Jun 272017
 
Beast + Electron

We have just released Beast version 0.11.0: Beast 0.11.0 Announcement

The announcement gives a high level overview of the changes (Soundfont support, multi threaded signal processing, new packaging, etc) and links to all the details like NEWS, tarballs, the binary package and shortlogs.

In this post, I’d like to extend a bit on where we’re going next. Beast has come a long way from its first lines of code drafted in 1996 and has seen long periods of inactivity due to numerous personal reasons on my part and also Stefan’s. I can’t begin to describe how much the Beast project owes to Stefan’s involvement, development really thrives whenever he manages to put some weight behind the project. He’s initiated major shifts in the project, contributed lots of demos, instruments and of course code.

Lately we were able to devote some time to Beast again, and with that reformulated its future directions. One important change was packaging, which already made it into the 0.11.0 release. This allows us to provide an easily installable binary package that extracts into /opt/. It’s available as a DEB for now, and we hope other package formats will follow.

Another major area of change that I’m working on behind the scenes is the UI technology. The current UI has huge deficits and lacks in workflow optimizations compared to other DAWs. Stefan has several big improvements planned for the workflow as do I, but in the past Gtk+ has not been helping with making those changes easy. Rapicorn was one attempt at fixing that, and while in theory it can provide a lot more flexibility in shaping the UI, based on concise declarations and use of SVG elements, it is still far away from reaching the degree of flexibility needed for our plans.

So far indeed, that I’ve had to seriously reconsider the approach and look for alternatives. Incidentally, the vast majority of feature needs and ideas I’ve had for the toolkit area appear to already be readily accessible through web technologies that have impressively advanced in the last few years.

Though we’re not planning to move Beast into an online application, we can still leverage these technologies through the electron project, which is an open source project providing HTML & CSS rendering plus Javascript on the desktop using libchromiumcontent from Google Chrome.

In my eyes it makes little sense to replicate much of the W3C specified features in desktop toolkits like Gtk+, Qt, Rapicorn which are much lesser staffed than the major browser projects, especially if we have a way to utilize recent browser improvements on the desktop.

So in effect I’ve changed plans for Beast’s future UI technology and started to construct a new interface based on web technologies running in electron. It’s interesting to change desktop UI development like this to say the least, and I’m curious about how long it takes to get up to par with current Gtk+ Beast functionality. I have some ideas how to address real time display of volume and frequency meters, but I’m still unsure how to best tackle large track view / clip view displays with wide scrolling and zooming ranges, given the choice between DOM elements and an HTML5 canvas.

Apart from the UI, we have several sound Library improvements pending integration. Stefan wants to finally complete Jack driver support, and as always there are some interesting plugin implementations in the queue that are awaiting completion.

If you want to help with any of the development steps outlined or just track Beast’s evolution, you can join our mailing list. Although the occasional face to face meeting helps us with setting development directions, we’re doing our best with keeping everything documented and open for discussions on the list.

UPDATE: Stefan just released his first Beast screencast: Walkthrough: making music with BEAST 0.11.0

Dec 272016
 

33c3

The last update has been a while, so with the new year around the corner and sitting in c-base @ 33c3, I’ll do my best to sum up what’s been going on in Rapicorn and Beast development since the last releases.

Now both projects make use of extended instruction sets (SIMD) that have been present in CPUs for the last 8 – 10 years, such as MMX, SSE, SSE2, SSE3 and CMPXCHG16B. Also both projects now support easy test builds in Docker images, which makes automated testing for different Linux distributions from travis-ci much simpler and more reproducible. Along the way, both got finally fixed up to fully support clang++ builds, although clang++ still throws a number of warnings. This means we can use clang++ based development and debugging tools now! A lot of old code that became obsolete or always remained experimental could be removed (and still is being removed).

Beast got support for using multiple CPU cores in its synthesis engine, we are currently testing performance improvements and stability of this addition. Rapicorn gained some extra logic to allow main loop integration with a GMainContext, which allows Beast to execute a Gtk+ and a Rapicorn event loop in the same thread.

Rapicorn widgets now always store coordinates relative to their parents, and always buffer drawings in per-widget surfaces. This allowed major optimizations to the size negotiation process so renegotiations can now operate much more fine grained. The widget states also got an overhaul and XML nodes now use declare=”…” attributes when new widgets are composed. Due to some rendering changes, librsvg modifications could be obsoleted, so librapicorn now links against a preinstalled librsvg. RadioButton, ToggleButton, SelectableItem and new painter widgets got added, as well as a few convenience properties.

After setting up an experimental Rapicorn build with Meson, we got some new ideas to speed up and improve the autotools based builds. I.e. I managed to do a full Rapicorn build with meson and compare that to autotools + GNU Make. It turns out Meson had two significant speed advantages:

  1. Meson builds files from multiple directories in parallel;
  2. Meson configuration happens a lot faster than what the autoconf scripts do.

Meson also has/had a lot of quirks (examples #785, #786, #753) and wasn’t really easier to use than our GNU Make setup. At least for me – given that I know GNU Make very well. The number one advantage of Meson was overcome with migrating Rapicorn to use a non-recursive Makefile (I find dependencies can still be expressed much better in Make than Meson), since parallel GNU Make can be just as fast as Ninja for small to medium sized projects.

The number two issue is harder to beat though. Looking at our configure.ac file, there where a lot of shell and compiler invocations I could remove, simply by taking the same shortcuts that Meson does, e.g. detect clang or gcc and then devise a batch of compiler flags instead of testing compiler support for each flag individually. Executing ./configure takes ca 3 seconds now, which isn’t too bad for infrequent invocations. The real culprit is autoreconf though, which takes over 12 seconds to regenerate everything after a configure.ac or related change (briefly looking into that, it seems aclocal takes longer than all of autoconf, automake, autoheader and libtoolize together).

 

PS: I’m attending 33C3 in Hamburg atm, so drop me a line (email or twitter) if you’re around and like to chat over coffee.

Jul 022015
 

Rapicorn 'visitor' branch

Trying to keep it up, here’s an update on recent developments in Rapicorn and Beast.

Git Branches

For now, Rapicorn and Beast are using Git branches the following way:

  • Topic branches are created for each change. Where possible, commits should compile and pass all tests (i.e. pass make check installcheck).
  • Once completed, topic branches are merged into the master branch. For intermediate merges of huge branches, I’ve recently been adding [ongoing] to the merge commit message. As an aside, branch merges should probably be more elaborate in the future to make devlog articles easier to write and potentially more accurate.
  • The master branch must always compile and pass all tests.
  • OpenHub: The OpenHub repo links have been adjusted to point at Rapicorn’s and Beast’s master branch. Because of problems with spammers and a corresponding reimplementations, code statistic updates on the OpenHub are platform currently stalled however.
    https://www.openhub.net/p/beast-bse
    https://www.openhub.net/p/rapicorn

Hello and goodbye clang++

Rapicorn C++11 code currently compiles with g++-4.7 and upwards. An initial attempt was made at making the C++11 code compile with clang++-3.4 but the incompatibilities are currently too numerous. A few good fixes have come out of this and are merged into master now, but further work on this branch probably has to wait for a newer clang++ version.

New Widgets

Rapicorn is growing more widgets that implement state rendering via SVG element matching. Recent additions are:

  • LayerPainter – A container that allows rendering widgets on top of each other.
  • ElementPainter – A container that displays state dependent SVG image elements.
  • FocusPainter – An ElementPainter that decorates its child according to focus changes.

IDL Improvements

Several changes around Rapicorn’s IDL compiler and support code made it into master recently:

  • The IDL layer got bind() and connect() mthods (on the ObjectBroker interface). This models the IDL setup phase after the zeromq API. Beast makes use of this when setting up IDL interface layers in the UI and in BSE.
  • The Python binding was rewritten using Cython. Instead of invoking a heap of generated Python glue code and talking to the message passing interfaces directly, the Python binding now sits on top of the C++ binding. This makes the end results operate much faster, is less complex on the maintenance side and more functional with regards to the Python API offered. As an added bonus, it also eases testing of the C++ bindings.
  • And just to prove the previous point, the new Cython port uncovered a major issue lurking in the C++ IDL handling of objects in records and sequences. At least since the introduction of remote reference counting, client side object handles and server side object references are implemented and treated in fundamentally different ways. This requires records (struct) and sequences (std::vector) to have separate implementation types on the client and server sides. Thus, the client and server types are now prefixed with ClnT_ and SrvT_ respectively. Newly generated typedef aliases are hiding the prefixes from user code.
  • IDL files don’t need ‘ = 0 ‘ postfixes for methods any more. After all, generating non-virtual methods wasn’t really used anyway.
  • The Enum introspection facilities got rewritten so things like the Enum name are also accessible now. This area probably isn’t fully finished yet, for future Any integration a more versatile API is needed still.
  • Auxillary information for properties is now accessible through an __aida_aux_data__() method on generated interfaces.
  • Generated records now provide a template method __accept__<>(Visitor) to visit all record fields by value reference and name string. Exemplary visitor implementations are provided to serialize/deserialize records to XML and INI file formats.

BEAST Developments

For the most part, changes in Beast are driving or chasing Rapicorn at the moment. This means that often the tip of Rapicorn master is required to build Beast’s master branch. Here is why:

  • Beast now uses RAPIDRES(1) to embedd compressed files. Rapicorn::Blob and Rapicorn::Res make these accessible.
  • Beast now makes use of Rapicorn’s IDL compiler to generate beastrc config structures and to add a new ‘Bse‘ IDL layer into libbse that allows the UI code to interface with Bse objects via C++ interfaces. Of course, lots of additional porting work is needed to complete this.
  • Beast procedures (a kind of ‘remote method’ implemented in C with lots of boilerplate code) are now migrated to C++ methods one by one which majorly simplifies the code base, but also causes lots of laborious adaptions on the call sites, the UI and the undo system. An excursion into the changes this brings for the undo implementation is provided in DevLog: A day of templates.
  • The GParamSpec introspection objects for properties that Beast uses for GUI generation can now be constructed from __aida_aux_data__()  strings, which enabled the beastrc config structure migration.
  • An explanatory file HACKING.md was added which describes the ongoing migration efforts and provides help in accessing the object types involved.

What’s next?

For the moment, porting the object system in Beast from GObject to IDL based C++11 interfaces and related procedure, signal and property migrations is keeping me more than busy. I’ll try to focus on completing the majority of work in this area first. But for outlooks, adding a Python REPL might make a good followup step. 😉

Jun 272015
 

c++11Yesterday I spent some 14+ hours on getting a templated undo method wrapper going.
Just to throw it all away this morning.

Here’s what I was trying to achieve, the C version of BEAST implements undo as follows:

// bse_track_remove_tick():
BseTrack *track;
uint tick;
BsePart *part;
bse_item_push_undo_proc (track, "insert-part", tick, part);

That is, it queues an undo step, that if executed, will call the “insert-part” procedure
on a BseTrack object that inserts a BsePart object at a ‘tick’.
This all happens through a varargs interface with lots of magic behind the scenes. In
particular the reference to ‘part’ is tricky. Future modifications to the BseTrack (or
project) may cause the removal and destruction of the BsePart object involved here.
While the execution of future undo steps will re-create a BsePart to be inserted here
before the step at hand is executed, the ‘part’ object pointer will have to be changed
to the re-created one instead of the destroyed one.
To achieve this, bse_item_push_undo_proc() internally converts the ‘part’ pointer into
a serializable descriptor string that allows to re-identify the BsePart object and the
undo machinery will resolve that before “insert-part” is called.

Now on to C++. I wanted the new pendant in the C++ version of Beast to look like:

// TrackImpl::remove_tick():
TrackImpl *this;
const uint tick;
PartImpl &part;
push_undo ("Remove Tick", *this, &TrackImpl::insert_part, tick, part);

But…

Under the hood that means push_undo() (which is a template method on ItemImpl, a base type of TrackImpl) needs to process its variable argument list to:

  • A) Put each argument into a wrapper structure and store away the argument list (i.e. std::tuple<Wrapper<Args>…>).
  • B) Special case the wrapper structure for objects to store a descriptor internally (i.e. template specialisation on Wrapper<Arg> for Arg=ItemImpl& or derived).
  • C) Copy the wrapped argument list into a closure to be called when the undo step is executed.
  • D) When the closure is called, “unwrap” each of the wrapped arguments to yield its original type (i.e. construct a std::tuple<Args…> from std::tuple<Wrapper<Args>…>).
  • E) When unwrapping an object, resolve the descriptor stored internally (i.e. put more magic into Wrapper<Arg> to yield a valid Arg& object).
  • F) Construct a variable argument call to &TrackImpl::insert_part(…) (i.e. apply a C++ argument pack).

In short, I got A, B, C, D, F working after significant efforts.
A is somewhat straight forward with C++11 variable template arguments. C can be accomplished with a C++11 lambda capture list and F involves copying over std::integer_sequence from the C++14 proposals and hacking its std::apply() template to support instance + method calls. Last, D can be implemented in a related fashion to F.
What’s left is B and E, i.e. writing a wrapper that will store and yield ordinary arguments such as int or std::string and convert ItemImpl& derived types back and forth between a string representation.
Probably laborious but doable — or so I thought.

It turns out that because of all the argument and tuple packing hassle (template recursion, integer sequencing and more) involved in implementing A, D, F, it would be hard to pass needed serialization context into Wrapper<>. And what’s much worse is that g++-4.9 started to choke on template errors during the Wrapper<> development, aborting with “confused by earlier errors” after pages and pages of template error messages. clang++-3.4 isn’t yet capable of processing the C++11 used by Rapicorn, so it wasn’t of help here either (I plan on another attempt at porting my C++11 code to be clang++ compatible once I get my hands on a newer clang++ version).
I.e. in the end, I gave up after an overlong day in the middle of E, everything else having been accomplished. g++-4.9 choking was a main let down, but probably even more important is that I had the necessary state and mood to process multiple pages of template error messages yesterday, but the same cannot be expected of every push_undo() user in the future if any push_undo() argument ever mismatches.

This morning, I threw away yesterdays templating excess and within an hour got an alternative interface to work:

// undoing part removal needs an undo_descriptor because future
// deletions may invalidate and recreate the part object
TrackImpl *this;
const uint tick;
PartImpl &part;
UndoDescriptor<PartImpl> part_descriptor = undo_descriptor (part);
auto lambda = [tick, part_descriptor] (TrackImpl &self) {
  PartImpl &part = self.undo_resolve (part_descriptor);
  self.insert_part (utick, part);
};
push_undo ("Remove Tick", *this, lambda);

That is, this interface is fully type-safe, but the ‘part’ wrapping has to be done manually, which involves writing a small lambda around TrackImpl::insert_part(). If any argument of the lambda or push_undo() calls is erroneous, the compiler will point at a single failing variable assignment in the implementation of push_undo<>() and list the mismatching arguments.
That is much more digestible than multiple template recursion error pages, so it’s a plus on the side of future maintenance.

The short version of push_undo<>() that takes a method pointer instead of a lambda is still available for implementing undo steps that don’t involve object references, incidentally covering the majority of uses.

Feb 052011
 
Last week the Beast project went live with a new website that has been in the making since December:


beast.testbit.eu
The old website was several years old, adding or changing content was very cumbersome and bottlenecked on Stefan or me. All edits had to go into the source code repository, adding content pages meant editing web/Makefile.am and changing a menu entry required the entire site to be rebuilt and re-synced. Also beast.gtk.org went offline for several weeks due to hosting problems at UC Berkeley.
So in the last few weeks the Beast website has been gradually moved from beast.gtk.org to beast.testbit.eu and a different hosting service that has more resources available. In the last few years, I’ve gained experiences with Plone, Drupal, DokuWiki, Confluence, a beast-specific markup parser, Joomla, WordPress, etc. They all have their up and down sides, and while I prefer WordPress for my own blog, I’ve settled on MediaWiki for the new Beast website.
Running the new site entirely as a wiki makes the contents easily accessible for everyone willing to contribute and MediaWiki’s markup is what most people already know or are likely to learn in the future. MediaWiki must be the hardest tested collaborative editing tool available, turns out to be impressively feature rich compared to other Wiki engines, has a rich set of extensions, scripting facilities and due to Wikipedia weight a reliable maintenance future.
Much of the previously hand crafted code used for site generation and operation becomes obsolete with the migration, like the screenshot gallery PHP snippets. The entire build-from-source process can be eliminated, and running a dedicated Beast theme on MediaWiki allows editing of the menu structure in a wiki page.
Also MediaWiki allows running multiple front ends under different domains and with different themes on the same Wiki database, which allowed me to merge the Beast site and testbit.eu to reduce maintenance.
A small set of patches/extensions were used to tune MediaWiki for the site’s needs:
  • Enhancing the builtin search logic, so it automatically resorts to partial matches for empty result lists.
  • Adjusting Expires/Cache-Control headers to utilize mod_disk_cache – this increases the number of possible requests per-second by more than a factor of one hundred.
  • Adding support for [[local:/path/to/download/area]] URLs, to refer to downloadable files from within wiki pages.
It took a while to migrate contents gradually into MediaWiki format, as some files had to be migrated from a very old ErfurtWiki installation, some came from the source code repository and some were available in HTML only. Big Kudos to David Iberri, his online html2wiki converter (html2wiki on CPAN) has been a huge help in the process.
I hope the new site is well received, have fun with it!

Sep 102010
 

BEAST/BSE and BSE-ALSA version 0.7.2 are available for download:

    http://beast.gtk.org/beast-ftp/v0.7/beast-0.7.2.tar.bz2
    http://beast.gtk.org/beast-ftp/v0.7/bse-alsa-0.7.2.tar.bz2

BEAST is a music composition and modular synthesis application. The “Bedevilled” portion of the names has no religious background, please refer to the About page for more details.

    Homepage:           http://beast.gtk.org/
    Downloads:          http://beast.gtk.org/beast-ftp/
    Feedback:           http://mail.gnome.org/mailman/listinfo/beast/

The 0.7.2 release provides new plugins and instruments, and a long list of bug fixes, improvements and translation updates.

TRANSLATORS: Please help us to improve the BEAST translations at Transifex.

Overview of Changes in BEAST/BSE 0.7.2:

  • Moved Beast/BSE to GNU LGPL, use AS-IS license for examples
  • Module changes and additions: ArtsCompressor – Relicensed to LGPL with permission from Matthias Kretz BseContribSampleAndHold – Relicensed to LGPL with permission from Artem Popov DavXTalStrings – Use deterministic random numbers for unit tests BseNoise – Improved random number generator
  • Switched to autogenerated ChangeLogs
  • Error bell can be muted in beast preferences dialog
  • Added multisample creation/editing command line tool: bsewavetool
  • Support adjustable volume, pitching and drum envelopes in .bsewave files
  • Added Retro Acoustic drum kit [Tim, Stefan]
  • New loadable Instruments/Effects:
    - BQS Bass Drum E8012 [Tim, Stefan]
    - BQS Slow Hum [Stefan, William DeVore]
    - FSM Fresh Water Bass instrument [Krzysztof Foltman]
    - FSM Growl Bass instrument [Krzysztof Foltman]
    - FSM Synth String Sweep [Krzysztof Foltman]
    
  • Added support for loading 32bit and 24bit PCM-format WAV files
  • Added support for gcc-4.4 and automake-1.10
  • Added support for guile-1.8, guile-1.6 remains as minimum requirement
  • Various fixes, improvements and much improved test coverage.
  • Bug fixes: #452604, #468229, #344388, #451086, #450724, #454121, #491552, #450490, #441936, #336766, #433431, #474332, #474244, #456879, #456408, #424897 [Tim Janik, Stefan Westerfeld]
  • Migrated translation support to use awk, sed and po/Makefile.am.
  • Updated German translation [Mario Blättermann]
  • Updated Italien translation [Michele Petrecca]
  • Updated Occitan translation [Yannig Marchegay]
  • Updated Brazilian Portugues translation [Leonardo Ferreira Fontenelle]
  • Updated British English translation [David Lodge]
  • Updated Spanish translation [Jorge Gonzalez]
  • Updated Slovenian translation [Andrej Znidarsic]
  • Updated Danish translation [Joe Hansen]
  • Updated French translation [Bruno Brouard]
  • Added Norwegian bokmal translation [Kjartan Maraas]
  • Added Ukrainian translation [Maxim V. Dziumanenko]

Overview of Changes in BSE-ALSA 0.7.2:

  • Fixes for automake-1.10 builds
  • Moved Beast/BSE to GNU LGPL

UPDATES: Updated translator instructions and uploaded a new release tarball with build fixes.

Sep 212007
 

We’ve been fairly busy recently with resolving the milestone bugs of the next Beast release. The good news is that pretty much all of the hard issues are sorted out by now, the bad news is that according to the release plan some essential release features are still missing 😉

It’s the large recent work that Stefan Westerfeld (who btw finally decided to start his own blog) has put into shaping up the remaining bits of bsewavetool (man page draft).
This is a very handy tool, that’ll finally be installed for public consumption in the upcoming release (it has been developed in SVN since 2004!). It can clip/normalize/ogg-encode/highpass/lowpass/upsample/downsample/etc chunks from the BSE multi sample files, and is normally used for shell-scripting the construction of sample kits from an unsorted pile of raw unprocessed sample data (note to self: blog some example use cases after the release).

Another addition are interesting new instruments by Krzysztof Foltman.

Oh – and before i forget, the synthesis link section on the website got some new updates as well: Beast Synthesis Links,

Also, Stefan pointed me at a YouTube video of Beast the other day:

The video is very nicely done, but it has the usual YouTubeish low quality artefacts.