Start aRts-builder.

You need a Synth_AMAN_PLAY-module to hear the output you are creating. So create a Synth_AMAN_PLAY-module by selecting Modules->Synthesis->SoundIO->Synth_AMAN_PLAY and clicking on the empty module space. Put it below the fifth line or so, because we'll add some stuff above.

The module will have a parameter title (leftmost port), and autoRestoreID (besides the leftmost port) for finding it. To fill these out, doubleclick on these ports, select constant value and type tutorial in the edit box. Click OK to apply.

Select File->Execute structure. You will hear absolutely nothing. The play module needs some input yet... ;) If you have listened to the silence for a while, click OK and go to Step 2

Create a Synth_WAVE_SIN module (from Modules->Synthesis->Waveforms) and put it above the Synth_AMAN_PLAY module. (Leave one line space in between).

As you see, it produces some output, but requires a pos as input. First lets put the output to the speakers. Click on the out port of the Synth_WAVE_SIN and then on the left port of Synth_AMAN_PLAY. Voila, you have connected two modules.

All oscillators in aRts don't require a frequency as input, but a position in the wave. The position should be between 0 and 1, which maps for a standard Synth_WAVE_SIN object to the range 0..2*pi. To generate oscillating values from a frequency, a Synth_FREQUENCY modules is used.

Create a Synth_FREQUENCY module (from Modules+Synthesis+Oscillation & Modulation) and connect it's ‘pos’ output to the ‘pos’ input of your Synth_WAVE_SIN. Specify the frequency port of the FREQUENCY generator as constant value 440.

Select File->Execute structure. You will hear a sinus wave at 440 Hz on one of your speakers. If you have listened to it for a while, click OK and go to Step 3.

Ok, it would be nicer if you would hear the sin wave on both speakers. Connect the right port of Synth_PLAY to the outvalue of the Synth_WAVE_SIN as well.

Create a Synth_SEQUENCE object (from Modules->Synthesis->Midi & Sequencing). It should be at the top of the screen. If you need more room you can move the other modules by selecting them (to select multiple modules use Shift), and dragging them around.

Now connect the frequency output of Synth_SEQUENCE to the frequency input of the Synth_FREQUENCY module. Then specify the sequence speed as constant value 0.13 (the speed is the leftmost port).

Now go to the rightmost port (sequence) of Synth_SEQUENCE and type in as constant value A-3;C-4;E-4;C-4; this specifies a sequence. More to that in the Module Reference.

Select File->Execute Structure. You will hear a nice sequence playing. If you have enjoyed the feeling, click OK and go to Step 4.

Create a Synth_PSCALE module (from Modules->Synthesis->Envelopes). Disconnect the outvalue of the SIN wave by doubleclicking it and choosing not connected. Connect

Finally, set the PSCALE top to some value, for instance 0.1.

How that works now: The Synth_SEQUENCE gives additional information about the position of the note it is playing right now, while 0 means just started and 1 means finished. The Synth_PSCALE module will scale the audio stream that is directed through it from a volume 0 (silent) to 1 (original loudness) back to 0 (silent). According to the position. The position where the peak should occur can be given as pos. 0.1 means that after 10% of the note has been played, the volume has reached its maximum, and starts decaying afterwards.

Select File->Execute Structure. You will hear a nice sequence playing. If you have enjoyed the feeling, click OK and go to Step 5.

Start another aRts-builder

Put a Synth_AMAN_PLAY into it, configure it to a sane name. Put a Synth_BUS_DOWNLINK into it and:

Start executing the structure. As expected, you hear nothing, ... not yet.

Go back to the structure with the Synth_WAVE_SIN stuff and replace the Synth_AMAN_PLAY module by an Synth_BUS_UPLINK, and configure the name to audio (or fred if you like). Deleting modules works with selecting them and choosing Edit->delete from the menu (or pressing the Del key).

Hit File+Execute structure. You will hear the sequence with scaled notes, transported over the bus.

If you want to find out why something like this can actually be useful, click OK (in the aRts-builder that is executing the Synth_SEQUENCE stuff, you can leave the other one running) and go to Step 6.

Choose File->Rename structure from the menu of the artsbuilder which contains the Synth_SEQUENCE stuff, and call it tutorial. Hit OK.

Choose File->Save

Start yet another aRts-builder and choose File->Load, and load the tutorial again.

Now you can select File->Execute structurein both aRts-builders having that structure. You'll now hear two times the same thing. Depending on the time when you start it it will sound more or less nice.

Another thing that is good to do at this point in time is: start Noatun, and play some mp3. Start artscontrol. Go to View->View audio manager. What you will see is Noatun and your ‘tutorial’ playback structure playing something. The nice thing you can do is this: doubleclick on Noatun. You'll now get a list of destinations. And see? You can assign Noatun to send it's output via the audio bus your tutorial playback structure provides.

Finally, now you should be able to turn your sin wave into an real instrument. This only makes sense if you have something handy that could send MIDI events to aRts. I'll describe here how you can use some external keyboard, but a midibus aware sequence like Brahms will work as well.

First of all, clean up on your desktop until you only have one aRts-builder with the sine wave structure running (not executing). Then, three times go to Ports->Create IN audio signal, and three times to Ports->Create OUT audio signal. Place the ports somewhere.

Finally, go to Ports+Change positions and names and call the ports frequency, velocity, pressed, left, right, done.

Finally, you can delete the Synth_SEQUENCE module, and rather connect connect the frequency input port of the structure to the Synth_FREQUENCY frequency port. Hm. But what do do about pos?

We don't have this, because with no algorithm in the world, you can predict when the user will release the note he just pressed on the midi keyboard. So we rather have a pressed parameter instead that just indicates wether the user still holds down the key. (pressed = 1: key still hold down, pressed = 0: key released)

That means the Synth_PSCALE object also must be replaced now. Plug in a Synth_ENVELOPE_ADSR instead (from Modules->Synthesis->Envelopes). Connect:

Set the parameters attack to 0.1, decay to 0.2, sustain to 0.7, release to 0.1.

Another thing we need to think of is that the instrument structure somehow should know when it is ready playing and then be cleaned up, because otherwise it would be never stopped even if the note has been released. Fortunately, the ADSR envelope knows when the will be nothing to hear anymore, since it anyway scales the signal to zero at some point after the note has been released.

This is indicated by setting the done output to 1. So connect this to the done output of the structure. The structure will be removed as soon as done goes up to 1.

Rename your structure to instrument_tutorial (from File->Rename structure. Then, save it using save as (the default name offered should be instrument_tutorial now).

Start artscontrol, and go to View->Midi Manager, and choose Add->aRts Synthesis Midi Output. Finally, you should be able to select your instrument (tutorial) here.

Open a terminal and type midisend. You'll see that midisend and the instrument are listed now in the aRts MIDI manager. After selecting both and hitting connect, we're finally done. Take your keyboard and start playing (of course it should be connected to your computer).

You now should be able to work with aRts. Here are a few tips what you could try to improve with your structures now:

If you have created something great, please consider providing it for the aRts web page. Or for inclusion into the next release.

Suppose you have an application called ‘mousepling’(that should make a ‘pling’ sound if you click on a button. The latency is the time between your finger clicking the mouse button and you hearing the pling. The latency in this setup composes itself out of certain latencies, that have different causes.

In this simple application, latency occurs at these places:

The first three items are latencies external to aRts. They are interesting, but beyond the scope of this document. Nevertheless be aware that they exist, so that even if you have optimized everything else to really low values, you may not necessarily get exactly the result you calculated.

Telling the server to play something involves usually one single MCOP call. There are benchmarks which confirm that, on the same host with unix domain sockets, telling the server to play something can be done about 9000 times in one second with the current implementation. I expect that most of this is kernel overhead, switching from one application to another. Of course this value changes with the exact type of the parameters. If you transfer a whole image with one call, it will be slower than if you transfer only one long value. For the returncode the same is true. However for ordinary strings (such as the filename of the wav file to play) this shouldn't be a problem.

That means, we can approximate this time with 1/9000 sec, that is below 0.15 ms. We'll see that this is not relevant.

Next is the time between the server starting playing and the soundcard getting something. The server needs to do buffering, so that when other applications are running, such as your X11 server or ‘mousepling’ application no dropouts are heard. The way this is done under Linux® is that there are a number fragments of a size. The server will refill fragments, and the soundcard will play fragments.

So suppose there are three fragments. The server refills the first, the soundcard starts playing it. The server refills the second. The server refills the third. The server is done, other applications can do something now.

As the soundcard has played the first fragment, it starts playing the second and the server starts refilling the first. And so on.

The maximum latency you get with all that is (number of fragments)*(size of each fragment)/(samplingrate * (size of each sample)). Suppose we assume 44kHz stereo, and 7 fragments a 1024 bytes (the current aRts defaults), we get 40 ms.

These values can be tuned according to your needs. However, the CPU usage increases with smaller latencies, as the sound server needs to refill the buffers more often, and in smaller parts. It is also mostly impossible to reach better values without giving the soundserver realtime priority, as otherwise you'll often get drop-outs.

However, it is realistic to do something like 3 fragments with 256 bytes each, which would make this value 4.4 ms. With 4.4ms delay the idle CPU usage of aRts would be about 7.5%. With 40ms delay, it would be about 3% (of a PII-350, and this value may depend on your soundcard, kernel version and others).

Then there is the time it takes the pling sound to get from the speakers to your ear. Suppose your distance from the speakers is 2 meters. Sound travels at a speed of 330 meters per second. So we can approximate this time with 6 ms.

Streaming applications are those that produce their sound themselves. Assume a game, which outputs a constant stream of samples, and should now be adapted to replay things via aRts. To have an example: when I press a key, the figure which I am playing jumps, and a boing sound is played.

First of all, you need to know how aRts does streaming. Its very similar to the I/O with the soundcard. The game sends some packets with samples to the sound server. Lets say three packets. As soon as the sound server is done with the first packet, it sends a confirmation back to the game that this packet is done.

The game creates another packet of sound and sends it to the server. Meanwhile the server starts consuming the second sound packet, and so on. The latency here looks similar like in the simple case:

The external latencies, as above, are beyond the scope of this document.

Obviously, the streaming latency depends on the time it takes all packets that are used for streaming to be played once. So it is (number of packets)*(size of each packet)/(samplingrate * (size of each sample))

As you see that is the same formula as applies for the fragments. However for games, it makes no sense to do such small delays as above. I'd say a realistic configuration for games would be 2048 bytes per packet, use 3 packets. The resulting latency would be 35ms.

This is based on the following: assume that the game renders 25 frames per second (for the display). It is probably safe to assume that you won't notice a difference of sound output of one frame. Thus 1/25 second delay for streaming is acceptable, which in turn means 40ms would be okay.

Most people will also not run their games with realtime priority, and the danger of drop-outs in the sound is not to be neglected. Streaming with 3 packets a 256 bytes is possible (I tried that) - but causes a lot of CPU usage for streaming.

For server side latencies, you can calculate these exactly as above.

There are a lot of factors which influence _CPU usage in a complex scenario, with some streaming applications and some others, some plugins on the server etc. To name a few:

For raw CPU usage for calculations, if you play two streams, simultaneuosly you need to do additions. If you apply a filter, some calculations are involved. To have a simplified example, adding two streams involves maybe four CPU cycles per addition, on a 350Mhz processor, this is 44100*2*4/350000000 = 0.1% CPU usage.

aRts internal scheduling: aRts needs to decide which plugin when calculates what. This takes time. Take a profiler if you are interested in that. Generally what can be said is: the less realtime you do (i.e.. the larger blocks can be calculated at a time) the less scheduling overhead you have. Above calculating blocks of 128 samples at a time (thus using fragment sizes of 512 bytes) the scheduling overhead is probably not worth thinking about it.

Integer to float conversion overhead: aRts uses floats internally as data format. These are easy to handle and on recent processors not slower than integer operations. However, if there are clients which play data which is not float (like a game that should do its sound output via aRts), it needs to be converted. The same applies if you want to replay the sounds on your soundcard. The soundcard wants integers, so you need to convert.

Here are numbers for a Celeron, approx. ticks per sample, with -O2 +egcs 2.91.66 (taken by Eugene Smith <hamster@null.ru>). This is of course highly processor dependant:

convert_mono_8_float: 14
convert_stereo_i8_2float: 28
convert_mono_16le_float: 40
interpolate_mono_16le_float: 200
convert_stereo_i16le_2float: 80
convert_mono_float_16le: 80

So that means 1% CPU usage for conversion and 5% for interpolation on this 350 MHz processor.

MCOP protocol overheadL MCOP does, as a rule of thumb, 9000 invocations per second. Much of this is not MCOPs fault, but relates to the two kernel causes named below. However, this gives a base to do calculations what the cost of streaming is.

Each data packet transferred through streaming can be considered one MCOP invocation. Of course large packets are slower than 9000 packets/s, but its about the idea.

Suppose you use packet sizes of 1024 bytes. Thus, to transfer a stream with 44kHz stereo, you need to transfer 44100*4/1024 = 172 packets per second. Suppose you could with 100% cpu usage transfer 9000 packets, then you get (172*100)/9000 = 2% CPU usage due to streaming with 1024 byte packets.

That are approximations. However, they show, that you would be much better off (if you can afford it for the latency), to use for instance packets of 4096 bytes. We can make a compact formula here, by calculating the packet size which causes 100% CPU usage as 44100*4/9000 = 19.6 samples, and thus getting the quick formula:

streaming CPU usage in percent = 1960/(your packet size)

which gives us 0.5% CPU usage when streaming with 4096 byte packets.

Kernel process/context switching: this is part of the MCOP protocol overhead. Switching between two processes takes time. There is new memory mapping, the caches are invalid, whatever else (if there is a kernel expert reading this - let me know what exactly are the causes). This means: it takes time.

I am not sure how many context switches Linux® can do per second, but that number isn't infinite. Thus, of the MCOP protocol overhead I suppose quite a bit is due to context switching. In the beginning of MCOP, I did tests to use the same communication inside one process, and it was much faster (four times as fast or so).

Kernel: communication overhead: This is part of the MCOP protocol overhead. Transferring data between processes is currently done via sockets. This is convenient, as the usual select() methods can be used to determine when a message has arrived. It can also be combined with other I/O sources as audio I/O, X11 server or whatever else easily.

However, those read and write calls cost certainly processor cycles. For small invocations (such as transferring one midi event) this is probably not so bad, for large invocations (such as transferring one video frame with several megabytes) this is clearly a problem.

Adding the usage of shared memory to MCOP where appropriate is probably the best solution. However it should be done transparent to the application programmer.

Take a profiler or do other tests to find out how much exactly current audio streaming is impacted by the not using sharedmem. However, its not bad, as audio streaming (replaying mp3) can be done with 6% total CPU usage for artsd and artscat (and 5% for the mp3 decoder). However, this includes all things from the necessary calculations up do the socket overhead, thus I'd say in this setup you could perhaps save 1% by using sharedmem.

These are done with the current development snapshot. I also wanted to try out the real hard cases, so this is not what everyday applications should use.

I wrote an application called streamsound which sends streaming data to aRts. Here it is running with realtime priority (without problems), and one small serverside (volume-scaling and clipping) plugin:

 4974 stefan    20   0  2360 2360  1784 S       0 17.7  1.8   0:21 artsd
 5016 stefan    20   0  2208 2208  1684 S       0  7.2  1.7   0:02 streamsound
 5002 stefan    20   0  2208 2208  1684 S       0  6.8  1.7   0:07 streamsound
 4997 stefan    20   0  2208 2208  1684 S       0  6.6  1.7   0:07 streamsound

Each of them is streaming with 3 fragments a 1024 bytes (18 ms). There are three such clients running simultaneously. I know that that does look a bit too much, but as I said: take a profiler and find out what costs time, and if you like, improve it.

However, I don't think using streaming like that is realistic or makes sense. To take it even more to the extreme, I tried what would be the lowest latency possible. Result: you can do streaming without interruptions with one client application, if you take 2 fragments of 128 bytes between aRts and the soundcard, and between the client application and aRts. This means that you have a total maximum latency of 128*4/44100*4 = 3 ms, where 1.5 ms is generated due to soundcard I/O and 1.5 ms is generated through communication with aRts. Both applications need to run realtimed.

But: this costs an enormous amount of CPU. This example cost you about 45% of my P-II/350. I also starts to click if you start top, move windows on your X11 display or do disk I/O. All these are kernel issues. The problem is that scheduling two or more applications with realtime priority cost you an enormous amount of effort, too, even more if the communicate, notify each other etc..

Finally, a more real life example. This is aRts with artsd and one artscat (one streaming client) running 16 fragments a 4096 bytes:

 5548 stefan    12   0  2364 2364  1752 R       0  4.9  1.8   0:03 artsd
 5554 stefan     3   0   752  752   572 R       0  0.7  0.5   0:00 top
 5550 stefan     2   0  2280 2280  1696 S       0  0.5  1.7   0:00 artscat

Each namespace declaration corresponds to a ‘module’ declaration in the MCOP IDL.

// mcop idl

module M {
    interface A
    {
    }
};

interface B;

In this case, the generated C++ code for the IDL snippet would look like this:

// C++ header

namespace M {
    /* declaration of A_base/A_skel/A_stub and similar */
    class A {        // Smartwrapped reference class
        /* [...] */
    };
}

/* declaration of B_base/B_skel/B_stub and similar */
class B {
    /* [...] */
};

So when referring the classes from the above example in your C++ code, you would have to write M::A, but only B. However, you can of course use ‘using M’ somewhere - like with any namespace in C++.

There is one global namespace called ‘Arts’, which all programs and libraries that belong to aRts itself use to put their declarations in. This means, that when writing C++ code that depends on aRts, you normally have to prefix every class you use with Arts::, like this:

int main(int argc, char **argv)
{
    Arts::Dispatcher dispatcher;
    Arts::SimpleSoundServer server(Arts::Reference("global:Arts_SimpleSoundServer"));

    server.play("/var/foo/somefile.wav");

The other alternative is to write a using once, like this:

using namespace Arts;

int main(int argc, char **argv)
{
    Dispatcher dispatcher;
    SimpleSoundServer server(Reference("global:Arts_SimpleSoundServer"));

    server.play("/var/foo/somefile.wav");
    [...]

In IDL files, you don't exactly have a choice. If you are writing code that belongs to aRts itself, you'll have to put it into module aRts.

// IDL File for aRts code:
#include <artsflow.idl>
module Arts {        // put it into the Arts namespace
    interface Synth_TWEAK : SynthModule
    {
        in audio stream invalue;
        out audio stream outvalue;
        attribute float tweakFactor;
    };
};

If you write code that doesn't belong to aRts itself, you should not put it into the ‘Arts’ namespace. However, you can make an own namespace if you like. In any case, you'll have to prefix classes you use from aRts.

// IDL File for code which doesn't belong to aRts:
#include <artsflow.idl>

// either write without module declaration, then the generated classes will
// not use a namespace:
interface Synth_TWEAK2 : Arts::SynthModule
{
    in audio stream invalue;
    out audio stream outvalue;
    attribute float tweakFactor;
};

// however, you can also choose your own namespace, if you like, so if you
// write an application "PowerRadio", you could for instance do it like this:
module PowerRadio {
    struct Station {
        string name;
        float frequency;
    };

    interface Tuner : Arts::SynthModule {
        attribute Station station;     // no need to prefix Station, same module
        out audio stream left, right;
    };
};

Often, in interfaces, casts, method signatures and similar, MCOP needs to refer to names of types or interfaces. These are represented as string in the common MCOP datastructures, while the namespace is always fully represented in the C++ style. This means the strings would contain ‘M::A’ and ‘B’, following the example above.

Note this even applies if inside the IDL text the namespace qualifiers were not given, since the context made clear which namespace the interface A was meant to be used in.

Using threads isn't possible on all platforms. This is why aRts was originally written without using threading at all. For almost all problems, for each threaded solution to the problem, there is a non-threaded solution that does the same.

For instance, instead of putting audio output in a seperate thread, and make it blocking, aRts uses non-blocking audio output, and figures out when to write the next chunk of data using select().

However, aRts (in very recent versions) at least provides support for people who do want to implement their objects using threads. For instance, if you already have code for an mp3 player, and the code expects the mp3 decoder to run in a seperate thread, it's usally the easiest thing to do to keep this design.

The aRts/MCOP implementation is built along sharing state between seperate objects in obvious and non-obvious ways. A small list of shared state includes:

All of the above objects don't expect to be used concurrently (i.e. called from seperate threads at the same time). Generally there are two ways of solving this:

aRts follows the first approach: you will need a lock whenever you talk to any of these objects. The second approach is harder to do. A hack which tries to achieve this is available at http://space.twc.de/~stefan/kde/download/arts-mt.tar.gz, but for the current point in time, a minimalistic approach will probably work better, and cause less problems with existing applications.

You can get/release the lock with the two functions:

Generally, you don't need to acquire the lock (and you shouldn't try to do so), if it is already held. A list of conditions when this is the case is:

There are also some exceptions of functions. which you can only call in the main thread, and for that reason you will never need a lock to call them:

But that is it. For everything else that is somehow related to aRts, you will need to get the lock, and release it again when done. Always. Here is a simple example:

class SuspendTimeThread : Arts::Thread {
public:
    void run() {
        /*
         * you need this lock because:
         *  - constructing a reference needs a lock (as global: will go to
         *    the object manager, which might in turn need the GlobalComm
         *    object to look up where to connect to)
         *  - assigning a smartwrapper needs a lock
         *  - constructing an object from reference needs a lock (because it
         *    might need to connect a server)
         */
        Arts::Dispatcher::lock();
        Arts::SoundServer server = Arts::Reference("global:Arts_SoundServer");
        Arts::Dispatcher::unlock();

        for(;;) {            /*
             * you need a lock here, because
             *  - dereferencing a smartwrapper needs a lock (because it might
             *    do lazy creation)
             *  - doing an MCOP invocation needs a lock
             */
            Arts::Dispatcher::lock();
            long seconds = server.secondsUntilSuspend();
            Arts::Dispatcher::unlock();

            printf("seconds until suspend = %d",seconds);
            sleep(1);
        }
    }
}

The following threading related classes are currently available:

See the links for documentation.

As references can point to remote objects, the servers containing these objects can crash. What happens then?

There are other conditions of failure, such as network disconnection (suppose you remove the cable between your server and client while your application runs). However their effect is the same like a server crash.

Overall, it is of course a consideration of policy how strictly you try to trap communcation errors throughout your application. You might follow the ‘if the server crashes, we need to debug the server until it never crashes again’ method, which would mean you need not bother about all these problems.

An object, to exist, must be owned by someone. If it isn't, it will cease to exist (more or less) immediately. Internally, ownership is indicated by calling _copy(), which increments an reference count, and given back by calling _release(). As soon as the reference count drops to zero, a delete will be done.

As a variation of the theme, remote usage is indicated by _useRemote(), and dissolved by _releaseRemote(). These functions lead a list which server has invoked them (and thus owns the object). This is used in case this server disconnects (i.e. crash, network failure), to remove the references that are still on the objects. This is done in _disconnectRemote().

Now there is one problem. Consider a return value. Usually, the return value object will not be owned by the calling function any longer. It will however also not be owned by the caller, until the message holding the object is received. So there is a time of ‘ownershipless’ objects.

Now, when sending an object, one can be reasonable sure that as soon as it is received, it will be owned by somebody again, unless, again, the receiver dies. However this means that special care needs to be taken about object at least while sending, probably also while receiving, so that it doesn't die at once.

The way MCOP does this is by ‘tagging’ objects that are in process of being copied across the wire. Before such a copy is started, _copyRemote is called. This prevents the object from being freed for a while (5 seconds). Once the receiver calls _useRemote(), the tag is removed again. So all objects that are send over wire are tagged before transfer.

If the receiver receives an object which is on his server, of course he will not _useRemote() it. For this special case, _cancelCopyRemote() exists to remove the tag manually. Other than that, there is also timer based tag removal, if tagging was done, but the receiver didn't really get the object (due to crash, network failure). This is done by the ReferenceClean class.

Streams define multimedia data, one of the most important components of a module. Streams are defined in the following format:

[ async ] in|out [ multi ] type stream name [ , name ] ;

Streams have a defined direction in reference to the module, as indicated by the required qualifiers in or out. The type argument defines the type of data, which can be any of the types described later for attributes (not all are currently supported). Many modules use the stream type audio, which is an alias for float since that is the internal data format used for audio stream. Multiple streams of the same type can defined in the same definition uisng comma separated names.

Streams are by default synchronous, which means they are continuous flows of data at a constant rate, such as PCM audio. The async qualifier specifies an asynchronous stream, which is used for non-continuous data flows. The most common example of an async stream is MIDI messages.

The multi keyword, only valid for input streams, indicates that the interface supports a variable number of inputs. This is useful for implementing devices such as mixers that can accept any number of input streams.

Attributes are data associated with an instance of an interface. They are declared like member variables in C++, and can can use any of the primitive types boolean, byte, long, string, or float. You can also use user-defined struct or enum types as well as variable sized sequences using the syntax sequence<type>. Attributes can optionally be marked readonly.

As in C++, methods can be defined in interfaces. The method parameters are restricted to the same types as attributes. The keyword oneway indicates a method which returns immediately and is executed asynchronously.

Several standard module interfaces are already defined for you in aRts, such as StereoEffect, and SimpleSoundServer.

A simple example of a module taken from aRts is the constant delay module, found in the file kdemultimedia/arts/modules/artsmodules.idl. The interface definition is listed below.

interface Synth_CDELAY : SynthModule {
        attribute float time;
        in audio stream invalue;
        out audio stream outvalue;
};

This modules inherits from SynthModule. That interface, defined in artsflow.idl, defines the standard methods implemented in all music synthesizer modules.

The CDELAY effect delays a stereo audio stream by the time value specified as a floating point parameter. The interface definition has an attribute of type float to store the delay value. It defines two input audio streams and two output audio streams (typical of stereo effects). No methods are required other than those it inherits.

There are various requirements for how a module can do streaming. To illustrate this, consider these examples:

The first case is the simplest: upon receiving 200 samples of input the module produces 200 samples of output. It only produces output when it gets input.

The second case produces different numbers of output samples when given 200 input samples. It depends what conversion is performed, but the number is known in advance.

The third case is even worse. From the outset you cannot even guess how much data 200 input bytes will generate (probably a lot more than 200 bytes, but...).

The last case is a module which becomes active by itself, and sometimes produces data.

In aRtss-0.3.4, only streams of the first type were handled, and most things worked nicely. This is probably what you need most when writing modules that process audio. The problem with the other, more complex types of streaming, is that they are hard to program, and that you don't need the features most of the time. That is why we do this with two different stream types: synchronous and asynchronous.

Synchronous streams have these characteristics:

Asynchronous streams, on the other hand, have this behaviour:

When you declare streams, you use the keyword ‘async’ to indicate you want to make an asynchronous stream. So, for instance, assume you want to convert an asynchronous stream of bytes into a synchronous stream of samples. Your interface could look like this:

interface ByteStreamToAudio : SynthModule {
    async in byte stream indata;   // the asynchonous input sample stream

    out audio stream left,right;   // the synchronous output sample streams
};

Suppose you decided to write a module to produce sound asynchronously. Its interface could look like this:

interface SomeModule : SynthModule
{
    async out byte stream outdata;
};

How do you send the data? The first method is called ‘push delivery’. With asynchronous streams you send the data as packets. That means you send individual packets with bytes as in the above example. The actual process is: allocate a packet, fill it, send it.

Here it is in terms of code. First we allocate a packet:

DataPacket<mcopbyte> *packet = outdata.allocPacket(100);

The we fill it:

// cast so that fgets is happy that it has a (char *) pointer
char *data = (char *)packet->contents;

// as you can see, you can shrink the packet size after allocation
// if you like
if(fgets(data,100,stdin))
    packet->size = strlen(data);
else
    packet->size = 0;

Now we send it:

packet->send();

This is quite simple, but if we want to send packets exactly as fast as the receiver can process them, we need another approach, the ‘pull delivery’ method. You ask to send packets as fast as the receiver is ready to process them. You start with a certain amount of packets you send. As the receiver processes one packet after another, you start refilling them with fresh data, and send them again.

You start that by calling setPull. For example:

outdata.setPull(8, 1024);

This means that you want to send packets over outdata. You want to start sending 8 packets at once, and as the receiver processes some of them, you want to refill them.

Then, you need to implement a method which fills the packets, which could look like this:

void request_outdata(DataPacket<mcopbyte> *packet)
{
    packet->size = 1024;  // shouldn't be more than 1024
    for(int i = 0;i < 1024; i++)
        packet->contents[i] = (mcopbyte)'A';
    packet->send();
}

Thats it. When you don't have any data any more, you can start sending packets with zero size, which will stop the pulling.

Note that it is essential to give the method the exact name request_streamname.

We just discussed sending data. Receiving data is much much simpler. Suppose you have a simple ToLower filter, which simply converts all letters in lowercase:

interface ToLower {
    async in byte stream indata;
    async out byte stream outdata;
};

This is really simple to implement; here is the whole implementation:

class ToLower_impl : public ToLower_skel {
public:
    void process_indata(DataPacket<mcopbyte> *inpacket)
    {
        DataPacket<mcopbyte> *outpacket = outdata.allocPacket(inpacket->size);

        // convert to lowercase letters
        char *instring = (char *)inpacket->contents;
        char *outstring = (char *)outpacket->contents;

        for(int i=0;i<inpacket->size;i++)
            outstring[i] = tolower(instring[i]);

        inpacket->processed();
        outpacket->send();
    }
};

REGISTER_IMPLEMENTATION(ToLower_impl);

Again, it is essential to name the method process_streamname.

As you see, for each arriving packet you get a call for a function (the process_indata call in our case). You need to call the processed() method of a packet to indicate you have processed it.

Here is an implenetation tip: if processing takes longer (i.e. if you need to wait for soundcard output or something like that), don't call processed immediately, but store the whole data packet and call processed only as soon as you really processed that packet. That way, senders have a chance to know how long it really takes to do your work.

As synchronization isn't so nice with asynchronous streams, you should use synchronous streams wherever possible, and asynchronous streams only when necessary.

Suppose you have 2 objects, for example an AudioProducer and an AudioConsumer. The AudioProducer has an output stream and AudioConsumer has an input one. Each time you want to connect them, you will use those 2 streams. The first use of defaulting is to enable you to make the connection without specifying the ports in that case.

Now suppose the teo objects above can handle stereo, and each have a ‘left’ and ‘right’ port. You'd still like to connect them as easily as before. But how can the connecting system know which output port to connect to which input port? It has no way to correctly map the streams. Defaulting is then used to specify several streams, with an order. Thus, when you connect an object with 2 default output streams to another one with 2 default input streams, you don't have to specify the ports, and the mapping will be done correctly.

Of course, this is not limited to stereo. Any number of streams can be made default if needed, and the connect function will check that the number of defaults for 2 object match (in the required direction) if you don't specify the ports to use.

The syntax is as follows: in the IDL, you can use the default keyword in the stream declaration, or on a single line. For example:

interface TwoToOneMixer {
    default in audio stream input1, input2;
    out audio stream output;
};

In this example, the object will expect its two input ports to be connected by default. The order is the one specified on the default line, so an object like this one:

interface DualNoiseGenerator {
    out audio stream bzzt, couic;
    default couic, bzzt;
};

Will make connections from ‘couic’ to ‘input1’, and ‘bzzt’ to ‘input2’ automatically. Note that since there is only one output for the mixer, it will be made default in this case (see below). The syntax used in the noise generator is useful to declare a different order than the declaration, or selecting only a few ports as default. The directions of the ports on this line will be looked up by mcopidl, so don't specify them. You can even mix input and output ports in such a line, only the order is important.

There are some rules that are followed when using inheritance:

When implementing objects that have attributes, you need to send change notifications whereever an attribute changes. The code for doing this looks like this:

 void KPoti_impl::value(float newValue)
 {
     if(newValue != _value)
     {
         _value = newValue;
         value_changed(newValue); // <- send change notification
     }
 }

It is strongly recommended to use code like this for all objects you implement, so that change notifications can be used by other people. You should however void sending notifications too often, so if you are doing signal processing, it is probably the best if you keep track when you sent your last notification, so that you don't send one with every sample you process.

It will be especially useful to use change notifications in conjunction with scopes (things that visualize audio data for instance), gui elements, control widgets, and monitoring. Code using this is in kdelibs/arts/tests, and in the experimental artsgui implementation, which you can find under kdemultimedia/arts/gui.

The .mcoprc file (in each user's home directory) can be used to configure MCOP in some ways. Currently, the following is possible:

An example which uses all of the above is:

# $HOME/.mcoprc file
GlobalComm=Arts::X11GlobalComm

# if you are a developer, it might be handy to add a directory in your home
# to the trader/extension path to be able to add components without
# installing them
TraderPath="/opt/kde2/lib/mcop","/home/joe/mcopdevel/mcop"
ExtensionPath="/opt/kde2/lib","/home/joe/mcopdevel/lib"

If you have used CORBA before, you will see that MCOP is much the same thing. In fact, aRts prior to version 0.4 used CORBA.

The basic idea of CORBA is the same: you implement objects (components). By using the MCOP features, your objects are not only available as normal classes from the same process (via standard C++ techniques) - they also are available to remote servers transparently. For this to work, the first thing you need to do is to specify the interface of your objects in an IDL file - just like CORBA IDL. There are only a few differences.

// CORBA idl
interface Account {
  void deposit( in long amount );
  void withdraw( in long amount );
  long balance();
};

// MCOP idl
interface Account {
  void deposit( long amount );
  void withdraw( long amount );
  long balance();
};

// CORBA idl
struct Line {
    long x1,y1,x2,y2;
};
typedef sequence<Line> LineSeq;
interface Plotter {
    void draw(in LineSeq lines);
};

// MCOP idl
struct Line {
    long x1,y1,x2,y2;
};
interface Plotter {
    void draw(sequence<Line> lines);
};

// MCOP idl
interface Synth_ADD : SynthModule {
    in audio stream signal1,signal2;
    out audio stream outvalue;
};

// MCOP idl: hello.idl
interface Hello {
    void hello(string s);
    string concat(string s1, string s2);
    long sum2(long a, long b);
};

// C++ header file - include hello.h somewhere
class Hello_impl : virtual public Hello_skel {
public:
    void hello(const string& s);
    string concat(const string& s1, const string& s2);
    long sum2(long a, long b);
};

// C++ implementation file

// as you see string's are passed as const string references
void Hello_impl::hello(const string& s)
{
    printf("Hello '%s'!\n",s.c_str());
}

// when they are a returncode they are passed as "normal" strings
string Hello_impl::concat(const string& s1, const string& s2)
{
    return s1+s2;
}

long Hello_impl::sum2(long a, long b)
{
    return a+b;
}

    Hello_impl server;

    string reference = server._toString();
    printf("%s\n",reference.c_str());

Dispatcher::the()->run();

// this code can run anywhere - not necessarily in the same process
// (it may also run on a different computer/architecture)

    Hello *h = Hello::_fromString([the object reference printed above]);

    if(h)
        h->hello("test");
    else
        printf("Access failed?\n");

Since MCOP servers will listen on a TCP port, potentially everybody (if you are on the Internet) may try to connect MCOP services. Thus, it is important to authenticate clients. MCOP uses the md5-auth protocol.

The md5-auth protocol does the following to ensure that only selected (trusted) clients may connect to a server:

To give each client the secret cookie, MCOP will (normally) put it in the mcop directory (under /tmp/mcop-USER/secret-cookie). Of course, you can copy it to other computers. However, if you do so, use a secure transfer mechanism, such as scp (from ssh).

The authentication of clients uses the following steps:

This algorithm should be secure, given that

The MCOP protocol will start every new connection with an authentication process. Basically, it looks like this:

To see that the security actually works, we should look at how messages are processed on unauthenticated connections:

Since KDE dropped CORBA completely, and is using DCOP everywhere instead, naturally the question arises why aRts isn't doing so. After all, DCOP support is in KApplication, is well-maintained, supposed to integrate greatly with libICE, and whatever else.

Since there will be (potentially) a lot of people asking whether having MCOP besides DCOP is really necessary, here is the answer. Please don't get me wrong, I am not trying to say ‘DCOP is bad’. I am just trying to say ‘DCOP isn't the right solution for aRts’ (while it is a nice solution for other things).

First, you need to understand what exactly DCOP was written for. Created in two days during the KDE-TWO meeting, it was intended to be as simple as possible, a really ‘lightweight’ communication protocol. Especially the implementation left away everything that could involve complexity, for instance a full blown concept how data types shall be marshalled.

Even although DCOP doesn't care about certain things (like: how do I send a string in a network-transparent manner?) - this needs to be done. So, everything that DCOP doesn't do, is left to Qt™ in the KDE apps that use DCOP today. This is mostly type management (using the Qt™ serialization operator).

So DCOP is a minimal protocol which perfectly enables KDE applications to send simple messages like ‘open a window pointing to http://www.kde.org’ or ‘your configuration data has changed’. However, inside aRts the focus lies on other things.

The idea is, that little plugins in aRts will talk involving such data structures as ‘midi events’ and ‘songposition pointers’ and ‘flow graphs’.

These are complex data types, which must be sent between different objects, and be passed as streams, or parameters. MCOP supplies a type concept, to define complex data types out of simpler ones (similar to structs or arrays in C++). DCOP doesn't care about types at all, so this problem would be left to the programmer - like: writing C++ classes for the types, and make sure they can serialize properly (for instance: support the Qt™ streaming operator).

But that way, they would be inaccessible to everything but direct C++ coding. Specifically, you could not design a scripting language, that would know all types plugins may ever expose, as they are not self describing.

Much the same argument is valid for interfaces as well. DCOP objects don't expose their relationships, inheritance hierarchies, etc. - if you were to write an object browser which shows you ‘what attributes has this object got’, you'd fail.

While Matthias told me that you have a special function ‘functions’ on each object that tells you about the methods that an object supports, this leaves out things like attributes (properties), streams and inheritance relations.

This seriously breaks applications like aRts-builder. But remember: DCOP was not so much intended to be an object model (as Qt™ already has one with moc and similar), nor to be something like CORBA, but to supply inter-application communication.

Why MCOP even exists is: it should work fine with streams between objects. aRts makes heavily use of small plugins, which interconnect themselves with streams. The CORBA version of aRts had to introduce a very annoying split between ‘the SynthModule objects’, which were the internal work modules that did do the streaming, and ‘the CORBA interface’, which was something external.

Much code cared about making interaction between ‘the SynthModule objects’ and ‘the CORBA interface’ look natural, but it didn't, because CORBA knew nothing at all about streams. MCOP does. Look at the code (something like simplesoundserver_impl.cc). Way better! Streams can be declared in the interface of modules, and implemented in a natural looking way.

One can't deny it. One of the reasons why I wrote MCOP was speed. Here are some arguments why MCOP will definitely be faster than DCOP (even without giving figures).

An invocation in MCOP will have a six-‘long’-header. That is:

After that, the parameters follow. Note that the demarshalling of this is extremely fast. You can use table lookups to find the object and the method demarshalling function, which means that complexity is O(1) [ it will take the same amount of time, no matter how many objects are alive, or how many functions are there ].

Comparing this to DCOP, you'll see, that there are at least

These are much more painful to demarshall, as you'll need to parse the string, search for the function, etc..

In DCOP, all requests are running through a server (DCOPServer). That means, the process of a synchronous invocation looks like this:

In MCOP, the same invocation looks like this:

Say both were implemented correctly, MCOPs peer-to-peer strategy should be faster by a factor of two, than DCOPs man-in-the-middle strategy. Note however that there were of course reasons to choose the DCOP strategy, which is namely: if you have 20 applications running, and each app is talking to each app, you need 20 connections in DCOP, and 200 with MCOP. However in the multimedia case, this is not supposed to be the usual setting.

I tried to compare MCOP and DCOP, doing an invocation like adding two numbers. I modified testdcop to achieve this. However, the test may not have been precise on the DCOP side. I invoked the method in the same process that did the call for DCOP, and I didn't know how to get rid of one debugging message, so I used output redirection.

The test only used one object and one function, expect DCOPs results to decrease with more objects and functions, while MCOPs results should stay the same. Also, the dcopserver process wasn't connected to other applications, it might be that if many applications are connected, the routing performance decreases.

The result I got was that while DCOP got slightly more than 2000 invocations per second, MCOP got slightly more than 8000 invocations per second. That makes a factor of 4. I know that MCOP isn't tuned to the maximum possible, yet. (Comparision: CORBA, as implemented with mico, does something between 1000 and 1500 invocations per second).

If you want ‘harder’ data, consider writing some small benchmark app for DCOP and send it to me.

CORBA had the nice feature that you could use objects you implemented once, as ‘seperate server process’, or as ‘library’. You could use the same code to do so, and CORBA would transparently descide what to do. With DCOP, that is not really intended, and as far as I know not really possible.

MCOP on the other hand should support that from the beginning. So you can run an effect inside artsd. But if you are a wave editor, you can choose to run the same effect inside your process space as well.

While DCOP is mostly a way to communicate between apps, MCOP is also a way to communicate inside apps. Especially for multimedia streaming, this is important (as you can run multiple MCOP objects parallely, to solve a multimedia task in your application).

Although MCOP does not currently do so, the possibilities are open to implement quality of service features. Something like ‘that MIDI event is really really important, compared to this invocation’. Or something like ‘needs to be there in time’.

On the other hand, stream transfer can be integrated in the MCOP protocol nicely, and combined with QoS stuff. Given that the protocol may be changed, MCOP stream transfer should not really get slower than conventional TCP streaming, but: it will be easier and more consistent to use.

There is no need to base a middleware for multimedia on Qt™. Deciding so, and using all that nice Qt™-streaming and stuff, will easily lead to the middleware becoming a Qt™-only (or rather KDE-only) thing. I mean: as soon as I'll see the GNOMEs using DCOP, too, or something like that, I am certainly proven wrong.

While I do know that DCOP basically doesn't know about the data types it sends, so that you could use DCOP without using Qt™, look at how it is used in daily KDE usage: people send types like QString, QRect, QPixmap, QCString, ..., around. These use Qt™-serialization. So if somebody choose to support DCOP in a GNOME program, he would either have to claim to use QString,... types (although he doesn't do so), and emulate the way Qt™ does the streaming, or he would send other string, pixmap and rect types around, and thus not be interoperable.

Well, whatever. aRts was always intended to work with or without KDE, with or without Qt™, with or without X11, and maybe even with or without Linux® (and I have even no problems with people who port it to a popular non-free operating systems).

It is my position that non-GUI-components should be written non-GUI-dependant, to make sharing those among wider amounts of developers (and users) possible.

I see that using two IPC protocols may cause inconveniences. Even more, if they are both non-standard. However, for the reasons given above, switching to DCOP is no option. If there is significant interest to find a way to unite the two, okay, we can try. We could even try to make MCOP speak IIOP, then we'd have a CORBA ORB ;).

I talked with Matthias Ettrich a bit about the future of the two protocols, and we found lots of ways how things could go on. For instance, MCOP could handle the message communication in DCOP, thus bringing the protocols a bit closer together.

So some possible solutions would be:

However, it may not be the worst possibility to use each protocol for everything it was intended for (there are some big differences in the design goals), and don't try to merge them into one.

The aRts C API was designed to make it easy to writing and port plain C applications to the aRts sound server. It provides streaming functionality (sending sample streams to artsd), either blocking or non-blocking. For most applications you simply remove the few system calls that deal with your audio device and replace them with the appropriate aRts calls.

I did two ports as a proof of concept: mpg123 and quake. You can get the patches from here. Feel free to submit your own patches to the maintainer of aRts or of multimedia software packages so that they can integrate aRts support into their code.

Sending audio to the sound server with the API is very simple:

Here is a small example program that illustrates this:

#include <stdio.h>
#include <artsc.h>
int main()
{
    arts_stream_t stream;
    char buffer[8192];
    int bytes;
    int errorcode;

    errorcode = arts_init();
    if (errorcode < 0)
    {
        fprintf(stderr, "arts_init error: %s\n", arts_error_text(errorcode));
        return 1;
    }

    stream = arts_play_stream(44100, 16, 2, "artsctest");

    while((bytes = fread(buffer, 1, 8192, stdin)) > 0)
    {
        errorcode = arts_write(stream, buffer, bytes);
        if(errorcode < 0)
        {
            fprintf(stderr, "arts_write error: %s\n", arts_error_text(errorcode));
            return 1;
        }
    }

    arts_close_stream(stream);
    arts_free();

    return 0;
}

To easily compile and link programs using the aRts C API, the artsc-config utility is provided which knows which libraries you need to link and where the includes are. It is called using

artsc-config --libs

to find out the libraries and

artsc-config --cflags

to find out additional C compiler flags. The example above could have been compiled using the command line:

cc -o artsctest artsctest.c `artsc-config --cflags` `artsc-config --libs`

cc -o artsctest artsctest.c `artsc-config --cflags` `artsc-config --libs`

[TODO: generate the documentation for artsc.h using kdoc]