Colin.Guthr.ie Illegitimi non carborundum

29Oct/0910

So how does the KDE PulseAudio support work anyway?

So I think it's probably worth me writing up just exactly how the PulseAudio support in KDE's Phonon library actually works and why using it will give you some nice extra features!

This article follows on from my previous articles here and here. If you've not read them, then it would make sense to give them a quick glance :) (also as this was composed on my phone while on a flight, please forgive me any silly spelling errors I didn't spot when reviewing it!!)

First of all it's probably worth explaining what happens and what the phonon backend itself needs to be aware of and how it can interoperate with pulse.

So when libphonon is used in an application, a connection to PulseAudio will be attempted. If that connection fails everything will work as currently and nothing much changes. If the connection succeeds, we establish if "module-device-manager" is loaded. This is a module specifically written to implement the routing policy (an ordered priority list of devices for each category of sound produced - e.g. Notifications, Music or Video). If this module is not detected we offer a reduced/cut-down PulseAudio integration where the user will only see a single, virtual "PulseAudio" sound "device" listed in the KDE configuration system's multimedia section. If m-d-m is detected however, users will get the full rich experience. All of the devices ever seen by PulseAudio (I.e. Built in sound cards, PCI devices, USB speakers and headsets, Bluetooth devices and Apple Airtunes devices etc.) will be listed. If the device is not currently available, it will be shown greyed out as expected and you can of course reorder them to suit your own preferences.

So all that is well and good from a user perspective but "how does it work internally?" I hear you scream!

The Nitty Gritty

When m-d-m is used the device preference order is not actually stored in phonon itself as is the case when PulseAudio is not used, it is instead passed across to PulseAudio which takes over responsibility for keeping this priority list up to date. Why do it this way? Well while the preferences stored in KDE work fine for phonon apps they do not cover any non-phonon ones (which still make up the vast majority of sound producing applications on Linux). As someone involved with sound on Linux I've often been asked by confused users why their preferences are not honoured in e.g. Audacious or other non-phonon apps. As PulseAudio is integrated lower down the sound stack than phonon, ensuring it knows your device order preferences also ensures that it can route your audio appropriately according to your preferences for all apps! A clear bonus :)

Now as you may know phonon has it's own API calls to move streams to specific devices. This is something also handled in PulseAudio so it makes sense to simply wrap up the phonon API around these PA call which is exactly what I do! However, there is a complication. As phonon has pluggable backends, we don't actually speak to pulse directly to do the sound output (our connection to the PulseAudio daemon is only for querying and control purposes) so we don't know which stream is ours when the audio is actually playing. To get around this obscurity introduced by the pluggable backends, we take advantage of a facility in PulseAudio to attach arbitrary metadata to our streams by way of setting special environment variables. When phonon is instructed to start outputting audio, we set both the PulseAudio "media.role" property (this is the equivalent of the Phonon Categories system and we happily have an easy 1:1 mapping) and we generate a GUID for our stream and assign it as the "phonon.streamid" property. Our control connection sees all audio streams played in PulseAudio and can use this streamid to map back the PulseAudio streams to the phonon streams. This then allows us to process the stream move requests! It's a bit of a round about road to get this capability but it is the only way we can skirt around the obscurity provided by a pluggable backend system in a general way (the opaque interfaces provided by e.g. Xine, GStreamer and other backends would not expose this info directly anyway).

"But wait!" I hear those of you paying close attention scream. "Didn't you say above that PulseAudio will do the routing for us according to our preferences? Why not let it do that for phonon streams as well as non-phonon streams and forget about the whole GUID thing?" Well, if truth be told, I did start off doing it this way but it had two major issues. The first was the "Test" button not working properly - it always resulted in the highest priority device being used, no the one the user had selected! Not really ideal and more than a little confusing I'm sure you'll agree! The second problem related to visual feedback. When you reorder your devices or plug in a device of higher priority when sound is currently playing, phonon will visually inform the user of the device change. Handing over wholesale to PulseAudio meant that this feedback was lost. Again this is certainly not what I'd call seamless integration!

So at present the stream will be routed to the correct device by PulseAudio and then Phonon will also issue a move command which should in most cases be a NOOP. Likewise when devices are hotplugged (or unplugged!) PulseAudio will route them correctly and phonon will follow this up with a specific move request which will again end up being a NOOP. While this may seem pointless it does allow for the phonon API to be use properly (as needed by the test button) and give visual feedback)

Backends

So what does it mean for a backend? Well the backend needs to be somewhat aware of PulseAudio. Thankfully this is pretty trivial thanks to the PulseSupport class I've written. The backend should ask PulseSupport if pulse is detected if so it should check for and enable it's own PulseAudio output system. If this cannot be fond or activated it can tell PulseSupport that it's a no go and everything should fall back to how things work without any PulseAudio support. There are a couple other bits and bobs that a backend needs to do to work fully but that's the bulk of it.

So I hope I've provided a good and in-depth overview of how this support works. Please keep any comments on topic. If you want to post comments about PA in general, please do so on (and after reading!) this article.

This support is included in the upcoming Mandriva 2010.0 Release which will be hitting the streets very soon.

Share and Enjoy:
  • Digg
  • StumbleUpon
  • del.icio.us
  • Facebook
  • Yahoo! Buzz
  • Twitter
  • Google Bookmarks
  • Identi.ca
  • Slashdot
Comments (10) Trackbacks (2)
  1. Impressive!

  2. aww, I feel bad now for doing my usual dance: Upgrade Kubuntu, sound broken, ps -A | grep pulse, O_O, sudo aptitude remove pulse, Sound works again :)

    Good to see people work on this, I am sure that in the future when everyone has 3 sound devices pulseaudio will be really interesting.

    • *buntu is the wrong system to try out PulseAudio on anyway, see the PA developer’s blog at http://0pointer.de/blog
      Looks like very bad management on Ubuntu’s part there.

      • This is certainly true in the past and while Ubuntu are still doing some things a bit differently for no real reason, the guys involved are much more active with us upstream and are certainly providing a much better PulseAudio experience these days. So while Lennart is right in his comments, it’s not really as bad as it could be :)

  3. Nice to see work being done here. Two questions though.

    1. Since the backends need to be aware of PA, what are their current status regarding that? Especially Xine I suppose since I at least never have had much luck with the gstreamer one.

    2. Any hope for this to become a part of KDE 4.4? I have no intention of switching distro just for this…

    • Hi,

      Both the Xine and GStreamer backends work with this patchset in the same way. Sadly the xine-pulse output layer itself is not particularly great in my experience. We ship GStreamer by default so my focus has not been on improving that side of things. I’m quietly hoping that the Xine backend will disappear and be replaced by either GStreamer (if it gets a bit more love) or VLC etc. Having spoken to the guy who wrote the xine-pulse output layer, I believe there is not much flexibility in the API at their end. That said, I’m all for getting PA more widely adopted and if I do find time to take a look at that weak link in the chain, I certainly will.

      As for getting into 4.4 I’m not sure of the time constraints there. I am waiting for a Qt code review of the phonon changes prior to officially landing it and we should really push an offical PulseAudio 0.9.20 release too to ensure that module-device-manager is available officially without patching.

      That said, you are more than welcome to poke the relevant people in your distro of choice and point out this work. I’d happily discuss with the people who do the integration work any issues or questions they may have.

      Hope this helps :)

  4. So I see by Esben that PulseAudio still sucks. In Kubuntu 9.04 all I saw was repeated random threading-related crashes handling only 2 sound cards. I’m in no hurry to try PulseAudio in 9.10 either, as I doubt it ever gets better (it’s a shame too, I’d like to use it). Yeah I guess I’m glad PulseAudio will get a phonon backend but Esben, I personally would be more glad PulseAudio actually worked properly haha.

    So this PulseAudio phonon is not in standard KDE but only Mandriva then?

    • Well Esben (along with many others) has taken the “quick” way out. There is nothing wrong with that and I don’t want to say this is “wrong” etc., but the evidence is simply not there to say “PulseAudio sucks”. Configuring and deploying PA is a complex process. Many proplems with the low level drivers have been fixed very recently and with upgrades old user preferences can get in the way of a working system. With phonon especially, integration with PA is lacking which can often lead to configuration problems that cause things to go silent. When this happens it’s usually not too difficult to work out what is wrong and fix the configuration, but the easiest way is to obviously disable PA and the old configuration probably starts working again. This in no way reflects on PA itself, merely the user configuration and the ease with which the distro in question deals with that. Hope that makes sense!

      As for the patches, I’m not aware of other distros using this patchset yet, but they are obviously most welcome to do so (no one has asked me any questions so it could be that they are using it and it “just works” but that’s probably a bit unlikely!!). I’m very much working with upstream to get these patches committed officially but we will need to wait until PulseAudio 0.9.20 is released before we can do that as it contains the necessary support for this (in Mandriva I backport the support to PA 0.9.19).

  5. Hi, I only found your blog post(s) through many clicks via bugtracker of pulseaudio (http://pulseaudio.org/ticket/706), then searching google for the module-device-manager. These posts really should get listed on planetkde, I am shure I am not the only one who is excited about this. I would be so happy, if sound plug&play would work for all apps (via pulse-alsa-emulation) in kde and could be controlled from there. Also the priority-list is exactly what I need. Hope to see this in kde-4.4.

    • Hi Peter – My recent posts are indeed now listed on planetkde so it should be more exposed. The main work is heading for 4.4 release and some upcoming changes to kmix should also make it into kde 4.5 :)


Leave a comment