As well as taking part in various discussions, I was able to spend some time cooking up a UI to control the configuration of PulseAudio and the various cards attached.When adding PulseAudio support to the various parts of KDE that need it (Phonon, KMix), there was an important part of the puzzle missing: a card profile selector, and a sink/source port selector. I had always intended to include this functionality somewhere, but the KMix framework didn't really allow for it neatly (I could have created a separate dialog of course but it didn't quite feel right).

The eagle eyed readers may have seen a sneak preview of this feature when looking at the KDE-specific help page I wrote up on the PulseAudio website. So I give you this: the Speaker Setup tab in System Settings.

Speaker Setup GUI

The little icon in the middle is your user icon, so you'll see this differently unless your a weird stalker type and have set your icon to mine :s

The various drop downs allow fully control over all cards that are attached to the system. The buttons on the main pane allow you to test each speaker separately. In order to test the speakers, libcanberra is used. libcanberra is an implementation of the Free Desktop Sound Theme specification. It allows this test to be implemented with minimum hassle and I'm not personally interested in reinventing the wheel, hence the use of this library and the additional dependency.  Some people dislike using libcanberra in KDE (as was apparent from some discussions at the Multimedia Sprint), but I believe the reasons were often personal ones and not related to the usefulness of the code. If someone really wants to factor this out, they can but I have no interest in doing so and will prefer to use the existing implementation whenever possible (and this would include any future implementation I may or may not do with regards to support the Sound Theme Specification in KDE (or maybe Qt) directly)

When a 5.1 surround system is presented, the GUI is obviously a bit more advanced:

Speaker Setup GUI (5.1)

For those naysayers, I've tried my best to ensure that the compile will work fine without PulseAudio installed. It should also degrade gracefully when it is compiled with PA support, but PA is not configured to be used at runtime. The whole tab will simply not be available.

There is still one small problem left in that I don't handle disconnection/reconnection yet. This causes the GUI to crash if the PA server is stopped and restarted. This is not typically something that happens, but it's still something I will fix shortly all the same.

This code is now in trunk (r1154776) so feel free to try it out and report other bugs etc. This GUI is also included in Mandriva Cooker (I did want to include it prior to 2010.1 release, but the timing didn't work out - tho' it probably would have been OK considering the delays that cropped up in the release process). I expect this functionality to be included in any updated/backported versions of KDE for 2010.1.

For reference, in case you didn't spot it yourself, this GUI was heavily inspired by the gnome-speaker-setup utility.

So the trip to Randa was pretty uneventful. My bus did bread down on the way to the airport and as we were waiting for a replacement, the bus after the one I caught overtook us! We did catch it up and ultimately were asked to get on it at the RBS HQ, which did little to settle my nerves about catching the flight on time. In the end the latter bus was fine time-wise anyway. I was pleased to find an Ian Rankin book at the airport I'd not read yet, so quickly purchased it and rushed on through.

When I arrived in Geneva, I had missed some other folks that were there an hour or so before me, but I managed to board the right train and headed towards my destination. I had an hour to wait for the transfer in Visp, but the hot weather meant I really wasn't that bothered!

When I arrived I was greeted by a several folks from various KDE folks. While names were familiar the only familiar face was Helio whom I'd worked with on various Mandriva projects and met at Akademy 2007. Everyone was really nice and the house was fantastic.

I really can't say enough good things about the organisation. Mario was fantastic in his admin duties and the convenience of the house, where accommodation, meals and hacking/discussions sessions were all in one place, was perfect for this kind of event.

I met lots of really nice people and put faces to several names. It was especially great to meet the Amarok guys whom I've conversed with in the past on the mailing list and on IRC and it was good to meet with Christian Esken who is the current kmix maintainer seeing as I've pushed a lot of PulseAudio related patches into that project of late. Meeting Jean-Baptiste Kempf was another highlight. I've learned a great deal about VLC, or rather VideoLAN as it should be more accurately called over the past few days and definitely think of it differently now (in very much a good way - and for clarity I am really referring to the scope of the project here - I wasn't fully aware of how flexible it is until JB's presentation!).

In terms of actual productivity, I was able to start and complete (mostly!) a project to enable easy setup and testing of mutli-channel sound cards on KDE using PulseAudio. This work was heavily inspired by the tool Lennart wrote for Gnome - gnome-speaker-setup, but I've integrated it nicely into the same KCM module that is used for configure device preferences. I'll blog about this in more depth in the coming weeks. There is a very slim chance it can get into Mandriva 2010.1, but due to the timings it's rather unlikely at this stage.

We were also able to have what I think was a productive first meeting with the Qt developers in Brisbane, Australia, facilitated by Knut Yrvin. I wont go into too much detail here as I have a much longer and more detailed blog entry on this topic to post shortly. Keep your eyes peeled for that one.

The return journey was also uneventful and I arrived in Geneva at around 3:30pm and proceeded to walk in the exact opposite direction to what I should have done to get to my hostel. Thank goodness for GPS, although as it's designed for driving I did continue walking in the wrong direction for some time. That said, the weather was lovely and having a saunter around the centre of Geneva was rather pleasant! I did eventually find the hostel and subsequently went for a more organised stroll along the shore and sat in a little bar and devoured a few more chapters of my book while supping a couple cold beers - purely for their refreshment value on a hot day mind you!

So I'd like to sign off with a big thank you to all the people who helped make this happen. For KDE e.V. for sponsorship and especially to Mario for his organisation, but also to all the new friends and those friends who now have faces rather than just IRC nicks

Until next time...

Sadly the GStreamer backend was plagued with some problems. I wont go into the details (primary because I don't really know them all!) but many KDE developers felt that it just wasn't appropriate and decided to write a Xine backend for Phonon instead (NOTE: This is not actually correct. Please read comment by Kevin Kofler below). This matured quickly, but it was always had it's own set of problems that resulted in many hacks to be introduced at the application level. This was very bad as it meant that those of us (Mandriva included) who stuck with and helped fix the GStreamer backend were sometimes left a little out in the cold due to these hacks. Ultimately this story has concluded with the Xine backend being more or less abandoned now, and renewed focus being put on the new VLC backend. VLC, or more accurately "VideoLAN", is a multimedia framework, very similar to GStreamer. It has a very well maintained code base and support for numerous codecs and formats. It is also totally cross platform, working with Windows, OSX and of course Linux and several embedded OSs to boot. Having a single Phonon backend with uniform capabilities and support across all platforms is very desirable, so progress with this is quite rapid and an official release of the Phonon-VLC backend will be available in a matter of weeks.

But this is all about Phonon, not Qt Multimedia or Qt Mobility. So where does this all fit in? Well, Phonon has ultimately not really been developed much by Qt. That's not to say they have totally forgotten about it. I am assured that bugs reported will be looked at, time permitting, and that patches from downstream will be tested and merged, but it's fairly clear that it will not be actively developed further and no new features will come from the Qt side. Their new babies are Qt Multimedia and Qt Mobility.

Qt Multimedia, and arguably to a larger extent, Qt Mobility has been developed with a strong focus on mobile usage. Now that Qt is owned by Nokia this is not surprising! So what was the motivation for this and how does it differ from Phonon? Well, like Phonon it also makes use of existing projects to provide the core media decode capabilities. It will continue to use GStreamer for most "linuxy" systems and others for e.g. Windows and OSX. It differs however with regards to focus. It encompasses several additional features regarding the various needs for a typical mobile environment, such as still image and video capture (think cameras on a mobile phone).

There are several other key differences too but I wouldn't like to go into detail just now (I'm not really familiar enough with either Qt MM or Qt Mob to comment in any great depth), but suffice to say that this general abandonment of innovation in Phonon in favour of a new project has left several people in the KDE multimedia community feeling rather uncertain about what to do next; which horse to back!

Here at the KDE Multimedia Sprint in Randa, Switzerland, we were lucky enough to discuss the various issues with the Qt Multimedia/Mobility guys in Brisbane, Australia via a video conference. Well it was supposed to be a video conference, but despite it being between two groups of multimedia geeks, video completely failed us (various firewall issues for one solution and the lack of admin rights at the remote end for another!), meant we had to make do with a normal phone call! Such is life! Anyway, they were kind enough to give us an overview of Qt Multimedia/Mobility and let us know their approximate roadmap.

The outcome of this meeting was more or less that KDE Multimedia focus for the short term will remain with Phonon but the Qt guys will try and be more open about their plans and try and work better with the community longer term. At present, this is an uphill struggle as the VLC project has been very active in the KDE community for the last little while and seems to fit the needs in both the short and relatively long term of KDE, albeit in a way that is not currently exposed via Phonon API. This previous lack of communication is something that will be hard to overcome, but using a Qt integrated and supported solution definitely has advantages in the future - if they really mean it this time! The fact that the code is also used on Nokia mobile platforms give a fairly good reassurance that this will indeed be the case at least!

The rather obvious question of "Why not extend Phonon?" was also raised. While it was difficult to hear exactly the responses, the general reasoning seemed to be that there were some design mistakes in Phonon that basically meant that each individual feature that needed to be added, actually needed to be done at the backend level first (for each and every backend) and then exposed out through the API. With the Qt Mobility approach, more of the core features and functionality can be implemented once, with it only reaching out the the underlying platform specific system for specific and defined operations.

We enquired about the possibility of having a single Qt Multimedia backend written for Phonon, that allowed the Phonon API to continue in use at the application level while using the underlying systems/framework provided by Qt Mobility and/or Multimedia and deprecating all other Phonon backends. This is attractive in the sense that applications to not need to change horses mid stream and can gradually move over to using a pure Qt Multimedia API if indeed that is determined to be the most desirable outcome. That said it is also unattractive in some ways too. This adds yet another layer of abstraction to a system that many people argue is a layer of abstraction too far in itself! i.e. Application uses Phonon which uses Qt Multimedia which uses GStreamer; Why not just Application uses GStreamer? (there are of course many reasons to add a layer of abstraction; cross platform support being a primary one; but two layers is arguably not the best situation, especially on mobile platforms where wasted CPU cycles really hurt). That said, it would still be an acceptable stepping stone for most people. Regardless, there was no actual commitment to this eventuality from the Qt side. It was seen as a good idea but it's likely not something that there will be time (aka budget) for developing in the near term, which is understandable.

Many of the problems with Phonon still remain however. Qt Mobility will still require the setup of different underlying support libraries on different platforms. Just as before, GStreamer will be required for it to work on Linux and downstream (distributions and, to a lesser extent, users) will need to ensure that this is all configured and working correctly. As with the original Phonon, this backend will be different on different platforms, which is not ideal for the downstream.

So in the short to medium term (e.g. the next one to two years at least), Phonon will continue to be the primary media framework on KDE, and development on the Phonon-VLC backend will be seen as the best way forward to provide a standard experience across all platforms. GStreamer as a phonon backend will also continue, although the mplayer and xine backends should be considered deprecated.

With regards to Qt Multimedia and Mobility development itself, the Qt guys will try to be more open; opening a private mailing list to more access and generally being more communicative.

Those of you who regularly read this blog will be no doubt wondering where the PulseAudio connection is. Well, from what I gather, this is a primary difference of Qt Mobility and Qt Multimedia. Qt Mutlimedia seems to use ALSA directly and thus is not ideally suited for mobile situations (PulseAudio's timer based scheduling now being pretty much universally accepted as being the preferred approach on mobile platforms). Qt Mobility does actually use PulseAudio indirectly via GStreamer, but it does not seem to do anything special with regards to the "buffer-time" or "latency-time" attributes of the pulsesink (not that pulsesink is actually referred to directly in the appropriate place anyway from what I can tell). These attributes map directly to the tlength and minreq attributes of the buffer metrics in a PulseAudio stream. These are very important when it comes to mobile environments as the general aim is to provide as high a latency as possible. Higher latency means lower power usage (and for those wondering, this does not necessarily affect A/V sync on video playback - it's just about how much data you pump into the audio buffers before sleeping until you know it'll be almost empty). For a system that deals with audio playback on a mobile device, this is very important.

Now Phonon is also similarly ignorant to these properties when using PulseAudio. The the help of folk from Intel we're going to look at increasing the defaults in Phonon when used with GStreamer pulsesink to see if higher latencies would bring benefits without any drawbacks at the application level (I don't think any of the uses of Phonon will have problems with this approach), but we'll also have to consider how best to expose this via the Phonon API to allow the application level to state more explicitly if this matters. When capture APIs become possible on Phonon this will become important. The whole question of how much thought has gone into this kind of power saving is Qt Mobility is still very much at the forefront of my mind. I hope to be able to discuss things in more depth with the developers in due course, hopefully influencing them to extend the API in a similar was as described above.

More generally, those of us in the KDE community will try to get involved with Qt Multimedia/Mobility but until such times as it is easily configurable on the Desktop or there is a Phonon backend based on it, it will be hard to get involved too deeply.

Overall I think everyone has an open mind, but with the current focus on VLC and the functionality it provides, it will be the most interesting bits for most KDE Multimedia guys for the short term.

With regards to all of the above, I hope I've made fair and levelled comments. If you feel this is not the case, please tell me so in the comments below. I will happily retract or rephrase or reconsider any point if a suitable argument is made. I do not intend to represent anyone else in the KDE community in a way they do not feel is accurate so I'll be happy to both comment and edit the article with suitable annotation if this is deemed necessary.

Edits:

1. 27th May: Clarification of my incorrect statement of when/why Xine backend was introduced. Thanks to those who pointed it out.

Over the last fee weeks and months there has been growing concern about the reforms proposed by in the Digital Economy Bill which is going today for a third reading in the House of Commons (UK Parliament).

This legislation as it stands is quite frankly horribly Draconian in it's approach and makes a mockery of the legal system's overarching principle of the right to be presumed innocent until proven guilty.

But before I talk about the content of the bill itself, I have to write about the rather interesting event that took place yesterday in concerning the second reading.

Twittersphere/Public Engagement

On Twitter, a medium I quite often ridicule but still watch and occasionally participate with curious interest (I feel our ability to be hypocritical is what separates us humans from the animals :p) we saw a world-wide trend for the tag #debill. Looking at the results flow in during the Parliamentary debate broadcast live on BBC Parliament was very surprising for me. Sure, I expect people to be online discussing every nuance of the latest #drwho as it aired on Saturday. It's a nerdy show (I like it!) and it appeals to lots of people who are part of the "nu media" generation, so it's no surprise that the two mediums converge. But watching BBC Parliament and having a "twitstorm" of comment regarding it was, for me at least, the first time I've seen such public participation in politics for a long long time.

Politicians are often lamenting on voter apathy, but this showed that there is real interest in the political process when the process truly engages our interest. Ultimately I vote in the politician I believe will do the best job for me and those people and causes I care about. That's why I vote. I don't want to have a referendum on every contentious issue. I don't have the time to read the EU constitution and make an informed judgement on it's contents and the legal ramifications therein. I'm have neither the qualifications nor the time to deal with this. That's why my tax money goes towards paying people who are (or at least are supposed to be) qualified and do have the time. Having referendums usually means that the verbal diarrhoea that is spouted by the tabloid press often has major sway in decision making. I certainly do not want that. But it truly dismays me to see what went on in the chamber yesterday. Time after time, politicians stood up and lamented on the way this bill had been pushed through in the wash (the time period before parliament is dissolved before a general election) without proper scrutiny and without full disclosure (several MPs were not even able to read the amendments before the second reading!!), but then went on to say, "but I'll support it". Sorry? Excuse me? What? WHAT? If this is democracy in action, I feel sick.

This debate attracted tens of thousands of people to follow and debate a very important matter for the future of Britain's economy and society. It attracted many young people who had not before taken any interest in politics to listen and debate the issues. This was a real opportunity to engage the young and the apathetic. To make sure they valued and used their vote in the coming election. This is the generate that discuss everything with everyone. For each person involved, their opinions and concerns would have reached five more people. This was social media primed and ready. And what happened? Everyone involved said that the process was flawed but they still carried on with it? This is wrong, wrong, wrong. If the process if flawed, throw it out and start again. It doesn't matter what you are doing, that is the only outcome that can be honourably and morally supported. Time after time, it was stated that if they don't support this flawed procedure it would be tantamount to authorising illegal file sharing.... excuse me? How exactly? Due process not followed, start again. How does that say to anyone: "It's OK to download stuff in violation of the creators copyright"? It doesn't. Anyone who thinks it does is either stupid or has other agendas and thinks we, the public, are stupid and they can get away with this approach.

Sharing is Bad m'kay?

So on to the bill itself. There are many things I've read about the bill that are disagreeable to just about everyone except large media companies who want to protect the status quo. I don't deny that file sharing is something that needs to be more closely regulated - that doesn't mean I support an outright ban. It's pretty accepted that a degree of piracy actually encourages paid consumption. Microsoft secretly loves a degree of piracy as it locks in more people to their walled garden ecosystem. While I don't have any directly figures, the people I know who file share are also some of the largest consumers of paid media content. In order to support the "creative industries" (i.e. the economy) which is a question that must be when analysing any part of this bill, we have to look at how society works and how to encourage legitimate consumption of media. To encourage things in this way we need to stop assuming that the default stance of prohibition and punitive responses to accusations are the right way to go. There are tens of thousands of people scrambling to be part of the creative industry - ways to encourage this are critical and that means thinking about the little guys, not just the incumbent, dominant corporations that have a significant interest in maintaining the status quo.

Virtually everyone in the UK who owns a personal media player (e.g. an iPod, an iRiver, most mobile phones and that one guy who has a Zune), has infringed copyright by converting a CD they own to some other format - e.g. MP3. This is morally acceptable to approximately 100% of people, but yet it is copyright infringement. How many people have lent a DVD or Blu-Ray to a friend or invited friends over to watch a film? Have you ever read the disclaimer at the start of a movie? Sharing is Bad m'kay?

The rhetoric about supporting the creative industries is just that, rhetoric. If they really cared about supporting a digital economy they'd be looking at things with unbiased, fresh and open minds.

Theft

MP after MP stood up and talked about theft. I cannot think of a single example they gave that constituted theft. They are using this term incorrectly and it is biased towards the goals they want to achieve. It is simply not acceptable that they are deliberately skewing the debate via this technique. Theft means denying someone of property. If I break into your house and steal your coffee table, it's theft. If I break into your house and bring power tools and wood, then diligently make an exact replica of your coffee table then it's copyright infringement (assuming the table is a copyrighted work). And it's making a mess and loud noises probably too. But it's NOT theft. To call it theft just shows the complete ignorance of the matter being discussed or it shows that other agendas are taking priority over their responsibilities to debate and discuss things without prejudice. I feel I've had my democratic right stolen from me. No wait... not stolen, violated!

Pipes

The "Internet" is also being seen as a single thing. It's not. It's a myriad of different protocols and services all running on one pipe. The Internet is actually very similar in concept to electricity. I have an electricity supply to my house, using this generic resource I can plug in lamps, ovens, fridges, heating and safety equipment. I can also power my computer. And in a similar way my Internet supply allows me to access my Facebook photos from my digital photo frame, I can watch TV over iPlayer, I can browse the world wide web, I can read emails and I can share files. If I am accused of file sharing and I cannot prove my innocence, this bill proposes that I will have my Internet supply stopped. Excuse me? My whole internet supply? Why not my electricity supply? The Internet is a generic resource, just like electricity. I used the internet to share files, but I also used electricity. Hell in the old days I use electricity to record a record to a tape it's copyright infringement and electricity allowed me to do it! Why is it that my whole Internet supply is being restricted and not my whole electricity supply? I cannot see why one of these drivers for the copyright infringement is being chosen over the other? Why not just exclude me from the small subsection of the Internet that I used to do this bad thing? Oh, because it's possible to encrypt content and tunnel data via proxies and generally hide what you are doing if you are that way inclined. You can't achieve this technically with any degree of certainty, so you just ban the whole lot? I can use electricity for illegal acts just as I use the internet, you can't exclude me technically from using electricity for illegal things, but no-one would even consider turning off power as a punishment.

And this could turn out to be a serious problem. While I believe it's an inevitable consequence of over regulation in any medium, this legislation as proposed will generally foster the use of encryption technologies. The protocols that are quite open and available to DPI (deep packet inspection) will become locked down and very difficult to monitor. The general population will have more knowledge about encrypting and easier access to encryption systems. Information that our security services can snoop quite easily just now will become much, much harder to deal with. Disorganised groups of would be terrorists will be able to organise themselves with a greater degree of privacy. While this is something that will happen anyway, encouragement in this area is hardly likely to please the guys over at DARPA. Now this isn't really a major point of my article but I had to justify the catchy subject somehow.

So back to the main point, the punitive mechanisms here are just fundamentally wrong. Almost everyone accesses the internet via router these days so almost every house becomes an ISP, responsible for dishing out access to laptops, PVRs, STBs, phones, photo frames and desktop machines.... and toasters and fridges and whatever else is "Internet enabled" tomorrow. While those on the outside could potentially prove that some copyright infringement happened on my pipe, the could not prove who within the pipe is responsible. Therefore the owner of the pipe is held liable. That's basically everyone in the UK being held liable as an ISP. And if I as an ISP allowed, it why is my upstream ISP not equally liable? The technical process here is simple too ambiguous to lay any groundwork for sensible action. The principles need to be rethought and better drafted.

Shining stars

I could talk about other parts of the bill at length, but all in all it's not worth it. The third reading is today and there is little time to wax lyrical. There were some shining stars present, Tom Watson and Fiona Mactaggart were two excellent examples and I thoroughly support them in what they said. Keep up the good work.

David Cameron was yesterday talking about "listening to the ignored". Well here is your chance David. Why not listen to the tens of thousands of people who are up in arms about this and are being completely ignored. Take a principled stand and shoot down this bill. If your party gets power at the General Election (and to be honest I really don't know who I want in power right now - they all look like pretty miserable options), then all you'll do is moan about this bill when it all goes wrong anyway. So why support it? Take a stand on principle and throw it out. If you do get power, do it right and that way you'll get all the praise if it goes well.... though you'll also get all the blame if it goes wrong which is why I guess your party is supporting it with one hand while punching it with the other.

Democracy in action. I feel so much more apathetic now that I did yesterday.

I think that one twitterite got it right yesterday when he said: "Excellent, a bill proposed by the unelected, debated by the ignorant and voted on by the absent. #debill"

As a packager, it's important for me to be able to test the packages I produce so having a system that is solely running the latest and greatest upstream versions is not desirable. In order to do this I don't maintain two separate installations (that's too complicated and too much effort), rather I build all the upstream stuff into it's own prefix and then run it from there. It's totally separate from my system binaries (and shouldn't ever need root). Cleaning up is as simple as an rm -rf.So when you have a fresh checkout (git clone or svn checkout) of the code in question, I typically run the following:

[colin@jimmy kdemultimedia4]$ mkdir build
[colin@jimmy kdemultimedia4]$ cd build
[colin@jimmy build]$ kdecmake ..
[colin@jimmy build]$ make install

Nothing too magic there... but what is this strange "kdecmake" command? Well it's a simple little script I wrote to handle the options I need. This is the contents of that script::

#!/bin/bash

PREFIX=$HOME/Development/Personal/KDE/prefix
LIB=lib
if [ "$(uname -m)" = "x86_64" ]; then
  LIB=lib64
fi

exec cmake -DCMAKE_MODULE_PATH=$HOME/Development/Personal/Phonon/cmake-hacks -DCMAKE_INSTALL_PREFIX=$PREFIX -DLIB_INSTALL_DIR=$PREFIX/$LIB -DCMAKE_BUILD_TYPE=debugfull $*

Nothing too exciting there, just a few options etc. The "prefix" I choose is $HOME/Development/Personal/KDE/prefix. This is the one and only folder I need to kill when I want to trash my "install" and start again. There is a little cmake hack there for Phonon development - this is due to the FindPhonon.cmake always finding the system Phonon and not my custom one. The hack in there is just to make a FindPhonon.cmake that forces the detected version to my custom prefix. It's the ugliest bit of the whole setup, so don't worry too much about it if you don't hack on Phonon!

So this will build and install the applications but how about running them? Well I have another little script for that which is sourced into the current environment and then used to run the applications itself:

[colin@jimmy ~]$ . kde4env.sh

[KDE: colin@jimmy ~]$ which kmix
/home/colin/Development/Personal/KDE/prefix/bin/kmix
[KDE: colin@jimmy ~]$ kmix --nofork

So this is how the application can be run, it finds the right version and the libraries etc. are all loaded from the right place too. Quite handy all in all Here are the contents of the kde4env.sh file:

# To be sourced

PREFIX=$HOME/Development/Personal/KDE/prefix
LIB=lib
if [ "$(uname -m)" = "x86_64" ]; then
  LIB=lib64
fi

export PKG_CONFIG_DIRS=$PREFIX/$LIB/pkgconfig
export XDG_DATA_DIRS=$PREFIX/share:$XDG_DATA_DIRS
export PATH=$PREFIX/bin:$PATH
export LD_LIBRARY_PATH=$PREFIX/$LIB:$LD_LIBRARY_PATH
export KDEDIRS=$PREFIX:/usr

kbuildsycoca4 --noincremental

export PS1="$(echo -n "$PS1" | sed "s|\\\u@|KDE: \\\u@|")"

Now one warning. This approach uses your existing home and thus settings. If you're running a newer version of the app in question it may upgrade it's data along the way and subsequently prevent it working with the system version. To get around this, you can set the KDEHOME environment variable in the kde4env.sh file and keep things totally separate.

Now, for some specific debug info concerning Amarok and perhaps Akonadi. Both these applications use an embedded or private version of MySQL to store their data. This is great but it can complicate issues when trying to debug the data in the database itself. Here is a very simple script I wrote (with helpful guidance from Leo on the Amarok team):

#!/bin/bash

cd ~/.kde4/share/apps/amarok
trap 'kill $(jobs -p)' EXIT
/usr/sbin/mysqld --defaults-file=`pwd`/my.cnf --default-storage-engine=MyISAM --datadir=`pwd`/mysqle --socket=`pwd`/sock --skip-grant-tables --skip-networking&
sleep 1
mysql --socket=`pwd`/sock amarok --prompt="Amarok> "

This little script will start a version of mysqld specifically for the files/data in the Amarok data folder. It then starts a mysql client that connects to that daemon. When the client exits the mysqld process used is automatically killed. It's probably not a good idea to run this command at the same time as Amarok itself runs, but it's excellent for debugging and checking the data after a change etc.

Nothing in this article is particularly new or ground breaking, but hopefully it helps some people in some of their tasks and makes contributing that bit easier!

Firstly, it was pointed out to me that the use of a QEventLoop was not the best approach when connecting to PulseAudio from a KDE app. I used this technique in Phonon and while it seemed to work fine in testing it did trigger some problems in some unlikely places: namely this bug as reported against Okular! So I changed how the connection test to PulseAudio is run to make it blocking. This has so far appeared to fix the problems \o/

Now on to something interesting I've recently added support in Phonon for "hijacking" the volume change requests to allow them to be pushed directly to PulseAudio, totally avoiding the backend itself. In additional it is possible to listen to changes from PulseAudio and then emit the relevant Phonon signal to let applications know the volume has changed. This allows volumes changed in a PulseAudio mixer (pavucontrol and now KMix too!) to be reflected in the Application GUI (should it present it to the user).

Phonon provides it's own VolumeSlider widget that applications can use. I had to make some changes to this to avoid some complicated race conditions. As volume changes in PulseAudio are asynchronous, the change request may be followed some time later by the actual update notification. For this reason the it was not possible to use the "traditional" blockSignals(true) approach. I settled on two flags to achieve the necessary protection and it seems to be working well so far in my testing.

Then I tested Amarok and found it didn't work at all This is because Amarok has it's own volume slider widget and does not listen for any volume changes from the Phonon layer. I was able to make the necessary changes to support this without too much trouble, but in doing so I had to change the mute behaviour slightly. Due to (I'm told) legacy reasons, when the volume is muted, Amarok would also set the volume to 0. This was reflected in the GUI by the slider showing 0. While it would be possible to keep this behaviour it's not really desirable these days (the OSD always said "Volume: 0% (muted)" when muted which made it rather redundant to show the %age!) when mute is supported properly. If GUI feedback of mute status on the slider itself is desired (the image does change to a mute symbol already) then it would likely be more appropriate to change the slider colour to grey or similar (which is also how other media players - e.g. VLC represent things).

Anyway, with these issues fixed and suitable support for the feedback race conditions added to the Amarok slider code too, I've now got a rather nicely integrated solution

Here is a quick screencast I recorded to show you how all these changes: Phonon, KMix & Amarok all work together

(direct link)

As always, these latest changes are in Mandriva Cooker. The Amarok changes are not in the 2010.1 Beta 1 just recently released, but they will be part of Beta 2

Having now become quite good friends with Jim and Cat from Pure Mountain Holidays with whom I stayed last year, I reserved a space in their Chalet and booked my flights (from Glasgow as they were about a third of the price of the Edinburgh equivalent!).

I've had six days on the hill. The snow started off a little the worse for wear due to it not having been topped up for about three weeks. This wasn't ideal but it was still pretty good. As the week progressed the snow got a lot worse due to the temperature getting a lot higher (18 degrees at peak!). You can't have it both ways, and the sunshine was really nice even if the snow did suffer.

Jim Barker while out on our big tour

On Thursday myself, Jim and two other guests (Nigel and Ross) went for a fairly big trek up towards Avoriaz and over to Switzerland. It was technically Jim's day off (thanks for being so lenient Cat!) so it was very decent of him to drive us up there and come play with us (even tho' playing on the hill is something I'd also be happy to spend my day off doing!!). I tracked the trip on my phone and you can see the results on Google Earth (or on Google Maps if you don't have Google Earth installed). The tracks are quite hard to follow and there are a few GPS errors but it's pretty smart all in all.

We spend exactly 2 hours, 22 minutes and 22 seconds going downhill (although I did forget to turn off the tracking half way up one lift!) and covered a distance of about 24 miles. I only tracked the actual downhill sections and turned it off on the chairlifts - partly to conserve battery, but also to keep the average speed accurate. I averaged 10 miles an hour, which is pretty slow really, but maxed out at 40mph which I think was a fluke! All in all a great day out.

On the penultimate day, the snow came (as predicted) and while it was pretty wet in the morning the afternoon brightened up to give us some awesome powder. I made some fresh tracks through the trees on a few runs but came a cropper when I followed some tracks deeper into the woods which promptly stopped when they reached a small stream. Undeterred I ploughed on and over the stream and into the bank on the other side which I intended to hike over to get to the piste I knew was just on the other side. It looked a lot bigger when I was standing under it, but I eventually hauled myself over after having sunk to my waist on a couple of occasions!

A massive jump seen from the chairlift

The last day we woke to find it had been snowing for quite some time and as I trudged to the hill, the clouds parted and the sun shone for some of the best power/sun combo I've ever experienced. Pisted runs with about a foot of fresh snow on top made for some great sweeping carves and lovely spray behind. Awesome! It sadly started to cloud over again in the afternoon and the snow came on heavy and visibility became a problem but the morning more than made up for this slightly annoying end to the week.

All in all another great time with Jim and Cat. The other guests were all very friendly and good fun to chill out with. Until next year?

I wonder if the banana I left on my car roof at the airport will still be there when I get back?

So how does PulseAudio route your audio and what the hell do you mean by "audio routing" anyway? Well, PA handles multiple audio devices ("sinks" for output, "sources" for capture). When an application wants to play sound it says "Hey PulseAudio, play this!!", and PA tries it's best to comply. The application will typically not care about which device to actually output too - they delegate this job to PA (and ultimately to the user via PA's tools such as pavucontrol, gnome-volume-control and (more recently) kmix/phonon too). These tools allow you to move the stream to your preferred sink - no need for an unfamiliar GUI inside a particular application for choosing which audio device to use; users know where to look in their desktop environment to control sound settings. From an HCI/usability perspective I think this is very important (although incumbent users need to shake of their natural assumption that apps should provide a way to configure audio device settings).

With a fresh user account PA will attempt to calculate a suitable default device to use. It does this by assigning each sink an internal score of appropriateness. This is just to determine our initial defaults so it doesn't matter too much if we get this wrong, but obviously it's nice to try and get things right first time! So once our default device is chosen, when a stream is played, it will use this sink. Simple huh? Well not quite. What if I change the default while a stream is playing? My stream is moved across to the new default right? No. Setting the default device does not do this. It acts as a fallback, it's not an "active" default. If I stop my app and play it again, it will be played on the new fallback, but it wont move when the stream is "live".

Now this is just the very beginning so if you've become overexcited by now, best take a cold shower (and probably look up psychiatrists in the phone book...) as I'm about to move on to look at the "stream restore" database...

Stream Restoration

So the stream restore database is handled by module-stream-restore (m-s-r). It's part of the default PA install so 99.9% of users are likely using it. What this module does is to track when a user specifically moves a stream to a new device. When the user uses e.g. pavucontrol or kmix to move a particular stream to a new sink, this triggers a mechanism inside m-s-r that causes the sink name to be saved for that application. If that application appears at any point in the future and tries to play sound, m-s-r will try it's best to ensure that this sink is used. If the device is not currently available, we will ultimately use the fallback/default instead. If this saved sink becomes available at some point during the lifetime of the stream, the stream will be moved across automatically (which differs from the behaviour when setting the default sink).

Now it gets complicated. Despite my terminology above, m-s-r does not use "applications" per-se when it saves the device choice. It will look at a stream and select a "stream restore id" for it based on several bits of metadata attached to the stream. Firstly it will look to see if the stream has a role. If it does, the role is used as the identifying factor, not the individual application. So e.g. for event sounds, the "stream restore id" will actually be "sink-input-by-media-role:event" regardless of which application actually produced it. As m-s-r is responsible for storing volume, mute and device preferences, this means that all event sounds will have the same volume and will be played on the same device. If the stream does not have a role, PA checks for a few other things to try and create it's "stream restore id": it checks for an "application id" (a specific property set AFAIK only by a select few applications), an "application name" (which 99% of PA clients provide) and finally it checks for a media name property (which is pretty unlikely IMO, although I may have missed a use case here). Once the "stream restore id" is chosen it is saved for that stream and we will always use this rule when updating our saved volume, mute and device settings thereafter.

When a user moves the stream (using one of the afore mentioned GUI apps - or via any app that makes use of the pa_context_move_sink_input_by_*()  APIs (also applies to source_outputs)), the "stream restore rule" they are ultimately changing will depend on the metadata the application has provided PA in the first place. So for example, Amarok will be tagged with a "music" role, and thus when you use kmix to move the Amarok stream to a different device you are not just moving Amarok, you are actually moving all streams tagged as "music". (Update 1) But wait! It does not actually move the currently playing "music" streams. They will only be moved the next time a PA stream is created (this could be when the track changes, or it could be when the app restarts - it depends on implementation), or when the stream restore database is written by some PA client which requests the "apply_immediately" flag set.

This whole approach can be summed up with two flow diagrams:

Device Manager

OK, so that about covers m-s-r, now I'll talk a bit about module-device-manager (m-d-m). This is a module I developed principally for integration with KDE. The GUI exposed for the Phonon preferences in KDE is such that it lists all devices that have ever been available on the device even if they are not currently available. This allows them to be listed and arranged into a priority list. Whenever the highest priority device is available, it will be used. It's a relatively simple UI and one that users can easily understand and work with. For a degree of flexibility, the GUI allows for different priority lists to be configured for different Categories (which in PulseAudio terminology are called Roles). So m-d-m provides an API and Protocol extension to implement such a routing policy directly in PA which turned the KDE GUI effectively into a frontend to PulseAudio (internally this is abstracted via Phonon but this is not really important). So this m-d-m will route your audio for you to the appropriate device. Due to how it is implemented, m-s-r will still take priority, so if you move a particular stream and the device choice is saved by m-s-r, it will take priority over the role-based priority list of m-d-m. I have now provided a way in kmix to delete any application specific rules in m-s-r such that the m-d-m routing (and thus what is displayed in Phonon preferences) will be used again should an override have been in place.

Critique

Right so that's how things work. Now let's rip it apart and moan a little about the shitty bits.

• A default that is not a default: First things first. This is a question we get a lot: "I've set my default device but <insert victim application here> doesn't want to use it". Now that you know how the routing works due to the excellent overview above, you can probably work out what's going on here. Either m-s-r has a specific device saved for that application's role or the application itself, and that overrides the "default" choice; the user was playing the stream when they set the default device but this wont be honoured until a new stream is created; or m-d-m is implementing a role-based priority list preference (which in all cases overrides the default button due to having a priority list for all cases - one for each role and one for the default case). All of the above is flexible but it sure as hell is not clear to the user what the hell is going on. There needs to be some kind of unification and automatic reaction here. Having a simple "default" button IMO makes sense. It's a very easy concept to grasp for users and we should try to make it work in the most case and explain more clearly in the various GUIs as to why it may not work in certain circumstances.
• Why, Why, WHY?: Things are not clear to the user what is going on. In a typical setup we have m-s-r and it remembers one thing. Unless you know how it works, you cannot configure a scenario when you have three or more devices and have it "just work" for you. For me three devices is not uncommon. At work I have set of USB speakers I use and at home I have Airtunes and network-based PA servers I use too. I want them to "just work" without me having to fire up a GUI every time I want to use them. A primary problem just now is that unless you've taken the time to read and understand the above descriptions, it's very unclear to users as to how the default device works and what m-s-r does and how it operates. Clarity and transparency is needed here, or some way of making it "just work" without needing such and explanation.
• You gotta role with it?: Yes that is a rather contrived "problem", but the point is valid. As we go down the path of encouraging applications to include as much metadata as possible in their streams, assuming we ever reach such a zenith, then the ultimate end result is that all streams have their role specified. When this happens a fundamentally useful part of PulseAudios design is effectively crippled in the default setup: m-s-r will always pick it's stream restore id of "sink-input-by-role:fooo". All application's will no longer be able to chose their own individual audio device; you will only be able to pick the device for that whole class of application. All music players must go to one device. All video players to another, but nothing in between. Some applications set their role generically but can be used for other purposes e.g. totem/dragon are video players but may be being used for just music in a given instance - I'd like to use my preferred music device not my preferred video device but I don't want to affect all video players with this choice. Obviously it's better if a multi-purpose app can work out it's role dynamically based on content but the principle point still stands - you cannot move streams anymore (with the exception of the afore mentioned caveat for currently playing streams when the move is initiated), only whole classes of streams. Some may argue that this is OK/acceptable. Personally I don't think so.
• I don't want an override anymore: Say you've picked an override for your stream/role with m-s-r. Say you've now decided you don't want that override and just want to use the default device now? None of the GUI tools out there just now allow you to reset this (with the exception of the latest kmix in my Git Repository at the time of writing).

How do you solve a problem like Myrouting?

So the above problems basically make things a bit too much like a black box. It's not clear what's going on and it doesn't offer enough flexibility in many cases either. How do we solve this? I'll now outline what I think is the best possible implementation of a routing system and how it will work.

Overall Operation

Firstly we need to remove the internal concept of a single "default" device. As we've discussed before within the PA community, we will move to a priority list concept. A significant difference to the KDE approach of showing the device priority list for users to tweak, we will ensure that exposing this list is not a strict requirement for operation. In order to service this requirement we will still offer the existing UIs for setting the default device, but the internal behaviour will change. Setting a device as default would simply move it to the top of the priority list. This will ensure that simpler UIs can still operate and whenever the user prefers a device above other others they just set it as default. In order to get the perfect priority list, they may have to click the "default" button a couple of times with a given setup, but it will stabilise for them very quickly (for reference, this default priory list will be the equivalent of the current m-d-m priority list with the psuedo-role: "none").

As well as the default priority list, we will also implement similar lists for each role (again very similar to the current m-d-m role-priority lists). When a stream has a role, we will simply use this list rather than the overall default one. If a given role does not have a priority list of it's own, the default one will be used (in KDE the phonon GUI will likely enforce that a list exists for each role, but has GUI techniques to keep it in sync).

In addition to the above, each individual application will also have it's own priority list. Again this is fully optional. If such a list does not exist, the role-based list will be used instead, ultimately falling back to the default list if no role is present.

When a new device appears it will be added to the default priority list only. It is open for debate as to where this device should appear in this list - top or bottom, but this could ultimately be a user preference. The routing logic should ultimately check the most appropriate list first, try to find the highest priority device in the list that is currently available. If none are available it will use the next most appropriate list ultimately falling back to the default list which will always contain an available device.

New APIs

New APIs will be made available (either as extensions or baked in - details TBC). These APIs will allow the querying and editing of these priority lists (thus facilitating GUIs such as the one in KDE). When editing these lists the changes they reflect will always be immediately applied to any running streams. APIs will also be exposed to delete a  role-based list or an application specific list (NB the application list could potentially also be cleared via the existing APIs by passing PA_INVALID_INDEX or an empty or NULL device name).

Existing APIs

The existing APIs for setting the default device and moving a stream to a given device will still operate as just now but will operate differently internally.

As mentioned previously, setting a default device will update the overall default priority list. This change will be propagated to any currently running stream immediately. Thus a fresh user account with no priority lists saved, the default priority list will be used for all streams. Clicking on the default device will move any running streams to that new device. And thus the very basic use case will operate in an intuitive way.

If a user moves a specific stream to a new device via the (pa_context_move_sink_input_by_*()  style APIs) this will trigger the update of the application specific priority list. This is a stark change to the existing behaviour which will update the details for the currently used device choice mechanism. If the priority list for particular role is to be updated by a given GUI application, then the new APIs above should be used to achieve this  result. If a stream is moved and it does not yet have it's own priority list one will be created for it automatically (containing only the chosen device initially.

GUIs

So what can the GUIs show? Well there are a lot of options now:

• A very simple GUI which just shows a list of devices and allows a single default to be shown.
• A more complex GUI that shows all devices in the default list and allows the user to order them.
• One that shows both the default and the known roles and allows a default to be chosen for each (or expose the full list for user ordering)
• etc. etc.

(FWIW although it's possible I don't think it would be necessary to ever expose the per-application list although there should be a method for indicating that such a list exists and is in use and allow the user to delete it and user a higher level priority list).

Caveats

With regards to event sounds specifically, we want to make sure that these are handled differently to a regular stream produced by an application. For example I may want to use rhythm box to output sound to my Airtunes device and implement this via an application specific priority list. Any event sounds for this should not be pushed to this device. I'm not sure what the best solution to this conundrum is, but I suspect that a simple property set on the stream that says "do not allow per-application priority list updating" would be sufficient. This still allows streams to be moved to the device, but this wont be saved for later restoration (obviously moving an event sound is really hard due to their duration, so this logic is more for the principle rather than the practicality).

(Update 1) As I propose to apply changes whenever they happen if you run e.g. two paplay processes at the same time and change one to use a specific device, my logic dictates that both streams would be moved. This is a regression on the current behaviour where only when the stream is restarted will it be moved, but to be honest I don't think it's much of a problem. If it is desired to have specific control over a given instance of an application, then we need to either a public API to not save the move (much like our internal one), or support a proplist property that suppresses this (e.g. no API change but a loose "standard" for performing this type of operation. If you want to run an application in a specific way, then you can always set a property for the e.g. the "application id" as mentioned above which would make it appear as a unique application compared to running it normally. So flexibility still exists here (albeit rather complex), but we probably do want to deal with the default cases well and make the more niche ones possible via some degree of extra work.

Overall

OK, so this is quite an exhaustive proposal and I've not nailed down the exact best way to implement it, but I think that this approach is ultimately the most flexible possible that supports all of the interfaces desired and still exposes the flexibility of the PulseAudio system. I have not given much thought to the storage of volume/mute status (which m-s-r currently tracks along with it's single device preference).  My gut feeling here is a fairly similar approach where you should be allowed to set the default volume, a per-device volume and a per-application volume. The former two being used as defaults and as soon as the per-application volume is changed via any API call, we save it with that specific application for future reuse. APIs can set the default volume and the per-role volumes and this will be propagated to applications that do not have their own specific override in place.

So in summary, this is how I think things should work. I don't think it's actually as complicated to implement as I've made it sound (would probably only take about three or four times as long as it's taken me to write about it!!) and I believe it keeps the power and flexibility of the system, still allows for minimal control interfaces yet allowing more exposed and complex ones if required.

Updates:

[1] 16th Feb: Add notes about how moving a stream with a role does not move any currently live streams and how the proposed solution would prevent running two instances of the same app and moving the device independently for each (as one would follow the other) unless the application id was overridden for one of the instances.

I've spent quite a lot of time refining the initial support I added a few weeks back. What follows is a brief summary of the changes/improvements/bugfixes.

Firstly, some of the tabs (namely "Capture Streams" were not shown unless an application was actively capturing while kmix started... somewhat unlikely really :p This is now fixed.

Then there was a problem with Profiles. These would be created automatically when you went to the "Configure Channels..." dialog. There were actually several problems relating to profiles which ultimately lead to controls going AWOL - with no way in the GUI to see them even if the backend itself was actually aware of their presence. The concept of GUIProfiles does not really tie in too well with the concept of "Dynamic Mixers" (where controls come and go), but I think I've managed to get it all working nicely now.

And finally we come to something I said I wasn't going to do. In my last post I explained that moving streams to different devices was something best left to the System Settings -> Multimedia dialog. Lots of people commented on that post saying that individual stream moving is still needed. Well, not wanting to disappoint the public, I have now added relevant patches to enable this in KMix. The UI is not ideal yet, but you can access it via the context menu of Playback and Capture streams by right clicking on the sliders and selecting an option from the "Move" submenu.

There were several other small cosmetic fixes (like using the right icons for devices/applications in the GUI) too and a really annoying race condition due to a singleton class taking too long to start up that got fixed too.

All in all I'm rather pleased with the outcome for now. It's not perfect by any means (if the PA server dies there is not much feedback), but it's certainly a good start. Failing any massive problems with the build I've just submitted to Mandriva Cooker, I'll sync this to subversion trunk for everyone else to play with.

I had to make more changes to the core of kmix than I originally intended. I've tried to ensure I always remain backwards compatible but some subtle differences in profile loading certainly exists. I'm not sure if these are avoidable (I don't think so) so this may be a sticking point... time will tell.

Have fun

Quick screenie before a more detailed description:

What does this mean?

Well it means you will typically get three or four tabs in KMix that represent (in order), "Playback Devices", "Capture Devices", "Playback Stream" and "Capture Streams".

All the physical (and virtual) cards' output "sinks" will appear under "Playback Devices". I'm sure the names are pretty obvious so I wont explain further!

The functionality is close to pavucontrol, but with three important exceptions:

1. There is no equiv of the "Configuration" tab to change "Profile" for a given card.
2. There is no way to change "Ports" for a given Sink/Source (not all sound hardware supports this and it is intended to make this automatic in some cases - i.e. Headphone Jack Sensing - once it becomes reliable upstream in ALSA, but sometimes it will still be user choice (e.g. Amplified vs. Non-Amplified))
3. There is no way to move an given application from one output to another.

For 1 and 2 I intend to (eventually) provide a KCM to go alongside the Multimedia tab. For 3 you can move "categories" of applications to different devices via the Phonon settings in System Settings -> Multimedia already thanks to my previous patches.

Where can I get it?

It's already in Mandriva Cooker or you can grab it from my Git repo here: http://colin.guthr.ie/git/kdemultimedia/log/?h=pulse

The master branch is upstream svn. The pulse branch is my changes. Do a clone then issue:

 git diff master..pulse >mypatch.patch

or just build it directly.

Hopefully it'll be committed to trunk soon and if all goes well it can be easily backported to Mandriva 2010 too.

Caveats

Every time a new slider appears it is added to the "Shortcuts" system. This is OK for hardware devices, but for every application it can get a bit much.... I'll try and find a way to disable this (see "What's Left" below).

PulseAudio supports a pretty crazy limit on the number of channels a device or stream can have. Kmix only has support for a fairly standard set of elements. In come cases not all channels of a given device/stream may be shown in kmix due to this limitation. Stereo -> 5.1 setups should work fine tho'.

You Suck, I use PA but I want a Real Man's ALSA mixer!

Whatever floats your boat baby!

 $  KMIX_PULSEAUDIO_DISABLE=1 kmix

Knock yourself out!

What's Left?

I have two remaining issues that I do not think are show stoppers:

1. Everytime a new device shows up a new Global Shortcut dialog appears. I don't think this is any different to ALSA but as I now have per-app volume control, this dialog is also shown everytime a new application plays sound. It only happens the first time a given application plays a sound, but it could still be considered annoying by some.
2. If there are no capture streams running at startup the tab for that will never be displayed - likewise if the stream restore module in PA is not loaded (unlikely) the "Playback Streams" tab will never be displayed.
3. There are various things that could be more efficient (e.g. refreshing the GUI for a new device or application current redraws all tabs, not just the one that has changed when a new application appears or disappears)
4. Make KMix dumber! KMix is pretty clever and it tries to do some smart things like saving and restoring volume for you. But when PA is used, it knows better, so I need to ensure that KMix does not do any saving/restoring of actual volume values.
5. Use application icons for per-application streams. Just for eye candy, it would be nice to use the applications own icon in the GUI.