So the trip to Randa was pretty uneventful. My bus did bread down on the way to the airport and as we were waiting for a replacement, the bus after the one I caught overtook us! We did catch it up and ultimately were asked to get on it at the RBS HQ, which did little to settle my nerves about catching the flight on time. In the end the latter bus was fine time-wise anyway. I was pleased to find an Ian Rankin book at the airport I'd not read yet, so quickly purchased it and rushed on through.

When I arrived in Geneva, I had missed some other folks that were there an hour or so before me, but I managed to board the right train and headed towards my destination. I had an hour to wait for the transfer in Visp, but the hot weather meant I really wasn't that bothered!

When I arrived I was greeted by a several folks from various KDE folks. While names were familiar the only familiar face was Helio whom I'd worked with on various Mandriva projects and met at Akademy 2007. Everyone was really nice and the house was fantastic.

I really can't say enough good things about the organisation. Mario was fantastic in his admin duties and the convenience of the house, where accommodation, meals and hacking/discussions sessions were all in one place, was perfect for this kind of event.

I met lots of really nice people and put faces to several names. It was especially great to meet the Amarok guys whom I've conversed with in the past on the mailing list and on IRC and it was good to meet with Christian Esken who is the current kmix maintainer seeing as I've pushed a lot of PulseAudio related patches into that project of late. Meeting Jean-Baptiste Kempf was another highlight. I've learned a great deal about VLC, or rather VideoLAN as it should be more accurately called over the past few days and definitely think of it differently now (in very much a good way - and for clarity I am really referring to the scope of the project here - I wasn't fully aware of how flexible it is until JB's presentation!).

In terms of actual productivity, I was able to start and complete (mostly!) a project to enable easy setup and testing of mutli-channel sound cards on KDE using PulseAudio. This work was heavily inspired by the tool Lennart wrote for Gnome - gnome-speaker-setup, but I've integrated it nicely into the same KCM module that is used for configure device preferences. I'll blog about this in more depth in the coming weeks. There is a very slim chance it can get into Mandriva 2010.1, but due to the timings it's rather unlikely at this stage.

We were also able to have what I think was a productive first meeting with the Qt developers in Brisbane, Australia, facilitated by Knut Yrvin. I wont go into too much detail here as I have a much longer and more detailed blog entry on this topic to post shortly. Keep your eyes peeled for that one.

The return journey was also uneventful and I arrived in Geneva at around 3:30pm and proceeded to walk in the exact opposite direction to what I should have done to get to my hostel. Thank goodness for GPS, although as it's designed for driving I did continue walking in the wrong direction for some time. That said, the weather was lovely and having a saunter around the centre of Geneva was rather pleasant! I did eventually find the hostel and subsequently went for a more organised stroll along the shore and sat in a little bar and devoured a few more chapters of my book while supping a couple cold beers - purely for their refreshment value on a hot day mind you!

So I'd like to sign off with a big thank you to all the people who helped make this happen. For KDE e.V. for sponsorship and especially to Mario for his organisation, but also to all the new friends and those friends who now have faces rather than just IRC nicks

Until next time...

Sadly the GStreamer backend was plagued with some problems. I wont go into the details (primary because I don't really know them all!) but many KDE developers felt that it just wasn't appropriate and decided to write a Xine backend for Phonon instead (NOTE: This is not actually correct. Please read comment by Kevin Kofler below). This matured quickly, but it was always had it's own set of problems that resulted in many hacks to be introduced at the application level. This was very bad as it meant that those of us (Mandriva included) who stuck with and helped fix the GStreamer backend were sometimes left a little out in the cold due to these hacks. Ultimately this story has concluded with the Xine backend being more or less abandoned now, and renewed focus being put on the new VLC backend. VLC, or more accurately "VideoLAN", is a multimedia framework, very similar to GStreamer. It has a very well maintained code base and support for numerous codecs and formats. It is also totally cross platform, working with Windows, OSX and of course Linux and several embedded OSs to boot. Having a single Phonon backend with uniform capabilities and support across all platforms is very desirable, so progress with this is quite rapid and an official release of the Phonon-VLC backend will be available in a matter of weeks.

But this is all about Phonon, not Qt Multimedia or Qt Mobility. So where does this all fit in? Well, Phonon has ultimately not really been developed much by Qt. That's not to say they have totally forgotten about it. I am assured that bugs reported will be looked at, time permitting, and that patches from downstream will be tested and merged, but it's fairly clear that it will not be actively developed further and no new features will come from the Qt side. Their new babies are Qt Multimedia and Qt Mobility.

Qt Multimedia, and arguably to a larger extent, Qt Mobility has been developed with a strong focus on mobile usage. Now that Qt is owned by Nokia this is not surprising! So what was the motivation for this and how does it differ from Phonon? Well, like Phonon it also makes use of existing projects to provide the core media decode capabilities. It will continue to use GStreamer for most "linuxy" systems and others for e.g. Windows and OSX. It differs however with regards to focus. It encompasses several additional features regarding the various needs for a typical mobile environment, such as still image and video capture (think cameras on a mobile phone).

There are several other key differences too but I wouldn't like to go into detail just now (I'm not really familiar enough with either Qt MM or Qt Mob to comment in any great depth), but suffice to say that this general abandonment of innovation in Phonon in favour of a new project has left several people in the KDE multimedia community feeling rather uncertain about what to do next; which horse to back!

Here at the KDE Multimedia Sprint in Randa, Switzerland, we were lucky enough to discuss the various issues with the Qt Multimedia/Mobility guys in Brisbane, Australia via a video conference. Well it was supposed to be a video conference, but despite it being between two groups of multimedia geeks, video completely failed us (various firewall issues for one solution and the lack of admin rights at the remote end for another!), meant we had to make do with a normal phone call! Such is life! Anyway, they were kind enough to give us an overview of Qt Multimedia/Mobility and let us know their approximate roadmap.

The outcome of this meeting was more or less that KDE Multimedia focus for the short term will remain with Phonon but the Qt guys will try and be more open about their plans and try and work better with the community longer term. At present, this is an uphill struggle as the VLC project has been very active in the KDE community for the last little while and seems to fit the needs in both the short and relatively long term of KDE, albeit in a way that is not currently exposed via Phonon API. This previous lack of communication is something that will be hard to overcome, but using a Qt integrated and supported solution definitely has advantages in the future - if they really mean it this time! The fact that the code is also used on Nokia mobile platforms give a fairly good reassurance that this will indeed be the case at least!

The rather obvious question of "Why not extend Phonon?" was also raised. While it was difficult to hear exactly the responses, the general reasoning seemed to be that there were some design mistakes in Phonon that basically meant that each individual feature that needed to be added, actually needed to be done at the backend level first (for each and every backend) and then exposed out through the API. With the Qt Mobility approach, more of the core features and functionality can be implemented once, with it only reaching out the the underlying platform specific system for specific and defined operations.

We enquired about the possibility of having a single Qt Multimedia backend written for Phonon, that allowed the Phonon API to continue in use at the application level while using the underlying systems/framework provided by Qt Mobility and/or Multimedia and deprecating all other Phonon backends. This is attractive in the sense that applications to not need to change horses mid stream and can gradually move over to using a pure Qt Multimedia API if indeed that is determined to be the most desirable outcome. That said it is also unattractive in some ways too. This adds yet another layer of abstraction to a system that many people argue is a layer of abstraction too far in itself! i.e. Application uses Phonon which uses Qt Multimedia which uses GStreamer; Why not just Application uses GStreamer? (there are of course many reasons to add a layer of abstraction; cross platform support being a primary one; but two layers is arguably not the best situation, especially on mobile platforms where wasted CPU cycles really hurt). That said, it would still be an acceptable stepping stone for most people. Regardless, there was no actual commitment to this eventuality from the Qt side. It was seen as a good idea but it's likely not something that there will be time (aka budget) for developing in the near term, which is understandable.

Many of the problems with Phonon still remain however. Qt Mobility will still require the setup of different underlying support libraries on different platforms. Just as before, GStreamer will be required for it to work on Linux and downstream (distributions and, to a lesser extent, users) will need to ensure that this is all configured and working correctly. As with the original Phonon, this backend will be different on different platforms, which is not ideal for the downstream.

So in the short to medium term (e.g. the next one to two years at least), Phonon will continue to be the primary media framework on KDE, and development on the Phonon-VLC backend will be seen as the best way forward to provide a standard experience across all platforms. GStreamer as a phonon backend will also continue, although the mplayer and xine backends should be considered deprecated.

With regards to Qt Multimedia and Mobility development itself, the Qt guys will try to be more open; opening a private mailing list to more access and generally being more communicative.

Those of you who regularly read this blog will be no doubt wondering where the PulseAudio connection is. Well, from what I gather, this is a primary difference of Qt Mobility and Qt Multimedia. Qt Mutlimedia seems to use ALSA directly and thus is not ideally suited for mobile situations (PulseAudio's timer based scheduling now being pretty much universally accepted as being the preferred approach on mobile platforms). Qt Mobility does actually use PulseAudio indirectly via GStreamer, but it does not seem to do anything special with regards to the "buffer-time" or "latency-time" attributes of the pulsesink (not that pulsesink is actually referred to directly in the appropriate place anyway from what I can tell). These attributes map directly to the tlength and minreq attributes of the buffer metrics in a PulseAudio stream. These are very important when it comes to mobile environments as the general aim is to provide as high a latency as possible. Higher latency means lower power usage (and for those wondering, this does not necessarily affect A/V sync on video playback - it's just about how much data you pump into the audio buffers before sleeping until you know it'll be almost empty). For a system that deals with audio playback on a mobile device, this is very important.

Now Phonon is also similarly ignorant to these properties when using PulseAudio. The the help of folk from Intel we're going to look at increasing the defaults in Phonon when used with GStreamer pulsesink to see if higher latencies would bring benefits without any drawbacks at the application level (I don't think any of the uses of Phonon will have problems with this approach), but we'll also have to consider how best to expose this via the Phonon API to allow the application level to state more explicitly if this matters. When capture APIs become possible on Phonon this will become important. The whole question of how much thought has gone into this kind of power saving is Qt Mobility is still very much at the forefront of my mind. I hope to be able to discuss things in more depth with the developers in due course, hopefully influencing them to extend the API in a similar was as described above.

More generally, those of us in the KDE community will try to get involved with Qt Multimedia/Mobility but until such times as it is easily configurable on the Desktop or there is a Phonon backend based on it, it will be hard to get involved too deeply.

Overall I think everyone has an open mind, but with the current focus on VLC and the functionality it provides, it will be the most interesting bits for most KDE Multimedia guys for the short term.

With regards to all of the above, I hope I've made fair and levelled comments. If you feel this is not the case, please tell me so in the comments below. I will happily retract or rephrase or reconsider any point if a suitable argument is made. I do not intend to represent anyone else in the KDE community in a way they do not feel is accurate so I'll be happy to both comment and edit the article with suitable annotation if this is deemed necessary.

Edits:

1. 27th May: Clarification of my incorrect statement of when/why Xine backend was introduced. Thanks to those who pointed it out.