Web Engines Hackfest 2014

During the last days I attended the Web Engines Hackfest 2014 in A Coruña, which was kindly hosted by Igalia in their office. This gave me some time to work again on WebKit, and especially improve and clean up its GStreamer media backend, and even more important of course an opportunity to meet again all the great people working on it.

Apart from various smaller and bigger cleanups, and getting 12 patches merged (thanks to Philippe Normand and Gustavo Noronha reviewing everything immediately), I was working on improving the WebAudio AudioDestination implementation, which allows websites to programmatically generate audio via Javascript and output it. This should be in a much better state now.

But the biggest chunk of work was my attempt for a cleaner and actually working reimplementation of the Media Source Extensions. The Media Source Extensions basically allow a website to provide a container or elementary stream for e.g. the video tag via Javascript, and can for example be used to implement custom streaming protocols. This is still far from finished, but at least it already works on YouTube with their Javascript DASH player and should give a good starting point for finishing the implementation. I hope to have some more time for continuing this work, but any help would be welcome.

Next to the Media Source Extensions, I also spent some time on reviewing a few GStreamer related patches, e.g. the WebAudio AudioSourceProvider implementation by Philippe, or his patch to use GStreamer’s OpenGL library directly in WebKit instead of reimplementing many pieces of it.

I also took a closer look at Servo, Mozilla’s research browser engine written in Rust. It looks like a very promising and well designed project (both, Servo and Rust actually!). I’m sure I’ll play around with Rust in the near future, and will also try to make some time available to work a bit on Servo. Thanks also to Martin Robinson for answering all my questions about Servo and Rust.

And then there were of course lots of discussions about everything!

Good things are going to happen in the future, and WebKit still seems to be a very active project with enthusiastic people 🙂
I hope I’ll be able to visit next year’s Web Engines Hackfest again, it was a lot of fun.

GStreamer Playback API

Update: the code is now also available on GitHub which probably makes it easier for a few people to use it and contribute. Just send pull requests or create issues in the issue tracker of GitHub.

Over the last years I noticed that I was copying too much code to create simple GStreamer based playback applications. After talking to other people at GUADEC this year it was clear that this wasn’t only a problem on my side but a general problem. So here it is, a convenience API for creating GStreamer based playback applications: GstPlayer.

The API is really simple but is still missing many features:

GstPlayer *  gst_player_new       (void);

void         gst_player_play      (GstPlayer * player);
void         gst_player_pause     (GstPlayer * player);
void         gst_player_stop      (GstPlayer * player);
void         gst_player_seek      (GstPlayer * player, GstClockTime position);
void         gst_player_set_uri   (GstPlayer * player, const gchar * uri);
...

Additionally to that there are a few other properties (which are not only exposed as setters/getters but also as GObject properties), and signals to be notified about position changes, errors, end-of-stream and other useful information. You can find the complete API documentation here. In general the API is modeled similar to other existing APIs like Android’s MediaPlayer and iOS’ AVPlayer.

Included are also some very, very simple commandline, GTK+, Android (including nice Java bindings) and iOS apps. An APK for the Android app can also be found here. It provides no way to start a player, but whenever there is a video or audio file to be played it will be proposed as a possible application via the Android intent system.

In the end the goal is to have a replacement for most of the GStreamer code in e.g. GNOME‘s Totem video player, Enlightenment‘s Emotion or really any other playback application, and then have it integrated in a gst-plugins-base library (or a separate module with other convenience APIs).

While this is all clearly only the start, I hope that people already take a look at this and consider using it for their projects, provide patches, or help making the included sample apps really useful and nice-looking. Apps for other platforms (e.g. a Qt app, or one written in other languages like C# or Python) would also be nice to have. And if you’re an Android or iOS or Qt developer and have no idea about GStreamer you can still help by creating an awesome user interface 🙂 Ideally I would like to get the Android and iOS app into such a good shape that we can upload them to the app stores as useful GStreamer playback applications, which we could then also use to point people to a good demo.

If you’re interested and have some time to work on it or try it, please get in contact with me.

FOMS Workshop 2013

This week and next week I’m going to be in San Francisco and will attend the Foundations of Open Media Standards and Software (FOMS) workshop 2013. Thanks to Silvia Pfeiffer, the other organisers and the sponsors of the event (Google and brightcove) for making this possible by sponsoring me.

Topics that are going to be discussed are WebRTC, Web multimedia in general, open codecs and related topics and the list of attendees seems very diverse to me. I expect lots of interesting discussions and a few interesting and hopefully productive days 🙂

If anybody else is in that area currently and wants to meet for a coffee, beer or some food on the 2-3 days before and after FOMS, please write me a mail or use some other communication channel 🙂

GStreamer 1.2.0 released and other updates

This blog post might be a bit late but nonetheless, here are some updates that happened in GStreamer in the last week 🙂

GStreamer 1.2.0 released

On Tuesday we released GStreamer 1.2.0. This was just in time for GNOME 3.10, which depends on it and was released on Wednesday (Congrats btw! Many great improvements).

The changes between GStreamer 1.0 and 1.2 can be seen from the release announcement mail. In summary there are many, many nice new features, API improvements and even more bugfixes that we did not include in the 1.0.x bugfix releases because they required larger changes.

Following the 1.2.0 release we plan to do regular 1.2.x bugfix releases with important bugfixes, while at the same time develop new features and in general more complicated changes in the already opened 1.3 development release series (which is the GIT master branch now). The 1.3 release series will later lead to the next stable 1.4 release series. Needless to say that 1.4 will be completely backwards-compatible with 1.2 and 1.0.

GStreamer 1.2.0 binaries

Two days later on Thursday we then released the binaries for GStreamer 1.2.0 for Android (ARM), Mac OS X and Windows. This will allow to take advantage of all the new changes on these platforms too and provide a better cross-platform multimedia experience.

We also finally moved cerbero, the build system used to generate the binaries for all these platforms, to GStreamer’s freedesktop.org GIT and got rid of all the ancient 0.10 cruft that was still in there. This GIT repository is now the official place from where you can get the latest version of cerbero for building GStreamer, and will also be updated for all future 1.2.x bugfix releases. Hopefully it will also be kept up-to-date for GIT master, i.e. the 1.3 development release series.

iOS binaries and build support

Over the last days I also worked on porting the remaining parts of GStreamer 1.0 to iOS (5.1 and newer versions), and also adapted the cerbero build system to automatically build GStreamer 1.0 for iOS and create an installer package from it. iOS binaries were already available in the past from a third-party website (gstreamer.com) but only for the old and deprecated 0.10 version. My changes are based on this.

It’s now also possible to build GStreamer and all its dependencies via cerbero with the Xcode 5.0 toolchain for OS X and iOS. Thanks to more strict compiler checks and behaviour changes, some changes in the build systems and the code were necessary.

Official binaries for iOS will be provided by the GStreamer project in the next days.

Others

gstreamer-vaapi was already ported to the new 1.2 API for better hardware integration and the next release should contain it. Finally it can do all that without using unstable GStreamer API.

Also gnonlin, gst-editing-services and gst-python finally saw a 1.x based (pre-) release. The final 1.2.0 releases should be there in the next days, and hopefully a new PiTiVi release will follow soonish.

GStreamer conference 2013

In October the GStreamer Conference 2013 will happen in Edinburgh. There will be many interesting talks by the GStreamer developers but also other people and companies, who will give talks about what they achieved with GStreamer.

I’ll give a talk about how to integrate hardware capabilities into GStreamer, and which new APIs were added to make this much easier or possible at all (depending on what kind of integration is needed).