Archive for January, 2012

12.01 was the first monthly cycle of the year 2012 for the Linaro Multimedia Work Group. It was a short cycle as it included both the Christmas/New Years celebrations as well as the Chinese New Year celebration which is of no surprise many take vacation to spend with their friends and family.

Even with the short cycle work was accomplished and I’d like to share some of the highlights with you.

Audio

One of our major goals for this cycle was the return of working audio. This is a fairly difficult area especially on Linux as the between the kernel, alsa and pulse audio, there’s plenty of room for regressions to occur. In past months we’ve been bit strongly by those regressions such that when it comes to audio on the Linaro LEB, I’ve taken a very inbox zero approach. It’s all broke unless proven otherwise. Let’s fix it!

So as a team we focused on audio on Panda, Panda ES and imx53. All of these boards have output via an audio jack as well as out HDMI when HDMI is used for video out. Wei Feng along with the respective ti and freescale landing teams worked hard to address the kernel drivers, ucm configs which are packaged with alsa and updates to pulse audio to support ucm.

With Panda there is an existing bug where DVI video is busted. Because of this if you are validating the audio jack you have to set amixer settings appropriately so when using speaker-test or aplay you’ll get output.

Here we are on release day and audio is working for our aplay and speaker-test tests. It’s a start but not quite a whole perfect user experience. We do see bugs with GUI tools that sit above. Are the bugs in the tools or in lower portions of the audio stack, that’s a good question. We know we have a start. A line in the sand which we will not back from but clearly more work ahead.

Kurt is working on the audio stack for panda on Android. On Jan 6th the Android kernel team delivered the needed support for audio. It’s a start, we look forward to enabling ucm as well as other audio stack elements in the days and weeks ahead.

Codecs

Mans started implementing optimizations for libav involving vp8.He’s a master coder and we look forward to his upstream submissions sometime in February but probably after LC since Linaro Connect and ELC are the first part of February.

Rony worked on the NEON optimized version of speex. It is part of the Linaro Android release now. Good job Rony! Rony has also worked to become on of our Lava experts.

I worked on libjpeg-turbo for android. I pull in some upstream updates fixing up the android specific patches. I then pulled in some existing qualcomm optimizations of 565 and 8888 encode and decode. I had to rework the patches a bit to get them to fit into libjpeg-turbo but in the end it appeared to be well worth it yielding about a 100% performance improvement in skia-bench.  There are some STE patches that accomplish the same that were recently given to me. I am going update the STE implementation so it fits with libjpeg-turbo and hold a bake off. Winner goes upstream.

UMM/Video

Rob added eglImage rendering support to qtmobility. This dramatically improved HW video decode so for the linaro-tv xmbc image and Ubuntu TV efforts on panda, it looks spectacular. Ricardo Salveti wrote about this in his blog, including demos. Be sure to check it out at : http://rsalveti.wordpress.com/2012/01/06/hw-video-decode-and-xbmc-ubuntu-linaro/

Kan worked on xbmc. Specifically he’s looking at a way to improve the way video is played on xbmc using gstreamer as well as general improvements for xbmc that make it really shine on arm hardware. We’ll have a demo at Linaro Connect in February.

Benjamin worked the the continuous memory allocator debug blueprint. This is part of the larger UMM (unified memory manager) effort that Linaro is leading. A special shout out to Benjamin for becoming and helping out others with getting testcases into Lava. Lava is our automated test system. It new to Linaro and having someone take the time to education and help others is really valuable.

Finally

I hope you’ll agree it’s a good set of accomplishments for a significantly shortened cycle. I can’t thank the team enough for their efforts. Look for more accomplishments in 12.02.

The wiki page that holds all this information but in a much more raw form can be found here: http://wiki.linaro.org/TomGall/DraftPlan/2012.01Plan

Our 12.02 plan is starting to take form here : http://wiki.linaro.org/TomGall/DraftPlan/2012.02Plan

skia_bench on android measures a number of things. One portion of the benchmark measures libjpeg performance, specifically for 565 and 8888 image types that are specific to android.

Android (including the latest ice cream sandwich release) uses the old and quite crusty libjpeg library. This library while functional is missing a great deal of optimization. The libjpeg-turbo project (http://libjpeg-turbo.org) is a souped up and more importantly drop in replacement API compatible version of libjpeg. It is compatible with both versions 6.x and 8 that are in wide use across many a distro. However because libjpeg doesn’t have simd (Single instruction, multiple data) optimizations using NEON on ARM for instance, distributions have been pitching libjpeg for libjpeg-turbo.

At Linaro our Android Ice Cream Sandwich for instance replaces libjpeg with libjpeg-turbo. Likewise we recently worked with Ubuntu and as a result Precise (version 12.04) now includes libjpeg-turbo.

So what kind of performance bump can one see by switching from libjpeg to libjpeg-turbo?

https://wiki.linaro.org/TomGall/LibJpeg8  includes lots of raw numbers on intel and ARM machines. In short performance improvements measured by tjbench  are on the order of 200% to 300%.

libjpeg-turbo doesn’t include support for Android. So earlier this year, we ported libjpeg-turbo so it would include Android dependency on the aforementioned  nonstandard 8888 and 565 formats. This was a good first step however, no work had been done to optimize for 565 or 8888. Thus the performance for 565 and 8888 was about the same for libjpeg and libjpeg-turbo.

I’d like to see Android switch to use libjpeg-turbo so this week it was time to do some optimization that would give credence to that desire. Optimizations that can be measure with skia_bench would be the way to go.

Hack. Hack. Hack.

The results? A comparison of android’s libjpeg,  libjpeg-turbo without 565 or 8888 optimizations and libjpeg-turbo with 565 and 8888 optimizations can be found at : https://wiki.linaro.org/TomGall/SkiaBenchNumbers . Smaller numbers are better.

Half the time not bad or put another way, the newly improved libjpeg-turbo is 2 times faster than the old libjpeg! I’m sure you’d like that improvement on your Android phone or tablet!  I would!

Now we get to another part of the story that is important also reflects another aspect of what makes Linaro an important leader in ARM for Linux and Android.  You see the optimization code for 565 and 8888 already sort of existed. It was sitting out in a git archive more or less gathering dust. The essential step of putting it together with the already optimized libjpeg-turbo hadn’t been done. The code also hadn’t been pushing upstream to the communities that would most benefit. In Linaro we want Linux and Android on ARM to shine.

While it’s all put together and it works, that’s great. Now comes the most important steps, getting google’s Android engineers (and perhaps cyanogen mod too!) to accept it so that all might benefit. That’s the bar for success we aim for at Linaro and we will succeed.

Code : git://git.linaro.org/people/tomgall/libjpeg-turbo/libjpeg-turbo.git

Pull from the android branch.   git checkout -b android origin/android

Have fun!