Our first Graphics Working Group session of this connect was today. We reviewed the ongoing efforts to optimize various media libraries for AARCH64. James Yu and Ragesh Radhakrishnan talked about their work involving libpng, libjpeg-turbo, libvpx and pixman. With libjpeg-turbo it features a refresh of the android port. This allows libjpeg-turbo to be a drop in replacement for libjpeg which is currently part of AOSP. The performance difference is clear. Libjpeg-turbo contains quite a number of optimization work over the past few years while libjpeg has languished for a variety of unfortunately political reasons. The reasons? libjpeg-turbo is better than twice as fast as libjpeg.
James talked his libvpx work which of cource includes VP8 and VP9 support. He’s essentially replaced the past hand coded assembler with a version that used neon intrinsics. Comparing hand coded vs neon intrinsics on ARMv7, there’s a bit of a degrade in performance that needs to be looked into. It’s nearly 10%. In prior efforts with libpng, a similar switch yielded statistically no change in performance.
On the good news side, progress continues on porting past optimized libraries so that they are also optimized for AARCH64 in preparation for the arrival of real hardware.
The other important goal of the session was to gather input on the next set of libraries that should be optimized for AARCH64. We’ve a limited amount of time. We’ve a limited amount of hands. We won’t have everything optimized by the time real hardware shows up. We want to optimize first what is viewed as most important.
Unfortunately we didn’t get a lot of direction. There seems to be a feeling that codecs and such that are in android should be considered first priority. Tho there was some rightful dissent on that concept from those interested in traditional linux.
Besides leaning towards android the other concept that seemed to be there was we should give priority for video over audio. This makes sense as while default fallbacks for audio use more CPU, they generally won’t completely consume a CPU with loss of quality as compared to a video codec that will suffer in framework and quality.
libav was discussed as being potentially important to 3rd party application developers. There doesn’t appear to be solid usage numbers that have been identified as of yet as far as how much libav is in use by 3rd party application developers on Android. I would presume there must be some as libav is a good choice for “odd” formats.
In the afternoon we reviewed our internal list of codecs and media libs for porting and optimizing for AARCH64 and with some other attendees from connect we assembled our list for the “next” libs to put time and attention into. We’ll be reviewing this with others first but generally speaking it looks something approx like this:
- webrtc (audio portion)
- aac (from AOSP)
libav is a discussion point yet so that might very well find it’s way onto the list.