Multiple Videos Encoding and Decoding in Android on Firefly-RK3288 Development Board

Some applications like digital signage and video surveillance may require simultaneous video playback, and encoding to send over the network. ARM platforms may not be powerful enough to achieve those tasks with the processor only, but luckily many ARM SoCs include powerful video processing unit (VPU) capable of doing both hardware video encoding and decoding, leaving the CPU cores mostly free for other tasks.

Firefly team has demoed such capabilities on their Firefly-RK3288 development board by displaying 7 videos on a single screen, including two previews from two USB cameras attached to the board, which are then encoded and decoded on the fly and displayed on the screen. The last three videos are played from 3 different files. So that means that 5 video decode and 2 video encode are running simultaneously.  The board can achieve 26 to 30fps with 720p videos, and around 13 to 15 fps with 1080p videos in that demo.

The demo was published in September 2015, but I just found out about it via one Rockchip tweet, which claimed “Multiple video coding&decoding case of Firefly-RK3288 is open source for maker now!”. While the demo firmware with MultiCodecs app is available from Firefly website, I did not find any source code, so I asked on twitter… To my surprise I got an answer with a link to download .zip   (passwordqtsr), with source code for MultiCodecs app (with some close-source binary libraries), and to control the UVC cameras.

Google translation of the readme file:

System: Android 4.4

Implementation:
In SDK libraries required to compile and execute the file:
1, will be copied to the folder uvc to frameworks/av/ directory
2, In the folder frameworks/av/uvc, run “mm -B” to compile

Add Android system services:
1. Edit device/rockchip/rksdk/init.rc
2 added services:

service uvcd/system/bin/uvcserver class main user root group system audio camera inet net_bt net_bt_admin net_bw_acct drmrpc mediadrm ioprio rt 4

1 2 3 4 5

service uvcd/system/bin/uvcserver class main user root group system audio camera inet net_bt net_bt_admin net_bw_acct drmrpc mediadrm ioprio rt 4

Replace libRkOMX_Resourcemanager.so:
Android 4.4 can patch: 0001-Android-vpu-omx-support-5-decodes-and-2-encodes.patch
Other versions can be replaced manually own respective positions.

Then you can compile SDK, package generated img file to burn on the board.
Taken to ensure that the following files exist:
out/target/product/rk3288_box/system/bin/uvcserver
out/target/product/rk3288_box/system/lib/libmp4v2.so
out/target/product/rk3288_box/system/lib/libuvc-bunchen-jni.so

Nextinstall MultiCodecs.apk:
1, path: MultiCodecs/bin/MultiCodecs.apk
2, through adb install apk installed or copied to the board can be installed

Then you have to prepare three mp4 format video, named video1.mp4, video2.mp4, video3.mp4, and copied to the board /mnt/ sdcard/ below.
Uvc then plug in two cameras, run MultiCodecs, will be able to see the effects..

So you could try by yourself using Android 4.4 for the board, adjust the app to your needs, or integrate the solution into your own app.

Jean-Luc Aufranc (CNXSoft)

Jean-Luc started CNX Software in 2010 as a part-time endeavor, before quitting his job as a software engineering manager, and starting to write daily news, and reviews full time later in 2011.

Support CNX Software! Donate via cryptocurrencies, become a Patron on Patreon, or purchase goods on Amazon or Aliexpress