Open-source H.265/HEVC Hantro G2 decoder driver added to NXP i.MX 8M in Linux

Getting open-source multimedia drivers on Arm Linux is one of the most difficult tasks, that’s why there’s no much talk about open-source GPU drivers for 2D & 3D graphics acceleration, but work on video hardware decoding and encoding is also a challenge.

We’ve previously seen Bootlin work on Cedrus open-source driver for Allwinner VPU (Video Processing Unit), but Collabora has been working on open-source drivers for VeriSilicon’s Hantro G1 and G2 VPU found in some Rockchip, NXP, and Microchip processors.

The company previously managed to have Hantro G1 open-source driver for JPEG, MPEG-2, VP8, and H.264 codecs, but H.265/HEVC relies on Hantro G2, and the patch for H.265 hardware video decoding on NXP i.MX 8M Quad has just been submitted to mainline Linux.

Benjamin Gaignard explains more in his commit message:

The IMX8MQ got two VPUs but until now only G1 has been enabled This series aim to add the second VPU (aka G2) and provide basic HEVC decoding support.

To be able to decode HEVC it is needed to add/update some of the structures in the uapi. In addition of them one HANTRO dedicated control is required to inform the driver of the numbre of bits to skip at  the beginning of the slice header. The hardware require to allocate few auxiliary buffers to store the references frame, scaling list or tile size data.

The driver has been tested with fluster test suite stream.
For example with this command: ./fluster.py run -ts JCT-VC-HEVC_V1 -d GStreamer-H.265-V4L2SL-Gst1.0

The test application relies on the Gstreamer framework and a V4L2 Hantro driver, and you can check out the details by running the command:

For reference, v4l2-compliance is part of v4l-utils package, and in Debian/Ubuntu can be installed with apt:

The “m” option is used to select the /dev/media? node, e.g. /dev/media1.

Benjamin further explains that Hantro G1/G2 are stateless accelerators meaning they do no need firmware to operate, which makes those better suited for open-source implementation in a blog post on Collabora. Specifically, the driver is divided into two parts with a kernel driver (Video4Linux2 Hantro driver), and a userspace component such as GStreamer and FFMPEG.

The open-source Hantro driver is still in the staging directory reserved for WiP code, so further work will be needed to make it a stable API and have it used by userland programs by default. Going forward, Collabora will also work on VP9 decoding and V4L2 hardware-accelerated encoding.

Jean-Luc Aufranc (CNXSoft)

Jean-Luc started CNX Software in 2010 as a part-time endeavor, before quitting his job as a software engineering manager, and starting to write daily news, and reviews full time later in 2011.

Support CNX Software! Donate via cryptocurrencies, become a Patron on Patreon, or purchase goods on Amazon or Aliexpress