Tomeu Vizoso has been working on an open-source driver for NPU (Neural Processing Unit) found in Rockchip RK3588 SoC in the last couple of months, and the project has nicely progressed with object detection working fine at 30 fps using the SSDLite MobileDet model and just one of the three cores from the AI accelerator.
Many recent processors include AI accelerators that work with closed-source drivers, but we had already seen reverse-engineering works on the Allwinner V831’s NPU a few years ago, and earlier this year, we noted that Tomeu Vizoso released the Etvaniv open-source driver that works on Amlogic A311D’s Vivante NPU. Tomeu has now also started working on porting his Teflon TensorFlow Lite driver to the Rockchip RK3588 NPU which is closely based on NVIDIA’s NVDLA open-source IP.
He started his work in March leveraging the reverse-engineering work already done by Pierre-Hugues Husson and Jasbir Matharu and was quickly able to run TensorFLow Lite’s Conv2D and DepthwiseConv2D operations. Only two weeks later, MobileNetv1 model could run on the Pine64 QuartzPro64 SBC with the same performance level as the blob (closed-source binary).
Work was much easier than on the Verisilicon Vivante NPU because lots of the reverse-engineering work was done, and NVDLA is open-source so at least some documentation was available, which was not the case for the Vivante NPU. Nevertheless, it took only four weeks (not full-time) to have the object detection shown below work on the Rockchip RK3588’s NPU at 30 FPS.
You’ll find the source code for the Teflon project on Freedesktop website, and you can also the status of the project on Tomeu’s blog. Next up, Tomeu plans to write a kernel driver for Linux mainline in the drivers/accel subsystem. There’s still much work to be done and it’s unclear how long it will take, especially since he is working on different NPUs and will split his time between each implementation unless additional contributors join the project(s).
Jean-Luc started CNX Software in 2010 as a part-time endeavor, before quitting his job as a software engineering manager, and starting to write daily news, and reviews full time later in 2011.
Support CNX Software! Donate via cryptocurrencies, become a Patron on Patreon, or purchase goods on Amazon or Aliexpress
