RT-Thread Vision board review - Part 1: OpenMV on Renesas RA8D1 Cortex-M85 microcontroller

I am always interested in real-time operating systems (RTOS) for microcontrollers (MCUs) with my past backgrounds in µC/OS-II, mbed, and FreeRTOS. When the opportunity arose to get my hands on the RT-Thread Vision Board, thanks to the RT-Thread team and CNX Software, I was excited to check it out. This board, a collaboration between RT-Thread and Renesas, packs a powerful Renesas RA8D1 Cortex-M85 MCU and comes pre-loaded with OpenMV firmware. OpenMV’s MicroPython engine lets you jump right into embedded vision development, perfect for experimenting with computer vision tasks. But the real power lies in RT-Thread’s ability to handle tasks very quickly, which we’ll explore with C/C++ development in part two. This first part will focus on getting you familiar with the hardware using the OpenMV firmware, making it a smooth entry point for beginners. Plus, I have a collection of other Renesas evaluation boards, so you can bet I’ll be trying out RT-Thread on those as well! Stay tuned!

Renesas evaluation boards — RT-Thread Vision board and other Renesas boards in my stock.

The RT-Thread Vision board comes loaded with hardware perfect for embedded vision development. The heart of the board is a 5-megapixel camera (OV5640) for capturing high-quality images and videos. It also comes with ample storage (8MB flash and 32MB SDRAM) to hold image data. The board integrates an additional AT32F425 microcontroller (ARM Cortex-M4) that acts as a DAP-Link debugger, allowing easy programming and debugging via USB. For wireless connectivity needs, the RT-Thread Vision board includes a separate RW007 WiFi module (Realtek RTL8710BN inside) that handles communication through SPI/UART using simple AT commands. This makes it ideal for projects that require remote data transfer or control. And the best part? The board’s schematics are available on the official Github repository, making it easy for experienced developers to customize the hardware for their specific projects.

What is RT-Thread?

Since its debut in 2006, RT-Thread RTOS initially focused on delivering a lightweight kernel tailored for MCUs with limited resources. As the Internet of Things (IoT) paradigm gained traction, RT-Thread evolved into a comprehensive embedded software platform. The RT-Thread architecture now offers rich kernel, middleware, and software layers supporting file systems, networking stacks, and GUI components. Moreover, RT-Thread claims to incorporate security mechanisms spanning multiple layers (quite interesting but should take time to explore). I also found several interesting RT-Thread projects listed on the CNX Software website: $25 Renesas “HMI Board”, Art-Pi STM32H750 Cortex-M7, and $4 DshanMCU Pitaya Lite.

RT-Thread's architecture — Software architecture of RT-Thread RTOS.

RT-Thread plays nicely with a wide range of MCUs, from classic ARM cores (ARM7/9/11) to modern Cortex-M and Cortex-A processors. It’s not picky and can even work with x86, RISC-V, and MIPS32 chips! To cater to different needs, RT-Thread comes in two flavors. The kernel-only Nano version is perfect for resource-constrained MCUs (3KB of ROM and 1.2 KB of RAM claimed). For more complex projects, the full-featured standard version provides a rich set of extras like file systems, networking tools, and even popular software packages. RT-Thread project provides an SDK called RT-Thread Studio, letting you choose hardware, configure the kernel, pick the tools you need, and integrate pre-built software packages. It then uses the GCC toolchain to build your firmware. But if you prefer a different environment, RT-Thread can also export projects for use with popular IDEs like Keil MDK-ARM or IAR.

Unlike MCU-oriented RTOSes like FreeRTOS, RT-Thread goes a step further. It offers a rich set of pre-built features like file systems and networking stacks, saving time and effort when building complex projects. This makes it a strong alternative to bulky operating systems like Linux, which can be overkill for resource-constrained MCUs. We can access FinSH – a terminal utility – on the RT-Thread Vision board by connecting the USB-DBG port to a computer, and then using a serial terminal program like TeraTerm (115200, 8, n, 1). We can type commands, such as list thread to check active threads.

CLI with FinSH — FinSH as a console terminal.

Another interesting feature is the dynamic module that can import binary modules (.mo programs, .so libraries) at runtime. This makes your software more flexible and adaptable. Imagine creating small programs and loading them onto the board whenever you need them, just like apps on your smartphone! You can check an innovative project on Hackster.io that demonstrates compiling Arduino-like code into binaries, storing them on a file system, and executing them as applications within RT-Thread. Despite its comprehensive structure, an academic study has revealed that RT-Thread’s real-time scheduling capabilities are on par with other leading RTOSes like FreeRTOS and uC-OS/II. This makes RT-Thread a great choice for projects where timing is crucial.

The RT-Thread project sells several evaluation boards on their website through AliExpress with prices ranging from a budget-friendly $29 to $49 for the more powerful option (prices checked on June 1, 2024).

RT-Thread Vision board has two options: a $29 camera-only board and a $39 camera-and-LCD board.
ART-Pi board with a STM32H750 Cortex-M7 microcontroller ($49).
The “RT-Thread×Renesas×LVGL HMI board” uses the Renesas RA6M3 microcontroller and works with the LVGL graphics library for embedded GUI. Price of a kit with the board and a 4.3-inch display is $36.

These affordable boards suggest that RT-Thread project positions these evaluation boards to encourage developers to explore and adopt their RTOS platform.

RT-Thread webstore — RT-Thread evaluation boards on AliExpress.

Renesas RA8D1 microcontroller

While ARM Cortex-M architecture is still around, core options are expanding beyond single-digit series like M0~M7. With dual-digit series like M33 and M55 popping up, the architecture changes reflect the growing needs of IoT devices, which require stronger security and the ability to handle artificial intelligence (AI) tasks. Renesas enters the high-performance MCU arena with the RA8 series. This powerhouse boasts a 32-bit ARM Cortex-M85 core with the Helium MVE (M-Profile Vector Extension) for digital signal processing (DSP) and machine learning (ML) applications. This translates to faster processing and the ability to tackle more complex tasks for cutting-edge IoT projects. The RA8 series challenges other top-performance offerings such as ST Microelectronics’ STM32H7, Infineon’s PSOC Edge E84, and Nordic Semiconductor’s nRF7002.

RA8D1 block diagram — Hardware architecture of RA8D1 microcontroller.

I am very impressed with the Renesas RA8D1 powering the RT-Thread Vision board. The Renesas RA8D1 MCU powering the RT-Thread Vision board includes an LCD controller and a 2D graphics drawing engine on top of the CEU camera interface. These features, combined with the muscle for DSP and ML features, make the RA8D1 a great choice for embedded vision and edge computing projects. The best part? The RT-Thread Vision board lets you try for just $29, a fraction of the cost of the EK-RA8D1 evaluation board ($332). Sure, it might lack the Ethernet port of the official board, but that’s a small trade-off for such an affordable entry point.

There are de-facto standard connectors for prototyping, making it easy to swap components and build projects quickly. Familiar options include Grove from SeeedStudio, Pmod from Digilent, and the multi-pin group connectors found on Arduino shields and Raspberry Pi HATs. The RT-Thread Vision board opts for a versatile Raspberry Pi HAT-compatible connector that includes commonly used interfaces like UART, SPI, I2C, and PWM. The beauty lies in its compatibility – you can leverage existing Raspberry Pi HATs with the RT-Thread Vision board. Plus, the color-coded connector (red for 5V, yellow for 3.3V, etc.) simplifies those initial connections, saving you time and frustration as you bring your project to life.

Getting started with OpenMV on the RT-Thread Vision Board

I’m particularly impressed by the inclusion of OpenMV as a pre-loaded software package on the RT-Thread Vision board. OpenMV is a popular choice for embedded vision projects, especially among students and hobbyists in mobile robotics. With OpenMV handling image processing tasks like line following or object detection, the RT-Thread Vision board becomes a powerful core for building a mobile robot brain. This, combined with the compact size and affordable price tag of the RT-Thread Vision board, makes it a compelling platform for building budget-friendly mobile robots.

OpenMV was created to fill a gap in the market for embedded vision boards that were both programmable and easy to use. It accomplishes this by leveraging MicroPython, a simplified version of Python designed for MCUs. OpenMV’s ease of use and powerful image processing features make it a popular choice for building robots. OpenMV’s ease of use extends beyond the coding language itself. The OpenMV IDE provides a user-friendly environment for developing your vision projects. By connecting with the USB-OTG port, the IDE functions as a code editor, allowing you to write and edit your MicroPython scripts. But that’s not all! The IDE also offers a built-in image preview feature, letting you see what your camera sees in real-time. This makes it easy to visualize and fine-tune your image processing algorithms. Additionally, the IDE provides tools to check image characteristics like color histograms and frame statistics, further aiding in the development process.

OpenMV IDE with RT-Thread Vision board — Running “hello, world” in RT-Thread Vision board with OpenMV IDE.

OpenMV firmware is built as a library in a MicroPython environment. We can write Python (subset) code to capture images from the camera using the sensor module and do image processing using the image module. A simplified code structure is similar to Arduino code, i.e. a setup section to get everything ready and a main loop that runs continuously. The following sample code demonstrates how to detect objects using find_blobs() method. Anyone who wants to explore more OpenMV features on RT-Thread Vision board can check this YouTube link demonstrating several vision-based detection capabilities.

import sensor
import time

sensor.reset()                               # Reset and initialize the sensor.
sensor.set_pixformat(sensor.RGB565)          # Set pixel format to RGB565 (or GRAYSCALE)
sensor.set_framesize(sensor.QVGA)            # Set frame size to QVGA (320x240)
sensor.skip_frames(time=2000)                # Wait for settings take effect.
clock = time.clock()                         # Create a clock object to track the FPS.
thresholds = [(0, 100, 56, 95, 41, 74),      # generic_red_thresholds
              (0, 100, -128, -22, -128, 99), # generic_green_thresholds
              (0, 100, -128, 98, -128, -16)] # generic_blue_thresholds

while True:
    clock.tick()                             # Update the FPS clock.
    img = sensor.snapshot()                  # Take a picture and return the image.
    Bs = img.find_blobs(thresholds)          # find objects with color thresholds
    for b in Bs:                             # loop drawing over detected blobs
        img.draw_rectangle(b[0], b[1], b[2], b[3])
    print(clock.fps())                       # print FPS performance

import sensor

import time

sensor.reset() # Reset and initialize the sensor.

sensor.set_pixformat(sensor.RGB565) # Set pixel format to RGB565 (or GRAYSCALE)

sensor.set_framesize(sensor.QVGA) # Set frame size to QVGA (320x240)

sensor.skip_frames(time=2000) # Wait for settings take effect.

clock = time.clock() # Create a clock object to track the FPS.

thresholds = [(0, 100, 56, 95, 41, 74), # generic_red_thresholds

(0, 100, -128, -22, -128, 99), # generic_green_thresholds

(0, 100, -128, 98, -128, -16)] # generic_blue_thresholds

while True:

clock.tick() # Update the FPS clock.

img = sensor.snapshot() # Take a picture and return the image.

Bs = img.find_blobs(thresholds) # find objects with color thresholds

for b in Bs: # loop drawing over detected blobs

img.draw_rectangle(b[0], b[1], b[2], b[3])

print(clock.fps()) # print FPS performance

OpenMV's find_blobs() — Running OpenMV’s find_blobs() in RT-Thread Vision board.

Benchmarking RT-Thread Vision Board

The RT-Thread Vision board boasts the latest Renesas RA8D1 with a cutting-edge Arm Cortex-M85 core, which claims to be the highest performing ARM Cortex-M processor. Since Part 1 of the review focuses on rapid prototyping, let’s put this claim to the test at the application level. I have a WeAct STM32H743 board with a similar clock speed (480 MHz) and OpenMV installed. This setup allows us to compare the performance of single-digit vs. dual-digit ARM Cortex-M cores. In this benchmark, we’ll take a simplified approach, focusing on frames per second (FPS) while running the same code with a live camera preview. Both boards use their own cameras for image capture – the RT-Thread Vision board has an OV5640 sensor, while the WeAct STM32H743 uses an OV2640. This will give us a real-world comparison of how these two core architectures handle image processing tasks.

RT-Thread VIsion board vs WeAct STM32H743

Connecting the OpenMV IDE to the boards revealed a slight difference in how they’re recognized by the software. The RT-Thread Vision board appears as OpenMV Cam M4 version 4.5.0, indicating that the firmware was probably built with ARM Cortex-M4 settings. While the WeAct STM32F743 board is seen as OpenMV Cam M7 version 4.4.1. This might influence the benchmark results as the RT-Thread Vision board’s firmware might be limited compared to the WeAct board. We’ll keep this in mind while interpreting the data. To assess the real-world performance, we’ll benchmark three key vision tasks:

Raw image capture to test the performance of the camera interface.
Blob detection to test image processing ability.
Face detection to test the computing of data flowing along the Haar Cascade pipeline.

Face detection task — Benchmark with OpenMV’s Haar Cascade face detection.

OpenMV benchmark results in FPS (higher is better).

Note that the RT-Thread Vision board is at a bit of a disadvantage in this benchmark because the firmware was built with an older architecture. Surprisingly, the RT-Thread Vision board performed better than the WeAct board in both blob detection and face detection tasks. This suggests that the faster processing speed of the Arm Cortex-M85 core compensates for the potential limitations observed in the raw image capture test (18.0 FPS < 26.5 FPS). Our results align with the CoreMark scores, where the RA8D1 with the M85 core achieves a higher score (6.38 CoreMark/MHz) than the STM32H743 with the M7 core (5.05 CoreMark/MHz). Imagine the potential performance if the OpenMV firmware were fully optimized to utilize the capabilities of the M85 core! We could see a more significant performance difference in favor of the RT-Thread Vision board for these image processing tasks.

What’s next?

The RT-Thread Vision board has picked our interest with its powerful MCU and pre-loaded OpenMV software. We explored its potential for prototyping and the performance gap between ARM Cortex-M cores. But the journey continues! Next, we’ll dive into the world of RTOS features using the RT-Thread Studio. We’ll dissect the specific features offered by RT-Thread: how they handle software packages beyond just vision tasks.

Our aim is SQLite, a popular database management system. Including databases in MCU applications unlocks several benefits: efficient data storage and retrieval, improved data integrity, and scalability for handling growing data volumes. We’ll explore how RT-Thread facilitates the use of SQLite and other software components, giving us a deeper understanding of the RT-Thread Vision board’s inner workings and how it manages hardware-software interaction for smooth execution.