TRACEPaw sensorized paw helps legged robots "feel the floor" with Arduino Nicla Vision

Our four-legged friends don’t walk on tarmac the same way as they do on ice or sand as they can see and feel the floor with their eyes and nerve endings and adapt accordingly. The TRACEPaw open-source project, which stands for “Terrain Recognition And Contact force Estimation through Sensorized Legged Robot Paw“, aims to bring the same capabilities to legged robots.

Autonomous Robots Lab achieves this through the Arduino Nicla Vision board leveraging its camera and microphone to run machine learning models on the STM32H7 Cortex-M7 microcontroller in order to determine the type of terrain and estimate the force exercized on the leg.

But the camera is apparently not used to look at the terrain, but instead, at the deformation of the silicone hemisphere – made of “Dragon Skin” – at the end of the leg to estimate 3D force vectors, while the microphone is used to recognize terrain types in real-time with both data streams fed to force estimation and terrain classification machine learning models respectively so that the robot can adapt its locomotion strategies. The Arduino Nicla Vision also comes with a 6-axis motion sensor, but interestingly it’s not used at all by the project.

TRACEPaw is fully open-source with the hardware, source code, and documentation made available on GitHub with the latter explaining the following:

Mechanical design & electronics with CAD files (STEP/STL), BoM, assembly instructions, and manufacturing guidelines.
Software Setup – Guide to setting up the software environment (OpenMV firmware + TensorFlow Lite) for TRACEPaw, including instructions on data collection, model training & visualization, and the deployment process.
Training and test datasets for the vision-based force estimation and audio-based terrain classification capabilities of TRACEPaw.
Pretrained models both in Keras and TensorFlow Lite format.

You should be able to reproduce the complete leg provided you have access to a 4-axis CNC machine to manufacture the aluminum parts and a 3D printer to print the mold for the silicone sole that is manufactured using a “vacuum casting process” which probably means you’d need a vacuum casting machine as well…

Jean-Luc Aufranc (CNXSoft)

Jean-Luc started CNX Software in 2010 as a part-time endeavor, before quitting his job as a software engineering manager, and starting to write daily news, and reviews full time later in 2011.