Trying out Edge Impulse machine learning platform on XIAO BLE Sense board

I had seen the Edge Impulse development platform for machine learning on edge devices being used by several boards, but I hadn’t had an opportunity to try it out so far. So when Seeed Studio asked me whether I’d be interested to test the nRF52840-powered XIAO BLE Sense board, I thought it might be a good idea to review it with Edge Impulse as I had seen a motion/gesture recognition demo on the board.

It was quite a challenge as it took me four months to complete the review from the time Seeed Studio first contacted me, mostly due to poor communications from DHL causing the first boards to go to customs’ heaven, then wasting time with some of the worse instructions I had seen in a long time (now fixed), and other reviews getting in the way. But I finally managed to get it working (sort of), so let’s have a look.

XIAO BLE (Sense) and OLED display unboxing

Since the gesture recognition demo used an OLED display, I also asked for it and I received the XIAO BLE board (without sensor), the XIAO BLE Sense board, and the Grove OLED Display 0.66″.

Both boards are extremely tiny, and exactly the same except the XIAO BLE lacks the LSM6DS3TR onboard 6-axis IMU (bottom left side).

Some soldering…

Before loading firmware on the board, I had to solder the display to the board. I just cut the Grove cable and soldered the black and red wires to the power, and the white and yellow wires to I2C.

I do not have a 3D printer (another customs story, but I digress), so instead I used several layers of dual-sided sticky tape to attach the two boards together, but that’s really optional.

Arduino sketches for XIAO BLE Sense’s OLED display and accelerometer

It took me a while to find the instructions for the gesture recognition demo, as those were not listed in the video description, and I could not find anything about it in Seeed Studio wiki. Eventually I was given the link to the instructions, and the company modified the website to make it easier to find.

Before playing with Edge Impulse, we’ll run two Arduino sketches to check the OLED display and the accelerometer work as expected.

The first step is to add the board manager URL for Seeed Studio boards: https://files.seeedstudio.com/arduino/package_seeeduino_boards_index.json

Now we can install the package to support  Seeed nRF52 Boards…

… and once it’s done, let’s connect the board with a USB-C cable to our computer, and select “Seeed XIAO BLE Sense – nRF52840” board with default settings.

Let’s try a “Hello World” program to make sure our board works and the connection to the OLED display is fine.

But I did not work as expected, and I got an error at compile time:

The answer is on Adafruit website where we learn two important details:

• nRF52 requires Arduino 1.8.15 or greater, so you may have to upgrade to the latest version
• Linux requires the installation of adafruit-nrfutil

Since I’m using Ubuntu 20.04, I had to run:

Note this is NOT needed if you are using the Arduino IDE in Windows or MacOS. The utility was installed in $HOME/.local/bin, so you’ll need to add it to your path, and restart the Arduino IDE. This can be done temporarily in the command line:

Or you can change /etc/environment or ~/.bashrc file for permanently adding the folder to your PATH. The sketch could build fine, and the binary was flashed to the board without issue.

But nothing showed on the display. I added a serial.println debug message in the main loop to check it was indeed running, double-checked the connections with a multimeter, and I could not find any obvious solution. Seeed Studio told me to downgrade to version 1.0.0 of the Seeed nRF52 Boards package.

And it indeed did the trick!

Note that you won’t need to downgrade to version 1.0.0 anymore in the rest of the review, and it is recommended you use 2.6.1 and greater. The new “Hello World” sample looks like this:

Time to switch to testing the accelerometer demo. First, we’ll need to install LSM6DS3 Arduino library from Seeed Studio.

Note there’s also an official Arduino_LSM6DS3 which you may need to uninstall to avoid conflicts. Here’s the code:

Note that when I used Seeed nRF52 Boards v1.0.0 with that sample, I would get “decode error” messages, which went away with version 2.6.1. We’ll need to open the Serial monitor to check X, Y, Z values are showing up.

Important: we’ll still need the accelerator demo running for the next step. So if you play with other samples first, make sure the accelerator demo is running before switching to Edge Impulse.

Edge Impulse on XIAO BLE Sense

Now that we know our hardware is working as expected, let’s go to Edge Impulse Studio to get started. Let’s register, and build our first project.

We’ll select Accelerometer data.

We’ll also need to install Edge Impulse CLI in Linux (Ubuntu 20.04 for this review). This first requires installing NodeJS 14.x:

The default directory will be in /usr accessible by root, let’s change that to a user’s directory which we’ll also add to our path:

Now we can install Edge Impulse CLI:

You may have to exit the terminal and restart it for the new PATH to apply.  Now we can launch the edge-impulse-data-forwarder used for boards that are not officially supported by Edge Impulse (like XIAO BLE Sense):

The very first time the command is running, you’ll need to enter your username and password (not shown above). The utility scans serial devices, connects to edge impulse, tries to detect data from the serial port, and once done, asks us to name data fields (Ax, Ay, Az), the device (XIAO BLE SENSE), and will automatically attach it to the project we have just created. If there is more than one project in Edge Impulse, you’ll be asked to select the project first. That means it’s basically hardware agnostic, and as long as your board output accelerometer data to the serial interface it should work.

Let’s go back to Edge Impulse, click on Data acquisition, and we’ll see our device together with sensor parameters and data frequency settings.

Let’s set the sample size to 20,000 ms, define a label, click on Start Sampling, and move the board up and down with about 1 second internal for 20 seconds to acquire the data.

We’ll then need to split the data by clicking on the three dots in the raw section and choosing “Split Sample”. Click “+Add Segment” to add more section. We should repeat it until we’ve got around 20 segments representing the up and down movement.  If you moved slower or faster than 1s adjust the time in “Set segment length (ms.)”.

I’m using Firefox, and I had a weird bug where I could add a segment, but when I selected it to move it, it would jump by an offset to the right, sometimes outside of the screen. But if I kept pressing on the mouse button, and moving it left, it could bring it back in view. It was not exactly the most convenient, and I can’t zoom in too much or the box will get too much out of the display. Using Chrome or Microsoft Edge with Edge Impulse might be better.

Once we click on Split and we’ll see the 1-second data sample we’ve selected.

We can repeat data acquisition and splitting for other gestures such as left and right, clockwise circle, and anti-clockwise circle. But I’d recommend keeping it simple first, as we’ll see below.

At this point, you may see what looks like a problem with the data with the warning:

One or more of the labels in your dataset have a poor train/test split

To fix this, you can either capture test data for a shorter period (e.g. 2 seconds), or rebalance the dataset, by clicking on “Dashboard” in the left menu, and scroll down to find the “Perform train / test split” button.

We are now ready to create an impulse. Click on Create impulse -> Add a processing block -> Choose Spectral Analysis -> Add a learning block -> Choose Classification (Keras) -> Save Impulse

Click on Spectral features in the Spectral Analysis, then Save parameters and Generate features.

You’d expect the data to be cleanly separated, but there’s clearly some overlap, so the trained data is not ideal. I’ll try to go ahead nevertheless.

Click NN Classifier, then “Start training” which will take about one minute to process. We can then choose Unoptimized (float32)

There’s indeed low accuracy and the model is basically unusable with only the “anti-clockwise” sample being detected properly.

Let’s try again, but only with up and down, left and right, and circle (clockwise) and trying to keep each motion within one second.

The Features explorer chart looks much better with Blue, Orange, and Green dots in their own areas. It’s also possible to delete some samples that may cause issues. After training the results are not perfect, but we may still give it a try.

Let’s go through “Model testing” in the left menu.

That’s disappointing as only Circle clockwise works well, while “left and right” can be detected about half the time, and “up and down” being wrongly detected as left and right. So besides training the data, the operator must also be trained.

The important part is the method to create an Impulse, and over time we should e able to create better data. Let’s build an Arduino library by clicking on Deployment in the left menu, then Arduino Library,  Build, and finally download the .ZIP file.

Back to the Arduino IDE. Download the Arduino sample provided by Seeed Studio, which has gone through many revisions over the last few months.  Change the edge impulse header file (line 24 in the sample below) to match your own, and I also had to comment out the line with U8X8lib.h library. I also slightly modified the code, as I had not trained for “idle” as in their demo:

We need to add the ZIP library we’ve just downloaded from Edge Impulse to the Arduino Library, and we can now build and flash the code to the board. This will take around 5 minutes the first time, and about 2 minutes for subsequent builds.

Here’s the output from the serial console.

So each time a value is over 50 it will show the corresponding text (e.g. Circle), and if none of the results goes above 50, the program simply displays “Idle”.

Here’s what it looks like on video.

The circle is recognized (even if I go in the wrong way as in the video below), but I only got “left and right” a few times, and never “up and down”. So it should take some time to have a proper demo with the important part being the data acquisition and accurate splitting to make sure all samples for a particular gesture look about the same.

I’d like to thank Seeed Studio for sending XIAO BLE (Sense) boards and Grove OLED display for testing Edge Impulse. I’d just wish they got their documentation right the first time. If you are interested in reproducing the demo above the XIAO BLE Sense board is sold for $15.99, while the OLED display goes for $5.50, and it’s optional because we can see the results in the serial terminal.

Support CNX Software! Donate via cryptocurrencies, become a Patron on Patreon, or purchase goods on Amazon or Aliexpress