Synthetic Sensors Combine Multiple Sensors and Machine Learning for General-Purpose Sensing

Sensors can be used to get specific data for example temperature & humidity or light intensity, or you can combine an array of sensors and leverage sensor fusion to combines data from the sensors to improve accuracy of measurement or detect more complex situation.

Gierad Laput, Ph.D. student at Carnegie Mellon University, went a little further with what he (and the others he worked with) call Synthetic Sensors. Their USB powered hardware board includes several sensors, whose data can then be used after training through machine learning algorithms to detect specific events in a room, car, workshop, etc…

List of sensors in the above board (at frequency at which data is gathered):

PANASONIC GridEye AMG8833 IR thermal camera (10 Hz)
TCS34725 color to digital converter (10 Hz)
MAG3110F magnetometer (10 Hz)
BME280 temperature & humidity sensor, barometer (10 Hz)
MPU6500 accelerometer (4 kHz)
RSSI data out of 2.4 GHz WiFi module (10 Hz)
AMN21111 PIR Motion sensor (10 Hz)
ADMP401 microphone (17 kHz)
EMI data out of 100 mH inductor (0.5 MHz)

The chart below shows how it works. They first manually train the system to recognized events in the cloud based on sensor data, and after a while it basically run on auto-pilot detecting very specific events.

The best way to understand how powerful the solution is to check an example such as events occurring inside a car.

5 events can be detected:

Car start (accelerometer, magnetometer, and audio data)
Approaching highway (after which acceleration increases)
Windows Opened (temperature drops, pressure drops, humidity increases, wind noise)
Windows Closed (less noise, temperature rises, etc…)
Junction merge (deceleration + magnetometer data)

The system can also detect clouds based on color and illumination data. If you’d rather see what kind of event the system can detect in the home or office, watch the video below.

Further details can be found on Gierad’s SuperSensor page, or/and you can read the relevant research paper.

Thanks to TLS for the tip.

Jean-Luc Aufranc (CNXSoft)

Jean-Luc started CNX Software in 2010 as a part-time endeavor, before quitting his job as a software engineering manager, and starting to write daily news, and reviews full time later in 2011.