Vision FPGA SoM Integrates Audio, Vision and Motion-Sensing with Lattice iCE40 FPGA (Crowdfunding)

tinyVision.ai’s Vision FPGA SoM is a tiny Lattice iCE40 powered FPGA module with integrated vision, audio, and motion-sensing capability with a CMOS image sensor, an I2S MEMS microphone and a 6-axis accelerometer & gyroscope.

The module enables low power vision (10-20 mW) for battery-powered applications, can interface via SPI  to a host processor as a storage device, comes with open-source toolchain and sample code, and is optimized for volume production.

Vision FPGA SoM specifications:

• FPGA – Lattice iCE40UP5k FPGA with 5K LUT’s, 1 Mb RAM, 8 MAC units
• Memory – 64 Mbit QSPI SRAM for temporary data
• Storage – 4 Mbit QSPI Flash for FPGA bitstream/code storage
• Sensors
  • Himax HM01B0 CMOS image sensor
  • Knowles MEMS I2S microphone, expandable to a stereo configuration with an off-board I2S microphone
  • InvenSense IMU 60289 6-axis Gyro/accelerometer
• I/Os
  • 4x GPIOs with programmable IO voltage
  • SPI host interface with programmable IO voltage
• Misc – Tri-color LED, IR LED for low-light illumination
• Power Supply
  • Single 3.3 V operation that can supply 1.8 V and 1.2 V at 100 mA (max) to external devices using onboard LDO’s
  • External 3.3 V, 1.8 V, 1.2 V for lower power operation
• Dimensions – 3x 2 cm

The module support Neural Networks thanks to the Lattice SensAI toolchain using Tensorflow, Caffe, or Keras for model development, quantization, and mapping to the SensAI Neural Network engines. The Vision FPGA SOM can handle Inference at over 30 FPS at around 25 mW on average. Sample FPGA code for vision-based people detection and audio keyword detection will be provided together with documents and schematics on the project’s Github repo.

To get started, the company also provides two carrier boards with the first one dubbed “SOM Developer board” giving access to all IO to headers, micro USB power and FPGA programming, and standard Pmod and Qwiic connectors for expansion, as well as a smaller “SoM Breakout Board” with some IO accessible via 2.54mm pitch through holes along the sides. Note the latter needs an external SPI flash programmer to program a bitstream to the FPGA.

Vision FPGA SOM and development kits have launched on Crowd Supply with an $8,000 funding target. A $70 pledge should get you the module, but most backers will likely select the $80 reward with SoM and breakout board, or the $120 reward with the module and developer board. Shipping is free to the US but adds $5 to $30 to the rest of the world depending on the selected perk. Backers should expect their rewards to ship in December 2020 if everything goes according to plans.

Support CNX Software! Donate via cryptocurrencies, become a Patron on Patreon, or purchase goods on Amazon or Aliexpress