edge ai

Posted on by Jean-Luc Aufranc (CNXSoft) - 19 Comments on DeepSeek shown to run on Rockchip RK3588 with AI acceleration at about 15 tokens/s

DeepSeek shown to run on Rockchip RK3588 with AI acceleration at about 15 tokens/s

Rockchip RK3588 DeepSeek R1 NPU acceleration

DeepSeek R1 model was released a few weeks ago and Brian Roemmele claimed to run it locally on a Raspberry Pi at 200 tokens per second promising to release a Raspberry Pi image “as soon as all tests are complete”. He further explains the Raspberry Pi 5 had a few HATs including a Hailo AI accelerator, but that’s about all the information we have so far, and I assume he used the distilled model with 1.5 billion parameters. Jeff Geerling did his own tests with DeepSeek-R1 (Qwen 14B), but that was only on the CPU at 1.4 token/s,  and he later installed an AMD W7700 graphics card on it for better performance. Other people made TinyZero models based on DeepSeekR1 optimized for Raspberry Pi, but that’s specific to countdown and multiplication tasks and still runs on the CPU only. So I was happy to finally see Radxa release instructions to […]

Posted on by Jean-Luc Aufranc (CNXSoft) - 2 Comments on YOLO-Jevois leverages YOLO-World to enable open-vocabulary object detection at runtime, no dataset or training needed

YOLO-Jevois leverages YOLO-World to enable open-vocabulary object detection at runtime, no dataset or training needed

YOLO-Jevois general object detection by typing words

YOLO is one of the most popular edge AI computer vision models that detects multiple objects and works out of the box for the objects for which it has been trained on. But adding another object would typically involve a lot of work as you’d need to collect a dataset, manually annotate the objects you want to detect, train the network, and then possibly quantize it for edge deployment on an AI accelerator. This is basically true for all computer vision models, and we’ve already seen Edge Impulse facilitate the annotation process using GPT-4o and NVIDIA TAO to train TinyML models for microcontrollers. However, researchers at jevois.org have managed to do something even more impressive with YOLO-Jevois “open-vocabulary object detection”, based on Tencent AI Lab’s YOLO-World, to add new objects in YOLO at runtime by simply typing words or selecting part of the image. It also updates class definitions on […]

Posted on by Debashis Das - 2 Comments on Phison’s aiDAPTIV+ AI solution leverages SSDs to expand GPU memory for LLM training

Phison’s aiDAPTIV+ AI solution leverages SSDs to expand GPU memory for LLM training

Phison's aiDAPTIVCache family support 70B model

While looking for new and interesting products I found ADLINK’s DLAP Supreme series, a series of Edge AI devices built around the NVIDIA Jetson AGX Orin platform. But that was not the interesting part, what got my attention was it has support for something called the aiDAPTIV+ technology which made us curious. Upon looking we found that the aiDAPTIV+ AI solution is a hybrid (software and hardware) solution that uses readily available low-cost NAND flash storage to enhance the capabilities of GPUs to streamline and scale large-language model (LLM) training for small and medium-sized businesses. This design allows organizations to train their data models on standard, off-the-shelf hardware, overcoming limitations with more complex models like Llama-2 7B. The solution supports up to 70B model parameters with low latency and high-endurance storage (100 DWPD) using SLC NAND. It is designed to easily integrate with existing AI applications without requiring hardware changes, […]

Posted on by Debashis Das - No Comments on SECO’s SMARC-QCS5430 SMARC SoM and devkit feature Qualcomm QCS5430 SoC for Edge AI and 5G applications

SECO’s SMARC-QCS5430 SMARC SoM and devkit feature Qualcomm QCS5430 SoC for Edge AI and 5G applications

SOM SMARC QCS5430 SoM

SECO has announced early engineering samples for its SOM-SMARC-QCS5430 system-on-module (SoM) and devkit designed to support IoT and edge computing applications. Built around the Qualcomm QCS5430 processor this SMARC-compliant SoM targets industrial automation, robotics, smart cities, and surveillance.

The module also offers dual MIPI-CSI interfaces for camera and connectivity options including USB 3.1, PCIe Gen3, dual GbE, and optional Wi-Fi and Bluetooth. SECO’s DEV-KIT-SMARC industrial devkit includes all the necessary components for rapid prototyping and integration.

Posted on by Debashis Das - No Comments on Vecow ECX-4000 – Intel Core Ultra 200S-powered fanless Edge AI embedded system features up to 9 Ethernet ports

Vecow ECX-4000 – Intel Core Ultra 200S-powered fanless Edge AI embedded system features up to 9 Ethernet ports

Vecow ECX 4000 Intel 200S powred fanless Edge AI system

Taiwan-based company Vecow has recently launched the ECX-4000 series, an Intel Core Ultra 200S-powered fanless Edge AI embedded system with up to nine Ethernet ports including two 10G SFP+ cages, five 2.5GbE ports (with 4 supporting PoE+), and a gigabit Ethernet jack, SUMIT (Stackable Unified Module Interconnect Technology) expansion, and a 9V to 50V DC redundant power input. The ECX-4000 supports the whole line of Intel Core Ultra 200S Series of Processors (Arrow Lake). It comes with W880 PCH which gives access to various I/O options including USB 3.2 Gen 2 ports, RS-232/422/485 serial ports, sixteen isolated digital I/O (8x input, 8x output, optional), DisplayPort, HDMI, and DVI-I video outputs. Additionally, it offers M.2 Key B and Key E sockets for wireless modules, and expansion options including multiple storage interfaces such as SATA III ports and an M.2 Key-M socket. The system also features a range of power and remote […]

Posted on by Jean-Luc Aufranc (CNXSoft) - 2 Comments on M5Stack LLM630 Compute Kit features Axera AX630C Edge AI SoC for on-device LLM and computer vision processing

M5Stack LLM630 Compute Kit features Axera AX630C Edge AI SoC for on-device LLM and computer vision processing

M5Stack LLM630 Compute Kit

M5Stack LLM630 Compute Kit is an Edge AI development platform powered by Axera Tech AX630C AI SoC with a 3.2 TOPS NPU designed to run computer vision (CV) and large language model (LLM) tasks at the edge, in other words, on the device itself without access to the cloud. The LLM630 Compute Kit is also equipped with 4GB LPDDR4 and 32GB eMMC flash and supports both wired and wireless connectivity thanks to a JL2101-N040C Gigabit Ethernet chip and an ESP32-C6 module for 2.4GHz WiFi 6 connectivity. You can also connect a display and a camera through MIPI DSI and CSI connectors. M5Stack LLM630 Compute Kit specifications: SoC – Axera Tech (Aixin in China) AX630C CPU – Dual-core Arm Cortex-A53 @ 1.2 GHz; 32KB I-Cache, 32KB D-Cache, 256KB L2 Cache NPU – 12.8 TOPS @ INT4 (max), 3.2 TOPS @ INT8 ISP – 4K @ 30fps Video – Encoding: 4K; Decoding:1080p […]

Posted on by Debashis Das - No Comments on Intel Core Ultra 200S/200U/200H-powered COM-HPC Client modules support up to 192GB DDR5 memory, PCIe Gen5

Intel Core Ultra 200S/200U/200H-powered COM-HPC Client modules support up to 192GB DDR5 memory, PCIe Gen5

Portwell COM HPC client modules PCOM B887 and PCOM B886

Portwell PCOM-B887 and PCOM-B886 are two new COM-HPC client modules built around Intel Core Ultra 200S/200U/200H processors delivering high-performance computing and AI acceleration for industrial, edge, and AI-driven applications. The Portwell PCOM-B887 (Size C) module is built around the 200S Series, offers up to 36 TOPS, supports 192GB DDR5 memory, and features 42 PCIe lanes up to Gen5. The PCOM-B886 (Size B) module supports 200H/200U Series processors, which deliver up to 99 TOPS, support 96GB DDR5 memory, and include 24 PCIe lanes. Both modules feature various I/O options, including USB4, USB3.2 Gen2, and multiple display outputs. Portwell PCOM-B887 – COM-HPC Client Type Size C module The PCOM-B887 COM-HPC Client Type Size C module is powered by Intel Core Ultra 200S Series processors, which feature up to 36 TOPS of AI performance via an integrated neural processing unit (NPU). It supports up to 192GB of DDR5 memory at 4800MT/s with 42 […]

Posted on by Debashis Das - 1 Comment on Seeed Studio introduces ESP32-C3-based Modbus Vision RS485 and SenseCAP A1102 LoRaWAN outdoor Edge AI cameras

Seeed Studio introduces ESP32-C3-based Modbus Vision RS485 and SenseCAP A1102 LoRaWAN outdoor Edge AI cameras

Seeed Studio's Modbus Vision RS485 and SenseCAP A1102 outdoor Edge AI cameras

Seeed Studio has recently released the Modbus Vision RS485 and SenseCAP A1102 (LoRaWAN) outdoor Edge AI cameras based on ESP32-C3 SoC through the XIAO-ESP32C3 module for WiFi and the Himax WiseEye2 processor for vision AI. Both are IP66-rated AI vision cameras designed for home and industrial applications. The RS485 camera is designed for industrial systems and features a Modbus interface, making it suitable for factory automation and smart buildings. The SenseCAP A1102 uses LoRaWAN for long-range, low-power monitoring in remote locations. Both offer advanced AI for tasks like object detection and facial recognition. Besides its built-in RS485 interface, the Modbus Vision RS485 also supports LoRaWAN and 4G LTE connectivity via external Data Transfer Units (DTUs). With over 300 pre-trained AI models, the camera can do object detection and classification tasks making it suitable for industrial automation, smart agriculture, environmental monitoring, and other AI-driven applications requiring high performance. The SenseCAP A1102 […]