There are plenty of M.2 AI modules based on accelerators such as Hailo-8, MemryX MX3, or Axelera AI, but the Geniatech AIM M2 module is based on the Kinara Ara-2 40 TOPS AI accelerator that we’ve yet to cover here on CNX Software.
The Key-M module is designed to handle Generative AI and transformer-based models such as Stable Diffusion at a lower price point than competitors, and operates at a typical sub-2W power consumption in computer vision workloads. Target applications include AI assistants/Copilot, gaming, smart retail, physical security, and factory automation.
Geniatech AIM M2 specifications:
- AI Accelerator
- Kinara Ara-2 NPU with 40 TOPS of AI power
- Package – 17x17mm FCBBA
- System Memory – 16GB RAM (4GB/8GB Option)
- Host interface – PCIe Gen4 x4
- Security – Secure boot and encrypted memory access
- Misc – Heatsink cooling
- Supply Voltage – 3.3V
- Power Consumption – Under 2 Watts (typical)
- TDP – 12 Watts
- Dimensions – 22 x 80 mm (M.2 M-Key 2280 module)
- Temperature Range – Operating: 0 to 50°C; storage: -40 to 85°C
- Humidity – Operating: 5%-90% RH; storage: 5%-95% RH
Geniatech says the AIM M2 module, and by extension the Kinara Ara-2 AI accelerator, supports popular AI frameworks such as TensorFlow, Torchscript, PyTorch, ONNX, Caffe, and Mxnet. Supported models include Llama2.0 and Yolov8.
Drivers for Linux and Windows are available, and we’re told the module has been tested on NXP, Nvidia, Qualcomm, and AMD Xilinx targets. We can find more details on the Kinara website with some performance metrics for popular models:
- Stable Diffusion 1.4 – 7 seconds per image
- Llama-7b – 12 output tokens/s
- MobileNetV1 SSD – 974 IPS (1.02 ms latency)
- ResNet50 – 2ms latency
Kinara has its own KM-2 M.2 and KU-2 USB 3.0 modules/reference designs and recommends 2GB of RAM for conventional AI, and 8GB of RAM for generative AI.
While we are told the Kinara Ara-2 offers “unmatched cost-effectiveness”, I was unable to find pricing information for any of the modules above. More details about the Geniatech AIM-M2 can be found on the product page. They also have an AIM-B2 module with board-to-board connectors and using the same 40 TOPS Ara-2 chipset, but it’s unclear which carrier board it is designed for. [Update: the company sent a photo of the i.MX 8M Plus development kit they use with both M.2 and B2B sockets, and they also told us they would typically provide the hardware design documentation so they can develop their own carrier board for the AI module.
]
Jean-Luc started CNX Software in 2010 as a part-time endeavor, before quitting his job as a software engineering manager, and starting to write daily news, and reviews full time later in 2011.
Support CNX Software! Donate via cryptocurrencies, become a Patron on Patreon, or purchase goods on Amazon or Aliexpress
That can be interesting for laptops.
It’s cool that they’ve installed a moderately large amount of RAM onboard. These models are apparently 4266 MT/s. In a total of 64 bits this gives 34 GB/s, that’s quite decent (the same as what a rock5b can achieve) and more than 4 times higher than what PCIe4x4 can deliver. It would obviously have been even better with DDR5-8000+ or with 4 chips, but that’s a good step in the right direction if we want real accelerators that are capable of doing their job. And I agree with @Toxa above regarding the fact that it might help get AI closer to laptops.
“While we are told the Kinara Ara-2 offers “unmatched cost-effectiveness”, I was unable to find pricing information for any of the modules above.”
Brrrr. What price do we expect for the M.2 module? 50 Euro? 200 Euro?
I’m guessing 100 Euros. I’ve asked Geniatech, but they may or may not reply because their main market is B2B.