generative ai News - Page 2 of 3 - CNX Software

May 29, 2024May 29, 2024 by Jean-Luc Aufranc (CNXSoft) - 2 Comments

picoLLM is a cross-platform, on-device LLM inference engine

Large Language Models (LLMs) can run locally on mini PCs or single board computers like the Raspberry Pi 5 but with limited performance due to high memory usage and bandwidth requirements. That’s why Picovoice has developed the picoLLM Inference Engine cross-platform SDK optimized for running compressed large language models on systems running Linux (x86_64), macOS (arm64, x86_64), and Windows (x86_64), Raspberry Pi OS on Pi 5 and 4, Android and iOS mobile operating systems, as well as web browsers such as Chrome, Safari, Edge, and Firefox. Alireza Kenarsari, Picovoice CEO, told CNX Software that “picoLLM is a joint effort of Picovoice deep learning researchers who developed the X-bit quantization algorithm and engineers who built the cross-platform LLM inference engine to bring any LLM to any device and control back to enterprises”. The company says picoLLM delivers better accuracy than GPTQ when using Llama-3.8B MMLU (Massive Multitask Language Understanding) as a […]

May 23, 2024May 23, 2024 by Jean-Luc Aufranc (CNXSoft) - 6 Comments

EdgeCortix SAKURA-II Edge AI accelerator deliver up to 60 TOPS in an 8W power envelope

SAKURA-II M.2 and PCIe Edge AI accelerators

EdgeCortix has just announced its SAKURA-II Edge AI accelerator with its second-generation Dynamic Neural Accelerator (DNA) architecture delivering up to 60 TOPS (INT8) in an 8Watts power envelope and suitable to run complex generative AI tasks such as Large Language Models (LLMs), Large Vision Models (LVMs), and multi-modal transformer-based applications at the edge. Besides the AI accelerator itself, the company designed a range of M.2 modules and PCIe cards with one or two SAKURA-II chips delivering up to 120 TOPS with INT8, 60 TFLOPS with BF16 to enable generative AI in legacy hardware with a spare M.2 2280 socket or PCIe x8/x16 slot. SAKURA-II Edge AI accelerator SAKURA-II key specifications: Neural Processing Engine – DNA-II second-generation Dynamic Neural Accelerator (DNA) architecture Performance 60 TOPS (INT8) 30 TFLOPS (BF16) DRAM – Dual 64-bit LPDDR4x (8GB,16GB, or 32GB on board) DRAM Bandwidth – 68 GB/sec On-chip SRAM – 20MB Compute Efficiency – […]

May 22, 2024 by Jean-Luc Aufranc (CNXSoft) - 27 Comments

Snapdragon Dev Kit for Windows features Qualcomm Snapdragon X Elite Arm SoC for AI PC application development

Qualcomm Snapdragon Dev Kit for Windows is a mini PC-looking development platform based on the Snapdragon X Elite 12-core Arm processor with up to 75 AI TOPS of performance designed to help developers natively port apps to the Elite X SoC and develop new AI applications besides the Copilot+ AI PC features developed internally by Microsoft. Although it’s slightly bigger, the external design looks similar to the Windows Dev Kit 2023 with a Qualcomm Snapdragon 8cx Gen 3 compute platform, but internally, the new devkit features the much more powerful 4.3 GHz X Elite 12-core 64-bit Armv8 Oryon processor coupled with 32GB LPDDR5x RAM and 512GB NVMe SSD, and offering a range of ports and features such as USB4 and WiFi 7. Snapdragon Dev Kit for Windows (2024) specifications: SoC – Snapdragon X Elite (X1E-00-1DE) CPU – 12-core 64-bit Armv8 Oryon processor clocked at up to 3.8 GHz, or 4.3 […]

May 10, 2024May 10, 2024 by Jean-Luc Aufranc (CNXSoft) - No Comments

XGO-Rider is a 2-wheel self-balancing robot with an ESP32 controller plus either a Raspberry Pi CM4 or BBC Micro:bit (Crowdfunding)

XGO-Rider is a two-wheel self-balancing robot with an ESP32 controller for motor and servo control, USB-C charging, etc… and a choice between a Raspberry Pi CM4 module or a BBC Micro:bit board for display, audio, and camera (CM4-only). It’s not the first robot from Luwu Intelligence, since the company launched the XGO-Mini robot dog in 2021, followed by the XGO 2 Raspberry Pi CM4-powered desktop robotic dog with an arm which we reviewed last year. The new XGO-Rider builds on these earlier models but in a different form factor moving from four-legged robots to a 2-wheel self-balancing robot design with many of the same features including AI vision running on the Raspberry Pi CM4. XGO-Rider specifications: Host controller (one or the other) Raspberry Pi CM4 with 2GB RAM + ESP32 for main control, USB-C charging port, DIP switch BBC Micro:bit V2 + ESP32 for main control, USB-C charging port, DIP […]

April 26, 2024May 30, 2024 by Jean-Luc Aufranc (CNXSoft) - 1 Comment

Firefly AIBOX-1684X compact AI Box delivers 32 TOPS for large language models, image generation, video analytics, and more

Firefly AIBOX-1684X is a compact AI Box based on SOPHON BM1684X octa-core Arm Cortex-53 processor with a 32 TOPS AI accelerator suitable for large language models (LLM) such as Llama 2, Stable Diffusion image generation solution, and traditional CNN and RNN neural network architectures. Firefly had already released several designs based on the SOPHON BM1684X AI processor with the full-featured Firefly EC-A1684XJD4 FD Edge AI computer and the AIO-1684XQ motherboard, but the AIBOX-1684X AI Box offers the same level of performance, just without as many interfaces, in a compact enclosure measuring just 90.6 x 84.4 x 48.5 mm. AIBOX-1684X AI box specifications: SoC – SOPHGO SOPHON BM1684X CPU – Octa-core Arm Cortex-A53 processor @ up to 2.3 GHz TPU – Up to 32TOPS (INT8), 16 TFLOPS (FP16/BF16), 2 TFLOPS (FP32) VPU Up to 32-channel H.265/H.264 1080p25 video decoding Up to 32-channel 1080p25 HD video processing (decoding + AI analysis) Up […]

April 4, 2024 by Jean-Luc Aufranc (CNXSoft) - 6 Comments

Hailo-10 M.2 Key-M module brings Generative AI to the edge with up to 40 TOPS of performance

Hailo-10 M.2 module generative AI for the edge

Hailo-10 is a new M.2 Key-M module that brings Generative AI capabilities to the edge with up to 40 TOPS of performance at low power. It targets AI PCs supporting only the Windows 11 operating system on x86 or Aarch64 targets at this time. Hailo claims the Hailo-10 is faster and more energy efficient than integrated neural processing unit (NPU) solutions found in Intel SoCs and delivers at least twice the performance at half the power of Intel’s Core Ultra “AI Boost” NPU. Hailo-10 module specifications: AI accelerator – Hailo-10H System Memory – 8GB LPDDR4 on module Host interface – 4-lane PCIe Gen 3 Power consumption – Less than 3.5W (typical) for the chip Form factor – M.2 Key M 2242 / 2280 Supported AI frameworks – TensorFlow, TensorFlow Lite, Keras, PyTorch & ONNX The Hailo-10 can run Llama2-7B with up to 10 tokens per second (TPS) at under 5W […]

January 9, 2024January 10, 2024 by Tomisin Olujinmi - 8 Comments

The EQSP32 is a no-code, no-solder Industrial Internet of Things Controller powered by a generative AI assistant (Crowdfunding)

The EQSP32 controller is a complete, end-to-end solution for IoT applications that recently launched on Kickstarter. It is a compact, wireless Industrial IoT controller based on the ESP32-S3 wireless SoC with a 250MHz dual-core processor, 512KB of RAM, and 8MB of flash memory. The product leverages artificial intelligence and code for automation projects can be generated automatically by the bundled generative AI programming assistant. The EQSP32 controller features 16 terminals that can be configured as analog or digital inputs, or as digital outputs. Switches, pushbuttons, keypads, LED strips, sensors, servos, potentiometers, etc., can be connected to these terminals. It is similar to the EdgeBox-Edge-100 we covered a while back but lacks an Ethernet port and uses less power overall. EQSP32 specifications: SoC: Espressif Systems ESP32-S3 dual-core Tensilica LX7 microcontroller @ 240 MHz, 512KB RAM Memory – 8MB flash Network Connectivity: Bluetooth, WiFi USB – USB-C programming port I/O: 16 multipurpose […]

January 9, 2024January 8, 2024 by Jean-Luc Aufranc (CNXSoft) - 5 Comments

Ambarella N1 SoC brings Generative AI to the edge for video analytics, robotics, industrial applications

Ambarella has been working on adding support for generative AI and multi-modal large language models (LLMs) to its AI Edge processors including the new 50W N1 SoC series with server-grade performance and the 5W CV72S for mid-range applications at a fraction of the power-per-inference of leading GPU solutions. Last year, Generative AI was mostly offered as a cloud solution, but we’ve also seen LLM running on single board computers thanks to open-source projects such as Llama2 and Whispter, and analysts such as Alexander Harrowell, Principal Analyst, at Omdia expect that “virtually every edge application will get enhanced by generative AI in the next 18 months”. The Generative AI and LLM solutions running on Ambarella AI Edge processors will be used for video analytics, robotics, and various industrial applications. Compared to GPUs and other AI accelerators, Ambarella provides AI Edge SoC solutions that are up to 3x more power-efficient per generated […]