AI in a box offline LLM solution leverages Rockchip RK3588S’ NPU (Crowdfunding)
Useful Sensors “AI in a box” LLM (large language model) solution works offline with complete privacy and leverages the NPU in Rockchip RK3588S processor for conversational AI similar to ChatGPT but without an internet connection or registration required. The AI box prototype currently relies on off-the-shelf hardware, specifically the Radxa ROCK 5A SBC with 8GB RAM, housed in a plastic enclosure, and the code relies on open-source models like Whisper speech-to-text model and Llama2 language models. Besides conversational AI where you can interact with the box as if you were talking to a person, the AI in a Box can also be useful for other use cases: Live Captions – The box can display subtitles/closed captions for a live event or help in situations where people have trouble hearing a conversation using the audio input. Live Translation – It can also translate various languages close to real-time. Simply select […]