You’d typically need hardware with a large amount of memory and bandwidth and multiple GPUs, if you want to run the latest large language models (LLMs), such as DeepSeek R1 with 671 billion parameters. But such hardware is not affordable or even available to most people, and the Exo software works around that as a distributed LLM solution working on a cluster of computers with or without NVIDIA GPUs, smartphones, and/or single board computers like Raspberry Pi boards. In some ways, exo works like distcc when compiling C programs over a build farm, but targets AI workloads such as LLMs instead. Key features of Exo software: Support for LLaMA (MLX and tinygrad), Mistral, LlaVA, Qwen, and Deepseek. Dynamic Model Partitioning – The solution splits up models based on the current network topology and device resources available in order to run larger models than you would be able to on any […]
exo software – A distributed LLM solution running on a cluster of computers, smartphones, or SBCs
