You dream of running the biggest AI models on your computer, but like me, you do not have the budget to rent a datacenter & mldr; It’s stupid anyway.

So if I told you that there is a solution to transform your existing machines into a Distributed AI clusterthanks to Exoa free project that allows you to connect all your devices to pool their computing power.

With exo, you will be able to cluster your MacBook, your iPhone, your Android tablet and even your Raspberry Pi to distribute the load and thus run the most strong open source models as strong as Llama,, Mistral Or Qwen.

Exo screenshot

The principle is simple & mldr; Exo analyzes the memory and the resources available on each connected device, then intelligently cut the AI ​​model into pieces which will be distributed throughout the network. The more devices you add, the more you increase the total capacity of your cluster.

To start with Exo, simply install Python 3.12+ then execute these commands:

git clone https://github.com/exo-explore/exo.git
cd exo
source install.sh

Once installed, then launch the command exo on each device. The system will then automatically detect the other nodes on the network, like that, no need for twisted configuration! Everything works in P2P.

A chatgpt -style web interface is then accessible on http: // localhost: 52415.

To operate, Exo uses a partitioning strategy called Ring Memory Weighted Partitioning where each device performs a number of layers of the model proportional to its available memory. You can thus integrate Exo into your existing applications thanks to its API compatible with that of Chatgpt.

Many models are already supported like Llama (MLX and Tinygrad versions), Mistral, Llava, Qwen or Deepseek.

If you want to see more, A tutorial is available here. Me on my side, I do not have enough machines available to run exo in a comfortable way but a youtuber did the test with 5 mac studio so I share your video for you:

https://www.youtube.com/watch?v=JU0NDY2KWLW


Thanks to Lorenper and Letsar for this excellent discovery!

Discover Exo on Github


Source link

Categorized in:

Technology