Analysis · Articles

Clustering two machines for bigger models: worth it?

Two boxes can hold a model neither could alone. But the tools are early, and the network is now part of your AI.

Published Oct 27, 2026 Updated Oct 27, 2026 6 min read

We earn from qualifying purchases via affiliate links. Disclosure.

Thumbnail: “Two Boxes, One Model” with two Minisforum MS-S1 MAX maker photos joined by a 10 GbE link — Illustration by FitMyModel with maker product photos (local review copy).

Key takeaways

llama.cpp’s RPC backend can spread a model across machines over TCP.
Its own docs call it a fragile, insecure proof of concept.
Minisforum says two MS-S1 MAX units ran a 235B model at Q4. That’s the maker’s claim; we haven’t checked it.

Can’t afford one machine with 256 GB of memory? The idea is tempting: link two and split the model between them. It works, but you should know what you’re signing up for.

How it works

llama.cpp’s RPC backend talks to ggml-rpc-server instances on other hosts and offloads work to them over TCP. Each host can contribute its GPU, Metal device or CPU. Meta says its 816B Muse Spark Open can run on two M5 Ultra Mac Studios using MLX.

Illustration: two machines joined by a 10 GbE cable, each holding half of one model — Illustration: llama.cpp RPC splits one model across machines over the network.

The warning label

“This example and the RPC backend are currently in a proof-of-concept development stage. As such, the functionality is fragile and insecure.”

From llama.cpp RPC backend README

The same README says never to run the RPC server on an open network. Treat a cluster as an experiment on a private, wired network.

What the network needs

The link between the boxes now carries model data, so faster is better. The MS-S1 MAX’s store page lists dual 10GbE and USB4 V2 at 80 Gbps. For a two-box setup, a direct cable is enough. For more boxes, a 10 GbE switch such as the MikroTik CRS304 connects them. We haven’t measured how much a slow link costs you.

MikroTik CRS304-4XG-IN fanless switch with four 10 GbE ports — MikroTik CRS304: four 10 GbE ports for three or more boxes. Maker product photo (local review copy).

Minisforum MS-S1 MAX mini workstation — The MS-S1 MAX lists dual 10GbE and USB4 V2. Maker product photo (local review copy).

Worth it?

Worth it if you already own one 128 GB box and want to try models bigger than 128 GB. Not worth it as your first plan. One bigger machine is simpler, and today’s tools are labeled proof of concept. See our Clustering and NAS guide.