Tools · SPBoucher
Menu

exo

by Exo Labs · Application · also known as exo-explore, exo labs

Open-source software that pools several Apple Silicon Macs (and Linux hosts) into one local inference cluster, sharding large models across devices with MLX

Apache-2.0 licenseOpen sourceFreeLocalSelf-hostedSelf-hostable
Visit official site Compare

Link verified Nov 2, 2026

Overview

exo is an open-source project from Exo Labs (Apache 2.0) that connects multiple everyday devices into a single cluster for running large language models locally. Devices on the same network, or linked by Thunderbolt cables, discover each other automatically; exo builds a live view of the topology, including memory, link latency and bandwidth, and splits a model across nodes so that models too large for any one machine can still run. It supports tensor parallelism across devices and, on recent macOS versions, RDMA over Thunderbolt 5 to cut inter-node latency.

The inference backend is MLX with MLX distributed for communication, so Apple Silicon Macs are the primary target; Linux hosts currently run on CPU, with GPU support described as in development. A macOS app and a web dashboard manage the cluster and offer a chat interface, models load from the Hugging Face Hub, and the server exposes OpenAI Chat Completions, OpenAI Responses, Claude Messages and Ollama-compatible endpoints so existing clients work unchanged. Published examples run very large open-weight models across four Mac Studios.

exo is under rapid development: the repository is active daily, all nodes must run the same version, and APIs and installation steps change often. It is well suited to experimenters with several Macs who want to run models beyond a single machine's memory, less so to teams needing a stable, supported serving stack.

Editorial note on uncertainty. The project is pre-1.0 in its source tree and changes quickly; capabilities described reflect the README on 2026-11-02 and may shift. Exo Labs also lists a business offering but publishes no pricing.

Capabilities

  • Automatic device discovery over LAN or Thunderbolt
  • Topology-aware model sharding and tensor parallelism
  • MLX inference backend with MLX distributed communication
  • OpenAI-, Claude- and Ollama-compatible API endpoints
  • Web dashboard and macOS app for cluster management
  • Model loading from the Hugging Face Hub

Typical workflows

  1. Run a model larger than one Mac's memory across several Macs
  2. Point an OpenAI-compatible client at a home cluster
  3. Benchmark Thunderbolt versus Ethernet interconnects

Strengths

  • Uses hardware already owned instead of a GPU server
  • Open source with a permissive license
  • Drop-in API compatibility with common clients

Limitations

  • Apple Silicon-centric; Linux is CPU-only for now
  • Fast-moving project with frequent breaking changes
  • All nodes must run identical versions
  • No long-term support or stability guarantees

Prerequisites

  • Two or more Apple Silicon Macs on one network (macOS 15.1 or later for the app)
  • Fast interconnect for good performance

Who it is for

  • Experimenters with several Macs
  • Researchers exploring distributed local inference

Alternatives & comparison

  • llama.cpp

    C/C++ inference engine for LLMs and vision-language models with GGUF quantized weights, Metal, CUDA, Vulkan and other backends, a CLI and an OpenAI-compatible server

    Open sourceOpen source
  • MLX LM

    Python package and CLI for running, serving, quantizing and fine-tuning large language models on Apple Silicon with MLX

    Open sourceOpen source
  • Ollama

    Open-source runtime and model manager for running open-weight LLMs locally on macOS, Windows and Linux, with a REST API, OpenAI- and Anthropic-compatible endpoints and optional cloud models

    Open sourceOpen source
  • vLLM

    High-throughput LLM inference and serving engine with PagedAttention, continuous batching, quantization and an OpenAI-compatible server for GPU and accelerator clusters

    Open sourceOpen source
Compare exo with its alternatives →

Sources & verification

Primary sources consulted for this record. Claims are tied to the fields each source supports; the automated link checker re-verifies every URL on a schedule.

Confidence
Medium
Reviewed
Nov 1, 2026
Links
Verified · Nov 2, 2026

Spotted an error?

Broken link, outdated license, wrong category, or a tool that no longer exists? Tell us; reports are reviewed by the editor.

Kubernetes

Cloud Native Computing Foundation · Containers & virtualization

In-depth entry

Open-source system for scheduling, scaling and operating containerized workloads across a cluster of machines

Apache-2.0Open sourceFreeSelf-hostedNov 2, 2026

Ray

Ray project (Anyscale and contributors) · Distributed & parallel computing

In-depth entry

Open-source Python framework for distributing tasks, actors and ML workloads across a cluster, with libraries for data processing, training, tuning, serving and reinforcement learning

Apache-2.0Open sourceFreeLocalSelf-hostedBroken

GNU Parallel

GNU Project (Ole Tange) · Distributed & parallel computing

Shell tool that runs commands in parallel over input lines on one or several machines, as a drop-in alternative to xargs and shell loops

GPL-3.0-or-laterOpen sourceFreeLocalSelf-hostedNov 2, 2026

Slurm Workload Manager

SchedMD LLC · Distributed & parallel computing

Open-source cluster resource manager and job scheduler for Linux clusters, allocating nodes, launching jobs and managing queues on many HPC systems

GPL-2.0-or-laterOpen sourceFreeSelf-hostedNov 2, 2026

Apache Spark

Apache Software Foundation · Analytical engines & OLAP

Distributed data processing engine with DataFrame and SQL APIs in Python, Scala, Java and R, plus structured streaming and MLlib

Apache-2.0Open sourceFreeLocalSelf-hostedNov 2, 2026

Open MPI

Open MPI consortium · Distributed & parallel computing

Open-source implementation of the MPI message-passing standard used to run tightly coupled parallel programs across the nodes of a cluster

BSD-3-Clause-Open-MPIOpen sourceFreeLocalSelf-hostedBroken