2 tools
Exo Labs · Distributed & parallel computing
Open-source software that pools several Apple Silicon Macs (and Linux hosts) into one local inference cluster, sharding large models across devices with MLX
vLLM project (originated at UC Berkeley Sky Computing Lab) · Local LLM inference & serving
High-throughput LLM inference and serving engine with PagedAttention, continuous batching, quantization and an OpenAI-compatible server for GPU and accelerator clusters
Page 1 of 1
Share this view: /tools?category=distributed-computing&use=run-local-models