Tools · SPBoucher
Menu

MLX LM

In-depth entry

by Apple Inc. · Library · also known as mlx-lm, mlx_lm

Python package and CLI for running, serving, quantizing and fine-tuning large language models on Apple Silicon with MLX

MIT licenseOpen sourceFreeLocalSelf-hostable
Visit official site Compare

Link verified Nov 2, 2026

Overview

MLX LM is the LLM toolkit of Apple's MLX project: a Python package, published on PyPI as mlx-lm under the MIT license, that loads transformer language models from the Hugging Face Hub and runs them on Apple Silicon. It began as part of the mlx-examples repository and now lives in its own repository maintained by the MLX contributors.

The package exposes a small set of command-line entry points and a matching Python API. mlx_lm.generate and mlx_lm.chat produce text or hold an interactive session; mlx_lm.convert downloads a Hub checkpoint, converts it to MLX format and optionally quantizes it (4-bit is the common choice) and uploads the result; mlx_lm.server starts a local HTTP server with an OpenAI-style chat completions endpoint; mlx_lm.lora performs LoRA or full fine-tuning, including on quantized models. In Python, load, generate and stream_generate cover most scripting needs, with hooks for custom samplers and logits processors. Prompt caching, a rotating KV cache for long contexts and batch generation are built in, and mx.distributed lets inference or fine-tuning span several Macs.

MLX LM is the usual way a Mac owner runs a quantized Llama, Qwen, Mistral or Gemma model locally without a GUI, and it underpins many of the pre-converted models in the mlx-community Hub organization. Typical users are developers building local assistants, researchers fine-tuning small models, and operators serving models on Mac Studios.

Limits: it only runs on Apple Silicon, only architectures that have been ported are supported (new families need code contributions), very large models are bounded by unified memory, and the server is meant for local or trusted-network use rather than as a hardened production gateway.

Capabilities

  • Text generation and interactive chat from the command line or Python
  • Conversion and quantization of Hugging Face checkpoints to MLX format
  • Local OpenAI-compatible HTTP server
  • LoRA and full fine-tuning, including on quantized models
  • Prompt caching and rotating KV cache for long contexts
  • Distributed inference and fine-tuning across several Macs

Typical workflows

  1. Download, quantize and run a chat model on a MacBook
  2. Serve a local model to tools that speak the OpenAI API
  3. Fine-tune a small model on domain data with LoRA
  4. Publish a converted model to the mlx-community Hub organization

Strengths

  • Minimal setup: pip install and one command
  • Direct access to thousands of Hub models
  • MIT license and active development alongside MLX
  • Python API that fits into existing scripts

Limitations

  • Apple Silicon only
  • Only ported model architectures are supported
  • Model size is bounded by the Mac's unified memory
  • Server is not designed as a production gateway with auth or multi-tenancy

Prerequisites

  • Mac with Apple Silicon
  • Python 3.11 or newer
  • Hugging Face Hub access for model downloads

Who it is for

  • Developers running local LLMs on Macs
  • Researchers fine-tuning small models
  • Operators serving models on Apple Silicon workstations

Alternatives & comparison

Ollama and LM Studio wrap llama.cpp-style engines in a packaged runtime or desktop app and run on macOS, Linux and Windows; MLX LM is a Python library tied to MLX and Apple Silicon, which makes it more scriptable and gives it fine-tuning, at the cost of portability. llama.cpp is the C/C++ engine with GGUF quantization that those tools use; MLX LM uses MLX's own quantized format. vLLM targets high-throughput GPU serving on Linux and has no comparable role on a Mac.

  • exo

    Open-source software that pools several Apple Silicon Macs (and Linux hosts) into one local inference cluster, sharding large models across devices with MLX

    Open sourceOpen source
  • Foundation Models (Apple)

    Apple's Swift framework giving apps access to the on-device Apple Intelligence language model with guided generation, tool calling and sessions, on OS 26 and later

    ProprietaryFree
  • llama.cpp

    C/C++ inference engine for LLMs and vision-language models with GGUF quantized weights, Metal, CUDA, Vulkan and other backends, a CLI and an OpenAI-compatible server

    Open sourceOpen source
  • LM Studio

    Proprietary desktop app for discovering, downloading and running local LLMs with llama.cpp and MLX engines, a chat UI, an OpenAI-compatible server, CLI, SDKs and a headless daemon

    ProprietaryFreemium
  • Ollama

    Open-source runtime and model manager for running open-weight LLMs locally on macOS, Windows and Linux, with a REST API, OpenAI- and Anthropic-compatible endpoints and optional cloud models

    Open sourceOpen source
  • Transformers

    Hugging Face's model-definition library for text, vision, audio, video and multimodal models, with Pipeline, Trainer and generate APIs used as the reference by training and inference engines

    Open sourceOpen source
  • Unsloth

    Open-source framework for fine-tuning and running LLMs with reduced memory, covering LoRA, QLoRA, full fine-tuning and RL, GGUF export, Dynamic quantizations and a local Studio/Desktop UI

    Open sourceOpen source
  • vLLM

    High-throughput LLM inference and serving engine with PagedAttention, continuous batching, quantization and an OpenAI-compatible server for GPU and accelerator clusters

    Open sourceOpen source
Compare MLX LM with its alternatives →

Sources & verification

Primary sources consulted for this record. Claims are tied to the fields each source supports; the automated link checker re-verifies every URL on a schedule.

Confidence
High
Reviewed
Nov 1, 2026
Links
Verified · Nov 2, 2026

Spotted an error?

Broken link, outdated license, wrong category, or a tool that no longer exists? Tell us; reports are reviewed by the editor.

MLX

Apple Inc. · Apple Silicon ML & MLX

In-depth entry

Apple's open-source array framework for machine learning on Apple Silicon, with NumPy-like Python, C++ and Swift APIs and unified CPU/GPU memory

MITOpen sourceFreeLocalNov 2, 2026

Core ML

Apple Inc. · Apple Silicon ML & MLX

Apple's on-device machine-learning runtime for iOS and macOS apps, with the open-source coremltools package for converting PyTorch and other models to its format

BSD-3-ClauseFreeLocalNov 2, 2026

Metal

Apple Inc. · Apple Silicon ML & MLX

Apple's low-level GPU programming framework for graphics and parallel compute on iOS, macOS and visionOS, the layer beneath MLX, Core ML and most Apple Silicon ML runtimes

ProprietaryFreeLocalNov 2, 2026

Create ML

Apple Inc. · Apple Silicon ML & MLX

Apple's app and Swift framework for training image, text, sound, tabular and activity models on a Mac without writing training code, exporting directly to Core ML

ProprietaryFreeLocalNov 2, 2026

exo

Exo Labs · Distributed & parallel computing

Open-source software that pools several Apple Silicon Macs (and Linux hosts) into one local inference cluster, sharding large models across devices with MLX

Apache-2.0Open sourceFreeLocalSelf-hostedNov 2, 2026

Foundation Models (Apple)

Apple Inc. · Apple Silicon ML & MLX

Apple's Swift framework giving apps access to the on-device Apple Intelligence language model with guided generation, tool calling and sessions, on OS 26 and later

ProprietaryFreeLocalNov 2, 2026