Ollama

Name: Ollama
Author: Ollama

In-depth entry

by Ollama · Application

Open-source runtime and model manager for running open-weight LLMs locally on macOS, Windows and Linux, with a REST API, OpenAI- and Anthropic-compatible endpoints and optional cloud models

MIT licenseOpen sourceFreeUsage-basedPaidEnterpriseLocalSelf-hostedCloudSelf-hostable

Visit official site Compare

Link verified Nov 2, 2026

Overview

Ollama is an MIT-licensed runtime for running open-weight language models on your own machine. A single install on macOS, Windows or Linux (or the official Docker image) provides a background server, a CLI (ollama run, ollama pull, ollama launch) and, on desktop platforms, a chat app. Models are fetched from the Ollama library by name and stored as layered blobs; a Modelfile lets you derive variants with a different system prompt, parameters or adapter. Inference is built on llama.cpp's GGML engine, with Metal acceleration on Apple Silicon and CUDA or ROCm on discrete GPUs.

The server listens on port 11434 and exposes Ollama's native REST API plus OpenAI- and Anthropic-compatible endpoints, which is why many editors, agents and frameworks treat Ollama as a drop-in local backend. Official Python and JavaScript SDKs cover chat, generation, embeddings and tool calling, and ollama launch starts integrations such as Claude Code or OpenClaw against a local model. Since 2025 the company also sells cloud-hosted models through the same CLI and API, on pay-as-you-go, Pro, Max, Team and Enterprise plans, while local inference remains free and unmetered.

In a research or small-team setting Ollama is the quickest way to serve an embedding or chat model on a Mac Studio for private workloads. Its limits are those of a single-node consumer runtime: throughput and concurrency are far below vLLM on server GPUs, model choice is restricted to the GGUF conversions in the library or ones you import, and quantized models trade some quality for memory. Behavior on headless macOS machines under launchd also needs care, since the desktop app and the CLI server are separate processes.

Capabilities

One-command model download and execution from a curated library
Native REST API plus OpenAI- and Anthropic-compatible endpoints on port 11434
Modelfile for custom system prompts, parameters and adapters
Official Python and JavaScript SDKs with tool calling and embeddings
Metal, CUDA and ROCm acceleration; CPU fallback
Desktop app on macOS and Windows; Docker image for servers
Optional cloud models billed per token through the same interface
`ollama launch` to connect coding agents to local models

Typical workflows

Serve a quantized chat model on a Mac for a private assistant
Generate embeddings locally for a vector index
Point an OpenAI-compatible client at localhost to test without API costs
Mix local and cloud models from one CLI

Strengths

Simple installation and model management
Broad client ecosystem through OpenAI-compatible API
MIT license, local use free and unmetered
Works well on Apple Silicon unified memory

Limitations

Single-node, modest concurrency compared with server engines
Only GGUF-packaged models unless imported manually
Cloud models are proprietary hosted services with separate terms
Separate desktop app and CLI server processes can confuse headless setups

Prerequisites

macOS, Windows or Linux with enough RAM or VRAM for the chosen model
Optional: NVIDIA or AMD GPU drivers

Who it is for

Developers prototyping LLM features offline
Researchers running private inference on workstations
Home-lab and small-team operators

Alternatives & comparison

llama.cpp is the underlying engine; using it directly gives more control and the newest model support at the cost of manual builds and model files. LM Studio offers a richer GUI and MLX support on Apple Silicon but is proprietary. vLLM targets high-throughput GPU serving with continuous batching and is not aimed at laptops. mlx-lm is Apple's own path for Apple Silicon and avoids GGUF conversion but has a smaller client ecosystem.

exo
Open-source software that pools several Apple Silicon Macs (and Linux hosts) into one local inference cluster, sharding large models across devices with MLX
Open sourceOpen source
Foundation Models (Apple)
Apple's Swift framework giving apps access to the on-device Apple Intelligence language model with guided generation, tool calling and sessions, on OS 26 and later
ProprietaryFree
Hugging Face Hub
Hosting platform for open models, datasets and Spaces demo apps built on git repositories and Xet storage, with Inference Providers, Endpoints, Jobs and an open-source Python client and CLI
MixedFreemium
llama.cpp
C/C++ inference engine for LLMs and vision-language models with GGUF quantized weights, Metal, CUDA, Vulkan and other backends, a CLI and an OpenAI-compatible server
Open sourceOpen source
LM Studio
Proprietary desktop app for discovering, downloading and running local LLMs with llama.cpp and MLX engines, a chat UI, an OpenAI-compatible server, CLI, SDKs and a headless daemon
ProprietaryFreemium
Mistral AI
French model provider offering open-weight and proprietary models through the Studio API, OCR and speech models, the Vibe agent for productivity and coding, and self-deployment options
MixedFreemium
MLX LM
Python package and CLI for running, serving, quantizing and fine-tuning large language models on Apple Silicon with MLX
Open sourceOpen source
vLLM
High-throughput LLM inference and serving engine with PagedAttention, continuous batching, quantization and an OpenAI-compatible server for GPU and accelerator clusters
Open sourceOpen source

Compare Ollama with its alternatives →

Sources & verification

Primary sources consulted for this record. Claims are tied to the fields each source supports; the automated link checker re-verifies every URL on a schedule.

License
https://github.com/ollama/ollama/blob/main/LICENSE
Supports: license_spdx, source_model
checked Nov 1, 2026
Official documentation
Ollama documentation
https://docs.ollama.com/
Supports: capabilities, interfaces, integrations
checked Nov 1, 2026
Official site
Ollama
https://ollama.com/
Supports: name, summary
checked Nov 1, 2026
Pricing page
https://ollama.com/pricing
Supports: pricing_model
Pay as you go, Pro, Max, Team, Enterprise for cloud models; local use unlimited
checked Nov 1, 2026
Repository
ollama/ollama
https://github.com/ollama/ollama
Supports: operating_systems, languages, capabilities
checked Nov 1, 2026

Confidence: High
Reviewed: Nov 1, 2026
Links: Verified · Nov 2, 2026

Spotted an error?

MLX

Apple Inc. · Apple Silicon ML & MLX

Apple's open-source array framework for machine learning on Apple Silicon, with NumPy-like Python, C++ and Swift APIs and unified CPU/GPU memory

MITOpen sourceFreeLocalNov 2, 2026

Ray

Ray project (Anyscale and contributors) · Distributed & parallel computing

Open-source Python framework for distributing tasks, actors and ML workloads across a cluster, with libraries for data processing, training, tuning, serving and reinforcement learning

Apache-2.0Open sourceFreeLocalSelf-hostedBroken

Unsloth

Unsloth AI · Training & fine-tuning

Open-source framework for fine-tuning and running LLMs with reduced memory, covering LoRA, QLoRA, full fine-tuning and RL, GGUF export, Dynamic quantizations and a local Studio/Desktop UI

Apache-2.0Open sourceFreeLocalSelf-hostedNov 2, 2026

Core ML

Apple Inc. · Apple Silicon ML & MLX

Apple's on-device machine-learning runtime for iOS and macOS apps, with the open-source coremltools package for converting PyTorch and other models to its format

BSD-3-ClauseFreeLocalNov 2, 2026

Transformers

Hugging Face · Training & fine-tuning

Hugging Face's model-definition library for text, vision, audio, video and multimodal models, with Pipeline, Trainer and generate APIs used as the reference by training and inference engines