Tools · SPBoucher
Menu

pandas

In-depth entry

by pandas development team (NumFOCUS) · Library

The standard Python DataFrame library for labeled tabular data, time series and I/O to CSV, Excel, SQL, Parquet and more

BSD-3-Clause licenseOpen sourceFreeLocalSelf-hostable
Visit official site Compare

Link verified Nov 2, 2026

Overview

pandas is the long-established Python library for tabular and time-series data, developed by an open community under the NumFOCUS umbrella and released under the BSD 3-Clause license. Its DataFrame and Series objects carry a labeled index, which drives alignment in arithmetic, joins, reshaping and time-based resampling. Readers and writers cover CSV, Excel, JSON, Parquet, ORC, HDF5, SQL databases, Stata, SAS and SPSS files, and the groupby, pivot, merge and window machinery handles most everyday analysis. Since version 2.0 columns can be backed by Apache Arrow arrays in addition to NumPy, improving string handling and interchange with PyArrow, Polars and DuckDB; the 3.0 line continues to move default string storage and copy semantics in that direction.

pandas is the lingua franca of the Python data ecosystem: scikit-learn, statsmodels, matplotlib, seaborn, Plotly and most data APIs accept or return DataFrames, and nearly every tutorial assumes it. Economists and financial researchers rely on its DatetimeIndex, business-day offsets, resampling and rolling statistics.

The library executes eagerly on a single thread and keeps data in memory, so performance and memory use become limiting on tens of millions of rows; Polars, DuckDB or Dask are the usual escapes. Its API accumulated several ways to do the same thing (loc, iloc, chained indexing, copy-versus-view ambiguities), which is a frequent source of bugs that recent versions address with copy-on-write. Mixed-type object columns and legacy NaN handling also complicate strict typing.

Capabilities

  • Labeled DataFrame and Series with index alignment
  • Readers and writers for CSV, Excel, JSON, Parquet, SQL, HDF5, Stata and more
  • GroupBy, pivot, merge, reshape and window operations
  • Time-series resampling, offsets and rolling statistics
  • Arrow-backed and NumPy-backed column dtypes
  • Missing-data handling and categorical types
  • Integration with matplotlib plotting

Typical workflows

  1. Clean and reshape survey or market data in a notebook
  2. Build features for scikit-learn or statsmodels models
  3. Load SQL query results for exploratory analysis
  4. Resample intraday prices to daily bars

Strengths

  • Largest ecosystem and documentation base in Python data work
  • Extensive I/O coverage including statistical package formats
  • Rich time-series tooling
  • BSD license

Limitations

  • Single-threaded eager execution
  • In-memory only; large datasets need other engines
  • Multiple overlapping idioms and copy/view pitfalls
  • Object dtype columns are slow and loosely typed

Prerequisites

  • Python 3 with NumPy; PyArrow recommended for Parquet and Arrow dtypes

Who it is for

  • Data analysts and scientists
  • Economists and finance researchers
  • Students learning data analysis in Python

Alternatives & comparison

Polars offers a multithreaded, lazily optimized alternative with a different expression API and no index. DuckDB brings SQL and out-of-core execution to the same files and converts to pandas when needed. NumPy underlies pandas and is preferable for homogeneous numeric arrays. Dask parallelizes a pandas-like API across cores or machines. R's data.frame and tidyverse are the closest equivalents outside Python.

  • Apache Arrow

    Language-independent columnar in-memory format and libraries that let DuckDB, Polars, pandas, Spark and databases exchange tables without copying

    Open sourceOpen source
  • Dask

    Python library for parallel and distributed computing that scales pandas, NumPy and custom task graphs from a laptop to a cluster

    Open sourceOpen source
  • DuckDB

    In-process analytical SQL database that queries Parquet, CSV and Arrow data directly from Python, R, Node or the CLI

    Open sourceOpen source
  • edgartools

    MIT-licensed Python library for SEC EDGAR: company and filing objects, XBRL financial statements, Form 3/4/5, 13F and proxy parsing into pandas, with the SEC identity requirement built in

    Open sourceFree
  • GeoPandas

    Python library extending pandas with geometry columns, spatial operations, reprojection, spatial joins and file I/O for vector geodata

    Open sourceOpen source
  • NumPy

    Foundational Python library for n-dimensional arrays, vectorized math, linear algebra, random numbers and the array API used by the scientific stack

    Open sourceOpen source
  • Pandera

    Lightweight Python library for declaring and validating DataFrame schemas for pandas, Polars, PySpark and other backends, with type-hint style models

    Open sourceOpen source
  • Polars

    Rust-based DataFrame library with Python, Rust, Node and R bindings offering lazy query optimization, multithreading and Arrow-native memory

    Open sourceOpen source
  • R

    Language and environment for statistical computing and graphics, with the CRAN package archive and a strong base in academic statistics and econometrics

    Open sourceOpen source
  • vectorbt

    Python library for vectorized backtesting and portfolio analysis built on pandas, NumPy and Numba, with a commercial PRO edition

    Source-availableFree
Compare pandas with its alternatives →

Sources & verification

Primary sources consulted for this record. Claims are tied to the fields each source supports; the automated link checker re-verifies every URL on a schedule.

Confidence
High
Reviewed
Nov 1, 2026
Links
Verified · Nov 2, 2026

Spotted an error?

Broken link, outdated license, wrong category, or a tool that no longer exists? Tell us; reports are reviewed by the editor.

Apache Arrow

Apache Software Foundation · Data catalogs & formats

In-depth entry

Language-independent columnar in-memory format and libraries that let DuckDB, Polars, pandas, Spark and databases exchange tables without copying

Apache-2.0Open sourceFreeLocalNov 2, 2026

GeoPandas

GeoPandas developers (NumFOCUS affiliated) · GIS & mapping

In-depth entry

Python library extending pandas with geometry columns, spatial operations, reprojection, spatial joins and file I/O for vector geodata

BSD-3-ClauseOpen sourceFreeLocalNov 2, 2026

Apache DataFusion

Apache Software Foundation · Analytical engines & OLAP

Extensible Rust query engine built on Apache Arrow offering SQL and DataFrame APIs, Parquet readers and an optimizer for embedding in data systems

Apache-2.0Open sourceFreeLocalNov 2, 2026