Directory

Language-independent columnar in-memory format and libraries that let DuckDB, Polars, pandas, Spark and databases exchange tables without copying

Extensible Rust query engine built on Apache Arrow offering SQL and DataFrame APIs, Parquet readers and an optimizer for embedding in data systems

Open table format for data lakes that adds ACID snapshots, schema and partition evolution and time travel over Parquet, ORC or Avro files

Open columnar file format for analytical data with compression, encodings and statistics, readable by nearly every data engine and library

Distributed data processing engine with DataFrame and SQL APIs in Python, Scala, Java and R, plus structured streaming and MLlib

Open-source columnar OLAP database server for fast aggregations over billions of rows, with MergeTree storage, SQL and a managed ClickHouse Cloud

Nonprofit open repository of petabyte-scale web crawl data since 2008, published as WARC, WAT and WET files on AWS Open Data with URL and columnar indexes

Python library for parallel and distributed computing that scales pandas, NumPy and custom task graphs from a laptop to a cluster

Usage-priced market data platform with normalized order-book (MBO/MBP), trades, OHLCV and reference schemas for futures, options and equities, delivered via historical, live and batch APIs in the DBN format

Open table format with a transaction log over Parquet files, created by Databricks and hosted by the Linux Foundation, with Spark, Rust and Python implementations

In-process analytical SQL database that queries Parquet, CSV and Arrow data directly from Python, R, Node or the CLI

Python library extending pandas with geometry columns, spatial operations, reprojection, spatial joins and file I/O for vector geodata

Rust-based DataFrame library with Python, Rust, Node and R bindings offering lazy query optimization, multithreading and Arrow-native memory

Distributed SQL query engine that federates queries across data lakes, warehouses and databases through connectors, formerly PrestoSQL