Directory

Nonprofit open repository of petabyte-scale web crawl data since 2008, published as WARC, WAT and WET files on AWS Open Data with URL and columnar indexes

Apple's on-device machine-learning runtime for iOS and macOS apps, with the open-source coremltools package for converting PyTorch and other models to its format

Open-source crawling and scraping library for Node.js/TypeScript and Python that unifies HTTP and headless-browser crawlers with queues, proxies, fingerprints and storage

Apple's app and Swift framework for training image, text, sound, tabular and activity models on a Mac without writing training code, exporting directly to Core ML

Membership organization that registers DOIs for scholarly content and exposes the metadata through a free public REST API, content negotiation and the paid Metadata Plus service

End-to-end and component testing framework that runs tests inside the browser with an interactive runner, paired with the commercial Cypress Cloud

Modular JavaScript library for bespoke, data-driven web graphics: scales, axes, shapes, layouts, geo projections, transitions and DOM selections

Python library for parallel and distributed computing that scales pandas, NumPy and custom task graphs from a laptop to a cluster

Open table format with a transaction log over Parquet files, created by Databricks and hosted by the Linux Foundation, with Spark, Rust and Python implementations

Secure-by-default JavaScript and TypeScript runtime from Deno Land with web-standard APIs, built-in tooling and an optional hosted Deploy service

React-based static site generator from Meta for documentation sites with MDX content, versioning, localization and a plugin architecture

Québec's shared open-data portal where provincial ministries and municipalities publish datasets under a Creative Commons Attribution licence

Python causal-inference library organized around model, identify, estimate and refute steps, with graphical causal models for attribution, counterfactuals and root-cause analysis

Free diagramming editor (formerly diagrams.net) for flowcharts, network, UML and architecture diagrams, available as web app, desktop app and integrations

TypeScript ORM with SQL-like query builder, schema defined in code, migrations through Drizzle Kit, and support for PostgreSQL, MySQL and SQLite drivers

Python framework for programming language-model pipelines with typed signatures and modules, then optimizing prompts and weights automatically against a metric

Python library for heterogeneous treatment-effect estimation with machine learning: double ML, doubly robust learners, causal forests, meta-learners, IV and policy learning

MIT-licensed Python library for SEC EDGAR: company and filing objects, XBRL financial statements, Form 3/4/5, 13F and proxy parsing into pandas, with the SEC identity requirement built in

Pluggable static analysis tool for JavaScript and TypeScript that finds problems in code through configurable rules and can fix many automatically

Statistical office of the European Union publishing harmonised EU and member-state data through the Data Browser, bulk downloads and SDMX/JSON APIs

Open-source virtual whiteboard for hand-drawn-style diagrams with live collaboration, embeddable as a React component, plus a hosted Excalidraw+ service

Open-source software that pools several Apple Silicon Macs (and Linux hosts) into one local inference cluster, sharding large models across devices with MLX

Minimal, unopinionated web framework for Node.js with routing and middleware, the long-standing default for HTTP servers in the Node ecosystem

Log-scanning intrusion-prevention daemon that temporarily bans IP addresses showing repeated authentication failures by updating firewall rules