Directory

Mature open-source Python framework for writing crawlers (spiders) that fetch pages asynchronously, extract structured items and export them through pipelines

API that returns live search-engine result pages from Google and other engines as structured JSON, with localization, device and search-type parameters

Open-source backend platform built around a dedicated PostgreSQL database with auto-generated APIs, authentication, file storage, realtime subscriptions and edge functions

Salesforce's visual analytics platform: Tableau Desktop authoring (now with a free edition), Tableau Cloud and Server for sharing, and Tableau Next for CRM analytics

Web search, extraction, crawling and research API designed for LLM agents, returning concise source content with SDKs, framework integrations and a hosted MCP server

PostgreSQL extension for time-series data adding hypertables, compression and continuous aggregates, from the company now named Tiger Data

Distributed SQL query engine that federates queries across data lakes, warehouses and databases through connectors, formerly PrestoSQL

GPL-licensed in-memory search engine in C++ offering typo-tolerant instant search, faceting, vector and hybrid search, with Typesense Cloud hosting

The Internet Archive's public web archive for browsing historical snapshots of web pages, with Save Page Now, Availability, CDX and Memento APIs

Open-source vector database in Go with pluggable vectorizer modules, hybrid BM25 and vector search, multi-tenancy and a managed cloud

Web-data company behind Scrapy offering Zyte API for ban-resistant fetching, browser rendering and automatic extraction, plus Scrapy Cloud for hosting spiders