Directory

Python workflow orchestrator for scheduling and monitoring DAGs of tasks, with hundreds of provider packages and managed offerings from several clouds

Distributed data processing engine with DataFrame and SQL APIs in Python, Scala, Java and R, plus structured streaming and MLlib

Asset-oriented Python orchestrator that models pipelines as software-defined data assets with lineage, checks and a hosted Dagster+ option

Foundational Python library for n-dimensional arrays, vectorized math, linear algebra, random numbers and the array API used by the scientific stack