Directory

Python workflow orchestrator for scheduling and monitoring DAGs of tasks, with hundreds of provider packages and managed offerings from several clouds

Distributed data processing engine with DataFrame and SQL APIs in Python, Scala, Java and R, plus structured streaming and MLlib

Asset-oriented Python orchestrator that models pipelines as software-defined data assets with lineage, checks and a hosted Dagster+ option

Rust-based DataFrame library with Python, Rust, Node and R bindings offering lazy query optimization, multithreading and Arrow-native memory