Dagster
The foundation of a modern data platform.
Dagster is the enterprise orchestration platform that puts developer experience first. With fully serverless or hybrid deployments, native branching, built-in cataloging, and cost observability, you can now expect more from your orchestration solution.
Offload operational concerns
Accelerate your work with Dagster . Scale at will and contain costs while eliminating infrastructure concerns.
Dagster self-service plans include everything you need to run a single production deployment with built-in CI/CD support.
With the Dagster enterprise plans, you can run multiple deployments with enhanced support and security.
Dagster is the cloud-native orchestrator for the whole development lifecycle
Define assets in Code
Write regular Python functions, and Dagster infers the DAG without any tedious work.
Learn more
Develop pipelines locally
Don't fight against complex local dependencies or catch typos after long build pipelines. Run all of Dagster locally with one pip install.
Integrate the entire data stack
Use your existing code for dbt, Airbyte, Fivetran, Snowflake, Bigquery, and others.
Unit tests
Unit-test your data applications, separate business logic from environments, and set explicit expectations on uncontrollable inputs.
Branch Deployments
Deploy confidently to production by first testing new features in automatically managed staging environments.
Sensor and schedule testing
Manually trigger a test evaluation of a sensor or schedule and view the results from the UI
Customized Deployments
Enterprise plans are configurable to include any number of Deployments based on your organization’s needs.
Integrated CI/CD
Go from pull request to production effortlessly with continuous code deployments.
Environment variables
Parameterize your data pipelines without modifying code or insecurely hard-coding database credentials.
Debug runs
Observe individual runs and query logs for detailed diagnostics, discover the most time-consuming tasks via a Gantt chart, re-execute subsets of a run, and more.
Track asset lineage
Get details on each asset: Freshness, status, schema, metadata, and dependencies displayed in one consolidated view.
Alerts
Trigger Slack or email notifications on run failure/success, schedule/sensor tick failure, and more.
“Dagster has been instrumental in empowering our development team to deliver insights at 20x velocity. Inception to insight is down to 2 days vs 6 months before.”
Gu Xie
Head of Data Engineering at Group 1001
Go beyond traditional orchestration
Thanks to its asset-oriented framework, Dagster provides an always-up-to-date catalog of your key assets.
The catalog is designed to support the work of pipeline builders as well as downstream data consumers seeking to build upon these assets.
Data pipelines must deliver trusted data, meaning data that is up-to-date and fit-for-purpose.
With Dagster's Asset Checks, you can specify your exact expectations for a given asset and how the pipeline should respond if the expectations are not met.
Dagster Insights provides unique tracking of the cost to materialize individual assets. It also provides longitudinal reporting on any metadata related to a given asset, such as data reliability.
Gain transparency into the origin of each column of data. Understand the impact of changes to upstream data sources or transformations on downstream systems and reports.
Two deployment options
Serverless
Effortless spin-up
Let us host the entire orchestration engine. Spin-up in Serverless is completely effortless. Just write a Python file and we do the rest: no Dockerfiles, no Kubernetes, no provisioning.
See the docs.
Free for 30 days
Hybrid
Bring your own compute
Bring your compute platform and let Dagster host the control plane, which does not see your code or data. Maximize flexibility and security while offloading the vast majority of operational burden. Egress-only requirements means no network headaches. Dagster handles patching, upgrades, and the service SLA. (architecture overview | Read the docs)
Free for 30 days
Integrated security and role-based access management
We built role-based access control and component-level isolation into Dagster because security is never a one-size-fits-all solution.
