Dagster
Assets, checks, and why the DAG is a data graph
5 questions with solutions
- Q1DatabricksStripeShopify
What is a software-defined asset, in one sentence?
Solution
A table (or file, or report) you declare with its dependencies; the scheduler materializes it. You are not scheduling a Python function that happens to write a table — you are scheduling the table.
- Q2AirbnbNetflixSpotify
Asset checks vs dbt tests. Who owns which?
Solution
dbt tests grain and relationships in the warehouse. Asset checks are the orchestrator saying this materialization is not fit for downstream. Both. Checks without dbt leave SQL untested. dbt without checks leave a green DAG on an empty table.
- Q3UberLyftDoorDash
A late partition arrived. What must not happen?
Solution
A full rebuild of every downstream asset. Partitioned assets + targeted backfill. If your graph cannot backfill one day, it is a batch script wearing assets.
- Q4Capital OneJPMorganDeloitte
When do you keep Airflow instead of moving to Dagster?
Solution
A large heterogeneous ops estate, vendor sensors, and a team already staffed on Airflow. Move when the DAG is secretly a data graph and assets would delete half the glue. Fashion is not a reason.
- Q5StripeSquareBlock
Where do warehouse credentials live?
Solution
In a resource / I/O manager, not in the op. Same rule as Airflow connections. If the PR has a password, it fails the screen.