DataLane
← All stacks

Dagster

Assets, checks, and why the DAG is a data graph

5 questions with solutions

  1. Q1DatabricksStripeShopify

    What is a software-defined asset, in one sentence?

    Solution

    A table (or file, or report) you declare with its dependencies; the scheduler materializes it. You are not scheduling a Python function that happens to write a table — you are scheduling the table.

  2. Q2AirbnbNetflixSpotify

    Asset checks vs dbt tests. Who owns which?

    Solution

    dbt tests grain and relationships in the warehouse. Asset checks are the orchestrator saying this materialization is not fit for downstream. Both. Checks without dbt leave SQL untested. dbt without checks leave a green DAG on an empty table.

  3. Q3UberLyftDoorDash

    A late partition arrived. What must not happen?

    Solution

    A full rebuild of every downstream asset. Partitioned assets + targeted backfill. If your graph cannot backfill one day, it is a batch script wearing assets.

  4. Q4Capital OneJPMorganDeloitte

    When do you keep Airflow instead of moving to Dagster?

    Solution

    A large heterogeneous ops estate, vendor sensors, and a team already staffed on Airflow. Move when the DAG is secretly a data graph and assets would delete half the glue. Fashion is not a reason.

  5. Q5StripeSquareBlock

    Where do warehouse credentials live?

    Solution

    In a resource / I/O manager, not in the op. Same rule as Airflow connections. If the PR has a password, it fails the screen.

↑↓ navigate openesc close