Airbyte in Production: A Successful Sync of Zero Rows Is Still an Outage

Revenue on the exec dashboard was $0 at 07:10. Airbyte showed every Stripe sync Succeeded. The destination table had zero new rows for 26 hours. The cursor had advanced on empty pages after a permission change on the API key. The connector did its job. The job state lied about the business.

I stopped treating the Airbyte UI as freshness. A successful sync is “the worker finished.” It is not “the grain arrived.” That is the same class of miss I wrote about in Python quality checks and data contracts: the producer (here, the sync) must fail or ticket when volume breaks the band.

Raw is a landing zone

Airbyte’s useful default is a raw table: JSON blob, extracted_at, and a source cursor. Normalization into typed columns is optional and, in my experience, a mediocre silver. I load raw, then model.

flowchart LR
  src["SaaS / DB"] --> ab["Airbyte sync"]
  ab -->|"Succeeded"| raw["Bronze: JSON + cursor"]
  raw --> test["Volume / freshness check"]
  test -->|band ok| dbt["dbt silver / marts"]
  test -->|0 rows| page["Page: silent sync"]
  dbt --> dash["Dashboard"]

The connector ends at bronze. The check is what makes “Succeeded” mean something.

-- Warehouse check after every sync. Fail the DAG if this is 0
-- on a day that is not a known holiday / source outage.
SELECT
    date_trunc('hour', _airbyte_extracted_at) AS hr,
    count(*) AS rows
FROM raw.stripe_charges
WHERE _airbyte_extracted_at >= current_timestamp - interval '36 hours'
GROUP BY 1
ORDER BY 1;

I do not let Looker read raw.*. JSON keys appear and disappear. Analysts will JSON_EXTRACT a field that vanished on Tuesday. Silver casts, renames, and tests the grain. Raw is for replay. Typed silver is a dbt model with a unique_key — the write contract in incremental models, not a connector checkbox.

Self-host versus Cloud

Self-host: you own the workers, the disk that holds the sync state, and the 2 a.m. OOM. I have lost a weekend to a stuck normalization pod on a cluster that also ran Airflow. Cloud: you own the bill and a status page. I use Cloud for a handful of long-tail SaaS sources I refuse to write extractors for. I self-host when the destination is unusual or the row volume makes Cloud pricing silly.

Neither removes the volume check. Cloud “success” is the same empty-page lie.

For databases I already know, I still prefer Debezium on Postgres when I need deletes and a replication slot I can alarm. Airbyte’s CDC connectors are fine for “get it in the warehouse this month.” They are not a substitute for slot hygiene on a primary you cannot afford to fill.

Airbyte does not replace dbt

I have inherited “transformation” steps inside the connector that encoded net revenue. They were untested, unreviewed, and wrong after a Stripe field change. Load raw. Model in dbt. Incremental silver with a unique_key is a write contract — not a checkbox in the sync. The connector handles auth, pagination, and schema discovery. It does not own fct_revenue.

Schema drift: additive fields in raw are normal. I version the bronze table or keep a VARIANT column so old rows stay valid. Breaking changes belong in a contract with the source owner, not in a silent column drop that turns a dbt not_null red for a week while nobody looks.

-- Silver: parse once, test the grain. Do not SELECT from raw in BI.
SELECT
    _airbyte_raw_id                          AS raw_id,
    (_airbyte_data:"id"::string)             AS charge_id,
    (_airbyte_data:"amount"::number) / 100.0 AS amount,
    (_airbyte_data:"status"::string)         AS status,
    _airbyte_extracted_at                    AS extracted_at
FROM raw.stripe_charges
QUALIFY row_number() OVER (
    PARTITION BY _airbyte_data:"id"::string
    ORDER BY _airbyte_extracted_at DESC
) = 1;

Pitfalls

Alerting only on Succeeded. Empty pages, a cursor that walks past a permission error, and a source that returns 200 with [] all look green. Volume is the SLA.

Normalization as the mart. Airbyte’s typed tables are a starting point. They will not match your grain after the third schema change. Keep them off the semantic layer.

Self-host on the same cluster as Airflow without quotas. A stuck sync fills the disk. The scheduler dies. Two products, one outage.

CDC through Airbyte on a primary you cannot fill. If the source is Postgres you already operate, Debezium plus a slot alert is the honest path. Airbyte CDC is fine until the WAL conversation starts.

Letting analysts live in raw. You will spend a quarter explaining why a JSON key vanished. Silver or nothing.

Resetting a cursor to “fix” a gap without a volume check. The next sync can look Succeeded and still skip the day you cared about. After a reset I run the 36-hour row query before I call it closed.

I keep a one-page runbook: which connections are Cloud versus self-host, who owns the API key rotation, and the weekday minimum rows per source. The Stripe key lived in a shared vault nobody rotated. The connector was fine. The secret was not. After rotation I do not trust the next green check until the 36-hour row query shows a normal weekday band.

The Stripe incident fix was 40 lines: a freshness-and-volume test on raw.stripe_charges that fails the Airflow task below 500 rows in 24 hours on a weekday. Holidays get a calendar exception, not a disabled test. The dashboard stayed empty until we rotated the key. The difference is we found it at 07:12 from the DAG, not at 09:40 from the CFO.

Succeeded was never the SLA. Rows were. Airbyte is a loader. The check is what makes the load honest.