DataLane
← All stacks

AI / RAG / MLOps

Retrieval, features, and the boundary between DE and ML

10 questions with solutions

  1. Q1OpenAIAnthropicNotion

    Why is RAG an ETL problem?

    Solution

    Ingest, chunk, embed, version, ACL-filter, then retrieve. Failures are usually stale chunks, missing metadata, and documents the model should never see — not the model brand. Treat the index like a serving table with a freshness SLO.

  2. Q2UberAirbnbMeta

    Where does a data engineer stop in MLOps?

    Solution

    Reliable feature tables, training-data DAGs, point-in-time joins, and serving logs. Model architecture and eval design belong with ML. Interviewers want the boundary, not a PyTorch lecture. If you cannot explain training/serving skew, you are not done.

  3. Q3PineconeGoogleMicrosoft

    Why do you store the embedding model version next to every vector?

    Solution

    Embeddings from model A and model B are not comparable. A silent model upgrade without a backfill mixes spaces and tanks retrieval. Version the model, the chunker, and the source document hash. This is data versioning, not ML trivia.

  4. Q4Capital OneUnitedHealthPalantir

    ACL-aware retrieval. What breaks if you embed docs and forget permissions?

    Solution

    The index will retrieve a document the caller cannot see. Filter on ACL metadata at query time (or partition indexes by tenant). “We will prompt the model not to reveal it” is not a control. This question shows up at every bank and healthcare loop.

  5. Q5StripeShopifyZendesk

    LLM in a batch pipeline: four patterns that survive production.

    Solution

    Classify, extract structured fields, summarize into a stored column, route (ticket → queue). Always: schema, retries, cost cap, and a deterministic fallback. Never call a chat API from a BI dashboard on every refresh.

  6. Q6UberDoorDashInstacart

    Feature store vs “a gold table named features.” What is the extra work?

    Solution

    Point-in-time correct joins (no leakage), online/offline parity, and a serving path with low latency. A gold table is a start. If training used yesterday’s snapshot and serving uses a live join that can see the label, you shipped leakage.

  7. Q7OpenAIAnthropicNotion

    Eval for a RAG system. What does a data engineer own?

    Solution

    A labeled retrieval set, freshness of chunks, and logs of what was retrieved. “The model felt better” is not an eval.

  8. Q8Capital OneUnitedHealthPalantir

    PII in prompts. How do you not leak the warehouse into the vendor?

    Solution

    Redact or tokenize before the call, keep a deny-list, and do not paste raw rows into a chat UI. Contracts and DLP, not hope.

  9. Q9UberDoorDashMeta

    Online vs offline features for the same model.

    Solution

    Same definition, two stores, a point-in-time story. If online computes a different join, you shipped skew.

  10. Q10SnowflakeGoogleDatabricks

    When is “just use the warehouse LLM function” the wrong answer?

    Solution

    When you need a specific model, tight latency, or tool-use the warehouse cannot host. Cortex/BigQuery ML are ELT steps, not every product.

↑↓ navigate↵ openesc close