polars cheat sheet

Expressions, lazy scans, group by, joins, and the streaming engine that let one machine handle what used to need a cluster.

ProgrammingIntermediate7 sections

Reading and scanning

pl.scan_parquet("s3://lake/orders/**/*.parquet"): scan_ builds a LazyFrame and pushes projections and filters down into the reader, while read_ pulls everything into memory first.
pl.read_csv("orders.csv", schema_overrides={"order_id": pl.Int32}, try_parse_dates=True): schema_overrides replaced the old dtypes argument in polars 1.0. Type inference only reads the first 100 rows by default.
pl.scan_csv("orders.csv", infer_schema_length=None): Scans the whole file to infer types. This is the fix for a column that looks like an integer for 100 rows and then hits an alphanumeric id.
pl.read_database_uri(query="select * from orders", uri=dsn): Uses ConnectorX and returns Arrow buffers directly, skipping the row-by-row DBAPI conversion that dominates a pandas read_sql.
lf.sink_parquet("out.parquet", compression="zstd"): Executes the lazy query straight to disk in batches, so peak memory tracks the batch size rather than the size of the result.

df.with_columns( (pl.col("revenue") - pl.col("cost")).alias("margin"), pl.col("region").str.to_uppercase(), ): with_columns takes any number of expressions and evaluates them in parallel. Expressions in one call cannot see each other's output, so chain a second call.
pl.when(pl.col("amount") > 100).then(pl.lit("high")).otherwise(pl.lit("low")): The polars conditional. Every branch must resolve to the same dtype, so wrap literals in pl.lit rather than relying on inference.
pl.col("amount").sum().over("customer_id"): A window function that keeps row count intact, matching a SQL OVER clause. Add mapping_strategy="join" when the inner expression changes length.
pl.col(pl.String).str.strip_chars(): Expressions can select by dtype, so one call cleans every string column. Combine with pl.exclude to skip key columns you must not touch.
pl.col("payload").str.json_path_match("$.user.id"): Extracts one field without decoding the whole document. Full parsing with str.json_decode should be given an explicit schema to avoid a scan.

df.filter((pl.col("amount") > 100) & (pl.col("region") == "EU")): Parentheses are mandatory because & binds tighter than the comparison operators in Python. Passing several arguments to filter ANDs them for you.
df.select(cs.numeric() - cs.by_name("id")): The polars.selectors module supports set algebra over column groups, which beats hand-maintaining a list of forty metric columns.
df.sort("amount", descending=True, nulls_last=True): Nulls sort first by default, which quietly puts the empty rows at the top of every worst-offenders report you build.
df.unique(subset=["order_id"], keep="last", maintain_order=True): Deduplication does not preserve row order unless asked, and maintain_order gives up some parallelism on large frames.
df.head(10).glimpse(): glimpse prints dtypes and leading values column by column, which stays readable in a terminal where a 60-column frame does not.

df.group_by("region").agg( pl.col("amount").sum().alias("total"), pl.col("order_id").n_unique().alias("orders"), ): group_by replaced groupby in polars 1.0. Aggregations are ordinary expressions, so anything valid in select works inside agg.
df.group_by("region", maintain_order=True): Group order is nondeterministic by default because groups are built in parallel. Only pay for ordering when the output is user-facing.
pl.col("amount").filter(pl.col("status") == "paid").sum(): Filtering inside an aggregation gives conditional sums in a single pass, the polars form of SUM(CASE WHEN ... THEN ... END).
df.group_by_dynamic("ts", every="1h", period="24h", closed="right"): Tumbling and sliding windows over time. The index column must be sorted, and every shorter than period produces deliberately overlapping windows.
pl.col("amount").sort_by("ts").last(): Latest value per group ordered by another column, which removes the usual sort-then-deduplicate pass entirely.

df.join(dim, on="customer_id", how="left", validate="m:1"): validate raises on unexpected fan-out instead of silently multiplying rows. polars keeps one key column for equi-joins, so pass coalesce=False to see both sides.
df.join_asof(quotes, on="ts", by="symbol", strategy="backward", tolerance="1s"): The point-in-time join. Both frames must be sorted on the asof key, and tolerance takes a duration string rather than a timedelta object.
df.join(other, how="anti", on="order_id"): Anti and semi joins are first class, so "which rows never made it downstream" is one call instead of a join plus an indicator filter.
pl.concat([a, b], how="diagonal_relaxed"): Unions frames with different columns and supercasts mismatched dtypes. Plain vertical concat raises on any schema difference at all.
df.explode("items").unnest("payload"): explode expands a list column to one row per element and unnest flattens a struct into columns. Chaining them is the standard JSON flattening pass.

lf.collect(engine="streaming"): Runs the query in batches so it can exceed available memory. This replaced the older collect(streaming=True) flag during the 1.x line.
lf.explain(optimized=True): Prints the physical plan. Read it bottom up and confirm the projection and selection appear at the scan node, which proves pushdown happened.
lf.profile(): Returns the result plus a per-node timing frame, the fastest way to learn which join is actually consuming the runtime.
df.lazy().filter(...).collect(): Wrapping an eager frame still buys cross-operation optimization. The tradeoff is that errors surface at collect rather than on the offending line.
pl.Config.set_streaming_chunk_size(50_000): Smaller batches cut peak memory in the streaming engine at the cost of per-batch overhead. Tune it only after a real out-of-memory failure.

df.to_pandas(use_pyarrow_extension_array=True): Preserves nulls and avoids a copy where the memory layout allows. Without the flag an integer column containing nulls degrades to float64.
df.to_arrow(): Free, because polars already stores Arrow memory. This is the cheapest handoff to DuckDB, an Iceberg writer, or an Arrow Flight endpoint.
pl.from_pandas(pdf, include_index=False): The pandas index is dropped unless requested, because polars has no index concept. Row position is the only ordering that exists.
df.null_count(): polars keeps null and NaN separate, so a float column can hold both. is_null does not match NaN and is_nan does not match null.
df.estimated_size("mb"): Reports the in-memory Arrow footprint. Compare it against the compressed Parquet size before assuming an eager read will fit.

From DataLane — tutorials at/blog, practice SQL live in theplayground.