DevEx for developers who don't write code

Next week we're releasing Bauplan 0.3, which brings typed data contracts to every pipeline. Each step now declares the tables it reads and writes, column by column: the type of each column, whether it can be null, what it means in plain language and, where declared, which upstream column it comes from.

Bauplan enforces these contracts at every stage. Your type checker flags mistakes in the editor, the Bauplan API rejects an inconsistent pipeline before any compute starts, and a run whose output breaks its contract fails without publishing anything.

Essentially, definitions are expressed entirely in code and travel with the data: they are written into the table's Iceberg metadata, versioned alongside it and readable by any Iceberg client and by agents through our MCP server. The same contracts hold across SQL and Python steps.

How we got here

Bauplan's previous SDK was designed to make pipelines quick to write. A step was a Python function with two decorators: one for its environment, one for the table it produces. Here is a step that reads two columns from a table of Titanic passengers and computes the survival rate by age:

import bauplan


@bauplan.python("3.12", pip={"polars": "1.37"})
@bauplan.model(materialization_strategy="REPLACE")
def survival_rate_by_age(
    passengers=bauplan.Model("titanic", columns=["Age", "Survived"]),
):
    import polars as pl

    df = pl.DataFrame(passengers)
    return (
        df.group_by(pl.col("Age").floor())
        .agg(pl.col("Survived").mean().alias("survival_rate"))
        .sort("Age")
        .to_arrow()
    )

Here is the same step in Bauplan 0.3:

from typing import Annotated

import bauplan
import pyarrow as pa


class SurvivalColumns(bauplan.TableSchema):
    """The two passenger columns used for survival analysis."""

    Age: Annotated[
        bauplan.Float64 | None,
        bauplan.TableField(doc="A passenger's age in years."),
    ]
    Survived: Annotated[
        bauplan.Int64 | None,
        bauplan.TableField(doc="1 if the passenger survived, 0 otherwise."),
    ]


class SurvivalRateSchema(bauplan.TableSchema):
    """Passenger survival rate by age, in one-year bins."""

    Age: Annotated[
        bauplan.Float64 | None,
        bauplan.TableField(
            doc="Age bin: the passenger's age rounded down to the year.",
            lineage=SurvivalColumns["Age"],
        ),
    ]
    survival_rate: Annotated[
        bauplan.Float64 | None,
        bauplan.TableField(
            doc="Share of passengers in the bin who survived, from 0 to 1.",
            lineage=SurvivalColumns["Survived"],
        ),
    ]


@bauplan.python("3.12", pip={"polars": "1.37"})
@bauplan.model(materialization_strategy="REPLACE")
def survival_rate_by_age(
    passengers: Annotated[
        pa.Table,
        bauplan.Model(
            "titanic",
            projection_schema=SurvivalColumns,
            filter="Age IS NOT NULL",
        ),
    ],
) -> Annotated[pa.Table, SurvivalRateSchema]:
    """Bins passengers by age and returns the survival rate per bin."""
    import polars as pl

    df = pl.DataFrame(passengers)
    return (
        df.group_by(pl.col("Age").floor())
        .agg(pl.col("Survived").mean().alias("survival_rate"))
        .sort("Age")
        .to_arrow()
    )

It's a lot more verbose! The same logic requires more than three times the code. That is because each extra line spells out something that was left implicit before: the columns going in and out and their types, which of them can be null (Float64 | None may be missing, a bare Float64 may not), what each one means semantically, and, through lineage, where an output column comes from. For a platform built around two decorators and one command, that looks like a step backward.

However, we made the trade on purpose, because who writes pipeline code has changed and consequently, so has the concept of DevEx.

Developer experience is changing

A growing share of pipeline code is written by a coding agent, with humans reviewing the output. That is because writing code is now nearly free, while the bottleneck has moved to verification and trust. Engineers have always known that code is read far more often than it is written. Agents push that ratio to the extreme. Writing takes seconds; reviewing the result, and deciding whether to trust it, still takes the same amount of a person's time.

If someone asks whether the pipeline’s numbers are right, answering means reading code you didn’t write and reconstructing the choices an agent made silently. Is Age expressed in years or months? Were passengers with no recorded age dropped or counted as zero? Did Thursday's change to the source table alter what this step reads? Swap passengers for revenue, and now the question comes from your CFO: is this column monthly or annual, gross or net?

Good software architects already work this way with APIs. They agree on the interface first (check out our Gropius library for API design!), because the interface is the contract between teams, it functions as implicit documentation, and it is hard to change later.

A pipeline's interfaces are its tables. Bauplan 0.3 lets you and your agent write those down first and fill in the transformation after. It helps the agent too: with the upstream schema declared in code, it doesn't have to guess column names, types or meaning. Expressing assets as Python classes makes specifications executable: it removes translation layers, provides machine-checkable contracts, and retains legibility for humans.

The experience is no longer designed for the developer who writes the code: it is designed for an agent and for the developer who verifies the code written by that agent.

The payoff: errors caught early, and meaning that travels with the data

Most pipeline bugs happen at boundaries: where one step hands a table to the next or where a pipeline reads a source table someone else owns. A column gets renamed upstream, a type changes or nulls show up where none were expected. In most stacks, you find out when a dashboard looks wrong.

With a contract at every boundary, we can catch these mistakes at the same three points software teams rely on:

• In your editor, like a type checker. The SDK ships type stubs, so ty or any Python type checker flags a misspelled type, an incorrect argument, or a nonexistent schema class locally, in seconds. Write description= where the SDK expects doc=, and you find out before anything leaves your laptop:

error[unknown-argument]: Argument `description` does not match any known parameter of `TableField.__init__`
  --> models.py:12:28
   |
12 |         bauplan.TableField(description="A passenger's age in years."),
   |                            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

‍

• ‍When the run is planned, like a compiler. Before any compute starts, Bauplan parses the whole pipeline and checks every boundary: what each step reads against what the upstream step declares it produces. Suppose a downstream step reads survival_rate_by_age but declares Age as an Int64, while the step above produces a Float64. The run stops in about a second:

Error: Unknown: Found 1 error while parsing your code:
 - Model "bauplan.downstream":
    - "data": failed to project "RateInput" onto model "bauplan.survival_rate_by_age":
        - "Age" has incompatible projection:
            - "DataType.Float64" cannot be cast to "DataType.Int64"

• ‍When the data is produced, like a runtime assertion. The output contract is exact. If a step returns a missing column, an extra column, the wrong type, or a nullable column where the contract says required, the run fails with ModelOutputContractError. This is the default, with no flag to turn on. And because every run publishes atomically, a failed run leaves your tables exactly as they were.

Types catch structural mistakes. They can't tell you whether revenue is monthly or annual. That is what the doc on each field is for: a definition written next to the code that produces the column. Bauplan saves those definitions, along with nullability, in the table's Iceberg metadata, versioned with the data, so they can't fall out of sync with it. Any Iceberg client can read them, and agents can reach them through our MCP server. This is the foundation of a semantic layer.

Every question now has an answer in the code: Age is a passenger's age in years, passengers with no recorded age are filtered out at the source, and the output says where its columns come from. And if Thursday's upstream change had broken the contract, Friday's run would have failed loudly instead of shipping a wrong chart.

Why Bauplan can do this

Data contracts are not a new idea. What is usually missing is enforcement: a contract kept in a separate YAML file or a catalog describes what a table should look like, but nothing stops a run that violates it. Three design choices let Bauplan enforce contracts instead of just documenting them:

• Code first, in any language. The contract is a Python class in the same file as the transformation, so it is versioned, reviewed and run together with it. SQL and Python steps share the same contracts. Unlike in dbt, Python is not an afterthought; unlike in Airflow, SQL is not a black box.
• Open storage. Schemas, nullability and definitions are written to standard Iceberg metadata, not to a proprietary catalog.
• Git-for-data transactions. Every run happens on a branch and publishes atomically; branches isolate changes, commits leave an audit trail and rollbacks undo them.

Together, they serve one goal, a lakehouse that is correct by design: never plan a mistyped pipeline, never run an inconsistent plan, never publish a faulty run.

Try it

pip install bauplan
bauplan init
bauplan run

bauplan init scaffolds a typed version of the example above. The semantic annotations guide and the type contract reference cover the rest.

Want to see it live? On October 27, we're hosting a webinar, Fail Before You Run: Typed Data Contracts for AI-Written Pipelines: an agent builds a model from a contract, catches its own mistake before anything runs, fixes it on a branch, and merges only once the data matches the spec.

Coming next: a new Rust-based parser and planner that checks a pipeline in milliseconds instead of seconds, a bauplan check command to run these checks without starting a run, and a language server for full editor support. We'll share more detailed release notes on October 29. Stay tuned!