Generate a Data Pipeline from a Prompt with Bauplan Skills for Claude Code

In this demo, we build a full data pipeline in Bauplan using Claude Code and the Bauplan Skills plugin. Starting from a single query prompt, the agent scaffolds the project, generates the models, runs dry-runs on a feature branch, catches and fixes a Polars scalar error, and materializes two governed tables to main. The whole workflow happens without touching production data.

This is the second demo in the Bauplan Skills series. The first covered safe ingestion of S3 data into Iceberg tables on a feature branch. This one picks up where that left off, using the table created during ingestion as the source for the new pipeline. The same Skill library handles both, so Claude is not relearning Bauplan each session.

The demo opens with a uv project and Claude Code running side by side, with the Bauplan Skills plugin installed from the Anthropic plugin marketplace. A verbose query prompt gives the agent enough context to work without loops or hallucinations, and Claude begins by reading the project structure, checking Git history, and scaffolding a new pipeline folder. From there, it auto-generates the bauplan_project.yml with the namespace, model names, and description, and writes a models.py file against the Bauplan SDK, following the Skill template. Ruff runs the syntax check on the generated code, and two Polars-based models come out: Total Content Consumption for top-line metrics and Per User Consumption grouped at the user ID level.

Claude then creates a feature branch and runs a dry-run of the pipeline. It catches a Polars scalar error, offers a fix, and runs again. The second dry-run executes the full DAG, monitoring jobs and handling exit codes as it materializes the two tables in the branch namespace. Once the tables are in place, Claude merges the feature branch into main, runs a SQL query to validate the data, and we open the Bauplan UI to review the final governed tables sitting in production.

Why this matters: agents can design and test pipelines on a branch, not on prod. Errors surface in isolation. The DAG runs against real data, but the data is governed by the branch. Only validated output reaches main.