Setting up Skill Issues: Generate Evals and Optimize Agent Skills

In this walkthrough, we set up the Skill Issues repository: installing the environment with just, configuring the .env file for local and cloud runs, generating a dataset for a domain, and running the optimization loop against a skill.

Skill Issues is an open source framework for two things: generating evaluation datasets for data engineering workflows, including the destructive operations most evals skip, and optimizing agent skills against those evals with GEPA. Because Bauplan gives you Git-for-data, destructive tasks can run in parallel against a production lakehouse, and every operation is immutable and programmatically auditable. That means verifiers can check what an agent actually did, rather than regexing terminal output or using an LLM as a judge.