Founding Backend Engineer
Company vision
We are building new infrastructure for data and AI workloads, motivated by two large tailwinds:
Python is the lingua franca of AI and data, and more generally one of the most widespread languages on the planet (and growing!);
Object storage is consolidating as the main persistence layer for enterprise applications: infinitely scalable, durable, cheap.
As the need for Python data-driven applications grows, today’s platforms - built for the “Big Data” era, where a specialized workforce would move slowly to build back-end systems - are not going to cut it: they are too complex to learn, too expensive to scale and just too bloated to be effective.
We believe that the future is code-first, and that full stack developers with no Big Data experience should be able to build reliable data pipelines backed by S3 leveraging the abstractions they already know: functions, tables, packages, Git.
Bauplan today is (super fast containerized) Python functions + Git-for-data (on S3): our clients already run tens of thousands of jobs every week through our serverless abstractions, and Bauplan is used effectively by senior data scientists, devOps and automation engineers, as well as interns. Our users run and deploy code from their laptop directly in the cloud, without ever having to maintain dedicated infrastructure.
To make the complexity invisible to the final user (“it just works”), we are innovating across the entire spectrum of the data life-cycle: API / DAG design, function scheduling, ephemeral query engines over S3, performant data sharing, table format semantics and much more.
Full job description
The founding team (three founders and the first founding engineer) is the same as their previous company, acquired in 2019 by an AI public company. The company is well-funded and supported by top Silicon Valley funds, as well as individuals from esteemed institutions such as NVIDIA, Docker, AWS, Redis Labs, as well as founders from other top startups in the field: investors and advisors include McArthur Fellow and renowned AI professor Chris Re (Stanford), computer scientist and entrepreneur Ihab Ilyas (Waterloo), Pandas and Apache Arrow co-creator Wes McKinney, and the founder of CockroachDB Spencer Kimball.
Our client is focused on building a next generation serverless data platform that can revolutionize how data engineers and scientists work. Their goal is to abstract data infrastructure away from engineers and data scientists, so that they can focus solely on building data driven applications with concepts they already know.
They’re building exciting features and tackling hard problems in the data space, often starting from an unconventional point of view. The initial system was brought to market a few months ago, and it’s live at enterprise scale: they are looking for additional engineers to work on the next iteration of many of their most exciting features, such as storage optimization and improved function scheduling. The team has a long history of cutting edge R&D in service of their product vision, regularly shares its progress in the developer and academic community, and often contributes to popular open source tools.
The early stage comes with hard problems: willingness and ability to continually learn new things, make intelligent trade-offs, challenge conventional wisdom, and working with blurry lines are imperative.
As a Founding Engineer, you will have a unique opportunity to help shape the technology and culture of the company and be part of a team that is passionate about making a positive impact in the industry. If you are a highly motivated engineer who is passionate about building innovative, elegant, simple solutions to complex problems, this job is for you!
What you will do
You will be responsible for refining the foundations of our platform and kick-off new exciting features. You will be working with the rest of the engineering team to prioritize, design and execute the company vision. In particular:
Implement performant systems to run data workloads over S3.
Analyze existing systems in the light of customer workloads and product requirements, propose and implement enhancements.
Own the development of new features, starting from user-facing requirements and building on / refining existing abstractions together with the rest of the team.
Evaluate approaches in the literature to recommend innovative solutions to the peculiar challenges we encounter as the platform evolves.
Help with the development of our internal and external libraries, and proactively works at the integrations with open source tooling whenever necessary.
Work with the team to assist customers with triaging issues, debugging problems, and generally ensure a good experience for everybody on the platform, minimizing incidents and providing clarifications as needed.
Collaborate with others to ensure quality of codebase and continuously improve our development cycle (CI/CD, iteration speed, security etc.).
What we are looking for:
Excellent at:
communication skills: writing and reading design docs, giving and receiving feedback, lead technical discussions, communicate challenges and solutions effectively at different resolution levels;
Python and Golang
modern software development practices: Git, Docker, infra-as-code, CI/CD, testing, open telemetry etc.
async programming and concurrency.
Extensive experience with cloud-based infrastructure (AWS preferred) and distributed systems design, with expertise in managing high-availability and fault-tolerant systems
Experience / working knowledge of:
distributed systems (e.g. k8s for webservices, Spark for data transformation, or custom distributed infrastructure);
cloud services, with focus on AWS to start;
a dialect of SQL.
Nice to have
Working knowledge of C++ / Rust;
working knowledge of existing FaaS frameworks (e.g. OpenWhisk, OpenFaas);
previous experience with data-specific abstractions (distributed transactions, query planning, data catalogs etc.);
theoretical knowledge related to correctness, verification, simulation of distributed systems.
What we offer
Founding Distributed Systems Engineer at Bauplan
We're an equal opportunity employer. All applicants will be considered for employment without attention to race, color, religion, sex, sexual orientation.