Software Engineer Infrastructure
Posted Updated
Description
You will design, build, and operate distributed infrastructure for model training and inference across thousands of accelerators. You will maintain orchestration, scheduling, storage, and resource-management systems; improve reliability and observability; resolve distributed failures; and develop internal Python and Go libraries and APIs.
Responsibilities
- Design, build, and operate distributed systems for large-scale model training and inference
- Build and maintain orchestration, scheduling, storage, and resource-management infrastructure
- Improve infrastructure reliability, performance, and observability
- Translate infrastructure needs into robust, well-abstracted systems
- Debug and resolve distributed failures across networking, storage, compute, and scheduling
- Write and maintain internal Python and Go libraries and APIs
Requirements
- Expertise designing and developing large-scale distributed systems
- Strong proficiency in Python and Go
- Experience building, deploying, and operating production infrastructure at scale
- Knowledge of consensus, consistency, fault tolerance, and networking
- Experience with ML infrastructure, training orchestration, job schedulers, or distributed storage and data systems
- Experience operating large-scale GPU or TPU clusters
- Experience with container orchestration such as Kubernetes and infrastructure-as-code
- Contributions to open-source infrastructure projects
Benefits
- Health benefits
- Dental benefits
- Vision benefits
- Unlimited PTO
- Paid parental leave
- Relocation support as needed