Software Engineer Evaluation Platform Infra

You will design, build, and maintain a platform for authoring, running, tracking, and analyzing model evaluations. You will work across evaluation libraries, distributed backend systems, data pipelines, APIs, and user-facing applications. You will make results reproducible, partner with researchers, and turn bespoke workflows into self-serve systems.

Responsibilities

• Design, build, and maintain the model-evaluation platform.
• Deliver end-to-end capabilities across evaluation libraries, distributed backend systems, data pipelines, APIs, and user-facing applications.
• Build abstractions for evaluation tasks, environments, graders, datasets, and model outputs.
• Ensure reproducibility and trustworthiness through versioning, provenance, observability, failure recovery, and quality controls.
• Partner with researchers to identify bottlenecks and convert bespoke workflows into self-serve systems.

Requirements

• Bachelor’s degree or equivalent practical experience in computer science, engineering, machine learning, or a related field.
• Two years of post-graduate software engineering or ML engineering experience, excluding internships.
• Experience building or maintaining evaluations, benchmarks, graders, or model-quality systems for large language or multimodal models.
• Software engineering fundamentals and experience building reliable, maintainable systems.
• Proficiency in a backend programming language.
• Experience with databases, data pipelines, distributed systems, or data-intensive infrastructure.
• Ability to work across the stack and own projects through deployment and operation.
• Experience collaborating with cross-functional partners and subject-matter experts.

Benefits

• Health, dental, and vision benefits.
• Unlimited PTO.
• Paid parental leave.
• Relocation support as needed.