Senior Software Engineer

You will build and operate an evaluation framework for computer-use-agent benchmarks. You will help researchers and forward deployed engineers integrate benchmarks, establish quality standards and checks, improve scheduling and observability, and ensure results are reproducible. You will work across cluster infrastructure, browser and desktop environments, customer workflows, and integrations.

Responsibilities

• Support researchers and forward deployed engineers in integrating benchmarks
• Define benchmark-entry standards and build enforcement checks
• Improve scheduling and observability for evaluation jobs
• Ensure reproducible results across trials
• Tune clusters and build benchmark-specific tooling
• Gather customer measurement requirements and automate them into evaluation harnesses

Requirements

• 5+ years of backend development experience
• Production Python experience
• Experience building test, QA, or evaluation tooling
• Experience operating distributed systems on Kubernetes in a public cloud
• Experience building and shipping end-to-end systems
• Experience with REST or GraphQL APIs and external-service integrations
• Knowledge of relational and non-relational databases
• Knowledge of message queues such as SQS, RabbitMQ, or Kafka
• Experience with metrics, tracing, and monitoring