Member of Technical Staff Evaluation Infrastructure

You will develop reliable, efficient evaluation infrastructure that runs evaluations at scale. You will work on new evaluations and internal tooling, maintain evaluation signal quality, and identify and resolve performance bottlenecks across inference, fleet management, and asynchronous evaluation orchestration.

Responsibilities

• Build reliable, efficient, and easy-to-use infrastructure for evaluations at scale
• Develop new evaluations and internal tooling
• Maintain evaluation signal quality
• Identify and resolve performance bottlenecks across the evaluation infrastructure stack
• Optimize inference, capacity fleet management, and asynchronous evaluation orchestration

Requirements

• Experience building, debugging, and optimizing large-scale distributed systems
• Willingness to solve hard problems across all layers of the stack

Benefits

• Equity
• Medical coverage
• Vision coverage
• Dental coverage
• 401(k) retirement plan
• Short-term disability insurance
• Long-term disability insurance
• Life insurance
• Discounts and perks