Member of Technical Staff Evaluation Execution

You will run models on evaluation tasks, integrate them into agent scaffolds, inspect results, and improve evaluation software and infrastructure. You will communicate findings through graphs and written reports, coordinate sensitive evaluation work with stakeholders, and manage multiple concurrent evaluation projects.

Responsibilities

• Run models on evaluation tasks and review results carefully
• Integrate models into agent scaffolds and evaluation infrastructure
• Build software that improves evaluation speed and informativeness
• Design graphs and communicate conclusions to varied audiences
• Manage concurrent evaluation projects and stakeholders
• Coordinate sensitive evaluations with leadership, lab contacts, and regulators

Requirements

• Software engineering and infrastructure fundamentals
• Ability to debug unfamiliar systems from logs and fix performance bottlenecks
• Ability to work quickly and prioritize high-impact work
• High attention to detail