Senior Software Engineer AI Benchmarking

You will build, scale, maintain, and improve biosecurity evaluations for frontier and open-source AI models. You will develop model adapters and internal assessment tools, troubleshoot agent behavior, contribute cloud infrastructure, conduct analyses, and support the public dashboard of AI biology capability trends.

Responsibilities

• Assess frontier and open-source AI models using biosecurity evaluations
• Build model adapters and troubleshoot unexpected agent behavior
• Build, scale, maintain, and improve biosecurity evaluations
• Improve evaluations using agent frameworks, jailbreaking methods, and elicitation strategies
• Develop internal tools for scalable capability assessments
• Contribute to cloud infrastructure for evaluations and analyses
• Contribute analyses to the public AI biology capabilities dashboard

Requirements

• 5+ years of software engineering experience in industry roles and/or open-source contributions
• Python fluency
• Docker proficiency
• AWS proficiency, particularly ECS
• Experience building tools that use or evaluate LLMs or AI agents
• Knowledge of coding practices and coding-agent use
• Code quality, readability, and human oversight of human- and AI-written code

Benefits

• Health, dental, and vision insurance
• Yearly mental health allowance
• Flexible work hours
• Unlimited paid time off
• Minimum 12 weeks of fully paid parental leave
• Fully paid conference sponsorships
• Yearly development allowance
• Employer 401(k) match up to 4% of total salary