Member of Technical Staff Answer Quality and Evals

You will build shared evaluation infrastructure and a platform for replaying and analyzing agent traces. You will operate scalable data systems, diagnose quality problems, turn findings into improvements, and collaborate with technical and product partners.

Responsibilities

• Build shared evaluation infrastructure
• Develop a platform for replaying and analyzing agent traces
• Build and operate scalable systems for processing, storing, and monitoring interaction, trace, and evaluation data
• Turn answer-quality problems into evaluations, analyses, and product improvements
• Shape how answer quality is measured and improved

Requirements

• 4+ years of software, data, or machine learning engineering experience
• Python
• SQL
• System design
• Data modeling
• Distributed systems
• Big-data systems
• Distributed compute
• Large-scale storage
• High-volume data pipelines
• Production system operation

Benefits

• Health insurance
• Dental insurance
• Vision insurance
• Retirement benefits
• Fitness account
• Commuter account
• Dependent care account