Member of Technical Staff Search Quality Analyst

You will diagnose search-pipeline quality issues, create metrics, and build training datasets and LLM-as-a-judge labeling pipelines. You will improve snippet and page-selection algorithms, and design and analyze A/B experiments to validate improvements.

Responsibilities

• Diagnose quality issues in the search pipeline
• Design metrics to measure search quality
• Build model-training datasets and LLM-as-a-judge labeling pipelines
• Improve search snippet quality and page-selection algorithms for indexing
• Design and analyze A/B experiments

Requirements

• 4+ years of data analyst or related experience
• Production-grade coding skills at a mid-level backend engineer level
• SQL
• Python
• Experience designing metrics, building LLM-as-a-judge labeling pipelines, training production ML models, or designing evaluations with known ground truth