Member of Technical Staff Search Crawler Analyst

You will diagnose quality issues in the crawling pipeline and improve page selection for indexing. You will design metrics and experiments, build training datasets and LLM-as-a-judge labeling pipelines, and train small production models that improve search and answer-system quality.

Responsibilities

• Find and diagnose crawling-pipeline quality issues
• Train models that optimize pipeline components
• Build model-training datasets and LLM-as-a-judge labeling pipelines
• Improve page-selection algorithms for indexing
• Design and analyze experiments to validate improvements

Requirements

• 4+ years of data analyst, ML engineer, or related experience
• Production-grade coding skills comparable to a mid-level backend engineer
• Experience designing metrics from scratch
• Experience training production ML models with measurable metric improvements