Staff AI Product Engineer

You will own architecture for model serving, reinforcement learning, post-training, and developer-facing interfaces. You will set standards for inference performance, model efficiency, fine-tuning, APIs, SDKs, and platform tooling. You will resolve cross-team architectural questions, coach engineers, and evaluate serving engines and accelerator technologies.

Responsibilities

• Set technical direction for inference serving architecture
• Drive model-efficiency strategy and performance trade-offs
• Resolve systemic serving performance and reliability issues
• Own RL and post-training architecture
• Define shared inference and training infrastructure for RL loops
• Establish standards for fine-tuning services and data workflows
• Set standards for developer APIs, SDKs, tooling, versioning, and rate limiting
• Coach engineers and represent the domain in architecture reviews

Requirements

• 8–12 years of engineering experience with production AI systems
• Experience setting technical direction for AI systems
• Expertise in LLM inference, caching, batching, scheduling, speculative decoding, and low-precision inference
• Hands-on expertise in RL for LLMs, including RLHF, DPO, reward modelling, or agentic RL
• Experience designing developer-facing APIs and SDKs
• Experience with control-plane and data-plane architecture
• Experience with GPU or accelerator workloads
• Knowledge of CUDA or ROCm
• Proficiency in Python and PyTorch
• Understanding of transformers and LLM production behaviour
• Experience with distributed compute and parallelism

Benefits

• Medical coverage
• Dental coverage
• Vision coverage
• Flexible paid time off
• Parental leave
• Retirement plan participation