Member of Technical Staff Search Core Indexing and Data Pipeline Stream

You will design and build core search-engine components, including indexing pipelines, storage formats, retrieval algorithms, and ranking systems. You will develop streaming and batch data-processing systems for search-index construction. You will optimize hardware and systems performance, address multithreading and concurrency challenges, and help shape foundational architecture and implementation decisions.

Responsibilities

• Design and build core search-engine components
• Develop indexing pipelines and data storage formats
• Develop streaming and batch data-processing systems for search-index construction
• Optimize hardware performance through low-level systems tuning
• Address multithreading, concurrency, and system-level optimization challenges
• Build retrieval algorithms and ranking systems

Requirements

• 3 years of systems programming experience
• Rust
• C++
• C
• Python
• Data processing system
• YT
• Kafka
• Hadoop
• Algorithm
• Data structure
• Multithreading
• Distributed service
• Linux internals
• System call
• Networking
• Memory model
• Kernel tuning
• Memory management
• Cache efficiency
• SIMD
• Profiling