Software Engineer Data Infrastructure

You will build and maintain scalable data infrastructure for high-throughput machine-learning workflows. You will develop and optimize large-scale data pipelines and batch jobs, improve platform reliability and performance, integrate open-source tools, and troubleshoot data systems.

Responsibilities

• Build and maintain scalable data infrastructure for machine-learning workflows
• Develop and optimize large-scale data pipelines and batch processing jobs
• Contribute to reliable, high-performance data platform architecture and implementation
• Integrate open-source tools and improve infrastructure through monitoring and tuning
• Improve data reliability, scalability, and operational excellence
• Evaluate and adopt programming languages and frameworks for data infrastructure
• Troubleshoot and tune data infrastructure performance
• Define and refine data infrastructure development practices

Requirements

• Proficiency in Python or similar languages and willingness to learn Python
• Experience with large-scale, high-throughput data infrastructure
• Familiarity with distributed computing frameworks such as Ray, Spark, or Beam
• Ability to design and optimize data pipelines for machine-learning research and internal teams
• Understanding of data engineering at scale
• Experience sourcing, integrating, and optimizing diverse large datasets

Benefits

• Equity