Software Engineer Data Infrastructure

You will design and build data pipelines that ingest, process, and transform user-vote data for model evaluation. You will improve pipeline reliability and data quality, scale infrastructure, and create dashboards and reports for evaluation insights.

Responsibilities

• Design and build data pipelines for user-vote data
• Ingest, process, and transform data into model-evaluation features
• Collaborate to identify necessary data
• Build result dashboards and reports
• Ensure pipeline integrity, data quality, and reliability
• Scale data infrastructure for growing volumes and analytical needs

Requirements

• 5+ years of software engineering experience focused on data engineering and big data technologies
• SQL proficiency
• Proficiency in Python, Scala, or R
• Experience with data processing and pipeline frameworks such as Apache Spark or Ray Data
• Experience with Databricks or Snowflake
• Experience designing, implementing, optimizing, and debugging production data pipelines

Benefits

• Equity
• Medical benefits
• Dental benefits
• Vision benefits
• Additional health and wellness support programs