Senior Data Engineer

Build and operate streaming and batch data pipelines for market, trading, and portfolio data. Design lakehouse and time-series layers, own data contracts and schema evolution, implement data quality, lineage, observability, and self-healing, and provide self-serve tooling for data products and AI agents.

Responsibilities

• Build resilient streaming and batch pipelines for market, trading, and portfolio data.
• Build self-serve SDKs, patterns, templates, and AI agents for data products.
• Own data contracts and schema evolution.
• Design lakehouse and time-series layers around consumer query patterns.
• Build data governance and quality frameworks with validation, lineage, ownership, and self-healing.
• Build derived analytics including spreads, VWAP, order book microstructure, portfolio views, exposure, and performance.
• Make observability, cost, and performance first-class.
• Treat infrastructure as code using Docker, Terraform, and CI/CD.
• Write documentation and partner with architecture, infrastructure, platform, and other teams.

Requirements

• 8+ years of building production data systems.
• Strong proficiency in Python and SQL.
• Strong understanding of data modelling for streaming and analytical workloads.
• Experience designing and operating Kafka, Redpanda, MSK, or Kinesis streaming systems.
• Production experience with ClickHouse, TimescaleDB, QuestDB, or similar time-series stores.
• Experience with lakehouse architecture, table layout, partitioning, and compaction.
• Experience building idempotent, self-healing systems with safe reprocessing.
• Experience with Docker, Terraform, and CI/CD.
• Experience instrumenting logs, metrics, and traces.
• Experience designing data quality, governance, contracts, validation, lineage, and ownership.
• Understanding of financial market data including order books, trades, reference data, portfolios, and exposures.
• Ability to design, ship, operate, and improve end-to-end data systems.
• Nice to have: Apache Iceberg or Delta Lake experience.
• Nice to have: DataHub or similar metadata and lineage platform familiarity.
• Nice to have: Rust familiarity.

Benefits

• Flexible hours
• Remote-first work
• Business-hours on-call shared across the team
• Regular online get-togethers
• Yearly onsite
• Autonomy on how you work
• Strong cross-functional partners
• Competitive salary package with various benefits