Data Engineer GCP
Posted Updated
Description
Data Engineer
We’re looking for an experienced Data Engineer to build, scale, optimize, and maintain reliable data platforms. You’ll work on real-time pipelines, ML infrastructure, attribution data processing, and cloud-native solutions in a high-volume production environment.
Tech Stack
- GCP
- Apache Spark, dbt, BigQuery
- Python, SQL, Scala
- Apache Airflow
- Terraform, GitHub Actions, Docker, Kubernetes
- VertexAI
Must Have
- Over 3 years of experience in Data Engineering and building production-scale data platforms.
- Strong programming skills in Python and SQL.
- Advanced experience with data modeling, ETL development, and multiple data formats.
- Strong knowledge of Google Cloud Platform and cloud-native architecture design.
- Expert knowledge of Apache Airflow for workflow orchestration and automation.
- Hands-on experience with Apache Spark for batch and high-volume streaming workloads.
- Good understanding of CI/CD and DevOps tools such as GitHub Actions or Kubernetes.
Nice to Have
- Experience with MLOps and ML deployment in production, preferably with Vertex AI.
- Practical experience with Terraform for Infrastructure as Code.
- Experience building scalable REST APIs for data or ML services.
- Strong testing practices, including TDD and unit/integration tests for Spark and Airflow.
- Knowledge of Scala for high-performance data processing.