Resumo da vaga

Senior Data Engineer (Modern Data Platform & AI) (all genders)| Berlin, hybrid

Requisitos e responsabilidades

Conteúdo da vaga extraído em seções para revisão mais rápida.

  • Data Modeling & Transformation:Design necessary data models and transformations to curate raw data. Develop, optimize and maintain existing data models, pipelines, and transformations to support analytics, reporting, and AI use cases such as but not limited to curating, transforming, annotating and modeling data.
  • Design necessary data models and transformations to curate raw data.
  • Develop, optimize and maintain existing data models, pipelines, and transformations to support analytics, reporting, and AI use cases such as but not limited to curating, transforming, annotating and modeling data.
  • Data Platform Architecture:Architect and contribute in implementing a scalable, modern data platform, including data lakehouse or warehouse, to support real-time/near-real-time data flows from Kafka to downstream consumers. Optimize ETL/ELT pipelines using tools like DBT, Spark, or Airflow, bridging upstream (e.g. Debezium, MSK) and downstream processes. Evaluate and integrate new technologies to support hybrid monolith-microservices architecture and ML and AI enablement. Ensure seamless migrations and minimal disruptions during platform evolution.
  • Architect and contribute in implementing a scalable, modern data platform, including data lakehouse or warehouse, to support real-time/near-real-time data flows from Kafka to downstream consumers.
  • Optimize ETL/ELT pipelines using tools like DBT, Spark, or Airflow, bridging upstream (e.g. Debezium, MSK) and downstream processes.
  • Evaluate and integrate new technologies to support hybrid monolith-microservices architecture and ML and AI enablement.
  • Ensure seamless migrations and minimal disruptions during platform evolution.
  • Real-Time Data Integration: Build and optimize real-time data pipelines using Kafka, Spark, and Delta Live Tables.
  • Data Governance & Quality: Support the team lead in establishing and enforcing data governance frameworks, including data lineage, quality standards, catalogue, metadata management, SSOT for business glossaries/CBC terms, and policies to ensure reliable reporting. Ensure the existence of, or adaptation to, full Data Life Cycle Management (DLCM) and end-to-end testing.
  • Support the team lead in establishing and enforcing data governance frameworks, including data lineage, quality standards, catalogue, metadata management, SSOT for business glossaries/CBC terms, and policies to ensure reliable reporting.
  • Ensure the existence of, or adaptation to, full Data Life Cycle Management (DLCM) and end-to-end testing.
  • AI/ML Enablement: Collaborate with the team to integrate AI/ML capabilities, such as feature engineering and model serving, to accelerate data products for market penetration and operational efficiency, as well as operationalizing ML models and integrate AI into business processes.
  • Knowledge Sharing: Mentor the team on best practices, modern tools (e.g., Databricks, Snowflake, AI adaptation and integrations like Cursor/CodeRabbit), and cloud-native scalability. And last but not least foster a culture of innovation and continuous improvement.
  • Stakeholder Collaboration: Collaborate with Product Analytics, domain teams, and business to deliver data solutions that drive value and are aligned with business needs.

Details

  • Design necessary data models and transformations to curate raw data.
  • Develop, optimize and maintain existing data models, pipelines, and transformations to support analytics, reporting, and AI use cases such as but not limited to curating, transforming, annotating and modeling data.
  • Architect and contribute in implementing a scalable, modern data platform, including data lakehouse or warehouse, to support real-time/near-real-time data flows from Kafka to downstream consumers.
  • Optimize ETL/ELT pipelines using tools like DBT, Spark, or Airflow, bridging upstream (e.g. Debezium, MSK) and downstream processes.
  • Evaluate and integrate new technologies to support hybrid monolith-microservices architecture and ML and AI enablement.
  • Ensure seamless migrations and minimal disruptions during platform evolution.
  • Support the team lead in establishing and enforcing data governance frameworks, including data lineage, quality standards, catalogue, metadata management, SSOT for business glossaries/CBC terms, and policies to ensure reliable reporting.
  • Ensure the existence of, or adaptation to, full Data Life Cycle Management (DLCM) and end-to-end testing.

Skills

  • Master's degree in Computer Science, Data Engineering, or related field (or equivalent experience)
  • 10+ years of experience in data engineering, with 5+ years in senior roles focused on modern architectures.
  • Excellent communication and collaboration skills, the ability to drive change and influence stakeholders, and a passion for mentoring, coaching, and sharing knowledge
  • Proven expertise in designing, developing & maintaining data lakehouses/DWH (e.g., Databricks Delta Lake, Snowflake) and transformations (e.g., DBT, SQL/Python/Spark).
  • Strong experience with cloud platforms such as AWS services (S3, Athena, MSK/Kafka, Terraform) and real-time streaming (e.g., Kafka, Spark Structured Streaming, Flink).
  • Hands-on knowledge of data governance tools (e.g., Unity Catalog, Collibra) for lineage, quality, catalogs, and SSOT.
  • Familiarity in AI/ML pipelines and MLOps (e.g., MLflow, feature stores) and complex system integration within modern data technologies.
  • Proficiency in CI/CD for data, and tools like Git, Airflow, or dbt Cloud.
  • Experience with large-scale data modeling (DataVault, dimensional, schema-on-read) and optimizing for self-service analytics.
Vagas similares

Mantenha uma lista reserva.

Ver stack
FocoData EngineeringÁrea da vaga
Sinal de senioridadeSeniorNível do candidato
StackAWS, CI/CD, PythonSkills principais
Localização37 países aceitosElegibilidade

Stack

Use estas tags para comparar vagas remotas similares.

Elegibilidade de localização

Candidatos devem aplicar apenas quando o país do perfil estiver listado aqui.

Fluxo de contratação

O WithMira mostra a vaga e depois envia candidatos para a aplicação da empresa.

1Confira fit da vaga, stack e elegibilidade de localização no WithMira.
2Abra a página de aplicação da empresa pelo link rastreado.
3Salve a vaga ou assine oportunidades similares antes de sair.
Aplicar no site da empresaSite da empresaAbrir link