Lead Data Engineer- Databricks

Requisitos e responsabilidades

Conteúdo da vaga extraído em seções para revisão mais rápida.

Details

Serve as the technical lead for the project—owning solution design decisions, guiding implementation standards, and mentoring other engineers through coaching, reviews, and knowledge sharing.
Design, develop, and maintain complex ETL pipelines using Databricks, ensuring scalable, high-performance data integration across multiple source systems.
Implement and optimize medallion architecture within Databricks, establishing clear data zones (raw, curated, trusted) to support governed, enterprise-wide reporting.
Develop and refine dimensional data models that enable unified, analytics-ready views of business domains and support automated dashboarding and KPI frameworks.
Collaborate closely with cross-functional teams (data stewards, IT, business stakeholders) to translate operational requirements into technical solutions, proactively clarifying dependencies and driving alignment.
Contribute to architectural decisions, leveraging your expertise to recommend best practices, challenge assumptions, and ensure data platform durability and flexibility.
Identify and address integration challenges, data quality issues, and process bottlenecks early, providing actionable insights and thoughtfully pushing back when project risks or inefficiencies arise.
Support knowledge transfer and documentation, empowering colleagues and clients to maintain and evolve data solutions independently.

Requirements

Databricks Expertise: 3–5 years of hands-on experience developing in Databricks, with relevant certifications (attach proof). Proven ability to build, optimize, and troubleshoot production-grade ETL pipelines.
Medallion Architecture: Demonstrated experience designing and implementing medallion data models (raw, curated, trusted zones) in Databricks environments.
Dimensional Modeling: Strong skills in developing dimensional models to support business intelligence, reporting, and single source of truth requirements.
Complex ETL Development: Advanced proficiency in orchestrating data ingestion, transformation, and integration workflows across multiple systems and formats.
Business Acumen: Ability to understand operational requirements, translate them into technical deliverables, and communicate the business impact of engineering decisions.
Self-Starter: Demonstrated capacity to take initiative, work independently, and drive projects forward in ambiguous or evolving contexts.
Azure Ecosystem (Preferred): Experience with Azure Data Lake, Data Factory, and related services is a strong advantage but not required.
Data Visualization Design (Preferred): Experience with human centric design principles with data visualization tools such as Power BI, Qlik, or Tableau.
Clear Communication: Proactive, constructive communication skills with the confidence to provide feedback, challenge assumptions, and advocate for best practices.
Embrace OZ Digital’s values in daily work: adapting and learning, focusing on results, working collaboratively, maintaining a positive outlook, and leading with empathy.

Vagas similares