Senior Data Platform Engineer

Details

• You have a passion for designing, implementing, and operating stable, scalable, and efficient data systems
• Deep experience with streaming architectures and event-driven data pipelines
• Strong focus on schema governance, data quality, and data lifecycle management
• Proven track record of architecting, building, deploying, and supporting scalable production systems across 7+ years of experience
• Enjoys leading through individual technical contribution while mentoring and supporting the growth of other data platform engineers
• Strong commitment to software engineering and data infrastructure best practices
• Curiosity and interest in emerging AI technologies and applications
• Prefer BS, MS, or Ph. D. in Computer Science with 7+ years of proven relevant industry experience
• 5+ years experience working on Cloud Platforms (AWS primarily or GCP) and open-source technologies supporting data infrastructure, platforms & tools
• 5+ years experience with Big Data query engines & data stores such as Databricks, Snowflake, Redshift, etc.
• Great communication skills capable of working cross functionally with different teams in Engineering and prioritizing critical needs for Data Platform
• Experience programming, building and deploying products that rely on large-scale distributed systems in Python, Go, Scala/Java, or other similar languages
• 5+ years experience with large-scale, distributed systems design and development with an understanding of scaling, replication, consistency, and high availability
• 5+ years experience with Infrastructure as Code (IaC) tools like Terraform or Ansible.
• Experience with compute layer like EC2, ECS, EKS or similar
• Implementing secure systems/datastores with least privilege access (e.g. IAM) and encryption
• Experience using streaming technologies like Apache Kafka, Kinesis, Flink, etc.
• Implementing quality monitoring/alerting using tools like Prometheus, Grafana or Datadog
• Experience with Dagster or other orchestration tooling
• Experience with data modelling tooling such as dbt
• Design, implement, and maintain a streaming-first data platform built on Apache Kafka, Protobuf, and Apache Iceberg, enabling near real-time data availability across the organization
• Build and operate a hybrid Databricks and Snowflake architecture: leveraging Databricks for data engineering, streaming processing, and AI/ML workloads, and Snowflake as the governed serving layer for analytics and business reporting
• Own the unified storage layer to ensure all consumers operate on a consistent and governed data foundation
• Champion platform standards, tooling, and best practices across the data organization