Senior Data Engineer| Apache NiFi

Conteúdo da vaga extraído em seções para revisão mais rápida.

Contractor: Between €260/day and €350/day
Permanent (Portugal/Spain): between €38,000 gross per year and€45,000 gross per year

Design, develop, and maintain data pipelines using Apache NiFi, ensuring efficient ingestion and transformation
Integrate structured and unstructured data from multiple sources, including Salesforce, APIs, SFTP, SAP, and CRM, into cloud-based and on-premise databases.
Ensure data quality, security, and governanceacross diverse sources and destinations.
Optimise and troubleshoot NiFi workflows to improve performance, reliability, and scalability
Work with APIs to fetch and send real-time data for external integrations (e.g., CustomerGauge).
Manage and operate NiFi architecture within Kubernetes environments
Collaborate with data scientists, analysts, and engineers to ensure data availability and usability
Improve and optimise ETL/ELT processes for large-scale data processing
Use NiFi Registry and Toolkit to maintain version control and streamline deployments across environments.
DevelopBash and Python scripts for automation, monitoring, and workflow execution.
ImplementCI/CD pipelines for data pipeline deployments and workflow automation.
Analyse and resolve data pipeline issues, ensuring high availability and reliability.

Strong expertise inApache NiFi, including flow design, performance tuning, version control, and security configurations.
Experience in data ingestion, transformation, and processing for both batchandreal-time workflows.
Proficiency inSQLandNoSQLdatabases, such as PostgreSQL, MySQL, and MongoDB.
Experience with big data technologies, including Apache Kafka, Spark, Hadoop, and HDFS.
Familiarity with cloud platforms (AWS,Azure, orGCP) and their data services (e.g., AWS Redshift, Azure Synapse, Google BigQuery).
Proficiency in at least one programming language, such asPythonor Java.
Experience with CI/CD pipelines, version control tools (Git, Jenkins), and automation frameworks.
Strong problem-solving skills and ability to work independently in a remote environment
Excellent communication and stakeholder management skills
Fluency in English
Must be Europe-based (Portugal/Spain preferred for permanent option)

Experience with data warehousing solutions (Snowflake, Redshift, BigQuery)
Understanding ofcontainerizationand orchestration tools, including Dockerand Kubernetes.
Knowledge of data security and compliance best practices(e.g., GDPR, HIPAA)
Experience with data lakes and real-time analytics solutions
Background in consulting or client-facing roles

Bottom of the rangeYou meet the core requirements and will need ramp-up time and support.
Middle of the rangeYou are fully autonomous from Day 1 and deliver consistently.
Top of the rangeYou are a reference for the role, mentor others, and raise the bar for the team.

A market leader in Data Intelligence
A company where people, trust, and diversity are core values
An environment that values ownership, flexibility, and innovation
A place where different backgrounds and perspectives are not just welcomed = they are essential

Vagas similares