Senior Data Engineer (Equity- Only)

Details

• Design, optimize, and maintain tables and data structures to support recommendation and trending content data.
• Work with structured data storage solutions, including PostgreSQL and JSONB, to manage recommendation and interaction data.
• Implement and refine recommendation algorithms (e.g., collaborative filtering, content-based, and hybrid approaches) to enhance relevancy.
• Use similarity search libraries like Annoy or Faiss to optimize recommendation speed and accuracy.
• Continuously evaluate recommendation logic to better serve user preferences, ensuring real-time delivery.
• Aggregate, analyze, and process user interaction data to support recommendations and trending content.
• Design efficient queries and implement aggregation methods to capture relevant data and insights for recommendations.
• Identify, troubleshoot, and resolve data handling issues to ensure accurate recommendation delivery.
• Optimise queries, processing workflows, and containerised services for high performance and scalability within AWS ECS.
• Proven experience as a Data Engineer or Backend Engineer, with a focus on recommendation systems.
• Proficient in SQL and database management, especially with PostgreSQL and JSONB for structured data handling.
• Solid understanding of recommendation algorithms (collaborative filtering, content-based, hybrid approaches).
• Experience with similarity search libraries such as Annoy or Faiss.
• Strong programming skills in Python, with experience in building backend logic for data-intensive applications in a containerised environment.
• Familiarity with AWS ECS for container management, including task scheduling and scaling.
• Experience using AWS EventBridge to trigger workflows or automate tasks in response to application events.
• Analytical skills for data aggregation, querying, and insights generation.
• Strong debugging and optimisation skills for handling large-scale data processing in cloud-based environments.
• Knowledge of data aggregation pipelines, ETL processes, and data handling at scale.
• Familiarity with additional AWS services (e.g., S3, Lambda) for data storage and event-driven architectures.
• Experience with machine learning libraries or tools used in recommendation systems.