NirYu
Data Engineer – Web Scraping, LLM Pipelines and Scalable Data Infrastructure
Remote Data Engineering role with clear candidate location fit.
PostedJul 5, 2026
Eligible countries1 accepted country
Seniority signalMiddle
Work settingRemote
Accepted candidate locations
USA
Role overview
Data Engineer – Web Scraping, LLM Pipelines and Scalable Data Infrastructure
Requirements and responsibilities
Readable role content extracted into sections for faster review.
Key Responsibilities:
- Build new structured datasets, including scraping accelerators, Form D filings and dynamic web sources.
- Develop automated ETL pipelines that parse, clean and transform content using LLMs.
- Define and maintain database schemas in Supabase or PostgreSQL.
- Create evaluation frameworks to measure and compare LLM performance across pipeline components.
- Contribute to the design of scalable data architectures using GCP services.
- Improve reliability, observability and deployment workflows for scraping and data processing systems.
Requirements:
- 4+ years of experience building data pipelines, backend services and automated data processing systems.
- Strong background in web scraping with tools like Scrapy, Playwright or similar.
- Experience deploying pipelines on cloud platforms such as GCP or AWS.
- Solid knowledge of ETL frameworks, workflow orchestration (Airflow) and modern data stores (BigQuery, PostgreSQL).
- Comfortable working with Docker and API frameworks like FastAPI.
- Clear, fluent communication in English.
Similar roles
Keep a backup shortlist.
Docker, PostgreSQL 5 accepted countries
Lead Full Stack EngineerKepler GroupView role AWS, PostgreSQL 13 accepted countries
Senior Backend Engineer (AdTech)Leap ToolsView role AWS, PostgreSQL 13 accepted countries
Senior Backend EngineerLeap ToolsView role Python USA
Senior Data EngineerTop Us Wealth Management FirmView role Stack
Use these tags to compare similar remote roles.
Location eligibility
Candidates should apply only when their profile country is listed here.
Your profileCountry not setSign in to check your country against this role.
Hiring flow
WithMira shows the role, then sends candidates to the company application.
1Check role fit, stack, and location eligibility in WithMira.
2Open the company application page from the tracked apply link.
3Save the role or subscribe for similar opportunities before leaving.