LLMOps Platform Engineer

Details

• Build and operate the LLM Platform:
• Develop model routing, prompt registry, and orchestration services for multi-model workflows.Integrate external LLM APIs (OpenAI, Anthropic, Mistral) and internal finetuned models.
• Develop model routing, prompt registry, and orchestration services for multi-model workflows.
• Integrate external LLM APIs (OpenAI, Anthropic, Mistral) and internal finetuned models.
• Enable fast, safe experimentation:
• Implement automated evaluation pipelines (offline + online) with golden sets, rubrics, and regression detection.Support CI/CD for prompt and model changes, with rollback and approval gates.
• Implement automated evaluation pipelines (offline + online) with golden sets, rubrics, and regression detection.
• Support CI/CD for prompt and model changes, with rollback and approval gates.
• Collaborate cross-functionally:
• Partner with product pods to instrument RAG pipelines and prompt versioning.Work with deep learning and data teams to integrate structured and unstructured retrieval into LLM workflows.
• Partner with product pods to instrument RAG pipelines and prompt versioning.
• Work with deep learning and data teams to integrate structured and unstructured retrieval into LLM workflows.
• Optimize performance and cost:
• Profile latency, token usage, and caching strategies.Build observability and monitoring for LLM calls, embeddings, and agent behaviors.
• Profile latency, token usage, and caching strategies.
• Build observability and monitoring for LLM calls, embeddings, and agent behaviors.
• Ensure reliability and safety:
• Implement guardrails (toxicity, PII filters, jailbreak detection).Maintain policy enforcement and audit logging for AI usage.
• Implement guardrails (toxicity, PII filters, jailbreak detection).
• Maintain policy enforcement and audit logging for AI usage.
• Develop model routing, prompt registry, and orchestration services for multi-model workflows.
• Integrate external LLM APIs (OpenAI, Anthropic, Mistral) and internal finetuned models.
• Implement automated evaluation pipelines (offline + online) with golden sets, rubrics, and regression detection.
• Support CI/CD for prompt and model changes, with rollback and approval gates.
• Partner with product pods to instrument RAG pipelines and prompt versioning.
• Work with deep learning and data teams to integrate structured and unstructured retrieval into LLM workflows.
• Profile latency, token usage, and caching strategies.
• Build observability and monitoring for LLM calls, embeddings, and agent behaviors.
• Implement guardrails (toxicity, PII filters, jailbreak detection).
• Maintain policy enforcement and audit logging for AI usage.
• 5+ years of experience in applied ML, NLP, or ML infrastructure engineering.
• Strong coding skills in Python and experience with frameworks like LangChain, LlamaIndex, or Haystack.
• Solid understanding of retrieval-augmented generation (RAG), embeddings, vector databases, and evaluation methodologies.
• Experience deploying models or AI systems in production environments (AWS, GCP, or Azure).
• Familiarity with prompt management, LLM observability, and CI/CD automation for AI workflows.
• Experience with model serving (vLLM, Triton, Ray Serve, KServe).
• Understanding of LLM evaluation frameworks (OpenAI Evals, Promptfoo, Arize Phoenix, TruLens).
• Background in sports analytics, data engineering, or multimodal (video/text) systems.
• Exposure to Responsible AI practices (guardrails, safety evals, fairness testing).
• Competitive Salary and Bonus Plan
• Comprehensive health insurance plan
• Retirement savings plan (401k) with company match
• Remote working environment
• A flexible, unlimited time off policy
• Generous paid holiday schedule - 13 in total including Monday after the Super Bowl