Engineering Manager- Model Performance

Details

• Baseten Embeddings Inference: The fastest embeddings solution available
• The Baseten Inference Stack
• Driving model performance optimization
• Lead, mentor, and manage a team of engineers focused on developing and optimizing ML model inference and performance.
• Oversee technical strategy and architecture decisions, driving improvements across our engineering organization.
• Collaborate with cross-functional teams to ensure seamless integration and scalability of ML models in production environments.
• Dive into the codebase of frameworks like TensorRT, PyTorch, CUDA, and others to identify and solve complex performance bottlenecks.
• Drive the development and deployment of large-scale optimization techniques for various ML models, especially large language models (LLMs).
• Own the full lifecycle of projects from inception through delivery, including planning, execution, and resource management.
• Foster a collaborative, inclusive team environment that encourages continuous learning and growth.
• Bachelor’s, Master’s, or Ph.D. in Computer Science, Engineering, or a related field.
• 5+ years of professional experience in software engineering, with at least 2 years in a technical leadership role.
• Proven experience managing and mentoring teams of engineers.
• Expertise in one or more programming languages, such as Python, C++, or Go.
• In-depth understanding of ML model performance optimization, especially using libraries such as PyTorch, TensorRT, and CUDA.
• Strong knowledge of containerization (Docker) and orchestration systems (Kubernetes).
• Experience with production-level AI/ML solutions, including scaling and deploying large models.
• Ability to balance hands-on technical work with team leadership and project management.
• Experience enhancing the performance of large language models (LLMs) or similar AI systems.
• Familiarity with LLM optimization techniques such as quantization, speculative decoding, or continuous batching.
• Deep knowledge of GPU architecture and performance tuning.
• Previous experience in a high-growth startup environment.
• Competitive compensation, including meaningful equity.
• 100% coverage of medical, dental, and vision insurance for employee and dependents
• Flexible PTO policy including company wide Winter Break (our offices are closed from Christmas Eve to New Year's Day!)
• Paid parental leave
• Fertility and family-building stipend through Carrot
• Company-facilitated 401(k)
• Exposure to a variety of ML startups, offering unparalleled learning and networking opportunities.