mercor
Software Engineer- Evaluation Author- AI Trainer
Remote Code Evaluation Author role with clear candidate location fit.
PostedJun 14, 2026
Eligible countries1 accepted country
Seniority signalMiddle
Work settingRemote
Accepted candidate locations
USA
Role overview
Software Engineer- Evaluation Author- AI Trainer
Requirements and responsibilities
Readable role content extracted into sections for faster review.
Role Responsibilities
- Author non-trivial coding tasks with golden solutions and automated verifiers.
- Design rubrics and grade agent trajectories and model outputs.
- Improve task and rubric quality through structured review.
- Evaluate the accuracy and depth of AI-generated content to strengthen reasoning and rigor in model outputs.
- Work independently and asynchronously to meet deadlines while improving AI model performance.
QualificationsMust-Have
- 5+ years of software engineering at a real product organization (big tech or venture-backed startup).
- Strong code quality, systems design, debugging, and testing discipline.
- Clear written communication (you write instructions others follow).
Preferred
- Familiarity with AI coding tools and evals.
Interview Process
- Short Mercor Technical Screen.
- Live Code Review Session.
- Domain Expert Interview.
- You're paid $200 for completing all three, regardless of outcome.
Application Process (Takes 20–30 mins to complete)
- Upload resume
- AI interview based on your resume
- Submit form
Resources & Support
- For details about the interview process and platform information, please check: https://talent.docs.mercor.com/welcome
- For any help or support, reach out to: support@mercor.com
Similar roles
Keep a backup shortlist.
Stack
Use these tags to compare similar remote roles.
Location eligibility
Candidates should apply only when their profile country is listed here.
Your profileCountry not setSign in to check your country against this role.
Hiring flow
WithMira shows the role, then sends candidates to the company application.
1Check role fit, stack, and location eligibility in WithMira.
2Open the company application page from the tracked apply link.
3Save the role or subscribe for similar opportunities before leaving.