Project Overview:

Join a growing community of professionals advancing the next wave of AI. As an AI Trainer, you’ll play a hands-on role by analyzing and providing feedback on data to improve LLM performance, helping ensure that the next generation of AI technology is accurate and trustworthy.

We are seeking a skilled software engineer to work as a project consultant in our AI Labor Marketplace. This is not a full-time employment position — you will be engaged as an expert project consultant on a contract basis.

Location: U.S.-based experts only

Engagement: Part-time, project-based expert evaluation work

Work Type: Remote

Project Summary:

Contributors will design and evaluate realistic software engineering tasks, including bug resolution, feature implementation, refactoring/migration, and test generation. Work includes both creating complex coding scenarios and reviewing peer submissions for quality and accuracy.

Consultant Engagement Terms:

This is a project-based consultant role. Consultants will be paid on a per-project basis; hourly rates are estimates based on anticipated completion time. Consultants control their own schedule, provide their own tools, and may simultaneously provide services to other vendors/employers (subject to those vendors’ allowances).

Responsibilities:

Contributors will:

• Design and implement multi-file coding tasks across bug fixing, feature development, refactoring, and testing
• Write clear natural-language specifications and reference implementations
• Develop and extend unit and integration test suites
• Review peer-generated tasks for correctness, clarity, and realism
• Identify edge cases, ambiguities, and potential failure modes
• Ensure alignment between specifications, code, and expected outputs

Expected Outcomes:

• High-quality, production-realistic coding tasks
• Complete and correct reference implementations
• Robust test coverage and validation artifacts
• Structured, actionable peer review feedback

Qualifications:

• Advanced professional written proficiency in English
• 5–7 years of professional software engineering experience
• Strong proficiency in Python and JavaScript/TypeScript; working knowledge of Java, C#, or Go
• Backend or full‑stack development experience in production systems
• Experience with testing frameworks (e.g., pytest, Jest, JUnit, xUnit, Go testing)
• Proven ability to debug and navigate large, multi‑file codebases
• Experience with code reviews, refactoring, and production migrations
• Prior experience with data annotation, evaluation, or structured labeling for AI/ML systems
• Experience defining solution trajectories and writing evaluation rubrics
• Familiarity with LLM evaluation, prompt design, or benchmarking workflows