Project Overview

Join a growing community of professionals advancing the next wave of AI. As an AI Trainer, you’ll play a hands-on role by analyzing and providing feedback on data to improve LLM performance, helping ensure that the next generation of AI technology is accurate and trustworthy.

We are seeking a skilled Healthcare Specialist to work as a project consultant in our AI Labor Marketplace. This is not a full-time employment position — you will be engaged as an expert project consultant on a contract basis.

Location: U.S.-based experts only

Engagement: Part-time, project-based expert evaluation work

Work Type: Remote

Project Summary

This project supports the development of an AI benchmarking framework designed to measure how effectively artificial intelligence models perform real-world, economically valuable knowledge work. The benchmark evaluates AI systems against tasks commonly performed by healthcare professionals across clinical, operational, administrative, quality, regulatory, and patient-care environments.

Contributors will create expert benchmark outputs, define evaluation criteria, and assess AI-generated work products across a wide range of healthcare scenarios. Example tasks may include clinical documentation review, care coordination planning, healthcare operations analysis, quality improvement initiatives, utilization review, healthcare policy interpretation, patient communication evaluation, healthcare compliance assessment, population health planning, and healthcare decision-support workflows. The goal is to establish rigorous, expert-driven standards for measuring AI performance on practical healthcare-related professional work.

Consultant Engagement Terms

This is a project-based consultant role. Consultants will be paid on a per-project basis; hourly rates are estimates based on anticipated completion time. Consultants control their own schedule, provide their own tools, and may simultaneously provide services to other vendors/employers (subject to those vendors’ allowances).

Responsibilities

Contributors will:

• Create expert benchmark responses for healthcare-related professional scenarios
• Develop scoring rubrics that define high-quality healthcare reasoning, documentation, communication, and decision-making standards
• Evaluate AI-generated analyses, recommendations, care-related content, and operational deliverables against established benchmarks
• Assess the accuracy, completeness, clinical appropriateness, operational practicality, and professional quality of AI-generated outputs
• Review AI-generated work involving patient care workflows, healthcare administration, quality improvement, care coordination, utilization management, healthcare compliance, and clinical documentation
• Identify factual inaccuracies, clinical errors, unsupported recommendations, workflow issues, patient-safety concerns, and material omissions
• Apply professional judgment to determine whether outputs meet real-world healthcare standards and best practices
• Provide written rationale supporting evaluation decisions and scoring outcomes
• Participate in calibration exercises and quality review activities when required

Expected Outcomes

• High-quality benchmark responses reflecting current healthcare practices and professional standards
• Clear, defensible evaluation rubrics aligned with real-world healthcare workflows
• Consistent and reliable assessment of AI-generated healthcare-related work products
• Identification of strengths, weaknesses, risks, and performance gaps in model outputs
• Expert evaluations that support rigorous benchmarking of AI systems on economically valuable healthcare work
• Documentation that enables repeatable and scalable model evaluation

Qualifications

• Minimum 5 years of professional experience in one or more of the following areas:
• Medicine
• Nursing
• Advanced Practice Nursing
• Physician Assistant Practice
• Rehabilitation Therapy
• Healthcare Administration
• Care Management
• Population Health
• Clinical Operations
• Utilization Management
• Healthcare Quality and Patient Safety
• Health Information Management
• Public Health
• Healthcare Compliance
• Case Management
• Active healthcare license, certification, registration, or credential preferred where applicable
• Strong understanding of healthcare workflows, patient care processes, healthcare regulations, quality standards, and professional practice requirements
• Experience reviewing, creating, or evaluating clinical documentation, care plans, operational processes, healthcare policies, quality initiatives, or patient-facing communications
• Ability to analyze complex healthcare scenarios and articulate professional reasoning clearly in writing
• Strong attention to detail and ability to identify clinical inaccuracies, patient-safety concerns, workflow issues, and compliance risks
• Experience working in hospitals, health systems, clinics, payer organizations, rehabilitation settings, public health organizations, or healthcare consulting environments preferred
• Relevant credentials are preferred where applicable (e.g., MD, DO, RN, NP, PA-C, PT, OT, SLP, RHIA, CPHQ, CCM, CHC, FACHE, or equivalent)
• U.S.-based