Remote | Machine Learning & NLP Research Evaluator — $75–$105/hour

  • New York, New York, United States
  • Contractor
  • Remote

Job Description:

We are sharing a specialised part-time consulting opportunity for machine learning and natural language processing professionals experienced in model development, language understanding, retrieval systems, transformer architectures, Python workflows, and advanced AI evaluation.

This role supports a remote collaboration focused on improving how advanced AI systems reason across machine learning and natural language processing tasks. Selected professionals will apply their technical expertise to design realistic ML and NLP problems, develop rigorous reference solutions, evaluate model behavior, identify capability gaps, and contribute to high-quality technical assessment workflows.

Key Responsibilities

Machine Learning & NLP Task Development

  • Design challenging, real-world machine learning and NLP problems based on applied industry, research, or open-source experience
  • Develop tasks involving model training, language understanding, text generation, retrieval, classification, ranking, and applied ML pipelines
  • Create problems that target specific technical weaknesses in advanced AI systems
  • Calibrate task difficulty, expected behavior, and evaluation criteria against modern ML and NLP standards

Reference Solution & Evaluation Environment Creation

  • Develop clear specifications, reference solutions, and supporting technical materials
  • Prepare executable tests, validation logic, and evaluation components using Python where applicable
  • Integrate tasks into structured development and assessment environments
  • Review reference implementations for correctness, reproducibility, and technical quality

Model Performance Evaluation

  • Evaluate model outputs across machine learning, natural language processing, retrieval, generation, and agentic workflows
  • Assess outputs for correctness, robustness, consistency, and alignment with task requirements
  • Identify failures involving reasoning, implementation, data handling, model behavior, or language interpretation
  • Document evaluation decisions clearly and support findings with technical evidence

Capability Gap Analysis

  • Identify tasks where meaningful model-performance headroom remains
  • Classify technical failures according to their underlying cause, severity, and impact
  • Distinguish between reasoning errors, modeling limitations, implementation issues, and evaluation constraints
  • Collaborate with other subject-matter experts to maintain consistent and accurate assessment standards

Ideal Profile

Strong candidates may have:

  • Deep hands-on experience in machine learning or natural language processing through industry work, academic research, graduate study, or open-source contributions
  • Practical proficiency in Python used in research, production, or technical development environments
  • Strong understanding of modern ML and NLP methods, including model training, evaluation, and transformer-based systems
  • Experience with large language models, retrieval systems, language generation, or applied machine learning pipelines
  • Ability to design technically rigorous tasks and analyze model behavior independently
  • Strong written communication and time-management skills
  • Availability to contribute approximately 20 hours per week
  • Current location in the United States

Educational Background

  • A degree in computer science, machine learning, artificial intelligence, computational linguistics, data science, mathematics, or a related technical field is helpful
  • Graduate-level research or doctoral training in machine learning, NLP, artificial intelligence, or a related discipline is highly relevant
  • Professional experience developing or evaluating production ML or NLP systems is also highly valuable
  • Equivalent hands-on experience through established open-source, research, or applied technical contributions may be considered

Nice to Have

  • Experience with PyTorch, TensorFlow, JAX, Hugging Face, or comparable machine learning frameworks
  • Familiarity with transformers, large language models, embeddings, retrieval-augmented generation, or sequence modeling
  • Experience building datasets, benchmarks, evaluation harnesses, or reproducible ML pipelines
  • Knowledge of standard machine learning and NLP metrics, error-analysis methods, and validation practices
  • Experience with information retrieval, text classification, semantic search, summarization, or language generation
  • Previous involvement in AI training, model evaluation, technical benchmarking, or structured data review
  • Open-source contributions related to machine learning, NLP, evaluation tooling, or model infrastructure

Why This Opportunity

  • Apply advanced machine learning and NLP expertise to structured remote work
  • Contribute to the development of stronger language reasoning and model evaluation systems
  • Design realistic tasks aligned with your technical specialization and practical experience
  • Use your ability to identify subtle model failures and assess complex technical outputs
  • Participate in flexible, high-impact technical work with competitive hourly compensation

Contract Details

  • Part-time W-2 contingent employment arrangement
  • Fully remote within the United States
  • Expected commitment of approximately 20 hours per week
  • Competitive rates between $75–$105 per hour depending on experience, technical depth, and project scope
  • Role-based engagement involving collaboration with technical experts and evaluation teams
  • Work schedules and engagement details may be adjusted depending on scope, performance, and business needs
  • Work will not involve access to confidential or proprietary information from any employer, client, or institution

About the Platform

This opportunity is available through 24-MAG LLC. We connect experienced professionals with remote consulting opportunities across technical, evaluation, and project-based workstreams.

By submitting this application, you acknowledge that your information may be processed by 24-MAG LLC for recruitment and opportunity matching in accordance with our Privacy Policy: https://www.24-mag.com/privacy-policy.