Create rigorous mathematics coding problems, verified Python solutions, unit tests, and discriminative evaluation cases for frontier AI models. Perform quality control through the Central Task Platform, refine tasks based on feedback, and optimize datasets for LLM evaluation. The role requires advanced mathematics expertise, scientific programming, deterministic outputs, mathematical research or writing experience, and familiarity with AI-generated code assessment.
Gramian Consultancy is a boutique consultancy specializing in IT professional services and engineering talent solutions. With a strong background in software engineering and leadership, we help companies build high-performing teams by matching them with professionals who truly fit their needs.
Role Overview
We are seeking a mathematics-focused professional to support an advanced AI research initiative developing high-quality STEM coding datasets. In this role, you will create rigorous mathematical problems, implement verified Python solutions, and develop evaluation tests designed to assess the capabilities of frontier AI models. The position combines mathematical problem-solving, scientific programming, and AI quality evaluation, with a strong emphasis on accuracy, determinism, and first-submission quality.
CONTRACT: Contractor assignment, 4 weeks
COMMITMENT: Hourly, up to 40h/week
LOCATIONS: Remote - Bangladesh, Brazil, Colombia, Egypt, Ghana, India, Indonesia, Pakistan, Turkey, Vietnam
PROCESS: Interest Check Form, CV and Google Scholar review
Key Responsibilities
- Author mathematical coding tasks containing one main problem and at least 3 logically connected sub-problems that progressively build toward the main solution.
- Implement verified golden solutions in Python with complete unit test coverage.
- Design discriminative test cases that distinguish correct from incorrect AI-generated outputs.
- Perform quality control validation using the Central Task Platform (CTP), including Tier 1 structure checks and Tier 2 quality rubrics.
- Refine problem specifications, solutions, and test cases based on QC feedback.
- Optimize tasks to meet Pass@K evaluation criteria across multiple LLM judges, including GPT, Gemini, and Nemotron.
- Ensure mathematical correctness, well-posedness, determinism, and compliance with project quality standards.
- Maintain high first-submission quality and minimize rework, targeting consistent L1 approval.
- Participate in review sessions, feedback meetings, and project standups during required overlap hours.
Requirements
- Master's degree or PhD in Mathematics.
- Strong Python programming skills with experience in scientific computing.
- Experience using scientific Python libraries such as NumPy, SciPy, SymPy, or relevant domain-specific tools.
- Ability to formulate rigorous mathematical problems with clear constraints and expected outputs.
- Experience implementing scientific or mathematical solutions with unit tests and deterministic outputs.
- Prior experience in AI data annotation, scientific research, or scientific writing.
- Familiarity with LLM evaluation frameworks, coding benchmarks, or AI-generated code assessment.
- Published research or academic project experience in mathematics or another STEM discipline.
Similar Jobs
Cloud • Information Technology • Internet of Things • Machine Learning • Software • Cybersecurity • Infrastructure as a Service (IaaS)
Lead end-to-end OSS/BSS enterprise architecture for large, multi-country transformation programs across hybrid-cloud and on-premises environments. Define integration designs, governance standards, reusable architecture patterns, and strategic bid solutions. Engage senior customer and technology leaders, assess delivery risks, oversee architecture across active programs, and mentor architects. The role requires extensive telecommunications and IT architecture experience, strong integration expertise, TOGAF or equivalent certification, industry standardization experience, and comfort with global travel.
Top Skills:
Hybrid CloudOn-Premises EnvironmentsOss/BssTm ForumTogaf
Artificial Intelligence • Cloud • Information Technology • Consulting
Serves as the primary post-sales technical contact for customers, coordinating service delivery, escalations, SLA reviews, risk mitigation, network health checks, configuration and design reviews, upgrades, and deployment planning. Advises customers on network performance, reliability, security, and topology improvements while partnering with support, professional services, engineering, and account teams. Requires advanced IP/MPLS and data center networking expertise, customer-facing experience, fluent French, and the ability to work independently.
Top Skills:
BgpData Center SwitchingEvpnHpe JuniperIpsecIpv6ItilMpls VpnMulticastNatOspfQosSd-WanSegment RoutingStpTcp/IpTraffic EngineeringVlanVrrpVxlanWireless Networking
Big Data • Food • Hardware • Machine Learning • Retail • Automation • Manufacturing
Leads owned and earned digital ecosystem strategy across websites, social media, direct messaging, apps, influencer marketing, and content campaigns. Develops personalized content frameworks, manages agencies, enables GenAI and DAM adoption, and tracks performance through analytics, SEO, social listening, and consumer insights. Partners with Brand, CX, Media, Data, and Insights teams to optimize consumer journeys, content effectiveness, and brand relevance.
Top Skills:
Digital Asset Management (Dam)Generative AiMarketing Analytics ToolsSeoSocial Listening ToolsSocial Media PlatformsTagging And Tracking Tools
What you need to know about the Delhi Tech Scene
Delhi, India's capital city, is a place where tradition and progress co-exist. While Old Delhi is known for its rich history and bustling markets, New Delhi is defined by its modern architecture. It's clear the region places a strong emphasis on preserving its cultural heritage while embracing technological advancements, particularly in artificial intelligence, which plays a central role in shaping the city's tech landscape, fueled by investments in research and development.
