AI Research Lead - Evaluation

AI Research Lead – Evaluation page is loaded## AI Research Lead – Evaluationlocations: Londontime type: Full timeposted on: Posted Todayjob requisition id: R-02694**Department:** Technology**Team:** AI**Location:** London with hybrid ways of working**Position type:** Short Term Contract (Inside IR35) until end of Dec 2026, with potential to extend**What the hiring manager says***\”As the AI Research Lead, you will be at the forefront of developing and maintaining the GSMA’s Open Telco AI benchmarks and working with members on evaluating low-resource language models. This role is critical to ensuring our evaluation pipelines are robust and transparent, directly impacting the quality and reliability of our Members AI solutions. You’ll have the opportunity to collaborate with leading telecom operators and frontier AI ecosystem partners, making a tangible impact on industry best practice and innovation.\”***About the Team**You’ll join a dynamic, cross-functional team dedicated to advancing AI capabilities in the telecom sector. Our team is growing rapidly, with a culture of collaboration and technical excellence. We value curiosity, initiative, and a drive to set new benchmarks for the industry.**About the role** You will own and maintain the evaluation pipeline for open telco benchmarks, designing and implementing new benchmarks in collaboration with telecom operator members and AI partners. You’ll lead the integration and benchmarking of new models, including member-submitted LLMs, and guide the expansion of benchmarking to include AI agents and diverse architectures. You’ll also provide technical support for members building low-resource language LLM initiatives. Success in this role means delivering robust, transparent benchmarking processes and supporting the community in adopting best practices.**About You**You are passionate about machine learning model evaluation and have demonstrable experience with open-source evaluation frameworks (such as HELM or lm-eval-harness). You thrive in collaborative environments, working with technical partners and cross-functional stakeholders. Your experience developing or evaluating local language LLMs gives you a unique perspective on linguistic, cultural, and resource constraints. You are adept at managing versioned pipelines and benchmarking reports, and you bring an understanding of telco or enterprise AI.**About your skills** You’ll possess :* Strong experience in machine learning model evaluation and benchmarks.* Familiarity with open-source evaluation frameworks (e.g., HELM, lm-eval-harness).* Experience collaborating with technical partners and cross-functional stakeholders.* Demonstrated ability to manage versioned pipelines and benchmarking reports.* Understanding of telco or enterprise AI use cases (a strong plus).* Proven experience developing or evaluating Local Language LLMs, with an understanding of unique linguistic, cultural, and resource constraints (a strong plus)* Communication, Analysis, Project Management, Innovation, Stakeholder Management.We strive to offer a meaningful and inclusive application experience for all candidates. Should you require any accommodations or adjustments due to a disability or for any other reason during the hiring process, please contact talent@gsma.com with your request.**Contract type**Short term Contractor**Worker type**Contingent Worker## **What We Offer**Working at the GSMA offers you unparalleled access to the mobile industry. We offer a chance to truly shape the direction of mobile, whatever your role. By joining the GSMA, you will be exposed to a fast-paced rapidly evolving environment, working on global solutions, genuinely fascinating and industry-changing projects and a stimulating and dynamic environment designed to enable you to flourish.In addition to architect-designed offices and competitive compensation, our benefits include fantastic learning & development opportunities, generous holiday allowances, four additional days off for professional development and many others.To learn more about the GSMA, visit our , our page and our page.Being You at the GSMAWe care deeply about diversity, equity and inclusivity and aspire to be the best at it. Your well-being and work/life balance is important, so flexi-time and remote working is available to all staff. We\’re keen to ensure everyone is equal, represented and connected so we particularly encourage applications from all demographics. The sucess of the GSMA year on year will continue to be contributed by people from all walks of life.**GSMA Values**Our values not only drive our culture – they shape how we work and interact inside and outside our global organisation.**Passionately driven**We approach everything we do with unparalleled capability, tenacity and commitment, knowing that the challenging scale, pace and complexity of our work is what leads to its world-changing impact.**Insightful leaders**We continually develop and engage our expertise, insight and creativity so that we’re always ready to respond to the changing landscape with authority, agility and nuance.**Stronger together**We lean on each other so the industry can lean on us, embracing our diversity by actively seeking out perspectives and skill sets beyond our own, fuelling each other’s successes and constantly asking how we can help.Underpinning our values is our collective mindset to show up purposefully as good human beings every day, in every situation. When we’re at our best – we are collaborative, considerate and compassionate to others, and we create a safe space for one another to thrive, assuming positive intent in our colleagues. And if we aren’t at our best and the pressure is on – we feel free to be ourselves but still remain curious, lean into the tough stuff and we are always respectful to others and accountable for the part we play.### #J-18808-Ljbffr