Machine Learning Engineer For Educational Assessment

Description

Accountable to the Head of AI for Assessment Innovation, the overall purpose of this role is to develop models and algorithms as required by new assessment products and services.

The post holder will research and develop AI capabilities that can enable new assessment products, increase the breadth of assessment services on offer and help shape long-term tech innovation and solutions.

They will ideate and develop proofs of concept and prototypes and ensure they are cutting-edge, relevant and fit-for-purpose.

Research, development and evaluation of AI solutions for assessment are key enablers in a range of diversification, digitisation and customer programmes.

The AI for Assessment Innovation team is AQA’s in-house AI for assessment lab, providing services and solutions alongside and in collaboration with contractors and partners.

The team’s responsibilities are

• Research and development of AI features for new products or as part of contracted services
• Ed Tech partnership support through targeted evaluations and testing of third-party AI tools
• Providing AI for assessment expertise to the whole AQA group and advancing AQA’s knowledge and know-how

The role sits within the AI for Assessment Innovations team, in the Assessment Research and Innovation business area.

Reporting to the Head of AI for Assessment, the role collaborates with a team of AI researchers, developers and managers and will have line management responsibility for AI for Assessment apprentices.

Activities

• AI model development for assessment Design, build, and refine machine learning models that support educational assessment use cases, such as automated marking (e. g., essays, short answers), feedback generation and learner support, skill estimation, proficiency modelling, and adaptive testing.
• Select appropriate modelling approaches (e. g., NLP models, classical ML, or deep learning) based on pedagogical and product requirements.
• Conduct rigorous experimentation, including hyperparameter tuning and ablation studies, to improve model performance and fairness.

Educational assessment research

• Work with complex educational datasets (e. g., learner responses, interaction logs, assessment outcomes).
• Design evaluation frameworks that go beyond accuracy to include fairness and bias across learner groups, marking reliability and consistency, alignment with human marking standards and mark schemes
• Work closely with psychometricians, assessment experts and product teams to translate educational requirements into technical solutions.
• Incorporate domain knowledge (e. g., marking schemes, assessment objectives, curriculum standards) into model design.
• From prototype to operationalisation
• Develop scalable pipelines for data processing, model training, validation, and deployment.
• Collaborate and support the teams responsible for integrating models into production systems
• Contribute to CI/CD workflows, model versioning, and reproducibility practices.
• Responsible AI and governance
• Identify, assess, and mitigate risks related to bias, fairness, and misuse in assessment AI systems.
• Contribute to the development of explainable and transparent AI systems suitable for high-stakes exams or classroom use.
• Work with the relevant AQA teams to ensure compliance with relevant regulatory and ethical standards in education.
• Documentation and knowledge sharing
• Document model architectures, decisions, evaluation results, and limitations.
• Communicate findings clearly to both technical and non-technical stakeholders.
• Contribute to internal best practices, reusable components, and knowledge sharing across teams.

To be successful in this role, you will need to demonstrate

Essential

• Motivation A keen interest in the education or educational assessment sector, and a drive to furthering AQA’s mission.
• Machine learning and NLP expertise Understanding of machine learning techniques, including supervised learning, model evaluation and optimisation
• Natural Language Processing (NLP) for text-based assessment and some knowledge of multi-modal models
• Experience building end-to-end ML systems from data ingestion to deployment.
• Familiarity with model interpretability techniques (e. g., SHAP, LIME).
• Engineering Proficient in Python and core ML/data libraries (e. g., Py Torch/Tensor Flow, Scikit-learn, Pandas).
• Knowledge or experience with production systems such as API development, containerisation and cloud platforms
• Solid understanding of software engineering practices: version control, testing, modular design.
• Research skills Experience working with real-world datasets, including noisy or incomplete data.
• Understanding of evaluation methodologies, particularly in contexts where ground truth may be subjective (e. g. human marking).
• Analytical and problem-solving skills Ability to translate ambiguous, domain-specific problems into structured ML solutions.
• Strong critical thinking when interpreting model outputs in high-stakes contexts.
• Attention to detail, particularly when working with sensitive learner data and evaluation outcomes.
• Communication and Collaboration Ability to work effectively in multidisciplinary teams.
• Strong communication skills, including explaining technical concepts to educators and non-technical stakeholders.
• Experience contributing to collaborative development environments.
• Education and Experience Bachelor’s or Master’s degree in Computer Science, Machine Learning, Data Science, or a related field, or equivalent experience
• Hands-on experience in machine learning or applied AI, acquired in a range of contexts

Desirable

• Interest in or experience with education technology, assessment systems, or learning analytics.
• Awareness of concepts relevant to assessment such as reliability and validity, computerised adaptive testing (CAT), Item response theory (IRT) or similar psychometric models
• Experience in education, assessment, or a related
• #J-18808-Ljbffr