Lead Data & AI Engineer

We’re looking for a Lead Data & AI Engineer to join our team in the UK in a hybrid working mode. In this role, you will design and deliver advanced data solutions and AI-assisted capabilities on modern cloud platforms. You’ll build scalable data pipelines, optimize data quality and governance and enable ML feature engineering for analytics and intelligent applications. The role requires hands-on technical expertise in data engineering and AI integration while driving reliability, performance and security for enterprise-scale systems.ResponsibilitiesDesign and implement robust data architectures using cloud-native technologiesBuild large-scale ETL/ELT workflows to process heterogeneous datasetsCreate data models optimized for AI/ML pipelines and advanced analyticsDevelop streaming and batch pipelines leveraging tools like Azure Data Factory and DatabricksOperationalize ML solutions, integrating feature stores, model registries and inference endpointsCollaborate with data scientists to deploy and monitor AI models using enterprise MLOps frameworksIntegrate GenAI-assisted development methods into data workflows for automation and efficiencyEnsure data governance, lineage, cataloging and quality frameworks across platformsOptimize platform performance, manage compute cost efficiency and enable observability for critical workloadsContribute to best practices, mentoring engineering teams in advanced data and AI engineeringRequirementsMinimum 8+ years working in data engineering, with proven architecture and leadership experienceAdvanced proficiency in SQL for performance tuning at large scaleStrong programming skills in Python, with applied experience in data engineering workflowsExpertise in PySpark for distributed data processingHands-on experience with Databricks, including Delta Lake and performance optimizationKnowledge of Azure Data Factory for orchestration of pipelines (ETL/ELT)Familiarity with AI/ML pipeline development, including integration of models into productionDemonstrated exposure to Gen AI-assisted development workflows for accelerating data engineering tasksStrong understanding of CI/CD processes for data and ML pipelines such as GitHub Actions, Azure DevOpsAbility to manage enterprise-scale solutions across global environments with compliance in mindNice to havePrompt Engineering knowledge and experience building RAG workflowsFamiliarity with Microsoft Foundry platformsVersion control and CI/CD experience with GitHubUnderstanding of ETL/ELT optimization patterns beyond Azure stackKnowledge of data mesh or lakehouse architectural conceptsHands-on exposure to dbt (data build tool), MKDocs and similar developer productivity toolingWe offerEPAM Employee Stock Purchase Plan (ESPP)Protection benefits including life assurance, income protection and critical illness coverPrivate medical insurance and dental careEmployee Assistance ProgramCompetitive group pension planCyclescheme, Techscheme and season ticket loansVarious perks such as free Wednesday lunch in-office, on-site massages and regular social eventsLearning and development opportunities including in-house training and coaching, professional certifications, and coursesIf otherwise eligible, participation in the discretionary annual bonus programIf otherwise eligible and hired into a qualifying level, participation in the discretionary Long-Term Incentive (LTI) Program #J-18808-Ljbffr