b Overview /b p As Lead Data AI Engineer, you will design and deliver data-driven, AI-enabled capabilities on cloud platforms in a hybrid UK setting. You’ll build scalable data pipelines and governance to support analytics and intelligent applications, while ensuring reliability, performance and security for enterprise workloads. You will work with cross-functional teams to operationalize ML solutions, integrate GenAI-assisted methods, and drive data quality across platforms. This role offers the chance to shape data architecture and ML workflow practices at scale, with a focus on efficiency and governance. /p b Responsibilities /b ul li Design and implement cloud-native data architectures /li li Build large-scale ETL/ELT workflows for heterogeneous datasets /li li Create data models optimized for AI/ML pipelines and analytics /li li Develop streaming and batch pipelines using Azure Data Factory and Databricks /li li Operationalize ML solutions with feature stores, model registries, and inference endpoints /li li Collaborate with data scientists to deploy and monitor AI models via enterprise MLOps /li li Integrate GenAI-assisted development into data workflows for automation /li li Ensure data governance, lineage, cataloging and quality across platforms /li li Optimize performance, control compute costs, and enable observability for critical workloads /li li Mentor engineering teams and contribute to best practices in data and AI engineering /li /ul b Key requirements /b ul li Minimum 8+ years in data engineering with architecture and leadership experience /li li Advanced SQL proficiency for large-scale performance tuning /li li Strong Python programming skills with data engineering workflow experience /li li Expertise in PySpark for distributed processing /li li Hands-on Databricks experience, including Delta Lake and optimization /li li Knowledge of Azure Data Factory for ETL/ELT orchestration /li li Familiarity with AI/ML pipeline development and production model integration /li li Experience with Gen AI-assisted development workflows for data tasks /li li Understanding of CI/CD for data and ML pipelines (GitHub Actions, Azure DevOps) /li li Ability to manage enterprise-scale solutions across global, compliant environments /li li Prompt engineering knowledge and experience building RAG workflows /li li Knowledge of Microsoft Foundry platforms /li li Version control and CI/CD experience with GitHub /li li Understanding of ETL/ELT optimization patterns beyond Azure /li li Knowledge of data mesh or lakehouse concepts /li li Hands-on exposure to dbt, MkDocs, and similar developer tools /li /ul ul li Leading and mentoring teams /li li Cross-functional collaboration /li li Communication and stakeholder management /li li Python /li li SQL /li li PySpark /li /ul
Lead Data & AI Engineer employer: EPAM Systems
Join a dynamic and innovative team in London where your expertise as a Delivery Director will be valued and nurtured. We pride ourselves on a collaborative work culture that fosters continuous improvement and professional growth, offering you the chance to work alongside award-winning teams in the Retail/Consumer Products industry. With a focus on client satisfaction and strategic project delivery, you'll find meaningful and rewarding employment in an environment that champions creativity and technical excellence.
