Project description
This is a strategic data engineering engagement with our client to architect and plan the migration of their entire data processing and ETL estate from Matillion to AWS Glue — a foundational shift in how one of the world's largest financial market infrastructure companies handles its data pipelines.During this Mobilisation Phase, you'll work jointly with engineering teams to reverse-engineer the existing landscape, design the target-state architecture across every layer (infrastructure, data processing, workflows, dependencies, and operating model), and build the detailed delivery blueprint that will greenlight the full-scale migration.The work is technically rich and highly collaborative: you'll review and validate job inventories spanning hundreds of ETL workflows, define reusable migration patterns and templates, design a validation and reconciliation framework, run a proof-of-concept to stress-test the approach, and navigate rigorous internal governance — from Architectural Significance Assessments through Architectural Review Boards to a formal Gate 1 decision.
Responsibilities
- Target-state architecture design — define and own the target-state AWS architecture across compute, data, and orchestration layers
- HLD/LLD ownership — produce and own High-Level Design and Low-Level Design documentation for migration workloads
- ARB submissions — prepare and present architecture proposals to the Architecture Review Board for approval
- Technical decisions — make and document key technical decisions across the migration program, balancing trade-offs (cost, performance, scalability)
- Pattern definition — define reusable architecture and migration patterns for use across teams/pods
- Agentic AI framework architecture — design the architecture for the Agentic AI migration framework, including its integration with the broader technical landscape
SKILLS
Must have
- 5+ years experience
- hands-on experince with AWS Migration
- AWS Glue, Step Functions, Lambda, EventBridge — deep hands-on experience designing solutions using these core AWS services
- IaC (Terraform) — strong experience defining and managing infrastructure as code with Terraform
- Data platform architecture — proven experience architecting data platforms (data lakes, warehouses, pipelines) at scale
- PySpark — hands-on experience with PySpark for large-scale data processing
- CI/CD — experience designing and implementing CI/CD pipelines for infrastructure and data workflows
- AI/ML frameworks — working knowledge of AI/ML frameworks and their application within architecture design
- GitHub Copilot — practical experience using GitHub Copilot within development/architecture workflows
Nice to have
- AWS certification (Data Analytics Specialty or Solutions Architect)
- Experience with BI tools (QuickSight, Tableau, Power BI)
- Infrastructure as Code experience (Terraform/CloudFormation)
- Industry experience relevant to your business
- Exposure to streaming data (Kinesis) or ML pipelines (SageMaker)
- experince in financial domain
#J-18808-Ljbffr
Senior Technical Lead employer: Luxoft
As a leading bank, we pride ourselves on fostering a dynamic and inclusive work environment that encourages professional growth and collaboration. Our Training Business Analyst role offers the opportunity to engage in impactful projects while working alongside talented teams dedicated to compliance and innovation. With a strong commitment to employee development and a culture that values diverse perspectives, we provide our staff with the tools and support needed to thrive in their careers.
