- Design and build data pipelines and ETL processes into a Lakehouse architecture across cloud (AWS) and on premises platforms
- Implement change data capture and event-driven patterns to ingest data from source systems
- Develop data models for the pipeline layer
- Handle non-functional requirements including security, performance and metadata labelling
- Write unit and integration tests for pipeline components
- Work within a defined SDLC using Git-based CI/CD tooling
- Contribute to sprint delivery as part of an integrated development team
Experience building data pipelines within Lakehouse architectures - Comfortable with Apache tooling, for data engineering work
- Experience with event sourcing, change data capture and event-driven architectures
- Java and/or Groovy, with a solid grasp of object-oriented design principles
- Experience working in an SDLC with an integrated team on CI/CD tooling (Git)
Nice-to-have
- Apache NiFi
- AWS, particularly EMR, S3 and SQS
- Awareness of domain-driven design
- Linux and Scripting
- Postgres or PL/SQL
#J-18808-Ljbffr
