At a Glance
- Tasks: Design and optimise data pipelines using cutting-edge technologies like Azure, PySpark, and Airflow.
- Company: Join Circana, a global leader committed to inclusivity and continuous growth.
- Benefits: Flexible work environment, competitive salary, and opportunities for professional development.
- Other info: Be part of a dynamic team that values curiosity, accountability, and collaboration.
- Why this job: Make a real impact by solving client challenges with innovative data solutions.
- Qualifications: 8+ years in data engineering with strong skills in Python, SQL, and cloud technologies.
The predicted salary is between 70000 - 90000 € per year.
At Circana, we are fueled by our passion for continuous learning and growth, we seek and share feedback freely, and we celebrate victories both big and small in an environment that is flexible and accommodating to our work and personal lives. We’re a global company dedicated to fostering inclusivity and belonging. We value and celebrate the unique experiences, cultures, and viewpoints that each individual brings. By embracing a wide range of backgrounds, skills, expertise, and beyond, we create a stronger, more innovative environment for our employees, clients, and communities. With us, you can always bring your full self to work.
Join our Global Professional Services team as a Lead Data Engineer, where you will be instrumental in bridging the gap between client needs, business objectives, and cutting-edge technology solutions, particularly supporting our new Private Cloud clients. This role is for more than just a service provider; we are seeking a consultant-minded go-getter who is client-obsessed, a liquid thinker who thrives at pace, and is bold and brave in challenging the status quo. We are a team that values being part of a successful business and building partnerships based on trust, deep knowledge, respect, and unwavering support. We are passionate about technology, data and AI and how it can be leveraged to solve our clients' most pressing business challenges.
You will be a key player in understanding our clients' world, becoming their trusted partner, and proactively identifying opportunities for innovation and efficiency. Curiosity, accountability, and a positive, questioning mindset are at the core of our team's DNA – we don't just aim to meet expectations, we strive to exceed them. You will work collaboratively with global Circana teams, acting as the vital link to ensure seamless, unified support and strategic technological partnership for our clients.
In this role we are seeking a skilled and motivated Data Engineer to join a growing Global Team based in the UK. You will be responsible for designing, building, and maintaining robust data pipelines and infrastructure on the Azure cloud platform. You will leverage your expertise in PySpark, Apache Spark, and Apache Airflow to process and orchestrate large-scale data workloads, ensuring data quality, efficiency, and scalability. If you have a passion for Data Engineering and a desire to make a significant impact, we encourage you to apply!
Duties & Responsibilities
- Design, develop, and optimize scalable DATA workflows using Python, PySpark, and Airflow
- Implement real-time and batch data processing using Spark
- Enforce best practices for data quality, governance, and security throughout the data lifecycle
- Ensure data availability, reliability and performance through monitoring and automation
Cloud Data Engineering:
- Manage cloud infrastructure and cost optimization for data processing workloads
- Implement CI/CD pipelines for data workflows to ensure smooth and reliable deployments
Big Data & Analytics:
- Build and optimize large-scale data processing pipelines using Apache Spark and PySpark
- Implement data partitioning, caching, and performance tuning for Spark-based workloads
- Work with diverse data formats (structured and unstructured) to support advanced analytics and machine learning initiatives
Workflow Orchestration (Airflow)
- Design and maintain DAGs (Directed Acyclic Graphs) in Airflow to automate complex data workflows
- Monitor, troubleshoot, and optimize job execution and dependencies
Team Leadership & Collaboration
- Lead a team of data engineers, providing technical guidance and mentorship
- Foster a collaborative environment and promote best practices for coding standards, version control, and documentation
Required Skills & Experience:
- Client facing role so strong communication and collaboration skills are vital (Ideally gained in a Consulting organisation)
- 8+ years of experience in data engineering with expertise in Azure, PySpark, Spark, and Airflow
- Strong programming skills in Python, SQL with the ability to write efficient and maintainable code
- Deep understanding of Spark internals (RDDs, DataFrames, DAG execution, partitioning, etc.)
- Experience with Airflow DAGs, scheduling, and dependency management
- Knowledge of Git, Docker, Kubernetes, Terraform, and apply best practices of DevOps for CI/CD workflows
- Excellent problem-solving skills and ability to optimize large-scale data processing
- Experience in leading teams and working in Agile/Scrum environments
- A proven track record of working effectively with global remote teams
Bonus Points:
- Experience with data modeling and data warehousing concepts
- Familiarity with data visualization tools and techniques
- Knowledge of machine learning algorithms and frameworks
What are we looking for?
- An authentic optimist with a genuine passion for solving problems and driving progress
- Intensely curious – you love to understand the 'why' and 'how' behind things, especially when it comes to technology and client challenges
- Highly accountable for your actions, your deliverables, and the success of your clients
- Comfortable and adept at building strong, professional relationships with both external clients and internal technical and business teams
- Deeply committed to giving your best for your team and ensuring our technology solutions exceed client expectations
- Confident in making informed decisions, weighing options and considering the technical and business implications
- Highly organised, with meticulous attention to detail in managing documentation, communication, and time
- Passionate about technology, Data, AI, and their application in solving real-world business problems
- Previous experience translating client requirements into technical specifications
- Previous experience building relationships with clients
ETL Data Lead Developer in London employer: Circana
At Circana, we pride ourselves on being an exceptional employer that champions continuous learning and growth within a flexible and inclusive work environment. Our commitment to celebrating diverse perspectives fosters innovation and collaboration, while our focus on employee development ensures that you can thrive in your career as a Lead Data Engineer. Join us in the UK, where you will not only contribute to cutting-edge technology solutions but also be part of a supportive team that values curiosity, accountability, and meaningful partnerships.
StudySmarter Expert Advice🤫
We think this is how you could land ETL Data Lead Developer in London
✨Tip Number 1
Network like a pro! Get out there and connect with people in the industry. Attend meetups, webinars, or even just grab a coffee with someone who works at Circana. Building relationships can open doors that applications alone can't.
✨Tip Number 2
Show off your skills! If you’ve got a portfolio or any projects that highlight your data engineering prowess, make sure to share them during interviews. It’s a great way to demonstrate your expertise in PySpark, Spark, and Airflow.
✨Tip Number 3
Prepare for the interview by understanding Circana's values and culture. They love curiosity and accountability, so think of examples from your past where you've embodied these traits. This will help you stand out as a perfect fit!
✨Tip Number 4
Don’t forget to apply through our website! It’s the best way to ensure your application gets seen by the right people. Plus, it shows you’re genuinely interested in joining our amazing team at Circana.
We think you need these skills to ace ETL Data Lead Developer in London
Some tips for your application 🫡
Tailor Your Application:Make sure to customise your CV and cover letter for the ETL Data Lead Developer role. Highlight your experience with Azure, PySpark, and Airflow, and show how your skills align with what we’re looking for at Circana.
Showcase Your Curiosity:We love a curious mind! In your application, share examples of how you've tackled challenges or sought out new knowledge in data engineering. This will resonate with our team’s values and show us you’re a great fit.
Be Clear and Concise:When writing your application, keep it straightforward. Use clear language and avoid jargon where possible. We appreciate well-structured applications that get straight to the point while still showcasing your personality.
Apply Through Our Website:Don’t forget to submit your application through our website! It’s the best way for us to receive your details and ensures you’re considered for the role. Plus, it shows you’re keen on joining our team!
How to prepare for a job interview at Circana
✨Know Your Tech Inside Out
Make sure you brush up on your knowledge of Azure, PySpark, and Apache Spark. Be ready to discuss how you've used these technologies in past projects, and think of specific examples where you optimised data workflows or solved complex problems.
✨Showcase Your Client-Facing Skills
Since this role is client-facing, prepare to demonstrate your communication and collaboration skills. Think of instances where you've successfully built relationships with clients or worked in a consulting environment, and be ready to share those stories.
✨Emphasise Your Curiosity and Problem-Solving
Circana values curiosity and a questioning mindset. Prepare to discuss how you've approached challenges in the past, what questions you asked, and how your curiosity led to innovative solutions. This will show that you're not just about meeting expectations but exceeding them.
✨Be Ready for Team Dynamics
As a Lead Data Engineer, you'll be leading a team. Think about your leadership style and how you foster collaboration. Be prepared to talk about how you've mentored others, promoted best practices, and contributed to a positive team environment.
