Find a job 
Featured Companies 
Featured Universities 
For Companies At a Glance
- Tasks: Design and maintain data pipelines using Azure, PySpark, and Airflow.
- Company: Circana values continuous learning, diversity, and a flexible work environment.
- Benefits: Enjoy remote work options, a supportive culture, and recognition as a Great Place To Work.
- Why this job: Make a significant impact in data engineering while collaborating with a global team.
- Qualifications: Experience in data engineering, strong Python skills, and knowledge of Azure and Spark required.
- Other info: Join a team that champions curiosity, inclusivity, and accountability.
The predicted salary is between 43200 - 72000 £ per year.
At Circana, we are fueled by our passion for continuous learning and growth, we seek and share feedback freely, and we celebrate victories both big and small in an environment that is flexible and accommodating to our work and personal lives. We have a global commitment to diversity, equity, and inclusion as we believe in the undeniable strength that diversity brings to our business, employees, clients, and communities. With us, you can always bring your full self to work. Join our inclusive, committed team to be a challenger, own outcomes, and stay curious together. Circana is proud to be Certified by Great Place To Work.
We are seeking a skilled and motivated Data Engineer to join a growing Global Team based in the UK. In this role, you will be responsible for designing, building, and maintaining robust data pipelines and infrastructure on the Azure cloud platform. You will leverage your expertise in PySpark, Apache Spark, and Apache Airflow to process and orchestrate large-scale data workloads, ensuring data quality, efficiency, and scalability. If you have a passion for data engineering and a desire to make a significant impact, we encourage you to apply!
Job Responsibilities
- Design, develop, and optimize scalable data workflows using Python, PySpark, and Airflow
- Implement real-time and batch data processing using Spark
- Enforce best practices for data quality, governance, and security throughout the data lifecycle
- Ensure data availability, reliability and performance through monitoring and automation
- Manage cloud infrastructure and cost optimization for data processing workloads
- Implement CI/CD pipelines for data workflows to ensure smooth and reliable deployments
- Build and optimize large-scale data processing pipelines using Apache Spark and PySpark
- Implement data partitioning, caching, and performance tuning for Spark-based workloads
- Work with diverse data formats (structured and unstructured) to support advanced analytics and machine learning initiatives
- Design and maintain DAGs (Directed Acyclic Graphs) in Airflow to automate complex data workflows
- Monitor, troubleshoot, and optimize job execution and dependencies
- Lead a team of data engineers, providing technical guidance and mentorship
- Foster a collaborative environment and promote best practices for coding standards, version control, and documentation
Requirements
- This is a client-facing role; strong communication and collaboration skills are vital
- Experience in data engineering with expertise in Azure, PySpark, Spark, and Airflow
- Strong programming skills in Python, SQL with the ability to write efficient and maintainable code
- Deep understanding of Spark internals (RDDs, DataFrames, DAG execution, partitioning, etc.)
- Experience with Airflow DAGs, scheduling, and dependency management
- Knowledge of Git, Docker, Kubernetes, Terraform, and apply best practices of DevOps for CI/CD workflows
- Excellent problem-solving skills and ability to optimize large-scale data processing
- Experience in leading teams and working in Agile/Scrum environments
- A proven track record of working effectively with global remote teams
Desirable:
- Experience with data modelling and data warehousing concepts
- Familiarity with data visualization tools and techniques
- Knowledge of machine learning algorithms and frameworks
What we offer:
As well as the technical skills, experience and attributes that are required for the role, our shared behaviours sit at the core of our organization. Therefore, we always look for people who can continuously champion these behaviours throughout the business within their day-to-day role:
- Stay Curious: Being hungry to learn and grow, always asking the big questions
- Seek Clarity: Embracing complexity to create clarity and inspire action
- Own the Outcome: Being accountable for decisions and taking ownership of our choices
- Centre on the Client: Relentlessly adding value for our customers
- Be a Challenger: Never complacent, always striving for continuous improvement
- Champion Inclusivity: Fostering trust in relationships engaging with empathy, respect, and integrity
- Commit to each other: Contributing to making Circana a great place to work for everyone
This position can be located in the following area(s): Remote or Bracknell, UK
Lead Data Engineer (Remote) employer: Circana
Contact Detail:
Circana Recruiting Team
StudySmarter Expert Advice 🤫
We think this is how you could land Lead Data Engineer (Remote)
✨Tip Number 1
Familiarise yourself with the specific technologies mentioned in the job description, such as Azure, PySpark, and Apache Airflow. Having hands-on experience or projects that showcase your skills in these areas can set you apart from other candidates.
✨Tip Number 2
Highlight your leadership experience, especially if you've led teams in Agile/Scrum environments. Be prepared to discuss how you've fostered collaboration and best practices in previous roles, as this aligns with the company's values.
✨Tip Number 3
Demonstrate your problem-solving skills by preparing examples of how you've optimised large-scale data processing in past projects. This will show your ability to handle the challenges that come with the role.
✨Tip Number 4
Engage with the company’s culture by researching their commitment to diversity, equity, and inclusion. Be ready to share how you embody these values in your work and interactions, as this is crucial for fitting into their team.
We think you need these skills to ace Lead Data Engineer (Remote)
Some tips for your application 🫡
Tailor Your CV: Make sure your CV highlights relevant experience in data engineering, particularly with Azure, PySpark, and Apache Spark. Use specific examples to demonstrate your skills in building data pipelines and managing cloud infrastructure.
Craft a Compelling Cover Letter: In your cover letter, express your passion for data engineering and how your values align with Circana's commitment to diversity, equity, and inclusion. Mention your experience leading teams and working in Agile environments to showcase your leadership skills.
Showcase Technical Skills: Clearly outline your technical expertise in Python, SQL, and tools like Airflow and Docker. Provide examples of projects where you implemented CI/CD pipelines or optimised data processing workflows to demonstrate your problem-solving abilities.
Highlight Soft Skills: Since this is a client-facing role, emphasise your strong communication and collaboration skills. Share instances where you've successfully worked with global remote teams or mentored others, showcasing your ability to foster a collaborative environment.
How to prepare for a job interview at Circana
✨Showcase Your Technical Skills
Be prepared to discuss your experience with Azure, PySpark, Apache Spark, and Airflow in detail. Highlight specific projects where you designed and optimised data pipelines, and be ready to explain the technical challenges you faced and how you overcame them.
✨Demonstrate Leadership Qualities
Since this role involves leading a team, share examples of your leadership experience. Talk about how you've mentored others, fostered collaboration, and promoted best practices in previous roles. This will show that you can guide a team effectively.
✨Emphasise Communication Skills
As this is a client-facing role, strong communication is key. Prepare to discuss how you've successfully communicated complex technical concepts to non-technical stakeholders. This will demonstrate your ability to bridge the gap between technical and non-technical teams.
✨Align with Company Values
Circana values curiosity, accountability, and inclusivity. Be ready to discuss how you embody these values in your work. Share instances where you've sought clarity in complex situations or taken ownership of outcomes, as this will resonate well with their culture.
