At a Glance
- Tasks: Design and maintain scalable data platforms and pipelines for analytics and machine learning.
- Company: Join a dynamic team in a remote-first innovative tech company.
- Benefits: Attractive salary, flexible working hours, and opportunities for professional growth.
- Other info: Great career advancement opportunities in a supportive team culture.
- Why this job: Make an impact with cutting-edge technology in a collaborative environment.
- Qualifications: Experience in Python, PySpark, AWS, and strong problem-solving skills.
The predicted salary is between 77000 - 77000 £ per year.
We are seeking an experienced and highly motivated Data Engineer to join our growing team. In this role, you will be responsible for designing, developing, and maintaining scalable data platforms and pipelines that support business intelligence, analytics, machine learning, and operational reporting initiatives. You will work closely with data analysts, software engineers, architects, and business stakeholders to deliver robust, high-performance data solutions in a cloud-native AWS environment. The ideal candidate has strong expertise in PySpark, Python, Apache Airflow, AWS services, Terraform, and modern DevOps practices.
Key Responsibilities
- Data Engineering & Pipeline Development
- Design, develop, and maintain scalable, reliable, and efficient data pipelines using PySpark and Python.
- Build high-volume batch and real-time data processing solutions capable of handling large-scale datasets.
- Develop, optimize, and monitor ETL/ELT workflows to ensure data quality, consistency, and availability.
- Implement data transformation, cleansing, enrichment, and validation processes.
- Troubleshoot and resolve data pipeline failures, bottlenecks, and performance issues.
- Workflow Orchestration
- Design and manage complex workflows using Apache Airflow.
- Create and maintain DAGs with robust scheduling, dependency management, alerting, and recovery mechanisms.
- Monitor workflow execution and proactively address failures or performance concerns.
- Implement workflow best practices to ensure reliability and maintainability.
- Cloud Data Architecture (AWS)
- Architect and implement cloud-native data solutions on AWS.
- Develop scalable and secure data platforms leveraging: Amazon S3, Amazon Redshift, AWS Glue, AWS Lambda, Amazon EMR, API Gateway, Amazon CloudWatch, AWS IAM.
- Ensure adherence to security, governance, and compliance standards.
- Optimise cloud resources for performance and cost efficiency.
- Infrastructure as Code
- Provision and manage AWS infrastructure using Terraform.
- Develop reusable Terraform modules and templates.
- Implement infrastructure automation to support development, testing, and production environments.
- Maintain version-controlled infrastructure and deployment processes.
- DevOps & CI/CD
- Design and maintain CI/CD pipelines using GitHub Actions.
- Automate testing, deployment, monitoring, and infrastructure updates.
- Support continuous integration and continuous delivery best practices.
- Collaborate with engineering teams to improve deployment reliability and efficiency.
- Performance Optimisation
- Optimise Spark applications for scalability and efficiency.
- Conduct performance tuning of distributed data processing jobs.
- Identify and resolve resource utilisation issues across cloud and distributed environments.
- Implement monitoring and logging strategies to improve observability.
- Collaboration & Data Governance
- Partner with business stakeholders, analysts, and engineering teams to understand data requirements.
- Contribute to data architecture decisions and long-term platform strategy.
- Establish and promote data governance, quality, and security best practices.
- Document systems, processes, and technical solutions to support maintainability and knowledge sharing.
Required Skills & Experience
- Strong experience with Python and PySpark.
- Hands-on expertise with Apache Airflow.
- Extensive experience working with AWS cloud services.
- Strong knowledge of Amazon Redshift, AWS Glue, S3, Lambda, EMR, API Gateway, CloudWatch, and IAM.
- Experience with Terraform and Infrastructure as Code (IaC).
- Proficiency with Git, GitHub Actions, and CI/CD pipelines.
- Solid understanding of distributed data processing and Spark optimization.
- Experience designing scalable data architectures and data models.
- Strong SQL skills and understanding of data warehousing concepts.
- Excellent troubleshooting, analytical, and problem-solving abilities.
- Strong communication and collaboration skills.
Data Engineer in Warrington employer: Athsai
Join a forward-thinking company that values innovation and collaboration, offering a fully remote work environment for the Data Engineer role. With a competitive salary of £77K, we provide our employees with opportunities for professional growth, access to cutting-edge technologies, and a culture that prioritises teamwork and knowledge sharing. Our commitment to data governance and quality ensures that you will be part of impactful projects that drive business intelligence and analytics in a cloud-native AWS setting.
StudySmarter Expert Advice🤫
We think this is how you could land Data Engineer in Warrington
✨Tip Number 1
Network like a pro! Reach out to your connections in the data engineering field, attend meetups, and engage in online forums. You never know who might have the inside scoop on job openings or can refer you directly.
✨Tip Number 2
Show off your skills! Create a portfolio showcasing your projects, especially those involving PySpark, AWS, and Apache Airflow. This gives potential employers a taste of what you can do and sets you apart from the crowd.
✨Tip Number 3
Prepare for interviews by brushing up on common data engineering questions and practical scenarios. Practice explaining your thought process when solving problems, as this will demonstrate your analytical skills and technical expertise.
✨Tip Number 4
Don't forget to apply through our website! We love seeing candidates who are genuinely interested in joining our team. Plus, it’s a great way to ensure your application gets the attention it deserves.
We think you need these skills to ace Data Engineer in Warrington
Some tips for your application 🫡
Tailor Your CV:Make sure your CV highlights your experience with Python, PySpark, and AWS services. We want to see how your skills match the role, so don’t be shy about showcasing relevant projects or achievements!
Craft a Compelling Cover Letter:Your cover letter is your chance to shine! Use it to explain why you’re passionate about data engineering and how your background makes you a perfect fit for our team. Keep it engaging and personal!
Showcase Your Projects:If you've worked on any cool data projects, mention them! Whether it's a personal project or something from a previous job, we love seeing practical examples of your skills in action.
Apply Through Our Website:We encourage you to apply directly through our website. It’s the best way for us to receive your application and ensures you’re considered for the role. Plus, it’s super easy!
How to prepare for a job interview at Athsai
✨Know Your Tech Stack
Make sure you brush up on your knowledge of Python, PySpark, and Apache Airflow. Be ready to discuss how you've used these technologies in past projects, especially in relation to building data pipelines and workflows.
✨Showcase Your Cloud Skills
Familiarise yourself with AWS services like S3, Redshift, and Glue. Prepare examples of how you've architected cloud-native solutions and optimised resources for performance and cost efficiency. This will show that you can hit the ground running.
✨Demonstrate Problem-Solving Abilities
Be prepared to talk about specific challenges you've faced in data engineering, particularly around troubleshooting pipeline failures or performance issues. Use the STAR method (Situation, Task, Action, Result) to structure your answers.
✨Collaboration is Key
Highlight your experience working with cross-functional teams, including data analysts and software engineers. Discuss how you've contributed to data governance and quality practices, as well as how you communicate technical concepts to non-technical stakeholders.
