At a Glance
- Tasks: Lead quality assurance in a cutting-edge cloud data engineering programme.
- Company: Join Scrumconnect, a dynamic digital transformation consultancy.
- Benefits: Fully remote role with competitive salary and inclusive culture.
- Other info: Diversity-driven workplace with excellent growth opportunities.
- Why this job: Make an impact in innovative tech while working with top-tier teams.
- Qualifications: Experience in automated testing and cloud data environments required.
The predicted salary is between 60000 - 75000 £ per year.
About Scrumconnect Consulting is a UK-based digital transformation consultancy delivering agile, secure, and scalable technology solutions for public and private sector clients. This is a fully remote role based in India, supporting UK operations and client engagements. The successful candidate will work closely with UK leadership teams and must be comfortable operating within UK regulatory, governance, and delivery frameworks.
About the Role We are looking for an experienced Test Manager to lead quality assurance and testing activities within a large-scale cloud data engineering programme operating across a modern AWS-native technology ecosystem, including Apache Airflow, Amazon Athena, AWS Glue, S3, EMR, DynamoDB, and related cloud services.
This role combines hands-on technical leadership with test strategy ownership. You will be responsible for defining and driving the overall testing approach across automated pipelines, data workflows, cloud infrastructure, and release delivery while managing risks, improving testing standards, and mentoring engineering teams.
The ideal candidate will have strong experience in Big Data and Data Lake environments, including testing both structured and unstructured datasets, and will be comfortable operating in highly automated CI/CD-driven delivery environments.
You will work closely with engineering, DevOps, architecture, and stakeholder teams to ensure quality is embedded throughout the software delivery lifecycle.
Key Responsibilities
- Define, implement, and manage the overall test strategy across cloud-native data platforms and applications
- Lead end-to-end testing activities across data pipelines, cloud services, APIs, and distributed systems
- Establish quality standards, governance processes, and testing best practices across teams
- Drive a quality-first engineering culture across the programme
- Provide leadership, mentoring, and coaching to Test Engineers and QA team members
Automated Testing & Frameworks
- Design, implement, and oversee scalable automated test frameworks for data and application testing
- Build and maintain automated test suites across AWS cloud infrastructure and data workflows
- Enhance existing automation frameworks to improve reliability, coverage, maintainability, and reporting
- Ensure automated testing is fully integrated within GitLab CI/CD pipelines for rapid feedback and release confidence
Big Data & Data Lake Testing
- Lead testing activities across Big Data and Data Lake solutions containing structured and unstructured datasets
- Validate large-scale data ingestion, transformation, and processing workflows
- Ensure data integrity, transformation accuracy, reconciliation, and performance under load
- Oversee PySpark-based data validation and interrogation activities on AWS EMR
- Support testing and monitoring of workflows orchestrated through Apache Airflow
- Drive root cause analysis for production data issues and platform defects
Risk Management & Reporting
- Identify, assess, and communicate quality risks across releases and environments
- Analyse test execution metrics, defects, trends, and quality indicators
- Translate technical findings into clear, prioritised reporting for technical and non-technical stakeholders
- Provide regular status reporting to programme leadership and client stakeholders
CI/CD & DevOps
- Collaborate with DevOps and engineering teams to improve release quality and deployment confidence
- Ensure automated test execution within GitLab CI/CD pipelines
- Support infrastructure testing and validation within Terraform-managed cloud environments
- Contribute to continuous improvement initiatives across testing and delivery processes
Technology Stack
Apache Airflow, Amazon Athena, AWS S3, AWS Glue, AWS EMR, AWS EC2, AWS ECR, AWS DynamoDB, AWS CloudWatch, AWS IAM, Python, SQL, Java, Bash, GitLab CI/CD, Jupyter Notebooks, Apache Spark / PySpark, Terraform, Docker
Key Skills
- Strong proficiency in Python, SQL, Java, and scripting languages such as Bash
- Hands-on experience testing Big Data platforms and Data Lake solutions
- Strong experience with AWS EMR, Apache Airflow, and Jupyter Notebooks
- Experience performing PySpark-based data interrogation, validation, and testing
- Strong knowledge of AWS services including S3, Glue, Athena, EMR, CloudWatch, IAM, DynamoDB, EC2, and ECR
- Experience designing, implementing, and managing enterprise-scale automated test frameworks
- Strong understanding of CI/CD practices using GitLab pipelines
- Experience with infrastructure-as-code tools such as Terraform
- Familiarity with Docker and containerised testing environments
- Understanding of dimensional data models and slowly changing dimensions
- Experience validating structured and unstructured data workflows
- Strong analytical and root cause analysis skills
- Ability to communicate technical risks clearly to stakeholders at all levels
- Experience leading and mentoring QA/Test Engineering teams
Skills & Experience Required
- Proven experience as a Test Manager, Lead Test Engineer, or QA Lead within cloud and data engineering environments
- Strong background in automated testing for cloud-native and data-centric platforms
- Experience testing large-scale data pipelines, transformation processes, and distributed systems
- Experience managing or leading QA/testing teams in Agile delivery environments
- Strong stakeholder management and communication skills
- Experience establishing test governance, reporting, and quality metrics
- Strong problem-solving and decision-making capability
- Experience working within regulated or security-conscious delivery environments
- SFIA Level 5 capability — provides leadership, influences strategy, and drives quality improvements
- Active SC clearance (mandatory)
Diversity and Inclusion At Scrumconnect Consulting, we believe diversity drives innovation and stronger outcomes for our teams and clients. We are committed to creating an inclusive environment where every individual is respected, valued, and supported. We welcome applications from candidates of all backgrounds, experiences, and perspectives.
Data Test Engineer (SC Cleared) in Newcastle employer: Scrumconnect Limited
At Scrumconnect, we pride ourselves on being an exceptional employer, offering a dynamic work culture that champions collaboration and continuous learning. Our commitment to employee growth is evident through our investment in training and development opportunities, ensuring that our Data Engineers are equipped with the latest skills in cloud technologies and data engineering practices. With a focus on meaningful public service projects, you will have the chance to make a significant impact while enjoying the flexibility of remote work and a supportive team environment.
StudySmarter Expert Advice🤫
We think this is how you could land Data Test Engineer (SC Cleared) in Newcastle
✨Get Involved in Data Science Meetups
Tap into local data science meetups or workshops to connect with fellow enthusiasts and professionals. These events are goldmines for networking, and sometimes even lead directly to job openings at companies like Scrumconnect Limited!
✨Show Off Your Projects
Start building a public portfolio showcasing your data science projects on platforms like GitHub or personal websites. Highlight unique analyses or models you've developed. This not only demonstrates your skills but also gets your name out there for roles like Data Test Engineer (SC Cleared) at Scrumconnect Limited.
✨Leverage Professional Networks
Join professional bodies related to data science, like the Data Science Society or similar organisations. Getting involved can lead to mentorship opportunities and insider knowledge about full-time positions at companies like Scrumconnect Limited.
✨Apply Directly through Our Website
When you find a suitable opening like Data Test Engineer (SC Cleared) at Scrumconnect Limited, make sure to apply directly through our website. It gives you an edge and shows you're keen to join our team. Plus, who doesn’t love a direct application? It’s easier than navigating through job boards!
We think you need these skills to ace Data Test Engineer (SC Cleared) in Newcastle
Some tips for your application 🫡
Show Off Your Projects:In the world of data science, your projects can speak volumes about your skills. Make sure to showcase a few key projects in your CV or portfolio, especially those that highlight your ability to work with data sets, build models, or use relevant tools like Python, R, or SQL. Don’t forget to include links to any GitHub repositories if applicable!
Quantify Your Achievements:Employers love numbers! When drafting your CV, highlight your achievements with quantifiable results. For instance, mention how your data analysis led to a certain percentage increase in efficiency or revenue at a previous job or project. These details can really make your application pop!
Craft a Tailored Cover Letter:For a full-time role at Scrumconnect Limited, your cover letter should reflect your passion for data science and your excitement about the specific projects or values of the company. Dive into why you’re a good fit, how your skills align with their needs, and any unique perspectives you can bring to the team.
Stand Out with Relevant Courses and Certifications:Although experience talks, relevant courses or certifications can be your ticket to impressing hiring managers at Scrumconnect Limited. Mention any standout courses you've completed that equipped you with essential skills, such as machine learning certifications or data visualisation courses. This shows your commitment to continuously developing your skills in the field!
How to prepare for a job interview at Scrumconnect Limited
✨Brush Up on Your Statistics
For a data science role, we need to seriously sharpen our statistics skills. Get ready to tackle technical questions on probability distributions, hypothesis testing, and regression analysis. These are often the bread and butter of data science interviews, so don't just skim over them!
✨Showcase Your Projects
Prepare a killer portfolio showcasing your data science projects. We should include details about the datasets used, the tools and techniques applied, and the impact of your findings. If we can walk them through a particularly challenging project or a cool visualisation that had real-world implications, it’ll really make us stand out!
✨Get Comfortable with Python and R
Most data science positions require us to be proficient in programming languages like Python and R. We should practice common libraries like pandas, NumPy, and scikit-learn, and be ready for live coding exercises or algorithm questions. Showing off our coding chops can really impress the interviewers at Scrumconnect Limited!
✨Prepare for Case Studies
Expect to encounter real-world case studies during the interview. We might be asked how we’d approach a data problem or analyse a dataset to extract insights. It's essential to think out loud and demonstrate our problem-solving process so that the interviewer can see our logical thinking in action.
