Senior Data Engineer in Worcester

Salary

£? - ? per year

Requirements

• Strong experience with the Elastic Stack (Elasticsearch, Logstash, Kibana)
• Proven expertise with Apache NiFi
• Extensive experience designing and building enterprise-scale data pipelines
• Strong understanding of secure data architectures and data governance
• Experience operating within regulated or security-conscious environments
• Ability to design and deliver end-to-end data engineering solutions
• Excellent troubleshooting and performance optimisation skills
• Experience working within government, defence, or other highly regulated sectors
• Knowledge of Apache Kafka, Spark, or Hadoop
• Experience with Docker and Kubernetes
• Familiarity with monitoring and observability tools such as Prometheus, Grafana, or ELK
• Understanding of machine learning algorithms and their implementation
• Experience with Infrastructure as Code tools such as Terraform and Ansible

Responsibilities

• Design, develop, and maintain secure, scalable data pipelines using the Elastic Stack (Elasticsearch, Logstash, Kibana) and Apache NiFi
• Build and manage data ingestion, transformation, and integration processes
• Ensure data quality, integrity, security, and compliance with governance standards
• Collaborate with data architects and security teams within highly secure environments
• Monitor and manage large-scale, real-time data flows, optimising performance and reliability
• Develop robust data models to support analytics and reporting capabilities
• Create dashboards and visualisations using Kibana to drive operational insights
• Troubleshoot, debug, and tune data pipelines and platform performance
• Implement high availability, backup, and disaster recovery strategies
• Produce clear technical documentation, including architecture, workflows, and security protocols

Technologies

• Docker
• ELK
• ElasticSearch
• Grafana
• Hadoop
• Support
• Kafka
• Kibana
• Kubernetes
• Machine Learning
• Prometheus
• Security
• Spark
• Terraform
• Ansible

More

We are seeking an experienced Senior Data Engineer to join a high-profile programme delivering secure, scalable data platforms within a highly regulated environment. This is an exciting opportunity to work on complex data engineering challenges using cutting-edge technologies, with the flexibility of compressed working patterns. The role is fully onsite in Worcester, with the option to work a compressed schedule such as three extended working days while being compensated for a full five-day week. We offer a 12-month rolling contract, outside IR35, with a rate of £70-£75 per hour and the opportunity to work on complex, mission-critical data platforms within a long-term, high-profile programme.