AWS OpenSearch Consultant in London

The Role

• This role will have a significant impact on delivering innovative solutions for one of the leading retailers in the UK, driving digital transformation and enhancing customer experiences.

• You’ll work on projects that leverage cutting-edge technologies such as AWS OpenSearch, Kafka, Kotlin, Java Spring boot, Terraform, event-driven architectures, and cloud platforms, contributing to scalable, secure, and high-performing systems.

Your responsibilities:

• Design, implement, and manage scalable OpenSearch architectures to support enterprise search, logging, monitoring, and analytics workloads.

• Develop and maintain Infrastructure as Code (IaC) using Terraform for automated provisioning, configuration, and environment consistency.

• Configure and optimise OpenSearch clusters, indexing strategies, dashboards, and query performance to ensure high availability and reliability.

• Integrate Kafka and other data ingestion pipelines to enable real-time log collection, processing, and search capabilities.

• Troubleshoot performance bottlenecks, conduct root cause analysis, and implement proactive monitoring and capacity planning.

• Collaborate with cross-functional teams to deliver cloud-native solutions, ensuring security, compliance, and operational excellence across AWS environments.

• Drive automation, best practices, and technical governance while mentoring teams on OpenSearch, observability, and platform optimisation strategies.

Key Responsibilities

• Design OpenSearch platform architecture and search solutions.

• Define data ingestion and indexing strategies using Kafka and ELK.

• Lead OpenSearch implementation, optimisation, and upgrades.

• Develop and manage infrastructure using Terraform.

• Establish monitoring, logging, and operational best practices.

• Drive performance tuning, troubleshooting, and incident resolution.

• Support production deployments and ensure platform reliability.

Hands-on Skills Required – OpenSearch Senior Consultant

• Design and manage scalable OpenSearch cluster architectures.

• Implement log ingestion pipelines using Kafka, Logstash, and Beats.

• Build and automate infrastructure using Terraform and AWS services.

• Configure monitoring, alerting, and analytics dashboards.

• Optimize search, indexing, and cluster performance.

• Troubleshoot production issues and ensure platform reliability.

• Implement security, governance, and operational best practices.

Technologies

• Search Engine: AWS OpenSearch

• Languages: Kotlin, Java 21+, Python, JavaScript/TypeScript

• Frameworks: Spring Boot 3.x, Ktor, React (SSR), Apache Airflow

• Data Stores: DynamoDB, Redis, S3

• Messaging: Kafka (Confluent Cloud)

• AI/ML: Azure OpenAI (embeddings), Kumo platform (relevancy), custom Python ML models

• CI/CD: GitHub Actions, Terraform, Docker, Bosun