# Senior Data EngineerIndia ContractAre you a Senior Data Engineer with 8+ years of experience in building scalable data platforms and pipelines? Apply now for a remote long-term contract opportunity with a leading client in London.## Requirements* 8+ years of experience in Data Engineering* Strong expertise in Python and extensive experience with PySpark and distributed data processing* Expert-level knowledge of AWS services including EMR, Glue, Lambda, S3, Athena, Redshift, ECS/EKS, Step Functions, EventBridge, IAM, CloudWatch, and SNS/SQS* Hands-on experience building data lakes and modern data platforms* Strong understanding of ETL/ELT frameworks and data modelling* Strong SQL skills and query optimization experience* Experience with NoSQL databases, particularly DynamoDB, including partitioning, indexing, and data access patterns* Hands-on experience with OpenSearch/Elasticsearch, including designing indexes, search schemas, query optimisation, and search performance tuning* Working knowledge of Java and understanding of object-oriented programming and software engineering best practices* Experience implementing analytics and monitoring dashboards* Strong problem-solving and analytical skills* Ability to work in a fast-paced, cloud-native environment* Excellent communication and stakeholder management skills* Strong ownership and accountability mindset with ability to mentor and guide team members* (Desirable) Experience with streaming platforms such as Kafka or Kinesis* (Desirable) Experience with Airflow or other workflow orchestration tools* (Desirable) Experience with Docker and Kubernetes* (Desirable) Exposure to Data Lake and Lakehouse architecture* (Desirable) Experience in Agile/Scrum development environments* (Desirable) Experience with Apache Kafka, Apache Airflow, Terraform, Databricks, Delta Lake, Snowflake, CI/CD pipelines, or Machine Learning data pipelines* (Desirable) Bachelor's or Master's degree in Computer Science, Engineering, or a related field* Language(s): English (fluent)## Job Description* Design, develop, and maintain large-scale data pipelines using Python and PySpark* Build and optimise batch and near real-time data processing solutions* Design scalable data architectures on AWS* Develop ETL/ELT workflows for ingesting, transforming, and publishing datasets* Work with structured and semi-structured data from multiple sources* Design and maintain NoSQL database solutions using DynamoDB* Implement search and analytics solutions using OpenSearch* Optimise data processing jobs for performance, scalability, and cost efficiency* Collaborate with product teams, architects, and software engineers to define data requirements* Monitor, troubleshoot, and resolve data platform and pipeline issues* Implement data quality, governance, security, and observability best practices* Participate in code reviews and mentor junior engineers #J-18808-Ljbffr
Senior Data Engineer
