At a Glance
- Tasks: Lead technical architecture and design data pipelines for AI/ML projects.
- Company: Innovative tech firm focused on cutting-edge data solutions.
- Benefits: Competitive salary, flexible work options, and opportunities for professional growth.
- Other info: Exciting projects with potential for career advancement in a collaborative environment.
- Why this job: Join a dynamic team and shape the future of data engineering.
- Qualifications: 7+ years in AWS data engineering with strong skills in S3, Glue, and OpenSearch.
The predicted salary is between 80000 - 100000 £ per year.
Role summary: The overall technical lead and architect. Designs the metadata schema, builds the simulation onboarding pipeline, deploys metadata embedding pipeline and OpenSearch k-NN vector store, and authors data export format spec for AI/ML use case. This role is the deepest technical seat on the engagement:
- Key responsibilities
- Run the Sprint 1 architecture review of the existing UAT codebase (S3 + Glue + S3 Tables + OpenSearch + Athena) and deliver written gap findings.
- Design the metadata schema, taxonomy, and field catalogue (Light, Brain, Power).
- Tune data orchestration — Glue jobs, Athena queries, S3 Tables config, scheduling.
- Lead the deep-dive technical sessions with analysts on visualization requirements.
- Build and validate the simulation data onboarding pipeline against real data — including the 30 GB-per-run acoustic spectra dataset.
- Configure and validate the OpenSearch k-NN vector store and the Bedrock embedding pipeline.
- Author the AI/ML data export format specification and the AI onboarding pattern document.
- Co-design the API middleware blueprint with the Cloud Infrastructure Architect.
Must-have
- Principal-level hands-on data engineering on AWS — 7+ years.
- Deep production experience with S3, S3 Tables, Glue, Athena, and OpenSearch (including k-NN / vector search).
- Built and shipped vector embedding workloads.
- Strong metadata modelling and data taxonomy design experience for scientific or engineering domains.
- Comfort working with Parquet, JSON-LD, and large binary scientific data formats (mesh, time-series, spectra).
- Python proficiency; PySpark / Glue job tuning experience.
Nice-to-have / differentiators
- Prior simulation / CAE / HPC data lake experience (Ansys, Siemens NX, BETA CAE, OpenFOAM, etc.).
- Familiarity with surrogate model training data pipelines.
- Experience with SageMaker Unified Studio or comparable governed data-mesh tooling (in case of required integration).
- Multi-cloud data engineering (AWS GCP) experience.
- Published or contributed to AWS data architecture patterns or blueprints.
Data Engineer in Dartford employer: Response Informatics
As a leading employer in the tech industry, we offer Data Engineers an exceptional opportunity to work at the forefront of data architecture and engineering. Our collaborative work culture fosters innovation and creativity, while our commitment to employee growth ensures that you will have access to continuous learning and development opportunities. Located in a vibrant tech hub, we provide a dynamic environment where your contributions directly impact cutting-edge AI/ML projects, making your work both meaningful and rewarding.
StudySmarter Expert Advice🤫
We think this is how you could land Data Engineer in Dartford
✨Tip Number 1
Network like a pro! Reach out to your connections in the data engineering field and let them know you're on the hunt for a role. Attend meetups or webinars related to AWS, OpenSearch, or data engineering to meet potential employers and learn about job openings.
✨Tip Number 2
Show off your skills! Create a portfolio showcasing your projects, especially those involving S3, Glue, and Athena. This will give you an edge during interviews and help you demonstrate your hands-on experience with the tools mentioned in the job description.
✨Tip Number 3
Prepare for technical interviews by brushing up on your Python and PySpark skills. Practice coding challenges that focus on data manipulation and orchestration, as these are crucial for the role. We recommend using platforms that simulate real-world scenarios to get you ready.
✨Tip Number 4
Don't forget to apply through our website! We have a streamlined application process that makes it easy for you to showcase your qualifications. Plus, it gives us a chance to see your application directly and consider you for the role.
We think you need these skills to ace Data Engineer in Dartford
Some tips for your application 🫡
Tailor Your CV:Make sure your CV is tailored to the Data Engineer role. Highlight your experience with AWS, S3, Glue, and OpenSearch, as these are key for us. Use specific examples that showcase your hands-on skills and achievements.
Craft a Compelling Cover Letter:Your cover letter is your chance to shine! Explain why you're passionate about data engineering and how your background aligns with our needs. Don’t forget to mention any relevant projects or experiences that demonstrate your expertise.
Showcase Your Technical Skills:In your application, be sure to highlight your technical skills, especially in metadata modelling and data orchestration. We love seeing concrete examples of how you've tackled challenges in previous roles, so don’t hold back!
Apply Through Our Website:We encourage you to apply through our website for the best chance of getting noticed. It’s super easy, and it helps us keep track of your application. Plus, we can’t wait to see what you bring to the table!
How to prepare for a job interview at Response Informatics
✨Know Your Tech Inside Out
Make sure you’re well-versed in the technologies mentioned in the job description, especially AWS services like S3, Glue, and OpenSearch. Brush up on your Python skills and be ready to discuss your hands-on experience with vector embedding workloads.
✨Prepare for Technical Deep-Dives
Expect to dive deep into technical discussions during the interview. Prepare to explain your past projects, particularly those involving metadata schema design and data orchestration. Be ready to showcase how you’ve tackled challenges in previous roles.
✨Showcase Your Problem-Solving Skills
Think of specific examples where you identified gaps in existing systems or processes and how you addressed them. This could relate to architecture reviews or tuning Glue jobs. Highlight your analytical thinking and how it led to successful outcomes.
✨Familiarise Yourself with AI/ML Use Cases
Since the role involves authoring data export formats for AI/ML, brush up on relevant use cases and be prepared to discuss how you would approach these tasks. Understanding the implications of your work in a broader context will impress your interviewers.
