Описание: Simple Machines is a global, independent technology consultancy that designs and builds modern data platforms, intelligent systems, and bespoke software across data engineering, software engineering, and AI. It helps enterprises, scale-ups, and government turn complex data into products, platforms, and actionable decisions.
Задачи
- Own the end-to-end architecture of modern, cloud-native data platforms
- Design scalable data ecosystems using data mesh, data products, and data contracts
- Make architectural decisions across ingestion, storage, processing, and access layers
- Ensure platforms are secure, compliant, and production-grade by design
- Design and deliver cloud-native data platforms using Databricks, Snowflake, AWS, and GCP
- Integrate with client systems to enable scalable, consumer-oriented data access
- Build and optimise batch and real-time pipelines
- Work with streaming and event-driven technologies such as Kafka, Flink, Kinesis, and Pub/Sub
- Orchestrate workflows using Airflow, Dataflow, and Glue
- Process and transform large datasets using Spark and Flink
- Design production-ready systems
- Work across relational, NoSQL, and analytical data stores
- Optimise storage formats and access patterns
- Implement secure, compliant data solutions with security by design
- Embed governance while maintaining developer velocity
- Work directly with clients to understand problems and shape solutions
- Translate business needs into pragmatic engineering decisions
- Act as a trusted technical advisor
- Set engineering standards, patterns, and best practices across teams
- Review designs and code, providing technical direction and mentorship
- Improve data quality, testing, observability, and operational excellence
Требования
- Strong Python and SQL skills
- Deep experience with Spark and modern data platforms such as Databricks and Snowflake
- Solid understanding of cloud data services in AWS or GCP
- Demonstrated ownership of large-scale data platform architectures
- Strong data modelling and architectural decision-making skills
- Ability to balance performance, cost, and complexity trade-offs
- Experience building and operating large-scale data pipelines in production
- Experience with multiple storage technologies and formats
- Infrastructure-as-code experience with Terraform or Pulumi
- Experience with CI/CD pipelines using tools such as GitHub Actions or ArgoCD
- Experience with data testing and quality frameworks such as dbt, Great Expectations, or Soda
- Experience in consulting or professional services environments
- Strong consulting instincts and ability to challenge assumptions and guide clients toward better outcomes
- Ability to mentor senior engineers and influence technical culture
Условия
No conditions specified
#J-18808-Ljbffr
data engineer in data platforms in London employer: Enfint
As a leading innovator in AI products for major publishers, our company offers an inspiring work environment where creativity and technology intersect. We prioritise employee growth through continuous learning opportunities and foster a collaborative culture that values diverse perspectives. Located in a vibrant city, we provide competitive salaries, relocation support, and the chance to make a real impact in the media landscape.
