Responsibilities
- Design, develop, and maintain metadata-driven data pipelines using Azure Data Factory and Databricks.
- Build and implement end-to-end metadata frameworks, ensuring scalability and reusability.
- Optimize data workflows leveraging SparkSQL and Pandas for large-scale data processing.
- Collaborate with cross‑functional teams to integrate data solutions into enterprise architecture.
- Implement CI/CD pipelines for automated deployment and testing of data solutions.
- Ensure data quality, governance, and compliance with organizational standards.
- Provide technical leadership and take complete ownership of assigned projects.
Technical Skills Required
- Azure Data Factory (ADF): Expertise in building and orchestrating data pipelines.
- Databricks: Hands‑on experience with notebooks, clusters, and job scheduling.
- Pandas: Advanced data manipulation and transformation skills.
- SparkSQL: Strong knowledge of distributed data processing and query optimization.
- CI/CD: Experience with Azure DevOps, Git, or similar for automated deployments.
- Metadata-driven architecture: Proven experience designing and implementing metadata frameworks.
- Programming: Proficiency in Python and/or Scala for data engineering tasks.
Seniority level
- Associate
Employment type
- Contract
Job function
- Information Technology
- IT Services and IT Consulting
Referrals increase your chances of interviewing at Response Informatics by 2x.
Location: Salford, England, United Kingdom.
Senior Data Engineer in Salford employer: Response Informatics
As a Field Service Engineer in Glasgow, you will join a dynamic team that values innovation and excellence in network installation and support. Our company fosters a collaborative work culture, offering extensive training and development opportunities to help you grow your skills in cutting-edge technologies. With a commitment to employee well-being and a supportive environment, we ensure that our engineers thrive both professionally and personally while working on exciting projects.
