We are seeking an experienced Data Engineer to join our team. As a Data Engineer, you will play a crucial role in designing, building, and maintaining our data infrastructure and systems. You will collaborate with cross-functional teams to ensure efficient and reliable data pipelines, implement data integration solutions, and support data-driven initiatives. This position requires strong technical expertise, problem-solving skills, and a deep understanding of data engineering principles. Roles & Responsibilities:
- Design and Develop Data Infrastructure: Architect, build, and optimize scalable data infrastructure, including data warehouses, data lakes, and ETL/ELT processes.
- Data Integration: Implement efficient and reliable data integration pipelines, ensuring the smooth flow of data from various sources to target systems.
- Data Modelling and Schema Design: Design and implement data models and schemas that support business requirements and optimize data retrieval and analysis.
- Data Quality and Governance: Develop and implement data quality checks, data validation processes, and data governance practices to maintain data integrity and accuracy.
- Performance Optimization: Identify performance bottlenecks, optimize data processing and storage systems, and improve overall data pipeline efficiency.
- Data Security: Ensure data privacy and security by implementing appropriate access controls, encryption techniques, and data protection measures.
- Collaboration: Collaborate with cross-functional teams, including data scientists, analysts, and software engineers, to understand their data needs and provide technical support.
- Documentation and Monitoring: Create and maintain technical documentation, data dictionaries, and monitoring systems to track data pipeline performance and identify issues.
- Troubleshooting and Issue Resolution: Investigate and resolve data-related issues, including data quality issues, system failures, and performance degradation.
- Data Architecture and Technology Evaluation: Stay updated with industry trends and emerging technologies in data engineering and evaluate their potential impact and suitability for the organization.
Applicants must be eligible to work in the specified location
Key Skills: Data, ETL/ELT, cloud, Shell/Bash, Data Modelling,
- 5+ years of experience in data engineering, data integration, or related roles.
- Strong proficiency in database management/development and working experience with relational databases, data modelling, and schema design.
- Proficiency in data integration tools, ETL/ELT frameworks, and data pipeline orchestration tools.
- Strong programming skills in languages such as Python/Java.
- Understanding of data warehousing concepts, dimensional modelling, and data governance practices.
- Knowledge of data security and privacy principles, as well as experience implementing data security measures.
- Working experience with cloud platforms (eg, AWS, Azure, GCP) and their data services (eg, AWS Glue, Azure Data Factory, GCP Dataflow).
- Having knowledge of Docker & Kubernetes will be added advantage
- Have good understanding on Shell/Bash Scripting
- Good analytical skills to present and understand statistical trends and forecast
- Excellent problem-solving skills, attention to detail, and the ability to work effectively in a team environment
#J-18808-Ljbffr
Data Engineer [Data, ETL/ELT, cloud, Shell/Bash, Data Modelling] in Sheffield employer: GIOS Technology Limited
As a leading employer in the tech sector, we offer a dynamic work environment that fosters innovation and collaboration. Our hybrid model across major UK cities allows for flexibility while providing access to cutting-edge projects in observability and data engineering. We prioritise employee growth through continuous learning opportunities and a culture that values diverse expertise, making us an ideal choice for professionals seeking meaningful and impactful careers.
