Job Description
The financial domain/banking is a must in recent projects.
Key Responsibilities:
- Infrastructure & Deployment Pipelines: Build, automate, and maintain CI/CD pipelines for data ingestion and processing workflows.
- Set up efficient deployment pipelines for data pipelines in development and production.
- Collaboration with Architecture & Engineering Teams: Collaborate with the Domain Architect and development teams to implement cloud-based analytics platforms.
- Provide DevOps support and expertise to optimize the development lifecycle.
- Monitoring & Optimization: Implement tools and processes for continuous monitoring, logging, and performance tuning of data pipelines.
- Develop fault-tolerant and scalable solutions for managing terabytes of data daily.
- Environment Automation: Utilize Infrastructure as Code (IaC) to ensure repeatable and consistent infrastructure deployment (e.g., Terraform, ARM templates).
- Optimize resource allocation and ensure cost-efficiency in cloud operations.
- Documentation: Develop and maintain comprehensive documentation of DevOps workflows, processes, and systems.
- Security & Compliance: Ensure deployment infrastructure meets data security and compliance standards.
- Implement role-based access control and security policies for development and production environments.
Required Skills & Qualifications:
- 5+ years as Azure DevOps
- Strong experience with Azure DevOps tools (Azure Pipelines, Repos, etc.).
- Experience with: Azure Data Lake, Azure Event Hubs, Azure Databricks
- Infrastructure as Code: Terraform or ARM templates
- CI/CD pipeline design for data workflows
- Performance monitoring and tuning for data engineering
- Understanding of data governance and role-based access control (RBAC)
- Cost optimization in cloud environments
Contact Detail:
Vallum Associates Recruiting Team