Role description:
We are seeking a highly skilled Senior Azure Data Engineer to design, develop, and execute enterprise-scale data migration strategies. The ideal candidate must have hands-on expertise in formulating structured data migration payloads, leveraging the Microsoft Azure Data stack. Associate will build high-performance pipelines and data migration engines using Azure Synapse Analytics, Azure Data Factory (ADF), PySpark, Python, and robust SQL scripting.
Key responsibilities:
Pipeline & Payload Architecture
• Design migration payloads: Construct optimized, transactional data packages and schema mappings
• Orchestrate ELT/ETL: Build fault-tolerant automated ingestion pipelines using Azure Data Factory.
• Implement CDC: Configure Change Data Capture (CDC) logic in Azure Synapse Analytics for incremental delta loads
Computational Processing & Transformations
• Write PySpark notebook jobs: Develop data transformations, file partitioning, and data cleansing using Spark DataFrames.
• Create custom scripts: Build native object-oriented utility applications in Python for API payload validation.
• Optimize data schemas: Author advanced SQL queries, stored procedures, and analytical views to model Dedicated SQL Pools.
Technical Core
• Cloud Orchestration Tools: Azure Data Factory (ADF) and Azure Synapse Pipelines.
• Data Warehousing Warehouses: Azure Synapse Analytics (Serverless & Dedicated SQL Pools).
• Languages & Environments: Python, PySpark, Spark SQL, and T-SQL.
• Cloud Infrastructure & Storage: Azure Data Lake Storage Gen2 (ADLS), Azure SQL Databases.
• Framework Formats: Comprehensive file format processing including , Delta Lake, and JSON.
Key skills/knowledge/experience:
• Extensive hands on experience on Azure Synapse Analytics, Azure Data Factory (ADF), PySpark, Python, and robust SQL scripting.
• Minimum Experience Needed: 5+ years of active cloud data engineering experience.
Good to Have
Official Microsoft DP-203: Azure Data Engineer Associate professional certification prefential
