At a Glance
- Tasks: Design and optimise enterprise-scale data platforms using Databricks Lakehouse.
- Company: Join a leading tech firm focused on innovative data solutions.
- Benefits: Attractive salary, flexible working options, and opportunities for professional growth.
- Other info: Collaborative environment with strong career advancement potential.
- Why this job: Be at the forefront of data technology and make a significant impact.
- Qualifications: Expertise in Databricks, Apache Spark, and cloud platforms required.
The predicted salary is between 63000 - 77000 £ per year.
We are seeking a Senior Databricks Architect to design, implement, and optimize enterprise-scale data platforms using the Databricks Lakehouse Platform .
The ideal candidate will have deep expertise in Databricks architecture, Apache Spark, Delta Lake, data engineering, cloud platforms, and enterprise data solutions.
Key Responsibilities
- Design and architect Databricks Lakehouse solutions for large-scaleenterprise data platforms.
- Lead end-to-end Databricks implementation, including architecture, development, deployment, and optimization.
- Define Databricks workspace architecture, clusterstrategies, security models, and operational standards.
- Design and developscalable data pipelinesusing Databricks, Apache Spark, Py Spark, SQL, and Delta Lake .
- Implement data ingestionframeworks supporting batchand real-timedata processing.
• Design Delta Lake architectures including
- Delta tables
- Medallion architecture (Bronze/Silver/Gold layers)
- Data quality frameworks
- Schema evolution andoptimization
• Perform Databricks performancetuning
- Cluster optimization
- Spark job optimization
- Query tuning
- Cost optimization
• Implement Databricks governanceand security using
- Unity Catalog
- Access controls
- Data lineage
- Audit capabilities
- Lead migration projectsfrom legacy dataplatforms to Databricks Lakehouse.
- Define best practicesfor Databricks CI/CD, Dev Ops, andenvironment management.
- Collaborate with dataengineers, data scientists, analysts, and enterprisearchitects.
- Provide technical guidance, architecture documentation, and design reviews.
Skills & Experience
- Strong experience in dataengineering, big data, or solution architecture.
- Strong hands-onexperience with Databricks Platform .
• Expert knowledge of
- Apache Spark
- Py Spark / Scala
- Delta Lake
- SQL
- Data Lakehouse architecture
- ETL/ELT development
- Experience designing anddelivering enterprise Databrickssolutions.
• Strong experience withcloud platforms
- Azure Databricks (preferred)
- AWS Databricks
- Google Cloud Databricks
• Experience with
- Databricks Workflows
- Jobs orchestration
- Notebooks
- Repos
- Cluster management
- Unity Catalog
- Preferred Skills
- Databricks Certified Data Engineer Professional /Databricks Certified Architect.
• Experience with
- MLflow
- Databricks Machine Learning
- Feature Store
- Model deployment
- MLOps
• Knowledge of streamingtechnologies
- Structured Streaming
- Kafka
- Experience with datagovernance, compliance, and enterprise securityframeworks.
- Key Competencies
- Strong Databricks architectureand implementation expertise.
- Ability to designscalableand cost-efficient Lakehouse solutions.
- Strong troubleshooting andperformance optimizationskills.
- Ability to leadtechnical discussions withenterprise stakeholders.
- #J-18808-Ljbffr
Senior Databricks Solution Architect in Norwich employer: UNISON Group
Unison Group is an exceptional employer located in the vibrant city of Norwich, offering a dynamic work culture that fosters innovation and collaboration. Employees benefit from comprehensive professional development opportunities, competitive compensation, and a supportive environment that values work-life balance. Joining Unison Group means being part of a forward-thinking team dedicated to leveraging cutting-edge technology in data solutions, making a meaningful impact in the industry.
