At a Glance
- Tasks: Design and develop high-performance data pipelines using Databricks, PySpark, and SQL.
- Company: Join a globally recognised financial solutions provider with a focus on innovation.
- Benefits: Enjoy a flexible hybrid work model, comprehensive benefits, and education reimbursement.
- Other info: Explore career development opportunities in a stable, international company.
- Why this job: Make an impact by driving cloud migration and optimising data solutions for top investment banks.
- Qualifications: 5-7 years of data engineering experience with strong Databricks and cloud migration skills.
The predicted salary is between 60000 - 80000 £ per year.
Crisil Integral IQ delivers solutions and analytics to top financial institutions, driving strategic transformation, risk optimization, and operational excellence. Our offerings across research, risk, lending, analytics and operations have empowered clients to navigate complex markets, mitigate risks and unlock new opportunities. Our domain expertise, innovative solutions, and future-ready technologies such as AI and data science give clients the confidence to accelerate growth and achieve sustainable competitive advantage. Our globally diverse workforce operates in the Americas, Asia-Pacific, Europe, Australia and the Middle East.
We are looking for a Databricks Senior Developer who will join our Team supporting one of the biggest investment banks.
Key responsibilities:- Design and Develop Data Pipelines: Build scalable, high-performance data pipelines using Databricks, PySpark, and SQL. Develop and optimize ETL/ELT workflows for batch and streaming data processing.
- Lead Cloud Migration Initiatives: Drive and support the migration of on-premises data platforms to cloud environments (preferably Azure). Refactor legacy data pipelines and ensure seamless transition to modern cloud-based architectures. Convert and optimize legacy SQL dialects into Spark SQL for improved performance and scalability.
- Data Platform Engineering: Develop and manage data solutions within the Databricks Lakehouse architecture. Ensure reliability, scalability, and performance of data processing workloads in production environments.
- Collaboration & Stakeholder Engagement: Work closely with data architects, analysts, and business stakeholders to understand data requirements and deliver solutions. Provide technical guidance and mentorship to junior engineers.
- Data Quality & Governance: Implement data quality frameworks, validation rules, and monitoring mechanisms. Ensure data accuracy, consistency, and compliance across pipelines.
- CI/CD and DevOps Practices: Design and maintain CI/CD pipelines for data engineering workflows. Apply best practices for version control, automated testing, and deployment.
- Workflow Orchestration: Design, implement, and maintain orchestration pipelines using tools such as Databricks Workflows, Apache Airflow, dbt. Monitor job performance and troubleshoot failures proactively.
- Cloud Integration & Optimization: Integrate Databricks with Azure services (e.g., Data Factory, Azure Functions, storage accounts). Optimize cluster configurations, job execution, and cost utilization.
- Performance Tuning & Optimization: Analyze and optimize queries, Spark jobs, and data models for efficiency and reduced processing time. Implement best practices for partitioning, caching, and data storage formats.
- Documentation & Best Practices: Maintain clear technical documentation for pipelines, workflows, and system architecture. Promote coding standards, reusable components, and best engineering practices.
- 5 to 7 years of data engineering experience, with at least 3+ years of hands-on Databricks experience in production environments.
- Migration Expertise: Proven experience in migrating on-premises data systems to cloud platforms.
- Strong proficiency in Python, PySpark, and SQL.
- Experience in converting legacy SQL dialects to Spark SQL is a plus.
- Familiarity with cloud computing concepts, preferably within Azure ecosystems.
- Experience integrating cloud functions or runtime services with data platforms is desirable.
- Solid understanding of CI/CD, data quality frameworks, and orchestration tools such as Databricks Workflows, Airflow, or dbt.
- Stable Employment in an International Company: Be part of a globally recognized organization that values stability and growth.
- Secure Employment Contract: Enjoy the peace of mind that comes with a solid employment contract.
- Flexible Hybrid Work Model: Experience the best of both worlds with our hybrid work model, allowing you to work from the office just 3 times a week and enjoy the flexibility of remote work the rest of the time.
- Comprehensive Benefits Package.
- Employee Referral Program: Help us grow our team and get rewarded through our Employee Referral Program.
- Education Reimbursement Policy: Invest in your future with our education reimbursement policy, supporting your continuous learning and professional development.
- Career Development Opportunities: Explore numerous internal and external opportunities to advance your career within the Finance Domain. We are committed to your growth and success.
Databricks Senior Developer in City of London employer: Crisil Integral IQ
Crisil Integral IQ is an exceptional employer, offering a stable and secure work environment within a globally recognised organisation. With a flexible hybrid work model, comprehensive benefits, and a strong commitment to employee growth through education reimbursement and career development opportunities, we empower our team members to thrive in their roles while supporting one of the largest investment banks. Join us to be part of a diverse workforce that values innovation and collaboration in driving strategic transformation and operational excellence.
StudySmarter Expert Advice🤫
We think this is how you could land Databricks Senior Developer in City of London
✨Get Involved in Data Science Meetups
Tap into local data science meetups or workshops to connect with fellow enthusiasts and professionals. These events are goldmines for networking, and sometimes even lead directly to job openings at companies like Crisil Integral IQ!
✨Show Off Your Projects
Start building a public portfolio showcasing your data science projects on platforms like GitHub or personal websites. Highlight unique analyses or models you've developed. This not only demonstrates your skills but also gets your name out there for roles like Databricks Senior Developer at Crisil Integral IQ.
✨Leverage Professional Networks
Join professional bodies related to data science, like the Data Science Society or similar organisations. Getting involved can lead to mentorship opportunities and insider knowledge about full-time positions at companies like Crisil Integral IQ.
✨Apply Directly through Our Website
When you find a suitable opening like Databricks Senior Developer at Crisil Integral IQ, make sure to apply directly through our website. It gives you an edge and shows you're keen to join our team. Plus, who doesn’t love a direct application? It’s easier than navigating through job boards!
We think you need these skills to ace Databricks Senior Developer in City of London
Some tips for your application 🫡
Show Off Your Projects:In the world of data science, your projects can speak volumes about your skills. Make sure to showcase a few key projects in your CV or portfolio, especially those that highlight your ability to work with data sets, build models, or use relevant tools like Python, R, or SQL. Don’t forget to include links to any GitHub repositories if applicable!
Quantify Your Achievements:Employers love numbers! When drafting your CV, highlight your achievements with quantifiable results. For instance, mention how your data analysis led to a certain percentage increase in efficiency or revenue at a previous job or project. These details can really make your application pop!
Craft a Tailored Cover Letter:For a full-time role at Crisil Integral IQ, your cover letter should reflect your passion for data science and your excitement about the specific projects or values of the company. Dive into why you’re a good fit, how your skills align with their needs, and any unique perspectives you can bring to the team.
Stand Out with Relevant Courses and Certifications:Although experience talks, relevant courses or certifications can be your ticket to impressing hiring managers at Crisil Integral IQ. Mention any standout courses you've completed that equipped you with essential skills, such as machine learning certifications or data visualisation courses. This shows your commitment to continuously developing your skills in the field!
How to prepare for a job interview at Crisil Integral IQ
✨Brush Up on Your Statistics
For a data science role, we need to seriously sharpen our statistics skills. Get ready to tackle technical questions on probability distributions, hypothesis testing, and regression analysis. These are often the bread and butter of data science interviews, so don't just skim over them!
✨Showcase Your Projects
Prepare a killer portfolio showcasing your data science projects. We should include details about the datasets used, the tools and techniques applied, and the impact of your findings. If we can walk them through a particularly challenging project or a cool visualisation that had real-world implications, it’ll really make us stand out!
✨Get Comfortable with Python and R
Most data science positions require us to be proficient in programming languages like Python and R. We should practice common libraries like pandas, NumPy, and scikit-learn, and be ready for live coding exercises or algorithm questions. Showing off our coding chops can really impress the interviewers at Crisil Integral IQ!
✨Prepare for Case Studies
Expect to encounter real-world case studies during the interview. We might be asked how we’d approach a data problem or analyse a dataset to extract insights. It's essential to think out loud and demonstrate our problem-solving process so that the interviewer can see our logical thinking in action.
