Find a job 
Featured Companies 
Featured Universities 
For Companies At a Glance
- Tasks: Design and build large-scale Python web scraping systems for dynamic, high-security environments.
- Company: Join a leading tech recruiter in Data & AI with a focus on innovation.
- Benefits: Enjoy a competitive salary, remote work flexibility, and opportunities for professional growth.
- Why this job: Leverage AI tools to tackle complex scraping challenges and make a real impact.
- Qualifications: Strong Python skills and experience with scraping protected platforms required.
- Other info: Collaborative environment with excellent career advancement opportunities.
The predicted salary is between 36000 - 60000 £ per year.
We are looking for a Senior Python Scraping Engineer to design, build, and operate large-scale, resilient web scraping systems, with a strong focus on Google and Google-like search environments. This role sits at the intersection of data engineering, reverse engineering, and systems reliability, and is critical to delivering accurate, timely, and trustworthy data at scale.
You will work on complex scraping problems where naïve approaches do not work - dynamic rendering, sophisticated anti-bot defences, frequent markup changes, and high-volume data collection. We are leaders in adopting AI-assisted and agentic coding workflows. This role is ideal for someone who believes in leveraging AI tools to accelerate development and wants to deepen their skills in this area.
What You'll Do
- Design, build, and maintain large-scale Python scraping systems that operate reliably against Google and other highly defended websites.
- Develop scrapers for dynamic, JavaScript-heavy pages using browser automation and hybrid scraping approaches.
- Continuously adapt to changes in page structure, request flows, and anti-automation mechanisms.
- Engineer robust extraction pipelines that prioritise data accuracy, consistency, and observability.
- Implement and optimise proxy, fingerprinting, and request-routing strategies to maximise stability and throughput.
- Monitor scrape health, detect failures early, and debug production issues quickly and methodically.
- Optimise performance, cost, and latency across scraping infrastructure.
- Collaborate closely with data engineers, data scientists, and product teams to ensure scraped data is usable and trusted.
- Document scraping logic, architectural decisions, and operational runbooks for long-term maintainability.
Required Skills and Experience
- Strong professional experience with Python, including production-grade scraping systems.
- Deep, hands-on experience scraping Google or similarly protected platforms.
- Solid understanding of HTTP, TLS, cookies, headers, redirects, and browser networking behaviour.
- Experience with browser automation frameworks such as Playwright, Selenium, Puppeteer, or equivalents.
- Strong knowledge of HTML parsing, DOM traversal, and data extraction techniques.
- Proven experience handling rate limiting, CAPTCHAs, IP rotation, and bot detection systems.
- Comfortable designing and operating asynchronous and concurrent scraping architectures.
- Experience running scraping systems at scale in cloud environments.
- Excellent debugging skills and the ability to reason about complex, non-obvious failure modes.
- Hands-on experience with headless browser fingerprinting and anti-bot evasion techniques.
Python Web Scrapping Engineer - Competitive salary - Remote in City of London employer: Tenth Revolution Group
Contact Detail:
Tenth Revolution Group Recruiting Team
StudySmarter Expert Advice 🤫
We think this is how you could land Python Web Scrapping Engineer - Competitive salary - Remote in City of London
✨Tip Number 1
Network like a pro! Reach out to folks in the industry, join relevant online communities, and attend meetups. You never know who might have the inside scoop on job openings or can refer you directly.
✨Tip Number 2
Show off your skills! Create a portfolio showcasing your web scraping projects. Include examples of how you've tackled complex problems, especially with dynamic sites. This will give potential employers a taste of what you can do.
✨Tip Number 3
Prepare for interviews by brushing up on common scraping challenges and solutions. Be ready to discuss your experience with tools like Selenium or Puppeteer, and how you've overcome obstacles like CAPTCHAs or rate limiting.
✨Tip Number 4
Apply through our website! We’re always on the lookout for talented individuals like you. Plus, it’s a great way to ensure your application gets seen by the right people.
We think you need these skills to ace Python Web Scrapping Engineer - Competitive salary - Remote in City of London
Some tips for your application 🫡
Tailor Your CV: Make sure your CV highlights your experience with Python and web scraping. We want to see how you've tackled complex scraping problems, especially with Google-like platforms. Customising your CV for this role will show us you're serious about joining our team!
Show Off Your Skills: Don’t just list your skills; demonstrate them! Include specific examples of projects where you’ve used browser automation frameworks or handled anti-bot defences. We love seeing real-world applications of your expertise!
Keep It Clear and Concise: When writing your application, clarity is key. Use straightforward language and avoid jargon unless it’s relevant. We appreciate a well-structured application that gets straight to the point—this shows us you can communicate effectively!
Apply Through Our Website: We encourage you to apply directly through our website. This not only streamlines the process but also ensures your application reaches us quickly. Plus, it’s super easy—just a few clicks and you’re done!
How to prepare for a job interview at Tenth Revolution Group
✨Know Your Python Inside Out
Make sure you brush up on your Python skills, especially in relation to web scraping. Be prepared to discuss your experience with production-grade scraping systems and any challenges you've faced while scraping Google or similar platforms.
✨Familiarise Yourself with Anti-Bot Techniques
Since this role involves dealing with sophisticated anti-bot defences, it’s crucial to understand techniques like IP rotation, CAPTCHA handling, and browser fingerprinting. Be ready to share specific examples of how you've tackled these issues in past projects.
✨Showcase Your Problem-Solving Skills
Prepare to discuss complex scraping problems you've encountered and how you resolved them. Highlight your debugging skills and your ability to reason through non-obvious failure modes, as this will demonstrate your critical thinking and adaptability.
✨Emphasise Collaboration and Documentation
This role requires close collaboration with data engineers and product teams. Be sure to mention any experiences where you’ve worked in a team setting and how you documented your scraping logic and architectural decisions for long-term maintainability.
