At a Glance
- Tasks: Build innovative data pipelines and ensure top-notch data quality for AI evaluations.
- Company: Mercor, a leader in AI partnerships and data engineering.
- Benefits: Competitive hourly rate, opportunities for promotion, and a collaborative work environment.
- Other info: Remote work available; strong contributors can advance quickly.
- Why this job: Join a cutting-edge team and make a real impact in the AI field.
- Qualifications: 3+ years in data engineering, expertise in dbt, Airflow, or warehouse design.
Mercor is partnering with leading AI labs to advance frontier agent evaluations in data engineering. As a Data Engineering Expert, you'll build long-horizon pipeline tasks that mirror the work you already do, each paired with a deterministic rubric that grades agent performance against verifiable ground truth. Tasks need to have checkable answers; no open-ended essays, no subjective judgment calls.
Expect to build scenarios across:
- Pipelines: ETL/ELT and dbt models that produce a specified output table, incremental logic with defined watermark behavior
- Orchestration and quality: Airflow/Dagster DAGs that pass a test suite, data quality tests with known pass/fail cases
- Warehouse design: schemas matching a defined contract, performance targets tied to a measured query-time budget
Who we're looking for:
- BS or MS in CS or related; 3+ years in data engineering or analytics engineering
- Expertise in one or more of the following: dbt model development, pipeline orchestration (Airflow, Dagster, Prefect), warehouse design (Snowflake, BigQuery, Redshift, Databricks), data quality and testing
- Comfortable reading and producing data engineering artifacts: dbt models, DAGs, schema docs, data contracts, test suites
- Clear written communication; can articulate reasoning step by step and encode it into deterministic rubrics
- Located in the United States or Canada
Compensation: $90–$125/hr depending on domain depth and prior experience. Strong contributors are promoted based on task quality and throughput.
Senior Data Engineer: Pipelines, Orchestration & Quality employer: Obsidian
Mercor is an exceptional employer that fosters a collaborative and innovative work culture, particularly for Senior Data Engineers. With a focus on cutting-edge AI projects, employees benefit from competitive compensation, opportunities for professional growth, and the chance to work with leading experts in the field. Located in the United States or Canada, Mercor offers a dynamic environment where your contributions are recognised and rewarded, making it an ideal place for those seeking meaningful and impactful work.
StudySmarter Expert Advice🤫
We think this is how you could land Senior Data Engineer: Pipelines, Orchestration & Quality
✨Tip Number 1
Network like a pro! Reach out to folks in the data engineering field on LinkedIn or at meetups. We all know that sometimes it’s not just what you know, but who you know that can help you land that dream job.
✨Tip Number 2
Show off your skills! Create a portfolio showcasing your best projects, especially those involving pipelines and orchestration. We want to see your work in action, so make it easy for potential employers to see what you can do.
✨Tip Number 3
Prepare for interviews by practising common data engineering scenarios. We recommend running through some mock interviews with friends or using online platforms. The more comfortable you are with the technical questions, the better you'll perform!
✨Tip Number 4
Don’t forget to apply through our website! We’ve got loads of opportunities waiting for talented data engineers like you. Plus, applying directly shows your enthusiasm and commitment to joining our team.
We think you need these skills to ace Senior Data Engineer: Pipelines, Orchestration & Quality
Some tips for your application 🫡
Tailor Your CV:Make sure your CV is tailored to the Senior Data Engineer role. Highlight your experience with pipelines, orchestration, and data quality. We want to see how your skills match what we're looking for!
Showcase Your Projects:Include specific projects that demonstrate your expertise in dbt model development and pipeline orchestration. We love seeing real examples of your work, so don’t hold back!
Clear Communication is Key:Since clear written communication is crucial for this role, ensure your application is well-structured and easy to read. Use bullet points where necessary and articulate your reasoning step by step.
Apply Through Our Website:We encourage you to apply through our website for a smoother process. It helps us keep track of applications and ensures you don’t miss any important updates from us!
How to prepare for a job interview at Obsidian
✨Know Your Pipelines
Make sure you can talk confidently about your experience with ETL/ELT processes and dbt models. Be ready to discuss specific projects where you've built pipelines, focusing on the outputs and any challenges you faced.
✨Master Orchestration Tools
Familiarise yourself with Airflow and Dagster, as these are key to the role. Prepare examples of how you've used these tools to create DAGs and ensure data quality, and be ready to explain your thought process behind your designs.
✨Showcase Your Communication Skills
Since clear written communication is crucial, practice articulating your reasoning step by step. You might be asked to explain a complex concept or a project, so think about how you can break it down into understandable parts.
✨Understand Data Quality Testing
Be prepared to discuss your approach to data quality and testing. Have examples ready that demonstrate how you've implemented known pass/fail cases in your previous work, and how you ensured the integrity of your data.
