Find a job 
Featured Companies 
Featured Universities 
For Companies At a Glance
- Tasks: Evaluate and enhance multilingual datasets for large language models, focusing on cultural nuances.
- Company: Join a forward-thinking team dedicated to improving AI through cultural and linguistic insights.
- Benefits: Flexible working hours and the chance to work remotely while making an impact.
- Why this job: Be part of a mission to ensure AI understands diverse cultures and languages better.
- Qualifications: Native English speaker with a Master's degree and experience in LLM evaluation preferred.
- Other info: Engage 25 to 30 hours per week in a dynamic and innovative environment.
We are looking for linguistically and culturally aware professionals to support the evaluation and enhancement of multilingual prompt-response datasets for large language models (LLMs). This role involves rubric design, evaluation of translations and model outputs, prompt creation, and red teaming focused on identifying and surfacing cultural nuances and biases in LLM behaviour.
Key Responsibilities:
- Rubric Definition & Prompt Evaluation
- Update rubric definitions with region/language-specific examples to ensure cultural and linguistic relevance.
- Identify the need for additional rubrics tailored to specific languages or regional contexts.
- Review prompts translated from English into the target language and revise where translations appear unnatural or inaccurate.
- Writing of thoughtful prompts which can test the cultural awareness of LLM models.
- Rate prompt-response pairs using a standardized evaluation template based on rubrics and provide detailed justifications to base the findings.
- Document problematic outputs and annotate them with clear explanations of rubric violations or cultural insensitivities.
Required Qualifications:
- Native proficiency in the target language (English) and deep familiarity with cultural norms in the corresponding region.
- Experience in LLM evaluation, content moderation, or linguistic QA preferred.
- Strong attention to detail with the ability to identify subtle issues in language use, tone, and cultural references.
- Comfortable working in spreadsheets and evaluation templates.
- Master’s degree in relevant stream.
Preferred Qualifications:
- Prior experience with prompt engineering or LLM testing.
- Familiarity with tools such as Gemini, ChatGPT or similar LLM platforms.
- Ability to clearly articulate reasoning behind rubric ratings or prompt edits.
Required Engagement: 25 to 30 hours per week.
Content Writer employer: Innodata Inc.
Contact Detail:
Innodata Inc. Recruiting Team
StudySmarter Expert Advice 🤫
We think this is how you could land Content Writer
✨Tip Number 1
Familiarise yourself with the latest trends in LLM evaluation and cultural nuances. This will not only help you understand the role better but also allow you to engage in informed discussions during interviews.
✨Tip Number 2
Network with professionals in the field of LLM evaluation and content moderation. Attend relevant webinars or join online forums where you can share insights and learn from others' experiences.
✨Tip Number 3
Prepare examples of your previous work that demonstrate your attention to detail and understanding of cultural contexts. Being able to discuss specific instances where you've identified biases or improved translations will set you apart.
✨Tip Number 4
Stay updated on the tools and platforms used in LLM testing, such as Gemini and ChatGPT. Familiarity with these tools will show your commitment to the role and your ability to hit the ground running.
We think you need these skills to ace Content Writer
Some tips for your application 🫡
Understand the Role: Before applying, make sure you fully understand the responsibilities of a Content Writer for LLM Evaluation. Familiarise yourself with rubric design, prompt evaluation, and cultural nuances in language.
Tailor Your CV: Highlight your relevant experience in LLM evaluation, content moderation, or linguistic QA. Make sure to showcase any specific projects or roles that demonstrate your attention to detail and cultural awareness.
Craft a Compelling Cover Letter: Write a cover letter that reflects your passion for language and culture. Discuss your familiarity with the target language and how your background aligns with the job requirements, particularly in rubric definition and prompt evaluation.
Showcase Relevant Skills: In your application, emphasise your proficiency in using tools like Gemini or ChatGPT. Mention any experience with prompt engineering or LLM testing, as these are preferred qualifications that can set you apart.
How to prepare for a job interview at Innodata Inc.
✨Showcase Your Linguistic Skills
Make sure to highlight your native proficiency in English and any other languages you may know. Discuss your familiarity with cultural norms and how they influence language use, as this is crucial for the role.
✨Demonstrate Attention to Detail
Prepare examples that showcase your ability to identify subtle issues in language use, tone, and cultural references. This could involve discussing past experiences where you successfully navigated these challenges.
✨Familiarise Yourself with LLMs
Research large language models and their evaluation processes. Be ready to discuss your experience with tools like Gemini or ChatGPT, and how you've used them in previous roles or projects.
✨Prepare Thoughtful Prompts
Think about how you would create prompts that test cultural awareness in LLMs. Bring a few examples to the interview to demonstrate your understanding of rubric design and prompt evaluation.
