Senior Machine Learning Engineer - Orchestration - Applied AI & LLMs (x/f/m)

About the Company

Responsibilities

• ● Define and own the evaluation strategy for our AI agentic system - metrics, protocols, datasets, and tooling
• ● Implement and maintain automated evaluation pipelines to monitor model quality, safety, and alignment across iterations
• ● Run systematic experiments to assess reasoning, factuality, robustness, and user experience
• ● Collaborate closely with model developers and research scientists to provide insights and drive iterative improvement
• ● Contribute to research and internal knowledge sharing on LLM evaluation methodologies and best practices
• ● Life at Doctolib Tech
• ● Our solutions are built on a single fully cloud-native platform that supports web and mobile app interfaces, multiple languages, and is adapted to country and healthcare specialty requirements.
• ● Our stack is composed of Rails, TypeScript, Java, Python, Kotlin, Swift, and React Native.
• ● We leverage AI ethically across our products to empower patients and health professionals. Discover our AI vision here .
• ● Want to learn more about our tech culture and environment Visit the Doctolib Tech site .

Requirements

• ● Before you read on: if you don't have the exact profile described below, but you feel this job description matches your skill set, we still encourage you to apply.
• ● You'll be a great fit if you:
• ● Hold an MSc or PhD in Computer Science, Machine Learning, Data Science, or a related field
• ● Have 7+ years of hands-on experience working with large language models (e.g., GPT, Claude, Llama, or BERT-like architectures)
• ● Have proven experience evaluating agentic or reasoning systems (e.g., autonomous agents, tool-using LLMs, dialogue systems, or task-oriented assistants)
• ● Have a strong track record in experiment design, metric definition, and evaluation automation
• ● Are able to bridge research and production, influencing both modeling and product decisions
• ● Have excellent communication skills, a collaborative mindset, and are fluent in English
• ● It would be fantastic if you:
• ● Have experience in the clinical or medical domain and sensitivity to ethical or regulatory challenges in healthcare AI

Benefits

• ● Free comprehensive health insurance (basic package) for you and your children
• ● days of paid vacation per year, plus up to 14 days of RTT
• ● Free mental health and coaching services through our partner Moka.care
• ● Work from abroad for up to 10 days per year thanks to our flexibility days policy
• ● Lunch vouchers (Swile card) worth 8.50 per working day, with 4.50 covered by Doctolib
• ● A subsidy from the work council to refund part of the membership to a sport club or a creative class
• ● % reimbursement of your public transport subscription
• ● ParentCare Program: Enjoy full salary coverage (100%) during your first month of birth leave, and 75% during the second, covered by Doctolib
• ● Enrollment in Doctolib's long-term employee value sharing plan called DoctoGrowth
• ● For caregivers and workers with disabilities, a package including an adaptation of the remote policy, extra days off for medical reasons, and psychological support
• ● Relocation support in case of international mobility
• ● Access to the best AI tools for coding, development and dedicated training
• ● Our interview process
• ● Recruiter Interview
• ● Technical Deep Dive
• ● Data System Design
• ● Behavioral Interview
• ● At least one reference check
• ● We want your experience to be clear, respectful, and transparent. Learn more about our hiring process on our candidate experience page .
• ● Permanent position