Location: Remote, North America

Language: Excellent verbal and written English communication skills are required

About the Opportunity
Are you passionate about advancing the future of artificial intelligence? We are seeking an Applied Research Scientist, LLM Evaluation & Post-Training to help shape how next-generation large language models are evaluated, refined, and trusted at scale. This is an opportunity to join a global technology organization at the forefront of AI innovation, where your research will directly influence the development of cutting-edge generative AI solutions.

Working alongside AI researchers, machine learning engineers, and language data specialists, you will design rigorous evaluation frameworks, conduct impactful research, and translate scientific insights into scalable solutions. Your work will help improve model quality, reliability, and performance while contributing to meaningful advancements in responsible AI.

What’s In It for You
Join a collaborative and research-driven environment where curiosity, experimentation, and innovation are encouraged. You'll have the opportunity to work alongside industry-leading AI professionals, contribute to emerging best practices in generative AI, and tackle complex challenges with real-world impact. This role offers the chance to influence both customer-facing solutions and the future direction of AI evaluation methodologies while continuing to grow your technical expertise.

Your Responsibilities
• You'll lead research initiatives focused on LLM evaluation methodologies and evaluation-driven post-training strategies for large language and multimodal models.
• You'll design and execute statistically rigorous experiments to assess how evaluation frameworks influence model performance and fine-tuning outcomes.
• You'll develop benchmark datasets, scoring methodologies, human and automated evaluation protocols, and robustness testing strategies.
• You'll analyze model behaviour, identify failure patterns, and recommend improvements to evaluation frameworks and model quality.
• You'll collaborate closely with AI researchers, machine learning engineers, and language data scientists to build scalable evaluation and post-training workflows.
• You'll engage with technical stakeholders to provide expert guidance on evaluation strategies, research findings, and implementation recommendations.
• You'll contribute technical documentation, reusable research assets, and thought leadership that advances best practices in LLM evaluation and generative AI.

Skills and Qualifications
• 5+ years of applied research experience in machine learning or artificial intelligence, with significant experience working with large language models or foundation models.
• PhD in Computer Science, Artificial Intelligence, Machine Learning, Statistics, Applied Mathematics, or a related quantitative discipline is strongly preferred. A Master's degree with equivalent experience will also be considered.
• Demonstrated expertise in LLM evaluation, benchmarking, alignment, post-training methodologies, or model quality research.
• Strong foundation in experimental design, statistical analysis, and scientific research methodologies.
• Advanced Python programming skills with experience building research experiments, evaluation pipelines, and analytical tools.
• Experience with modern machine learning frameworks such as PyTorch, Hugging Face Transformers, TensorFlow, or JAX.
• Exceptional communication skills with the ability to present complex technical findings, assumptions, and recommendations to both research and engineering audiences.

Compensation
This position offers a competitive base salary ranging from approximately CAD $246,550 to CAD $317,000 annually, depending on experience, skills, and qualifications.

Note from the Hiring Manager
"We're looking for someone who enjoys asking difficult research questions and turning them into practical solutions. If you're excited by experimentation, collaboration, and shaping how AI systems are evaluated and improved, we'd love to meet you."

Why Partner with Altis
If you've never worked with a staffing agency before, we make it easy. We work with top employers across Canada who have great jobs to fill, each vetted and verified by our team. When you apply for a job with Altis, we get to know you as a candidate and learn what your strengths are. Then, if you're a solid match, we handle all the logistics, advocating for you as a candidate for the role, providing access to coaching and connecting you directly with the hiring manager. And rest assured, all our services are free of cost for candidates.

We appreciate the time and effort all applicants invest in their submissions. Please note that only candidates shortlisted for this role will be contacted directly. However, your profile will remain under consideration for future opportunities that align with your experience and career goals.All qualified applicants will receive fair consideration for employment. We welcome individuals of all backgrounds, experiences, and identities including those who identify as women, members of racialized groups, Indigenous Peoples, persons with disabilities, and 2SLGBTQIA+ communities. If you require an accommodation, please review our accessibility policy and reach out to our accessibility officer with any questions. Our human recruiters review all applications and always make the final hiring decision. On occasion, we also use AI-assisted tools to help review applications.