Job Details

ML Analyst wanted for Alice's LLM special projects team to improve LLM agents, analyze model behavior, and build evaluation processes. Focus on new projects involving model quality evaluation and data preparation. Apply on our website.

We are looking for an ML Analyst to help develop Alice's LLM agents: improve response quality, analyze model behavior, and build evaluation and training processes. We are launching two new projects that require forming processes for evaluating model quality and preparing data. Details will be shared at the final stage! What tasks await you: • Improving the quality of LLM agents Study model responses, identify typical errors and growth areas, improve agent behavior through prompt engineering and context engineering. • Evaluation and comparison of models Participate in the acceptance of new model versions, analyze experiment results, and compare models by quality metrics. • Working with data and labeling Formulate labeling tasks, prepare data baskets, control labeling quality, and prepare training and learn datasets. • Development of quality evaluation tools Apply statistical methods for hypothesis testing and work with automatic response evaluation systems (e.g., LLM-as-a-Judge).

We expect you to: • Have a solid command of Python • Have experience with text data or NLP tasks • Be able to analyze model quality metrics • Have prepared and structured datasets • Be attentive to details, capable of understanding complex systems

Why it's good here: Yandex is a community. There are sports clubs, a book club, and an esports community. These are not all the bonuses - a full list is here: https://yandex.ru/jobs/pages/benefits?utm_campaign=ya_nanimaet