GPU Performance Engineer

Yandex is hiring a Senior GPU Performance Engineer to optimize GPU utilization, develop diagnostics tools, and implement modern solutions for ML infrastructure. Remote or hybrid work.

Responsibilities: - Improve GPU utilization efficiency by forming hypotheses and researching ways to increase efficiency, implementing and deploying the most profitable solutions. - Provide recommendations and best practices for performance improvement to maximize GPU infrastructure. - Optimize and profile: find bottlenecks in performance and eliminate them using profilers, optimize memory access, kernels, latency, and throughput. - Develop diagnostics tools: create and improve tools for quick identification and resolution of infrastructure issues affecting GPU utilization efficiency, stability, and speed of GPU computations (for both training and inference). - Research and implement modern solutions: study the latest approaches to organizing infrastructure for training and inference, evaluate their effectiveness, and implement them in real projects. - Analyze architecture, test, and integrate: closely interact with developers, ML engineers, and system architects. Participate in evaluating hardware solutions and propose improvements for future GPU generations, develop testing plans, create benchmarks, and analyze performance regressions.

Requirements: - Knowledge of Python and experience in system programming, developing libraries or frameworks. - Experience with PyTorch and distributed training via torch.distributed. - Applying parallelization approaches including data parallelism, tensor parallelism, pipeline parallelism, expert parallelism for distributed inference or training. - Interest in LLM and MLOps: understanding tasks and challenges related to operating large models in production. - Experience with NVIDIA GPUs and CUDA: understanding GPU architecture, developing or optimizing algorithms for CUDA, using Nsight, nvprof or similar tools. - Ability to optimize GPU application performance and improve GPU utilization efficiency. - Ability to analyze profiles and performance metrics. - Ability to read and optimize complex code. - Effective teamwork and willingness to share knowledge. Nice to have: - Proficiency in C/C++ or similar low-level languages. - Experience with RL training libraries for LLM: veRL, slime, NeMo-RL, SkyRL, and others. - Experience with inference libraries: vLLM, SGLang, and TRTLLM. - Experience optimizing for real production workloads, working with low-latency or real-time systems.

Work format: remote or hybrid.