Staff Machine Learning Engineer, LLM Inference Optimization

Nebius is seeking a Staff Machine Learning Engineer specializing in LLM Inference Optimization to join their team in Zurich, Switzerland. This role is crucial for developing and deploying efficient, high-performance inference endpoints for AI models.

Nebius AI Cloud is at the forefront of providing advanced cloud infrastructure for AI developers, enabling the creation and deployment of generative AI applications and accelerating scientific discoveries through optimized ML model training and execution.

If you are passionate about optimizing AI inference and want to shape the future of AI cloud infrastructure, apply now!

Company Overview

Nebius is pioneering a new era in cloud infrastructure for the global AI economy, offering a comprehensive AI cloud platform that supports developers and enterprises from data and model training to production deployment. Built by engineers for engineers, Nebius tackles complex challenges in compute, storage, networking, and applied AI, with a global footprint and a team of over 1,500 experts.

Quick Details

• Salary Range: Not Specified
• Job Type: Full-time
• Experience: 5 Year+
• Qualifications: Bachelors Degree

Key Responsibilities

• Own a platform-level optimization pillar such as quantization, speculative decoding, or KV-cache optimization.
• Create reusable evaluation, benchmarking, and rollout frameworks across multiple model families.
• Optimize LLM and VLM endpoints for latency, throughput, memory efficiency, and cost per token.
• Deploy, configure, benchmark, and extend inference engines like vLLM, SGLang, or TensorRT-LLM.
• Build and productionize model-compression workflows, including quantization and distillation.
• Implement advanced techniques like speculative decoding and continuous batching.

Requirements

• Strong Python and PyTorch engineering skills.
• Hands-on experience deploying or optimizing LLM, VLM, or high-throughput transformer inference systems.
• Practical knowledge of at least one modern inference stack (e.g., vLLM, SGLang, TensorRT-LLM).
• Strong understanding of transformer inference bottlenecks (KV cache, attention, memory bandwidth, etc.).
• Ability to quantitatively reason about performance and cost tradeoffs.
• Strong communication and collaboration skills.

Benefits

• Competitive compensation
• Career growth and learning opportunities
• Flexibility and ownership
• Collaborative and innovative culture
• Opportunity to work on impactful AI projects
• International environment and talented teams