About the Role

Principal Software Engineer role at Microsoft's AI Performance team focusing on optimizing AI inference performance for large language models. Responsibilities include benchmarking, debugging, and optimizing LLM performance on GPUs and Microsoft hardware. The role involves working across the AI software stack to enable efficient deployment of state-of-the-art models.

Key Responsibilities

• Identify and drive improvements to end-to-end inference performance of OpenAI and other state-of-the-art LLMs
• Measure, benchmark performance on Nvidia/AMD GPUs and first party Microsoft silicon
• Optimize and monitor performance of LLMs and build SW tooling for performance insights
• Design, implement, and test functions or components for AI/DNN/LLM frameworks and tools
• Communicate and collaborate with internal and external partners

Required Skills & Qualifications

Must Have:

• Bachelor's Degree in Computer Science or related technical field AND 6+ years technical engineering experience
• 4+ years' practical experience working on high performance applications and performance debug/optimization on CPUs/GPUs
• Coding experience in languages including C, C++, C#, Java, JavaScript, or Python
• Hands-on technical role requiring software design and development skills

Nice to Have:

• Master's Degree in Computer Science or related technical field AND 8+ years technical engineering experience
• Technical background in software engineering principles, computer architecture, GPU architecture, HW neural net acceleration
• Experience in end-to-end performance analysis and optimization of state-of-the-art LLMs, HPC applications
• Experience in DNN/LLM inference and experience in DL frameworks such as PyTorch, Tensorflow, or ONNX Runtime
• Familiarity with CUDA, ROCm, Triton

Benefits & Perks

• Industry leading healthcare