About the Role

Senior Software Engineer role on Microsoft's AI Frameworks team, focused on building end-to-end software stack for Microsoft's first-party AI accelerators. The position involves deep optimization across software and hardware stacks to enable state-of-the-art LLM training and inference. Work includes collaborating with hardware architects, compiler teams, and model experts to unlock the full potential of custom silicon.

Key Responsibilities

• Implement and optimize components of the AI software stack targeting Microsoft's 1P AI accelerators
• Collaborate with hardware, compiler, and model teams to develop high-performance solutions
• Contribute to framework integration work for PyTorch and ONNX with custom hardware backends
• Analyze performance bottlenecks and propose optimizations across framework, runtime, and hardware layers
• Write clean, maintainable, and well-tested code, and participate in design/code reviews

Required Skills & Qualifications

Must Have:

• Bachelor's Degree in Computer Science or related technical field AND 4+ years technical engineering experience
• Proficiency in C++ and/or Python with solid understanding of software engineering fundamentals
• Experience with modern large language model (LLM) serving technologies including distributed execution and inference optimization
• Strong problem-solving skills and ability to collaborate across teams

Nice to Have:

• Experience with deep learning frameworks such as PyTorch, TensorFlow, or ONNX
• Experience with GPU computing (CUDA programming, GPU kernel optimization, performance tuning)
• Familiarity with AI accelerator software stacks, graph compilers, or kernel libraries
• Understanding of large-scale distributed training or inference systems for LLMs
• Exposure to performance profiling and optimization tools
• Contributions to open-source AI frameworks or compilers

Benefits & Perks

• Industry leading healthcare