AI Framework Software Engineer

The AI Framework Software Engineer will join the software design task force for Artificial Intelligence (AI) hardware IPs (e.g., NPUs). In this role, you will be expected to:

1. 01

   Participate in the development and performance optimization of core modules in AI frameworks (e.g., PyTorch, vLLM, SGLang) tailored for domain-specific hardware backends.

2. 02

   Optimize model deployment, focusing on tuning algorithms such as model parallelism, data parallelism, asynchronous communication, model quantization, and kernel fusion.

3. 03

   Track cutting-edge LLM inference technologies (e.g., PagedAttention, Speculative Decoding) to deliver full-lifecycle solutions and drive their production rollout. Desirable Skills and Experience:

1. 01

   Expert programming skills in C/C++ and Python; handson experience with CUDA, cuTile, Tilelang, or Triton is a strong plus.

2. 02

   Indepth knowledge of the source code of at least one major AI framework or inference engine (e.g., PyTorch, vLLM, TensorRT, TVM); familiarity with adding new backend devices to these frameworks is a plus.

3. 03

   Deep understanding of AI theory and core concepts, with a strong intrinsic interest in inference engine acceleration.

4. 04

   Proven hands-on experience in software integration across the entire development lifecycle.