Software Development Engineer – AI/ML Networking Disaggregated Inference, Annapurna Labs , Elastic Collectives
Amazon
E-Commerce Services
Cupertino, CA - United States
Senior Specialist / Project Manager
Experteer Overview
In this role you will design and optimize the data-movement path that transfers KV caches and activations across accelerators and memory hierarchies. You’ll operate at the intersection of hardware, network, and software to push performance toward the hardware limits. You will collaborate with chip, runtime, and model teams to ship features for large-scale AI deployments. This is a fast-paced, mentorship‑driven environment at the frontier of AI infrastructure, offering meaningful impact on latency and throughput for production systems.
Compensation / Benefits
- flexible working hours
- mentorship and career growth
- health insurance
- 401(k) matching
- paid time off
- RSUs
Responsibilities
- Build and optimize low-level data-movement software for KV cache and activations across accelerators, servers, and heterogeneous memory
- Profile workloads to identify bottlenecks and close the gap to peak hardware performance
- Work across the stack from network transport to inference frameworks, collaborating with chip, runtime, and model teams
- Deliver features that scale to large clusters and customers in production for large AI models
Key requirements
- 3+ years of non-internship software development experience
- 2+ years in design/architecture of new or existing systems (reliability, scaling)
- Experience programming in at least one software language
- Strong C/C++ and Linux proficiency; memory management and writing fast code
Description
In this role you will design and optimize the data-movement path that transfers KV caches and activations across accelerators and memory hie…
Take your next career step
1M+ top positions worldwide with salary benchmarks
Be discreetly found and contacted by headhunters
Exclusively for senior-level professionals and executives
Already a member?

