magnifier icon

Software Development Engineer – AI/ML Networking Disaggregated Inference, Annapurna Labs , Elastic Collectives

Amazon

Amazon

E-Commerce Services

Cupertino, CA - United States

Senior Specialist / Project Manager

Experteer Overview

In this role you will design and optimize the data-movement path that transfers KV caches and activations across accelerators and memory hierarchies. You’ll operate at the intersection of hardware, network, and software to push performance toward the hardware limits. You will collaborate with chip, runtime, and model teams to ship features for large-scale AI deployments. This is a fast-paced, mentorship‑driven environment at the frontier of AI infrastructure, offering meaningful impact on latency and throughput for production systems.

Compensation / Benefits

  • flexible working hours
  • mentorship and career growth
  • health insurance
  • 401(k) matching
  • paid time off
  • RSUs

Responsibilities

  • Build and optimize low-level data-movement software for KV cache and activations across accelerators, servers, and heterogeneous memory
  • Profile workloads to identify bottlenecks and close the gap to peak hardware performance
  • Work across the stack from network transport to inference frameworks, collaborating with chip, runtime, and model teams
  • Deliver features that scale to large clusters and customers in production for large AI models

Key requirements

  • 3+ years of non-internship software development experience
  • 2+ years in design/architecture of new or existing systems (reliability, scaling)
  • Experience programming in at least one software language
  • Strong C/C++ and Linux proficiency; memory management and writing fast code

Description

In this role you will design and optimize the data-movement path that transfers KV caches and activations across accelerators and memory hie…
For members onlyMobile Experteer Ad

Take your next career step

  • 1M+ top positions worldwide with salary benchmarks

  • Be discreetly found and contacted by headhunters

  • Exclusively for senior-level professionals and executives

Already a member?