Inference Systems Engineer · LLM Serving & Admission Scheduling · Distributed Infrastructure

7+ years engineering resilient distributed systems at Microsoft and Amazon. Now specializing in ML-driven scheduling architectures that optimize LLM inference efficiency — from request admission to cluster resource allocation.

• vLLM Contributor — PR #41952 (Under Review): Fixed preemption ordering in PriorityRequestQueue to minimize KV cache recompute overhead.

• Clairvoyant — Go-based reverse proxy reducing HOL blocking in serial LLM backends (Ollama, llama.cpp) via ML-predicted SJF admission scheduling. 70–76% P50 latency reduction under burst load; 17% steady-state at ρ=0.74. arXiv preprint in prep.

• ACO Scheduler — Ant Colony Optimization job scheduler with LSTM-based load prediction and heterogeneous GPU/CPU/ARM64 affinity routing. +28% utilization gain validated against Alibaba and Google Borg traces. P99 latency <10ms.

• ServiceScope — LLM-powered AST dependency mapper for microservice blast-radius analysis. 190 files/sec, 0% inference failure, validated on Django (2,886 files). Zero external API calls.

Python · Go · Java · Bash
Expanding into: C++ / CUDA

LLM inference serving · Request scheduling & admission control · Distributed systems · ML for systems

📫 linkedin.com/in/aravindsundaresan