• Led a three-engineer team shipping a Triton Python backend fronting a vLLM-served LLM: p50 of 5 to 10 ms per signal, and a concurrency refactor that took sustained throughput from about 8 to 80 requests per second on the same hardware.
• Wrote the gRPC layer that lets any pipeline of the C++ speech framework behind Nova Sonic ship as a containerized service in under 300 lines of integration code; adopted org-wide, alongside a static service registry of my design.
• Led a RAG personalization effort for speech recognition, cutting LLM round trips per request from three to two and moving retrieval off the critical path.
• Consolidated a 500k-line C++ engine’s cloud and on-device builds into one source tree across 12 toolchains, replacing fork-and-patch releases that took days to weeks.
• Authored production AWS CDK infrastructure for served models, migrated 10k+ metrics to CDK-managed CloudWatch, and built a pipeline publishing 200+ live A/B dashboards.

I joined Amazon in late 2021 as a C++ engineer on Alexa’s speech recognition stack, and I’ve followed that stack through every reorg since: the infrastructure team for the classical ASR engine, the framework team that built the speech-to-speech LLM that became Nova Sonic, and now the team adapting that LLM for production speech workloads. Along the way I’ve been an infrastructure engineer, a framework engineer, and a tech lead, mostly in C++ and Python.

LLM Inference

The work I’d lead with. A voice assistant needs more from its speech model than a transcript: how confident it is, what language was spoken, what tone it carried. I led a three-engineer team building an NVIDIA Triton Python backend that sits in front of a vLLM-served LLM and derives those signals from the model’s own token-level outputs, teacher-forcing a task token after the transcript and reading the label off the softmax. No extra models to train or serve. We took it from prototype to production, at p50 latencies of about 5 to 10 ms per signal, and a concurrency refactor I led took sustained throughput from roughly 8 to 80 requests per second on the same hardware.

On the same model I led a personalization effort using retrieval-augmented generation: a user’s own contacts, device names and music are pulled from their catalogs with approximate-nearest-neighbor search and injected into the LLM’s prefill context, so the model transcribes their vocabulary instead of guessing at it.

The C++ Framework Under Nova Sonic

Before the LLM work I spent most of my time inside the C++ node-graph framework that Amazon’s speech services, including Nova Sonic, are built on. I wrote the framework’s generic gRPC wrapper, which lets any pipeline ship as a containerized service in a few hundred lines of integration code and became the org’s standard deployment pattern. I built its service locator, a static-initialization factory registry that lets consumers fetch node builders without depending on the implementing library, also adopted org-wide. And I did the Protobuf and gRPC integration that let generated service stubs be consumed as ordinary libraries across a large, many-toolchain codebase.

Developer Tooling

I like building tools for the team I’m on. At Amazon the two I’m proudest of are a Python provisioning system that deploys the full Nova Sonic inference stack, several Triton model containers plus the C++ orchestrator, onto a single developer machine, which took a fifteen-person team from sharing a couple of hand-built demo environments to everyone having their own; and an LLM-agent-driven on-call workflow that packages our CloudWatch queries, production traces and agent-agnostic instruction documents behind one CLI command, cutting the first half hour of an investigation to a couple of minutes. The instruction documents work unmodified across Claude Code, Kiro and Amazon’s internal agent.

Infrastructure

I authored the production AWS CDK infrastructure under our served models: ECS deployments, IAM, S3, and Route 53 DNS firewall protections. Earlier, on the speech engine’s infrastructure team, I migrated over ten thousand Alexa metrics to CDK-managed CloudWatch, built a pipeline that publishes two hundred-plus live A/B experiment dashboards, and stood up cloud CI/CD for a roughly 500k-line C++ codebase built across a dozen toolchains.