• Led a three-engineer team shipping a Triton Python backend fronting a vLLM-served LLM: p50 of 5 to 10 ms per signal, and a concurrency refactor that took sustained throughput from about 8 to 80 requests per second on the same hardware.
  • Wrote the gRPC layer that lets any pipeline of the C++ speech framework behind Nova Sonic ship as a containerized service in under 300 lines of integration code; adopted org-wide, alongside a static service registry of my design.
  • Led a RAG personalization effort for speech recognition, cutting LLM round trips per request from three to two and moving retrieval off the critical path.
  • Consolidated a 500k-line C++ engine’s cloud and on-device builds into one source tree across 12 toolchains, replacing fork-and-patch releases that took days to weeks.
  • Authored production AWS CDK infrastructure for served models, migrated 10k+ metrics to CDK-managed CloudWatch, and built a pipeline publishing 200+ live A/B dashboards.

I joined Amazon in late 2021 as a C++ engineer on Alexa’s speech recognition stack, and I’ve followed that stack through every reorg since: the infrastructure team for the classical ASR engine, the framework team that built the speech-to-speech LLM that became Nova Sonic, and now the team adapting that LLM for production speech workloads. Along the way I’ve been an infrastructure engineer, a framework engineer, and a tech lead, mostly in C++ and Python.

LLM Inference

The work I’d lead with. A voice assistant needs more from its speech model than a transcript: how confident it is, what language was spoken, what tone it carried. I led a three-engineer team building an NVIDIA Triton Python backend that sits in front of a vLLM-served LLM and derives confidence, sentiment and language ID from the model’s own token-level outputs, teacher-forcing a task token after the transcript and reading sentiment and language off the softmax, with an ONNX model over those outputs for confidence and a forced-alignment model alongside on the transcript. We took it from prototype to production, at p50 latencies of about 5 to 10 ms per signal, and a concurrency refactor I led took sustained throughput from roughly 8 to 80 requests per second on the same hardware.

On the same model I led a personalization effort using retrieval-augmented generation: a user’s own contacts, device names and music are pulled from their catalogs with approximate-nearest-neighbor search and injected into the LLM’s prefill context, so the model transcribes their vocabulary instead of guessing at it.

The C++ Framework Under Nova Sonic

Before the LLM work I spent most of my time inside the C++ node-graph framework that Amazon’s speech services, including Nova Sonic, are built on. I wrote the framework’s generic gRPC wrapper, which lets any pipeline ship as a containerized service in a few hundred lines of integration code and became the org’s standard deployment pattern. I built its service locator, a static-initialization factory registry that lets consumers fetch node builders without depending on the implementing library, also adopted org-wide. And I did the Protobuf and gRPC integration that let generated service stubs be consumed as ordinary libraries across a large, many-toolchain codebase.

Developer Tooling

I like building tools for the team I’m on. At Amazon the two I’m proudest of are a Python provisioning system that deploys the full Nova Sonic inference stack, several Triton model containers plus the C++ orchestrator, onto a single developer machine, which took a fifteen-person team from sharing a couple of hand-built demo environments to everyone having their own; and an LLM-agent-driven on-call workflow that packages our CloudWatch queries, production traces and agent-agnostic instruction documents behind one CLI command, cutting the first half hour of an investigation to a couple of minutes. The instruction documents work unmodified across Claude Code, Kiro and Amazon’s internal agent.

Infrastructure

I authored the production AWS CDK infrastructure under our served models: ECS deployments, IAM, S3, and Route 53 DNS firewall protections. Earlier, on the speech engine’s infrastructure team, I migrated over ten thousand Alexa metrics to CDK-managed CloudWatch, built a pipeline that publishes two hundred-plus live A/B experiment dashboards, and stood up cloud CI/CD for a roughly 500k-line C++ codebase built across a dozen toolchains.