One Build for Cloud and Device: Consolidating a 500k-Line C++ Engine

The legacy ASR engine, roughly 500k lines of C++, was built by its owners with GCC and plain CMake for the cloud. The on-device team consumed it as a library across about 7 ARM toolchains from the Android NDK plus 4 x86 gcc/clang variants, through a Conan-based build layer.
Before: on-device releases were hand-curated snapshots of the upstream engine, maintained in forks of every ASR package and re-merged periodically. Each release took days at minimum and often weeks, as toolchain-specific breakage surfaced and needed patches.
Fix: point all engine code at a single virtual build-system package. In the cloud dependency universe it resolves to bare CMake; in the device universe it resolves to the Conan layer, which drives the same CMake underneath. About 40 packages converted; one source tree builds for all 12 toolchains.
Kept it that way with a pre-merge analyzer that compiles every pull request against the device toolchains before it can land, so device compatibility is checked at merge time rather than discovered at release time.

My first role at Amazon was on the team building on-device speech recognition. We consumed the organization’s ASR engine, roughly 500k lines of C++, as a library, and the engine’s owners built it for exactly one world: cloud hosts, GCC, plain CMake. Our world was a spread of ARM targets with toolchains mostly from the Android NDK, a handful of x86 variants across gcc and clang, and a Conan-based build layer adapted to Amazon’s internal package system. Twelve toolchains in all, counting the cloud one, and the seam between the two build systems was where the pain lived.

What a device release cost

Because the two builds were separate, an on-device release was a hand-curated event. The device team kept forks of every ASR package, periodically merged upstream into them, and then worked through whatever broke. The recurring offenders: headers that libstdc++ pulls in transitively but libc++ does not, so code that compiled by accident under GCC failed to find memcpy or std::function under clang; dependencies declared in the cloud universe that simply did not exist in the device one; and floating-point differences between the cloud BLAS and the device one under Kaldi, which exposed tests with tolerances tuned to x86 and, more than once, a bare float == float. Each of those became a patch in the fork, and the whole process mirrored the cloud release process in overhead. A release took days at minimum and often weeks. When the engine’s infrastructure team was formed with a mandate to consolidate, this was the first thing to remove.

The build system is just another package

The property that made this solvable is a feature of Amazon’s internal build tooling: the build system a package uses is itself a declared dependency, and every build happens inside a pinned dependency universe in which each package name resolves to exactly one version. Think of a lockfile that spans an entire distribution rather than one project. A package declares “I build with X,” and which X that is gets decided by the universe it is built in, not by the package.

So we introduced a virtual build-system package, in the spirit of a Debian virtual package or a Conan provides, and converted all of the engine’s code to declare it as their build system. In the cloud universe, that name resolves to bare CMake, and nothing changes for the engine’s owners. In the device universes, it resolves to the Conan layer. Since Conan does not replace CMake but drives it, the device build became a thin layer over the same CMake the cloud build uses rather than a parallel build with its own drift. As a team we converted about 40 packages. The forks went away, and the same commit that builds for the cloud builds for every device target.

Keeping it consolidated

A single build is only worth something if it stays green, and the engine’s developers were not going to install seven ARM toolchains. Amazon’s pre-merge checks include a dry-run release, where a pull request is compiled by the internal build system against its dependency universe before it can merge. We wrote and maintained an additional analyzer that runs every ASR pull request through that same dry run against dedicated device universes, where the build system resolves to the Conan layer. A change that compiles for the cloud but breaks an NDK target fails the pull request, with the toolchain named, instead of failing a release weeks later. That analyzer is what turned “consolidated once” into “consolidated,” and it is the cloud CI/CD work I refer to elsewhere: the device layer ran on AWS CodeBuild, unusually for Amazon-internal code, which is how it came to touch CodeBuild, Conan and CMake at once.