AWS Emulator, built entirely by AI agents, passes 56.6% of its own fidelity tests

AWS has released AWS Emulator, a local stand-in for the AWS application programming interface (API) that developers can run on a laptop, a dev server or in a continuous integration pipeline. The AWS command-line interface and software development kits connect to it instead of AWS itself, so no AWS account and no service fee are required, according to the project’s documentation covered by MarkTechPost.

The notable part of the release is who built it. MarkTechPost reports that autonomous agents wrote and reviewed all of the emulator’s code, built from 438 AWS API models covering more than 200 services, with one Rust programming-language crate per service. AWS engineer Matthew Goldberg, cited by MarkTechPost, says the project began as a prompt to AWS’s Kiro agent system, which produced a workable Amazon SQS emulator in about 15 minutes before AWS split the broader effort across three teams of agents: ones that write tests from public API models, ones that replay those tests to find failures, and merge gates that check builds, code review, security and fidelity. Goldberg reports hundreds of concurrent agents fixing thousands of issues daily.

AWS publishes a scores.json file with every release measuring how closely the emulator matches real AWS behavior. MarkTechPost’s tally of the November 11 release, version v1.0.322, found 24,243 of 42,857 scored tests passing, or 56.6%, with 80 tests skipped. At the level of individual steps within those tests, 49,810 of 68,007 passed, or 73.2%. Results vary widely by service: AWS KMS passed 93.9% of its tests and Amazon SQS 89.8%, while AWS CloudFormation passed only 15.8%. Among core services tracked by MarkTechPost, S3 reached 79.1%, DynamoDB 77.3%, Lambda 46.4% and EC2 41.5%.

The emulator runs on Linux via RPM or Debian packages, or through Docker on macOS and Windows with a 2 gigabyte memory limit, according to AWS’s installation guide. Resources exist only in memory and disappear when the emulator stops. AWS’s documentation warns that the tool does not evaluate identity and access management permissions the way real AWS does, so it should not be used for security testing, and that it has been verified only in the us-west-2 region, with global services checked in us-east-1.

What this means

The emulator targets fast local and CI testing loops rather than production-equivalent behavior, and the published fidelity scores make clear where that holds up and where it does not. Control-plane operations like KMS and SQS fare well, while CloudFormation and cross-service workflows, which passed only 41.9% in MarkTechPost’s count, lag far behind.

MarkTechPost’s comparison table places AWS Emulator against LocalStack, Moto and Floci, noting it needs no vendor authentication token and is free under a CC0 license, unlike LocalStack’s paid tiers. What the reports do not establish is how the agent-built codebase will be maintained over time, or whether the fast release cadence, five versions between November 8 and 11, will hold as coverage gaps in services like CloudFormation get addressed.