Establishing verifiable security: Reproducible builds and AWS Nitro Enclaves

Recent security incidents across blockchain and broader IT sectors underscore the persistent risk of sophisticated attacks on software supply chains and build environments. Reproducible builds offer a powerful mitigation strategy by making sure that software compiled from the same source code and dependencies consistently produces identical binaries, making it possible to detect tampering.

In this post, we show you how reproducible builds enable decentralized verification of source code and build artifacts, creating a robust trust model that doesn’t rely on a single vendor.

We also show what are the requirements to make builds deterministic and reproducible, with a hands-on example of compiling non-trivial software such as a Multi-party computation (MPC) library and how to combine reproducible builds with AWS Nitro Enclaves for cryptographic remote attestation and runtime integrity.

Benefits of reproducible builds

Let’s now explore the benefits of reproducible builds, such as decentralized verification and enhanced transparency, trust and auditability.

Independent decentralized verification of software artifacts

Reproducible builds produce identical binary outputs from the same source code, regardless of where or by whom they’re built. This allows any third party to build from publicly available or shared source code and verify that the resulting binaries match exactly. Such independent verification helps ensure that the software artifact has not been altered or compromised during compilation, packaging, or distribution.

Enhanced transparency, trust, and auditability

Much like open source code invites peer review for bugs or logic flaws, reproducible builds invite verification that distributed binaries match their source. This transparency builds trust among users, developers, auditors and security researchers, reducing reliance on the integrity claims of a single vendor or distributor. Furthermore, reproducible builds significantly improve the auditability of changes because anyone can verify that a binary corresponds exactly to its source code. Audits can focus on reviewing human readable sources rather than opaque machine level binaries.

How to achieve reproducible builds

Before jumping into implementation, it’s important to understand two key concepts: determinism and reproducibility.

Determinism

A deterministic build consistently produces the same output artifacts from the same inputs, eliminating variability caused by timestamps, file order, or randomness.

Reproducibility

Reproducibility goes a step further – it means that anyone can independently recreate build artifacts and verify the output, regardless of where the build takes place. This requires pinned dependencies and build environment control.

High level reproducible build and attestation process

Before we dive deep into the different steps of enabling reproducible and remote attestable builds lets have a look at the high level process:

Source Code and Dependencies
- Stable source code
- Pinned source code dependencies
Controlled and Reproducible Build Environment
- Pinned container base image
- Pinned dependency versions
Deterministic Build Process
- Eliminate sources of non-determinism (timestamps, file order)
- Use reproducible build tools (for example Kaniko)
Reproducible Binary or Container
- Constantly measure and validate using SHA-256 hashes
Integrate with AWS Nitro Enclave
- Convert to Enclave Image File (EIF)
- Read individual Platform Configuration Register (PCR) measurements
Remote Cryptographic Attestation
- Run EIF in Nitro Enclave
- Generate cryptographic attestation document
- Distribute attestation document and remotely verify integrity and environment

Implementation guide

Let’s now see how these theoretical requirements can be translated into technical implementation steps. The high-level implementation guide consists of the following two steps:

Create a controlled build environment.
Eliminate sources of non-determinism and apply continuous verification and measurement.

The foundation of reproducible builds is a consistent, controlled, and portable build environment. The build environment must be identical even when executed on different systems. To achieve portability and consistency, it’s common to use container based build environments. To eliminate sources of variability, the base image version must be pinned down to the hash version, for example ubuntu:22.04@sha256:<digest> instead of ubuntu:latest and the versions of all required dependencies must be pinned down, for example, using apt-get install cmake=3.22.1-1ubuntu1.22.04.2 instead of apt-get install cmake.

Eliminate sources of non-determinism and apply continuous verification and measurement

Reproducible builds require eliminating non-deterministic elements such as timestamps and environmental variability during the build process. Tools such as Kaniko (--reproducible flag) and environment variables such as SOURCE_DATE_EPOCH standardize timestamps and file metadata in container images and compiled binaries.

The programming language and compiler configuration also play a critical role in achieving deterministic and reproducible builds. For example, Go(lang) introduced perfectly reproducible builds with v1.21, eliminating implicit inputs such as build paths or timestamps. C/C++ relies on compiler flags such as -ffile-prefix-map=old=new or -frandom-seed=string. In addition to programming language or compiler specific configuration, all build dependencies (including the compiler itself) must also be explicitly pinned to specific versions.

To measure reproducibility between different environments and configurations, use cryptographic hashes such as SHA-256 to quickly verify if builds are identical – matching hashes confirm reproducibility.

Prerequisites

For this walkthrough, you must have the following prerequisites:

An AWS Account
An AWS Nitro Enclave enabled EC2 parent instance. Refer to the following documentation to learn how to Prepare the enclave-enabled parent instance and how to Install the Nitro Enclaves CLI on Linux and start all required services.
Git and Docker installed on the workstation that you plan to do the walkthrough from

Example of deterministic and reproducible build of complex software

We now guide you through an end-to-end example of how to build complex software in a deterministic and reproducible way. Throughout the steps, we explain critical concepts and configuration items.

Clone the mpc-lib repository using the following command and change into the mpc-lib folder:
```
git clone https://github.com/fireblocks/mpc-lib.git && cd mpc-lib
```

Copy the following text snippet into docker/Dockerfile.reproducible-jammy

FROM ubuntu:jammy@sha256:2b7412e6465c3c7fc5bb21d3e6f1917c167358449fecac8176c6e496e5c1f05f AS builder
ENV DEBIAN_FRONTEND=noninteractive
RUN apt-get update && apt-get install -y \
    build-essential=12.9ubuntu3 \
    cmake=3.22.1-1ubuntu1.22.04.2 \
    libsecp256k1-dev=0.1~20210825-2 \
    libssl-dev=3.0.2-0ubuntu1.19 \
    uuid-dev=2.37.2-4ubuntu3.4 \
    && rm -rf /var/lib/apt/lists/* /var/cache/apt/archives/*
COPY . /usr/src/mpc-lib/
WORKDIR /usr/src/mpc-lib/
RUN mkdir build && cd build && cmake .. && make -j
FROM ubuntu:jammy@sha256:2b7412e6465c3c7fc5bb21d3e6f1917c167358449fecac8176c6e496e5c1f05f AS final
COPY --from=builder /usr/src/mpc-lib/build/src/common/libcosigner.so /usr/local/lib/

Note the following aspects of the Docker file:

The base image has been pinned to hash 2b7412e6465c3c7fc5bb21d3e6f1917c167358449fecac8176c6e496e5c1f05f
All build dependencies are pinned to specific versions such as 12.9ubuntu3 for build-essential
It consists of multiple stages (builder and final) to keep all the build dependencies out of the final image, thus keeping it minimal

Run the following command to build the mpc-lib using that Docker file:
```
docker run --rm -v "$(pwd)":/workspace gcr.io/kaniko-project/executor:v1.23.2 --dockerfile docker/Dockerfile.reproducible-jammy --destination mpc-lib-repro:1 --reproducible --custom-platform=linux/amd64 --no-push --tarPath=/workspace/image.tar
```
Note the following aspects on the provided build command:
- We use Kaniko (pinned to v1.23.2) to build our image. Kaniko provides rootless builds inside a container (the executor). This means that the build container does not require privileged access to a container daemon on the underlying host. Instead, each Dockerfile command is executed with user permissions.
- The --reproducible flag that invokes go-containerregistry’s mutate.Canonical zeros out timestamps and fields in the image’s config JSON like DockerVersion.
- Target architecture is linux/amd64 instead of defaulting to the current host’s architecture.
- Note that other OCI image build tools such as buildkit have similar capabilities respecting SOURCE_DATE_EPOCH, a standardized environment variable designed to help produce reproducible output from tools that implement it.
Run the following command to load the container:
```
docker load -i image.tar
```
You can now see the Docker container using the following command:
```
docker images | grep mpc-lib-repro
```
You should see an output like the following:
```
mpc-lib-repro 1 1e2a520c0c79 N/A 84.2MB
```
Now, you can inspect the image ID sha256 value:
```
docker inspect --format='{{.Id}}' mpc-lib-repro:1
```
The output should be sha256:1e2a520c0c799313ef271b7067a01f781a27dd3e81c4a006655b871ea3bd97fe for Linux/AMD64.

Building for Linux/ARM64 to run on Graviton platforms would return the following value: sha256:fbae58963e63340b83fe36cdbfc43013ce3e274f8863333707aaeecd1ecb7246

It should be the same across time and build environments. If you encounter differences in the sha256 hash value, you can use tools such as diffoscope and diffoci to locate differences in the OCI tarballs or images. Tools such as reprotest automate building the same source code under varying environments and compares the resulting binaries to detect reproducibility issues.

How to make reproducible builds remote attestable

In this section, we first introduce the concept of remote attestation, a mechanism to cryptographically verify both software integrity and runtime environment authenticity across distributed systems. We then give a brief introduction to AWS Nitro Enclaves and their remote attestation services, referred to as cryptographic attestation. Finally, we provide a step-by-step walkthrough of how containerized software can be turned into a remote attestable artifact.

Remote attestation

Attestation provides cryptographic proof that the binary matches the audited source code and that the binary executes in an isolated and verified environment. Remote attestation means that the verification occurs through the network and does not require direct access to the software or its runtime environment.

AWS Nitro Enclaves and cryptographic attestation

AWS Nitro enclaves are fully isolated virtual machines, hardened with no persistent storage, no interactive access, and no external networking. Nitro enclaves offer cryptographic attestation, which allows you to verify the enclave’s identity and to make sure that only authorized code is running inside the enclave. It also offers close integration with AWS Key Management Service (AWS KMS). See Integration with AWS KMS to learn more.

Example of integrating reproducible builds with AWS Nitro Enclaves cryptographic attestation

We now guide you through an end-to-end example of how to convert the reproducible Docker container from the first section into an attestable enclave image file and to enable remote integrity validation using cryptographic attestation.

Run the following command to start an Amazon Linux 2023 container with a specific hash and to install the required dependencies.
```
docker run -it -v "$(pwd)":/workspace -v /var/run/docker.sock:/var/run/docker.sock amazonlinux@sha256:fc7c82b2ba834045bdf454ef0f9e73d6fdf01166e08671037c8ffdaa9de2cac4
```
Inside the container, install the required dependencies:
```
dnf install -y aws-nitro-enclaves-cli-1.4.2-0.amzn2023 aws-nitro-enclaves-cli-devel-1.4.2-0.amzn2023
```
Note the following aspects of the nitro-cli environment:
- The amazonlinux Docker container is pinned to version sha256:fc7c82b2ba834045bdf454ef0f9e73d6fdf01166e08671037c8ffdaa9de2cac4
- nitro-cli version is pinned to v1.4.2
- nitro-cli requires access to the image produced in the previous section. Access can be granted by mapping docker.sock into the container as done in the command -v /var/run/docker.sock:/var/run/docker.sock
Now trigger the nitro-cli build-enclave command:
```
nitro-cli build-enclave --docker-uri mpc-lib-repro:1 --output-file /workspace/mpc-lib-repro.eif
```
Note that the previous command will fail if the Docker container is not available in the Docker registry being pointed to.

You should be able to see an output like the following for the Linux/AMD64 architecture:
```
Start building the Enclave Image...
Using the locally available Docker image...
Enclave Image successfully created.
{
  "Measurements": {
    "HashAlgorithm": "Sha384 { ... }",
    "PCR0": "fac540c30c11a357e1b79a4ff9e22dbd05d31bba4b589fa8f0421357994433904d7d921a21ab616441c609dbc1fe3d6f",
    "PCR1": "4b4d5b3661b3efc12920900c80e126e4ce783c522de6c02a2a5bf7af3a2b9327b86776f188e4be1c1c404a129dbda493",
    "PCR2": "53af4d92520250331676155d1b50cd1356a63c35f9295fd4dc5d7d1df84cce307cf4b774f40c64f696e7ee41bb017668"
  }
}
```
PCR0 for Linux/ARM64 builds would be 2fd0457a02a1d42c9d3af8cbc13fccbcd75dd29bd624b0658abcb4ee7b389a5c4c3d66fe9ee8617840561f0025e403f5.

You could also use eif_build to further customize the EIF build.

After exiting the enclave generation docker container, you will find the mpc-lib-repro.eif file in the local folder.

Production builds can also sign the EIF (nitro-cli sign-eif –eif-path …) by providing the signing certificate and key or AWS KMS key Amazon Resource Name (ARN) to nitro-cli. For a signed EIF, the enclave measurements will provide a PCR8 value, which can be used to identify the signing key.

In addition to PCR8, AWS Nitro enclaves provide PCR3 and PCR4, which are hashes of the parent instance’s AWS Identity and Access Management (IAM) role and Amazon Elastic Compute Cloud (Amazon EC2) instance ID, respectively.

PCR0, PCR1, and PCR2 in AWS Nitro enclaves are generated during enclave image build. PCR0 measures the enclave image file itself, PCR1 measures the Linux kernel and bootstrap data, and PCR2 measures the user application code. All of these PCRs describe the actual code and binaries loaded into the enclave, not external metadata, and thus are reproducible in this setup. See Cryptographic attestation for more information on the PCR values.

Run the enclave using the following command:

nitro-cli run-enclave --eif-path mpc-lib-repro.eif --memory 600 --cpu-count 2

Run the following command to list all running enclaves:

nitro-cli describe-enclaves

The command returns an output like the following:

[
  {
    "EnclaveName": "my-enclave",
    "...",
    "Measurements": {
    "HashAlgorithm": "Sha384 { ... }",
    "PCR0": "fac540c30c11a357e1b79a4ff9e22dbd05d31bba4b[...]",
    "..."
    }
  }
]

Request and validate the attestation document.

AWS Nitro Enclaves enables secure attestation by allowing enclaves to request a cryptographically signed document from the Nitro Secure Module (NSM), accessible only from within the enclave. The Nitro Hypervisor generates this document, embedding critical platform measurements like PCR0 (enclave image hash) and PCR8 (signing certificate fingerprint) if available, and signs it using the AWS Nitro Attestation PKI. A certificate chain is included, allowing AWS KMS and external parties to validate the document’s authenticity against the publicly trusted AWS Nitro enclave’s root certificate.

By verifying the document and its PCR values, a remote party gains:

Trusted environment assurance: The document’s valid signature confirms execution within an AWS-operated Nitro enclave, backed by hardware isolation.
Enclave identity validation: PCR0 matches the hash of the EIF, while PCR8 (if present) validates the EIF’s signing certificate.
Reproducibility verification: For open source or shared enclave code, third parties can rebuild the EIF from source, recompute PCRs, and compare them to the attestation document’s values. Matching PCRs prove the enclave runs unmodified, auditable code.

This workflow combines AWS’s trusted execution environment with reproducible builds, enabling decentralized verification of both what code runs and where it runs.

For a practical walkthrough of attestation document generation and verification, see the AWS Nitro Custom Attestation Workshop.

Clean up

To avoid incurring future charges, terminate the EC2 instance after it is no longer needed. Refer to the Terminate Amazon EC2 instances document to learn about different methods of deleting EC2 instances.

Conclusion

In this post, we introduced reproducible builds and remote attestation, two foundational techniques for securing the software supply chain. We showed how to build deterministic, reproducible binaries. Also, we explained how cryptographic hashes can be used to measure integrity and how reproducible builds can be integrated with AWS Nitro Enclaves attestation to help ensure integrity and decentralized verification.

In our next post, we’ll show how you can apply these concepts in building distributed applications using AWS Nitro Enclaves.

Now go, apply the reproducible build concepts to your software supply chains to enable verifiable and trustworthy builds across your organization!

AWS Web3 Blog