Standardised verifiable builds (use-ink#1148)

* docker image + tokio

* docker client draft

* refactoring execute func

* successful docker build

* persist flags in the main execution context

* docker readme + respect host flags

* add build steps

* better image, custom image arg, container reuse

* fix readme example

* fix docs

* update README

* docs fixes

* clarification in the docs

* use locked for cargo install

* concat install and cleanup commands

* use arg instead of env in dockerfile

* add version labels

* update comments

* decompose the function

* correct argument passing in dockerfile

* refactoring

* more fixes

* using remote registry + bell & whistles (progress bars)

* changelog entry

* fixes

* refactoring

* explicitly specify versions of apt packages

* refactoring

* add module docs

* refactoring

* doc fix and optionally use git in docker image

* Error handling for builds

* use single var in docker and refactor docker module

* notes on apple silicon

* remove unused arg in dockerfile

* use target path correctly

* handle building with relative paths + permissions

* more flags in docker image + fixes

* remove unused code

* add git info to docker metadata

* use env to detect running OS

* detect host OS

* non-root docker user

* mount contract to home dir in docker

* give permission to target to everyone

* fix typo

* use current uid and gid for docker

* conditional compilation

* dont cache ci template and use musl in docker

* OS specific code blocks

* print error logs

* remove the message

* calculate digest in separate func

* filter our status msgs in error

* update comments

* resolve other merge conflicts

* display container name

* properly trancate error output

* apply Andrews suggestions

* proper build step logs

* fix generation of ABI

* hash the cmd, not entrypoint for digest

* fix args parsing and add docs

* clippy fix

README.md

@@ -45,6 +45,9 @@ Modern releases of gcc and clang, as well as Visual Studio 2019+ should work.
   * (MacOS) `brew install openssl`
   * `cargo install cargo-dylint dylint-link`.
 
+* Step 4: (**Optional**) Install [Docker Engine](https://docs.docker.com/engine/install)
+to be able to produce verifiable builds.
+
 You can always update the `cargo-contract` binary to the latest version by running the Step 2.
 
 ### Installation using Docker Image
@@ -69,6 +72,16 @@ docker run --rm -it -v $(pwd):/sources paritytech/contracts-ci-linux:production
 If you want to reproduce other steps of CI process you can use the following
 [guide](https://github.com/paritytech/scripts#reproduce-ci-locally).
 
+### Verifiable builds
+
+Some block explorers require the Wasm binary to be compiled in the deterministic environment.
+To achieve it, you should build your contract using Docker image we provide:
+```bash
+cargo contract build --verifiable
+```
+
+You can find more detailed documentation how to use the image [here](/build-image/README.md)
+
 ## Usage
 
 You can always use `cargo contract help` to print information on available

build-image/.dockerignore

@@ -0,0 +1 @@
+target/

build-image/Dockerfile

@@ -0,0 +1,111 @@
+FROM docker.io/bitnami/minideb:bullseye-amd64 as slimmed-rust
+
+# The rust version to use
+# The 1.69 toolchain is temporarily required to build ink! contracts because of
+# https://github.com/paritytech/cargo-contract/issues/1139
+ARG RUST_VERSION=1.69
+# The cargo contract version to use
+ARG CARGO_CONTRACT_VERSION=3.0.1
+# Url to the cargo-contract repository to install from
+ARG CARGO_CONTRACT_GIT
+# Branch to use in git repository
+ARG CARGO_CONTRACT_BRANCH
+# Commit to use in git repository
+ARG CARGO_CONTRACT_REV
+# Tag to use in git repository
+ARG CARGO_CONTRACT_TAG
+# gcc package version
+ARG GCC_VERSION=4:10.2.1-1
+# wget package version
+ARG WGET_VERSION=1.21-1+deb11u1
+# g++ package version
+ARG G_VERSION=4:10.2.1-1
+ARG MUSL_V=1.2.2-1
+
+# metadata
+LABEL io.parity.image.vendor="Parity Technologies" \
+    io.parity.image.title="paritytech/contracts-verifiable" \
+    io.parity.image.description="Inherits from docker.io/bitnami/minideb:bullseye. \
+    rust nightly, clippy, rustfmt, miri, rust-src, rustc-dev, grcov, rust-covfix, \
+    llvm-tools-preview, cargo-contract, xargo, binaryen, parallel, codecov, ink, solang" \
+    io.parity.image.documentation="https://github.com/paritytech/cargo-contract/blob/master/\
+    build-image/README.md" \
+    io.parity.version.rust=${RUST_VERSION} \
+    io.parity.version.cargo-contract.version=${CARGO_CONTRACT_VERSION} \
+    io.parity.version.cargo-contract.git.repository=${CARGO_CONTRACT_GIT} \
+    io.parity.version.cargo-contract.git.branch=${CARGO_CONTRACT_BRANCH} \
+    io.parity.version.cargo-contract.git.revision=${CARGO_CONTRACT_REV} \
+    io.parity.version.cargo-contract.git.tag=${CARGO_CONTRACT_TAG} \
+    io.parity.version.gcc=${GCC_VERSION} \
+    io.parity.version.wget=${WGET_VERSION} \
+    io.parity.version.g_plus_plus=${G_VERSION} \
+    io.parity.version.musl=${MUSL_V}
+
+ENV RUSTUP_HOME=/usr/local/rustup \
+    CARGO_HOME=/usr/local/cargo \
+    PATH=/usr/local/cargo/bin:$PATH
+
+# Minimal Rust dependencies.
+RUN set -eux \
+    && apt-get update && apt-get -y install wget=${WGET_VERSION} \
+    && url="https://static.rust-lang.org/rustup/dist/x86_64-unknown-linux-gnu/rustup-init" \
+    && wget "$url" \
+    && chmod +x rustup-init \
+    && ./rustup-init -y --no-modify-path --profile minimal --component rust-src rustfmt --default-toolchain $RUST_VERSION  \
+    && rm rustup-init \
+    && chmod -R a+w $RUSTUP_HOME $CARGO_HOME \
+    && rustup --version \
+    && cargo --version \
+    && rustc --version \
+    && apt-get remove -y --auto-remove wget \
+    && apt-get -y install gcc=${GCC_VERSION} \
+    && rm -rf /var/lib/apt/lists/*
+
+FROM slimmed-rust as cc-builder
+ARG CARGO_CONTRACT_VERSION
+ARG GCC_VERSION
+ARG G_VERSION
+ARG MUSL_V
+ARG CARGO_CONTRACT_BRANCH
+ARG CARGO_CONTRACT_TAG
+ARG CARGO_CONTRACT_REV
+ARG CARGO_CONTRACT_GIT
+
+# This is important, see https://github.com/rust-lang/docker-rust/issues/85
+ENV RUSTFLAGS="-C target-feature=-crt-static"
+
+# Install required packages for `cargo-contract`
+RUN apt-get -y update && apt-get -y install gcc=${GCC_VERSION} g++=${G_VERSION} musl-dev=${MUSL_V} \
+    # Install cargo contract from git if the arg is set
+    && if [ -n "$CARGO_CONTRACT_GIT" ]; then \
+    COMMAND="cargo install --git ${CARGO_CONTRACT_GIT}" ; \
+    else \
+    COMMAND="cargo install cargo-contract --locked --version ${CARGO_CONTRACT_VERSION}" ;\
+    fi \
+    && if [ -n "$CARGO_CONTRACT_BRANCH" ] && [ -n "$CARGO_CONTRACT_GIT" ]; then \
+    COMMAND="${COMMAND} --branch ${CARGO_CONTRACT_BRANCH}" ; \
+    fi \
+    && if [ -n "$CARGO_CONTRACT_REV" ] && [ -n "$CARGO_CONTRACT_GIT" ]; then \
+    COMMAND="${COMMAND} --rev ${CARGO_CONTRACT_REV}" ; \
+    fi \
+    && if [ -n "$CARGO_CONTRACT_TAG" ] && [ -n "$CARGO_CONTRACT_GIT" ]; then \
+    COMMAND="cargo install --git ${CARGO_CONTRACT_GIT} --tag ${CARGO_CONTRACT_TAG}" ; \
+    fi \
+    && echo "Executing ${COMMAND}" \
+    && eval "${COMMAND}" \
+    # Cleanup after `cargo install`
+    && rm -rf ${CARGO_HOME}/"registry" ${CARGO_HOME}/"git" /root/.cache/sccache \
+    # apt clean up
+    && apt-get remove -y gnupg \
+    && apt-get autoremove -y \
+    && apt-get clean \
+    && rm -rf /var/lib/apt/lists/*
+
+FROM slimmed-rust as ink-dev
+
+COPY --from=cc-builder /usr/local/cargo/bin/cargo-contract /usr/local/bin/cargo-contract
+
+WORKDIR /contract
+
+# default entry point
+ENTRYPOINT ["cargo", "contract"]

build-image/README.md

@@ -0,0 +1,52 @@
+# Verifiable builds using Docker
+
+Docker image based on the minimalistic Debian image `bitnami/minideb:bullseye-amd64`.
+
+Used for reproducible builds in `cargo contract build --verifiable`
+
+**Rust versions:**
+
+Currently, the 1.69 toolchain is temporarily required to build ink! contracts because of https://github.com/paritytech/cargo-contract/issues/1139
+
+**Rust tools & toolchains:**
+
+We use stable releases from crates.io
+
+- `cargo-contract`
+- `wasm32-unknown-unknown`: The toolchain to compile Rust codebases for Wasm.
+
+[Click here](https://hub.docker.com/repository/docker/paritytech/contracts-verifiable) for the registry.
+
+## Usage
+
+The default entry point is `ENTRYPOINT [ "cargo", "contract", "build", "--release" ]`
+and work directory is set to `/contract`. You need to mount the folder with the contract to the container.
+
+```bash
+docker run -d \
+    --name ink-container \
+    --mount type=bind,source="$(pwd)",target="/contract" \
+    paritytech/contracts-verified
+```
+
+For multi-contract projects, like in the example below:
+```
+my-app/
+├─ ink-project-a/
+│  ├─ Cargo.toml
+│  ├─ lib.rs
+├─ ink-project-b/
+│  ├─ Cargo.toml
+│  ├─ lib.rs
+├─ rust-toolchain
+```
+Make sure to run the command inside `my-app` directory and specify a relative manifest path
+to the root contract:
+`cargo contract build --verifiable --release --manifest-path ink-project-a/Cargo.toml`
+
+
+**Apple Silicon performance**
+
+It is a known issue that running AMD64 image on the ARM host architecture significantly impacts the performance
+and build times. To solve this issues, enable _Use Rosetta for x86/amd64 emulation on Apple Silicon_ in
+_Settings_ -> _Features in development_ tab in Docker Desktop App.

crates/build/src/args.rs

@@ -45,7 +45,7 @@ impl TryFrom<&VerbosityFlags> for Verbosity {
 }
 
 /// Denotes if output should be printed to stdout.
-#[derive(Clone, Copy, Default, serde::Serialize, Eq, PartialEq)]
+#[derive(Clone, Copy, Default, serde::Serialize, serde::Deserialize, Eq, PartialEq)]
 pub enum Verbosity {
     /// Use default output
     #[default]
@@ -89,7 +89,15 @@ impl Network {
 
 /// Describes which artifacts to generate
 #[derive(
-    Copy, Clone, Default, Eq, PartialEq, Debug, clap::ValueEnum, serde::Serialize,
+    Copy,
+    Clone,
+    Default,
+    Eq,
+    PartialEq,
+    Debug,
+    clap::ValueEnum,
+    serde::Serialize,
+    serde::Deserialize,
 )]
 #[clap(name = "build-artifacts")]
 pub enum BuildArtifacts {
@@ -223,15 +231,18 @@ pub enum BuildMode {
     /// Functionality to output debug messages is build into the contract.
     #[default]
     Debug,
-    /// The contract is build without any debugging functionality.
+    /// The contract is built without any debugging functionality.
     Release,
+    /// the contract is built in release mode and in a deterministic environment.
+    Verifiable,
 }
 
 impl fmt::Display for BuildMode {
     fn fmt(&self, f: &mut fmt::Formatter<'_>) -> fmt::Result {
         match self {
             Self::Debug => write!(f, "debug"),
             Self::Release => write!(f, "release"),
+            Self::Verifiable => write!(f, "verifiable"),
         }
     }
 }