Convert to OCI builder #125
Conversation
There was a problem hiding this comment.
This is a good start! There are a lot of little nitpicks, but I tried to stay away from them and focus on the higher-level things first so we can hash out what the end state needs to look like before we get too buried in the nuts and bolts / implementation (although it's definitely useful to have an explicit implementation to point at for articulating and discussing those 👍).
I think at a really high level, this probably needs to be integrated deeper into the existing build process, especially so we can have it be fully reproducible (by building it inside that container/image). 🤔
oci.sh
Outdated
| schemaVersion: 2, | ||
| mediaType: "application/vnd.oci.image.index.v1+json", | ||
| manifests: $manifests | ||
| }' > "$ociroot/index.json" |
There was a problem hiding this comment.
Lumping all the architectures together in one big index.json won't actually work in the oci-import builder -- it requires exactly/only one image in index.json or (as I noted in #124) that we specify File: pointing to a JSON file that contains a single image descriptor.
As much as I'd like to lump all these into a single OCI layout here (it's very cute!), I think we'll actually have a much better time downstream if we instead do a separate OCI layout per architecture (amd64/oci/index.json, i386/oci/index.json, etc), especially since the sourceId for the image/build will currently be calculated based on the full contents of the OCI layout, not just the one architecture, so any update to any binaries will result in "rebuilding" all of them (which, to be fair, is something we should try to fix downstream, but implementing that correctly is very complex).
| } | tojson' | ||
| } | ||
|
|
||
| created="2025-01-21T23:32:32Z" |
There was a problem hiding this comment.
Any particular reason for this specific value? 😅
(long-term we probably need to scrape an appropriate value per-architecture from the latest Git commit to touch the relevant hello binary or something like that, but that might get recursive if we want to update the binary and the OCI layouts in one step 🤔 there's also an argument we could make that committing the binaries and the tarballs/OCI layouts is overkill and we should only commit the latter)
There was a problem hiding this comment.
Hahaha, that was s just the value from the current one in registry and forgot to change it
|
|
||
| mkdir -p $blobs | ||
|
|
||
| toBlob() { |
There was a problem hiding this comment.
You might be interested in https://gist.github.com/tianon/35528d9851ebf0b0767c04cec5e01bcf, particularly https://gist.github.com/tianon/35528d9851ebf0b0767c04cec5e01bcf#file-oci-put 😅
oci.sh
Outdated
| local arch="$1"; shift | ||
|
|
||
| path="./$arch/hello-world" | ||
| tar -C $path -cf - hello > tmp.tar |
There was a problem hiding this comment.
If we're going to commit these tarballs (and even if we're not, really), we should go out of our way to make sure they are generated reproducibly like we do in BusyBox, which is probably going to involve/require we build them inside a container or image (such as the Dockerfile.build we already have 😅): https://github.com/docker-library/busybox/blob/69136da0a606fde9f84b593f975060819960f42b/build.sh#L44-L53
| platform="" | ||
|
|
||
| case "$arch" in | ||
| arm64v8) | ||
| platform=$(jq -r -n --arg os $os --arg arch "arm64" --arg variant "v8" '{ | ||
| architecture: $arch, | ||
| os: $os, | ||
| variant: $variant | ||
| } | tojson') | ||
| ;; | ||
| arm32v*) | ||
| platform=$(jq -r -n --arg os $os --arg arch "arm" --arg variant "${arch/arm32/}" '{ | ||
| architecture: $arch, | ||
| os: $os, | ||
| variant: $variant | ||
| } | tojson') | ||
| ;; | ||
| *) | ||
| platform=$(jq -r -n --arg os $os --arg arch $arch '{ | ||
| architecture: $arch, | ||
| os: $os | ||
| } | tojson') | ||
| ;; | ||
| esac |
There was a problem hiding this comment.
We should let bashbrew interpret these for us instead, like we do in BusyBox: https://github.com/docker-library/busybox/blob/69136da0a606fde9f84b593f975060819960f42b/build.sh#L11-L13 😅
(however, we don't actually need this for index.json - only for the config blob)
LaurentGoderre
force-pushed
the
oci-builder
branch
3 times, most recently
from
4f247b6 to
046d48d
Compare
LaurentGoderre
force-pushed
the
oci-builder
branch
from
046d48d to
0688ddc
Compare
Addresses #124