Is the new LayersData field the compressed size?

[jchunkins]

I saw a recent change in this PR: #1626 and was wondering what the LayersData.Size field corresponds to. Is this the compressed size of the layer?

For example, docker reports this for the latest redis image:

$ docker image ls redis
REPOSITORY   TAG       IMAGE ID       CREATED        SIZE
redis        latest    9da089657551   29 hours ago   117MB

Another way to look at the image with docker:

$ docker history redis --human=false --format "{{.Size}}" | jq -s 'add' | numfmt --to=si
117M

NOTE: I think both of the above examples must be returning the size of the redis image AND its base image? Maybe?

This way seems to match value reported by skopeo:

$ docker manifest inspect -v redis | jq '.[] | select( .Descriptor.platform.architecture | contains("amd64")) | [.. | objects | .size] | add ' | numfmt --to=si
43M

The skopeo code that consumes this latest change reports this:

$ ./bin/skopeo --override-os linux inspect --no-tags docker://redis | jq '[.. | objects | .Size] | add' | numfmt --to=si 
43M

NOTE I assume these last two examples above only report the layers for the image itself?

At any rate, I have to assume LayersData.Size is the compressed image size, or maybe I am missing something. Perhaps @ningmingxiao can provide more insight into what the value represents.

Would it be possible to report compressed and uncompressed sizes, if for no other reason than completeness / convenience?

[ningmingxiao]

LayersData.Size is the compressed image size

[jchunkins]

@ningmingxiao Thanks for the confirmation.

Given the first set of commands in my original post (i.e. docker image ls and docker history), those sizes appear to be the uncompressed sizes. I tested this by extracting the redis image onto the file system, and the size matches the 117MB in those examples.

So the question I have, is there a way to provide the uncompressed size? Docker seems to have a way to do this, so just wondering if there's something easily done to provide that value in the results too.

[mtrmac]

LayersData.Size is the size of the representation as it exists in the source. That might be compressed or uncompressed, depending on the source (and possibly on the way it was written there).

In most cases (but not reliably all the time), the MIMEType field can help determine whether the blob is compressed or uncompressed.

The other size (compressed/uncompressed) is not typically available. The uncompressed size of a compressed layer can only be determined by decompressing; the compressed size of an uncompressed layer is not a well-defined number at all (it depends on compression settings).

[jchunkins]

@mtrmac So there's no way to make any guarantees about the LayersData.Size other than the amount of data stored in the source (and it could be compressed or uncompressed). This is different from the assertion @ningmingxiao made saying it's always compressed size.

[mtrmac]

% skopeo --override-os linux inspect docker://quay.io/libpod/alpine | grep 'MIME\|Size'  
            "MIMEType": "application/vnd.docker.image.rootfs.diff.tar.gzip",
            "Size": 2896510,
% skopeo --override-os linux copy docker://quay.io/libpod/alpine dir:t
…
% skopeo --override-os linux inspect dir:t | grep 'MIME\|Size'                
            "MIMEType": "application/vnd.docker.image.rootfs.diff.tar.gzip",
            "Size": 2896510,
% skopeo --override-os linux copy --dest-decompress docker://quay.io/libpod/alpine dir:t
…
% skopeo --override-os linux inspect dir:t | grep 'MIME\|Size'                                                                
            "MIMEType": "application/vnd.docker.image.rootfs.diff.tar",
            "Size": 5843968,

[jchunkins]

Tangential question... if a tool "mirrors" an image to a new destination and changes the compression type (i.e. compressed -> uncompressed OR uncompressed ->compressed), then I assume that changes the image digest too?

[mtrmac]

Yes it does.