1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
|
---
stage: Package
group: Package
info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/engineering/ux/technical-writing/#assignments
---
# Reduce Container Registry data transfers **(FREE)**
Depending on the frequency with which images or tags are downloaded from the Container Registry,
data transfers can exceed the GitLab limit. This page offers several recommendations and tips for
reducing the amount of data you transfer with the Container Registry.
are downloaded from the Container Registry, data transfers can exceed the GitLab limit. This page
offers several recommendations and tips for reducing the amount of data you transfer with the
Container Registry.
## Check data transfer use
Transfer usage is not available within the GitLab UI. [GitLab-#350905](https://gitlab.com/gitlab-org/gitlab/-/issues/350905)
is the epic tracking the work to surface this information. In the interim, GitLab team members may reach out to customers that have
significantly exceeded their transfer limits to better understand their use case and offer ways to reduce data transfer
usage.
## Determine image size
Use these tools and techniques to determine your image's size:
- [Skopeo](https://github.com/containers/skopeo):
use Skopeo's `inspect` command to examine layer count and sizes through API calls. You can
therefore inspect this data prior to running `docker pull IMAGE`.
- Docker in CI: examine and record the image size when using GitLab CI prior to pushing an image
with Docker. For example:
```yaml
docker inspect "$CI_REGISTRY_IMAGE:$IMAGE_TAG" \
| awk '/"Size": ([0-9]+)[,]?/{ printf "Final Image Size: %d\n", $2 }'
```
- [Dive](https://github.com/wagoodman/dive)
is a tool for exploring a Docker image, layer contents, and discovering ways to reduce its size.
## Reduce image size
### Use a smaller base image
Consider using a smaller base image, such as [Alpine Linux](https://alpinelinux.org/).
An Alpine image is around 5MB, which is several times smaller than popular base images such as
[Debian](https://hub.docker.com/_/debian).
If your application is distributed as a self-contained static binary, such as for Go applications,
you can also consider using Docker's [scratch](https://hub.docker.com/_/scratch/)
base image.
If you need to use a specific base image OS, look for `-slim` or `-minimal` variants, as this helps
to reduce the image size.
Also be mindful about the operating system packages you install on top of your base image. These can
add up to hundreds of megabytes. Try keeping the number of installed packages to the bare minimum.
[Multi-stage builds](#use-multi-stage-builds) can be a powerful ally in cleaning up transient build
dependencies.
You may also consider using tools such as these:
- [DockerSlim](https://github.com/docker-slim/docker-slim)
provides a set of commands to reduce the size of your container images.
- [Distroless](https://github.com/GoogleContainerTools/distroless) images contain only your
application and its runtime dependencies. They don't contain package managers, shells, or any
other programs you would expect to find in a standard Linux distribution.
### Minimize layers
Every instruction in your Dockerfile leads to a new layer, which records the file system changes
applied during such an instruction. In general, more or larger layers lead to larger images. Try to
minimize the number of layers to install the packages in the Dockerfile. Otherwise, this may cause
each step in the build process to increase the image size.
There are multiple strategies to reduce the number and size of layers. For example, instead of using
a `RUN` command per operating system package that you want to install (which would lead to a layer
per package), you can install all the packages on a single `RUN` command to reduce the number of
steps in the build process and reduce the size of the image.
Another useful strategy is to ensure that you remove all transient build dependencies and disable or
empty the operating system package manager cache before and after installing a package.
When building your images, make sure you only copy the relevant files. For Docker, using a
[`.dockerignore`](https://docs.docker.com/engine/reference/builder/#dockerignore-file)
helps ensure that the build process ignores irrelevant files.
You can use other third-party tools to minify your images, such as [DockerSlim](https://github.com/docker-slim/docker-slim).
Be aware that if used improperly, such tools may remove dependencies that your application needs to
operate under certain conditions. Therefore, it's preferable to strive for smaller images during the
build process instead of trying to minify images afterward.
### Use multi-stage builds
With [multi-stage builds](https://docs.docker.com/develop/develop-images/multistage-build/),
you use multiple `FROM` statements in your Dockerfile. Each `FROM` instruction can use a different
base, and each begins a new build stage. You can selectively copy artifacts from one stage to
another, leaving behind everything you don't want in the final image. This is especially useful when
you need to install build dependencies, but you don't need them to be present in your final image.
## Use an image pull policy
When using the `docker` or `docker+machine` executors, you can set a [`pull_policy`](https://docs.gitlab.com/runner/executors/docker.html#using-the-if-not-present-pull-policy)
parameter in your runner `config.toml` that defines how the runner works when pulling Docker images.
To avoid transferring data when using large and rarely updated images, consider using the
`if-not-present` pull policy when pulling images from remote registries.
## Use Docker layer caching
When running `docker build`, each command in `Dockerfile` results in a layer. These layers are kept
as a cache and can be reused if there haven't been any changes. You can specify a tagged image to be
used as a cache source for the `docker build` command by using the `--cache-from` argument. Multiple
images can be specified as a cache source by using multiple `--cache-from` arguments. This can speed
up your builds and reduce the amount of data transferred. For more information, see the
[documentation on Docker layer caching](../../../ci/docker/using_docker_build.md#make-docker-in-docker-builds-faster-with-docker-layer-caching).
## Move to GitLab Premium or Ultimate
GitLab data transfer limits are set at the tier level. If you need a higher limit, consider
upgrading to [GitLab Premium or Ultimate](https://about.gitlab.com/upgrade/).
## Purchase additional data transfer
Read more about managing your [data transfer limits](../../../subscriptions/gitlab_com/#purchase-more-storage-and-transfer).
## Related issues
- You may want to rebuild your image when the base Docker image is updated. However, the
[pipeline subscription limit is too low](https://gitlab.com/gitlab-org/gitlab/-/issues/225278)
to leverage this feature. As a workaround, you can rebuild daily or multiple times per day.
[GitLab-#225278](https://gitlab.com/gitlab-org/gitlab/-/issues/225278)
proposes raising the limit to help with this workflow.
|
