Container virtual registry

The GitLab container virtual registry is a local proxy you can use to cache container images from upstream registries. It acts as a pull-through cache, storing frequently-accessed images locally to reduce bandwidth usage and improve build performance.

Prerequisites

Before you can use the container virtual registry:

• Review the prerequisites to use the virtual registry.
• Configure authentication to the virtual registry. For more information, see Authenticate to the virtual registry.

When using the container virtual registry, remember the following restrictions:

• You can create up to 5 container virtual registries per top-level group.
• You can set only 5 upstreams to a given container virtual registry.
• Geo support is not implemented.

Manage virtual registries

Manage container virtual registries for your group.

You can also use the API.

Create a container virtual registry

To create a container virtual registry:

1. In the top bar, select Search or go to and find your group. This group must be at the top level.
2. Select Deploy > Virtual registry.
3. If you:
   • Have an existing registry, select Create registry. From the dropdown list, select Container.
   • Do not have an existing registry, from the dropdown list, select Container. Then, select Create registry.
4. Enter a Name and optional Description.
5. Select Create registry.

Manage upstream registries

Manage upstream container registries in a virtual registry.

Create a container upstream registry

Create a container upstream registry to connect to the virtual registry.

Prerequisites:

• You must have a container virtual registry. For more information, see Create a virtual registry.

To create a container upstream registry:

1. In the top bar, select Search or go to and find your group. This group must be at the top level.

2. Select Deploy > Virtual registry.

3. Under Registry types, select View registries.

4. Under the Registries tab, select a registry.

5. Select Add upstream. If the virtual registry has existing upstreams, from the dropdown list, select either:

   • Create new upstream to configure the upstream.
   • Link existing upstream > Select existing upstream.
     1. From the dropdown list, select an upstream.
     2. Optional. Select Test upstream to test the upstream connection before you create it.
     3. Select Add upstream.

6. Complete the fields.

   • Include both a username and password, or neither. If not set, a public (anonymous) request is used to access the upstream.

   • If you want to connect the upstream to Docker Hardened Images, use the following as the Upstream URL:

     https://dhi.io

   • Artifact caching period defaults to 24 hours. Set to 0 to disable cache entry checks.

   • If you want to test the upstream connection before you create it, select Test upstream.

7. Select Create upstream.

For more information about cache validity settings, see Set the cache validity period.

Authenticate with the container virtual registry

The container virtual registry stores and associates container images in a registry associated with your top-level group. To access container images, you must authenticate with your group’s container virtual registry.

To authenticate manually, run the following command:

echo "$CONTAINER_REGISTRY_PASSWORD" | docker login gitlab.example.com/virtual_registries/container/1 --username <your_username> --password-stdin

Or, configure authentication with any of the methods described in Authenticate to the virtual registry.

The container virtual registry follows the Docker v2 token authentication flow:

1. After client authentication, a JWT token issued to the client authorizes the client to pull container images.
2. The token expires according to its expiration time.
3. When the token expires, most Docker clients store user credentials and automatically request a new token without further action.

Pull container images from the virtual registry

To pull a container image through the virtual registry:

1. Authenticate with the virtual registry.

2. Use the virtual registry URL format to pull images:

   gitlab.example.com/virtual_registries/container/<registry_id>/<image_path>:<tag>

For example:

• Pull an image by its tag:

  docker pull gitlab.example.com/virtual_registries/container/1/library/alpine:latest

• Pull an image by digest:

  docker pull gitlab.example.com/virtual_registries/container/1/library/alpine@sha256:c9375e662992791e3f39e919b26f510e5254b42792519c180aad254e6b38f4dc

• Pull an image in a Dockerfile:

  FROM gitlab.example.com/virtual_registries/container/1/library/alpine:latest

• Pull an image in a .gitlab-ci.yml file:

  image: gitlab.example.com/virtual_registries/container/1/library/alpine:latest

When you pull an image, the virtual registry:

1. Checks if the image is already cached.
   1. If the image is cached and still valid based on the upstream’s cache_validity_hours setting, the image is served from the cache.
   2. If the image is not cached or the cache is invalid, the image is fetched from the configured upstream registry and cached.
2. Serves the image to your Docker client.

Virtual registry cache validation for images

An image tag like alpine:latest always pulls the most recent version of the image. The new version contains an updated image manifest. The container virtual registry does not pull a new image when the manifest changes.

Instead, the container virtual registry:

1. Checks the cache_validity_hours setting in the upstream to determine when an image manifest is invalid.
2. Sends a HEAD request to the upstream. If the manifest is invalid, a new image is pulled.

For example, if your pipeline pulls node:latest and you’ve set the cache_validity_period to 24 hours, the virtual registry caches the image and updates it either when the cache expires or node:latest changes in the upstream.

Troubleshooting

Authentication error: HTTP Basic: Access Denied

If you receive an HTTP Basic: Access denied error when authenticating against the virtual registry, refer to two-factor authentication troubleshooting.

Virtual registry connection failure

If a service alias is not set, the docker:20.10.16 image is unable to find the dind service, and an error like the following is thrown:

error during connect: Get http://docker:2376/v1.39/info: dial tcp: lookup docker on 192.168.0.1:53: no such host

To resolve this error, set a service alias for the Docker service:

services:
  - name: docker:20.10.16-dind
    alias: docker

Virtual registry authentication issues from CI/CD jobs

GitLab Runner authenticates automatically using the CI/CD job token. However, the underlying Docker engine is still subject to its authorization resolving process.

Misconfigurations in the authentication mechanism may cause HTTP Basic: Access denied and 403: Access forbidden errors.

You can use the job logs to view the authentication mechanism used to authenticate against the virtual registry:

Authenticating with credentials from $DOCKER_AUTH_CONFIG

Authenticating with credentials from /root/.docker/config.json

Authenticating with credentials from job payload (GitLab Registry)

Make sure you are using the expected authentication mechanism.

Not Found or 404 error when pulling image

Errors like these might indicate that:

• The user running the job does not have the Guest, Planner, Reporter, Developer, Maintainer, or Owner role for the group that owns the virtual registry.
• The virtual registry ID in the URL is incorrect.
• The upstream registry does not contain the requested image.
• The virtual registry has no upstreams configured.

Example error messages:

ERROR: gitlab.example.com/virtual_registries/container/1/library/alpine:latest: not found

ERROR: Job failed: failed to pull image "gitlab.example.com/virtual_registries/container/1/library/alpine:latest" with specified policies [always]:
Error response from daemon: error parsing HTTP 404 response body: unexpected end of JSON input: "" (manager.go:237:1s)

To resolve these errors:

1. Verify you have the Guest, Planner, Reporter, Developer, Maintainer, or Owner role for the group.
2. Confirm the virtual registry ID is correct.
3. Check that the virtual registry has at least one upstream configured.
4. Verify the image exists in the upstream registry.