Welcome to mirror list, hosted at ThFree Co, Russian Federation.

index.md « dependency_proxy « package_registry « packages « user « doc - gitlab.com/gitlab-org/gitlab-foss.git - Unnamed repository; edit this file 'description' to name the repository.
summaryrefslogtreecommitdiff
blob: e1d968e63c0e99de0f182092ddcc79386c4b387d (plain)
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
263
264
265
266
267
268
269
270
271
272
273
274
275
276
277
278
279
280
281
282
283
284
285
286
287
288
289
290
291
292
293
---
stage: Package
group: Package Registry
info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments
---

# Dependency proxy for packages **(PREMIUM ALL BETA)**

> - [Introduced](https://gitlab.com/groups/gitlab-org/-/epics/3610) in GitLab 16.6 [with a flag](../../../../administration/feature_flags.md) named `packages_dependency_proxy_maven`. Disabled by default.
> - This feature is in [Beta](../../../../policy/experiment-beta-support.md).

FLAG:
On self-managed GitLab, by default this feature is not available. To make it available, an administrator can [enable the feature flag](../../../../administration/feature_flags.md) named `packages_dependency_proxy_maven`.
On GitLab.com, this feature is not available.
The feature is not ready for production use.

The GitLab dependency proxy for packages is a local proxy for frequently pulled packages.
It is implemented as a pull-through cache that works at the project level.

Packages are pulled from the upstream package registry and automatically published to the
project's package registry. Subsequent identical requests are fulfilled with the project's
package registry. You can use the dependency proxy for packages to reduce unnecessary traffic
to the upstream registry.

## Enable the dependency proxy

To use the dependency proxy for packages, ensure your project is configured properly,
and that users who pull from the cache have the necessary authentication:

1. In the global configuration, if the following features are disabled, enable them:
   - The [`package` feature](../../../../administration/packages/index.md#enable-or-disable-the-package-registry). Enabled by default.
   - The [`dependency_proxy` feature](../../../../administration/packages/dependency_proxy.md#turn-on-the-dependency-proxy). Enabled by default.
1. In the project settings, if the [`package` feature](../../package_registry/index.md#disable-the-package-registry)
   is disabled, enable it. It is enabled by default.
1. [Add an authentication method](#configure-a-client). The dependency proxy supports the same [authentication methods](../index.md#authenticate-with-the-registry) as the package registry:
   - [Personal access token](../../../profile/personal_access_tokens.md)
   - [Project deploy token](../../../project/deploy_tokens/index.md)
   - [Group deploy token](../../../project/deploy_tokens/index.md)
   - [Job token](../../../../ci/jobs/ci_job_token.md)

## Advanced caching

When possible, the dependency proxy for packages uses advanced caching to store packages in the project's package registry.

Advanced caching verifies the coherence between the project's package registry
and the upstream package registry. If the upstream registry has updated files,
the dependency proxy uses them to update the cached files.

When advanced caching is not supported, the dependency proxy falls back to the default behavior:

- If the requested file is found in the project's package registry, it is returned.
- If the file is not found, it is fetched from the upstream package registry.

Advanced caching support depends on how the upstream package registry
responds to dependency proxy requests, and on
which package format you use.

::Tabs

:::TabTitle Maven

| Package registry                                                                                                                         | Advanced caching supported? |
|------------------------------------------------------------------------------------------------------------------------------------------|-----------------------------|
| [GitLab](../../maven_repository/index.md)                                                                                                | **{check-circle}** Yes      |
| [Maven Central](https://mvnrepository.com/repos/central)                                                                                 | **{check-circle}** Yes      |
| [Artifactory](https://jfrog.com/integration/maven-repository/)                                                                           | **{check-circle}** Yes      |
| [Sonatype Nexus](https://help.sonatype.com/repomanager3/nexus-repository-administration/formats/maven-repositories)                      | **{check-circle}** Yes      |
| [GitHub Packages](https://docs.github.com/en/packages/working-with-a-github-packages-registry/working-with-the-apache-maven-registry)    | **{dotted-circle}** No      |

::EndTabs

### Permissions

When the dependency proxy pulls a file, the following occurs:

1. The dependency proxy searches for a file in the project's package registry.
   This is a read operation.
1. The dependency proxy might publish a package file to the project's package registry.
   This is a write operation.

Whether both steps are executed depends on user permissions.
The dependency proxy uses the [same permissions as the package registry](../index.md#package-registry-visibility-permissions).

| Project visibility | Minimum [role](../../../../user/permissions.md#roles) | Can read package files? | Can write package files? | Behavior |
|--------------------|-------------------------------------------------------|-------------------------|--------------------------|----------|
| Public             | Anonymous                                             | **{dotted-circle}** No  | **{dotted-circle}** No   | Request rejected. |
| Public             | Guest                                                 | **{check-circle}** Yes  | **{dotted-circle}** No   | Package file returned from either the cache or the remote registry. |
| Public             | Developer                                             | **{check-circle}** Yes  | **{check-circle}** Yes   | Package file returned from either the cache or the remote registry. The file is published to the cache. |
| Internal           | Anonymous                                             | **{dotted-circle}** No  | **{dotted-circle}** No   | Request rejected |
| Internal           | Guest                                                 | **{check-circle}** Yes  | **{dotted-circle}** No   | Package file returned from either the cache or the remote registry. |
| Internal           | Developer                                             | **{check-circle}** Yes  | **{check-circle}** Yes   | Package file returned from either the cache or the remote registry. The file is published to the cache. |
| Private            | Anonymous                                             | **{dotted-circle}** No  | **{dotted-circle}** No   | Request rejected |
| Private            | Reporter                                              | **{check-circle}** Yes  | **{dotted-circle}** No   | Package file returned from either the cache or the remote registry. |
| Internal           | Developer                                             | **{check-circle}** Yes  | **{check-circle}** Yes   | Package file returned from either the cache or the remote registry. The file is published to the cache. |

At a minimum, any user who can use the dependency proxy can also use the project's package registry.

To ensure the cache is properly filled over time, you should make sure a user with at least the Developer role pulls packages with the dependency proxy.

## Configure a client

Configuring a client for the dependency proxy is similar to configuring a client for the [package registry](../supported_functionality.md#pulling-packages).

### For Maven packages

For Maven packages, [all clients supported by the package registry](../../maven_repository/index.md) are supported by the dependency proxy:

- `mvn`
- `gradle`
- `sbt`

For authentication, the Maven dependency proxy access all authentication methods accepted by the [Maven package registry](../../maven_repository/index.md#edit-the-client-configuration).
You should use the [Basic HTTP authentication](../../maven_repository/index.md#basic-http-authentication) method as it is less complex.

To configure the client:

1. Follow the instructions in [Basic HTTP authentication](../../maven_repository/index.md#basic-http-authentication).

   Make sure you use the endpoint URL `https://gitlab.example.com/api/v4/projects/<project_id>/dependency_proxy/packages/maven`.

1. Complete the configuration for your client:

::Tabs

:::TabTitle mvn

In the `pom.xml` file add a `repository` element:

```xml
<repositories>
  <repository>
    <id>gitlab-maven</id>
    <url>https://gitlab.example.com/api/v4/projects/<project_id>/dependency_proxy/packages/maven</url>
  </repository>
</repositories>
```

Where:

- `<project_id>` is the ID of the project to be used as a dependency proxy.
- `<id>` contains the name of the `<server>` used in the [authentication configuration](../../maven_repository/index.md#basic-http-authentication).

By default, Maven Central is checked first through the [Super POM](https://maven.apache.org/guides/introduction/introduction-to-the-pom.html#Super_POM).
However, you might want to force `mvn` to check the GitLab endpoint first. To do this, follow the instructions from the [request forward](../../maven_repository/index.md#additional-configuration-for-mvn).

:::TabTitle gradle

Add a `repositories` section to your [`build.gradle`](https://docs.gradle.org/current/userguide/tutorial_using_tasks.html) file.

- In Groovy DSL:

  ```groovy
  repositories {
      maven {
          url "https://gitlab.example.com/api/v4/projects/<project_id>/dependency_proxy/packages/maven"
          name "GitLab"
          credentials(PasswordCredentials) {
              username = 'REPLACE_WITH_NAME'
              password = gitLabPrivateToken
          }
          authentication {
              basic(BasicAuthentication)
          }
      }
  }
  ```

- In Kotlin DSL:

  ```kotlin
  repositories {
      maven {
          url = uri("https://gitlab.example.com/api/v4/projects/<project_id>/dependency_proxy/packages/maven")
          name = "GitLab"
          credentials(BasicAuthentication::class) {
              username = "REPLACE_WITH_NAME"
              password = findProperty("gitLabPrivateToken") as String?
          }
          authentication {
              create("basic", BasicAuthentication::class)
          }
      }
  }
  ```

In this example:

- `<project_id>` is the ID of the project to be used as a dependency proxy.
- `REPLACE_WITH_NAME` is explained in the [Basic HTTP authentication](../../maven_repository/index.md#basic-http-authentication) section.

:::TabTitle sbt

In your [`build.sbt`](https://www.scala-sbt.org/1.x/docs/Directories.html#sbt+build+definition+files), add the following lines:

```scala
resolvers += ("gitlab" at "https://gitlab.example.com/api/v4/projects/<project_id>/dependency_proxy/packages/maven")

credentials += Credentials("GitLab Packages Registry", "<host>", "<name>", "<token>")
```

In this example:

- `<project_id>` is the ID of the project to be used as a dependency proxy.
- `<host>` is the host present in the `<endpoint url>` without the protocol scheme or the port. Example: `gitlab.example.com`.
- `<name>` and `<token>` are explained in the [Basic HTTP authentication](../../maven_repository/index.md#basic-http-authentication) section.

::EndTabs

## Configure the remote registry

The dependency proxy must be configured with:

- The URL of the remote package registry.
- Optional. The required credentials.

To set those parameters:

1. Use the related [GraphQL mutation](../../../../api/graphql/reference/index.md#mutationupdatedependencyproxypackagessettings).

   A frontend to display and manage these settings is proposed by [issue 410726](https://gitlab.com/gitlab-org/gitlab/-/issues/410726).

1. Complete the configuration for your package format:

::Tabs

:::TabTitle Maven

Any Maven package registry can be connected to the dependency proxy. You can
authorize the connection with the Maven package registry username and password.

To set or update the remote Maven package registry, update the following fields on the settings object:

- `mavenExternalRegistryUrl` - The URL of the remote registry.
- `mavenExternalRegistryUsername` - Optional. The username to use with the remote registry.
- `mavenExternalRegistryPassword` - Optional. The password to use with the remote registry.

Example of the GraphQL mutation:

```graphql
mutation {
  updateDependencyProxyPackagesSettings(input: {  projectPath : <project full path>, enabled: true, mavenExternalRegistryUrl: <remote registry url>, mavenExternalRegistryUsername: <username>, mavenExternalRegistryPassword: <password> }) {
    dependencyProxyPackagesSetting {
      enabled
      mavenExternalRegistryUrl
      mavenExternalRegistryUsername
    }
    errors
  }
}
```

::EndTabs

## Troubleshooting

### Manual file pull errors

You can pull files manually with cURL.
However, you might encounter one of the following responses:

- `404 Not Found` - The dependency proxy setting object was not found because it doesn't exist, or because the [requirements](#enable-the-dependency-proxy) were not fulfilled.
- `401 Unauthorized` - The user was properly authenticated but did not have the proper permissions to access the dependency proxy object.
- `403 Forbidden` - There was an issue with the [GitLab license level](#enable-the-dependency-proxy).
- `502 Bad Gateway` - The remote package registry could not fulfill the file request. Verify the [dependency proxy settings](#configure-the-remote-registry).
- `504 Gateway Timeout` - The remote package registry timed out. Verify the [dependency proxy settings](#configure-the-remote-registry).

::Tabs

:::TabTitle Maven

```shell
curl --verbose "https://<username>:<personal access token>@gitlab.example.com/api/v4/projects/<project_id>/dependency_proxy/packages/maven/<group id and artifact id>/<version>/<file_name>"
```

- `<username>` and `<personal access token>` are the credentials to access the dependency proxy of the GitLab instance.
- `<project_id>` is the project ID.
- `<group id and artifact id>` are the [Maven package group ID and artifact ID](https://maven.apache.org/pom.html#Maven_Coordinates) joined with a forward slash.
- `<version>` is the package version.
- `file_name` is the exact name of the file.

For example, given a package with:

- group ID: `com.my_company`.
- artifact ID: `my_package`.
- version: `1.2.3`.

The request to manually pull a package is:

```shell
curl --verbose "https://<username>:<personal access token>@gitlab.example.com/api/v4/projects/<project_id>/dependency_proxy/packages/maven/com/my_company/my_package/1.2.3/my_package-1.2.3.pom"
```

::EndTabs