1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
263
264
265
266
267
268
269
270
271
272
273
274
275
276
277
278
279
280
281
282
283
284
285
286
287
288
289
290
291
292
293
294
295
296
297
298
299
300
301
302
303
304
305
306
307
308
309
310
311
312
313
314
315
316
317
318
319
320
321
322
323
324
325
326
327
328
329
330
331
332
333
334
335
336
337
338
339
|
---
stage: Package
group: Package Registry
info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments
---
# PyPI packages in the Package Registry **(FREE)**
> - [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/208747) in GitLab 12.10.
> - [Moved](https://gitlab.com/gitlab-org/gitlab/-/issues/221259) from GitLab Premium to GitLab Free in 13.3.
Publish PyPI packages in your project's Package Registry. Then install the
packages whenever you need to use them as a dependency.
The Package Registry works with:
- [pip](https://pypi.org/project/pip/)
- [twine](https://pypi.org/project/twine/)
For documentation of the specific API endpoints that the `pip` and `twine`
clients use, see the [PyPI API documentation](../../../api/packages/pypi.md).
Learn how to [build a PyPI package](../workflows/build_packages.md#pypi).
## Authenticate with the Package Registry
Before you can publish to the Package Registry, you must authenticate.
To do this, you can use:
- A [personal access token](../../../user/profile/personal_access_tokens.md)
with the scope set to `api`.
- A [deploy token](../../project/deploy_tokens/index.md) with the scope set to
`read_package_registry`, `write_package_registry`, or both.
- A [CI job token](#authenticate-with-a-ci-job-token).
### Authenticate with a personal access token
To authenticate with a personal access token, edit the `~/.pypirc` file and add:
```ini
[distutils]
index-servers =
gitlab
[gitlab]
repository = https://gitlab.example.com/api/v4/projects/<project_id>/packages/pypi
username = <your_personal_access_token_name>
password = <your_personal_access_token>
```
The `<project_id>` is either the project's
[URL-encoded](../../../api/rest/index.md#namespaced-path-encoding)
path (for example, `group%2Fproject`), or the project's ID (for example `42`).
### Authenticate with a deploy token
To authenticate with a deploy token, edit your `~/.pypirc` file and add:
```ini
[distutils]
index-servers =
gitlab
[gitlab]
repository = https://gitlab.example.com/api/v4/projects/<project_id>/packages/pypi
username = <deploy token username>
password = <deploy token>
```
The `<project_id>` is either the project's
[URL-encoded](../../../api/rest/index.md#namespaced-path-encoding)
path (for example, `group%2Fproject`), or the project's ID (for example `42`).
### Authenticate with a CI job token
> [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/202012) in GitLab 13.4.
To work with PyPI commands within [GitLab CI/CD](../../../ci/index.md), you
can use `CI_JOB_TOKEN` instead of a personal access token or deploy token.
For example:
```yaml
image: python:latest
run:
script:
- pip install build twine
- python -m build
- TWINE_PASSWORD=${CI_JOB_TOKEN} TWINE_USERNAME=gitlab-ci-token python -m twine upload --repository-url ${CI_API_V4_URL}/projects/${CI_PROJECT_ID}/packages/pypi dist/*
```
You can also use `CI_JOB_TOKEN` in a `~/.pypirc` file that you check in to
GitLab:
```ini
[distutils]
index-servers =
gitlab
[gitlab]
repository = https://gitlab.example.com/api/v4/projects/${env.CI_PROJECT_ID}/packages/pypi
username = gitlab-ci-token
password = ${env.CI_JOB_TOKEN}
```
### Authenticate to access packages within a group
Follow the instructions above for the token type, but use the group URL in place of the project URL:
```shell
https://gitlab.example.com/api/v4/groups/<group_id>/-/packages/pypi
```
## Publish a PyPI package
Prerequisites:
- You must [authenticate with the Package Registry](#authenticate-with-the-package-registry).
- Your [version string must be valid](#ensure-your-version-string-is-valid).
- The maximum allowed package size is 5 GB.
- You can't upload the same version of a package multiple times. If you try,
you receive the error `400 Bad Request`.
- PyPI packages are published using your projectID.
- If your project is in a group, PyPI packages published to your project registry are also available
at the group-level registry (see [Install from the group level](#install-from-the-group-level)).
You can then [publish a package by using twine](#publish-a-pypi-package-by-using-twine).
### Ensure your version string is valid
If your version string (for example, `0.0.1`) isn't valid, it gets rejected.
GitLab uses the following regex to validate the version string.
```ruby
\A(?:
v?
(?:([0-9]+)!)? (?# epoch)
([0-9]+(?:\.[0-9]+)*) (?# release segment)
([-_\.]?((a|b|c|rc|alpha|beta|pre|preview))[-_\.]?([0-9]+)?)? (?# pre-release)
((?:-([0-9]+))|(?:[-_\.]?(post|rev|r)[-_\.]?([0-9]+)?))? (?# post release)
([-_\.]?(dev)[-_\.]?([0-9]+)?)? (?# dev release)
(?:\+([a-z0-9]+(?:[-_\.][a-z0-9]+)*))? (?# local version)
)\z}xi
```
You can experiment with the regex and try your version strings by using this
[regular expression editor](https://rubular.com/r/FKM6d07ouoDaFV).
For more details about the regex, review this [documentation](https://www.python.org/dev/peps/pep-0440/#appendix-b-parsing-version-strings-with-regular-expressions).
### Publish a PyPI package by using twine
To publish a PyPI package, run a command like:
```shell
python3 -m twine upload --repository gitlab dist/*
```
This message indicates that the package was published successfully:
```plaintext
Uploading distributions to https://gitlab.example.com/api/v4/projects/<your_project_id>/packages/pypi
Uploading mypypipackage-0.0.1-py3-none-any.whl
100%|███████████████████████████████████████████████████████████████████████████████████████████| 4.58k/4.58k [00:00<00:00, 10.9kB/s]
Uploading mypypipackage-0.0.1.tar.gz
100%|███████████████████████████████████████████████████████████████████████████████████████████| 4.24k/4.24k [00:00<00:00, 11.0kB/s]
```
To view the published package, go to your project's **Packages and registries**
page.
If you didn't use a `.pypirc` file to define your repository source, you can
publish to the repository with the authentication inline:
```shell
TWINE_PASSWORD=<personal_access_token or deploy_token> TWINE_USERNAME=<username or deploy_token_username> python3 -m twine upload --repository-url https://gitlab.example.com/api/v4/projects/<project_id>/packages/pypi dist/*
```
If you didn't follow the steps on this page, ensure your package was properly
built, and that you [created a PyPI package with `setuptools`](https://packaging.python.org/tutorials/packaging-projects/).
You can then upload your package by using the following command:
```shell
python -m twine upload --repository <source_name> dist/<package_file>
```
- `<package_file>` is your package filename, ending in `.tar.gz` or `.whl`.
- `<source_name>` is the [source name used during setup](#authenticate-with-the-package-registry).
### Publishing packages with the same name or version
You cannot publish a package if a package of the same name and version already exists.
You must [delete the existing package](../../packages/package_registry/reduce_package_registry_storage.md#delete-a-package) first.
If you attempt to publish the same package
more than once, a `400 Bad Request` error occurs.
## Install a PyPI package
In [GitLab 14.2 and later](https://gitlab.com/gitlab-org/gitlab/-/issues/233413),
when a PyPI package is not found in the Package Registry, the request is forwarded to [pypi.org](https://pypi.org/).
Administrators can disable this behavior in the [Continuous Integration settings](../../admin_area/settings/continuous_integration.md).
WARNING:
When you use the `--index-url` option, do not specify the port if it is a default
port, such as `80` for a URL starting with `http` or `443` for a URL starting
with `https`.
### Install from the project level
To install the latest version of a package, use the following command:
```shell
pip install --index-url https://<personal_access_token_name>:<personal_access_token>@gitlab.example.com/api/v4/projects/<project_id>/packages/pypi/simple --no-deps <package_name>
```
- `<package_name>` is the package name.
- `<personal_access_token_name>` is a personal access token name with the `read_api` scope.
- `<personal_access_token>` is a personal access token with the `read_api` scope.
- `<project_id>` is either the project's [URL-encoded](../../../api/rest/index.md#namespaced-path-encoding)
path (for example, `group%2Fproject`), or the project's ID (for example `42`).
In these commands, you can use `--extra-index-url` instead of `--index-url`. However, using
`--extra-index-url` makes you vulnerable to dependency confusion attacks because it checks the PyPi
repository for the package before it checks the custom repository. `--extra-index-url` adds the
provided URL as an additional registry which the client checks if the package is present.
`--index-url` tells the client to check for the package on the provided URL only.
If you were following the guide and want to install the
`MyPyPiPackage` package, you can run:
```shell
pip install mypypipackage --no-deps --index-url https://<personal_access_token_name>:<personal_access_token>@gitlab.example.com/api/v4/projects/<your_project_id>/packages/pypi/simple
```
This message indicates that the package was installed successfully:
```plaintext
Looking in indexes: https://<personal_access_token_name>:****@gitlab.example.com/api/v4/projects/<your_project_id>/packages/pypi/simple
Collecting mypypipackage
Downloading https://gitlab.example.com/api/v4/projects/<your_project_id>/packages/pypi/files/d53334205552a355fee8ca35a164512ef7334f33d309e60240d57073ee4386e6/mypypipackage-0.0.1-py3-none-any.whl (1.6 kB)
Installing collected packages: mypypipackage
Successfully installed mypypipackage-0.0.1
```
### Install from the group level
To install the latest version of a package from a group, use the following command:
```shell
pip install --index-url https://<personal_access_token_name>:<personal_access_token>@gitlab.example.com/api/v4/groups/<group_id>/-/packages/pypi/simple --no-deps <package_name>
```
In this command:
- `<package_name>` is the package name.
- `<personal_access_token_name>` is a personal access token name with the `read_api` scope.
- `<personal_access_token>` is a personal access token with the `read_api` scope.
- `<group_id>` is the group ID.
In these commands, you can use `--extra-index-url` instead of `--index-url`. However, using
`--extra-index-url` makes you vulnerable to dependency confusion attacks because it checks the PyPi
repository for the package before it checks the custom repository. `--extra-index-url` adds the
provided URL as an additional registry which the client checks if the package is present.
`--index-url` tells the client to check for the package at the provided URL only.
If you're following the guide and want to install the `MyPyPiPackage` package, you can run:
```shell
pip install mypypipackage --no-deps --index-url https://<personal_access_token_name>:<personal_access_token>@gitlab.example.com/api/v4/groups/<your_group_id>/-/packages/pypi/simple
```
### Package names
GitLab looks for packages that use
[Python normalized names (PEP-503)](https://www.python.org/dev/peps/pep-0503/#normalized-names).
The characters `-`, `_`, and `.` are all treated the same, and repeated
characters are removed.
A `pip install` request for `my.package` looks for packages that match any of
the three characters, such as `my-package`, `my_package`, and `my....package`.
## Using `requirements.txt`
If you want pip to access your private registry, add the `--extra-index-url` parameter along with the URL for your registry to your `requirements.txt` file.
```plaintext
--extra-index-url https://gitlab.example.com/api/v4/projects/<project_id>/packages/pypi/simple
package-name==1.0.0
```
If this is a private registry, you can authenticate in a couple of ways. For example:
- Using your `requirements.txt` file:
```plaintext
--extra-index-url https://__token__:<your_personal_token>@gitlab.example.com/api/v4/projects/<project_id>/packages/pypi/simple
package-name==1.0.0
```
- Using a `~/.netrc` file:
```plaintext
machine gitlab.example.com
login __token__
password <your_personal_token>
```
## Troubleshooting
To improve performance, the pip command caches files related to a package. Note that pip doesn't remove data by
itself. The cache grows as new packages are installed. If you encounter issues, clear the cache with
this command:
```shell
pip cache purge
```
### Multiple `index-url` or `extra-index-url` parameters
You can define multiple `index-url` and `extra-index-url` parameters.
If you use the same domain name (such as `gitlab.example.com`) multiple times with different authentication
tokens, `pip` may not be able to find your packages. This problem is due to how `pip`
[registers and stores your tokens](https://github.com/pypa/pip/pull/10904#issuecomment-1126690115) during commands executions.
To workaround this issue, you can use a [group deploy token](../../project/deploy_tokens/index.md) with the
scope `read_package_registry` from a common parent group for all projects or groups targeted by the
`index-url` and `extra-index-url` values.
## Supported CLI commands
The GitLab PyPI repository supports the following CLI commands:
- `twine upload`: Upload a package to the registry.
- `pip install`: Install a PyPI package from the registry.
|