1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
263
264
265
266
267
268
269
270
271
272
273
274
275
276
277
278
279
280
281
282
283
284
285
286
287
288
289
290
291
292
293
294
295
296
297
298
299
300
301
302
303
304
305
306
307
308
309
310
311
312
313
314
315
316
317
318
319
320
321
322
323
324
325
326
327
328
329
330
331
332
333
334
335
336
337
338
339
340
341
342
343
344
345
346
347
348
349
350
|
---
stage: Package
group: Package
info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/engineering/ux/technical-writing/#assignments
---
# PyPI packages in the Package Registry
> - [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/208747) in [GitLab Premium](https://about.gitlab.com/pricing/) 12.10.
> - [Moved](https://gitlab.com/gitlab-org/gitlab/-/issues/221259) to GitLab Core in 13.3.
Publish PyPI packages in your project’s Package Registry. Then install the
packages whenever you need to use them as a dependency.
The Package Registry works with:
- [pip](https://pypi.org/project/pip/)
- [twine](https://pypi.org/project/twine/)
## Build a PyPI package
This section explains how to create a PyPI package.
If you already use PyPI and know how to build your own packages, go to the
[next section](#authenticate-with-the-package-registry).
### Install pip and twine
Install a recent version of [pip](https://pypi.org/project/pip/) and
[twine](https://pypi.org/project/twine/).
### Create a project
Create a test project.
1. Open your terminal.
1. Create a directory called `MyPyPiPackage`, and then go to that directory:
```shell
mkdir MyPyPiPackage && cd MyPyPiPackage
```
1. Create another directory and go to it:
```shell
mkdir mypypipackage && cd mypypipackage
```
1. Create the required files in this directory:
```shell
touch __init__.py
touch greet.py
```
1. Open the `greet.py` file, and then add:
```python
def SayHello():
print("Hello from MyPyPiPackage")
return
```
1. Open the `__init__.py` file, and then add:
```python
from .greet import SayHello
```
1. To test the code, in your `MyPyPiPackage` directory, start the Python prompt.
```shell
python
```
1. Run this command:
```python
>>> from mypypipackage import SayHello
>>> SayHello()
```
A message indicates that the project was set up successfully:
```plaintext
Python 3.8.2 (v3.8.2:7b3ab5921f, Feb 24 2020, 17:52:18)
[Clang 6.0 (clang-600.0.57)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> from mypypipackage import SayHello
>>> SayHello()
Hello from MyPyPiPackage
```
### Create a package
After you create a project, you can create a package.
1. In your terminal, go to the `MyPyPiPackage` directory.
1. Create a `setup.py` file:
```shell
touch setup.py
```
This file contains all the information about the package. For more information
about this file, see [creating setup.py](https://packaging.python.org/tutorials/packaging-projects/#creating-setup-py).
Because GitLab identifies packages based on
[Python normalized names (PEP-503)](https://www.python.org/dev/peps/pep-0503/#normalized-names),
ensure your package name meets these requirements. See the [installation section](#authenticate-with-a-ci-job-token)
for details.
1. Open the `setup.py` file, and then add basic information:
```python
import setuptools
setuptools.setup(
name="mypypipackage",
version="0.0.1",
author="Example Author",
author_email="author@example.com",
description="A small example package",
packages=setuptools.find_packages(),
classifiers=[
"Programming Language :: Python :: 3",
"License :: OSI Approved :: MIT License",
"Operating System :: OS Independent",
],
python_requires='>=3.6',
)
```
1. Save the file.
1. Execute the setup:
```shell
python3 setup.py sdist bdist_wheel
```
The output should be visible in a newly-created `dist` folder:
```shell
ls dist
```
The output should appear similar to the following:
```plaintext
mypypipackage-0.0.1-py3-none-any.whl mypypipackage-0.0.1.tar.gz
```
The package is now ready to be published to the Package Registry.
## Authenticate with the Package Registry
Before you can publish to the Package Registry, you must authenticate.
To do this, you can use:
- A [personal access token](../../../user/profile/personal_access_tokens.md)
with the scope set to `api`.
- A [deploy token](../../project/deploy_tokens/index.md) with the scope set to
`read_package_registry`, `write_package_registry`, or both.
- A [CI job token](#authenticate-with-a-ci-job-token).
### Authenticate with a personal access token
To authenticate with a personal access token, edit the `~/.pypirc` file and add:
```ini
[distutils]
index-servers =
gitlab
[gitlab]
repository = https://gitlab.example.com/api/v4/projects/<project_id>/packages/pypi
username = __token__
password = <your personal access token>
```
- `username` must be `__token__` exactly.
- Your project ID is on your project's home page.
### Authenticate with a deploy token
To authenticate with a deploy token, edit your `~/.pypirc` file and add:
```ini
[distutils]
index-servers =
gitlab
[gitlab]
repository = https://gitlab.example.com/api/v4/projects/<project_id>/packages/pypi
username = <deploy token username>
password = <deploy token>
```
Your project ID is on your project's home page.
### Authenticate with a CI job token
> [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/202012) in GitLab 13.4.
To work with PyPI commands within [GitLab CI/CD](../../../ci/README.md), you
can use `CI_JOB_TOKEN` instead of a personal access token or deploy token.
For example:
```yaml
image: python:latest
run:
script:
- pip install twine
- python setup.py sdist bdist_wheel
- TWINE_PASSWORD=${CI_JOB_TOKEN} TWINE_USERNAME=gitlab-ci-token python -m twine upload --repository-url https://gitlab.example.com/api/v4/projects/${CI_PROJECT_ID}/packages/pypi dist/*
```
You can also use `CI_JOB_TOKEN` in a `~/.pypirc` file that you check in to
GitLab:
```ini
[distutils]
index-servers =
gitlab
[gitlab]
repository = https://gitlab.example.com/api/v4/projects/${env.CI_PROJECT_ID}/packages/pypi
username = gitlab-ci-token
password = ${env.CI_JOB_TOKEN}
```
## Publish a PyPI package
When publishing packages, note that:
- You must [authenticate with the Package Registry](#authenticate-with-the-package-registry).
- Your [version string must be valid](#ensure-your-version-string-is-valid).
- The maximum allowed package size is 5 GB.
- You can't upload the same version of a package multiple times. If you try,
you receive the error `Validation failed: File name has already been taken`.
- You cannot publish PyPI packages to a group, only to a project.
### Ensure your version string is valid
If your version string (for example, `0.0.1`) isn't valid, it gets rejected.
GitLab uses the following regex to validate the version string.
```ruby
\A(?:
v?
(?:([0-9]+)!)? (?# epoch)
([0-9]+(?:\.[0-9]+)*) (?# release segment)
([-_\.]?((a|b|c|rc|alpha|beta|pre|preview))[-_\.]?([0-9]+)?)? (?# pre-release)
((?:-([0-9]+))|(?:[-_\.]?(post|rev|r)[-_\.]?([0-9]+)?))? (?# post release)
([-_\.]?(dev)[-_\.]?([0-9]+)?)? (?# dev release)
(?:\+([a-z0-9]+(?:[-_\.][a-z0-9]+)*))? (?# local version)
)\z}xi
```
You can experiment with the regex and try your version strings by using this
[regular expression editor](https://rubular.com/r/FKM6d07ouoDaFV).
For more details about the regex, review this [documentation](https://www.python.org/dev/peps/pep-0440/#appendix-b-parsing-version-strings-with-regular-expressions).
### Publish a PyPI package by using twine
To publish a PyPI package, run a command like:
```shell
python3 -m twine upload --repository gitlab dist/*
```
This message indicates that the package was published successfully:
```plaintext
Uploading distributions to https://gitlab.example.com/api/v4/projects/<your_project_id>/packages/pypi
Uploading mypypipackage-0.0.1-py3-none-any.whl
100%|███████████████████████████████████████████████████████████████████████████████████████████| 4.58k/4.58k [00:00<00:00, 10.9kB/s]
Uploading mypypipackage-0.0.1.tar.gz
100%|███████████████████████████████████████████████████████████████████████████████████████████| 4.24k/4.24k [00:00<00:00, 11.0kB/s]
```
To view the published package, go to your project's **Packages & Registries**
page.
If you didn't use a `.pypirc` file to define your repository source, you can
publish to the repository with the authentication inline:
```shell
TWINE_PASSWORD=<personal_access_token or deploy_token> TWINE_USERNAME=<username or deploy_token_username> python3 -m twine upload --repository-url https://gitlab.example.com/api/v4/projects/<project_id>/packages/pypi dist/*
```
If you didn't follow the steps on this page, ensure your package was properly
built, and that you [created a PyPI package with `setuptools`](https://packaging.python.org/tutorials/packaging-projects/).
You can then upload your package by using the following command:
```shell
python -m twine upload --repository <source_name> dist/<package_file>
```
- `<package_file>` is your package filename, ending in `.tar.gz` or `.whl`.
- `<source_name>` is the [source name used during setup](#authenticate-with-the-package-registry).
### Publishing packages with the same name or version
You cannot publish a package if a package of the same name and version already exists.
You must delete the existing package first. If you attempt to publish the same package
more than once, a `404 Bad request` error occurs.
## Install a PyPI package
To install the latest version of a package, use the following command:
```shell
pip install --extra-index-url https://__token__:<personal_access_token>@gitlab.example.com/api/v4/projects/<project_id>/packages/pypi/simple --no-deps <package_name>
```
- `<package_name>` is the package name.
- `<personal_access_token>` is a personal access token with the `read_api` scope.
- `<project_id>` is the project ID.
If you were following the guide and want to install the
`MyPyPiPackage` package, you can run:
```shell
pip install mypypipackage --no-deps --extra-index-url https://__token__:<personal_access_token>@gitlab.example.com/api/v4/projects/<your_project_id>/packages/pypi/simple
```
This message indicates that the package was installed successfully:
```plaintext
Looking in indexes: https://__token__:****@gitlab.example.com/api/v4/projects/<your_project_id>/packages/pypi/simple
Collecting mypypipackage
Downloading https://gitlab.example.com/api/v4/projects/<your_project_id>/packages/pypi/files/d53334205552a355fee8ca35a164512ef7334f33d309e60240d57073ee4386e6/mypypipackage-0.0.1-py3-none-any.whl (1.6 kB)
Installing collected packages: mypypipackage
Successfully installed mypypipackage-0.0.1
```
### Package names
GitLab looks for packages that use
[Python normalized names (PEP-503)](https://www.python.org/dev/peps/pep-0503/#normalized-names).
The characters `-`, `_`, and `.` are all treated the same, and repeated
characters are removed.
A `pip install` request for `my.package` looks for packages that match any of
the three characters, such as `my-package`, `my_package`, and `my....package`.
|