1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
263
264
265
266
267
268
269
270
271
272
273
274
275
276
277
278
279
280
281
282
283
284
285
286
287
288
289
290
291
292
293
294
295
296
297
298
299
300
301
302
303
304
305
306
307
308
309
310
311
312
313
314
315
316
317
318
319
320
321
322
323
324
325
326
327
328
329
330
331
332
333
|
---
stage: Package
group: Package
info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/engineering/ux/technical-writing/#designated-technical-writers
---
# GitLab PyPi Repository
> - [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/208747) in [GitLab Premium](https://about.gitlab.com/pricing/) 12.10.
> - [Moved](https://gitlab.com/gitlab-org/gitlab/-/issues/221259) to GitLab Core in 13.3.
With the GitLab PyPi Repository, every project can have its own space to store PyPi packages.
The GitLab PyPi Repository works with:
- [pip](https://pypi.org/project/pip/)
- [twine](https://pypi.org/project/twine/)
## Setting up your development environment
You need a recent version of [pip](https://pypi.org/project/pip/) and [twine](https://pypi.org/project/twine/).
## Enabling the PyPi Repository
NOTE: **Note:**
This option is available only if your GitLab administrator has
[enabled support for the Package Registry](../../../administration/packages/index.md).
After the PyPi Repository is enabled, it is available for all new projects
by default. To enable it for existing projects, or if you want to disable it:
1. Navigate to your project's **Settings > General > Visibility, project features, permissions**.
1. Find the Packages feature and enable or disable it.
1. Click on **Save changes** for the changes to take effect.
You should then be able to see the **Packages & Registries** section on the left sidebar.
## Getting started
This section covers creating a new example PyPi package to upload. This is a
quickstart to test out the **GitLab PyPi Registry**. If you already understand how
to build and publish your own packages, move on to the [next section](#adding-the-gitlab-pypi-repository-as-a-source).
### Create a project
Understanding how to create a full Python project is outside the scope of this
guide, but you can create a small package to test out the registry. Start by
creating a new directory called `MyPyPiPackage`:
```shell
mkdir MyPyPiPackage && cd MyPyPiPackage
```
After creating this, create another directory inside:
```shell
mkdir mypypipackage && cd mypypipackage
```
Create two new files inside this directory to set up the basic project:
```shell
touch __init__.py
touch greet.py
```
Inside `greet.py`, add the following code:
```python
def SayHello():
print("Hello from MyPyPiPackage")
return
```
Inside the `__init__.py` file, add the following:
```python
from .greet import SayHello
```
Now that the basics of our project is completed, we can test that the code runs.
Start the Python prompt inside your top `MyPyPiPackage` directory. Then run:
```python
>>> from mypypipackage import SayHello
>>> SayHello()
```
You should see an output similar to the following:
```plaintext
Python 3.8.2 (v3.8.2:7b3ab5921f, Feb 24 2020, 17:52:18)
[Clang 6.0 (clang-600.0.57)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> from mypypipackage import SayHello
>>> SayHello()
Hello from MyPyPiPackage
```
Once we've verified that the sample project is working as above, we can next
work on creating a package.
### Create a package
Inside your `MyPyPiPackage` directory, we need to create a `setup.py` file. Run
the following:
```shell
touch setup.py
```
This file contains all the information about our package. For more information
about this file, see [creating setup.py](https://packaging.python.org/tutorials/packaging-projects/#creating-setup-py).
For this guide, we don't need to extensively fill out this file, simply add the
below to your `setup.py`:
```python
import setuptools
setuptools.setup(
name="mypypipackage",
version="0.0.1",
author="Example Author",
author_email="author@example.com",
description="A small example package",
packages=setuptools.find_packages(),
classifiers=[
"Programming Language :: Python :: 3",
"License :: OSI Approved :: MIT License",
"Operating System :: OS Independent",
],
python_requires='>=3.6',
)
```
Save the file, then execute the setup like so:
```shell
python3 setup.py sdist bdist_wheel
```
If successful, you should be able to see the output in a newly created `dist`
folder. Run:
```shell
ls dist
```
And confirm your output matches the below:
```plaintext
mypypipackage-0.0.1-py3-none-any.whl mypypipackage-0.0.1.tar.gz
```
Our package is now all set up and ready to be uploaded to the **GitLab PyPi
Package Registry**. Before we do so, we next need to set up authentication.
## Adding the GitLab PyPi Repository as a source
### Authenticating with a personal access token
You need the following:
- A personal access token. You can generate a [personal access token](../../../user/profile/personal_access_tokens.md) with the scope set to `api` for repository authentication.
- A suitable name for your source.
- Your project ID which can be found on the home page of your project.
Edit your `~/.pypirc` file and add the following:
```ini
[distutils]
index-servers =
gitlab
[gitlab]
repository = https://gitlab.com/api/v4/projects/<project_id>/packages/pypi
username = __token__
password = <your personal access token>
```
### Authenticating with a deploy token
You need the following:
- A deploy token. You can generate a [deploy token](./../../project/deploy_tokens/index.md) with the `read_package_registry` and/or `write_package_registry` scopes for repository authentication.
- A suitable name for your source.
- Your project ID which can be found on the home page of your project.
Edit your `~/.pypirc` file and add the following:
```ini
[distutils]
index-servers =
gitlab
[gitlab]
repository = https://gitlab.com/api/v4/projects/<project_id>/packages/pypi
username = <deploy token username>
password = <deploy token>
```
## Uploading packages
When uploading packages, note that:
- The maximum allowed size is 50 Megabytes.
- You cannot upload the same version of a package multiple times. If you try, you receive the error `Validation failed: File name has already been taken`.
### Ensure your version string is valid
If your version string (for example, `0.0.1`) is invalid, it will be rejected. GitLab uses the following regex to validate the version string.
```ruby
\A(?:
v?
(?:([0-9]+)!)? (?# epoch)
([0-9]+(?:\.[0-9]+)*) (?# release segment)
([-_\.]?((a|b|c|rc|alpha|beta|pre|preview))[-_\.]?([0-9]+)?)? (?# pre-release)
((?:-([0-9]+))|(?:[-_\.]?(post|rev|r)[-_\.]?([0-9]+)?))? (?# post release)
([-_\.]?(dev)[-_\.]?([0-9]+)?)? (?# dev release)
(?:\+([a-z0-9]+(?:[-_\.][a-z0-9]+)*))? (?# local version)
)\z}xi
```
You can play around with the regex and try your version strings on [this regular expression editor](https://rubular.com/r/FKM6d07ouoDaFV).
For more details about the regex used, please check the [documentation here](https://www.python.org/dev/peps/pep-0440/#appendix-b-parsing-version-strings-with-regular-expressions))
### Upload packages with Twine
If you were following the guide above, then the `MyPyPiPackage` package should
be ready to be uploaded. Run the following command:
```shell
python3 -m twine upload --repository gitlab dist/*
```
If successful, you should see the following:
```plaintext
Uploading distributions to https://gitlab.com/api/v4/projects/<your_project_id>/packages/pypi
Uploading mypypipackage-0.0.1-py3-none-any.whl
100%|███████████████████████████████████████████████████████████████████████████████████████████| 4.58k/4.58k [00:00<00:00, 10.9kB/s]
Uploading mypypipackage-0.0.1.tar.gz
100%|███████████████████████████████████████████████████████████████████████████████████████████| 4.24k/4.24k [00:00<00:00, 11.0kB/s]
```
This indicates that the package was uploaded successfully. You can then navigate
to your project's **Packages & Registries** page and see the uploaded packages.
If you would rather not use a `.pypirc` file to define your repository source,
you can upload to the repository with the authentication inline:
```shell
TWINE_PASSWORD=<personal_access_token or deploy_token> TWINE_USERNAME=<username or deploy_token_username> python3 -m twine upload --repository-url https://gitlab.com/api/v4/projects/<project_id>/packages/pypi dist/*
```
If you did not follow the guide above, then you need to ensure your package
has been properly built and you [created a PyPi package with `setuptools`](https://packaging.python.org/tutorials/packaging-projects/).
You can then upload your package using the following command:
```shell
python -m twine upload --repository <source_name> dist/<package_file>
```
Where:
- `<package_file>` is your package filename, ending in `.tar.gz` or `.whl`.
- `<source_name>` is the [source name used during setup](#adding-the-gitlab-pypi-repository-as-a-source).
## Install packages
Install the latest version of a package using the following command:
```shell
pip install --index-url https://__token__:<personal_access_token>@gitlab.com/api/v4/projects/<project_id>/packages/pypi/simple --no-deps <package_name>
```
Where:
- `<package_name>` is the package name.
- `<personal_access_token>` is a personal access token with the `read_api` scope.
- `<project_id>` is the project ID.
If you were following the guide above and want to test installing the
`MyPyPiPackage` package, you can run the following:
```shell
pip install mypypipackage --no-deps --index-url https://__token__:<personal_access_token>@gitlab.com/api/v4/projects/<your_project_id>/packages/pypi/simple
```
This should result in the following:
```plaintext
Looking in indexes: https://__token__:****@gitlab.com/api/v4/projects/<your_project_id>/packages/pypi/simple
Collecting mypypipackage
Downloading https://gitlab.com/api/v4/projects/<your_project_id>/packages/pypi/files/d53334205552a355fee8ca35a164512ef7334f33d309e60240d57073ee4386e6/mypypipackage-0.0.1-py3-none-any.whl (1.6 kB)
Installing collected packages: mypypipackage
Successfully installed mypypipackage-0.0.1
```
## Using GitLab CI with PyPI packages
> [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/11678) in GitLab 13.4.
To work with PyPI commands within [GitLab CI/CD](./../../../ci/README.md), you can use
`CI_JOB_TOKEN` in place of the personal access token or deploy token in your commands.
For example:
```yaml
image: python:latest
run:
script:
- pip install twine
- python setup.py sdist bdist_wheel
- TWINE_PASSWORD=${CI_JOB_TOKEN} TWINE_USERNAME=gitlab-ci-token python -m twine upload --repository-url https://gitlab.com/api/v4/projects/${CI_PROJECT_ID}/packages/pypi dist/*
```
You can also use `CI_JOB_TOKEN` in a `~/.pypirc` file that you check into GitLab:
```ini
[distutils]
index-servers =
gitlab
[gitlab]
repository = https://gitlab.com/api/v4/projects/${env.CI_PROJECT_ID}/packages/pypi
username = gitlab-ci-token
password = ${env.CI_JOB_TOKEN}
```
|