diff options
author | GitLab Bot <gitlab-bot@gitlab.com> | 2020-11-19 11:27:35 +0300 |
---|---|---|
committer | GitLab Bot <gitlab-bot@gitlab.com> | 2020-11-19 11:27:35 +0300 |
commit | 7e9c479f7de77702622631cff2628a9c8dcbc627 (patch) | |
tree | c8f718a08e110ad7e1894510980d2155a6549197 /doc/user/packages/pypi_repository | |
parent | e852b0ae16db4052c1c567d9efa4facc81146e88 (diff) |
Add latest changes from gitlab-org/gitlab@13-6-stable-eev13.6.0-rc42
Diffstat (limited to 'doc/user/packages/pypi_repository')
-rw-r--r-- | doc/user/packages/pypi_repository/index.md | 363 |
1 files changed, 180 insertions, 183 deletions
diff --git a/doc/user/packages/pypi_repository/index.md b/doc/user/packages/pypi_repository/index.md index 33c37ee6a6c..83b29d5f53a 100644 --- a/doc/user/packages/pypi_repository/index.md +++ b/doc/user/packages/pypi_repository/index.md @@ -4,89 +4,83 @@ group: Package info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/engineering/ux/technical-writing/#designated-technical-writers --- -# GitLab PyPI Repository +# PyPI packages in the Package Registry > - [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/208747) in [GitLab Premium](https://about.gitlab.com/pricing/) 12.10. > - [Moved](https://gitlab.com/gitlab-org/gitlab/-/issues/221259) to GitLab Core in 13.3. -With the GitLab PyPI Repository, every project can have its own space to store PyPI packages. +Publish PyPI packages in your project’s Package Registry. Then install the +packages whenever you need to use them as a dependency. -The GitLab PyPI Repository works with: +The Package Registry works with: - [pip](https://pypi.org/project/pip/) - [twine](https://pypi.org/project/twine/) -## Setting up your development environment +## Build a PyPI package -You need a recent version of [pip](https://pypi.org/project/pip/) and [twine](https://pypi.org/project/twine/). +This section explains how to create a PyPI package. -## Enabling the PyPI Repository +If you already use PyPI and know how to build your own packages, go to the +[next section](#authenticate-with-the-package-registry). -NOTE: **Note:** -This option is available only if your GitLab administrator has -[enabled support for the Package Registry](../../../administration/packages/index.md). +### Install pip and twine -After the PyPI Repository is enabled, it is available for all new projects -by default. To enable it for existing projects, or if you want to disable it: +Install a recent version of [pip](https://pypi.org/project/pip/) and +[twine](https://pypi.org/project/twine/). -1. Navigate to your project's **Settings > General > Visibility, project features, permissions**. -1. Find the Packages feature and enable or disable it. -1. Click on **Save changes** for the changes to take effect. +### Create a project -You should then be able to see the **Packages & Registries** section on the left sidebar. +Create a test project. -## Getting started +1. Open your terminal. +1. Create a directory called `MyPyPiPackage`, and then go to that directory: -This section covers creating a new example PyPI package to upload. This is a -quickstart to test out the **GitLab PyPI Registry**. If you already understand how -to build and publish your own packages, move on to the [next section](#adding-the-gitlab-pypi-repository-as-a-source). + ```shell + mkdir MyPyPiPackage && cd MyPyPiPackage + ``` -### Create a project +1. Create another directory and go to it: -Understanding how to create a full Python project is outside the scope of this -guide, but you can create a small package to test out the registry. Start by -creating a new directory called `MyPyPiPackage`: + ```shell + mkdir mypypipackage && cd mypypipackage + ``` -```shell -mkdir MyPyPiPackage && cd MyPyPiPackage -``` +1. Create the required files in this directory: -After creating this, create another directory inside: + ```shell + touch __init__.py + touch greet.py + ``` -```shell -mkdir mypypipackage && cd mypypipackage -``` +1. Open the `greet.py` file, and then add: -Create two new files inside this directory to set up the basic project: + ```python + def SayHello(): + print("Hello from MyPyPiPackage") + return + ``` -```shell -touch __init__.py -touch greet.py -``` +1. Open the `__init__.py` file, and then add: -Inside `greet.py`, add the following code: + ```python + from .greet import SayHello + ``` -```python -def SayHello(): - print("Hello from MyPyPiPackage") - return -``` +1. To test the code, in your `MyPyPiPackage` directory, start the Python prompt. -Inside the `__init__.py` file, add the following: + ```shell + python + ``` -```python -from .greet import SayHello -``` +1. Run this command: -Now that the basics of our project is completed, we can test that the code runs. -Start the Python prompt inside your top `MyPyPiPackage` directory. Then run: + ```python + >>> from mypypipackage import SayHello + >>> SayHello() + ``` -```python ->>> from mypypipackage import SayHello ->>> SayHello() -``` - -You should see an output similar to the following: +A message indicates that the project was set up successfully: ```plaintext Python 3.8.2 (v3.8.2:7b3ab5921f, Feb 24 2020, 17:52:18) @@ -97,80 +91,81 @@ Type "help", "copyright", "credits" or "license" for more information. Hello from MyPyPiPackage ``` -Once we've verified that the sample project is working as above, we can next -work on creating a package. - ### Create a package -Inside your `MyPyPiPackage` directory, we need to create a `setup.py` file. Run -the following: +After you create a project, you can create a package. -```shell -touch setup.py -``` +1. In your terminal, go to the `MyPyPiPackage` directory. +1. Create a `setup.py` file: -This file contains all the information about our package. For more information -about this file, see [creating setup.py](https://packaging.python.org/tutorials/packaging-projects/#creating-setup-py). -GitLab identifies packages based on -[Python normalized names (PEP-503)](https://www.python.org/dev/peps/pep-0503/#normalized-names), -so ensure your package name meets these requirements. -See the [installation section](#install-packages) for more details. - -For this guide, we don't need to extensively fill out this file, simply add the -below to your `setup.py`: - -```python -import setuptools - -setuptools.setup( - name="mypypipackage", - version="0.0.1", - author="Example Author", - author_email="author@example.com", - description="A small example package", - packages=setuptools.find_packages(), - classifiers=[ - "Programming Language :: Python :: 3", - "License :: OSI Approved :: MIT License", - "Operating System :: OS Independent", - ], - python_requires='>=3.6', -) -``` + ```shell + touch setup.py + ``` -Save the file, then execute the setup like so: + This file contains all the information about the package. For more information + about this file, see [creating setup.py](https://packaging.python.org/tutorials/packaging-projects/#creating-setup-py). + Because GitLab identifies packages based on + [Python normalized names (PEP-503)](https://www.python.org/dev/peps/pep-0503/#normalized-names), + ensure your package name meets these requirements. See the [installation section](#authenticate-with-a-ci-job-token) + for details. -```shell -python3 setup.py sdist bdist_wheel -``` +1. Open the `setup.py` file, and then add basic information: + + ```python + import setuptools -If successful, you should be able to see the output in a newly created `dist` -folder. Run: + setuptools.setup( + name="mypypipackage", + version="0.0.1", + author="Example Author", + author_email="author@example.com", + description="A small example package", + packages=setuptools.find_packages(), + classifiers=[ + "Programming Language :: Python :: 3", + "License :: OSI Approved :: MIT License", + "Operating System :: OS Independent", + ], + python_requires='>=3.6', + ) + ``` + +1. Save the file. +1. Execute the setup: + + ```shell + python3 setup.py sdist bdist_wheel + ``` + +The output should be visible in a newly-created `dist` folder: ```shell ls dist ``` -And confirm your output matches the below: +The output should appear similar to the following: ```plaintext mypypipackage-0.0.1-py3-none-any.whl mypypipackage-0.0.1.tar.gz ``` -Our package is now all set up and ready to be uploaded to the **GitLab PyPI -Package Registry**. Before we do so, we next need to set up authentication. +The package is now ready to be published to the Package Registry. -## Adding the GitLab PyPI Repository as a source +## Authenticate with the Package Registry -### Authenticating with a personal access token +Before you can publish to the Package Registry, you must authenticate. -You need the following: +To do this, you can use: -- A personal access token. You can generate a [personal access token](../../../user/profile/personal_access_tokens.md) with the scope set to `api` for repository authentication. -- A suitable name for your source. -- Your project ID which can be found on the home page of your project. +- A [personal access token](../../../user/profile/personal_access_tokens.md) + with the scope set to `api`. +- A [deploy token](../../project/deploy_tokens/index.md) with the scope set to + `read_package_registry`, `write_package_registry`, or both. +- A [CI job token](#authenticate-with-a-ci-job-token). -Edit your `~/.pypirc` file and add the following: +### Authenticate with a personal access token + +To authenticate with a personal access token, edit the `~/.pypirc` file and add: ```ini [distutils] @@ -178,20 +173,17 @@ index-servers = gitlab [gitlab] -repository = https://gitlab.com/api/v4/projects/<project_id>/packages/pypi +repository = https://gitlab.example.com/api/v4/projects/<project_id>/packages/pypi username = __token__ password = <your personal access token> ``` -### Authenticating with a deploy token - -You need the following: +- `username` must be `__token__` exactly. +- Your project ID is on your project's home page. -- A deploy token. You can generate a [deploy token](./../../project/deploy_tokens/index.md) with the `read_package_registry` and/or `write_package_registry` scopes for repository authentication. -- A suitable name for your source. -- Your project ID which can be found on the home page of your project. +### Authenticate with a deploy token -Edit your `~/.pypirc` file and add the following: +To authenticate with a deploy token, edit your `~/.pypirc` file and add: ```ini [distutils] @@ -199,21 +191,58 @@ index-servers = gitlab [gitlab] -repository = https://gitlab.com/api/v4/projects/<project_id>/packages/pypi +repository = https://gitlab.example.com/api/v4/projects/<project_id>/packages/pypi username = <deploy token username> password = <deploy token> ``` -## Uploading packages +Your project ID is on your project's home page. -When uploading packages, note that: +### Authenticate with a CI job token -- The maximum allowed size is 50 Megabytes. -- You cannot upload the same version of a package multiple times. If you try, you receive the error `Validation failed: File name has already been taken`. +> [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/202012) in GitLab 13.4. + +To work with PyPI commands within [GitLab CI/CD](../../../ci/README.md), you +can use `CI_JOB_TOKEN` instead of a personal access token or deploy token. + +For example: + +```yaml +image: python:latest + +run: + script: + - pip install twine + - python setup.py sdist bdist_wheel + - TWINE_PASSWORD=${CI_JOB_TOKEN} TWINE_USERNAME=gitlab-ci-token python -m twine upload --repository-url https://gitlab.example.com/api/v4/projects/${CI_PROJECT_ID}/packages/pypi dist/* +``` + +You can also use `CI_JOB_TOKEN` in a `~/.pypirc` file that you check in to +GitLab: + +```ini +[distutils] +index-servers = + gitlab + +[gitlab] +repository = https://gitlab.example.com/api/v4/projects/${env.CI_PROJECT_ID}/packages/pypi +username = gitlab-ci-token +password = ${env.CI_JOB_TOKEN} +``` + +## Publish a PyPI package + +When publishing packages, note that: + +- The maximum allowed size is 50 MB. +- You can't upload the same version of a package multiple times. If you try, + you'll receive the error `Validation failed: File name has already been taken`. ### Ensure your version string is valid -If your version string (for example, `0.0.1`) is invalid, it will be rejected. GitLab uses the following regex to validate the version string. +If your version string (for example, `0.0.1`) isn't valid, it will be rejected. +GitLab uses the following regex to validate the version string. ```ruby \A(?: @@ -227,118 +256,86 @@ If your version string (for example, `0.0.1`) is invalid, it will be rejected. G )\z}xi ``` -You can play around with the regex and try your version strings on [this regular expression editor](https://rubular.com/r/FKM6d07ouoDaFV). +You can experiment with the regex and try your version strings by using this +[regular expression editor](https://rubular.com/r/FKM6d07ouoDaFV). -For more details about the regex used, please check the [documentation here](https://www.python.org/dev/peps/pep-0440/#appendix-b-parsing-version-strings-with-regular-expressions)) +For more details about the regex, review this [documentation](https://www.python.org/dev/peps/pep-0440/#appendix-b-parsing-version-strings-with-regular-expressions). -### Upload packages with Twine +### Publish a PyPI package by using twine -If you were following the guide above, then the `MyPyPiPackage` package should -be ready to be uploaded. Run the following command: +To publish a PyPI package, run a command like: ```shell python3 -m twine upload --repository gitlab dist/* ``` -If successful, you should see the following: +This message indicates that the package was published successfully: ```plaintext -Uploading distributions to https://gitlab.com/api/v4/projects/<your_project_id>/packages/pypi +Uploading distributions to https://gitlab.example.com/api/v4/projects/<your_project_id>/packages/pypi Uploading mypypipackage-0.0.1-py3-none-any.whl 100%|███████████████████████████████████████████████████████████████████████████████████████████| 4.58k/4.58k [00:00<00:00, 10.9kB/s] Uploading mypypipackage-0.0.1.tar.gz 100%|███████████████████████████████████████████████████████████████████████████████████████████| 4.24k/4.24k [00:00<00:00, 11.0kB/s] ``` -This indicates that the package was uploaded successfully. You can then navigate -to your project's **Packages & Registries** page and see the uploaded packages. +To view the published package, go to your project's **Packages & Registries** +page. -If you would rather not use a `.pypirc` file to define your repository source, -you can upload to the repository with the authentication inline: +If you didn't use a `.pypirc` file to define your repository source, you can +publish to the repository with the authentication inline: ```shell -TWINE_PASSWORD=<personal_access_token or deploy_token> TWINE_USERNAME=<username or deploy_token_username> python3 -m twine upload --repository-url https://gitlab.com/api/v4/projects/<project_id>/packages/pypi dist/* +TWINE_PASSWORD=<personal_access_token or deploy_token> TWINE_USERNAME=<username or deploy_token_username> python3 -m twine upload --repository-url https://gitlab.example.com/api/v4/projects/<project_id>/packages/pypi dist/* ``` -If you did not follow the guide above, then you need to ensure your package -has been properly built and you [created a PyPI package with `setuptools`](https://packaging.python.org/tutorials/packaging-projects/). +If you didn't follow the steps on this page, ensure your package was properly +built, and that you [created a PyPI package with `setuptools`](https://packaging.python.org/tutorials/packaging-projects/). -You can then upload your package using the following command: +You can then upload your package by using the following command: ```shell python -m twine upload --repository <source_name> dist/<package_file> ``` -Where: - - `<package_file>` is your package filename, ending in `.tar.gz` or `.whl`. -- `<source_name>` is the [source name used during setup](#adding-the-gitlab-pypi-repository-as-a-source). +- `<source_name>` is the [source name used during setup](#authenticate-with-the-package-registry). -## Install packages +## Install a PyPI package -Install the latest version of a package using the following command: +To install the latest version of a package, use the following command: ```shell -pip install --extra-index-url https://__token__:<personal_access_token>@gitlab.com/api/v4/projects/<project_id>/packages/pypi/simple --no-deps <package_name> +pip install --extra-index-url https://__token__:<personal_access_token>@gitlab.example.com/api/v4/projects/<project_id>/packages/pypi/simple --no-deps <package_name> ``` -Where: - - `<package_name>` is the package name. - `<personal_access_token>` is a personal access token with the `read_api` scope. - `<project_id>` is the project ID. -If you were following the guide above and want to test installing the -`MyPyPiPackage` package, you can run the following: +If you were following the guide and want to install the +`MyPyPiPackage` package, you can run: ```shell -pip install mypypipackage --no-deps --extra-index-url https://__token__:<personal_access_token>@gitlab.com/api/v4/projects/<your_project_id>/packages/pypi/simple +pip install mypypipackage --no-deps --extra-index-url https://__token__:<personal_access_token>@gitlab.example.com/api/v4/projects/<your_project_id>/packages/pypi/simple ``` -This should result in the following: +This message indicates that the package was installed successfully: ```plaintext -Looking in indexes: https://__token__:****@gitlab.com/api/v4/projects/<your_project_id>/packages/pypi/simple +Looking in indexes: https://__token__:****@gitlab.example.com/api/v4/projects/<your_project_id>/packages/pypi/simple Collecting mypypipackage - Downloading https://gitlab.com/api/v4/projects/<your_project_id>/packages/pypi/files/d53334205552a355fee8ca35a164512ef7334f33d309e60240d57073ee4386e6/mypypipackage-0.0.1-py3-none-any.whl (1.6 kB) + Downloading https://gitlab.example.com/api/v4/projects/<your_project_id>/packages/pypi/files/d53334205552a355fee8ca35a164512ef7334f33d309e60240d57073ee4386e6/mypypipackage-0.0.1-py3-none-any.whl (1.6 kB) Installing collected packages: mypypipackage Successfully installed mypypipackage-0.0.1 ``` -GitLab looks for packages using -[Python normalized names (PEP-503)](https://www.python.org/dev/peps/pep-0503/#normalized-names), -so the characters `-`, `_`, and `.` are all treated the same and repeated characters are removed. -A `pip install` request for `my.package` looks for packages that match any of -the three characters, such as `my-package`, `my_package`, and `my....package`. - -## Using GitLab CI with PyPI packages +### Package names -> [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/202012) in GitLab 13.4. - -To work with PyPI commands within [GitLab CI/CD](./../../../ci/README.md), you can use -`CI_JOB_TOKEN` in place of the personal access token or deploy token in your commands. +GitLab looks for packages that use +[Python normalized names (PEP-503)](https://www.python.org/dev/peps/pep-0503/#normalized-names). +The characters `-`, `_`, and `.` are all treated the same, and repeated +characters are removed. -For example: - -```yaml -image: python:latest - -run: - script: - - pip install twine - - python setup.py sdist bdist_wheel - - TWINE_PASSWORD=${CI_JOB_TOKEN} TWINE_USERNAME=gitlab-ci-token python -m twine upload --repository-url https://gitlab.com/api/v4/projects/${CI_PROJECT_ID}/packages/pypi dist/* -``` - -You can also use `CI_JOB_TOKEN` in a `~/.pypirc` file that you check into GitLab: - -```ini -[distutils] -index-servers = - gitlab - -[gitlab] -repository = https://gitlab.com/api/v4/projects/${env.CI_PROJECT_ID}/packages/pypi -username = gitlab-ci-token -password = ${env.CI_JOB_TOKEN} -``` +A `pip install` request for `my.package` looks for packages that match any of +the three characters, such as `my-package`, `my_package`, and `my....package`. |