diff options
Diffstat (limited to 'doc/ci')
-rw-r--r-- | doc/ci/README.md | 27 | ||||
-rw-r--r-- | doc/ci/caching/index.md | 24 | ||||
-rw-r--r-- | doc/ci/chatops/README.md | 27 | ||||
-rw-r--r-- | doc/ci/ci_cd_for_external_repos/bitbucket_integration.md | 49 | ||||
-rw-r--r-- | doc/ci/ci_cd_for_external_repos/github_integration.md | 49 | ||||
-rw-r--r-- | doc/ci/ci_cd_for_external_repos/index.md | 24 | ||||
-rw-r--r-- | doc/ci/docker/README.md | 4 | ||||
-rw-r--r-- | doc/ci/docker/using_docker_build.md | 42 | ||||
-rw-r--r-- | doc/ci/docker/using_docker_images.md | 63 | ||||
-rw-r--r-- | doc/ci/docker/using_kaniko.md | 42 | ||||
-rw-r--r-- | doc/ci/enable_or_disable_ci.md | 59 | ||||
-rw-r--r-- | doc/ci/environments.md | 4 | ||||
-rw-r--r-- | doc/ci/environments/protected_environments.md | 16 | ||||
-rw-r--r-- | doc/ci/multi_project_pipelines.md | 8 |
14 files changed, 316 insertions, 122 deletions
diff --git a/doc/ci/README.md b/doc/ci/README.md index 93b82a065b3..635cce13b4e 100644 --- a/doc/ci/README.md +++ b/doc/ci/README.md @@ -161,6 +161,33 @@ See also the [Why CI/CD?](https://docs.google.com/presentation/d/1OGgk2Tcxbpl7DJ As GitLab CI/CD has evolved, certain breaking changes have been necessary. These are: +#### 12.0 + +- [Use refspec to clone/fetch git + repository](https://gitlab.com/gitlab-org/gitlab-runner/issues/4069). +- [Old cache + configuration](https://gitlab.com/gitlab-org/gitlab-runner/issues/4070). +- [Old metrics server + configuration](https://gitlab.com/gitlab-org/gitlab-runner/issues/4072). +- [Remove + `FF_K8S_USE_ENTRYPOINT_OVER_COMMAND`](https://gitlab.com/gitlab-org/gitlab-runner/issues/4073). +- [Remove Linux distributions that reach + EOL](https://gitlab.com/gitlab-org/gitlab-runner/merge_requests/1130). +- [Update command line API for helper + images](https://gitlab.com/gitlab-org/gitlab-runner/issues/4013). +- [Remove old `git clean` + flow](https://gitlab.com/gitlab-org/gitlab-runner/issues/4175). + +#### 11.0 + +- No breaking changes. + +#### 10.0 + +- No breaking changes. + +#### 9.0 + - [CI variables renaming for GitLab 9.0](variables/deprecated_variables.md#gitlab-90-renamed-variables). Read about the deprecated CI variables and what you should use for GitLab 9.0+. - [New CI job permissions model](../user/project/new_ci_build_permissions_model.md). diff --git a/doc/ci/caching/index.md b/doc/ci/caching/index.md index d703e106e76..9a5a3624c73 100644 --- a/doc/ci/caching/index.md +++ b/doc/ci/caching/index.md @@ -1,3 +1,7 @@ +--- +type: index, concepts, howto +--- + # Cache dependencies in GitLab CI/CD GitLab CI/CD provides a caching mechanism that can be used to save time @@ -60,7 +64,7 @@ In summary: - Caches are disabled if not defined globally or per job (using `cache:`). - Caches are available for all jobs in your `.gitlab-ci.yml` if enabled globally. -- Caches can be used by subsequent pipelines of that very same job (a script in +- Caches can be used by subsequent pipelines of that same job (a script in a stage) in which the cache was created (if not defined globally). - Caches are stored where the Runner is installed **and** uploaded to S3 if [distributed cache is enabled](https://docs.gitlab.com/runner/configuration/autoscale.html#distributed-runners-caching). @@ -87,7 +91,7 @@ cache, when declaring `cache` in your jobs, use one or a mix of the following: that share their cache. - [Use sticky Runners](../runners/README.md#locking-a-specific-runner-from-being-enabled-for-other-projects) that will be only available to a particular project. -- [Use a `key`](../yaml/README.md#cachekey) that fits your workflow (e.g., +- [Use a `key`](../yaml/README.md#cachekey) that fits your workflow (for example, different caches on each branch). For that, you can take advantage of the [CI/CD predefined variables](../variables/README.md#predefined-environment-variables). @@ -420,7 +424,7 @@ mismatch and a few ideas how to fix it. Let's explore some examples. ---- +#### Examples Let's assume you have only one Runner assigned to your project, so the cache will be stored in the Runner's machine by default. If two jobs, A and B, @@ -462,8 +466,6 @@ job B: To fix that, use different `keys` for each job. ---- - In another case, let's assume you have more than one Runners assigned to your project, but the distributed cache is not enabled. We want the second time the pipeline is run, `job A` and `job B` to re-use their cache (which in this case @@ -526,3 +528,15 @@ Behind the scenes, this works by increasing a counter in the database, and the value of that counter is used to create the key for the cache by appending an integer to it: `-1`, `-2`, etc. After a push, a new key is generated and the old cache is not valid anymore. + +<!-- ## Troubleshooting + +Include any troubleshooting steps that you can foresee. If you know beforehand what issues +one might have when setting this up, or when something is changed, or on upgrading, it's +important to describe those, too. Think of things that may go wrong and include them here. +This is important to minimize requests for support, and to avoid doc comments with +questions that you know someone might ask. + +Each scenario can be a third-level heading, e.g. `### Getting error message X`. +If you have none to add when creating a doc, leave this section in place +but commented out to help encourage others to add to it in the future. --> diff --git a/doc/ci/chatops/README.md b/doc/ci/chatops/README.md index 5ecdf0f8c54..241134783da 100644 --- a/doc/ci/chatops/README.md +++ b/doc/ci/chatops/README.md @@ -1,3 +1,7 @@ +--- +type: index, concepts, howto +--- + # GitLab ChatOps > **Notes:** @@ -10,7 +14,10 @@ GitLab ChatOps provides a method to interact with CI/CD jobs through chat servic ## How it works -GitLab ChatOps is built upon two existing features, [GitLab CI/CD](../README.md) and [Slack Slash Commmands](../../user/project/integrations/slack_slash_commands.md). +GitLab ChatOps is built upon two existing features: + +- [GitLab CI/CD](../README.md). +- [Slack Slash Commands](../../user/project/integrations/slack_slash_commands.md). A new `run` action has been added to the [slash commands](../../integration/slash_commands.md), which takes two arguments: a `<job name>` to execute and the `<job arguments>`. When executed, ChatOps will look up the specified job name and attempt to match it to a corresponding job in [.gitlab-ci.yml](../yaml/README.md). If a matching job is found on `master`, a pipeline containing just that job is scheduled. Two additional [CI/CD variables](../variables/README.md#predefined-environment-variables) are passed to the job: `CHAT_INPUT` contains any additional arguments, and `CHAT_CHANNEL` is set to the name of channel the action was triggered in. @@ -22,9 +29,9 @@ After the job has finished, its output is sent back to Slack provided it has com Since ChatOps is built upon GitLab CI/CD, the job has all the same features and functions available. There a few best practices to consider however when creating ChatOps jobs: -* It is strongly recommended to set `only: [chat]` so the job does not run as part of the standard CI pipeline. -* If the job is set to `when: manual`, the pipeline will be created however the job will wait to be started. -* It is important to keep in mind that there is very limited support for access control. If the user who triggered the slash command is a developer in the project, the job will run. The job itself can utilize existing [CI/CD variables](../variables/README.html#predefined-environment-variables) like `GITLAB_USER_ID` to perform additional rights validation, however these variables can be [overridden](../variables/README.html#priority-of-environment-variables). +- It is strongly recommended to set `only: [chat]` so the job does not run as part of the standard CI pipeline. +- If the job is set to `when: manual`, the pipeline will be created however the job will wait to be started. +- It is important to keep in mind that there is limited support for access control. If the user who triggered the slash command is a developer in the project, the job will run. The job itself can utilize existing [CI/CD variables](../variables/README.html#predefined-environment-variables) like `GITLAB_USER_ID` to perform additional rights validation, however these variables can be [overridden](../variables/README.html#priority-of-environment-variables). ### Controlling the ChatOps reply @@ -59,3 +66,15 @@ You can find and download the official GitLab ChatOps icon here. ![GitLab ChatOps bot icon](img/gitlab-chatops-icon-small.png) [Download bigger image](img/gitlab-chatops-icon.png) + +<!-- ## Troubleshooting + +Include any troubleshooting steps that you can foresee. If you know beforehand what issues +one might have when setting this up, or when something is changed, or on upgrading, it's +important to describe those, too. Think of things that may go wrong and include them here. +This is important to minimize requests for support, and to avoid doc comments with +questions that you know someone might ask. + +Each scenario can be a third-level heading, e.g. `### Getting error message X`. +If you have none to add when creating a doc, leave this section in place +but commented out to help encourage others to add to it in the future. --> diff --git a/doc/ci/ci_cd_for_external_repos/bitbucket_integration.md b/doc/ci/ci_cd_for_external_repos/bitbucket_integration.md index 1c8e12d229c..9126eb65f07 100644 --- a/doc/ci/ci_cd_for_external_repos/bitbucket_integration.md +++ b/doc/ci/ci_cd_for_external_repos/bitbucket_integration.md @@ -1,8 +1,15 @@ +--- +type: howto +--- + # Using GitLab CI/CD with a Bitbucket Cloud repository **[PREMIUM]** -GitLab CI/CD can be used with Bitbucket Cloud by creating a -[CI/CD project](https://docs.gitlab.com/ee/user/project/ci_cd_for_external_repo.html) and connecting -your Git repository via URL. +GitLab CI/CD can be used with Bitbucket Cloud by: + +1. Creating a [CI/CD project](https://docs.gitlab.com/ee/user/project/ci_cd_for_external_repo.html). +1. Connecting your Git repository via URL. + +To use GitLab CI/CD with a Bitbucket Cloud repository: 1. In GitLab create a **CI/CD for external repo**, select **Repo by URL** and create the project. @@ -16,13 +23,13 @@ your Git repository via URL. with `api` scope. This will be used to authenticate requests from the web hook that will be created in Bitbucket to notify GitLab of new commits. -1. In Bitbucket from **Settings > Webhooks** create a new web hook to notify +1. In Bitbucket, from **Settings > Webhooks**, create a new web hook to notify GitLab of new commits. The web hook URL should be set to the GitLab API to trigger pull mirroring, using the Personal Access Token we just generated for authentication. - ``` + ```text https://gitlab.com/api/v4/projects/<NAMESPACE>%2F<PROJECT>/mirror/pull?private_token=<PERSONAL_ACCESS_TOKEN> ``` @@ -33,27 +40,27 @@ your Git repository via URL. After saving, test the web hook by pushing a change to your Bitbucket repository. -1. In Bitbucket create an **App Password** from **Bitbucket Settings > App +1. In Bitbucket, create an **App Password** from **Bitbucket Settings > App Passwords** to authenticate the build status script setting commit build statuses in Bitbucket. Repository write permissions are required. ![Bitbucket Cloud webhook](img/bitbucket_app_password.png) -1. In GitLab from **Settings > CI/CD > Environment variables** add variables to allow - communication with Bitbucket via the Bitbucket API. +1. In GitLab, from **Settings > CI/CD > Environment variables**, add variables to allow + communication with Bitbucket via the Bitbucket API: - `BITBUCKET_ACCESS_TOKEN`: the Bitbucket app password created above + `BITBUCKET_ACCESS_TOKEN`: the Bitbucket app password created above. - `BITBUCKET_USERNAME`: the username of the Bitbucket account + `BITBUCKET_USERNAME`: the username of the Bitbucket account. - `BITBUCKET_NAMESPACE`: set this if your GitLab and Bitbucket namespaces differ + `BITBUCKET_NAMESPACE`: set this if your GitLab and Bitbucket namespaces differ. - `BITBUCKET_REPOSITORY`: set this if your GitLab and Bitbucket project names differ + `BITBUCKET_REPOSITORY`: set this if your GitLab and Bitbucket project names differ. -1. In Bitbucket add a script to push the pipeline status to Bitbucket. +1. In Bitbucket, add a script to push the pipeline status to Bitbucket. > Note: changes made in GitLab will be overwritten by any changes made - upstream in Bitbucket. + > upstream in Bitbucket. Create a file `build_status` and insert the script below and run `chmod +x build_status` in your terminal to make the script executable. @@ -111,7 +118,7 @@ your Git repository via URL. 1. Still in Bitbucket, create a `.gitlab-ci.yml` file to use the script to push pipeline success and failures to Bitbucket. - ``` + ```yaml stages: - test - ci_status @@ -145,3 +152,15 @@ GitLab is now configured to mirror changes from Bitbucket, run CI/CD pipelines configured in `.gitlab-ci.yml` and push the status to Bitbucket. [pull-mirroring]: ../../workflow/repository_mirroring.md#pulling-from-a-remote-repository-starter + +<!-- ## Troubleshooting + +Include any troubleshooting steps that you can foresee. If you know beforehand what issues +one might have when setting this up, or when something is changed, or on upgrading, it's +important to describe those, too. Think of things that may go wrong and include them here. +This is important to minimize requests for support, and to avoid doc comments with +questions that you know someone might ask. + +Each scenario can be a third-level heading, e.g. `### Getting error message X`. +If you have none to add when creating a doc, leave this section in place +but commented out to help encourage others to add to it in the future. --> diff --git a/doc/ci/ci_cd_for_external_repos/github_integration.md b/doc/ci/ci_cd_for_external_repos/github_integration.md index 0e2acf957e0..612dcc93bc1 100644 --- a/doc/ci/ci_cd_for_external_repos/github_integration.md +++ b/doc/ci/ci_cd_for_external_repos/github_integration.md @@ -1,3 +1,7 @@ +--- +type: howto +--- + # Using GitLab CI/CD with a GitHub repository **[PREMIUM]** GitLab CI/CD can be used with **GitHub.com** and **GitHub Enterprise** by @@ -15,7 +19,7 @@ administrator: NOTE: **Note:** Due to a 10-token limitation on the [GitHub OAuth Implementation](https://developer.github.com/apps/building-oauth-apps/authorizing-oauth-apps/#creating-multiple-tokens-for-oauth-apps), -if you import more than 10 times, your oldest imported project's token will be +if you import more than 10 times, your oldest imported project's token will be revoked. See issue [#9147](https://gitlab.com/gitlab-org/gitlab-ee/issues/9147) for more information. @@ -31,23 +35,27 @@ for more information. 1. In GitHub, add a `.gitlab-ci.yml` to [configure GitLab CI/CD](../quick_start/README.md). -GitLab will import the project, enable [Pull Mirroring](../../workflow/repository_mirroring.md#pulling-from-a-remote-repository-starter), enable -[GitHub project integration](https://docs.gitlab.com/ee/user/project/integrations/github.html), and create a web hook -on GitHub to notify GitLab of new commits. +GitLab will: + +1. Import the project. +1. Enable [Pull Mirroring](../../workflow/repository_mirroring.md#pulling-from-a-remote-repository-starter). +1. Enable [GitHub project integration](https://docs.gitlab.com/ee/user/project/integrations/github.html). +1. Create a web hook on GitHub to notify GitLab of new commits. ## Connect with Personal Access Token -NOTE: **Note:** Personal access tokens can only be used to connect GitHub.com +NOTE: **Note:** +Personal access tokens can only be used to connect GitHub.com repositories to GitLab. If you are not using the [GitHub integration](../../integration/github.md), you can still perform a one-off authorization with GitHub to grant GitLab access your repositories: -1. Open https://github.com/settings/tokens/new to create a **Personal Access +1. Open <https://github.com/settings/tokens/new> to create a **Personal Access Token**. This token with be used to access your repository and push commit statuses to GitHub. - + The `repo` and `admin:repo_hook` should be enable to allow GitLab access to your project, update commit statuses, and create a web hook to notify GitLab of new commits. @@ -62,20 +70,23 @@ repositories: 1. In GitHub, add a `.gitlab-ci.yml` to [configure GitLab CI/CD](../quick_start/README.md). -GitLab will import the project, enable [Pull Mirroring](../../workflow/repository_mirroring.md#pulling-from-a-remote-repository-starter), enable -[GitHub project integration](https://docs.gitlab.com/ee/user/project/integrations/github.html), and create a web hook -on GitHub to notify GitLab of new commits. +GitLab will: + +1. Import the project. +1. Enable [Pull Mirroring](../../workflow/repository_mirroring.md#pulling-from-a-remote-repository-starter). +1. Enable [GitHub project integration](https://docs.gitlab.com/ee/user/project/integrations/github.html). +1. Create a web hook on GitHub to notify GitLab of new commits. ## Connect manually If the [GitHub integration](../../integration/github.md) is not enabled, or is enabled for a different GitHub instance, you GitLab CI/CD can be manually enabled for -your repository. +your repository: -1. In GitHub open https://github.com/settings/tokens/new create a **Personal +1. In GitHub open <https://github.com/settings/tokens/new> create a **Personal Access Token.** GitLab will use this token to access your repository and push commit statuses. - + Enter a **Token description** and update the scope to allow: `repo` so that GitLab can access your project and update commit statuses @@ -109,3 +120,15 @@ your repository. ![Create web hook](img/github_push_webhook.png) 1. In GitHub add a `.gitlab-ci.yml` to configure GitLab CI/CD. + +<!-- ## Troubleshooting + +Include any troubleshooting steps that you can foresee. If you know beforehand what issues +one might have when setting this up, or when something is changed, or on upgrading, it's +important to describe those, too. Think of things that may go wrong and include them here. +This is important to minimize requests for support, and to avoid doc comments with +questions that you know someone might ask. + +Each scenario can be a third-level heading, e.g. `### Getting error message X`. +If you have none to add when creating a doc, leave this section in place +but commented out to help encourage others to add to it in the future. --> diff --git a/doc/ci/ci_cd_for_external_repos/index.md b/doc/ci/ci_cd_for_external_repos/index.md index 450ffe512dc..5de88412121 100644 --- a/doc/ci/ci_cd_for_external_repos/index.md +++ b/doc/ci/ci_cd_for_external_repos/index.md @@ -1,3 +1,7 @@ +--- +type: index, howto +--- + # GitLab CI/CD for external repositories **[PREMIUM]** NOTE: **Note:** @@ -6,22 +10,26 @@ GitLab.com users until September 22nd, 2019. >[Introduced][ee-4642] in [GitLab Premium][eep] 10.6. -GitLab CI/CD can be used with GitHub or any other Git server. +GitLab CI/CD can be used with: + +- [GitHub](github_integration.md). +- [Bitbucket Cloud](bitbucket_integration.md). +- Any other Git server. + Instead of moving your entire project to GitLab, you can connect your external repository to get the benefits of GitLab CI/CD. -- [GitHub](github_integration.md) -- [Bitbucket Cloud](bitbucket_integration.md) - Connecting an external repository will set up [repository mirroring][mirroring] and create a lightweight project where issues, merge requests, wiki, and snippets disabled. These features [can be re-enabled later][settings]. -1. From your GitLab dashboard click **New project** -1. Switch to the **CI/CD for external repo** tab -1. Choose **GitHub** or **Repo by URL** -1. The next steps are similar to the [import flow](../../user/project/import/index.md) +To connect to an external repository: + +1. From your GitLab dashboard, click **New project**. +1. Switch to the **CI/CD for external repo** tab. +1. Choose **GitHub** or **Repo by URL**. +1. The next steps are similar to the [import flow](../../user/project/import/index.md). ![CI/CD for external repository project creation](img/ci_cd_for_external_repo.png) diff --git a/doc/ci/docker/README.md b/doc/ci/docker/README.md index 446f5b54f0c..f76471b50f2 100644 --- a/doc/ci/docker/README.md +++ b/doc/ci/docker/README.md @@ -1,9 +1,13 @@ --- comments: false +type: index --- # Docker integration +GitLab CI/CD can be combined with [Docker](https://www.docker.com) to enable +integration between the two. + The following documentation is available for using GitLab CI/CD with Docker: - [Using Docker images](using_docker_images.md). diff --git a/doc/ci/docker/using_docker_build.md b/doc/ci/docker/using_docker_build.md index 6920ce17b46..d8068bbb7f0 100644 --- a/doc/ci/docker/using_docker_build.md +++ b/doc/ci/docker/using_docker_build.md @@ -1,3 +1,7 @@ +--- +type: concepts, howto +--- + # Building Docker images with GitLab CI/CD GitLab CI/CD allows you to use Docker Engine to build and test docker-based projects. @@ -5,10 +9,10 @@ GitLab CI/CD allows you to use Docker Engine to build and test docker-based proj One of the new trends in Continuous Integration/Deployment is to: -1. Create an application image -1. Run tests against the created image -1. Push image to a remote registry -1. Deploy to a server from the pushed image +1. Create an application image. +1. Run tests against the created image. +1. Push image to a remote registry. +1. Deploy to a server from the pushed image. It's also useful when your application already has the `Dockerfile` that can be used to create and test an image: @@ -113,7 +117,7 @@ In order to do that, follow the steps: The above command will create a `config.toml` entry similar to this: - ``` + ```toml [[runners]] url = "https://gitlab.com/" token = TOKEN @@ -227,7 +231,7 @@ In order to do that, follow the steps: The above command will create a `config.toml` entry similar to this: - ``` + ```toml [[runners]] url = "https://gitlab.com/" token = REGISTRATION_TOKEN @@ -270,9 +274,9 @@ aware of the following implications: create containers with specific names, they may conflict with each other. - Sharing files and directories from the source repo into containers may not work as expected since volume mounting is done in the context of the host - machine, not the build container, e.g.: + machine, not the build container. For example: - ``` + ```sh docker run --rm -t -i -v $(pwd)/src:/home/app/src test-image:latest run_app_tests ``` @@ -337,7 +341,7 @@ NOTE: **Note:** The shared Runners on GitLab.com use the `overlay2` driver by default. By default, when using `docker:dind`, Docker uses the `vfs` storage driver which -copies the filesystem on every run. This is a very disk-intensive operation +copies the filesystem on every run. This is a disk-intensive operation which can be avoided if a different driver is used, for example `overlay2`. ### Requirements @@ -345,13 +349,13 @@ which can be avoided if a different driver is used, for example `overlay2`. 1. Make sure a recent kernel is used, preferably `>= 4.2`. 1. Check whether the `overlay` module is loaded: - ``` + ```sh sudo lsmod | grep overlay ``` If you see no result, then it isn't loaded. To load it use: - ``` + ```sh sudo modprobe overlay ``` @@ -359,7 +363,7 @@ which can be avoided if a different driver is used, for example `overlay2`. On Ubuntu systems, this is done by editing `/etc/modules`. Just add the following line into it: - ``` + ```text overlay ``` @@ -367,7 +371,7 @@ which can be avoided if a different driver is used, for example `overlay2`. You can enable the driver for each project individually by editing the project's `.gitlab-ci.yml`: -``` +```yaml variables: DOCKER_DRIVER: overlay2 ``` @@ -571,3 +575,15 @@ deploy: [docker-cap]: https://docs.docker.com/engine/reference/run/#runtime-privilege-and-linux-capabilities [2fa]: ../../user/profile/account/two_factor_authentication.md [pat]: ../../user/profile/personal_access_tokens.md + +<!-- ## Troubleshooting + +Include any troubleshooting steps that you can foresee. If you know beforehand what issues +one might have when setting this up, or when something is changed, or on upgrading, it's +important to describe those, too. Think of things that may go wrong and include them here. +This is important to minimize requests for support, and to avoid doc comments with +questions that you know someone might ask. + +Each scenario can be a third-level heading, e.g. `### Getting error message X`. +If you have none to add when creating a doc, leave this section in place +but commented out to help encourage others to add to it in the future. --> diff --git a/doc/ci/docker/using_docker_images.md b/doc/ci/docker/using_docker_images.md index 13c26bc5f47..d09efce7cc1 100644 --- a/doc/ci/docker/using_docker_images.md +++ b/doc/ci/docker/using_docker_images.md @@ -1,3 +1,7 @@ +--- +type: concepts, howto +--- + # Using Docker images GitLab CI in conjunction with [GitLab Runner](../runners/README.md) can use @@ -45,10 +49,10 @@ The `image` keyword is the name of the Docker image the Docker executor will run to perform the CI tasks. By default, the executor will only pull images from [Docker Hub][hub], -but this can be configured in the `gitlab-runner/config.toml` by setting +however this can be configured in the `gitlab-runner/config.toml` by setting the [Docker pull policy][] to allow using local images. -For more information about images and Docker Hub please read +For more information about images and Docker Hub, please read the [Docker Fundamentals][] documentation. ## What is a service @@ -95,8 +99,8 @@ required for the CI/CD job to proceed and is accessed by network. To make sure this works, the Runner: -1. checks which ports are exposed from the container by default -1. starts a special container that waits for these ports to be accessible +1. Checks which ports are exposed from the container by default. +1. Starts a special container that waits for these ports to be accessible. When the second stage of the check fails, either because there is no opened port in the service, or the service was not started properly before the timeout and the port is not @@ -106,7 +110,7 @@ In most cases it will affect the job, but there may be situations when the job will still succeed even if that warning was printed. For example: - The service was started a little after the warning was raised, and the job is - not using the linked service from the very beginning. In that case, when the + not using the linked service from the beginning. In that case, when the job needed to access the service, it may have been already there waiting for connections. - The service container is not providing any networking service, but it's doing @@ -143,9 +147,9 @@ job: If you need to have `php`, `node` and `go` available for your script, you should either: -- choose an existing Docker image that contains all required tools, or -- create your own Docker image, which will have all the required tools included - and use that in your job +- Choose an existing Docker image that contains all required tools. +- Create your own Docker image, which will have all the required tools included + and use that in your job. ### Accessing the services @@ -167,18 +171,18 @@ access to it from your build container under two hostnames to choose from: - `tutum-wordpress` - `tutum__wordpress` ->**Note:** +NOTE: **Note:** Hostnames with underscores are not RFC valid and may cause problems in 3rd party applications. The default aliases for the service's hostname are created from its image name following these rules: -- Everything after the colon (`:`) is stripped +- Everything after the colon (`:`) is stripped. - Slash (`/`) is replaced with double underscores (`__`) and the primary alias - is created + is created. - Slash (`/`) is replaced with a single dash (`-`) and the secondary alias is - created (requires GitLab Runner v1.1.0 or higher) + created (requires GitLab Runner v1.1.0 or higher). To override the default behavior, you can [specify a service alias](#available-settings-for-services). @@ -333,7 +337,7 @@ services: ``` The Runner will still start two containers using the `mysql:latest` image, -but now each of them will also be accessible with the alias configured +however now each of them will also be accessible with the alias configured in `.gitlab-ci.yml` file. ### Setting a command for the service @@ -408,8 +412,6 @@ you should check which one your Runner is using. Specifically: The syntax of `image:entrypoint` is similar to [Dockerfile's `ENTRYPOINT`][entrypoint]. ----- - Let's assume you have a `super/sql:experimental` image with some SQL database inside it and you would like to use it as a base image for your job because you want to execute some tests with this database binary. Let's also assume that @@ -443,7 +445,7 @@ image: Look for the `[runners.docker]` section: -``` +```toml [runners.docker] image = "ruby:2.1" services = ["mysql:latest", "postgres:latest"] @@ -469,11 +471,11 @@ image which is private and requires you to login into a private container regist Let's also assume that these are the login credentials: -| Key | Value | -|----------|---------------------------| -| registry | registry.example.com:5000 | -| username | my_username | -| password | my_password | +| Key | Value | +|:---------|:----------------------------| +| registry | `registry.example.com:5000` | +| username | `my_username` | +| password | `my_password` | To configure access for `registry.example.com:5000`, follow these steps: @@ -534,7 +536,8 @@ To configure access for `registry.example.com:5000`, follow these steps: You can add configuration for as many registries as you want, adding more registries to the `"auths"` hash as described above. -NOTE: **Note:** The full `hostname:port` combination is required everywhere +NOTE: **Note:** +The full `hostname:port` combination is required everywhere for the Runner to match the `DOCKER_AUTH_CONFIG`. For example, if `registry.example.com:5000/namespace/image:tag` is specified in `.gitlab-ci.yml`, then the `DOCKER_AUTH_CONFIG` must also specify `registry.example.com:5000`. @@ -551,8 +554,9 @@ service containers. For all possible configuration variables check the documentation of each image provided in their corresponding Docker hub page. -*Note: All variables will be passed to all services containers. It's not -designed to distinguish which variable should go where.* +NOTE: **Note:** +All variables will be passed to all services containers. It's not +designed to distinguish which variable should go where. ### PostgreSQL service example @@ -582,8 +586,9 @@ time. ## How to debug a job locally -*Note: The following commands are run without root privileges. You should be -able to run Docker with your regular user account.* +NOTE: **Note:** +The following commands are run without root privileges. You should be +able to run Docker with your regular user account. First start with creating a file named `build_script`: @@ -602,7 +607,7 @@ is specific to your project. Then create some service containers: -``` +```sh docker run -d --name service-mysql mysql:latest docker run -d --name service-postgres postgres:latest ``` @@ -614,7 +619,7 @@ respectively. They will both run in the background (`-d`). Finally, create a build container by executing the `build_script` file we created earlier: -``` +```sh docker run --name build -i --link=service-mysql:mysql --link=service-postgres:postgres ruby:2.1 /bin/bash < build_script ``` @@ -626,7 +631,7 @@ piped using STDIN to the bash interpreter which in turn executes the When you finish testing and no longer need the containers, you can remove them with: -``` +```sh docker rm -f -v build service-mysql service-postgres ``` diff --git a/doc/ci/docker/using_kaniko.md b/doc/ci/docker/using_kaniko.md index f354cdb398e..50f1ac3d54a 100644 --- a/doc/ci/docker/using_kaniko.md +++ b/doc/ci/docker/using_kaniko.md @@ -1,3 +1,7 @@ +--- +type: howto +--- + # Building images with kaniko and GitLab CI/CD > [Introduced](https://gitlab.com/gitlab-org/gitlab-ce/issues/45512) in GitLab 11.2. @@ -9,17 +13,18 @@ container images from a Dockerfile, inside a container or Kubernetes cluster. kaniko solves two problems with using the [docker-in-docker build](using_docker_build.md#use-docker-in-docker-executor) method: -1. Docker-in-docker requires [privileged mode](https://docs.docker.com/engine/reference/run/#runtime-privilege-and-linux-capabilities) - in order to function, which is a significant security concern. -1. Docker-in-docker generally incurs a performance penalty and can be quite slow. +- Docker-in-docker requires [privileged mode](https://docs.docker.com/engine/reference/run/#runtime-privilege-and-linux-capabilities) + in order to function, which is a significant security concern. +- Docker-in-docker generally incurs a performance penalty and can be quite slow. ## Requirements In order to utilize kaniko with GitLab, a [GitLab Runner](https://docs.gitlab.com/runner/) -using either the [Kubernetes](https://docs.gitlab.com/runner/executors/kubernetes.html), -[Docker](https://docs.gitlab.com/runner/executors/docker.html), or -[Docker Machine](https://docs.gitlab.com/runner/executors/docker_machine.html) -executors is required. +using one of the following executors is required: + +- [Kubernetes](https://docs.gitlab.com/runner/executors/kubernetes.html). +- [Docker](https://docs.gitlab.com/runner/executors/docker.html). +- [Docker Machine](https://docs.gitlab.com/runner/executors/docker_machine.html). ## Building a Docker image with kaniko @@ -34,14 +39,17 @@ few important details: - A Docker `config.json` file needs to be created with the authentication information for the desired container registry. ---- +In the following example, kaniko is used to: + +1. Build a Docker image. +1. Then push it to [GitLab Container Registry](../../user/project/container_registry.md). -In the following example, kaniko is used to build a Docker image and then push -it to [GitLab Container Registry](../../user/project/container_registry.md). The job will run only when a tag is pushed. A `config.json` file is created under `/kaniko/.docker` with the needed GitLab Container Registry credentials taken from the [environment variables](../variables/README.md#predefined-environment-variables) -GitLab CI/CD provides. In the last step, kaniko uses the `Dockerfile` under the +GitLab CI/CD provides. + +In the last step, kaniko uses the `Dockerfile` under the root directory of the project, builds the Docker image and pushes it to the project's Container Registry while tagging it with the Git tag: @@ -80,3 +88,15 @@ store: ... -----END CERTIFICATE-----" >> /kaniko/ssl/certs/ca-certificates.crt ``` + +<!-- ## Troubleshooting + +Include any troubleshooting steps that you can foresee. If you know beforehand what issues +one might have when setting this up, or when something is changed, or on upgrading, it's +important to describe those, too. Think of things that may go wrong and include them here. +This is important to minimize requests for support, and to avoid doc comments with +questions that you know someone might ask. + +Each scenario can be a third-level heading, e.g. `### Getting error message X`. +If you have none to add when creating a doc, leave this section in place +but commented out to help encourage others to add to it in the future. --> diff --git a/doc/ci/enable_or_disable_ci.md b/doc/ci/enable_or_disable_ci.md index 9934d543991..56200142055 100644 --- a/doc/ci/enable_or_disable_ci.md +++ b/doc/ci/enable_or_disable_ci.md @@ -1,29 +1,32 @@ --- -type: reference +type: howto --- # How to enable or disable GitLab CI/CD -To effectively use GitLab CI/CD, you need a valid [`.gitlab-ci.yml`](yaml/README.md) -file present at the root directory of your project and a -[runner](runners/README.md) properly set up. You can read our -[quick start guide](quick_start/README.md) to get you started. +To effectively use GitLab CI/CD, you need: + +- A valid [`.gitlab-ci.yml`](yaml/README.md) file present at the root directory + of your project. +- A [runner](runners/README.md) properly set up. + +You can read our [quick start guide](quick_start/README.md) to get you started. If you are using an external CI/CD server like Jenkins or Drone CI, it is advised to disable GitLab CI/CD in order to not have any conflicts with the commits status API. ---- - GitLab CI/CD is exposed via the `/pipelines` and `/jobs` pages of a project. Disabling GitLab CI/CD in a project does not delete any previous jobs. In fact, the `/pipelines` and `/jobs` pages can still be accessed, although it's hidden from the left sidebar menu. -GitLab CI/CD is enabled by default on new installations and can be disabled either -individually under each project's settings, or site-wide by modifying the -settings in `gitlab.yml` and `gitlab.rb` for source and Omnibus installations -respectively. +GitLab CI/CD is enabled by default on new installations and can be disabled +either: + +- Individually under each project's settings. +- Site-wide by modifying the settings in `gitlab.yml` and `gitlab.rb` for source + and Omnibus installations respectively. ## Per-project user setting @@ -40,10 +43,10 @@ and `gitlab.rb` for source and Omnibus installations respectively. Two things to note: -1. Disabling GitLab CI/CD, will affect only newly-created projects. Projects that - had it enabled prior to this modification, will work as before. -1. Even if you disable GitLab CI/CD, users will still be able to enable it in the - project's settings. +- Disabling GitLab CI/CD, will affect only newly-created projects. Projects that + had it enabled prior to this modification, will work as before. +- Even if you disable GitLab CI/CD, users will still be able to enable it in the + project's settings. For installations from source, open `gitlab.yml` with your editor and set `builds` to `false`: @@ -58,12 +61,32 @@ default_projects_features: builds: false ``` -Save the file and restart GitLab: `sudo service gitlab restart`. +Save the file and restart GitLab: + +```sh +sudo service gitlab restart +``` For Omnibus installations, edit `/etc/gitlab/gitlab.rb` and add the line: -``` +```ruby gitlab_rails['gitlab_default_projects_features_builds'] = false ``` -Save the file and reconfigure GitLab: `sudo gitlab-ctl reconfigure`. +Save the file and reconfigure GitLab: + +```sh +sudo gitlab-ctl reconfigure +``` + +<!-- ## Troubleshooting + +Include any troubleshooting steps that you can foresee. If you know beforehand what issues +one might have when setting this up, or when something is changed, or on upgrading, it's +important to describe those, too. Think of things that may go wrong and include them here. +This is important to minimize requests for support, and to avoid doc comments with +questions that you know someone might ask. + +Each scenario can be a third-level heading, e.g. `### Getting error message X`. +If you have none to add when creating a doc, leave this section in place +but commented out to help encourage others to add to it in the future. --> diff --git a/doc/ci/environments.md b/doc/ci/environments.md index 9f930833093..f2661c4bafd 100644 --- a/doc/ci/environments.md +++ b/doc/ci/environments.md @@ -92,7 +92,7 @@ deploy_staging: - master ``` -We have defined 3 [stages](yaml/README.md#stages): +We have defined three [stages](yaml/README.md#stages): - `test` - `build` @@ -135,7 +135,7 @@ In summary, with the above `.gitlab-ci.yml` we have achieved the following: > the name given in `.gitlab-ci.yml` (with any variables expanded), while the > second is a "cleaned-up" version of the name, suitable for use in URLs, DNS, > etc. - +> > Starting with GitLab 9.3, the environment URL is exposed to the Runner via > `$CI_ENVIRONMENT_URL`. The URL is expanded from `.gitlab-ci.yml`, or if > the URL was not defined there, the external URL from the environment is used. diff --git a/doc/ci/environments/protected_environments.md b/doc/ci/environments/protected_environments.md index ab5c0e2dbad..b72ebe838b8 100644 --- a/doc/ci/environments/protected_environments.md +++ b/doc/ci/environments/protected_environments.md @@ -1,3 +1,7 @@ +--- +type: concepts, howto +--- + # Protected Environments **[PREMIUM]** > [Introduced](https://gitlab.com/gitlab-org/gitlab-ee/merge_requests/6303) in [GitLab Premium](https://about.gitlab.com/pricing/) 11.3. @@ -47,3 +51,15 @@ Maintainers can: - Update existing protected environments at any time by changing the access in the **Allowed to Deploy** dropdown menu. - Unprotect a protected environment by clicking the **Unprotect** button for that environment. + +<!-- ## Troubleshooting + +Include any troubleshooting steps that you can foresee. If you know beforehand what issues +one might have when setting this up, or when something is changed, or on upgrading, it's +important to describe those, too. Think of things that may go wrong and include them here. +This is important to minimize requests for support, and to avoid doc comments with +questions that you know someone might ask. + +Each scenario can be a third-level heading, e.g. `### Getting error message X`. +If you have none to add when creating a doc, leave this section in place +but commented out to help encourage others to add to it in the future. --> diff --git a/doc/ci/multi_project_pipelines.md b/doc/ci/multi_project_pipelines.md index 1091129a9dc..50c8d82602b 100644 --- a/doc/ci/multi_project_pipelines.md +++ b/doc/ci/multi_project_pipelines.md @@ -26,6 +26,10 @@ and when hovering or tapping (on touchscreen devices) they will expand and be sh Multi-project pipelines are useful for larger products that require cross-project inter-dependencies, such as those adopting a [microservices architecture](https://about.gitlab.com/2016/08/16/trends-in-version-control-land-microservices/). +For a demonstration of how cross-functional development teams can use cross-pipeline +triggering to trigger multiple pipelines for different microservices projects, see +[Cross-project Pipeline Triggering and Visualization](https://about.gitlab.com/handbook/marketing/product-marketing/demo/#cross-project-pipeline-triggering-and-visualization-may-2019---1110). + ## Use cases Let's assume you deploy your web app from different projects in GitLab: @@ -167,10 +171,6 @@ In this scenario, the `UPSTREAM_BRANCH` variable with a value related to the upstream pipeline will be passed to the `downstream-job` job, and will be available within the context of all downstream builds. -### Demos - -[A click-through demo of cross-project pipeline is available](https://about.gitlab.com/handbook/marketing/product-marketing/demo/#cross-project-pipeline-triggering-and-visualization-may-2019---1110), demonstrates how cross-functional dev teams use cross-pipeline triggering to trigger multiple pipelines for different microservices projects. - ### Limitations Because bridge jobs are a little different to regular jobs, it is not |