From b6ede127294fd0df63facb9a65791c018067db9f Mon Sep 17 00:00:00 2001 From: Jason Roehm Date: Tue, 15 Mar 2016 09:46:09 -0400 Subject: add some documentation for `triggers` keyword [ci skip] --- doc/ci/yaml/README.md | 13 ++++++++++++- 1 file changed, 12 insertions(+), 1 deletion(-) (limited to 'doc/ci') diff --git a/doc/ci/yaml/README.md b/doc/ci/yaml/README.md index 5158e3c387c..510fab7340a 100644 --- a/doc/ci/yaml/README.md +++ b/doc/ci/yaml/README.md @@ -286,7 +286,7 @@ There are a few rules that apply to the usage of refs policy: * `only` and `except` are inclusive. If both `only` and `except` are defined in a job specification, the ref is filtered by `only` and `except`. * `only` and `except` allow the use of regular expressions. -* `only` and `except` allow the use of special keywords: `branches` and `tags`. +* `only` and `except` allow the use of special keywords: `branches`, `tags`, and `triggers`. * `only` and `except` allow to specify a repository path to filter jobs for forks. @@ -303,6 +303,17 @@ job: - branches ``` +In this example, `job` will run only for refs that are tagged, or if a build is explicitly requested +via an API trigger. + +```yaml +job: + # use special keywords + only: + - tags + - triggers +``` + The repository path can be used to have jobs executed only for the parent repository and not forks: -- cgit v1.2.3 From 0ce1f9387a0bb92b6e6656fa90e7d456f82aded1 Mon Sep 17 00:00:00 2001 From: Pat Turner Date: Thu, 17 Mar 2016 15:28:55 +0000 Subject: Fix for https://gitlab.com/gitlab-org/gitlab-ce/issues/14366 --- doc/ci/yaml/README.md | 2 ++ 1 file changed, 2 insertions(+) (limited to 'doc/ci') diff --git a/doc/ci/yaml/README.md b/doc/ci/yaml/README.md index a9b79bbdb1b..4a932dfeaff 100644 --- a/doc/ci/yaml/README.md +++ b/doc/ci/yaml/README.md @@ -279,6 +279,8 @@ job_name: | Keyword | Required | Description | |---------------|----------|-------------| | script | yes | Defines a shell script which is executed by runner | +| image | no | Use docker image, covered in [Using Docker Images](../docker/using_docker_images.md#define-image-and-services-from-.gitlab-ci.yml) | +| services | no | Use docker services, covered in [Use Docker - Using Docker Images](../docker/using_docker_images.md#define-image-and-services-from-.gitlab-ci.yml) | | stage | no | Defines a build stage (default: `test`) | | type | no | Alias for `stage` | | only | no | Defines a list of git refs for which build is created | -- cgit v1.2.3 From 08ed12d91e589a1e55c6662ce06fe1478e8f9fd5 Mon Sep 17 00:00:00 2001 From: Pat Turner Date: Thu, 17 Mar 2016 17:27:13 +0000 Subject: removed anchor as unsupported in markdown --- doc/ci/yaml/README.md | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) (limited to 'doc/ci') diff --git a/doc/ci/yaml/README.md b/doc/ci/yaml/README.md index 4a932dfeaff..3572b496e1a 100644 --- a/doc/ci/yaml/README.md +++ b/doc/ci/yaml/README.md @@ -279,8 +279,8 @@ job_name: | Keyword | Required | Description | |---------------|----------|-------------| | script | yes | Defines a shell script which is executed by runner | -| image | no | Use docker image, covered in [Using Docker Images](../docker/using_docker_images.md#define-image-and-services-from-.gitlab-ci.yml) | -| services | no | Use docker services, covered in [Use Docker - Using Docker Images](../docker/using_docker_images.md#define-image-and-services-from-.gitlab-ci.yml) | +| image | no | Use docker image, covered in [Using Docker Images](../docker/using_docker_images.md) | +| services | no | Use docker services, covered in [Use Docker - Using Docker Images](../docker/using_docker_images.md) | | stage | no | Defines a build stage (default: `test`) | | type | no | Alias for `stage` | | only | no | Defines a list of git refs for which build is created | -- cgit v1.2.3 From 44ca0305cde4329405fca08b75028a09adf0735a Mon Sep 17 00:00:00 2001 From: Pat Turner Date: Fri, 18 Mar 2016 08:32:56 +0000 Subject: Fix for https://gitlab.com/gitlab-org/gitlab-ce/issues/14366 --- doc/ci/yaml/README.md | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) (limited to 'doc/ci') diff --git a/doc/ci/yaml/README.md b/doc/ci/yaml/README.md index 3572b496e1a..4a932dfeaff 100644 --- a/doc/ci/yaml/README.md +++ b/doc/ci/yaml/README.md @@ -279,8 +279,8 @@ job_name: | Keyword | Required | Description | |---------------|----------|-------------| | script | yes | Defines a shell script which is executed by runner | -| image | no | Use docker image, covered in [Using Docker Images](../docker/using_docker_images.md) | -| services | no | Use docker services, covered in [Use Docker - Using Docker Images](../docker/using_docker_images.md) | +| image | no | Use docker image, covered in [Using Docker Images](../docker/using_docker_images.md#define-image-and-services-from-.gitlab-ci.yml) | +| services | no | Use docker services, covered in [Use Docker - Using Docker Images](../docker/using_docker_images.md#define-image-and-services-from-.gitlab-ci.yml) | | stage | no | Defines a build stage (default: `test`) | | type | no | Alias for `stage` | | only | no | Defines a list of git refs for which build is created | -- cgit v1.2.3 From 280d9dc23fb30bfc31f11c75c9c22251dde4b995 Mon Sep 17 00:00:00 2001 From: Pat Turner Date: Fri, 18 Mar 2016 11:46:19 +0000 Subject: fixed md anchor link --- doc/ci/yaml/README.md | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) (limited to 'doc/ci') diff --git a/doc/ci/yaml/README.md b/doc/ci/yaml/README.md index 4a932dfeaff..762b35859b9 100644 --- a/doc/ci/yaml/README.md +++ b/doc/ci/yaml/README.md @@ -279,8 +279,8 @@ job_name: | Keyword | Required | Description | |---------------|----------|-------------| | script | yes | Defines a shell script which is executed by runner | -| image | no | Use docker image, covered in [Using Docker Images](../docker/using_docker_images.md#define-image-and-services-from-.gitlab-ci.yml) | -| services | no | Use docker services, covered in [Use Docker - Using Docker Images](../docker/using_docker_images.md#define-image-and-services-from-.gitlab-ci.yml) | +| image | no | Use docker image, covered in [Using Docker Images](../docker/using_docker_images.md#define-image-and-services-from-gitlab-ciyml) | +| services | no | Use docker services, covered in [Using Docker Images](../docker/using_docker_images.md#define-image-and-services-from-gitlab-ciyml) | | stage | no | Defines a build stage (default: `test`) | | type | no | Alias for `stage` | | only | no | Defines a list of git refs for which build is created | -- cgit v1.2.3 From 1824fb0603c798ec467ea3529570031e7dbb2986 Mon Sep 17 00:00:00 2001 From: Florian Date: Tue, 5 Apr 2016 15:20:58 +0000 Subject: Fix broken link in CI quickstart docs The space between the [label] and the (link) caused it to be interpreted literally. --- doc/ci/quick_start/README.md | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) (limited to 'doc/ci') diff --git a/doc/ci/quick_start/README.md b/doc/ci/quick_start/README.md index 9aba4326e11..aae9ccae1d3 100644 --- a/doc/ci/quick_start/README.md +++ b/doc/ci/quick_start/README.md @@ -212,8 +212,8 @@ If you want to receive e-mail notifications about the result status of the builds, you should explicitly enable the **Builds Emails** service under your project's settings. -For more information read the [Builds emails service documentation] -(../../project_services/builds_emails.md). +For more information read the +[Builds emails service documentation](../../project_services/builds_emails.md). ## Builds badge -- cgit v1.2.3 From 71b5010a940c4b0364126ba7e2dfc0ce07b69f25 Mon Sep 17 00:00:00 2001 From: frodsan Date: Wed, 6 Apr 2016 22:45:56 +0000 Subject: [ci skip] Fix typo. --- doc/ci/yaml/README.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) (limited to 'doc/ci') diff --git a/doc/ci/yaml/README.md b/doc/ci/yaml/README.md index 4316f3c1f64..7da9b31e30d 100644 --- a/doc/ci/yaml/README.md +++ b/doc/ci/yaml/README.md @@ -38,7 +38,7 @@ services: - postgres before_script: - - bundle_install + - bundle install stages: - build -- cgit v1.2.3 From 875646a85e6fb7ab783d99c9c9eac406983e502c Mon Sep 17 00:00:00 2001 From: Alex Mayer Date: Thu, 7 Apr 2016 20:10:29 +0000 Subject: Fix Incorrect Quote In Docker Executor Example --- doc/ci/ssh_keys/README.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) (limited to 'doc/ci') diff --git a/doc/ci/ssh_keys/README.md b/doc/ci/ssh_keys/README.md index 210f9c3e849..d790015aca1 100644 --- a/doc/ci/ssh_keys/README.md +++ b/doc/ci/ssh_keys/README.md @@ -57,7 +57,7 @@ before_script: # WARNING: Use this only with the Docker executor, if you use it with shell # you will overwrite your user's SSH config. - mkdir -p ~/.ssh - - '[[ -f /.dockerinit ]] && echo -e "Host *\n\tStrictHostKeyChecking no\n\n" > ~/.ssh/config` + - '[[ -f /.dockerinit ]] && echo -e "Host *\n\tStrictHostKeyChecking no\n\n" > ~/.ssh/config' ``` As a final step, add the _public_ key from the one you created earlier to the -- cgit v1.2.3 From ca823065a905f8c19b2003a67e79831ba1ceff2e Mon Sep 17 00:00:00 2001 From: Benjamin Montgomery Date: Sat, 9 Apr 2016 20:51:15 +0000 Subject: actually enable artifacts --- doc/ci/build_artifacts/README.md | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) (limited to 'doc/ci') diff --git a/doc/ci/build_artifacts/README.md b/doc/ci/build_artifacts/README.md index 71db5aa5dc8..bd26da2af6d 100644 --- a/doc/ci/build_artifacts/README.md +++ b/doc/ci/build_artifacts/README.md @@ -31,7 +31,7 @@ To disable it site-wide, follow the steps below. 1. Edit `/etc/gitlab/gitlab.rb` and add the following line: ```ruby - gitlab_rails['artifacts_enabled'] = false + gitlab_rails['artifacts_enabled'] = true ``` 1. Save the file and [reconfigure GitLab][] for the changes to take effect. @@ -44,7 +44,7 @@ To disable it site-wide, follow the steps below. ```yaml artifacts: - enabled: false + enabled: true ``` 1. Save the file and [restart GitLab][] for the changes to take effect. -- cgit v1.2.3 From 1e99e56202c0c96d4a70fba95e04b4f41d24327a Mon Sep 17 00:00:00 2001 From: Benjamin Montgomery Date: Sun, 10 Apr 2016 21:14:40 +0000 Subject: Make the purpose of artifact config options clearer --- doc/ci/build_artifacts/README.md | 15 ++++++--------- 1 file changed, 6 insertions(+), 9 deletions(-) (limited to 'doc/ci') diff --git a/doc/ci/build_artifacts/README.md b/doc/ci/build_artifacts/README.md index bd26da2af6d..a21cd129862 100644 --- a/doc/ci/build_artifacts/README.md +++ b/doc/ci/build_artifacts/README.md @@ -1,7 +1,8 @@ # Introduction to build artifacts Artifacts is a list of files and directories which are attached to a build -after it completes successfully. +after it completes successfully. Artificats is enabled by default. _If you are searching for ways to use artifacts, jump to +[Defining artifacts in `.gitlab-ci.yml`](#defining-artifacts-in-gitlab-ciyml)._ Since GitLab 8.2 and [GitLab Runner] 0.7.0, build artifacts that are created by GitLab Runner are uploaded to GitLab and are downloadable as a single archive @@ -16,13 +17,9 @@ The artifacts browser will be available only for new artifacts that are sent to GitLab using GitLab Runner version 1.0 and up. It will not be possible to browse old artifacts already uploaded to GitLab. -## Enabling build artifacts - -_If you are searching for ways to use artifacts, jump to -[Defining artifacts in `.gitlab-ci.yml`](#defining-artifacts-in-gitlab-ciyml)._ +## Disabling build artifacts -The artifacts feature is enabled by default in all GitLab installations. -To disable it site-wide, follow the steps below. +To disable artifacts site-wide, follow the steps below. --- @@ -31,7 +28,7 @@ To disable it site-wide, follow the steps below. 1. Edit `/etc/gitlab/gitlab.rb` and add the following line: ```ruby - gitlab_rails['artifacts_enabled'] = true + gitlab_rails['artifacts_enabled'] = false ``` 1. Save the file and [reconfigure GitLab][] for the changes to take effect. @@ -44,7 +41,7 @@ To disable it site-wide, follow the steps below. ```yaml artifacts: - enabled: true + enabled: false ``` 1. Save the file and [restart GitLab][] for the changes to take effect. -- cgit v1.2.3 From 84ff30a6b2b522e35d608df311c58c180acb114e Mon Sep 17 00:00:00 2001 From: Benjamin Montgomery Date: Sun, 10 Apr 2016 21:16:06 +0000 Subject: fix typo --- doc/ci/build_artifacts/README.md | 4 +++- 1 file changed, 3 insertions(+), 1 deletion(-) (limited to 'doc/ci') diff --git a/doc/ci/build_artifacts/README.md b/doc/ci/build_artifacts/README.md index a21cd129862..9553bb11e9d 100644 --- a/doc/ci/build_artifacts/README.md +++ b/doc/ci/build_artifacts/README.md @@ -1,7 +1,9 @@ # Introduction to build artifacts Artifacts is a list of files and directories which are attached to a build -after it completes successfully. Artificats is enabled by default. _If you are searching for ways to use artifacts, jump to +after it completes successfully. This feature is enabled by default in all GitLab installations. + +_If you are searching for ways to use artifacts, jump to [Defining artifacts in `.gitlab-ci.yml`](#defining-artifacts-in-gitlab-ciyml)._ Since GitLab 8.2 and [GitLab Runner] 0.7.0, build artifacts that are created by -- cgit v1.2.3 From d86190eead0075d1b0ba7e73ff2efa8a610b5398 Mon Sep 17 00:00:00 2001 From: Achilleas Pipinellis Date: Mon, 11 Apr 2016 12:41:27 +0300 Subject: Add a note about installing Runners in ci/runners/README.md --- doc/ci/runners/README.md | 6 +++++- 1 file changed, 5 insertions(+), 1 deletion(-) (limited to 'doc/ci') diff --git a/doc/ci/runners/README.md b/doc/ci/runners/README.md index 295d953db11..36e90e21419 100644 --- a/doc/ci/runners/README.md +++ b/doc/ci/runners/README.md @@ -7,6 +7,10 @@ through the coordinator API of GitLab CI. A runner can be specific to a certain project or serve any project in GitLab CI. A runner that serves all projects is called a shared runner. +Ideally, GitLab Runner should not be installed on the same machine as GitLab. +Read the [requirements documentation](../../install/requirements.md#gitlab-runner) +for more information. + ## Shared vs. Specific Runners A runner that is specific only runs for the specified project. A shared runner @@ -140,7 +144,7 @@ to it. This means that if you have shared runners setup for a project and someone forks that project, the shared runners will also serve jobs of this project. -# Attack vectors in runners +## Attack vectors in Runners Mentioned briefly earlier, but the following things of runners can be exploited. We're always looking for contributions that can mitigate these [Security Considerations](https://gitlab.com/gitlab-org/gitlab-ci-multi-runner/blob/master/docs/security/index.md). -- cgit v1.2.3 From fd248b0f068ad0239d0d4ddc462f091eecfe981e Mon Sep 17 00:00:00 2001 From: Lee Date: Tue, 12 Apr 2016 16:13:31 -0500 Subject: (doc) fix typo to ssh keys doc url --- doc/ci/ssh_keys/README.md | 8 ++++---- 1 file changed, 4 insertions(+), 4 deletions(-) (limited to 'doc/ci') diff --git a/doc/ci/ssh_keys/README.md b/doc/ci/ssh_keys/README.md index d790015aca1..7f825e6a065 100644 --- a/doc/ci/ssh_keys/README.md +++ b/doc/ci/ssh_keys/README.md @@ -30,7 +30,7 @@ This is the universal solution which works with any type of executor ## SSH keys when using the Docker executor You will first need to create an SSH key pair. For more information, follow the -instructions to [generate an SSH key](../ssh/README.md). +instructions to [generate an SSH key](../../ssh/README.md). Then, create a new **Secret Variable** in your project settings on GitLab following **Settings > Variables**. As **Key** add the name `SSH_PRIVATE_KEY` @@ -63,7 +63,7 @@ before_script: As a final step, add the _public_ key from the one you created earlier to the services that you want to have an access to from within the build environment. If you are accessing a private GitLab repository you need to add it as a -[deploy key](../ssh/README.md#deploy-keys). +[deploy key](../../ssh/README.md#deploy-keys). That's it! You can now have access to private servers or repositories in your build environment. @@ -79,12 +79,12 @@ on, and use that key for all projects that are run on this machine. First, you need to login to the server that runs your builds. Then from the terminal login as the `gitlab-runner` user and generate the SSH -key pair as described in the [SSH keys documentation](../ssh/README.md). +key pair as described in the [SSH keys documentation](../../ssh/README.md). As a final step, add the _public_ key from the one you created earlier to the services that you want to have an access to from within the build environment. If you are accessing a private GitLab repository you need to add it as a -[deploy key](../ssh/README.md#deploy-keys). +[deploy key](../../ssh/README.md#deploy-keys). Once done, try to login to the remote server in order to accept the fingerprint: -- cgit v1.2.3 From f2aacc25fb5ded3c4aa70495d05aaad2cf9b23f8 Mon Sep 17 00:00:00 2001 From: Hyunwoo Jung Date: Wed, 13 Apr 2016 21:19:50 +0000 Subject: Fix typos in CI docs. --- doc/ci/quick_start/README.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) (limited to 'doc/ci') diff --git a/doc/ci/quick_start/README.md b/doc/ci/quick_start/README.md index 9aba4326e11..6a42a935abd 100644 --- a/doc/ci/quick_start/README.md +++ b/doc/ci/quick_start/README.md @@ -13,7 +13,7 @@ GitLab offers a [continuous integration][ci] service. If you and configure your GitLab project to use a [Runner], then each merge request or push triggers a build. -The `.gitlab-ci.yml` file tells the GitLab runner what do to. By default it +The `.gitlab-ci.yml` file tells the GitLab runner what to do. By default it runs three [stages]: `build`, `test`, and `deploy`. If everything runs OK (no non-zero return values), you'll get a nice green -- cgit v1.2.3 From 072b9c2c8fe466d43834c16bfbb565043a033fdf Mon Sep 17 00:00:00 2001 From: Achilleas Pipinellis Date: Thu, 14 Apr 2016 15:40:05 +0300 Subject: Add TOC to yaml README and an intro section [ci skip] --- doc/ci/yaml/README.md | 73 ++++++++++++++++++++++++++++++++++++++++----------- 1 file changed, 57 insertions(+), 16 deletions(-) (limited to 'doc/ci') diff --git a/doc/ci/yaml/README.md b/doc/ci/yaml/README.md index 7da9b31e30d..abb6e97e5e6 100644 --- a/doc/ci/yaml/README.md +++ b/doc/ci/yaml/README.md @@ -1,5 +1,47 @@ # Configuration of your builds with .gitlab-ci.yml +This document describes the usage of `.gitlab-ci.yml`, the file that is used by +GitLab Runner to manage your project's builds. + +If you want a quick introduction to GitLab CI, follow our +[quick start guide](../quick_start/README.md). + +--- + + + +**Table of Contents** *generated with [DocToc](https://github.com/thlorenz/doctoc)* + +- [.gitlab-ci.yml](#gitlab-ci-yml) + - [image and services](#image-and-services) + - [before_script](#before_script) + - [stages](#stages) + - [types](#types) + - [variables](#variables) + - [cache](#cache) + - [cache:key](#cache-key) +- [Jobs](#jobs) + - [script](#script) + - [stage](#stage) + - [only and except](#only-and-except) + - [tags](#tags) + - [when](#when) + - [artifacts](#artifacts) + - [artifacts:name](#artifacts-name) + - [dependencies](#dependencies) +- [Hidden jobs](#hidden-jobs) +- [Special YAML features](#special-yaml-features) + - [Anchors](#anchors) +- [Validate the .gitlab-ci.yml](#validate-the-gitlab-ci-yml) +- [Skipping builds](#skipping-builds) +- [Examples](#examples) + + + +--- + +## .gitlab-ci.yml + From version 7.12, GitLab CI uses a [YAML](https://en.wikipedia.org/wiki/YAML) file (`.gitlab-ci.yml`) for the project configuration. It is placed in the root of your repository and contains definitions of how your project should be built. @@ -23,12 +65,10 @@ Of course a command can execute code directly (`./configure;make;make install`) or run a script (`test.sh`) in the repository. Jobs are used to create builds, which are then picked up by -[runners](../runners/README.md) and executed within the environment of the -runner. What is important, is that each job is run independently from each +[Runners](../runners/README.md) and executed within the environment of the +Runner. What is important, is that each job is run independently from each other. -## .gitlab-ci.yml - The YAML syntax allows for using more complex job specifications than in the above example: @@ -71,7 +111,7 @@ There are a few reserved `keywords` that **cannot** be used as job names: This allows to specify a custom Docker image and a list of services that can be used for time of the build. The configuration of this feature is covered in -separate document: [Use Docker](../docker/README.md). +[a separate document](../docker/README.md). ### before_script @@ -86,7 +126,8 @@ The specification of `stages` allows for having flexible multi stage pipelines. The ordering of elements in `stages` defines the ordering of builds' execution: 1. Builds of the same stage are run in parallel. -1. Builds of next stage are run after success. +1. Builds of the next stage are run after the jobs from the previous stage + complete successfully. Let's consider the following example, which defines 3 stages: @@ -98,9 +139,9 @@ stages: ``` 1. First all jobs of `build` are executed in parallel. -1. If all jobs of `build` succeeds, the `test` jobs are executed in parallel. -1. If all jobs of `test` succeeds, the `deploy` jobs are executed in parallel. -1. If all jobs of `deploy` succeeds, the commit is marked as `success`. +1. If all jobs of `build` succeed, the `test` jobs are executed in parallel. +1. If all jobs of `test` succeed, the `deploy` jobs are executed in parallel. +1. If all jobs of `deploy` succeed, the commit is marked as `success`. 1. If any of the previous jobs fails, the commit is marked as `failed` and no jobs of further stage are executed. @@ -278,14 +319,14 @@ job_name: | Keyword | Required | Description | |---------------|----------|-------------| -| script | yes | Defines a shell script which is executed by runner | +| script | yes | Defines a shell script which is executed by Runner | | image | no | Use docker image, covered in [Using Docker Images](../docker/using_docker_images.md#define-image-and-services-from-gitlab-ciyml) | | services | no | Use docker services, covered in [Using Docker Images](../docker/using_docker_images.md#define-image-and-services-from-gitlab-ciyml) | | stage | no | Defines a build stage (default: `test`) | | type | no | Alias for `stage` | | only | no | Defines a list of git refs for which build is created | | except | no | Defines a list of git refs for which build is not created | -| tags | no | Defines a list of tags which are used to select runner | +| tags | no | Defines a list of tags which are used to select Runner | | allow_failure | no | Allow build to fail. Failed build doesn't contribute to commit status | | when | no | Define when to run build. Can be `on_success`, `on_failure` or `always` | | dependencies | no | Define other builds that a build depends on so that you can pass artifacts between them| @@ -294,7 +335,7 @@ job_name: ### script -`script` is a shell script which is executed by the runner. For example: +`script` is a shell script which is executed by the Runner. For example: ```yaml job: @@ -375,13 +416,13 @@ except master. ### tags -`tags` is used to select specific runners from the list of all runners that are +`tags` is used to select specific Runners from the list of all Runners that are allowed to run this project. -During the registration of a runner, you can specify the runner's tags, for +During the registration of a Runner, you can specify the Runner's tags, for example `ruby`, `postgres`, `development`. -`tags` allow you to run builds with runners that have the specified tags +`tags` allow you to run builds with Runners that have the specified tags assigned to them: ```yaml @@ -391,7 +432,7 @@ job: - postgres ``` -The specification above, will make sure that `job` is built by a runner that +The specification above, will make sure that `job` is built by a Runner that has both `ruby` AND `postgres` tags defined. ### when -- cgit v1.2.3 From b49069b72cbfa73b6f7bb7195659ba2958f02f7e Mon Sep 17 00:00:00 2001 From: Jamie Neubert Pedersen Date: Thu, 14 Apr 2016 16:04:18 +0000 Subject: change: "very demand" changed to "specific demand" This must be an error as the sentence doesn't make sense otherwise --- doc/ci/runners/README.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) (limited to 'doc/ci') diff --git a/doc/ci/runners/README.md b/doc/ci/runners/README.md index 295d953db11..c7df0713a3d 100644 --- a/doc/ci/runners/README.md +++ b/doc/ci/runners/README.md @@ -19,7 +19,7 @@ many projects, you can have a single or a small number of runners that handle multiple projects. This makes it easier to maintain and update runners. **Specific runners** are useful for jobs that have special requirements or for -projects with a very demand. If a job has certain requirements, you can set +projects with a specific demand. If a job has certain requirements, you can set up the specific runner with this in mind, while not having to do this for all runners. For example, if you want to deploy a certain project, you can setup a specific runner to have the right credentials for this. -- cgit v1.2.3 From a0afeefd76ebfacae59343f48e127828b27ba77a Mon Sep 17 00:00:00 2001 From: Kamil Trzcinski Date: Sun, 17 Apr 2016 09:02:42 -0400 Subject: Add CHANGELOG and documentation --- doc/ci/yaml/README.md | 10 ++++++++++ 1 file changed, 10 insertions(+) (limited to 'doc/ci') diff --git a/doc/ci/yaml/README.md b/doc/ci/yaml/README.md index abb6e97e5e6..54b06f10b95 100644 --- a/doc/ci/yaml/README.md +++ b/doc/ci/yaml/README.md @@ -15,6 +15,7 @@ If you want a quick introduction to GitLab CI, follow our - [.gitlab-ci.yml](#gitlab-ci-yml) - [image and services](#image-and-services) - [before_script](#before_script) + - [after_script](#after_script) - [stages](#stages) - [types](#types) - [variables](#variables) @@ -80,6 +81,9 @@ services: before_script: - bundle install +after_script: + - rm secrets + stages: - build - test @@ -104,6 +108,7 @@ There are a few reserved `keywords` that **cannot** be used as job names: | stages | no | Define build stages | | types | no | Alias for `stages` | | before_script | no | Define commands that run before each job's script | +| after_script | no | Define commands that run after each job's script | | variables | no | Define build variables | | cache | no | Define list of files that should be cached between subsequent runs | @@ -118,6 +123,11 @@ used for time of the build. The configuration of this feature is covered in `before_script` is used to define the command that should be run before all builds, including deploy builds. This can be an array or a multi-line string. +### after_script + +`after_script` is used to define the command that will be run after for all +builds. This has to be an array or a multi-line string. + ### stages `stages` is used to define build stages that can be used by jobs. -- cgit v1.2.3 From 38b15e35d48550a5621b8fc292cabc5670897a44 Mon Sep 17 00:00:00 2001 From: Kamil Trzcinski Date: Sun, 17 Apr 2016 09:10:47 -0400 Subject: Update CHANGELOG and add documentation --- doc/ci/yaml/README.md | 20 ++++++++++++++++++++ 1 file changed, 20 insertions(+) (limited to 'doc/ci') diff --git a/doc/ci/yaml/README.md b/doc/ci/yaml/README.md index 54b06f10b95..c626bee5703 100644 --- a/doc/ci/yaml/README.md +++ b/doc/ci/yaml/README.md @@ -30,6 +30,7 @@ If you want a quick introduction to GitLab CI, follow our - [artifacts](#artifacts) - [artifacts:name](#artifacts-name) - [dependencies](#dependencies) + - [before_script and after_script](#before_script-and-after_script) - [Hidden jobs](#hidden-jobs) - [Special YAML features](#special-yaml-features) - [Anchors](#anchors) @@ -342,6 +343,8 @@ job_name: | dependencies | no | Define other builds that a build depends on so that you can pass artifacts between them| | artifacts | no | Define list build artifacts | | cache | no | Define list of files that should be cached between subsequent runs | +| before_script | no | Override a set of commands that are executed before build | +| after_script | no | Override a set of commands that are executed after build | ### script @@ -686,6 +689,23 @@ deploy: script: make deploy ``` +### before_script and after_script + +It's possible to overwrite globally defined `before_script` and `after_script`: + +```yaml +before_script +- global before script + +job: + before_script: + - execute this instead of global before script + script: + - my command + after_script: + - execute this after my script +``` + ## Hidden jobs >**Note:** -- cgit v1.2.3 From c764b57f09ac55d181d17db018bdea9587fadea8 Mon Sep 17 00:00:00 2001 From: Kamil Trzcinski Date: Sun, 17 Apr 2016 09:11:37 -0400 Subject: Add note about version --- doc/ci/yaml/README.md | 3 +++ 1 file changed, 3 insertions(+) (limited to 'doc/ci') diff --git a/doc/ci/yaml/README.md b/doc/ci/yaml/README.md index 54b06f10b95..697445bceea 100644 --- a/doc/ci/yaml/README.md +++ b/doc/ci/yaml/README.md @@ -125,6 +125,9 @@ builds, including deploy builds. This can be an array or a multi-line string. ### after_script +>**Note:** +Introduced in GitLab 8.7 and GitLab Runner v1.2. + `after_script` is used to define the command that will be run after for all builds. This has to be an array or a multi-line string. -- cgit v1.2.3 From 5d89bdb16a9340c7d8b43efc03dba623f8fb4d31 Mon Sep 17 00:00:00 2001 From: Grzegorz Bizon Date: Sat, 16 Apr 2016 21:17:05 +0200 Subject: Add documentation for job-level build variables --- doc/ci/variables/README.md | 4 +++- doc/ci/yaml/README.md | 21 +++++++++++++++++++++ 2 files changed, 24 insertions(+), 1 deletion(-) (limited to 'doc/ci') diff --git a/doc/ci/variables/README.md b/doc/ci/variables/README.md index b0e53cbc261..35dac4c9337 100644 --- a/doc/ci/variables/README.md +++ b/doc/ci/variables/README.md @@ -70,12 +70,14 @@ These variables can be later used in all executed commands and scripts. The YAML-defined variables are also set to all created service containers, thus allowing to fine tune them. +Variables can be defined on a global level, but also on a job level. + More information about Docker integration can be found in [Using Docker Images](../docker/using_docker_images.md). ### User-defined variables (Secure Variables) **This feature requires GitLab Runner 0.4.0 or higher** -GitLab CI allows you to define per-project **Secure Variables** that are set in build environment. +GitLab CI allows you to define per-project **Secure Variables** that are set in build environment. The secure variables are stored out of the repository (the `.gitlab-ci.yml`). The variables are securely passed to GitLab Runner and are available in build environment. It's desired method to use them for storing passwords, secret keys or whatever you want. diff --git a/doc/ci/yaml/README.md b/doc/ci/yaml/README.md index abb6e97e5e6..ea7d6dfa465 100644 --- a/doc/ci/yaml/README.md +++ b/doc/ci/yaml/README.md @@ -23,6 +23,7 @@ If you want a quick introduction to GitLab CI, follow our - [Jobs](#jobs) - [script](#script) - [stage](#stage) + - [job variables](#job-variables) - [only and except](#only-and-except) - [tags](#tags) - [when](#when) @@ -174,6 +175,8 @@ These variables can be later used in all executed commands and scripts. The YAML-defined variables are also set to all created service containers, thus allowing to fine tune them. +Variables can be also defined on [job level](#job-variables). + ### cache >**Note:** @@ -324,6 +327,7 @@ job_name: | services | no | Use docker services, covered in [Using Docker Images](../docker/using_docker_images.md#define-image-and-services-from-gitlab-ciyml) | | stage | no | Defines a build stage (default: `test`) | | type | no | Alias for `stage` | +| variables | no | Define build variables on a job level | | only | no | Defines a list of git refs for which build is created | | except | no | Defines a list of git refs for which build is not created | | tags | no | Defines a list of tags which are used to select Runner | @@ -414,6 +418,23 @@ job: The above example will run `job` for all branches on `gitlab-org/gitlab-ce`, except master. +### job variables + +It is possible to define build variables using a `variables` keyword on a job +level. It works basically the same way like it's global-level equivalent but +allows you to define a job specific build variables. + +When `variables` keyword is used on a job level, it can override global YAML +build variables and predefined variables. + +Build variables priority is defined as follows: + +* predefined variables +* global YAML variables +* job YAML variables +* secure variables +* trigger variables + ### tags `tags` is used to select specific Runners from the list of all Runners that are -- cgit v1.2.3 From a2649739055842ce519fab5a8b0c68dbf23cb35a Mon Sep 17 00:00:00 2001 From: Grzegorz Bizon Date: Sat, 16 Apr 2016 22:04:59 +0200 Subject: Improve grammar in CI documentation for build vars --- doc/ci/variables/README.md | 11 +++++++---- 1 file changed, 7 insertions(+), 4 deletions(-) (limited to 'doc/ci') diff --git a/doc/ci/variables/README.md b/doc/ci/variables/README.md index 35dac4c9337..f321a618f3b 100644 --- a/doc/ci/variables/README.md +++ b/doc/ci/variables/README.md @@ -70,17 +70,20 @@ These variables can be later used in all executed commands and scripts. The YAML-defined variables are also set to all created service containers, thus allowing to fine tune them. -Variables can be defined on a global level, but also on a job level. +Variables can be defined at a global level, but also at a job level. More information about Docker integration can be found in [Using Docker Images](../docker/using_docker_images.md). ### User-defined variables (Secure Variables) **This feature requires GitLab Runner 0.4.0 or higher** -GitLab CI allows you to define per-project **Secure Variables** that are set in build environment. +GitLab CI allows you to define per-project **Secure Variables** that are set in +the build environment. The secure variables are stored out of the repository (the `.gitlab-ci.yml`). -The variables are securely passed to GitLab Runner and are available in build environment. -It's desired method to use them for storing passwords, secret keys or whatever you want. +The variables are securely passed to GitLab Runner and are available in the +build environment. +It's desired method to use them for storing passwords, secret keys or whatever +you want. **The value of the variable can be shown in build log if explicitly asked to do so.** If your project is public or internal you can make the builds private. -- cgit v1.2.3 From a2957291b3a28eff3988dfc8bdcff4b99b20a1b3 Mon Sep 17 00:00:00 2001 From: Grzegorz Bizon Date: Sun, 17 Apr 2016 20:08:02 +0200 Subject: Update documentation on job level build variables --- doc/ci/variables/README.md | 10 ++++++---- 1 file changed, 6 insertions(+), 4 deletions(-) (limited to 'doc/ci') diff --git a/doc/ci/variables/README.md b/doc/ci/variables/README.md index f321a618f3b..e1df9e5f543 100644 --- a/doc/ci/variables/README.md +++ b/doc/ci/variables/README.md @@ -1,17 +1,19 @@ ## Variables + When receiving a build from GitLab CI, the runner prepares the build environment. It starts by setting a list of **predefined variables** (Environment Variables) and a list of **user-defined variables** The variables can be overwritten. They take precedence over each other in this order: 1. Secure variables -1. YAML-defined variables +1. YAML-defined job-leve variables +1. YAML-defined global variables 1. Predefined variables For example, if you define: -1. API_TOKEN=SECURE as Secure Variable -1. API_TOKEN=YAML as YAML-defined variable +1. `API_TOKEN=SECURE` as Secure Variable +1. `API_TOKEN=YAML` as YAML-defined variable -The API_TOKEN will take the Secure Variable value: `SECURE`. +The `API_TOKEN` will take the Secure Variable value: `SECURE`. ### Predefined variables (Environment Variables) -- cgit v1.2.3 From 2972a991df0e7a0071de1803205814f2d105df46 Mon Sep 17 00:00:00 2001 From: Grzegorz Bizon Date: Mon, 18 Apr 2016 12:29:36 +0200 Subject: Add minor fixes in docs for job-specific variables --- doc/ci/variables/README.md | 3 ++- doc/ci/yaml/README.md | 15 +++++---------- 2 files changed, 7 insertions(+), 11 deletions(-) (limited to 'doc/ci') diff --git a/doc/ci/variables/README.md b/doc/ci/variables/README.md index e1df9e5f543..70fb81492d6 100644 --- a/doc/ci/variables/README.md +++ b/doc/ci/variables/README.md @@ -4,8 +4,9 @@ When receiving a build from GitLab CI, the runner prepares the build environment It starts by setting a list of **predefined variables** (Environment Variables) and a list of **user-defined variables** The variables can be overwritten. They take precedence over each other in this order: +1. Trigger variables 1. Secure variables -1. YAML-defined job-leve variables +1. YAML-defined job-level variables 1. YAML-defined global variables 1. Predefined variables diff --git a/doc/ci/yaml/README.md b/doc/ci/yaml/README.md index ea7d6dfa465..61475b45988 100644 --- a/doc/ci/yaml/README.md +++ b/doc/ci/yaml/README.md @@ -421,19 +421,14 @@ except master. ### job variables It is possible to define build variables using a `variables` keyword on a job -level. It works basically the same way like it's global-level equivalent but -allows you to define a job specific build variables. +level. It works basically the same way as its global-level equivalent but +allows you to define job-specific build variables. -When `variables` keyword is used on a job level, it can override global YAML +When the `variables` keyword is used on a job level, it overrides global YAML build variables and predefined variables. -Build variables priority is defined as follows: - -* predefined variables -* global YAML variables -* job YAML variables -* secure variables -* trigger variables +Build variables priority is defined in +[variables documentation](../variables/README.md). ### tags -- cgit v1.2.3 From 03e5873ae52f3c8c0efb7baa7d1a358a7c3e7974 Mon Sep 17 00:00:00 2001 From: Achilleas Pipinellis Date: Tue, 19 Apr 2016 19:04:58 +0300 Subject: Fix broken links [ci skip] --- doc/ci/deployment/README.md | 98 ------------------------------------ doc/ci/examples/deployment/README.md | 98 ++++++++++++++++++++++++++++++++++++ doc/ci/triggers/README.md | 2 +- 3 files changed, 99 insertions(+), 99 deletions(-) delete mode 100644 doc/ci/deployment/README.md create mode 100644 doc/ci/examples/deployment/README.md (limited to 'doc/ci') diff --git a/doc/ci/deployment/README.md b/doc/ci/deployment/README.md deleted file mode 100644 index 7d91ce6710f..00000000000 --- a/doc/ci/deployment/README.md +++ /dev/null @@ -1,98 +0,0 @@ -## Using Dpl as deployment tool -Dpl (dee-pee-ell) is a deploy tool made for continuous deployment that's developed and used by Travis CI, but can also be used with GitLab CI. - -**We recommend to use Dpl, if you're deploying to any of these of these services: https://github.com/travis-ci/dpl#supported-providers**. - -### Requirements -To use Dpl you need at least Ruby 1.8.7 with ability to install gems. - -### Basic usage -The Dpl can be installed on any machine with: -``` -gem install dpl -``` - -This allows you to test all commands from your shell, rather than having to test it on a CI server. - -If you don't have Ruby installed you can do it on Debian-compatible Linux with: -``` -apt-get update -apt-get install ruby-dev -``` - -The Dpl provides support for vast number of services, including: Heroku, Cloud Foundry, AWS/S3, and more. -To use it simply define provider and any additional parameters required by the provider. - -For example if you want to use it to deploy your application to heroku, you need to specify `heroku` as provider, specify `api-key` and `app`. -There's more and all possible parameters can be found here: https://github.com/travis-ci/dpl#heroku - -``` -staging: - type: deploy - - gem install dpl - - dpl --provider=heroku --app=my-app-staging --api-key=$HEROKU_STAGING_API_KEY -``` - -In the above example we use Dpl to deploy `my-app-staging` to Heroku server with api-key stored in `HEROKU_STAGING_API_KEY` secure variable. - -To use different provider take a look at long list of [Supported Providers](https://github.com/travis-ci/dpl#supported-providers). - -### Using Dpl with Docker -When you use GitLab Runner you most likely configured it to use your server's shell commands. -This means that all commands are run in context of local user (ie. gitlab_runner or gitlab_ci_multi_runner). -It also means that most probably in your Docker container you don't have the Ruby runtime installed. -You will have to install it: -``` -staging: - type: deploy - - apt-get update -yq - - apt-get install -y ruby-dev - - gem install dpl - - dpl --provider=heroku --app=my-app-staging --api-key=$HEROKU_STAGING_API_KEY - only: - - master -``` - -The first line `apt-get update -yq` updates the list of available packages, where second `apt-get install -y ruby-dev` install `Ruby` runtime on system. -The above example is valid for all Debian-compatible systems. - -### Usage in staging and production -It's pretty common in developer workflow to have staging (development) and production environment. -If we consider above example: we would like to deploy `master` branch to `staging` and `all tags` to `production` environment. -The final `.gitlab-ci.yml` for that setup would look like this: - -``` -staging: - type: deploy - - gem install dpl - - dpl --provider=heroku --app=my-app-staging --api-key=$HEROKU_STAGING_API_KEY - only: - - master - -production: - type: deploy - - gem install dpl - - dpl --provider=heroku --app=my-app-production --api-key=$HEROKU_PRODUCTION_API_KEY - only: - - tags -``` - -We created two deploy jobs that are executed on different events: -1. `staging` is executed for all commits that were pushed to `master` branch, -2. `production` is executed for all pushed tags. - -We also use two secure variables: -1. `HEROKU_STAGING_API_KEY` - Heroku API key used to deploy staging app, -2. `HEROKU_PRODUCTION_API_KEY` - Heroku API key used to deploy production app. - -### Storing API keys -In GitLab CI 7.12 a new feature was introduced: Secure Variables. -Secure Variables can added by going to `Project > Variables > Add Variable`. -**This feature requires `gitlab-runner` with version equal or greater than 0.4.0.** -The variables that are defined in the project settings are sent along with the build script to the runner. -The secure variables are stored out of the repository. Never store secrets in your projects' .gitlab-ci.yml. -It is also important that secret's value is hidden in the build log. - -You access added variable by prefixing it's name with `$` (on non-Windows runners) or `%` (for Windows Batch runners): -1. `$SECRET_VARIABLE` - use it for non-Windows runners -2. `%SECRET_VARIABLE%` - use it for Windows Batch runners diff --git a/doc/ci/examples/deployment/README.md b/doc/ci/examples/deployment/README.md new file mode 100644 index 00000000000..7d91ce6710f --- /dev/null +++ b/doc/ci/examples/deployment/README.md @@ -0,0 +1,98 @@ +## Using Dpl as deployment tool +Dpl (dee-pee-ell) is a deploy tool made for continuous deployment that's developed and used by Travis CI, but can also be used with GitLab CI. + +**We recommend to use Dpl, if you're deploying to any of these of these services: https://github.com/travis-ci/dpl#supported-providers**. + +### Requirements +To use Dpl you need at least Ruby 1.8.7 with ability to install gems. + +### Basic usage +The Dpl can be installed on any machine with: +``` +gem install dpl +``` + +This allows you to test all commands from your shell, rather than having to test it on a CI server. + +If you don't have Ruby installed you can do it on Debian-compatible Linux with: +``` +apt-get update +apt-get install ruby-dev +``` + +The Dpl provides support for vast number of services, including: Heroku, Cloud Foundry, AWS/S3, and more. +To use it simply define provider and any additional parameters required by the provider. + +For example if you want to use it to deploy your application to heroku, you need to specify `heroku` as provider, specify `api-key` and `app`. +There's more and all possible parameters can be found here: https://github.com/travis-ci/dpl#heroku + +``` +staging: + type: deploy + - gem install dpl + - dpl --provider=heroku --app=my-app-staging --api-key=$HEROKU_STAGING_API_KEY +``` + +In the above example we use Dpl to deploy `my-app-staging` to Heroku server with api-key stored in `HEROKU_STAGING_API_KEY` secure variable. + +To use different provider take a look at long list of [Supported Providers](https://github.com/travis-ci/dpl#supported-providers). + +### Using Dpl with Docker +When you use GitLab Runner you most likely configured it to use your server's shell commands. +This means that all commands are run in context of local user (ie. gitlab_runner or gitlab_ci_multi_runner). +It also means that most probably in your Docker container you don't have the Ruby runtime installed. +You will have to install it: +``` +staging: + type: deploy + - apt-get update -yq + - apt-get install -y ruby-dev + - gem install dpl + - dpl --provider=heroku --app=my-app-staging --api-key=$HEROKU_STAGING_API_KEY + only: + - master +``` + +The first line `apt-get update -yq` updates the list of available packages, where second `apt-get install -y ruby-dev` install `Ruby` runtime on system. +The above example is valid for all Debian-compatible systems. + +### Usage in staging and production +It's pretty common in developer workflow to have staging (development) and production environment. +If we consider above example: we would like to deploy `master` branch to `staging` and `all tags` to `production` environment. +The final `.gitlab-ci.yml` for that setup would look like this: + +``` +staging: + type: deploy + - gem install dpl + - dpl --provider=heroku --app=my-app-staging --api-key=$HEROKU_STAGING_API_KEY + only: + - master + +production: + type: deploy + - gem install dpl + - dpl --provider=heroku --app=my-app-production --api-key=$HEROKU_PRODUCTION_API_KEY + only: + - tags +``` + +We created two deploy jobs that are executed on different events: +1. `staging` is executed for all commits that were pushed to `master` branch, +2. `production` is executed for all pushed tags. + +We also use two secure variables: +1. `HEROKU_STAGING_API_KEY` - Heroku API key used to deploy staging app, +2. `HEROKU_PRODUCTION_API_KEY` - Heroku API key used to deploy production app. + +### Storing API keys +In GitLab CI 7.12 a new feature was introduced: Secure Variables. +Secure Variables can added by going to `Project > Variables > Add Variable`. +**This feature requires `gitlab-runner` with version equal or greater than 0.4.0.** +The variables that are defined in the project settings are sent along with the build script to the runner. +The secure variables are stored out of the repository. Never store secrets in your projects' .gitlab-ci.yml. +It is also important that secret's value is hidden in the build log. + +You access added variable by prefixing it's name with `$` (on non-Windows runners) or `%` (for Windows Batch runners): +1. `$SECRET_VARIABLE` - use it for non-Windows runners +2. `%SECRET_VARIABLE%` - use it for Windows Batch runners diff --git a/doc/ci/triggers/README.md b/doc/ci/triggers/README.md index 9f7c1bfe6a0..1848f6319d8 100644 --- a/doc/ci/triggers/README.md +++ b/doc/ci/triggers/README.md @@ -33,7 +33,7 @@ POST /projects/:id/trigger/builds The required parameters are the trigger's `token` and the Git `ref` on which the trigger will be performed. Valid refs are the branch, the tag or the commit -SHA. The `:id` of a project can be found by [querying the API](../api/projects.md) +SHA. The `:id` of a project can be found by [querying the API](../../api/projects.md) or by visiting the **Triggers** page which provides self-explanatory examples. When a rebuild is triggered, the information is exposed in GitLab's UI under -- cgit v1.2.3 From 0725c28069689ae958adfc140f0a47fa9596f641 Mon Sep 17 00:00:00 2001 From: Tomasz Maczukin Date: Sat, 16 Apr 2016 15:29:12 +0200 Subject: Add documentation of incremental trace update API [ci skip] --- doc/ci/api/builds.md | 108 +++++++++++++++++++++++++++++++++++++++++---------- 1 file changed, 87 insertions(+), 21 deletions(-) (limited to 'doc/ci') diff --git a/doc/ci/api/builds.md b/doc/ci/api/builds.md index d100e261178..119dbbe9386 100644 --- a/doc/ci/api/builds.md +++ b/doc/ci/api/builds.md @@ -26,48 +26,114 @@ This API uses two types of authentication: ### Runs oldest pending build by runner - POST /ci/api/v1/builds/register +``` +POST /ci/api/v1/builds/register +``` -Parameters: +| Attribute | Type | Required | Description | +|-----------|---------|----------|---------------------| +| `token` | string | yes | Unique runner token | - * `token` (required) - Unique runner token +``` +curl -X POST "https://gitlab.example.com/ci/api/v1/builds/register" -F "token=t0k3n" +``` ### Update details of an existing build - PUT /ci/api/v1/builds/:id +``` +PUT /ci/api/v1/builds/:id +``` + +| Attribute | Type | Required | Description | +|-----------|---------|----------|----------------------| +| `id` | integer | yes | The ID of a project | +| `token` | string | yes | Unique runner token | +| `state` | string | no | The state of a build | +| `trace` | string | no | The trace of a build | + +``` +curl -X PUT "https://gitlab.example.com/ci/api/v1/builds/1234" -F "token=t0k3n" -F "state=running" -F "trace=Running git clone...\n" +``` + +### Incremental build trace update + +Using this method you need to send trace content as a request body. You need also to provide the `Content-Range` header +with a range of sent trace part. Note, that you need to send parts in a proper order, so the begining of the part +must starts just after the end of the previous part. If you will mess the parts, then GitLab CI AIP will return `416 +Range Not Satisfiable` response with a header `Range: 0-X`, where `X` is the current trace length. + +For example: if you receive `Range: 0-11` in the response, then your next part must contains `Content-Range: 11-...` +header and a trace part covered by this range. + +For a valid update API will return `202` response with: +* `Build-Status: {status}` header containing current status of the build, +* `Range: 0-{length}` header with the current trace length. + +``` +PATCH /ci/api/v1/builds/:id/trace.txt +``` Parameters: - * `id` (required) - The ID of a project - * `token` (required) - Unique runner token - * `state` (optional) - The state of a build - * `trace` (optional) - The trace of a build +| Attribute | Type | Required | Description | +|-----------|---------|----------|----------------------| +| `id` | integer | yes | The ID of a build | + +Headers: + +| Attribute | Type | Required | Description | +|-----------------|---------|----------|-----------------------------------| +| `BUILD-TOKEN` | string | yes | The build authorization token | +| `Content-Range` | string | yes | Bytes range of trace that is sent | + +``` +curl -X PATCH "https://gitlab.example.com/ci/api/v1/builds/1234/trace.txt" -H "BUILD-TOKEN=build_t0k3n" -H "Content-Range=0-21" -d "Running git clone...\n" +``` + ### Upload artifacts to build - POST /ci/api/v1/builds/:id/artifacts +``` +POST /ci/api/v1/builds/:id/artifacts +``` -Parameters: +| Attribute | Type | Required | Description | +|-----------|---------|----------|-------------------------------| +| `id` | integer | yes | The ID of a build | +| `token` | string | yes | The build authorization token | +| `file` | mixed | yes | Artifacts file | - * `id` (required) - The ID of a build - * `token` (required) - The build authorization token - * `file` (required) - Artifacts file +``` +curl -X POST "https://gitlab.example.com/ci/api/v1/builds/1234/artifacts" -F "token=build_t0k3n" -F "file=@/path/to/file" +``` ### Download the artifacts file from build - GET /ci/api/v1/builds/:id/artifacts +``` +GET /ci/api/v1/builds/:id/artifacts +``` -Parameters: +| Attribute | Type | Required | Description | +|-----------|---------|----------|-------------------------------| +| `id` | integer | yes | The ID of a build | +| `token` | string | yes | The build authorization token | - * `id` (required) - The ID of a build - * `token` (required) - The build authorization token +``` +curl "https://gitlab.example.com/ci/api/v1/builds/1234/artifacts" -F "token=build_t0k3n" +``` ### Remove the artifacts file from build - DELETE /ci/api/v1/builds/:id/artifacts +``` +DELETE /ci/api/v1/builds/:id/artifacts +``` -Parameters: +| Attribute | Type | Required | Description | +|-----------|---------|----------|-------------------------------| +| ` id` | integer | yes | The ID of a build | +| `token` | string | yes | The build authorization token | - * ` id` (required) - The ID of a build - * `token` (required) - The build authorization token +``` +curl -X DELETE "https://gitlab.example.com/ci/api/v1/builds/1234/artifacts" -F "token=build_t0k3n" +``` -- cgit v1.2.3 From 4019b0d872bd2b183a06bdc832f5a100c8a88a00 Mon Sep 17 00:00:00 2001 From: Tomasz Maczukin Date: Tue, 19 Apr 2016 13:42:50 +0200 Subject: Fix some typos in documentation [ci skip] --- doc/ci/api/builds.md | 8 ++++---- 1 file changed, 4 insertions(+), 4 deletions(-) (limited to 'doc/ci') diff --git a/doc/ci/api/builds.md b/doc/ci/api/builds.md index 119dbbe9386..79761a893da 100644 --- a/doc/ci/api/builds.md +++ b/doc/ci/api/builds.md @@ -58,12 +58,12 @@ curl -X PUT "https://gitlab.example.com/ci/api/v1/builds/1234" -F "token=t0k3n" ### Incremental build trace update -Using this method you need to send trace content as a request body. You need also to provide the `Content-Range` header -with a range of sent trace part. Note, that you need to send parts in a proper order, so the begining of the part -must starts just after the end of the previous part. If you will mess the parts, then GitLab CI AIP will return `416 +Using this method you need to send trace content as a request body. You also need to provide the `Content-Range` header +with a range of sent trace part. Note that you need to send parts in the proper order, so the begining of the part +must start just after the end of the previous part. If you provide the wrong part, then GitLab CI API will return `416 Range Not Satisfiable` response with a header `Range: 0-X`, where `X` is the current trace length. -For example: if you receive `Range: 0-11` in the response, then your next part must contains `Content-Range: 11-...` +For example, if you receive `Range: 0-11` in the response, then your next part must contain a `Content-Range: 11-...` header and a trace part covered by this range. For a valid update API will return `202` response with: -- cgit v1.2.3 From c55f517719e8941ccedcf5167828bb840da3f302 Mon Sep 17 00:00:00 2001 From: Tomasz Maczukin Date: Mon, 25 Apr 2016 11:29:47 +0200 Subject: Fix php-docker example (for new docker versions) [ci skip] --- doc/ci/examples/php.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) (limited to 'doc/ci') diff --git a/doc/ci/examples/php.md b/doc/ci/examples/php.md index aeadd6a448e..db077927126 100644 --- a/doc/ci/examples/php.md +++ b/doc/ci/examples/php.md @@ -40,7 +40,7 @@ repository with the following content: #!/bin/bash # We need to install dependencies only for Docker -[[ ! -e /.dockerinit ]] && exit 0 +[[ ! -e /.dockerenv ]] && [[ ! -e /.dockerinit ]] && exit 0 set -xe -- cgit v1.2.3 From 9bd0f46a85c9f642722000de8317607c4ed567d4 Mon Sep 17 00:00:00 2001 From: Tomasz Maczukin Date: Mon, 25 Apr 2016 12:10:56 +0200 Subject: Replace '-n' with '--name' in docker related documentation [ci skip] --- doc/ci/docker/using_docker_images.md | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) (limited to 'doc/ci') diff --git a/doc/ci/docker/using_docker_images.md b/doc/ci/docker/using_docker_images.md index bd748f1b986..84212fb3c61 100644 --- a/doc/ci/docker/using_docker_images.md +++ b/doc/ci/docker/using_docker_images.md @@ -239,8 +239,8 @@ is specific to your project. Then create some service containers: ``` -docker run -d -n service-mysql mysql:latest -docker run -d -n service-postgres postgres:latest +docker run -d --name service-mysql mysql:latest +docker run -d --name service-postgres postgres:latest ``` This will create two service containers, named `service-mysql` and -- cgit v1.2.3 From 5fc6a7dc14aa05748049eebd0a53d5e615cd1a9a Mon Sep 17 00:00:00 2001 From: Tomasz Maczukin Date: Mon, 25 Apr 2016 13:51:17 +0200 Subject: Update using_docker_build.md, clarify the 'privileged' mode requirement [ci skip] --- doc/ci/docker/using_docker_build.md | 64 +++++++++++++++++++++++++++---------- 1 file changed, 48 insertions(+), 16 deletions(-) (limited to 'doc/ci') diff --git a/doc/ci/docker/using_docker_build.md b/doc/ci/docker/using_docker_build.md index 4b1788a9af0..bb2a6d1137d 100644 --- a/doc/ci/docker/using_docker_build.md +++ b/doc/ci/docker/using_docker_build.md @@ -8,7 +8,7 @@ This is one of new trends in Continuous Integration/Deployment to: 1. create application image, 1. run test against created image, -1. push image to remote registry, +1. push image to remote registry, 1. deploy server from pushed image It's also useful in case when your application already has the `Dockerfile` that can be used to create and test image: @@ -46,22 +46,22 @@ GitLab Runner then executes build scripts as `gitlab-runner` user. For more information how to install Docker on different systems checkout the [Supported installations](https://docs.docker.com/installation/). 3. Add `gitlab-runner` user to `docker` group: - + ```bash $ sudo usermod -aG docker gitlab-runner ``` 4. Verify that `gitlab-runner` has access to Docker: - + ```bash $ sudo -u gitlab-runner -H docker info ``` - + You can now verify that everything works by adding `docker info` to `.gitlab-ci.yml`: ```yaml before_script: - docker info - + build_image: script: - docker build -t my-docker-image . @@ -88,24 +88,56 @@ In order to do that follow the steps: --token RUNNER_TOKEN \ --executor docker \ --description "My Docker Runner" \ - --docker-image "gitlab/dind:latest" \ + --docker-image "docker:latest" \ --docker-privileged ``` - - The above command will register new Runner to use special [gitlab/dind](https://registry.hub.docker.com/u/gitlab/dind/) image which is provided by GitLab Inc. - The image at the start runs Docker daemon in [docker-in-docker](https://blog.docker.com/2013/09/docker-can-now-run-within-docker/) mode. + + The above command will register a new Runner to use special `docker:latest` image which is provided by Docker + creators. **Notice that it's using the `privileged` mode to start build and service containers.** If you want to use + [docker-in-docker](https://blog.docker.com/2013/09/docker-can-now-run-within-docker/) mode, you always have to use + `privileged = true` in your docker containers. + + The above command will create a `config.toml` entry similar to this: + + ``` + [[runners]] + url = "https://gitlab.com/ci" + token = TOKEN + executor = "docker" + [runners.docker] + tls_verify = false + image = "docker:latest" + privileged = true + disable_cache = false + volumes = ["/cache"] + [runners.cache] + Insecure = false + ``` + + If you want to use Shared Runners available on your GitLab CE/EE installation, to build docker images, then + make sure that your Shared Runners configuration have `privileged` mode set to `true`. 1. You can now use `docker` from build script: - + ```yaml + image: docker:latest + + services: + - docker:dind + before_script: - docker info - - build_image: + + build: + stage: build script: - - docker build -t my-docker-image . - - docker run my-docker-image /script/to/run/tests + - docker build -t my-docker-image . + - docker run my-docker-image /script/to/run/tests ``` -1. However, by enabling `--docker-privileged` you are effectively disables all security mechanisms of containers and exposing your host to privilege escalation which can lead to container breakout. -For more information, check out [Runtime privilege](https://docs.docker.com/reference/run/#runtime-privilege-linux-capabilities-and-lxc-configuration). \ No newline at end of file +1. However, by enabling `--docker-privileged` you are effectively disables all security mechanisms of containers and + exposing your host to privilege escalation which can lead to container breakout. + + For more information, check out [Runtime privilege](https://docs.docker.com/reference/run/#runtime-privilege-linux-capabilities-and-lxc-configuration). + +An example project using this approach can be found here: https://gitlab.com/gitlab-examples/docker. -- cgit v1.2.3 From 0ace42cdb90161ee7daf69b7a6a7af6ce4195208 Mon Sep 17 00:00:00 2001 From: Jason Roehm Date: Tue, 26 Apr 2016 21:41:52 -0400 Subject: fix: in recent versions of Docker, the /.dockerinit file doesn't exist; use /.dockerenv instead [ci skip] Signed-off-by: Jason Roehm --- doc/ci/ssh_keys/README.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) (limited to 'doc/ci') diff --git a/doc/ci/ssh_keys/README.md b/doc/ci/ssh_keys/README.md index 7f825e6a065..7c0fb225dac 100644 --- a/doc/ci/ssh_keys/README.md +++ b/doc/ci/ssh_keys/README.md @@ -57,7 +57,7 @@ before_script: # WARNING: Use this only with the Docker executor, if you use it with shell # you will overwrite your user's SSH config. - mkdir -p ~/.ssh - - '[[ -f /.dockerinit ]] && echo -e "Host *\n\tStrictHostKeyChecking no\n\n" > ~/.ssh/config' + - '[[ -f /.dockerenv ]] && echo -e "Host *\n\tStrictHostKeyChecking no\n\n" > ~/.ssh/config' ``` As a final step, add the _public_ key from the one you created earlier to the -- cgit v1.2.3 From 7c9233856e9f14f48811dc5678e98a036f2fa0de Mon Sep 17 00:00:00 2001 From: David de Boer Date: Wed, 27 Apr 2016 16:41:37 +0200 Subject: Add API doc example with query string for triggering build --- doc/ci/triggers/README.md | 6 ++++++ 1 file changed, 6 insertions(+) (limited to 'doc/ci') diff --git a/doc/ci/triggers/README.md b/doc/ci/triggers/README.md index 9f7c1bfe6a0..79ed512aabb 100644 --- a/doc/ci/triggers/README.md +++ b/doc/ci/triggers/README.md @@ -85,6 +85,12 @@ curl -X POST \ In this case, the project with ID `9` will get rebuilt on `master` branch. +Alternatively, you can pass the `token` and `ref` arguments in the query string: + +```bash +curl -X POST \ + "https://gitlab.example.com/api/v3/projects/9/trigger/builds?token=TOKEN&ref=master" +``` ### Triggering a build within `.gitlab-ci.yml` -- cgit v1.2.3 From 8b09dafb16726168ab78ff425c7c3d7c668ac5a5 Mon Sep 17 00:00:00 2001 From: Achilleas Pipinellis Date: Thu, 28 Apr 2016 13:04:53 +0300 Subject: Copyedit `using_docker_build.md` and fix links --- doc/ci/docker/using_docker_build.md | 37 ++++++++++++++++++++++++------------- 1 file changed, 24 insertions(+), 13 deletions(-) (limited to 'doc/ci') diff --git a/doc/ci/docker/using_docker_build.md b/doc/ci/docker/using_docker_build.md index bb2a6d1137d..5fb086b1dd9 100644 --- a/doc/ci/docker/using_docker_build.md +++ b/doc/ci/docker/using_docker_build.md @@ -75,15 +75,19 @@ For more information please checkout [On Docker security: `docker` group conside ## 2. Use docker-in-docker executor -Second approach is to use special Docker image with all tools installed (`docker` and `docker-compose`) and run build script in context of that image in privileged mode. +The second approach is to use the special Docker image with all tools installed +(`docker` and `docker-compose`) and run the build script in context of that +image in privileged mode. + In order to do that follow the steps: 1. Install [GitLab Runner](https://gitlab.com/gitlab-org/gitlab-ci-multi-runner/#installation). -1. Register GitLab Runner from command line to use `docker` and `privileged` mode: +1. Register GitLab Runner from the command line to use `docker` and `privileged` + mode: ```bash - $ sudo gitlab-runner register -n \ + sudo gitlab-runner register -n \ --url https://gitlab.com/ci \ --token RUNNER_TOKEN \ --executor docker \ @@ -92,10 +96,11 @@ In order to do that follow the steps: --docker-privileged ``` - The above command will register a new Runner to use special `docker:latest` image which is provided by Docker - creators. **Notice that it's using the `privileged` mode to start build and service containers.** If you want to use - [docker-in-docker](https://blog.docker.com/2013/09/docker-can-now-run-within-docker/) mode, you always have to use - `privileged = true` in your docker containers. + The above command will register a new Runner to use the special + `docker:latest` image which is provided by Docker. **Notice that it's using + the `privileged` mode to start the build and service containers.** If you + want to use [docker-in-docker] mode, you always have to use `privileged = true` + in your Docker containers. The above command will create a `config.toml` entry similar to this: @@ -114,8 +119,9 @@ In order to do that follow the steps: Insecure = false ``` - If you want to use Shared Runners available on your GitLab CE/EE installation, to build docker images, then - make sure that your Shared Runners configuration have `privileged` mode set to `true`. + If you want to use the Shared Runners available on your GitLab CE/EE + installation in order to build Docker images, then make sure that your + Shared Runners configuration has the `privileged` mode set to `true`. 1. You can now use `docker` from build script: @@ -126,7 +132,7 @@ In order to do that follow the steps: - docker:dind before_script: - - docker info + - docker info build: stage: build @@ -135,9 +141,14 @@ In order to do that follow the steps: - docker run my-docker-image /script/to/run/tests ``` -1. However, by enabling `--docker-privileged` you are effectively disables all security mechanisms of containers and - exposing your host to privilege escalation which can lead to container breakout. +1. However, by enabling `--docker-privileged` you are effectively disabling all + the security mechanisms of containers and exposing your host to privilege + escalation which can lead to container breakout. - For more information, check out [Runtime privilege](https://docs.docker.com/reference/run/#runtime-privilege-linux-capabilities-and-lxc-configuration). + For more information, check out the official Docker documentation on + [Runtime privilege and Linux capabilities][docker-cap]. An example project using this approach can be found here: https://gitlab.com/gitlab-examples/docker. + +[docker-in-docker]: https://blog.docker.com/2013/09/docker-can-now-run-within-docker/ +[docker-cap]: https://docs.docker.com/reference/run/#runtime-privilege-and-linux-capabilities -- cgit v1.2.3 From 80b30500e44b621c0ecd3c062349b34ddbb0a0fb Mon Sep 17 00:00:00 2001 From: Ben Bodenmiller Date: Fri, 6 May 2016 00:42:13 +0000 Subject: blost -> blog --- doc/ci/examples/README.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) (limited to 'doc/ci') diff --git a/doc/ci/examples/README.md b/doc/ci/examples/README.md index cc059dc4376..3a152c9116a 100644 --- a/doc/ci/examples/README.md +++ b/doc/ci/examples/README.md @@ -10,6 +10,6 @@ ## Outside the documentation -- [Blost post about using GitLab CI for iOS projects](https://about.gitlab.com/2016/03/10/setting-up-gitlab-ci-for-ios-projects/) +- [Blog post about using GitLab CI for iOS projects](https://about.gitlab.com/2016/03/10/setting-up-gitlab-ci-for-ios-projects/) - [Repo's with examples for various languages](https://gitlab.com/groups/gitlab-examples) - [The .gitlab-ci.yml file for GitLab itself](https://gitlab.com/gitlab-org/gitlab-ce/blob/master/.gitlab-ci.yml) -- cgit v1.2.3 From dd4fe8e9bbdd1f9874f20fbbd481bf56158f4f0c Mon Sep 17 00:00:00 2001 From: Takuya Noguchi Date: Sat, 7 May 2016 13:09:27 +0900 Subject: Update CI example docs. --- doc/ci/examples/README.md | 2 +- .../examples/test-and-deploy-python-application-to-heroku.md | 10 +++++----- doc/ci/examples/test-and-deploy-ruby-application-to-heroku.md | 6 +++--- 3 files changed, 9 insertions(+), 9 deletions(-) (limited to 'doc/ci') diff --git a/doc/ci/examples/README.md b/doc/ci/examples/README.md index cc059dc4376..31760d872b2 100644 --- a/doc/ci/examples/README.md +++ b/doc/ci/examples/README.md @@ -4,7 +4,7 @@ - [Test and deploy a Ruby application to Heroku](test-and-deploy-ruby-application-to-heroku.md) - [Test and deploy a Python application to Heroku](test-and-deploy-python-application-to-heroku.md) - [Test a Clojure application](test-clojure-application.md) -- [Using `dpl` as deployment tool](deployment/README.md) +- [Using `dpl` as deployment tool](../deployment/README.md) - Help your favorite programming language and GitLab by sending a merge request with a guide for that language. diff --git a/doc/ci/examples/test-and-deploy-python-application-to-heroku.md b/doc/ci/examples/test-and-deploy-python-application-to-heroku.md index a236da53fe9..e4d3970deac 100644 --- a/doc/ci/examples/test-and-deploy-python-application-to-heroku.md +++ b/doc/ci/examples/test-and-deploy-python-application-to-heroku.md @@ -8,7 +8,7 @@ This is what the `.gitlab-ci.yml` file looks like for this project: ```yaml test: script: - # this configures django application to use attached postgres database that is run on `postgres` host + # this configures Django application to use attached postgres database that is run on `postgres` host - export DATABASE_URL=postgres://postgres:@postgres:5432/python-test-app - apt-get update -qy - apt-get install -y python-dev python-pip @@ -37,7 +37,7 @@ production: ``` This project has three jobs: -1. `test` - used to test rails application, +1. `test` - used to test Django application, 2. `staging` - used to automatically deploy staging environment every push to `master` branch 3. `production` - used to automatically deploy production environmnet for every created tag @@ -61,12 +61,12 @@ gitlab-ci-multi-runner register \ --non-interactive \ --url "https://gitlab.com/ci/" \ --registration-token "PROJECT_REGISTRATION_TOKEN" \ - --description "python-3.2" \ + --description "python-3.5" \ --executor "docker" \ - --docker-image python:3.2 \ + --docker-image python:3.5 \ --docker-postgres latest ``` -With the command above, you create a runner that uses [python:3.2](https://registry.hub.docker.com/u/library/python/) image and uses [postgres](https://registry.hub.docker.com/u/library/postgres/) database. +With the command above, you create a runner that uses [python:3.5](https://hub.docker.com/r/_/python/) image and uses [postgres](https://hub.docker.com/r/_/postgres/) database. To access PostgreSQL database you need to connect to `host: postgres` as user `postgres` without password. diff --git a/doc/ci/examples/test-and-deploy-ruby-application-to-heroku.md b/doc/ci/examples/test-and-deploy-ruby-application-to-heroku.md index f5645d586ae..08c10d391ea 100644 --- a/doc/ci/examples/test-and-deploy-ruby-application-to-heroku.md +++ b/doc/ci/examples/test-and-deploy-ruby-application-to-heroku.md @@ -1,5 +1,5 @@ ## Test and Deploy a ruby application -This example will guide you how to run tests in your Ruby application and deploy it automatically as Heroku application. +This example will guide you how to run tests in your Ruby on Rails application and deploy it automatically as Heroku application. You can checkout the example [source](https://gitlab.com/ayufan/ruby-getting-started) and check [CI status](https://gitlab.com/ayufan/ruby-getting-started/builds?scope=all). @@ -32,7 +32,7 @@ production: ``` This project has three jobs: -1. `test` - used to test rails application, +1. `test` - used to test Rails application, 2. `staging` - used to automatically deploy staging environment every push to `master` branch 3. `production` - used to automatically deploy production environmnet for every created tag @@ -62,6 +62,6 @@ gitlab-ci-multi-runner register \ --docker-postgres latest ``` -With the command above, you create a runner that uses [ruby:2.2](https://registry.hub.docker.com/u/library/ruby/) image and uses [postgres](https://registry.hub.docker.com/u/library/postgres/) database. +With the command above, you create a runner that uses [ruby:2.2](https://hub.docker.com/r/_/ruby/) image and uses [postgres](https://hub.docker.com/r/_/postgres/) database. To access PostgreSQL database you need to connect to `host: postgres` as user `postgres` without password. -- cgit v1.2.3 From 95221312599a88b2f7898a22b2fa5b47d3b61f8c Mon Sep 17 00:00:00 2001 From: Takuya Noguchi Date: Sat, 7 May 2016 13:29:24 +0900 Subject: Update Docker Hub links. --- doc/ci/docker/using_docker_images.md | 6 +++--- doc/ci/examples/php.md | 6 +++--- doc/ci/services/mysql.md | 4 ++-- doc/ci/services/postgres.md | 2 +- doc/ci/services/redis.md | 2 +- 5 files changed, 10 insertions(+), 10 deletions(-) (limited to 'doc/ci') diff --git a/doc/ci/docker/using_docker_images.md b/doc/ci/docker/using_docker_images.md index 84212fb3c61..ab7369ce972 100644 --- a/doc/ci/docker/using_docker_images.md +++ b/doc/ci/docker/using_docker_images.md @@ -273,7 +273,7 @@ creation. [Docker Fundamentals]: https://docs.docker.com/engine/understanding-docker/ [hub]: https://hub.docker.com/ [linking-containers]: https://docs.docker.com/engine/userguide/networking/default_network/dockerlinks/ -[tutum/wordpress]: https://registry.hub.docker.com/u/tutum/wordpress/ -[postgres-hub]: https://registry.hub.docker.com/u/library/postgres/ -[mysql-hub]: https://registry.hub.docker.com/u/library/mysql/ +[tutum/wordpress]: https://hub.docker.com/r/tutum/wordpress/ +[postgres-hub]: https://hub.docker.com/r/_/postgres/ +[mysql-hub]: https://hub.docker.com/r/_/mysql/ [runner-priv-reg]: https://gitlab.com/gitlab-org/gitlab-ci-multi-runner/blob/master/docs/configuration/advanced-configuration.md#using-a-private-docker-registry diff --git a/doc/ci/examples/php.md b/doc/ci/examples/php.md index db077927126..26953014502 100644 --- a/doc/ci/examples/php.md +++ b/doc/ci/examples/php.md @@ -60,7 +60,7 @@ docker-php-ext-install pdo_mysql You might wonder what `docker-php-ext-install` is. In short, it is a script provided by the official php docker image that you can use to easilly install extensions. For more information read the the documentation at -. +. Now that we created the script that contains all prerequisites for our build environment, let's add it in `.gitlab-ci.yml`: @@ -92,7 +92,7 @@ Finally, commit your files and push them to GitLab to see your build succeeding The final `.gitlab-ci.yml` should look similar to this: ```yaml -# Select image from https://hub.docker.com/_/php/ +# Select image from https://hub.docker.com/r/_/php/ image: php:5.6 before_script: @@ -278,7 +278,7 @@ that runs on [GitLab.com](https://gitlab.com) using our publicly available Want to hack on it? Simply fork it, commit and push your changes. Within a few moments the changes will be picked by a public runner and the build will begin. -[php-hub]: https://hub.docker.com/_/php/ +[php-hub]: https://hub.docker.com/r/_/php/ [phpenv]: https://github.com/phpenv/phpenv [phpenv-installation]: https://github.com/phpenv/phpenv#installation [php-example-repo]: https://gitlab.com/gitlab-examples/php diff --git a/doc/ci/services/mysql.md b/doc/ci/services/mysql.md index c66d77122b2..aaf3aa77837 100644 --- a/doc/ci/services/mysql.md +++ b/doc/ci/services/mysql.md @@ -16,7 +16,7 @@ services: - mysql:latest variables: - # Configure mysql environment variables (https://hub.docker.com/_/mysql/) + # Configure mysql environment variables (https://hub.docker.com/r/_/mysql/) MYSQL_DATABASE: el_duderino MYSQL_ROOT_PASSWORD: mysql_strong_password ``` @@ -114,5 +114,5 @@ available [shared runners](../runners/README.md). Want to hack on it? Simply fork it, commit and push your changes. Within a few moments the changes will be picked by a public runner and the build will begin. -[hub-mysql]: https://hub.docker.com/_/mysql/ +[hub-mysql]: https://hub.docker.com/r/_/mysql/ [mysql-example-repo]: https://gitlab.com/gitlab-examples/mysql diff --git a/doc/ci/services/postgres.md b/doc/ci/services/postgres.md index 17d21dbda1c..f787cc0a124 100644 --- a/doc/ci/services/postgres.md +++ b/doc/ci/services/postgres.md @@ -110,5 +110,5 @@ available [shared runners](../runners/README.md). Want to hack on it? Simply fork it, commit and push your changes. Within a few moments the changes will be picked by a public runner and the build will begin. -[hub-pg]: https://hub.docker.com/_/postgres/ +[hub-pg]: https://hub.docker.com/r/_/postgres/ [postgres-example-repo]: https://gitlab.com/gitlab-examples/postgres diff --git a/doc/ci/services/redis.md b/doc/ci/services/redis.md index b281e8f9f60..80705024d2f 100644 --- a/doc/ci/services/redis.md +++ b/doc/ci/services/redis.md @@ -65,5 +65,5 @@ that runs on [GitLab.com](https://gitlab.com) using our publicly available Want to hack on it? Simply fork it, commit and push your changes. Within a few moments the changes will be picked by a public runner and the build will begin. -[hub-redis]: https://hub.docker.com/_/redis/ +[hub-redis]: https://hub.docker.com/r/_/redis/ [redis-example-repo]: https://gitlab.com/gitlab-examples/redis -- cgit v1.2.3 From 3c8265559a6965411c5c155a15fae4f2a822a532 Mon Sep 17 00:00:00 2001 From: Takuya Noguchi Date: Sat, 7 May 2016 14:54:36 +0900 Subject: Rename Docker with Docker Engine. --- doc/ci/docker/using_docker_build.md | 6 +++--- doc/ci/docker/using_docker_images.md | 2 +- 2 files changed, 4 insertions(+), 4 deletions(-) (limited to 'doc/ci') diff --git a/doc/ci/docker/using_docker_build.md b/doc/ci/docker/using_docker_build.md index 5fb086b1dd9..ca52a483a59 100644 --- a/doc/ci/docker/using_docker_build.md +++ b/doc/ci/docker/using_docker_build.md @@ -41,9 +41,9 @@ GitLab Runner then executes build scripts as `gitlab-runner` user. --description "My Runner" ``` -2. Install Docker on server. +2. Install Docker Engine on server. - For more information how to install Docker on different systems checkout the [Supported installations](https://docs.docker.com/installation/). + For more information how to install Docker Engine on different systems checkout the [Supported installations](https://docs.docker.com/engine/installation/). 3. Add `gitlab-runner` user to `docker` group: @@ -151,4 +151,4 @@ In order to do that follow the steps: An example project using this approach can be found here: https://gitlab.com/gitlab-examples/docker. [docker-in-docker]: https://blog.docker.com/2013/09/docker-can-now-run-within-docker/ -[docker-cap]: https://docs.docker.com/reference/run/#runtime-privilege-and-linux-capabilities +[docker-cap]: https://docs.docker.com/engine/reference/run/#runtime-privilege-and-linux-capabilities diff --git a/doc/ci/docker/using_docker_images.md b/doc/ci/docker/using_docker_images.md index ab7369ce972..56ac2195c49 100644 --- a/doc/ci/docker/using_docker_images.md +++ b/doc/ci/docker/using_docker_images.md @@ -64,7 +64,7 @@ You can see some widely used services examples in the relevant documentation of ### How is service linked to the build To better understand how the container linking works, read -[Linking containers together](https://docs.docker.com/userguide/dockerlinks/). +[Linking containers together][linking-containers]. To summarize, if you add `mysql` as service to your application, the image will then be used to create a container that is linked to the build container. -- cgit v1.2.3 From 01256eecfa39cf3edc3d0eb43e624f62e45df552 Mon Sep 17 00:00:00 2001 From: Tomasz Maczukin Date: Wed, 18 May 2016 21:34:58 -0500 Subject: Update 'after_script' introduction note --- doc/ci/yaml/README.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) (limited to 'doc/ci') diff --git a/doc/ci/yaml/README.md b/doc/ci/yaml/README.md index 7e9bced7616..63866d8c71c 100644 --- a/doc/ci/yaml/README.md +++ b/doc/ci/yaml/README.md @@ -128,7 +128,7 @@ builds, including deploy builds. This can be an array or a multi-line string. ### after_script >**Note:** -Introduced in GitLab 8.7 and GitLab Runner v1.2. +Introduced in GitLab 8.7 and requires Gitlab Runner v1.2 (not yet released) `after_script` is used to define the command that will be run after for all builds. This has to be an array or a multi-line string. -- cgit v1.2.3 From b253aa32c75cb5b0796ad6a7cd85cba78516ba26 Mon Sep 17 00:00:00 2001 From: Grzegorz Bizon Date: Fri, 6 May 2016 12:44:20 +0200 Subject: Add docs for a new configuration setting for runner --- doc/ci/runners/README.md | 8 +++++++- 1 file changed, 7 insertions(+), 1 deletion(-) (limited to 'doc/ci') diff --git a/doc/ci/runners/README.md b/doc/ci/runners/README.md index a06650b3387..e449d3dc61e 100644 --- a/doc/ci/runners/README.md +++ b/doc/ci/runners/README.md @@ -125,7 +125,13 @@ shared runners will only run the jobs they are equipped to run. For instance, at GitLab we have runners tagged with "rails" if they contain the appropriate dependencies to run Rails test suites. -### Be Careful with Sensitive Information +### Prevent runner with tags from picking jobs without tags + +You can configure runner to prevent it from picking jobs with tags when +runnner does not have tags assigned. This configuration setting is available +in GitLab interface when editting runner details. + +### Be careful with sensitive information If you can run a build on a runner, you can get access to any code it runs and get the token of the runner. With shared runners, this means that anyone -- cgit v1.2.3 From bf9cc351c28a349ca4c573978c869d2b90209d52 Mon Sep 17 00:00:00 2001 From: Grzegorz Bizon Date: Tue, 10 May 2016 13:19:25 +0200 Subject: Add minor corrections related to config of runner --- doc/ci/runners/README.md | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) (limited to 'doc/ci') diff --git a/doc/ci/runners/README.md b/doc/ci/runners/README.md index e449d3dc61e..b42d7a62ebc 100644 --- a/doc/ci/runners/README.md +++ b/doc/ci/runners/README.md @@ -127,9 +127,9 @@ the appropriate dependencies to run Rails test suites. ### Prevent runner with tags from picking jobs without tags -You can configure runner to prevent it from picking jobs with tags when -runnner does not have tags assigned. This configuration setting is available -in GitLab interface when editting runner details. +You can configure a runner to prevent it from picking jobs with tags when +the runnner does not have tags assigned. This setting is available on each +runner in *Project Settings* > *Runners*. ### Be careful with sensitive information -- cgit v1.2.3 From d7e714e4fd5bc45f4d68281b068068a0aa9bb6e1 Mon Sep 17 00:00:00 2001 From: Aurelio Jargas Date: Fri, 27 May 2016 21:28:45 +0000 Subject: Fix typo --- doc/ci/yaml/README.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) (limited to 'doc/ci') diff --git a/doc/ci/yaml/README.md b/doc/ci/yaml/README.md index 63866d8c71c..a3481f58c6c 100644 --- a/doc/ci/yaml/README.md +++ b/doc/ci/yaml/README.md @@ -348,7 +348,7 @@ job_name: | allow_failure | no | Allow build to fail. Failed build doesn't contribute to commit status | | when | no | Define when to run build. Can be `on_success`, `on_failure` or `always` | | dependencies | no | Define other builds that a build depends on so that you can pass artifacts between them| -| artifacts | no | Define list build artifacts | +| artifacts | no | Define list of build artifacts | | cache | no | Define list of files that should be cached between subsequent runs | | before_script | no | Override a set of commands that are executed before build | | after_script | no | Override a set of commands that are executed after build | -- cgit v1.2.3 From e6209a407e2e8abee99bb61f089e262c3a5e51e8 Mon Sep 17 00:00:00 2001 From: Kamil Trzcinski Date: Fri, 10 Jun 2016 13:26:36 +0200 Subject: Added description of artifacts:when --- doc/ci/yaml/README.md | 27 +++++++++++++++++++++++++++ 1 file changed, 27 insertions(+) (limited to 'doc/ci') diff --git a/doc/ci/yaml/README.md b/doc/ci/yaml/README.md index a3481f58c6c..39fad549a04 100644 --- a/doc/ci/yaml/README.md +++ b/doc/ci/yaml/README.md @@ -30,6 +30,7 @@ If you want a quick introduction to GitLab CI, follow our - [when](#when) - [artifacts](#artifacts) - [artifacts:name](#artifacts-name) + - [artifacts:when](#artifacts-when) - [dependencies](#dependencies) - [before_script and after_script](#before_script-and-after_script) - [Hidden jobs](#hidden-jobs) @@ -651,6 +652,32 @@ job: untracked: true ``` +#### artifacts:when + +>**Note:** +Introduced in GitLab 8.9 and GitLab Runner v1.3.0. + +`artifacts:when` is used to upload artifacts on build failure or despite the +failure. + +`artifacts:when` can be set to one of the following values: + +1. `on_success` - upload artifacts only when build succeeds. This is the default +1. `on_failure` - upload artifacts only when build fails +1. `always` - upload artifacts despite the build status + +--- + +**Example configurations** + +To upload artifacts only when build fails + +```yaml +job: + artifacts: + when: on_failure +``` + ### dependencies >**Note:** -- cgit v1.2.3 From 3714e1914b6b891274ab7d8b8db8f21dedd29e37 Mon Sep 17 00:00:00 2001 From: Kamil Trzcinski Date: Fri, 10 Jun 2016 21:25:48 +0200 Subject: Improve after review --- doc/ci/yaml/README.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) (limited to 'doc/ci') diff --git a/doc/ci/yaml/README.md b/doc/ci/yaml/README.md index 39fad549a04..0707555e393 100644 --- a/doc/ci/yaml/README.md +++ b/doc/ci/yaml/README.md @@ -670,7 +670,7 @@ failure. **Example configurations** -To upload artifacts only when build fails +To upload artifacts only when build fails. ```yaml job: -- cgit v1.2.3 From 897bc59761ad410728136308a20a184cbd9340c9 Mon Sep 17 00:00:00 2001 From: Kamil Trzcinski Date: Fri, 10 Jun 2016 13:26:36 +0200 Subject: Added description of artifacts:when --- doc/ci/yaml/README.md | 27 +++++++++++++++++++++++++++ 1 file changed, 27 insertions(+) (limited to 'doc/ci') diff --git a/doc/ci/yaml/README.md b/doc/ci/yaml/README.md index a3481f58c6c..39fad549a04 100644 --- a/doc/ci/yaml/README.md +++ b/doc/ci/yaml/README.md @@ -30,6 +30,7 @@ If you want a quick introduction to GitLab CI, follow our - [when](#when) - [artifacts](#artifacts) - [artifacts:name](#artifacts-name) + - [artifacts:when](#artifacts-when) - [dependencies](#dependencies) - [before_script and after_script](#before_script-and-after_script) - [Hidden jobs](#hidden-jobs) @@ -651,6 +652,32 @@ job: untracked: true ``` +#### artifacts:when + +>**Note:** +Introduced in GitLab 8.9 and GitLab Runner v1.3.0. + +`artifacts:when` is used to upload artifacts on build failure or despite the +failure. + +`artifacts:when` can be set to one of the following values: + +1. `on_success` - upload artifacts only when build succeeds. This is the default +1. `on_failure` - upload artifacts only when build fails +1. `always` - upload artifacts despite the build status + +--- + +**Example configurations** + +To upload artifacts only when build fails + +```yaml +job: + artifacts: + when: on_failure +``` + ### dependencies >**Note:** -- cgit v1.2.3 From d23b91b0d9b8db16801872c49a1fb1d3be3a7144 Mon Sep 17 00:00:00 2001 From: Kamil Trzcinski Date: Fri, 10 Jun 2016 21:25:48 +0200 Subject: Improve after review --- doc/ci/yaml/README.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) (limited to 'doc/ci') diff --git a/doc/ci/yaml/README.md b/doc/ci/yaml/README.md index 39fad549a04..0707555e393 100644 --- a/doc/ci/yaml/README.md +++ b/doc/ci/yaml/README.md @@ -670,7 +670,7 @@ failure. **Example configurations** -To upload artifacts only when build fails +To upload artifacts only when build fails. ```yaml job: -- cgit v1.2.3 From cf9c5b54c68218281ac066cac5d3c002fb72153a Mon Sep 17 00:00:00 2001 From: Kamil Trzcinski Date: Fri, 10 Jun 2016 21:53:57 +0200 Subject: Added documentation to artifacts expire --- doc/ci/yaml/README.md | 35 +++++++++++++++++++++++++++++++++++ 1 file changed, 35 insertions(+) (limited to 'doc/ci') diff --git a/doc/ci/yaml/README.md b/doc/ci/yaml/README.md index 0707555e393..d71ce6d6b13 100644 --- a/doc/ci/yaml/README.md +++ b/doc/ci/yaml/README.md @@ -31,6 +31,7 @@ If you want a quick introduction to GitLab CI, follow our - [artifacts](#artifacts) - [artifacts:name](#artifacts-name) - [artifacts:when](#artifacts-when) + - [artifacts:expire_in](#artifacts-expire_in) - [dependencies](#dependencies) - [before_script and after_script](#before_script-and-after_script) - [Hidden jobs](#hidden-jobs) @@ -678,6 +679,40 @@ job: when: on_failure ``` +#### artifacts:expire_in + +>**Note:** +Introduced in GitLab 8.9 and GitLab Runner v1.3.0. + +`artifacts:expire_in` is used to remove uploaded artifacts after specified time. +By default artifacts are stored on GitLab forver. +`expire_in` allows to specify after what time the artifacts should be removed. +The artifacts will expire counting from the moment when they are uploaded and stored on GitLab. + +After artifacts uploading you can use the **Keep** button on build page to keep the artifacts forever. + +Artifacts are removed every hour, but they are not accessible after expire date. + +The value of `expire_in` is a elapsed time. The example of parsable values: +- '3 mins 4 sec' +- '2 hrs 20 min' +- '2h20min' +- '6 mos 1 day' +- '47 yrs 6 mos and 4d' +- '3 weeks and 2 days' + +--- + +**Example configurations** + +To expire artifacts after 1 week from the moment that they are uploaded: + +```yaml +job: + artifacts: + expire_in: 1 week +``` + ### dependencies >**Note:** -- cgit v1.2.3 From 4209212bb80c8d92955bcc71fa8e6973b44cf59a Mon Sep 17 00:00:00 2001 From: Mark Pundsack Date: Tue, 7 Jun 2016 21:33:54 -0700 Subject: Add docker bind-mount as an option --- doc/ci/docker/using_docker_build.md | 78 ++++++++++++++++++++++++++++++++----- 1 file changed, 68 insertions(+), 10 deletions(-) (limited to 'doc/ci') diff --git a/doc/ci/docker/using_docker_build.md b/doc/ci/docker/using_docker_build.md index ca52a483a59..f98b2860e21 100644 --- a/doc/ci/docker/using_docker_build.md +++ b/doc/ci/docker/using_docker_build.md @@ -4,14 +4,14 @@ GitLab CI allows you to use Docker Engine to build and test docker-based project **This also allows to you to use `docker-compose` and other docker-enabled tools.** -This is one of new trends in Continuous Integration/Deployment to: +This is one of the new trends in Continuous Integration/Deployment to: -1. create application image, -1. run test against created image, -1. push image to remote registry, -1. deploy server from pushed image +1. create an application image, +1. run tests against the created image, +1. push image to a remote registry, +1. deploy server from the pushed image -It's also useful in case when your application already has the `Dockerfile` that can be used to create and test image: +It's also useful when your application already has the `Dockerfile` that can be used to create and test an image: ```bash $ docker build -t my-image dockerfiles/ $ docker run my-docker-image /script/to/run/tests @@ -19,10 +19,7 @@ $ docker tag my-image my-registry:5000/my-image $ docker push my-registry:5000/my-image ``` -However, this requires special configuration of GitLab Runner to enable `docker` support during build. -**This requires running GitLab Runner in privileged mode which can be harmful when untrusted code is run.** - -There are two methods to enable the use of `docker build` and `docker run` during build. +However, this requires special configuration of GitLab Runner to enable `docker` support during builds. There are three methods to enable the use of `docker build` and `docker run` during builds. ## 1. Use shell executor @@ -150,5 +147,66 @@ In order to do that follow the steps: An example project using this approach can be found here: https://gitlab.com/gitlab-examples/docker. +## 3. Bind Docker socket + +The third approach is to bind-mount `/var/run/docker.sock` into the container so that docker is available in the context of that image. + +In order to do that follow the steps: + +1. Install [GitLab Runner](https://gitlab.com/gitlab-org/gitlab-ci-multi-runner/#installation). + +1. Register GitLab Runner from the command line to use `docker` and `privileged` + mode: + + ```bash + sudo gitlab-runner register -n \ + --url https://gitlab.com/ci \ + --token RUNNER_TOKEN \ + --executor docker \ + --description "My Docker Runner" \ + --docker-image "docker:latest" \ + --docker-volumes /var/run/docker.sock:/var/run/docker.sock + ``` + + The above command will register a new Runner to use the special + `docker:latest` image which is provided by Docker. **Notice that it's using + the Docker daemon of the runner itself, and any containers spawned by docker commands will be siblings of the runner rather than children of the runner.** This may have complications and limitations that are unsuitable for your workflow. + + The above command will create a `config.toml` entry similar to this: + + ``` + [[runners]] + url = "https://gitlab.com/ci" + token = TOKEN + executor = "docker" + [runners.docker] + tls_verify = false + image = "docker:latest" + privileged = false + disable_cache = false + volumes = ["/usr/local/bin/docker:/usr/bin/docker", "/cache"] + [runners.cache] + Insecure = false + ``` + +1. You can now use `docker` from build script (note that you don't need to include the `docker:dind` service as in the option above): + + ```yaml + image: docker:latest + + before_script: + - docker info + + build: + stage: build + script: + - docker build -t my-docker-image . + - docker run my-docker-image /script/to/run/tests + ``` + +1. However, by sharing the docker daemon, you are effectively disabling all + the security mechanisms of containers and exposing your host to privilege + escalation which can lead to container breakout. + [docker-in-docker]: https://blog.docker.com/2013/09/docker-can-now-run-within-docker/ [docker-cap]: https://docs.docker.com/engine/reference/run/#runtime-privilege-and-linux-capabilities -- cgit v1.2.3 From d7664c7223cbd9e91e21beaf6ceb9ea7c2f294d8 Mon Sep 17 00:00:00 2001 From: Mark Pundsack Date: Tue, 7 Jun 2016 21:34:30 -0700 Subject: Add example using GitLab Container Registry --- doc/ci/docker/using_docker_build.md | 72 +++++++++++++++++++++++++++++++++++++ 1 file changed, 72 insertions(+) (limited to 'doc/ci') diff --git a/doc/ci/docker/using_docker_build.md b/doc/ci/docker/using_docker_build.md index f98b2860e21..fe2c5207cd1 100644 --- a/doc/ci/docker/using_docker_build.md +++ b/doc/ci/docker/using_docker_build.md @@ -208,5 +208,77 @@ In order to do that follow the steps: the security mechanisms of containers and exposing your host to privilege escalation which can lead to container breakout. +## Using the GitLab Container Registry + +Once you've built a Docker image, you can push it up to the built-in [GitLab Container Registry](../../container_registry/README.md). + +``` + build: + stage: build + script: + - docker login -u gitlab-ci-token -p $CI_BUILD_TOKEN registry.example.com + - docker build -t registry.example.com/group/project:latest . + - docker push registry.example.com/group/project:latest +``` + +Here's a more elaborate example that splits up the tasks into 4 stages, +including two tests that run in parallel. The build is stored in the container +registry and used by subsequent stages, downloading the image +when needed. Changes to `master` also get tagged as `latest` and deployed using +an application-specific deploy script: + +```yaml +image: docker:git +services: +- docker:dind + +stages: +- build +- test +- release +- deploy + +variables: + CONTAINER_TEST_IMAGE: registry.example.com/my-group/my-project:$CI_BUILD_REF_NAME + CONTAINER_RELEASE_IMAGE: registry.example.com/my-group/my-project:latest + +before_script: + - docker login -u gitlab-ci-token -p $CI_BUILD_TOKEN registry.example.com + +build: + stage: build + script: + - docker build --pull -t $CONTAINER_TEST_IMAGE . + - docker push $CONTAINER_TEST_IMAGE + +test1: + stage: test + script: + - docker pull $CONTAINER_TEST_IMAGE + - docker run $CONTAINER_TEST_IMAGE /script/to/run/tests + +test2: + stage: test + script: + - docker pull $CONTAINER_TEST_IMAGE + - docker run $CONTAINER_TEST_IMAGE /script/to/run/another/test + +release-image: + stage: release + script: + - docker pull $CONTAINER_TEST_IMAGE + - docker tag $CONTAINER_TEST_IMAGE $CONTAINER_RELEASE_IMAGE + - docker push $CONTAINER_RELEASE_IMAGE + only: + - master + +deploy: + stage: deploy + script: + - ./deploy.sh + only: + - master +``` + [docker-in-docker]: https://blog.docker.com/2013/09/docker-can-now-run-within-docker/ [docker-cap]: https://docs.docker.com/engine/reference/run/#runtime-privilege-and-linux-capabilities -- cgit v1.2.3 From 6841e76b45b44da9f749538dbae2bb1fc63d8ee4 Mon Sep 17 00:00:00 2001 From: Mark Pundsack Date: Tue, 7 Jun 2016 22:09:15 -0700 Subject: Add notes --- doc/ci/docker/using_docker_build.md | 6 ++++++ 1 file changed, 6 insertions(+) (limited to 'doc/ci') diff --git a/doc/ci/docker/using_docker_build.md b/doc/ci/docker/using_docker_build.md index fe2c5207cd1..a5f37366265 100644 --- a/doc/ci/docker/using_docker_build.md +++ b/doc/ci/docker/using_docker_build.md @@ -280,5 +280,11 @@ deploy: - master ``` +Notes: +1. You must log in to the container registry before running commands. Putting this in `before_script` will run it before each build job. +1. Using `docker build --pull` makes sure that Docker fetches any changes to base images before building just in case your cache is stale. It takes slightly longer, but means you don’t get stuck without security patches to base images. +1. Doing an explicit `docker pull` before each `docker run` makes sure to fetch the latest image that was just built. This is especially important if you are using multiple runners that cache images locally. Using the git SHA in your image tag makes this less necessary since each build will be unique and you shouldn't ever have a stale image, but it's still possible if you re-build a given commit after a dependency has changed. +1. You don't want to build directly to `latest` in case there are multiple builds happening simultaneously. + [docker-in-docker]: https://blog.docker.com/2013/09/docker-can-now-run-within-docker/ [docker-cap]: https://docs.docker.com/engine/reference/run/#runtime-privilege-and-linux-capabilities -- cgit v1.2.3 From d9cbe019866843132225d440754a20da0e937d00 Mon Sep 17 00:00:00 2001 From: Mark Pundsack Date: Tue, 7 Jun 2016 22:25:33 -0700 Subject: Moar commas --- doc/ci/docker/using_docker_build.md | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) (limited to 'doc/ci') diff --git a/doc/ci/docker/using_docker_build.md b/doc/ci/docker/using_docker_build.md index a5f37366265..4620eeac2b6 100644 --- a/doc/ci/docker/using_docker_build.md +++ b/doc/ci/docker/using_docker_build.md @@ -76,7 +76,7 @@ The second approach is to use the special Docker image with all tools installed (`docker` and `docker-compose`) and run the build script in context of that image in privileged mode. -In order to do that follow the steps: +In order to do that, follow the steps: 1. Install [GitLab Runner](https://gitlab.com/gitlab-org/gitlab-ci-multi-runner/#installation). @@ -151,7 +151,7 @@ An example project using this approach can be found here: https://gitlab.com/git The third approach is to bind-mount `/var/run/docker.sock` into the container so that docker is available in the context of that image. -In order to do that follow the steps: +In order to do that, follow the steps: 1. Install [GitLab Runner](https://gitlab.com/gitlab-org/gitlab-ci-multi-runner/#installation). -- cgit v1.2.3 From 84128441081869ff2bd260a92a7c0b43d68ca415 Mon Sep 17 00:00:00 2001 From: Mark Pundsack Date: Tue, 7 Jun 2016 22:26:59 -0700 Subject: Fix instructions --- doc/ci/docker/using_docker_build.md | 3 +-- 1 file changed, 1 insertion(+), 2 deletions(-) (limited to 'doc/ci') diff --git a/doc/ci/docker/using_docker_build.md b/doc/ci/docker/using_docker_build.md index 4620eeac2b6..62e48a6d8d6 100644 --- a/doc/ci/docker/using_docker_build.md +++ b/doc/ci/docker/using_docker_build.md @@ -155,8 +155,7 @@ In order to do that, follow the steps: 1. Install [GitLab Runner](https://gitlab.com/gitlab-org/gitlab-ci-multi-runner/#installation). -1. Register GitLab Runner from the command line to use `docker` and `privileged` - mode: +1. Register GitLab Runner from the command line to use `docker` and share `/var/run/docker.sock`: ```bash sudo gitlab-runner register -n \ -- cgit v1.2.3 From 9b30f26b988f69995d1f548f79020c23dfe1a9ea Mon Sep 17 00:00:00 2001 From: Mark Pundsack Date: Tue, 7 Jun 2016 23:39:38 -0700 Subject: Fix runner CLI instructions --- doc/ci/docker/using_docker_build.md | 8 ++++---- 1 file changed, 4 insertions(+), 4 deletions(-) (limited to 'doc/ci') diff --git a/doc/ci/docker/using_docker_build.md b/doc/ci/docker/using_docker_build.md index 62e48a6d8d6..3af4afbbefe 100644 --- a/doc/ci/docker/using_docker_build.md +++ b/doc/ci/docker/using_docker_build.md @@ -84,9 +84,9 @@ In order to do that, follow the steps: mode: ```bash - sudo gitlab-runner register -n \ + sudo gitlab-ci-multi-runner register -n \ --url https://gitlab.com/ci \ - --token RUNNER_TOKEN \ + --registration-token REGISTRATION_TOKEN \ --executor docker \ --description "My Docker Runner" \ --docker-image "docker:latest" \ @@ -158,9 +158,9 @@ In order to do that, follow the steps: 1. Register GitLab Runner from the command line to use `docker` and share `/var/run/docker.sock`: ```bash - sudo gitlab-runner register -n \ + sudo gitlab-ci-multi-runner register -n \ --url https://gitlab.com/ci \ - --token RUNNER_TOKEN \ + --registration-token REGISTRATION_TOKEN \ --executor docker \ --description "My Docker Runner" \ --docker-image "docker:latest" \ -- cgit v1.2.3 From 6ca1370c92dcf074af73562fb0fd613c8af45ce1 Mon Sep 17 00:00:00 2001 From: Mark Pundsack Date: Tue, 7 Jun 2016 23:50:26 -0700 Subject: Fix more instructions --- doc/ci/docker/using_docker_build.md | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) (limited to 'doc/ci') diff --git a/doc/ci/docker/using_docker_build.md b/doc/ci/docker/using_docker_build.md index 3af4afbbefe..aae37010508 100644 --- a/doc/ci/docker/using_docker_build.md +++ b/doc/ci/docker/using_docker_build.md @@ -24,16 +24,16 @@ However, this requires special configuration of GitLab Runner to enable `docker` ## 1. Use shell executor The simplest approach is to install GitLab Runner in `shell` execution mode. -GitLab Runner then executes build scripts as `gitlab-runner` user. +GitLab Runner then executes build scripts as the `gitlab-runner` user. 1. Install [GitLab Runner](https://gitlab.com/gitlab-org/gitlab-ci-multi-runner/#installation). 1. During GitLab Runner installation select `shell` as method of executing build scripts or use command: ```bash - $ sudo gitlab-runner register -n \ + $ sudo gitlab-ci-multi-runner register -n \ --url https://gitlab.com/ci \ - --token RUNNER_TOKEN \ + --registration-token REGISTRATION_TOKEN \ --executor shell --description "My Runner" ``` -- cgit v1.2.3 From db656a3987131816d47897b2424821b19ca147b0 Mon Sep 17 00:00:00 2001 From: Mark Pundsack Date: Wed, 8 Jun 2016 00:28:23 -0700 Subject: Fix more references to old gitlab-runner --- doc/ci/docker/using_docker_images.md | 2 +- doc/ci/examples/php.md | 4 ++-- doc/ci/runners/README.md | 6 +++--- 3 files changed, 6 insertions(+), 6 deletions(-) (limited to 'doc/ci') diff --git a/doc/ci/docker/using_docker_images.md b/doc/ci/docker/using_docker_images.md index 56ac2195c49..a849905ac6b 100644 --- a/doc/ci/docker/using_docker_images.md +++ b/doc/ci/docker/using_docker_images.md @@ -23,7 +23,7 @@ To use GitLab Runner with docker you need to register a new runner to use the `docker` executor: ```bash -gitlab-runner register \ +gitlab-ci-multi-runner register \ --url "https://gitlab.com/" \ --registration-token "PROJECT_REGISTRATION_TOKEN" \ --description "docker-ruby-2.1" \ diff --git a/doc/ci/examples/php.md b/doc/ci/examples/php.md index 26953014502..17e1c64bb8a 100644 --- a/doc/ci/examples/php.md +++ b/doc/ci/examples/php.md @@ -263,10 +263,10 @@ terminal execute: ```bash # Check using docker executor -gitlab-runner exec docker test:app +gitlab-ci-multi-runner exec docker test:app # Check using shell executor -gitlab-runner exec shell test:app +gitlab-ci-multi-runner exec shell test:app ``` ## Example project diff --git a/doc/ci/runners/README.md b/doc/ci/runners/README.md index b42d7a62ebc..400784da617 100644 --- a/doc/ci/runners/README.md +++ b/doc/ci/runners/README.md @@ -63,10 +63,10 @@ instance. Now simply register the runner as any runner: ``` -sudo gitlab-runner register +sudo gitlab-ci-multi-runner register ``` -Shared runners are enabled by default as of GitLab 8.2, but can be disabled with the +Shared runners are enabled by default as of GitLab 8.2, but can be disabled with the `DISABLE SHARED RUNNERS` button. Previous versions of GitLab defaulted shared runners to disabled. @@ -93,7 +93,7 @@ setup a specific runner for this project. To register the runner, run the command below and follow instructions: ``` -sudo gitlab-runner register +sudo gitlab-ci-multi-runner register ``` ### Making an existing Shared Runner Specific -- cgit v1.2.3 From e97af053eb24391df926cb7f7ca20d67a4ff03d0 Mon Sep 17 00:00:00 2001 From: Mark Pundsack Date: Wed, 8 Jun 2016 11:10:06 -0700 Subject: Fix docker volume --- doc/ci/docker/using_docker_build.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) (limited to 'doc/ci') diff --git a/doc/ci/docker/using_docker_build.md b/doc/ci/docker/using_docker_build.md index aae37010508..5df1fdd84c7 100644 --- a/doc/ci/docker/using_docker_build.md +++ b/doc/ci/docker/using_docker_build.md @@ -183,7 +183,7 @@ In order to do that, follow the steps: image = "docker:latest" privileged = false disable_cache = false - volumes = ["/usr/local/bin/docker:/usr/bin/docker", "/cache"] + volumes = ["/var/run/docker.sock", "/cache"] [runners.cache] Insecure = false ``` -- cgit v1.2.3 From 46114eddf0a2fc07f932fe45948a48896abbeb78 Mon Sep 17 00:00:00 2001 From: Mark Pundsack Date: Wed, 8 Jun 2016 11:39:31 -0700 Subject: Add more pros and cons for each docker approach --- doc/ci/docker/using_docker_build.md | 29 +++++++++++++++++++---------- 1 file changed, 19 insertions(+), 10 deletions(-) (limited to 'doc/ci') diff --git a/doc/ci/docker/using_docker_build.md b/doc/ci/docker/using_docker_build.md index 5df1fdd84c7..17ba953ca73 100644 --- a/doc/ci/docker/using_docker_build.md +++ b/doc/ci/docker/using_docker_build.md @@ -19,7 +19,7 @@ $ docker tag my-image my-registry:5000/my-image $ docker push my-registry:5000/my-image ``` -However, this requires special configuration of GitLab Runner to enable `docker` support during builds. There are three methods to enable the use of `docker build` and `docker run` during builds. +However, this requires special configuration of GitLab Runner to enable `docker` support during builds. There are three methods to enable the use of `docker build` and `docker run` during builds; each with their own tradeoffs. ## 1. Use shell executor @@ -67,7 +67,7 @@ GitLab Runner then executes build scripts as the `gitlab-runner` user. 5. You can now use `docker` command and install `docker-compose` if needed. -6. However, by adding `gitlab-runner` to `docker` group you are effectively granting `gitlab-runner` full root permissions. +However, by adding `gitlab-runner` to `docker` group you are effectively granting `gitlab-runner` full root permissions. For more information please checkout [On Docker security: `docker` group considered harmful](https://www.andreas-jung.com/contents/on-docker-security-docker-group-considered-harmful). ## 2. Use docker-in-docker executor @@ -138,12 +138,16 @@ In order to do that, follow the steps: - docker run my-docker-image /script/to/run/tests ``` -1. However, by enabling `--docker-privileged` you are effectively disabling all - the security mechanisms of containers and exposing your host to privilege - escalation which can lead to container breakout. +However, by enabling `--docker-privileged` you are effectively disabling all +the security mechanisms of containers and exposing your host to privilege +escalation which can lead to container breakout. For more information, check out the official Docker documentation on +[Runtime privilege and Linux capabilities][docker-cap]. - For more information, check out the official Docker documentation on - [Runtime privilege and Linux capabilities][docker-cap]. +Using docker-in-docker, each build is in a clean environment without the past +history. Concurrent builds work fine because every build get it's own instance of docker engine so they won't conflict with each other. But this also means builds can be slower because there's no caching of layers. + +By default `docker:dind` uses ``--storage-driver vfs` which is the slowest form +offered. An example project using this approach can be found here: https://gitlab.com/gitlab-examples/docker. @@ -203,9 +207,14 @@ In order to do that, follow the steps: - docker run my-docker-image /script/to/run/tests ``` -1. However, by sharing the docker daemon, you are effectively disabling all - the security mechanisms of containers and exposing your host to privilege - escalation which can lead to container breakout. +However, by sharing the docker daemon, you are effectively disabling all +the security mechanisms of containers and exposing your host to privilege +escalation which can lead to container breakout. For example, if a project +ran `docker rm -f $(docker ps -a -q)` it would remove the GitLab Runner +containers. + +Also, concurrent builds may not work; if your tests +create containers with specific names, they may conflict with each other. ## Using the GitLab Container Registry -- cgit v1.2.3 From 1c02ef9c144f3a8d40e31a21d82b5628e72d48e6 Mon Sep 17 00:00:00 2001 From: Mark Pundsack Date: Wed, 8 Jun 2016 12:00:17 -0700 Subject: Drop some 'however's --- doc/ci/docker/using_docker_build.md | 8 ++++---- 1 file changed, 4 insertions(+), 4 deletions(-) (limited to 'doc/ci') diff --git a/doc/ci/docker/using_docker_build.md b/doc/ci/docker/using_docker_build.md index 17ba953ca73..cc820d81144 100644 --- a/doc/ci/docker/using_docker_build.md +++ b/doc/ci/docker/using_docker_build.md @@ -19,7 +19,7 @@ $ docker tag my-image my-registry:5000/my-image $ docker push my-registry:5000/my-image ``` -However, this requires special configuration of GitLab Runner to enable `docker` support during builds. There are three methods to enable the use of `docker build` and `docker run` during builds; each with their own tradeoffs. +This requires special configuration of GitLab Runner to enable `docker` support during builds. There are three methods to enable the use of `docker build` and `docker run` during builds; each with their own tradeoffs. ## 1. Use shell executor @@ -67,7 +67,7 @@ GitLab Runner then executes build scripts as the `gitlab-runner` user. 5. You can now use `docker` command and install `docker-compose` if needed. -However, by adding `gitlab-runner` to `docker` group you are effectively granting `gitlab-runner` full root permissions. +By adding `gitlab-runner` to `docker` group you are effectively granting `gitlab-runner` full root permissions. For more information please checkout [On Docker security: `docker` group considered harmful](https://www.andreas-jung.com/contents/on-docker-security-docker-group-considered-harmful). ## 2. Use docker-in-docker executor @@ -138,7 +138,7 @@ In order to do that, follow the steps: - docker run my-docker-image /script/to/run/tests ``` -However, by enabling `--docker-privileged` you are effectively disabling all +By enabling `--docker-privileged` you are effectively disabling all the security mechanisms of containers and exposing your host to privilege escalation which can lead to container breakout. For more information, check out the official Docker documentation on [Runtime privilege and Linux capabilities][docker-cap]. @@ -207,7 +207,7 @@ In order to do that, follow the steps: - docker run my-docker-image /script/to/run/tests ``` -However, by sharing the docker daemon, you are effectively disabling all +By sharing the docker daemon, you are effectively disabling all the security mechanisms of containers and exposing your host to privilege escalation which can lead to container breakout. For example, if a project ran `docker rm -f $(docker ps -a -q)` it would remove the GitLab Runner -- cgit v1.2.3 From b393478f63ad2f4381996dc08111fc3393bf762e Mon Sep 17 00:00:00 2001 From: Mark Pundsack Date: Wed, 8 Jun 2016 12:11:44 -0700 Subject: Refactor notes --- doc/ci/docker/using_docker_build.md | 20 ++++++++++++++------ 1 file changed, 14 insertions(+), 6 deletions(-) (limited to 'doc/ci') diff --git a/doc/ci/docker/using_docker_build.md b/doc/ci/docker/using_docker_build.md index cc820d81144..5af6d36e83e 100644 --- a/doc/ci/docker/using_docker_build.md +++ b/doc/ci/docker/using_docker_build.md @@ -67,7 +67,8 @@ GitLab Runner then executes build scripts as the `gitlab-runner` user. 5. You can now use `docker` command and install `docker-compose` if needed. -By adding `gitlab-runner` to `docker` group you are effectively granting `gitlab-runner` full root permissions. +Notes: +* By adding `gitlab-runner` to `docker` group you are effectively granting `gitlab-runner` full root permissions. For more information please checkout [On Docker security: `docker` group considered harmful](https://www.andreas-jung.com/contents/on-docker-security-docker-group-considered-harmful). ## 2. Use docker-in-docker executor @@ -138,15 +139,16 @@ In order to do that, follow the steps: - docker run my-docker-image /script/to/run/tests ``` -By enabling `--docker-privileged` you are effectively disabling all +Notes: +* By enabling `--docker-privileged` you are effectively disabling all the security mechanisms of containers and exposing your host to privilege escalation which can lead to container breakout. For more information, check out the official Docker documentation on [Runtime privilege and Linux capabilities][docker-cap]. -Using docker-in-docker, each build is in a clean environment without the past +* Using docker-in-docker, each build is in a clean environment without the past history. Concurrent builds work fine because every build get it's own instance of docker engine so they won't conflict with each other. But this also means builds can be slower because there's no caching of layers. -By default `docker:dind` uses ``--storage-driver vfs` which is the slowest form +* By default, `docker:dind` uses ``--storage-driver vfs` which is the slowest form offered. An example project using this approach can be found here: https://gitlab.com/gitlab-examples/docker. @@ -207,15 +209,21 @@ In order to do that, follow the steps: - docker run my-docker-image /script/to/run/tests ``` -By sharing the docker daemon, you are effectively disabling all +Notes: +* By sharing the docker daemon, you are effectively disabling all the security mechanisms of containers and exposing your host to privilege escalation which can lead to container breakout. For example, if a project ran `docker rm -f $(docker ps -a -q)` it would remove the GitLab Runner containers. -Also, concurrent builds may not work; if your tests +* Concurrent builds may not work; if your tests create containers with specific names, they may conflict with each other. +* Sharing files and directories from the source repo into containers may not +work as expected since volume mounting is done in the context of the host +machine, not the build container. +e.g. `docker run --rm -t -i -v $(pwd)/src:/home/app/src test-image:latest run_app_tests` + ## Using the GitLab Container Registry Once you've built a Docker image, you can push it up to the built-in [GitLab Container Registry](../../container_registry/README.md). -- cgit v1.2.3 From b0cbeb18d1864ab36fb17c69d963321d745924fa Mon Sep 17 00:00:00 2001 From: Mark Pundsack Date: Wed, 8 Jun 2016 14:11:15 -0700 Subject: Remove unnecessary message --- doc/ci/docker/using_docker_build.md | 4 ---- 1 file changed, 4 deletions(-) (limited to 'doc/ci') diff --git a/doc/ci/docker/using_docker_build.md b/doc/ci/docker/using_docker_build.md index 5af6d36e83e..c44b1d7a0cc 100644 --- a/doc/ci/docker/using_docker_build.md +++ b/doc/ci/docker/using_docker_build.md @@ -117,10 +117,6 @@ In order to do that, follow the steps: Insecure = false ``` - If you want to use the Shared Runners available on your GitLab CE/EE - installation in order to build Docker images, then make sure that your - Shared Runners configuration has the `privileged` mode set to `true`. - 1. You can now use `docker` from build script: ```yaml -- cgit v1.2.3 From 6f834ecaa94a1da230c933c981b33634d937d8dd Mon Sep 17 00:00:00 2001 From: Mark Pundsack Date: Wed, 8 Jun 2016 14:17:03 -0700 Subject: Reformat notes --- doc/ci/docker/using_docker_build.md | 8 ++++---- 1 file changed, 4 insertions(+), 4 deletions(-) (limited to 'doc/ci') diff --git a/doc/ci/docker/using_docker_build.md b/doc/ci/docker/using_docker_build.md index c44b1d7a0cc..697b9f10163 100644 --- a/doc/ci/docker/using_docker_build.md +++ b/doc/ci/docker/using_docker_build.md @@ -67,7 +67,7 @@ GitLab Runner then executes build scripts as the `gitlab-runner` user. 5. You can now use `docker` command and install `docker-compose` if needed. -Notes: +### Notes * By adding `gitlab-runner` to `docker` group you are effectively granting `gitlab-runner` full root permissions. For more information please checkout [On Docker security: `docker` group considered harmful](https://www.andreas-jung.com/contents/on-docker-security-docker-group-considered-harmful). @@ -135,7 +135,7 @@ In order to do that, follow the steps: - docker run my-docker-image /script/to/run/tests ``` -Notes: +### Notes * By enabling `--docker-privileged` you are effectively disabling all the security mechanisms of containers and exposing your host to privilege escalation which can lead to container breakout. For more information, check out the official Docker documentation on @@ -205,7 +205,7 @@ In order to do that, follow the steps: - docker run my-docker-image /script/to/run/tests ``` -Notes: +### Notes * By sharing the docker daemon, you are effectively disabling all the security mechanisms of containers and exposing your host to privilege escalation which can lead to container breakout. For example, if a project @@ -292,7 +292,7 @@ deploy: - master ``` -Notes: +### Notes 1. You must log in to the container registry before running commands. Putting this in `before_script` will run it before each build job. 1. Using `docker build --pull` makes sure that Docker fetches any changes to base images before building just in case your cache is stale. It takes slightly longer, but means you don’t get stuck without security patches to base images. 1. Doing an explicit `docker pull` before each `docker run` makes sure to fetch the latest image that was just built. This is especially important if you are using multiple runners that cache images locally. Using the git SHA in your image tag makes this less necessary since each build will be unique and you shouldn't ever have a stale image, but it's still possible if you re-build a given commit after a dependency has changed. -- cgit v1.2.3 From 35ce04ef2e02e5b176c57567f2ddf82871af7639 Mon Sep 17 00:00:00 2001 From: Mark Pundsack Date: Wed, 8 Jun 2016 14:40:56 -0700 Subject: Move registry CI example to CI docs --- doc/ci/docker/using_docker_build.md | 22 ++++++++++++++++++---- 1 file changed, 18 insertions(+), 4 deletions(-) (limited to 'doc/ci') diff --git a/doc/ci/docker/using_docker_build.md b/doc/ci/docker/using_docker_build.md index 697b9f10163..33b1624d00b 100644 --- a/doc/ci/docker/using_docker_build.md +++ b/doc/ci/docker/using_docker_build.md @@ -73,7 +73,8 @@ For more information please checkout [On Docker security: `docker` group conside ## 2. Use docker-in-docker executor -The second approach is to use the special Docker image with all tools installed +The second approach is to use the special docker-in-docker (dind) +[Docker image](https://hub.docker.com/_/docker/) with all tools installed (`docker` and `docker-compose`) and run the build script in context of that image in privileged mode. @@ -222,10 +223,18 @@ e.g. `docker run --rm -t -i -v $(pwd)/src:/home/app/src test-image:latest run_ap ## Using the GitLab Container Registry -Once you've built a Docker image, you can push it up to the built-in [GitLab Container Registry](../../container_registry/README.md). +> **Note:** +This feature requires GitLab 8.8 and GitLab Runner 1.2. -``` +Once you've built a Docker image, you can push it up to the built-in [GitLab Container Registry](../../container_registry/README.md). For example, if you're using +docker-in-docker on your runners, this is how your `.gitlab-ci.yml` could look: + + +```yaml build: + image: docker:git + services: + - docker:dind stage: build script: - docker login -u gitlab-ci-token -p $CI_BUILD_TOKEN registry.example.com @@ -233,7 +242,12 @@ Once you've built a Docker image, you can push it up to the built-in [GitLab Con - docker push registry.example.com/group/project:latest ``` -Here's a more elaborate example that splits up the tasks into 4 stages, +You have to use the credentials of the special `gitlab-ci-token` user with its +password stored in `$CI_BUILD_TOKEN` in order to push to the Registry connected +to your project. This allows you to automate building and deployment of your +Docker images. + +Here's a more elaborate example that splits up the tasks into 4 pipeline stages, including two tests that run in parallel. The build is stored in the container registry and used by subsequent stages, downloading the image when needed. Changes to `master` also get tagged as `latest` and deployed using -- cgit v1.2.3 From a7caea9e3e6b624ada8d3dbabf13c2f9ad79b463 Mon Sep 17 00:00:00 2001 From: Mark Pundsack Date: Wed, 8 Jun 2016 15:41:27 -0700 Subject: Use docker:latest --- doc/ci/docker/using_docker_build.md | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) (limited to 'doc/ci') diff --git a/doc/ci/docker/using_docker_build.md b/doc/ci/docker/using_docker_build.md index 33b1624d00b..d5bc1d7406e 100644 --- a/doc/ci/docker/using_docker_build.md +++ b/doc/ci/docker/using_docker_build.md @@ -232,7 +232,7 @@ docker-in-docker on your runners, this is how your `.gitlab-ci.yml` could look: ```yaml build: - image: docker:git + image: docker:latest services: - docker:dind stage: build @@ -254,7 +254,7 @@ when needed. Changes to `master` also get tagged as `latest` and deployed using an application-specific deploy script: ```yaml -image: docker:git +image: docker:latest services: - docker:dind -- cgit v1.2.3 From f95791d4118e7a1ad85ab0f287784c5639182560 Mon Sep 17 00:00:00 2001 From: Mark Pundsack Date: Mon, 13 Jun 2016 22:32:01 -0700 Subject: Make Achilleas' suggested changes --- doc/ci/docker/using_docker_build.md | 56 ++++++++++++++++++------------------- 1 file changed, 28 insertions(+), 28 deletions(-) (limited to 'doc/ci') diff --git a/doc/ci/docker/using_docker_build.md b/doc/ci/docker/using_docker_build.md index d5bc1d7406e..77291597659 100644 --- a/doc/ci/docker/using_docker_build.md +++ b/doc/ci/docker/using_docker_build.md @@ -4,12 +4,12 @@ GitLab CI allows you to use Docker Engine to build and test docker-based project **This also allows to you to use `docker-compose` and other docker-enabled tools.** -This is one of the new trends in Continuous Integration/Deployment to: +One of the new trends in Continuous Integration/Deployment is to: 1. create an application image, 1. run tests against the created image, -1. push image to a remote registry, -1. deploy server from the pushed image +1. push image to a remote registry, and +1. deploy to a server from the pushed image. It's also useful when your application already has the `Dockerfile` that can be used to create and test an image: ```bash @@ -19,9 +19,13 @@ $ docker tag my-image my-registry:5000/my-image $ docker push my-registry:5000/my-image ``` -This requires special configuration of GitLab Runner to enable `docker` support during builds. There are three methods to enable the use of `docker build` and `docker run` during builds; each with their own tradeoffs. +This requires special configuration of GitLab Runner to enable `docker` support during builds. -## 1. Use shell executor +## Runner Configuration + +There are three methods to enable the use of `docker build` and `docker run` during builds; each with their own tradeoffs. + +### Use shell executor The simplest approach is to install GitLab Runner in `shell` execution mode. GitLab Runner then executes build scripts as the `gitlab-runner` user. @@ -67,11 +71,11 @@ GitLab Runner then executes build scripts as the `gitlab-runner` user. 5. You can now use `docker` command and install `docker-compose` if needed. -### Notes +> **Note:** * By adding `gitlab-runner` to `docker` group you are effectively granting `gitlab-runner` full root permissions. -For more information please checkout [On Docker security: `docker` group considered harmful](https://www.andreas-jung.com/contents/on-docker-security-docker-group-considered-harmful). +For more information please check out [On Docker security: `docker` group considered harmful](https://www.andreas-jung.com/contents/on-docker-security-docker-group-considered-harmful). -## 2. Use docker-in-docker executor +### Use docker-in-docker executor The second approach is to use the special docker-in-docker (dind) [Docker image](https://hub.docker.com/_/docker/) with all tools installed @@ -118,7 +122,7 @@ In order to do that, follow the steps: Insecure = false ``` -1. You can now use `docker` from build script: +1. You can now use `docker` in the build script: ```yaml image: docker:latest @@ -136,21 +140,19 @@ In order to do that, follow the steps: - docker run my-docker-image /script/to/run/tests ``` -### Notes -* By enabling `--docker-privileged` you are effectively disabling all +> **Notes:** +> * By enabling `--docker-privileged`, you are effectively disabling all the security mechanisms of containers and exposing your host to privilege escalation which can lead to container breakout. For more information, check out the official Docker documentation on [Runtime privilege and Linux capabilities][docker-cap]. - -* Using docker-in-docker, each build is in a clean environment without the past -history. Concurrent builds work fine because every build get it's own instance of docker engine so they won't conflict with each other. But this also means builds can be slower because there's no caching of layers. - -* By default, `docker:dind` uses ``--storage-driver vfs` which is the slowest form +> * Using docker-in-docker, each build is in a clean environment without the past +history. Concurrent builds work fine because every build gets it's own instance of docker engine so they won't conflict with each other. But this also means builds can be slower because there's no caching of layers. +> * By default, `docker:dind` uses `--storage-driver vfs` which is the slowest form offered. An example project using this approach can be found here: https://gitlab.com/gitlab-examples/docker. -## 3. Bind Docker socket +### Use Docker socket binding The third approach is to bind-mount `/var/run/docker.sock` into the container so that docker is available in the context of that image. @@ -172,14 +174,14 @@ In order to do that, follow the steps: The above command will register a new Runner to use the special `docker:latest` image which is provided by Docker. **Notice that it's using - the Docker daemon of the runner itself, and any containers spawned by docker commands will be siblings of the runner rather than children of the runner.** This may have complications and limitations that are unsuitable for your workflow. + the Docker daemon of the Runner itself, and any containers spawned by docker commands will be siblings of the Runner rather than children of the runner.** This may have complications and limitations that are unsuitable for your workflow. The above command will create a `config.toml` entry similar to this: ``` [[runners]] url = "https://gitlab.com/ci" - token = TOKEN + token = REGISTRATION_TOKEN executor = "docker" [runners.docker] tls_verify = false @@ -191,7 +193,7 @@ In order to do that, follow the steps: Insecure = false ``` -1. You can now use `docker` from build script (note that you don't need to include the `docker:dind` service as in the option above): +1. You can now use `docker` in the build script (note that you don't need to include the `docker:dind` service as when using the Docker in Docker executor): ```yaml image: docker:latest @@ -206,16 +208,14 @@ In order to do that, follow the steps: - docker run my-docker-image /script/to/run/tests ``` -### Notes +While the above method avoids using Docker in privileged mode, you should be aware of the following implications: * By sharing the docker daemon, you are effectively disabling all the security mechanisms of containers and exposing your host to privilege escalation which can lead to container breakout. For example, if a project ran `docker rm -f $(docker ps -a -q)` it would remove the GitLab Runner containers. - * Concurrent builds may not work; if your tests create containers with specific names, they may conflict with each other. - * Sharing files and directories from the source repo into containers may not work as expected since volume mounting is done in the context of the host machine, not the build container. @@ -306,11 +306,11 @@ deploy: - master ``` -### Notes -1. You must log in to the container registry before running commands. Putting this in `before_script` will run it before each build job. -1. Using `docker build --pull` makes sure that Docker fetches any changes to base images before building just in case your cache is stale. It takes slightly longer, but means you don’t get stuck without security patches to base images. -1. Doing an explicit `docker pull` before each `docker run` makes sure to fetch the latest image that was just built. This is especially important if you are using multiple runners that cache images locally. Using the git SHA in your image tag makes this less necessary since each build will be unique and you shouldn't ever have a stale image, but it's still possible if you re-build a given commit after a dependency has changed. -1. You don't want to build directly to `latest` in case there are multiple builds happening simultaneously. +Some things you should be aware of when using the Container Registry: +* You must log in to the container registry before running commands. Putting this in `before_script` will run it before each build job. +* Using `docker build --pull` makes sure that Docker fetches any changes to base images before building just in case your cache is stale. It takes slightly longer, but means you don’t get stuck without security patches to base images. +* Doing an explicit `docker pull` before each `docker run` makes sure to fetch the latest image that was just built. This is especially important if you are using multiple runners that cache images locally. Using the git SHA in your image tag makes this less necessary since each build will be unique and you shouldn't ever have a stale image, but it's still possible if you re-build a given commit after a dependency has changed. +* You don't want to build directly to `latest` in case there are multiple builds happening simultaneously. [docker-in-docker]: https://blog.docker.com/2013/09/docker-can-now-run-within-docker/ [docker-cap]: https://docs.docker.com/engine/reference/run/#runtime-privilege-and-linux-capabilities -- cgit v1.2.3 From 4c571041de6989d71f09fc326f8d6bee731f0b19 Mon Sep 17 00:00:00 2001 From: Mark Pundsack Date: Mon, 13 Jun 2016 22:36:28 -0700 Subject: Make minor grammar change --- doc/ci/docker/using_docker_build.md | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) (limited to 'doc/ci') diff --git a/doc/ci/docker/using_docker_build.md b/doc/ci/docker/using_docker_build.md index 77291597659..09a2d8b5966 100644 --- a/doc/ci/docker/using_docker_build.md +++ b/doc/ci/docker/using_docker_build.md @@ -72,8 +72,8 @@ GitLab Runner then executes build scripts as the `gitlab-runner` user. 5. You can now use `docker` command and install `docker-compose` if needed. > **Note:** -* By adding `gitlab-runner` to `docker` group you are effectively granting `gitlab-runner` full root permissions. -For more information please check out [On Docker security: `docker` group considered harmful](https://www.andreas-jung.com/contents/on-docker-security-docker-group-considered-harmful). +* By adding `gitlab-runner` to the `docker` group you are effectively granting `gitlab-runner` full root permissions. +For more information please read [On Docker security: `docker` group considered harmful](https://www.andreas-jung.com/contents/on-docker-security-docker-group-considered-harmful). ### Use docker-in-docker executor -- cgit v1.2.3 From aefb08cb6a8bd15415b641c385e790f941b72ced Mon Sep 17 00:00:00 2001 From: Mark Pundsack Date: Mon, 13 Jun 2016 22:42:46 -0700 Subject: Clarify dind example --- doc/ci/docker/using_docker_build.md | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) (limited to 'doc/ci') diff --git a/doc/ci/docker/using_docker_build.md b/doc/ci/docker/using_docker_build.md index 09a2d8b5966..36ff4dcf05a 100644 --- a/doc/ci/docker/using_docker_build.md +++ b/doc/ci/docker/using_docker_build.md @@ -122,7 +122,7 @@ In order to do that, follow the steps: Insecure = false ``` -1. You can now use `docker` in the build script: +1. You can now use `docker` in the build script (note the inclusion of the `docker:dind` service): ```yaml image: docker:latest @@ -141,7 +141,7 @@ In order to do that, follow the steps: ``` > **Notes:** -> * By enabling `--docker-privileged`, you are effectively disabling all +> * By enabling `--docker-privileged`, you are effectively disabling all of the security mechanisms of containers and exposing your host to privilege escalation which can lead to container breakout. For more information, check out the official Docker documentation on [Runtime privilege and Linux capabilities][docker-cap]. -- cgit v1.2.3 From 8df7d90d5a92b7d8aa26ac07b7391b4e86d63499 Mon Sep 17 00:00:00 2001 From: Mark Pundsack Date: Mon, 13 Jun 2016 22:45:43 -0700 Subject: De-note-ify --- doc/ci/docker/using_docker_build.md | 8 ++++---- 1 file changed, 4 insertions(+), 4 deletions(-) (limited to 'doc/ci') diff --git a/doc/ci/docker/using_docker_build.md b/doc/ci/docker/using_docker_build.md index 36ff4dcf05a..39eea740d18 100644 --- a/doc/ci/docker/using_docker_build.md +++ b/doc/ci/docker/using_docker_build.md @@ -140,14 +140,14 @@ In order to do that, follow the steps: - docker run my-docker-image /script/to/run/tests ``` -> **Notes:** -> * By enabling `--docker-privileged`, you are effectively disabling all of +Docker-in-Docker works well, and is our recommended configuration, but it is not without its own challenges: +* By enabling `--docker-privileged`, you are effectively disabling all of the security mechanisms of containers and exposing your host to privilege escalation which can lead to container breakout. For more information, check out the official Docker documentation on [Runtime privilege and Linux capabilities][docker-cap]. -> * Using docker-in-docker, each build is in a clean environment without the past +* Using docker-in-docker, each build is in a clean environment without the past history. Concurrent builds work fine because every build gets it's own instance of docker engine so they won't conflict with each other. But this also means builds can be slower because there's no caching of layers. -> * By default, `docker:dind` uses `--storage-driver vfs` which is the slowest form +* By default, `docker:dind` uses `--storage-driver vfs` which is the slowest form offered. An example project using this approach can be found here: https://gitlab.com/gitlab-examples/docker. -- cgit v1.2.3 From 6ed7fcad29d0b96b4513c2961c342d0309eda07e Mon Sep 17 00:00:00 2001 From: Mark Pundsack Date: Mon, 13 Jun 2016 22:47:54 -0700 Subject: Remove our --- doc/ci/docker/using_docker_build.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) (limited to 'doc/ci') diff --git a/doc/ci/docker/using_docker_build.md b/doc/ci/docker/using_docker_build.md index 39eea740d18..7f83f846454 100644 --- a/doc/ci/docker/using_docker_build.md +++ b/doc/ci/docker/using_docker_build.md @@ -140,7 +140,7 @@ In order to do that, follow the steps: - docker run my-docker-image /script/to/run/tests ``` -Docker-in-Docker works well, and is our recommended configuration, but it is not without its own challenges: +Docker-in-Docker works well, and is the recommended configuration, but it is not without its own challenges: * By enabling `--docker-privileged`, you are effectively disabling all of the security mechanisms of containers and exposing your host to privilege escalation which can lead to container breakout. For more information, check out the official Docker documentation on -- cgit v1.2.3 From e129f66d9e597f43f7a85243ddedf3de0fc4946a Mon Sep 17 00:00:00 2001 From: Kamil Trzcinski Date: Tue, 14 Jun 2016 14:43:16 +0200 Subject: Add gitlab-ci.yml documentation for environments --- doc/ci/yaml/README.md | 27 +++++++++++++++++++++++++++ 1 file changed, 27 insertions(+) (limited to 'doc/ci') diff --git a/doc/ci/yaml/README.md b/doc/ci/yaml/README.md index 0707555e393..0546fa50f1c 100644 --- a/doc/ci/yaml/README.md +++ b/doc/ci/yaml/README.md @@ -28,6 +28,7 @@ If you want a quick introduction to GitLab CI, follow our - [only and except](#only-and-except) - [tags](#tags) - [when](#when) + - [environment](#environment) - [artifacts](#artifacts) - [artifacts:name](#artifacts-name) - [artifacts:when](#artifacts-when) @@ -353,6 +354,7 @@ job_name: | cache | no | Define list of files that should be cached between subsequent runs | | before_script | no | Override a set of commands that are executed before build | | after_script | no | Override a set of commands that are executed after build | +| environment | no | Defines a name of environment to which deployment is done by this build | ### script @@ -524,6 +526,31 @@ The above script will: 1. Execute `cleanup_build_job` only when `build_job` fails 2. Always execute `cleanup_job` as the last step in pipeline. +### environment + +>**Note:** +Introduced in GitLab v8.9.0. + +`environment` is used to define that job does deployment to specific environment. +This allows to easily track all deployments to your environments straight from GitLab. + +If `environment` is specified and no environment under that name does exist a new one will be created automatically. + +The `environment` name must contain only letters, digits, '-' and '_'. + +--- + +**Example configurations** + +``` +deploy to production: + stage: deploy + script: git push production HEAD:master + environment: production +``` + +The `deploy to production` job will be marked as doing deployment to `production` environment. + ### artifacts >**Notes:** -- cgit v1.2.3 From 47c9b7d34ce0e4e842dba72cedd66671efc03be5 Mon Sep 17 00:00:00 2001 From: Achilleas Pipinellis Date: Thu, 9 Jun 2016 18:32:17 +0200 Subject: Update CI API docs - Move ci/api under api/ci - Clean up builds.md and runners.md - Replace old links with new ones - Add CI API links in ci/README.md --- doc/ci/README.md | 2 +- doc/ci/api/README.md | 21 +------- doc/ci/api/builds.md | 138 +------------------------------------------------- doc/ci/api/runners.md | 45 +--------------- 4 files changed, 4 insertions(+), 202 deletions(-) (limited to 'doc/ci') diff --git a/doc/ci/README.md b/doc/ci/README.md index 4abc45bf9bb..ef72df97ce6 100644 --- a/doc/ci/README.md +++ b/doc/ci/README.md @@ -14,5 +14,5 @@ - [Trigger builds through the API](triggers/README.md) - [Build artifacts](build_artifacts/README.md) - [User permissions](permissions/README.md) -- [API](api/README.md) +- [API](../../api/ci/README.md) - [CI services (linked docker containers)](services/README.md) diff --git a/doc/ci/api/README.md b/doc/ci/api/README.md index aea808007fc..4ca8d92d7cc 100644 --- a/doc/ci/api/README.md +++ b/doc/ci/api/README.md @@ -1,22 +1,3 @@ # GitLab CI API -## Purpose - -Main purpose of GitLab CI API is to provide necessary data and context for -GitLab CI Runners. - -For consumer API take a look at this [documentation](../../api/README.md) where -you will find all relevant information. - -## API Prefix - -Current CI API prefix is `/ci/api/v1`. - -You need to prepend this prefix to all examples in this documentation, like: - - GET /ci/api/v1/builds/:id/artifacts - -## Resources - -- [Builds](builds.md) -- [Runners](runners.md) +This document was moved to a [new location](../../api/ci/README.md). diff --git a/doc/ci/api/builds.md b/doc/ci/api/builds.md index 79761a893da..f5bd3181c02 100644 --- a/doc/ci/api/builds.md +++ b/doc/ci/api/builds.md @@ -1,139 +1,3 @@ # Builds API -API used by runners to receive and update builds. - -_**Note:** This API is intended to be used only by Runners as their own -communication channel. For the consumer API see the -[Builds API](../../api/builds.md)._ - -## Authentication - -This API uses two types of authentication: - -1. Unique runner's token - - Token assigned to runner after it has been registered. - -2. Using build authorization token - - This is project's CI token that can be found in Continuous Integration - project settings. - - Build authorization token can be passed as a parameter or a value of - `BUILD-TOKEN` header. This method are interchangeable. - -## Builds - -### Runs oldest pending build by runner - -``` -POST /ci/api/v1/builds/register -``` - -| Attribute | Type | Required | Description | -|-----------|---------|----------|---------------------| -| `token` | string | yes | Unique runner token | - - -``` -curl -X POST "https://gitlab.example.com/ci/api/v1/builds/register" -F "token=t0k3n" -``` - -### Update details of an existing build - -``` -PUT /ci/api/v1/builds/:id -``` - -| Attribute | Type | Required | Description | -|-----------|---------|----------|----------------------| -| `id` | integer | yes | The ID of a project | -| `token` | string | yes | Unique runner token | -| `state` | string | no | The state of a build | -| `trace` | string | no | The trace of a build | - -``` -curl -X PUT "https://gitlab.example.com/ci/api/v1/builds/1234" -F "token=t0k3n" -F "state=running" -F "trace=Running git clone...\n" -``` - -### Incremental build trace update - -Using this method you need to send trace content as a request body. You also need to provide the `Content-Range` header -with a range of sent trace part. Note that you need to send parts in the proper order, so the begining of the part -must start just after the end of the previous part. If you provide the wrong part, then GitLab CI API will return `416 -Range Not Satisfiable` response with a header `Range: 0-X`, where `X` is the current trace length. - -For example, if you receive `Range: 0-11` in the response, then your next part must contain a `Content-Range: 11-...` -header and a trace part covered by this range. - -For a valid update API will return `202` response with: -* `Build-Status: {status}` header containing current status of the build, -* `Range: 0-{length}` header with the current trace length. - -``` -PATCH /ci/api/v1/builds/:id/trace.txt -``` - -Parameters: - -| Attribute | Type | Required | Description | -|-----------|---------|----------|----------------------| -| `id` | integer | yes | The ID of a build | - -Headers: - -| Attribute | Type | Required | Description | -|-----------------|---------|----------|-----------------------------------| -| `BUILD-TOKEN` | string | yes | The build authorization token | -| `Content-Range` | string | yes | Bytes range of trace that is sent | - -``` -curl -X PATCH "https://gitlab.example.com/ci/api/v1/builds/1234/trace.txt" -H "BUILD-TOKEN=build_t0k3n" -H "Content-Range=0-21" -d "Running git clone...\n" -``` - - -### Upload artifacts to build - -``` -POST /ci/api/v1/builds/:id/artifacts -``` - -| Attribute | Type | Required | Description | -|-----------|---------|----------|-------------------------------| -| `id` | integer | yes | The ID of a build | -| `token` | string | yes | The build authorization token | -| `file` | mixed | yes | Artifacts file | - -``` -curl -X POST "https://gitlab.example.com/ci/api/v1/builds/1234/artifacts" -F "token=build_t0k3n" -F "file=@/path/to/file" -``` - -### Download the artifacts file from build - -``` -GET /ci/api/v1/builds/:id/artifacts -``` - -| Attribute | Type | Required | Description | -|-----------|---------|----------|-------------------------------| -| `id` | integer | yes | The ID of a build | -| `token` | string | yes | The build authorization token | - -``` -curl "https://gitlab.example.com/ci/api/v1/builds/1234/artifacts" -F "token=build_t0k3n" -``` - -### Remove the artifacts file from build - -``` -DELETE /ci/api/v1/builds/:id/artifacts -``` - -| Attribute | Type | Required | Description | -|-----------|---------|----------|-------------------------------| -| ` id` | integer | yes | The ID of a build | -| `token` | string | yes | The build authorization token | - -``` -curl -X DELETE "https://gitlab.example.com/ci/api/v1/builds/1234/artifacts" -F "token=build_t0k3n" -``` +This document was moved to a [new location](../../api/ci/builds.md). diff --git a/doc/ci/api/runners.md b/doc/ci/api/runners.md index 2f01da4bd76..b14ea99db76 100644 --- a/doc/ci/api/runners.md +++ b/doc/ci/api/runners.md @@ -1,46 +1,3 @@ # Runners API -API used by runners to register and delete themselves. - -_**Note:** This API is intended to be used only by Runners as their own -communication channel. For the consumer API see the -[new Runners API](../../api/runners.md)._ - -## Authentication - -This API uses two types of authentication: - -1. Unique runner's token - - Token assigned to runner after it has been registered. - -2. Using runners' registration token - - This is a token that can be found in project's settings. - It can be also found in Admin area » Runners settings. - - There are two types of tokens you can pass - shared runner registration - token or project specific registration token. - -## Runners - -### Register a new runner - -Used to make GitLab CI aware of available runners. - - POST /ci/api/v1/runners/register - -Parameters: - - * `token` (required) - Registration token - - -### Delete a runner - -Used to remove runner. - - DELETE /ci/api/v1/runners/delete - -Parameters: - - * `token` (required) - Unique runner token +This document was moved to a [new location](../../api/ci/runners.md). -- cgit v1.2.3 From 6d9ed76419641c48ab18f07e08e640008c72b906 Mon Sep 17 00:00:00 2001 From: Mark Pundsack Date: Tue, 14 Jun 2016 21:26:38 -0700 Subject: Document CI_BUILD_TOKEN --- doc/ci/variables/README.md | 2 ++ 1 file changed, 2 insertions(+) (limited to 'doc/ci') diff --git a/doc/ci/variables/README.md b/doc/ci/variables/README.md index 70fb81492d6..137b080a8f7 100644 --- a/doc/ci/variables/README.md +++ b/doc/ci/variables/README.md @@ -34,6 +34,7 @@ The `API_TOKEN` will take the Secure Variable value: `SECURE`. | **CI_BUILD_ID** | all | The unique id of the current build that GitLab CI uses internally | | **CI_BUILD_REPO** | all | The URL to clone the Git repository | | **CI_BUILD_TRIGGERED** | 0.5 | The flag to indicate that build was [triggered] | +| **CI_BUILD_TOKEN** | 1.2 | Token used for authenticating with the GitLab Container Registry | | **CI_PROJECT_ID** | all | The unique id of the current project that GitLab CI uses internally | | **CI_PROJECT_DIR** | all | The full path where the repository is cloned and where the build is ran | @@ -50,6 +51,7 @@ export CI_BUILD_TAG="1.0.0" export CI_BUILD_NAME="spec:other" export CI_BUILD_STAGE="test" export CI_BUILD_TRIGGERED="true" +export CI_BUILD_TOKEN="abcde-1234ABCD5678ef" export CI_PROJECT_DIR="/builds/gitlab-org/gitlab-ce" export CI_PROJECT_ID="34" export CI_SERVER="yes" -- cgit v1.2.3