diff options
author | Achilleas Pipinellis <axil@gitlab.com> | 2018-06-07 16:08:43 +0300 |
---|---|---|
committer | Achilleas Pipinellis <axil@gitlab.com> | 2018-06-07 16:21:25 +0300 |
commit | 135dc2c5b29fc8fe2563617882711364160e729c (patch) | |
tree | 31cfd5a70c97155d6180b42a5c4fe7743147ad18 /doc/topics | |
parent | 21469fa6b7364155c955fbef48880eb32419361d (diff) |
Port the Auto DevOps docs from EE
Diffstat (limited to 'doc/topics')
-rw-r--r-- | doc/topics/autodevops/img/autodevops_domain_variables.png | bin | 0 -> 8456 bytes | |||
-rw-r--r-- | doc/topics/autodevops/img/autodevops_multiple_clusters.png | bin | 0 -> 12619 bytes | |||
-rw-r--r-- | doc/topics/autodevops/index.md | 133 |
3 files changed, 100 insertions, 33 deletions
diff --git a/doc/topics/autodevops/img/autodevops_domain_variables.png b/doc/topics/autodevops/img/autodevops_domain_variables.png Binary files differnew file mode 100644 index 00000000000..b6f8864796f --- /dev/null +++ b/doc/topics/autodevops/img/autodevops_domain_variables.png diff --git a/doc/topics/autodevops/img/autodevops_multiple_clusters.png b/doc/topics/autodevops/img/autodevops_multiple_clusters.png Binary files differnew file mode 100644 index 00000000000..f4d101ca921 --- /dev/null +++ b/doc/topics/autodevops/img/autodevops_multiple_clusters.png diff --git a/doc/topics/autodevops/index.md b/doc/topics/autodevops/index.md index 478b9c9c0c6..4bcc7fc6512 100644 --- a/doc/topics/autodevops/index.md +++ b/doc/topics/autodevops/index.md @@ -41,6 +41,7 @@ project in an easy and automatic way: 1. [Auto Code Quality](#auto-code-quality) 1. [Auto SAST (Static Application Security Testing)](#auto-sast) 1. [Auto Dependency Scanning](#auto-dependency-scanning) +1. [Auto License Management](#auto-license-management) 1. [Auto Container Scanning](#auto-container-scanning) 1. [Auto Review Apps](#auto-review-apps) 1. [Auto DAST (Dynamic Application Security Testing)](#auto-dast) @@ -62,7 +63,7 @@ Auto DevOps provides great defaults for all the stages; you can, however, For an overview on the creation of Auto DevOps, read the blog post [From 2/3 of the Self-Hosted Git Market, to the Next-Generation CI System, to Auto DevOps](https://about.gitlab.com/2017/06/29/whats-next-for-gitlab-ci/). -## Prerequisites +## Requirements TIP: **Tip:** For self-hosted installations, the easiest way to make use of Auto DevOps is to @@ -112,25 +113,26 @@ NOTE: **Note:** If you do not have Kubernetes or Prometheus installed, then Auto Review Apps, Auto Deploy, and Auto Monitoring will be silently skipped. -### Auto DevOps base domain +## Auto DevOps base domain The Auto DevOps base domain is required if you want to make use of [Auto -Review Apps](#auto-review-apps) and [Auto Deploy](#auto-deploy). It is defined -either under the project's CI/CD settings while -[enabling Auto DevOps](#enabling-auto-devops) or in instance-wide settings in -the CI/CD section. -It can also be set at the project or group level as a variable, `AUTO_DEVOPS_DOMAIN`. +Review Apps](#auto-review-apps) and [Auto Deploy](#auto-deploy). It can be defined +in three places: -A wildcard DNS A record matching the base domain is required, for example, +- either under the project's CI/CD settings while [enabling Auto DevOps](#enabling-auto-devops) +- or in instance-wide settings in the **admin area > Settings** under the "Continuous Integration and Delivery" section +- or at the project or group level as a variable: `AUTO_DEVOPS_DOMAIN` (required if you want to use [multiple clusters](#using-multiple-kubernetes-clusters)) + +A wildcard DNS A record matching the base domain(s) is required, for example, given a base domain of `example.com`, you'd need a DNS entry like: ``` *.example.com 3600 A 1.2.3.4 ``` -where `example.com` is the domain name under which the deployed apps will be served, +In this case, `example.com` is the domain name under which the deployed apps will be served, and `1.2.3.4` is the IP address of your load balancer; generally NGINX -([see prerequisites](#prerequisites)). How to set up the DNS record is beyond +([see requirements](#requirements)). How to set up the DNS record is beyond the scope of this document; you should check with your DNS provider. Alternatively you can use free public services like [xip.io](http://xip.io) or @@ -146,6 +148,56 @@ If GitLab is installed using the [GitLab Omnibus Helm Chart], there are two options: provide a static IP, or have one assigned. For more information see the relevant docs on the [network prerequisites](../../install/kubernetes/gitlab_omnibus.md#networking-prerequisites). +## Using multiple Kubernetes clusters **[PREMIUM]** + +When using Auto DevOps, you may want to deploy different environments to +different Kubernetes clusters. This is possible due to the 1:1 connection that +[exists between them](../../user/project/clusters/index.md#multiple-kubernetes-clusters). + +In the [Auto DevOps template](https://gitlab.com/gitlab-org/gitlab-ci-yml/blob/master/Auto-DevOps.gitlab-ci.yml) +(used behind the scenes by Auto DevOps), there are currently 3 defined environment names that you need to know: + +- `review/` (every environment starting with `review/`) +- `staging` +- `production` + +Those environments are tied to jobs that use [Auto Deploy](#auto-deploy), so +except for the environment scope, they would also need to have a different +domain they would be deployed to. This is why you need to define a separate +`AUTO_DEVOPS_DOMAIN` variable for all the above +[based on the environment](../../ci/variables/README.md#limiting-environment-scopes-of-variables). + +The following table is an example of how the three different clusters would +be configured. + +| Cluster name | Cluster environment scope | `AUTO_DEVOPS_DOMAIN` variable value | Variable environment scope | Notes | +| ------------ | -------------- | ----------------------------- | ------------- | ------ | +| review | `review/*` | `review.example.com` | `review/*` | The review cluster which will run all [Review Apps](../../ci/review_apps/index.md). `*` is a wildcard, which means it will be used by every environment name starting with `review/`. | +| staging | `staging` | `staging.example.com` | `staging` | (Optional) The staging cluster which will run the deployments of the staging environments. You need to [enable it first](#deploy-policy-for-staging-and-production-environments). | +| production | `production` | `example.com` | `production` | The production cluster which will run the deployments of the production environment. You can use [incremental rollouts](#incremental-rollout-to-production). | + +To add a different cluster for each environment: + +1. Navigate to your project's **Operations > Kubernetes** and create the Kubernetes clusters + with their respective environment scope as described from the table above. + + ![Auto DevOps multiple clusters](img/autodevops_multiple_clusters.png) + +1. After the clusters are created, navigate to each one and install Helm Tiller + and Ingress. +1. Make sure you have [configured your DNS](#auto-devops-base-domain) with the + specified Auto DevOps domains. +1. Navigate to your project's **Settings > CI/CD > Variables** and add + the `AUTO_DEVOPS_DOMAIN` variables with their respective environment + scope. + + ![Auto DevOps domain variables](img/autodevops_domain_variables.png) + +Now that all is configured, you can test your setup by creating a merge request +and verifying that your app is deployed as a review app in the Kubernetes +cluster with the `review/*` environment scope. Similarly, you can check the +other environments. + ## Quick start If you are using GitLab.com, see our [quick start guide](quick_start_guide.md) @@ -154,13 +206,13 @@ Google Cloud. ## Enabling Auto DevOps -If you haven't done already, read the [prerequisites](#prerequisites) to make +If you haven't done already, read the [requirements](#requirements) to make full use of Auto DevOps. If this is your fist time, we recommend you follow the -[quick start guide](#quick-start). +[quick start guide](quick_start_guide.md). To enable Auto DevOps to your project: -1. Check that your project doesn't have a `.gitlab-ci.yml`, and remove it otherwise +1. Check that your project doesn't have a `.gitlab-ci.yml`, or remove it otherwise 1. Go to your project's **Settings > CI/CD > General pipelines settings** and find the Auto DevOps section 1. Select "Enable Auto DevOps" @@ -230,7 +282,7 @@ In GitLab Starter, differences between the source and target branches are also [shown in the merge request widget](https://docs.gitlab.com/ee/user/project/merge_requests/code_quality_diff.html). -### Auto SAST +### Auto SAST **[ULTIMATE]** > Introduced in [GitLab Ultimate][ee] 10.3. @@ -241,9 +293,9 @@ report is created, it's uploaded as an artifact which you can later download and check out. In GitLab Ultimate, any security warnings are also -[shown in the merge request widget](https://docs.gitlab.com/ee/user/project/merge_requests/sast.html). +[shown in the merge request widget](https://docs.gitlab.com/ee//user/project/merge_requests/sast.html). -### Auto Dependency Scanning +### Auto Dependency Scanning **[ULTIMATE]** > Introduced in [GitLab Ultimate][ee] 10.7. @@ -254,7 +306,20 @@ report is created, it's uploaded as an artifact which you can later download and check out. In GitLab Ultimate, any security warnings are also -[shown in the merge request widget](https://docs.gitlab.com/ee/user/project/merge_requests/dependency_scanning.html). +[shown in the merge request widget](https://docs.gitlab.com/ee//user/project/merge_requests/dependency_scanning.html). + +### Auto License Management **[ULTIMATE]** + +> Introduced in [GitLab Ultimate][ee] 11.0. + +License Management uses the +[License Management Docker image](https://gitlab.com/gitlab-org/security-products/license_management) +to search the project dependencies for their license. Once the +report is created, it's uploaded as an artifact which you can later download and +check out. + +In GitLab Ultimate, any licenses are also +[shown in the merge request widget](https://docs.gitlab.com/ee//user/project/merge_requests/license_management.html). ### Auto Container Scanning @@ -267,13 +332,13 @@ created, it's uploaded as an artifact which you can later download and check out. In GitLab Ultimate, any security warnings are also -[shown in the merge request widget](https://docs.gitlab.com/ee/user/project/merge_requests/container_scanning.html). +[shown in the merge request widget](https://docs.gitlab.com/ee//user/project/merge_requests/container_scanning.html). ### Auto Review Apps NOTE: **Note:** This is an optional step, since many projects do not have a Kubernetes cluster -available. If the [prerequisites](#prerequisites) are not met, the job will +available. If the [requirements](#requirements) are not met, the job will silently be skipped. CAUTION: **Caution:** @@ -295,7 +360,7 @@ up in the merge request widget for easy discovery. When the branch is deleted, for example after the merge request is merged, the Review App will automatically be deleted. -### Auto DAST +### Auto DAST **[ULTIMATE]** > Introduced in [GitLab Ultimate][ee] 10.4. @@ -306,9 +371,9 @@ issues. Once the report is created, it's uploaded as an artifact which you can later download and check out. In GitLab Ultimate, any security warnings are also -[shown in the merge request widget](https://docs.gitlab.com/ee/user/project/merge_requests/dast.html). +[shown in the merge request widget](https://docs.gitlab.com/ee//user/project/merge_requests/dast.html). -### Auto Browser Performance Testing +### Auto Browser Performance Testing **[PREMIUM]** > Introduced in [GitLab Premium][ee] 10.4. @@ -320,13 +385,14 @@ Auto Browser Performance Testing utilizes the [Sitespeed.io container](https://h /direction ``` -In GitLab Premium, performance differences between the source and target branches are [shown in the merge request widget](https://docs.gitlab.com/ee/user/project/merge_requests/browser_performance_testing.html). +In GitLab Premium, performance differences between the source +and target branches are [shown in the merge request widget](https://docs.gitlab.com/ee//user/project/merge_requests/browser_performance_testing.html). ### Auto Deploy NOTE: **Note:** This is an optional step, since many projects do not have a Kubernetes cluster -available. If the [prerequisites](#prerequisites) are not met, the job will +available. If the [requirements](#requirements) are not met, the job will silently be skipped. CAUTION: **Caution:** @@ -363,7 +429,7 @@ executed somewhere else, it cannot be accessed again. ### Auto Monitoring NOTE: **Note:** -Check the [prerequisites](#prerequisites) for Auto Monitoring to make this stage +Check the [requirements](#requirements) for Auto Monitoring to make this stage work. Once your application is deployed, Auto Monitoring makes it possible to monitor @@ -437,7 +503,7 @@ repo or by specifying a project variable: file in it, Auto DevOps will detect the chart and use it instead of the [default one](https://gitlab.com/charts/charts.gitlab.io/tree/master/charts/auto-deploy-app). This can be a great way to control exactly how your application is deployed. -- **Project variable** - Create a [variable](../../ci/variables/README.md#variables) +- **Project variable** - Create a [project variable](../../ci/variables/README.md#secret-variables) `AUTO_DEVOPS_CHART` with the URL of a custom chart to use. ### Customizing `.gitlab-ci.yml` @@ -493,22 +559,23 @@ also be customized, and you can easily use a [custom buildpack](#custom-buildpac | `POSTGRES_PASSWORD` | The PostgreSQL password; defaults to `testing-password`. Set it to use a custom password. | | `POSTGRES_DB` | The PostgreSQL database name; defaults to the value of [`$CI_ENVIRONMENT_SLUG`](../../ci/variables/README.md#predefined-variables-environment-variables). Set it to use a custom database name. | | `BUILDPACK_URL` | The buildpack's full URL. It can point to either Git repositories or a tarball URL. For Git repositories, it is possible to point to a specific `ref`, for example `https://github.com/heroku/heroku-buildpack-ruby.git#v142` | +| `SAST_CONFIDENCE_LEVEL` | The minimum confidence level of security issues you want to be reported; `1` for Low, `2` for Medium, `3` for High; defaults to `3`.| +| `DEP_SCAN_DISABLE_REMOTE_CHECKS` | Whether remote Dependency Scanning checks are disabled; defaults to `"false"`. Set to `"true"` to disable checks that send data to GitLab central servers. [Read more about remote checks](https://gitlab.com/gitlab-org/security-products/dependency-scanning#remote-checks).| | `STAGING_ENABLED` | From GitLab 10.8, this variable can be used to define a [deploy policy for staging and production environments](#deploy-policy-for-staging-and-production-environments). | | `CANARY_ENABLED` | From GitLab 11.0, this variable can be used to define a [deploy policy for canary environments](#deploy-policy-for-canary-environments). | | `INCREMENTAL_ROLLOUT_ENABLED`| From GitLab 10.8, this variable can be used to enable an [incremental rollout](#incremental-rollout-to-production) of your application for the production environment. | | `TEST_DISABLED` | From GitLab 11.0, this variable can be used to disable the `test` job. If the variable is present, the job will not be created. | -| `CODE_QUALITY_DISABLED` | From GitLab 11.0, this variable can be used to disable the `code_quality` job. If the variable is present, the job will not be created. | -| `LICENSE_MANAGEMENT_DISABLED` | From GitLab 11.0, this variable can be used to disable the `license_management` job. If the variable is present, the job will not be created. | +| `CODEQUALITY_DISABLED` | From GitLab 11.0, this variable can be used to disable the `codequality` job. If the variable is present, the job will not be created. | | `SAST_DISABLED` | From GitLab 11.0, this variable can be used to disable the `sast` job. If the variable is present, the job will not be created. | | `DEPENDENCY_SCANNING_DISABLED` | From GitLab 11.0, this variable can be used to disable the `dependency_scanning` job. If the variable is present, the job will not be created. | -| `CONTAINER_SCANNING_DISABLED` | From GitLab 11.0, this variable can be used to disable the `container_scanning` job. If the variable is present, the job will not be created. | +| `CONTAINER_SCANNING_DISABLED` | From GitLab 11.0, this variable can be used to disable the `sast:container` job. If the variable is present, the job will not be created. | | `REVIEW_DISABLED` | From GitLab 11.0, this variable can be used to disable the `review` and the manual `review:stop` job. If the variable is present, these jobs will not be created. | | `DAST_DISABLED` | From GitLab 11.0, this variable can be used to disable the `dast` job. If the variable is present, the job will not be created. | | `PERFORMANCE_DISABLED` | From GitLab 11.0, this variable can be used to disable the `performance` job. If the variable is present, the job will not be created. | TIP: **Tip:** Set up the replica variables using a -[project variable](../../ci/variables/README.md#variables) +[project variable](../../ci/variables/README.md#secret-variables) and scale your application by just redeploying it! CAUTION: **Caution:** @@ -583,7 +650,7 @@ staging environment and deploy to production manually. For this scenario, the `STAGING_ENABLED` environment variable was introduced. If `STAGING_ENABLED` is defined in your project (e.g., set `STAGING_ENABLED` to -`1` as a variable), then the application will be automatically deployed +`1` as a secret variable), then the application will be automatically deployed to a `staging` environment, and a `production_manual` job will be created for you when you're ready to manually deploy to production. @@ -596,7 +663,7 @@ A [canary environment](https://docs.gitlab.com/ee/user/project/canary_deployment before any changes are deployed to production. If `CANARY_ENABLED` is defined in your project (e.g., set `CANARY_ENABLED` to -`1` as a variable) then two manual jobs will be created: +`1` as a secret variable) then two manual jobs will be created: - `canary` which will deploy the application to the canary environment - `production_manual` which is to be used by you when you're ready to manually @@ -612,7 +679,7 @@ This will allow you to first check how the app is behaving, and later manually increasing the rollout up to 100%. If `INCREMENTAL_ROLLOUT_ENABLED` is defined in your project (e.g., set -`INCREMENTAL_ROLLOUT_ENABLED` to `1` as a variable), then instead of the +`INCREMENTAL_ROLLOUT_ENABLED` to `1` as a secret variable), then instead of the standard `production` job, 4 different [manual jobs](../../ci/pipelines.md#manual-actions-from-the-pipeline-graph) will be created: |