diff options
Diffstat (limited to 'doc/topics/autodevops/index.md')
| -rw-r--r-- | doc/topics/autodevops/index.md | 246 |
1 files changed, 134 insertions, 112 deletions
diff --git a/doc/topics/autodevops/index.md b/doc/topics/autodevops/index.md index 83bcb7950b5..88cd618a5c7 100644 --- a/doc/topics/autodevops/index.md +++ b/doc/topics/autodevops/index.md @@ -95,58 +95,85 @@ Auto DevOps. To make full use of Auto DevOps, you will need: +- **Kubernetes** (for Auto Review Apps, Auto Deploy, and Auto Monitoring) + + To enable deployments, you will need: + + 1. A [Kubernetes 1.5+ cluster](../../user/project/clusters/index.md) for the project. The easiest + way is to add a [new GKE cluster using the GitLab UI](../../user/project/clusters/add_remove_clusters.md#add-new-gke-cluster). + 1. NGINX Ingress. You can deploy it to your Kubernetes cluster by installing + the [GitLab-managed app for Ingress](../../user/clusters/applications.md#ingress), + once you have configured GitLab's Kubernetes integration in the previous step. + + Alternatively, you can use the + [`nginx-ingress`](https://github.com/helm/charts/tree/master/stable/nginx-ingress) + Helm chart to install Ingress manually. + + NOTE: **Note:** + If you are using your own Ingress instead of the one provided by GitLab's managed + apps, ensure you are running at least version 0.9.0 of NGINX Ingress and + [enable Prometheus metrics](https://github.com/kubernetes/ingress-nginx/blob/master/docs/examples/customization/custom-vts-metrics-prometheus/nginx-vts-metrics-conf.yaml) + in order for the response metrics to appear. You will also have to + [annotate](https://kubernetes.io/docs/concepts/overview/working-with-objects/annotations/) + the NGINX Ingress deployment to be scraped by Prometheus using + `prometheus.io/scrape: "true"` and `prometheus.io/port: "10254"`. + +- **Base domain** (for Auto Review Apps, Auto Deploy, and Auto Monitoring) + + You will need a domain configured with wildcard DNS which is going to be used + by all of your Auto DevOps applications. If you're using the + [GitLab-managed app for Ingress](../../user/clusters/applications.md#ingress), + the URL endpoint will be automatically configured for you. + + You will then need to [specify the Auto DevOps base domain](#auto-devops-base-domain). + - **GitLab Runner** (for all stages) Your Runner needs to be configured to be able to run Docker. Generally this means using either the [Docker](https://docs.gitlab.com/runner/executors/docker.html) or [Kubernetes](https://docs.gitlab.com/runner/executors/kubernetes.html) executors, with [privileged mode enabled](https://docs.gitlab.com/runner/executors/docker.html#use-docker-in-docker-with-privileged-mode). - The Runners do not need to be installed in the Kubernetes cluster, but the Kubernetes executor is easy to use and is automatically autoscaling. Docker-based Runners can be configured to autoscale as well, using [Docker Machine](https://docs.gitlab.com/runner/install/autoscaling.html). + If you have configured GitLab's Kubernetes integration in the first step, you + can deploy it to your cluster by installing the + [GitLab-managed app for GitLab Runner](../../user/clusters/applications.md#gitlab-runner). + Runners should be registered as [shared Runners](../../ci/runners/README.md#registering-a-shared-runner) for the entire GitLab instance, or [specific Runners](../../ci/runners/README.md#registering-a-specific-runner) - that are assigned to specific projects. -- **Base domain** (for Auto Review Apps and Auto Deploy) - - You will need a domain configured with wildcard DNS which is going to be used - by all of your Auto DevOps applications. + that are assigned to specific projects (the default if you have installed the + GitLab Runner managed application). - Read the [specifics](#auto-devops-base-domain). -- **Kubernetes** (for Auto Review Apps, Auto Deploy, and Auto Monitoring) - - To enable deployments, you will need: - - - Kubernetes 1.5+. - - A [Kubernetes cluster][kubernetes-clusters] for the project. - - A load balancer. You can use NGINX Ingress by deploying it to your - Kubernetes cluster by either: - - Using the [`nginx-ingress`](https://github.com/helm/charts/tree/master/stable/nginx-ingress) Helm chart. - - Installing the Ingress [GitLab Managed App](../../user/clusters/applications.md#ingress). - **Prometheus** (for Auto Monitoring) - To enable Auto Monitoring, you - will need Prometheus installed somewhere (inside or outside your cluster) and - configured to scrape your Kubernetes cluster. To get response metrics - (in addition to system metrics), you need to - [configure Prometheus to monitor NGINX](../../user/project/integrations/prometheus_library/nginx_ingress.md#configuring-nginx-ingress-monitoring). + To enable Auto Monitoring, you will need Prometheus installed somewhere + (inside or outside your cluster) and configured to scrape your Kubernetes cluster. + If you have configured GitLab's Kubernetes integration, you can deploy it to + your cluster by installing the + [GitLab-managed app for Prometheus](../../user/clusters/applications.md#prometheus). The [Prometheus service](../../user/project/integrations/prometheus.md) - integration needs to be enabled for the project, or enabled as a + integration needs to be enabled for the project (or enabled as a [default service template](../../user/project/integrations/services_templates.md) - for the entire GitLab installation. + for the entire GitLab installation). + + To get response metrics (in addition to system metrics), you need to + [configure Prometheus to monitor NGINX](../../user/project/integrations/prometheus_library/nginx_ingress.md#configuring-nginx-ingress-monitoring). If you do not have Kubernetes or Prometheus installed, then Auto Review Apps, Auto Deploy, and Auto Monitoring will be silently skipped. +One all requirements are met, you can go ahead and [enable Auto DevOps](#enablingdisabling-auto-devops). + ## Auto DevOps base domain -The Auto DevOps base domain is required if you want to make use of [Auto -Review Apps](#auto-review-apps) and [Auto Deploy](#auto-deploy). It can be defined -in any of the following places: +The Auto DevOps base domain is required if you want to make use of +[Auto Review Apps](#auto-review-apps), [Auto Deploy](#auto-deploy), and +[Auto Monitoring](#auto-monitoring). It can be defined in any of the following +places: - either under the cluster's settings, whether for [projects](../../user/project/clusters/index.md#base-domain) or [groups](../../user/group/clusters/index.md#base-domain) - or in instance-wide settings in the **admin area > Settings** under the "Continuous Integration and Delivery" section @@ -156,9 +183,15 @@ in any of the following places: The base domain variable `KUBE_INGRESS_BASE_DOMAIN` follows the same order of precedence as other environment [variables](../../ci/variables/README.md#priority-of-environment-variables). -NOTE: **Note** -`AUTO_DEVOPS_DOMAIN` environment variable is deprecated and -[is scheduled to be removed](https://gitlab.com/gitlab-org/gitlab-foss/issues/56959). +TIP: **Tip:** +If you're using the [GitLab managed app for Ingress](../../user/clusters/applications.md#ingress), +the URL endpoint should be automatically configured for you. All you have to do +is use its value for the `KUBE_INGRESS_BASE_DOMAIN` variable. + +NOTE: **Note:** +`AUTO_DEVOPS_DOMAIN` was [deprecated in GitLab 11.8](https://gitlab.com/gitlab-org/gitlab-foss/issues/52363) +and replaced with `KUBE_INGRESS_BASE_DOMAIN`. It was removed in +[GitLab 12.0](https://gitlab.com/gitlab-org/gitlab-foss/issues/56959). A wildcard DNS A record matching the base domain(s) is required, for example, given a base domain of `example.com`, you'd need a DNS entry like: @@ -179,77 +212,28 @@ Auto DevOps base domain to `1.2.3.4.nip.io`. Once set up, all requests will hit the load balancer, which in turn will route them to the Kubernetes pods that run your application(s). -NOTE: **Note:** -From GitLab 11.8, `KUBE_INGRESS_BASE_DOMAIN` replaces `AUTO_DEVOPS_DOMAIN`. -Support for `AUTO_DEVOPS_DOMAIN` was [removed in GitLab -12.0](https://gitlab.com/gitlab-org/gitlab-foss/issues/56959). - -## Using multiple Kubernetes clusters **(PREMIUM)** - -When using Auto DevOps, you may want to deploy different environments to -different Kubernetes clusters. This is possible due to the 1:1 connection that -[exists between them](../../user/project/clusters/index.md#multiple-kubernetes-clusters-premium). - -In the [Auto DevOps template] (used behind the scenes by Auto DevOps), there -are currently 3 defined environment names that you need to know: - -- `review/` (every environment starting with `review/`) -- `staging` -- `production` - -Those environments are tied to jobs that use [Auto Deploy](#auto-deploy), so -except for the environment scope, they would also need to have a different -domain they would be deployed to. This is why you need to define a separate -`KUBE_INGRESS_BASE_DOMAIN` variable for all the above -[based on the environment](../../ci/variables/README.md#limiting-environment-scopes-of-environment-variables). - -The following table is an example of how the three different clusters would -be configured. - -| Cluster name | Cluster environment scope | `KUBE_INGRESS_BASE_DOMAIN` variable value | Variable environment scope | Notes | -|--------------|---------------------------|-------------------------------------------|----------------------------|---| -| review | `review/*` | `review.example.com` | `review/*` | The review cluster which will run all [Review Apps](../../ci/review_apps/index.md). `*` is a wildcard, which means it will be used by every environment name starting with `review/`. | -| staging | `staging` | `staging.example.com` | `staging` | (Optional) The staging cluster which will run the deployments of the staging environments. You need to [enable it first](#deploy-policy-for-staging-and-production-environments). | -| production | `production` | `example.com` | `production` | The production cluster which will run the deployments of the production environment. You can use [incremental rollouts](#incremental-rollout-to-production-premium). | - -To add a different cluster for each environment: - -1. Navigate to your project's **Operations > Kubernetes** and create the Kubernetes clusters - with their respective environment scope as described from the table above. - -  - -1. After the clusters are created, navigate to each one and install Helm Tiller - and Ingress. Wait for the Ingress IP address to be assigned. -1. Make sure you have [configured your DNS](#auto-devops-base-domain) with the - specified Auto DevOps domains. -1. Navigate to each cluster's page, through **Operations > Kubernetes**, - and add the domain based on its Ingress IP address. - -Now that all is configured, you can test your setup by creating a merge request -and verifying that your app is deployed as a review app in the Kubernetes -cluster with the `review/*` environment scope. Similarly, you can check the -other environments. - ## Enabling/Disabling Auto DevOps -When first using Auto Devops, review the [requirements](#requirements) to ensure all necessary components to make +When first using Auto DevOps, review the [requirements](#requirements) to ensure all necessary components to make full use of Auto DevOps are available. If this is your fist time, we recommend you follow the [quick start guide](quick_start_guide.md). GitLab.com users can enable/disable Auto DevOps at the project-level only. Self-managed users can enable/disable Auto DevOps at the project-level, group-level or instance-level. -### At the instance level (Administrators only) +### At the project level -Even when disabled at the instance level, group owners and project maintainers can still enable -Auto DevOps at the group and project level, respectively. +If enabling, check that your project doesn't have a `.gitlab-ci.yml`, or if one exists, remove it. -1. Go to **Admin area > Settings > Continuous Integration and Deployment**. -1. Toggle the checkbox labeled **Default to Auto DevOps pipeline for all projects**. -1. If enabling, optionally set up the Auto DevOps [base domain](#auto-devops-base-domain) which will be used for Auto Deploy and Auto Review Apps. +1. Go to your project's **Settings > CI/CD > Auto DevOps**. +1. Toggle the **Default to Auto DevOps pipeline** checkbox (checked to enable, unchecked to disable) +1. When enabling, it's optional but recommended to add in the [base domain](#auto-devops-base-domain) + that will be used by Auto DevOps to [deploy your application](#auto-deploy) + and choose the [deployment strategy](#deployment-strategy). 1. Click **Save changes** for the changes to take effect. +When the feature has been enabled, an Auto DevOps pipeline is triggered on the default branch. + ### At the group level > [Introduced](https://gitlab.com/gitlab-org/gitlab-foss/issues/52447) in GitLab 11.10. @@ -266,19 +250,16 @@ When enabling or disabling Auto DevOps at group-level, group configuration will the subgroups and projects inside that group, unless Auto DevOps is specifically enabled or disabled on the subgroup or project. -### At the project level +### At the instance level (Administrators only) -If enabling, check that your project doesn't have a `.gitlab-ci.yml`, or if one exists, remove it. +Even when disabled at the instance level, group owners and project maintainers can still enable +Auto DevOps at the group and project level, respectively. -1. Go to your project's **Settings > CI/CD > Auto DevOps**. -1. Toggle the **Default to Auto DevOps pipeline** checkbox (checked to enable, unchecked to disable) -1. When enabling, it's optional but recommended to add in the [base domain](#auto-devops-base-domain) - that will be used by Auto DevOps to [deploy your application](#auto-deploy) - and choose the [deployment strategy](#deployment-strategy). +1. Go to **Admin area > Settings > Continuous Integration and Deployment**. +1. Toggle the checkbox labeled **Default to Auto DevOps pipeline for all projects**. +1. If enabling, optionally set up the Auto DevOps [base domain](#auto-devops-base-domain) which will be used for Auto Deploy and Auto Review Apps. 1. Click **Save changes** for the changes to take effect. -When the feature has been enabled, an Auto DevOps pipeline is triggered on the default branch. - ### Enable for a percentage of projects There is also a feature flag to enable Auto DevOps by default to your chosen percentage of projects. @@ -310,6 +291,53 @@ The available options are: - `master` branch is directly deployed to staging. - Manual actions are provided for incremental rollout to production. +## Using multiple Kubernetes clusters **(PREMIUM)** + +When using Auto DevOps, you may want to deploy different environments to +different Kubernetes clusters. This is possible due to the 1:1 connection that +[exists between them](../../user/project/clusters/index.md#multiple-kubernetes-clusters-premium). + +In the [Auto DevOps template] (used behind the scenes by Auto DevOps), there +are currently 3 defined environment names that you need to know: + +- `review/` (every environment starting with `review/`) +- `staging` +- `production` + +Those environments are tied to jobs that use [Auto Deploy](#auto-deploy), so +except for the environment scope, they would also need to have a different +domain they would be deployed to. This is why you need to define a separate +`KUBE_INGRESS_BASE_DOMAIN` variable for all the above +[based on the environment](../../ci/variables/README.md#limiting-environment-scopes-of-environment-variables). + +The following table is an example of how the three different clusters would +be configured. + +| Cluster name | Cluster environment scope | `KUBE_INGRESS_BASE_DOMAIN` variable value | Variable environment scope | Notes | +|--------------|---------------------------|-------------------------------------------|----------------------------|---| +| review | `review/*` | `review.example.com` | `review/*` | The review cluster which will run all [Review Apps](../../ci/review_apps/index.md). `*` is a wildcard, which means it will be used by every environment name starting with `review/`. | +| staging | `staging` | `staging.example.com` | `staging` | (Optional) The staging cluster which will run the deployments of the staging environments. You need to [enable it first](#deploy-policy-for-staging-and-production-environments). | +| production | `production` | `example.com` | `production` | The production cluster which will run the deployments of the production environment. You can use [incremental rollouts](#incremental-rollout-to-production-premium). | + +To add a different cluster for each environment: + +1. Navigate to your project's **Operations > Kubernetes** and create the Kubernetes clusters + with their respective environment scope as described from the table above. + +  + +1. After the clusters are created, navigate to each one and install Helm Tiller + and Ingress. Wait for the Ingress IP address to be assigned. +1. Make sure you have [configured your DNS](#auto-devops-base-domain) with the + specified Auto DevOps domains. +1. Navigate to each cluster's page, through **Operations > Kubernetes**, + and add the domain based on its Ingress IP address. + +Now that all is configured, you can test your setup by creating a merge request +and verifying that your app is deployed as a review app in the Kubernetes +cluster with the `review/*` environment scope. Similarly, you can check the +other environments. + ## Stages of Auto DevOps The following sections describe the stages of Auto DevOps. Read them carefully @@ -672,8 +700,6 @@ workers: ### Auto Monitoring -See the [requirements](#requirements) for Auto Monitoring to enable this stage. - Once your application is deployed, Auto Monitoring makes it possible to monitor your application's server and response metrics right out of the box. Auto Monitoring uses [Prometheus](../../user/project/integrations/prometheus.md) to @@ -687,18 +713,15 @@ The metrics include: - **Response Metrics:** latency, throughput, error rate - **System Metrics:** CPU utilization, memory utilization -In order to make use of monitoring you need to: - -1. [Deploy Prometheus](../../user/project/integrations/prometheus.md) into your Kubernetes cluster -1. If you would like response metrics, ensure you are running at least version - 0.9.0 of NGINX Ingress and - [enable Prometheus metrics](https://github.com/kubernetes/ingress-nginx/blob/master/docs/examples/customization/custom-vts-metrics-prometheus/nginx-vts-metrics-conf.yaml). -1. Finally, [annotate](https://kubernetes.io/docs/concepts/overview/working-with-objects/annotations/) - the NGINX Ingress deployment to be scraped by Prometheus using - `prometheus.io/scrape: "true"` and `prometheus.io/port: "10254"`. +To make use of Auto Monitoring: -To view the metrics, open the -[Monitoring dashboard for a deployed environment](../../ci/environments.md#monitoring-environments). +1. [Install and configure the requirements](#requirements). +1. [Enable Auto DevOps](#enablingdisabling-auto-devops) if you haven't done already. +1. Finally, go to your project's **CI/CD > Pipelines** and run a pipeline. +1. Once the pipeline finishes successfully, open the + [monitoring dashboard for a deployed environment](../../ci/environments.md#monitoring-environments) + to view the metrics of your deployed application. To view the metrics of the + whole Kubernetes cluster, navigate to **Operations > Metrics**.  @@ -1303,7 +1326,6 @@ curl --data "value=true" --header "PRIVATE-TOKEN: personal_access_token" https:/ ``` [ce-37115]: https://gitlab.com/gitlab-org/gitlab-foss/issues/37115 -[kubernetes-clusters]: ../../user/project/clusters/index.md [docker-in-docker]: ../../docker/using_docker_build.md#use-docker-in-docker-executor [review-app]: ../../ci/review_apps/index.md [container-registry]: ../../user/packages/container_registry/index.md |