Add latest changes from gitlab-org/gitlab@master

1 files changed, 63 insertions, 338 deletions

diff --git a/doc/user/project/clusters/index.md b/doc/user/project/clusters/index.md
index 26995809d84..fc946fc785c 100644
--- a/doc/user/project/clusters/index.md
+++ b/doc/user/project/clusters/index.md
@@ -28,12 +28,11 @@ Using the GitLab project Kubernetes integration, you can:
 - Use [Deploy Boards](#deploy-boards-premium). **(PREMIUM)**
 - Use [Canary Deployments](#canary-deployments-premium). **(PREMIUM)**
 - View [Pod logs](#pod-logs-ultimate). **(ULTIMATE)**
-
-You can also:
-
-- Connect and deploy to an [Amazon EKS cluster](eks_and_gitlab/index.html).
 - Run serverless workloads on [Kubernetes with Knative](serverless/index.md).
 
+See [Adding and removing Kubernetes clusters](add_remove_clusters.md) for details on how to
+set up integrations.
+
 ### Deploy Boards **(PREMIUM)**
 
 GitLab's Deploy Boards offer a consolidated view of the current health and
@@ -98,240 +97,10 @@ pods are annotated with:
 `$CI_ENVIRONMENT_SLUG` and `$CI_PROJECT_PATH_SLUG` are the values of
 the CI variables.
 
-## Adding and removing clusters
-
-There are two options when adding a new cluster to your project:
-
-- Associate your account with Google Kubernetes Engine (GKE) to
-  [create new clusters](#add-new-gke-cluster) from within GitLab.
-- Provide credentials to an
-  [existing Kubernetes cluster](#add-existing-kubernetes-cluster).
-
-### Add new GKE cluster
-
-TIP: **Tip:**
-Every new Google Cloud Platform (GCP) account receives [$300 in credit upon sign up](https://console.cloud.google.com/freetrial),
-and in partnership with Google, GitLab is able to offer an additional $200 for new GCP accounts to get started with GitLab's
-Google Kubernetes Engine Integration. All you have to do is [follow this link](https://cloud.google.com/partners/partnercredit/?PCN=a0n60000006Vpz4AAC) and apply for credit.
-
-NOTE: **Note:**
-The [Google authentication integration](../../../integration/google.md) must
-be enabled in GitLab at the instance level. If that's not the case, ask your
-GitLab administrator to enable it. On GitLab.com, this is enabled.
-
-#### Requirements
-
-Before creating your first cluster on Google Kubernetes Engine with GitLab's
-integration, make sure the following requirements are met:
-
-- A [billing account](https://cloud.google.com/billing/docs/how-to/manage-billing-account)
-  is set up and you have permissions to access it.
-- The Kubernetes Engine API and related service are enabled. It should work immediately but may take up to 10 minutes after you create a project. For more information see the
-  ["Before you begin" section of the Kubernetes Engine docs](https://cloud.google.com/kubernetes-engine/docs/quickstart#before-you-begin).
-
-#### Creating the cluster
-
-If all of the above requirements are met, you can proceed to create and add a
-new Kubernetes cluster to your project:
-
-1. Navigate to your project's **Operations > Kubernetes** page.
-
-   NOTE: **Note:**
-   You need Maintainer [permissions](../../permissions.md) and above to access the Kubernetes page.
-
-1. Click **Add Kubernetes cluster**.
-1. Click **Create with Google Kubernetes Engine**.
-1. Connect your Google account if you haven't done already by clicking the
-   **Sign in with Google** button.
-1. From there on, choose your cluster's settings:
-   - **Kubernetes cluster name** - The name you wish to give the cluster.
-   - **Environment scope** - The [associated environment](#setting-the-environment-scope-premium) to this cluster.
-   - **Google Cloud Platform project** - Choose the project you created in your GCP
-     console that will host the Kubernetes cluster. Learn more about
-     [Google Cloud Platform projects](https://cloud.google.com/resource-manager/docs/creating-managing-projects).
-   - **Zone** - Choose the [region zone](https://cloud.google.com/compute/docs/regions-zones/)
-     under which the cluster will be created.
-   - **Number of nodes** - Enter the number of nodes you wish the cluster to have.
-   - **Machine type** - The [machine type](https://cloud.google.com/compute/docs/machine-types)
-     of the Virtual Machine instance that the cluster will be based on.
-   - **Enable Cloud Run on GKE (beta)** - Check this if you want to use Cloud Run on GKE for this cluster. See the [Cloud Run on GKE section](#cloud-run-on-gke) for more information.
-   - **GitLab-managed cluster** - Leave this checked if you want GitLab to manage namespaces and service accounts for this cluster. See the [Managed clusters section](#gitlab-managed-clusters) for more information.
-1. Finally, click the **Create Kubernetes cluster** button.
-
-After a couple of minutes, your cluster will be ready to go. You can now proceed
-to install some [pre-defined applications](#installing-applications).
-
-NOTE: **Note:**
-GitLab requires basic authentication enabled and a client certificate issued for
-the cluster in order to setup an [initial service
-account](#access-controls). Starting from [GitLab
-11.10](https://gitlab.com/gitlab-org/gitlab-foss/issues/58208), the cluster
-creation process will explicitly request that basic authentication and
-client certificate is enabled.
-
-NOTE: **Note:**
-Starting from [GitLab 12.1](https://gitlab.com/gitlab-org/gitlab-foss/issues/55902), all GKE clusters created by GitLab are RBAC enabled. Take a look at the [RBAC section](#rbac-cluster-resources) for more information.
-
-### Add existing Kubernetes cluster
-
-NOTE: **Note:**
-Kubernetes integration is not supported for arm64 clusters. See the issue [Helm Tiller fails to install on arm64 cluster](https://gitlab.com/gitlab-org/gitlab-foss/issues/64044) for details.
-
-To add an existing Kubernetes cluster to your project:
-
-1. Navigate to your project's **Operations > Kubernetes** page.
-
-   NOTE: **Note:**
-   You need Maintainer [permissions](../../permissions.md) and above to access the Kubernetes page.
-
-1. Click **Add Kubernetes cluster**.
-1. Click **Add an existing Kubernetes cluster** and fill in the details:
-   - **Kubernetes cluster name** (required) - The name you wish to give the cluster.
-   - **Environment scope** (required) - The
-     [associated environment](#setting-the-environment-scope-premium) to this cluster.
-   - **API URL** (required) -
-     It's the URL that GitLab uses to access the Kubernetes API. Kubernetes
-     exposes several APIs, we want the "base" URL that is common to all of them,
-     e.g., `https://kubernetes.example.com` rather than `https://kubernetes.example.com/api/v1`.
-
-     Get the API URL by running this command:
-
-     ```sh
-     kubectl cluster-info | grep 'Kubernetes master' | awk '/http/ {print $NF}'
-     ```
-
-   - **CA certificate** (required) - A valid Kubernetes certificate is needed to authenticate to the cluster. We will use the certificate created by default.
-     - List the secrets with `kubectl get secrets`, and one should named similar to
-      `default-token-xxxxx`. Copy that token name for use below.
-     - Get the certificate by running this command:
-
-       ```sh
-       kubectl get secret <secret name> -o jsonpath="{['data']['ca\.crt']}" | base64 --decode
-       ```
-
-       NOTE: **Note:**
-       If the command returns the entire certificate chain, you need copy the *root ca*
-       certificate at the bottom of the chain.
-
-   - **Token** -
-     GitLab authenticates against Kubernetes using service tokens, which are
-     scoped to a particular `namespace`.
-     **The token used should belong to a service account with
-     [`cluster-admin`](https://kubernetes.io/docs/reference/access-authn-authz/rbac/#user-facing-roles)
-     privileges.** To create this service account:
-
-     1. Create a file called `gitlab-admin-service-account.yaml` with contents:
-
-        ```yaml
-        apiVersion: v1
-        kind: ServiceAccount
-        metadata:
-          name: gitlab-admin
-          namespace: kube-system
-        ---
-        apiVersion: rbac.authorization.k8s.io/v1beta1
-        kind: ClusterRoleBinding
-        metadata:
-          name: gitlab-admin
-        roleRef:
-          apiGroup: rbac.authorization.k8s.io
-          kind: ClusterRole
-          name: cluster-admin
-        subjects:
-        - kind: ServiceAccount
-          name: gitlab-admin
-          namespace: kube-system
-        ```
-
-     1. Apply the service account and cluster role binding to your cluster:
-
-        ```bash
-        kubectl apply -f gitlab-admin-service-account.yaml
-        ```
-
-        Output:
-
-        ```bash
-        serviceaccount "gitlab-admin" created
-        clusterrolebinding "gitlab-admin" created
-        ```
-
-     1. Retrieve the token for the `gitlab-admin` service account:
-
-        ```bash
-        kubectl -n kube-system describe secret $(kubectl -n kube-system get secret | grep gitlab-admin | awk '{print $1}')
-        ```
-
-        Copy the `<authentication_token>` value from the output:
-
-        ```yaml
-        Name:         gitlab-admin-token-b5zv4
-        Namespace:    kube-system
-        Labels:       <none>
-        Annotations:  kubernetes.io/service-account.name=gitlab-admin
-                      kubernetes.io/service-account.uid=bcfe66ac-39be-11e8-97e8-026dce96b6e8
-
-        Type:  kubernetes.io/service-account-token
-
-        Data
-        ====
-        ca.crt:     1025 bytes
-        namespace:  11 bytes
-        token:      <authentication_token>
-        ```
-
-     NOTE: **Note:**
-     For GKE clusters, you will need the
-     `container.clusterRoleBindings.create` permission to create a cluster
-     role binding. You can follow the [Google Cloud
-     documentation](https://cloud.google.com/iam/docs/granting-changing-revoking-access)
-     to grant access.
-
-   - **GitLab-managed cluster** - Leave this checked if you want GitLab to manage namespaces and service accounts for this cluster. See the [Managed clusters section](#gitlab-managed-clusters) for more information.
-
-   - **Project namespace** (optional) - You don't have to fill it in; by leaving
-     it blank, GitLab will create one for you. Also:
-     - Each project should have a unique namespace.
-     - The project namespace is not necessarily the namespace of the secret, if
-       you're using a secret with broader permissions, like the secret from `default`.
-     - You should **not** use `default` as the project namespace.
-     - If you or someone created a secret specifically for the project, usually
-       with limited permissions, the secret's namespace and project namespace may
-       be the same.
-
-1. Finally, click the **Create Kubernetes cluster** button.
-
-After a couple of minutes, your cluster will be ready to go. You can now proceed
-to install some [pre-defined applications](#installing-applications).
-
-### Enabling or disabling integration
-
-After you have successfully added your cluster information, you can enable the
-Kubernetes cluster integration:
-
-1. Click the **Enabled/Disabled** switch
-1. Hit **Save** for the changes to take effect
-
-To disable the Kubernetes cluster integration, follow the same procedure.
-
-### Removing integration
-
-NOTE: **Note:**
-You need Maintainer [permissions](../../permissions.md) and above to remove a Kubernetes cluster integration.
-
-NOTE: **Note:**
-When you remove a cluster, you only remove its relation to GitLab, not the
-cluster itself. To remove the cluster, you can do so by visiting the GKE
-dashboard or using `kubectl`.
-
-To remove the Kubernetes cluster integration from your project, simply click the
-**Remove integration** button. You will then be able to follow the procedure
-and add a Kubernetes cluster again.
-
 ## Cluster configuration
 
-This section covers important considerations for configuring Kubernetes
-clusters with GitLab.
+After [adding a Kubernetes cluster](add_remove_clusters.md) to GitLab, read this section that covers
+important considerations for configuring Kubernetes clusters with GitLab.
 
 ### Security implications
 
@@ -344,15 +113,6 @@ functionalities needed to successfully build and deploy a containerized
 application. Bear in mind that the same credentials are used for all the
 applications running on the cluster.
 
-### Cloud Run on GKE
-
-> [Introduced](https://gitlab.com/gitlab-org/gitlab/merge_requests/16566) in GitLab 12.4.
-
-You can choose to use Cloud Run on GKE in place of installing Knative and Istio
-separately after the cluster has been created. This means that Cloud Run
-(Knative), Istio, and HTTP Load Balancing will be enabled on the cluster at
-create time and cannot be [installed or uninstalled](../../clusters/applications.md) separately.
-
 ### GitLab-managed clusters
 
 > [Introduced](https://gitlab.com/gitlab-org/gitlab-foss/merge_requests/22011) in GitLab 11.5.
@@ -360,7 +120,7 @@ create time and cannot be [installed or uninstalled](../../clusters/applications
 
 You can choose to allow GitLab to manage your cluster for you. If your cluster is
 managed by GitLab, resources for your projects will be automatically created. See the
-[Access controls](#access-controls) section for details on which resources will
+[Access controls](add_remove_clusters.md#access-controls) section for details on which resources will
 be created.
 
 If you choose to manage your own cluster, project-specific resources will not be created
@@ -390,97 +150,6 @@ you can either:
 - Create an `A` record that points to the Ingress IP address with your domain provider.
 - Enter a wildcard DNS address using a service such as nip.io or xip.io. For example, `192.168.1.1.xip.io`.
 
-### Access controls
-
-When creating a cluster in GitLab, you will be asked if you would like to create either:
-
-- An [Attribute-based access control (ABAC)](https://kubernetes.io/docs/reference/access-authn-authz/abac/) cluster.
-- A [Role-based access control (RBAC)](https://kubernetes.io/docs/reference/access-authn-authz/rbac/) cluster.
-
-NOTE: **Note:**
-[RBAC](#rbac-cluster-resources) is recommended and the GitLab default.
-
-GitLab creates the necessary service accounts and privileges to install and run
-[GitLab managed applications](#installing-applications). When GitLab creates the cluster,
-a `gitlab` service account with `cluster-admin` privileges is created in the `default` namespace
-to manage the newly created cluster.
-
-  NOTE: **Note:**
-  Restricted service account for deployment was [introduced](https://gitlab.com/gitlab-org/gitlab-foss/issues/51716) in GitLab 11.5.
-
-When you install Helm into your cluster, the `tiller` service account
-is created with `cluster-admin` privileges in the `gitlab-managed-apps`
-namespace. This service account will be added to the installed Helm Tiller and will
-be used by Helm to install and run [GitLab managed applications](#installing-applications).
-Helm will also create additional service accounts and other resources for each
-installed application. Consult the documentation of the Helm charts for each application
-for details.
-
-If you are [adding an existing Kubernetes cluster](#add-existing-kubernetes-cluster),
-ensure the token of the account has administrator privileges for the cluster.
-
-The resources created by GitLab differ depending on the type of cluster.
-
-#### ABAC cluster resources
-
-GitLab creates the following resources for ABAC clusters.
-
-| Name                  | Type                 | Details                              | Created when               |
-|:----------------------|:---------------------|:-------------------------------------|:---------------------------|
-| `gitlab`              | `ServiceAccount`     | `default` namespace                         | Creating a new GKE Cluster |
-| `gitlab-token`        | `Secret`             | Token for `gitlab` ServiceAccount           | Creating a new GKE Cluster |
-| `tiller`              | `ServiceAccount`     | `gitlab-managed-apps` namespace             | Installing Helm Tiller     |
-| `tiller-admin`        | `ClusterRoleBinding` | `cluster-admin` roleRef                     | Installing Helm Tiller     |
-| Environment namespace | `Namespace`          | Contains all environment-specific resources | Deploying to a cluster     |
-| Environment namespace | `ServiceAccount`     | Uses namespace of environment               | Deploying to a cluster     |
-| Environment namespace | `Secret`             | Token for environment ServiceAccount        | Deploying to a cluster     |
-
-#### RBAC cluster resources
-
-GitLab creates the following resources for RBAC clusters.
-
-| Name                  | Type                 | Details                                                                                                    | Created when               |
-|:----------------------|:---------------------|:-----------------------------------------------------------------------------------------------------------|:---------------------------|
-| `gitlab`              | `ServiceAccount`     | `default` namespace                                                                                        | Creating a new GKE Cluster |
-| `gitlab-admin`        | `ClusterRoleBinding` | [`cluster-admin`](https://kubernetes.io/docs/reference/access-authn-authz/rbac/#user-facing-roles) roleRef | Creating a new GKE Cluster |
-| `gitlab-token`        | `Secret`             | Token for `gitlab` ServiceAccount                                                                          | Creating a new GKE Cluster |
-| `tiller`              | `ServiceAccount`     | `gitlab-managed-apps` namespace                                                                            | Installing Helm Tiller     |
-| `tiller-admin`        | `ClusterRoleBinding` | `cluster-admin` roleRef                                                                                    | Installing Helm Tiller     |
-| Environment namespace | `Namespace`          | Contains all environment-specific resources                                                                | Deploying to a cluster     |
-| Environment namespace | `ServiceAccount`     | Uses namespace of environment                                                                              | Deploying to a cluster     |
-| Environment namespace | `Secret`             | Token for environment ServiceAccount                                                                       | Deploying to a cluster     |
-| Environment namespace | `RoleBinding`        | [`edit`](https://kubernetes.io/docs/reference/access-authn-authz/rbac/#user-facing-roles) roleRef          | Deploying to a cluster     |
-
-NOTE: **Note:**
-Environment-specific resources are only created if your cluster is [managed by GitLab](#gitlab-managed-clusters).
-
-NOTE: **Note:**
-If your cluster was created before GitLab 12.2, it will use a single namespace for all project environments.
-
-#### Security of GitLab Runners
-
-GitLab Runners have the [privileged mode](https://docs.gitlab.com/runner/executors/docker.html#the-privileged-mode)
-enabled by default, which allows them to execute special commands and running
-Docker in Docker. This functionality is needed to run some of the
-[Auto DevOps](../../../topics/autodevops/index.md)
-jobs. This implies the containers are running in privileged mode and you should,
-therefore, be aware of some important details.
-
-The privileged flag gives all capabilities to the running container, which in
-turn can do almost everything that the host can do. Be aware of the
-inherent security risk associated with performing `docker run` operations on
-arbitrary images as they effectively have root access.
-
-If you don't want to use GitLab Runner in privileged mode, either:
-
-- Use shared Runners on GitLab.com. They don't have this security issue.
-- Set up your own Runners using configuration described at
-  [Shared Runners](../../gitlab_com/index.md#shared-runners). This involves:
-  1. Making sure that you don't have it installed via
-     [the applications](#installing-applications).
-  1. Installing a Runner
-     [using `docker+machine`](https://docs.gitlab.com/runner/executors/docker_machine.html).
-
 ### Setting the environment scope **(PREMIUM)**
 
 When adding more than one Kubernetes cluster to your project, you need to differentiate
@@ -636,6 +305,62 @@ record](https://en.wikipedia.org/wiki/Wildcard_DNS_record) such as `*.example.co
 in order to be able to reach your apps. If your external endpoint is an IP address,
 use an A record. If your external endpoint is a hostname, use a CNAME record.
 
+#### Deploy services to the cluster
+
+GitLab supports one-click deployment of helpful services to the cluster, many of
+which support Auto DevOps. Back on the Kubernetes cluster screen in GitLab, a
+list of applications is now available to deploy.
+
+First, install Helm Tiller, a package manager for Kubernetes. This enables
+deployment of the other applications.
+
+![Deploy Apps](img/deploy_apps.png)
+
+##### Deploying NGINX Ingress (optional)
+
+Next, if you would like the deployed app to be reachable on the internet, deploy
+the Ingress. Note that this will also cause an
+[Elastic Load Balancer](https://docs.aws.amazon.com/elasticloadbalancing/)
+to be created, which will incur additional AWS costs.
+
+Once installed, you may see a `?` for "Ingress IP Address". This is because the
+created ELB is available at a DNS name, not an IP address. To get the DNS name,
+run:
+
+```sh
+kubectl get service ingress-nginx-ingress-controller -n gitlab-managed-apps -o jsonpath="{.status.loadBalancer.ingress[0].hostname}"
+```
+
+Note that you may see a trailing `%` on some Kubernetes versions, **do not include it**.
+
+The Ingress is now available at this address and will route incoming requests to
+the proper service based on the DNS name in the request. To support this, a
+wildcard DNS CNAME record should be created for the desired domain name. For example,
+`*.myekscluster.com` would point to the Ingress hostname obtained earlier.
+
+![Create DNS](img/create_dns.png)
+
+##### Deploying the GitLab Runner (optional)
+
+If the project is on GitLab.com, free shared Runners are available and you do
+not have to deploy one. If a project specific Runner is desired, or there are no
+shared Runners, it is easy to deploy one.
+
+Simply click on the **Install** button for the GitLab Runner. It is important to
+note that the Runner deployed is set as **privileged**, which means it essentially
+has root access to the underlying machine. This is required to build docker images,
+and so is on by default.
+
+##### Deploying Prometheus (optional)
+
+GitLab is able to monitor applications automatically, utilizing
+[Prometheus](../integrations/prometheus.html). Kubernetes container CPU and
+memory metrics are automatically collected, and response metrics are retrieved
+from NGINX Ingress as well.
+
+To enable monitoring, simply install Prometheus into the cluster with the
+**Install** button.
+
 ## Deploying to a Kubernetes cluster
 
 A Kubernetes cluster can be the destination for a deployment job. If
@@ -658,7 +383,7 @@ GitLab CI/CD build environment.
 | Variable | Description |
 | -------- | ----------- |
 | `KUBE_URL` | Equal to the API URL. |
-| `KUBE_TOKEN` | The Kubernetes token of the [environment service account](#access-controls). |
+| `KUBE_TOKEN` | The Kubernetes token of the [environment service account](add_remove_clusters.md#access-controls). |
 | `KUBE_NAMESPACE` | The Kubernetes namespace is auto-generated if not specified. The default value is `<project_name>-<project_id>-<environment>`. You can overwrite it to use different one if needed, otherwise the `KUBE_NAMESPACE` variable will receive the default value. |
 | `KUBE_CA_PEM_FILE` | Path to a file containing PEM data. Only present if a custom CA bundle was specified. |
 | `KUBE_CA_PEM` | (**deprecated**) Raw PEM data. Only if a custom CA bundle was specified. |