diff options
Diffstat (limited to 'doc/user/infrastructure')
| -rw-r--r-- | doc/user/infrastructure/clusters/connect/new_gke_cluster.md | 94 | ||||
| -rw-r--r-- | doc/user/infrastructure/clusters/deploy/inventory_object.md | 69 | ||||
| -rw-r--r-- | doc/user/infrastructure/index.md | 42 | ||||
| -rw-r--r-- | doc/user/infrastructure/mr_integration.md | 4 | ||||
| -rw-r--r-- | doc/user/infrastructure/terraform_state.md | 2 |
5 files changed, 205 insertions, 6 deletions
diff --git a/doc/user/infrastructure/clusters/connect/new_gke_cluster.md b/doc/user/infrastructure/clusters/connect/new_gke_cluster.md new file mode 100644 index 00000000000..35af316d7ac --- /dev/null +++ b/doc/user/infrastructure/clusters/connect/new_gke_cluster.md @@ -0,0 +1,94 @@ +--- +stage: Configure +group: Configure +info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/engineering/ux/technical-writing/#assignments +--- + +# New GKE cluster through IaC + +Learn how to create a new cluster on Google Kubernetes Engine (GKE) through +[Infrastructure as Code (IaC)](../../index.md). + +This process combines the GitLab Terraform and Google Terraform providers +with Kubernetes to help you create GKE clusters and deploy them through +GitLab. + +This document describes how to set up a [group-level cluster](../../../group/clusters/index.md) on GKE by importing an example project to get you started. +You can then modify the project files according to your needs. + +**Prerequisites:** + +- A GitLab group. +- A GitLab user with the Maintainer role in the group. +- A [GitLab personal access token](../../../profile/personal_access_tokens.md) with `api` access, created by a user with at least the Maintainer role in the group. +- A [Google Cloud Platform (GCP) service account](https://cloud.google.com/docs/authentication/getting-started). + +**Steps:** + +1. [Import the example project](#import-the-example-project). +1. [Add your GCP credentials to GitLab](#add-your-gcp-credentials-to-gitlab). +1. [Configure your project](#configure-your-project). +1. [Deploy your cluster](#deploy-your-cluster). + +## Import the example project + +Start by [importing the example project by URL](../../../project/import/repo_by_url.md). Use `https://gitlab.com/gitlab-org/configure/examples/gitlab-terraform-gke.git` as URL. + +## Add your GCP credentials to GitLab + +After importing the project, you need to set up [CI environment variables](../../../../ci/variables/index.md) to associate your cluster on GCP to your group in GitLab. + +We advise that you [set environment variables through the GitLab UI](../../../../ci/variables/index.md#add-a-cicd-variable-to-a-project) +so that your credentials are not exposed through the code. To do so, follow the steps below. + +### Prepare your credentials on GCP + +1. Create a [GCP service account](https://cloud.google.com/docs/authentication/getting-started) to authenticate GCP with GitLab. It needs the following roles: `Computer Network Viewer`, `Kubernetes Engine Admin`, and `Service Account User`. +1. Download the JSON file with the service account key. +1. On your computer, encode the JSON file to `base64` (replace `/path/to/sa-key.json` to the path to your key): + + ```shell + base64 /path/to/sa-key.json | tr -d \\n` + ``` + +1. Use the output of this command as the `BASE64_GOOGLE_CREDENTIALS` environment variable in the next step. + +### Add your credentials to GitLab as environment variables + +1. On GitLab, from your project's sidebar, go to **Settings > CI/CD** and expand **Variables**. +1. Add your `GITLAB_TOKEN` ([personal access token](../../../profile/personal_access_tokens.md)). +1. Add the variable `BASE64_GOOGLE_CREDENTIALS` from the previous step. + +## Configure your project + +After authenticating with GCP, replace the project's defaults from the example +project with your own. To do so, edit the files as described below. + +Edit `gke.tf`: + +1. **(Required)** Replace the GCP `project` with a unique project name. +1. **(Optional)** Choose the `name` of your cluster. +1. **(Optional)** Choose the `region` and `zone` that you would like to deploy your cluster to. +1. Push the changes to your project's default branch. + +Edit `group_cluster.tf`: + +1. **(Required)**: Replace the `full_path` with the path to your group. +1. **(Optional)**: Choose your cluster base domain through `domain`. +1. **(Optional)**: Choose your environment through `environment_scope`. +1. Push the changes to your project's default branch. + +Refer to the [GitLab Terraform provider](https://registry.terraform.io/providers/gitlabhq/gitlab/latest/docs) and the [Google Terraform provider](https://registry.terraform.io/providers/hashicorp/google/latest/docs/guides/provider_reference) documentation for further resource options. + +## Deploy your cluster + +After adjusting the files in the previous step, manually trigger the deployment of your cluster. In GitLab: + +1. From your project's sidebar, go to **CI/CD > Pipelines**. +1. Select the dropdown icon (**{angle-down}**) next to the play icon (**{play}**). +1. Select **deploy** to manually trigger the deployment job. + +When the pipeline finishes successfully, you can see your new cluster: + +- In GCP: on your [GCP console's Kubernetes list](https://console.cloud.google.com/kubernetes/list). +- In GitLab: from your project's sidebar, select **Infrastructure > Kubernetes clusters**. diff --git a/doc/user/infrastructure/clusters/deploy/inventory_object.md b/doc/user/infrastructure/clusters/deploy/inventory_object.md new file mode 100644 index 00000000000..d5840641aab --- /dev/null +++ b/doc/user/infrastructure/clusters/deploy/inventory_object.md @@ -0,0 +1,69 @@ +--- +stage: Configure +group: Configure +info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/engineering/ux/technical-writing/#assignments +--- + +# Inventory object **(PREMIUM)** + +> [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/332227) in GitLab 14.0. + +An inventory object is a `ConfigMap` object for keeping track of the set of objects applied to a cluster. +When you remove objects from a manifest repository, GitLab Kubernetes Agent uses a corresponding inventory object to +prune (delete) objects from the cluster. + +The GitLab Kubernetes Agent creates an inventory object for each manifest project specified in the +`gitops.manifest_projects` configuration section. The inventory object has to be stored somewhere in the cluster. +The default behavior is: + +- The `namespace` used comes from `gitops.manifest_projects[].default_namespace`. If you don't specify this parameter + explicitly, the inventory object is stored in the `default` namespace. +- The `name` is generated from the numeric project ID of the manifest project and the numeric agent ID. + + This way the GitLab Kubernetes Agent constructs the name and local where the inventory object is + stored in the cluster. + +The GitLab Kubernetes Agent cannot locate the existing inventory object if you: + +- Change `gitops.manifest_projects[].default_namespace` parameter. +- Move manifests into another project. + +## Inventory object template + +The inventory object template is a `ConfigMap` object that allows you to configure the namespace and the name of the inventory +object. Store this template with manifest files in a single group. + +Example inventory object template: + +```yaml +apiVersion: v1 +kind: ConfigMap +metadata: + name: unique-name-for-the-inventory + namespace: my-project-namespace + labels: + cli-utils.sigs.k8s.io/inventory-id: unique-name-for-the-inventory +``` + +- The `namespace` and `name` fields configure where the real inventory object is created. +- The `cli-utils.sigs.k8s.io/inventory-id` label with its corresponding value is set on the inventory object, created + from this template. Make sure that the value is unique (for example, a string of random characters) and doesn't clash + with any existing or future inventory object templates. +- Objects tracked by this inventory object have the `config.k8s.io/owning-inventory` annotation set to the value of + the `cli-utils.sigs.k8s.io/inventory-id` label. +- The label's value doesn't have to match the `name` but it's convenient to have them set to the same value. +- Make sure that the `name` is unique so that it doesn't conflict with another inventory object in the same + namespace in the future. + +## Using GitOps with pre-existing Kubernetes objects + +The GitLab Kubernetes Agent treats manifest files in the manifest repository as the source of truth. When it applies +objects from the files to the cluster, it tracks them in an inventory object. If an object already exists, +GitLab Kubernetes Agent behaves differently based on the `gitops.manifest_projects[].inventory_policy` configuration. +Check the table below with the available options and when to use them. + +`inventory_policy` value | Description | +------------------------ | ------------------------------------------------------------------------------------------- | +`must_match` | This is the default policy. A live object must have the `config.k8s.io/owning-inventory` annotation set to the same value as the `cli-utils.sigs.k8s.io/inventory-id` label on the corresponding inventory object to be updated. Object is not updated and an error is reported if the values don't match or the object doesn't have the annotation. | +`adopt_if_no_inventory` | This mode allows to "adopt" an object if it doesn't have the `config.k8s.io/owning-inventory` annotation. Use this mode if you want to start managing existing objects using the GitOps feature. Once all objects have been "adopted", we recommend you to put the setting back into the default `must_match` mode to avoid any unexpected adoptions. | +`adopt_all` | This mode allows to "adopt" an object even if it has the `config.k8s.io/owning-inventory` annotation set to a different value. This mode can be useful if you want to migrate a set of objects from one agent to another one or from some other tool to the GitLab Kubernetes Agent. Once all objects have been "adopted", we recommend you to put the setting back into the default `must_match` mode to avoid any unexpected adoptions. | diff --git a/doc/user/infrastructure/index.md b/doc/user/infrastructure/index.md index 05ffab93f85..bdaae4b8225 100644 --- a/doc/user/infrastructure/index.md +++ b/doc/user/infrastructure/index.md @@ -16,7 +16,7 @@ GitLab, and support Terraform best practices. ## Quick Start Use the following `.gitlab-ci.yml` to set up a basic Terraform project integration -for GitLab versions 13.5 and later: +for GitLab versions 14.0 and later: ```yaml include: @@ -38,7 +38,7 @@ This template includes some opinionated decisions, which you can override: - Creating [four pipeline stages](https://gitlab.com/gitlab-org/gitlab/-/blob/master/lib/gitlab/ci/templates/Terraform.gitlab-ci.yml): `init`, `validate`, `build`, and `deploy`. These stages [run the Terraform commands](https://gitlab.com/gitlab-org/gitlab/-/blob/master/lib/gitlab/ci/templates/Terraform/Base.gitlab-ci.yml) - `init`, `validate`, `plan`, `plan-json`, and `apply`. The `apply` command only runs on `master`. + `init`, `validate`, `plan`, `plan-json`, and `apply`. The `apply` command only runs on the default branch. This video from January 2021 walks you through all the GitLab Terraform integration features: @@ -89,7 +89,7 @@ tools or rely on 3rd party solutions to streamline their IaC workflows. Read more on setting up and [using the merge request integrations](mr_integration.md). -## The GitLab terraform provider +## The GitLab Terraform provider WARNING: The GitLab Terraform provider is released separately from GitLab. @@ -101,3 +101,39 @@ owned by GitLab, where everyone can contribute. The [documentation of the provider](https://registry.terraform.io/providers/gitlabhq/gitlab/latest/docs) is available as part of the official Terraform provider documentations. + +## Create a new cluster through IaC + +Learn how to [create a new cluster on Google Kubernetes Engine (GKE)](clusters/connect/new_gke_cluster.md). + +## Troubleshooting + +### `gitlab_group_share_group` resources not detected when subgroup state is refreshed + +The GitLab Terraform provider can fail to detect existing `gitlab_group_share_group` resources +due to the issue ["User with permissions cannot retrieve `share_with_groups` from the API"](https://gitlab.com/gitlab-org/gitlab/-/issues/328428). +This results in an error when running `terraform apply` because Terraform attempts to recreate an +existing resource. + +For example, consider the following group/subgroup configuration: + +```plaintext +parent-group +├── subgroup-A +└── subgroup-B +``` + +Where: + +- User `user-1` creates `parent-group`, `subgroup-A`, and `subgroup-B`. +- `subgroup-A` is shared with `subgroup-B`. +- User `terraform-user` is member of `parent-group` with inherited `owner` access to both subgroups. + +When the Terraform state is refreshed, the API query `GET /groups/:subgroup-A_id` issued by the provider does not return the +details of `subgroup-B` in the `shared_with_groups` array. This leads to the error. + +To workaround this issue, make sure to apply one of the following conditions: + +1. The `terraform-user` creates all subgroup resources. +1. Grant Maintainer or Owner role to the `terraform-user` user on `subgroup-B`. +1. The `terraform-user` inherited access to `subgroup-B` and `subgroup-B` contains at least one project. diff --git a/doc/user/infrastructure/mr_integration.md b/doc/user/infrastructure/mr_integration.md index 6f8b1d8d569..66e00bab6ce 100644 --- a/doc/user/infrastructure/mr_integration.md +++ b/doc/user/infrastructure/mr_integration.md @@ -10,7 +10,7 @@ Collaborating around Infrastructure as Code (IaC) changes requires both code cha ## Output Terraform Plan information into a merge request -Using the [GitLab Terraform Report artifact](../../ci/yaml/README.md#artifactsreportsterraform), +Using the [GitLab Terraform Report artifact](../../ci/yaml/index.md#artifactsreportsterraform), you can expose details from `terraform plan` runs directly into a merge request widget, enabling you to see statistics about the resources that Terraform creates, modifies, or destroys. @@ -57,7 +57,7 @@ To manually configure a GitLab Terraform Report artifact requires the following 1. Define a `script` that runs `terraform plan` and `terraform show`. These commands pipe the output and convert the relevant bits into a store variable `PLAN_JSON`. This JSON is used to create a - [GitLab Terraform Report artifact](../../ci/yaml/README.md#artifactsreportsterraform). + [GitLab Terraform Report artifact](../../ci/yaml/index.md#artifactsreportsterraform). The Terraform report obtains a Terraform `tfplan.json` file. The collected Terraform plan report is uploaded to GitLab as an artifact, and is shown in merge requests. diff --git a/doc/user/infrastructure/terraform_state.md b/doc/user/infrastructure/terraform_state.md index 0b92ea46338..57db2b47de4 100644 --- a/doc/user/infrastructure/terraform_state.md +++ b/doc/user/infrastructure/terraform_state.md @@ -242,7 +242,7 @@ An example setup is shown below: Outputs from the data source can now be referenced in your Terraform resources using `data.terraform_remote_state.example.outputs.<OUTPUT-NAME>`. -You need at least [developer access](../permissions.md) to the target project +You need at least the [Developer role](../permissions.md) in the target project to read the Terraform state. ## Migrating to GitLab Managed Terraform state |