diff options
Diffstat (limited to 'doc/user/infrastructure')
8 files changed, 134 insertions, 102 deletions
diff --git a/doc/user/infrastructure/clusters/connect/new_eks_cluster.md b/doc/user/infrastructure/clusters/connect/new_eks_cluster.md index 87b8f510289..50899053cad 100644 --- a/doc/user/infrastructure/clusters/connect/new_eks_cluster.md +++ b/doc/user/infrastructure/clusters/connect/new_eks_cluster.md @@ -48,10 +48,13 @@ This project provides you with: ## Register the agent +FLAG: +In GitLab 14.10, a [flag](../../../../administration/feature_flags.md) named `certificate_based_clusters` changed the **Actions** menu to focus on the agent rather than certificates. The flag is [enabled on GitLab.com and self-managed](https://gitlab.com/groups/gitlab-org/configure/-/epics/8). + To create a GitLab agent for Kubernetes: 1. On the left sidebar, select **Infrastructure > Kubernetes clusters**. -1. Select **Actions**. +1. Select **Connect a cluster (agent)**. 1. From the **Select an agent** dropdown list, select `eks-agent` and select **Register an agent**. 1. GitLab generates a registration token for the agent. Securely store this secret token, as you will need it later. 1. GitLab provides an address for the agent server (KAS), which you will also need later. diff --git a/doc/user/infrastructure/clusters/connect/new_gke_cluster.md b/doc/user/infrastructure/clusters/connect/new_gke_cluster.md index 1ed8b0ef350..ab04187284d 100644 --- a/doc/user/infrastructure/clusters/connect/new_gke_cluster.md +++ b/doc/user/infrastructure/clusters/connect/new_gke_cluster.md @@ -6,6 +6,13 @@ info: To determine the technical writer assigned to the Stage/Group associated w # Create a Google GKE cluster +INFO: +Every new Google Cloud Platform (GCP) account receives [$300 in credit](https://console.cloud.google.com/freetrial), +and in partnership with Google, GitLab is able to offer an additional $200 for new +GCP accounts to get started with the GitLab integration with Google Kubernetes Engine. +[Follow this link](https://cloud.google.com/partners/partnercredit/?pcn_code=0014M00001h35gDQAQ#contact-form) +and apply for credit. + Learn how to create a new cluster on Google Kubernetes Engine (GKE) through [Infrastructure as Code (IaC)](../../index.md). This process uses the Google and Kubernetes Terraform providers create GKE clusters. You connect the clusters to GitLab @@ -48,10 +55,13 @@ with defaults for name, location, node count, and Kubernetes version. ## Register the agent +FLAG: +In GitLab 14.10, a [flag](../../../../administration/feature_flags.md) named `certificate_based_clusters` changed the **Actions** menu to focus on the agent rather than certificates. The flag is [enabled on GitLab.com and self-managed](https://gitlab.com/groups/gitlab-org/configure/-/epics/8). + To create a GitLab agent for Kubernetes: 1. On the left sidebar, select **Infrastructure > Kubernetes clusters**. -1. Select **Actions**. +1. Select **Connect a cluster (agent)**. 1. From the **Select an agent** dropdown list, select `gke-agent` and select **Register an agent**. 1. GitLab generates a registration token for the agent. Securely store this secret token, as you will need it later. 1. GitLab provides an address for the agent server (KAS), which you will also need later. diff --git a/doc/user/infrastructure/clusters/deploy/inventory_object.md b/doc/user/infrastructure/clusters/deploy/inventory_object.md index d76ef0b9359..d184d969ace 100644 --- a/doc/user/infrastructure/clusters/deploy/inventory_object.md +++ b/doc/user/infrastructure/clusters/deploy/inventory_object.md @@ -23,7 +23,7 @@ gitops: default_namespace: my-ns ``` -The agent creates an inventory object for every item in the `manifest_projects` list. +The agent creates an inventory object for every item in the `manifest_projects` list. The inventory object is stored in the namespace you specify for `default_namespace`. The name and location of the inventory object is based on: @@ -58,7 +58,7 @@ This action changes the location of the object in the cluster. inventory objects in the same namespace in the future. 1. Ensure the value for `cli-utils.sigs.k8s.io/inventory-id` is unique. This value is used for objects tracked by this inventory object. Their `config.k8s.io/owning-inventory` annotation is set to this value. - + The value doesn't have to match the `name` but it's convenient to set them to the same value. 1. Save the file with the manifest files as a single logical group. diff --git a/doc/user/infrastructure/clusters/migrate_to_gitlab_agent.md b/doc/user/infrastructure/clusters/migrate_to_gitlab_agent.md index 01530422e4a..61ec0a559f0 100644 --- a/doc/user/infrastructure/clusters/migrate_to_gitlab_agent.md +++ b/doc/user/infrastructure/clusters/migrate_to_gitlab_agent.md @@ -18,7 +18,7 @@ in GitLab 14.5. It is expected to be [turned off by default in 15.0](../../../update/deprecations.md#certificate-based-integration-with-kubernetes) and removed in GitLab 15.6. -If you are using the certificate-based integration, you should move to another workflow as soon as possible. +If you are using the certificate-based integration, you should move to another workflow as soon as possible. As a general rule, to migrate clusters that rely on GitLab CI/CD, you can use the [CI/CD workflow](../../clusters/agent/ci_cd_tunnel.md). @@ -60,7 +60,7 @@ To configure your Auto DevOps project to use the GitLab agent: 1. On the left sidebar, select **Infrastructure > Kubernetes clusters**. 1. From the certificate-based clusters section, open the cluster that serves the same environment scope. 1. Select the **Details** tab and disable the cluster. -1. To activate the changes, on the left sidebar, select **CI/CD > Variables > Run pipeline**. +1. To activate the changes, on the left sidebar, select **CI/CD > Pipelines** and then **Run pipeline**. For an example, [view this project](https://gitlab.com/gitlab-examples/ops/gitops-demo/hello-world-service). @@ -70,7 +70,11 @@ Follow the process for the [CI/CD workflow](../../clusters/agent/ci_cd_tunnel.md ## Migrate from GitLab Managed applications -Follow the process to [migrate from GitLab Managed Apps to the cluster management project](../../clusters/migrating_from_gma_to_project_template.md). +[GitLab Managed Apps (GMA)](../../clusters/applications.md#gitlab-managed-apps-deprecated) were deprecated in GitLab 14.0, and +the agent for Kubernetes does not support them. To migrate from GMA to the agent, go through the following steps: + +1. [Migrate from GitLab Managed Apps to a cluster management project](../../clusters/migrating_from_gma_to_project_template.md). +1. [Migrate the cluster management project to use the agent](../../clusters/management_project_template.md). ## Migrate a cluster management project diff --git a/doc/user/infrastructure/iac/index.md b/doc/user/infrastructure/iac/index.md index 3bc7495d4be..bc7a3c0d069 100644 --- a/doc/user/infrastructure/iac/index.md +++ b/doc/user/infrastructure/iac/index.md @@ -6,110 +6,121 @@ info: To determine the technical writer assigned to the Stage/Group associated w # Infrastructure as Code with Terraform and GitLab **(FREE)** -With Terraform in GitLab, you can use GitLab authentication and authorization with -your GitOps and Infrastructure-as-Code (IaC) workflows. -Use these features if you want to collaborate on Terraform code within GitLab or would like to use GitLab as a Terraform state storage that incorporates best practices out of the box. +To manage your infrastructure with GitLab, you can use the integration with +Terraform to define resources that you can version, reuse, and share: + +- Manage low-level components like compute, storage, and networking resources. +- Manage high-level components like DNS entries and SaaS features. +- Incorporate GitOps deployments and Infrastructure-as-Code (IaC) workflows. +- Use GitLab as a Terraform state storage. +- Store and use Terraform modules to simplify common and complex infrastructure patterns. + +<i class="fa fa-youtube-play youtube" aria-hidden="true"></i> Watch [a video overview](https://www.youtube.com/watch?v=iGXjUrkkzDI) of the features GitLab provides with the integration with Terraform. ## Integrate your project with Terraform > SAST test was [introduced](https://gitlab.com/groups/gitlab-org/-/epics/6655) in GitLab 14.6. -In GitLab 14.0 and later, to integrate your project with Terraform, add the following -to your `.gitlab-ci.yml` file: +The integration with GitLab and Terraform happens through GitLab CI/CD. +Use an `include` attribute to add the Terraform template to your project and +customize from there. -```yaml -include: - - template: Terraform.latest.gitlab-ci.yml +To get started, choose the template that best suits your needs: -variables: - # If you do not use the GitLab HTTP backend, remove this line and specify TF_HTTP_* variables - TF_STATE_NAME: default - TF_CACHE_KEY: default - # If your terraform files are in a subdirectory, set TF_ROOT accordingly - # TF_ROOT: terraform/production -``` +- [Latest template](#latest-terraform-template) +- [Stable template and advanced template](#stable-and-advanced-terraform-templates) -The `Terraform.latest.gitlab-ci.yml` template: +All templates: -- Uses the latest [GitLab Terraform image](https://gitlab.com/gitlab-org/terraform-images). -- Uses the [GitLab-managed Terraform state](#gitlab-managed-terraform-state) as +- Use the [GitLab-managed Terraform state](#gitlab-managed-terraform-state) as the Terraform state storage backend. -- Creates [four pipeline stages](https://gitlab.com/gitlab-org/gitlab/-/blob/master/lib/gitlab/ci/templates/Terraform.latest.gitlab-ci.yml): - `test`, `validate`, `build`, and `deploy`. These stages - [run the Terraform commands](https://gitlab.com/gitlab-org/gitlab/-/blob/master/lib/gitlab/ci/templates/Terraform/Base.latest.gitlab-ci.yml) - `test`, `validate`, `plan`, `plan-json`, and `apply`. The `apply` command only runs on the default branch. -- Runs the [Terraform SAST scanner](../../application_security/iac_scanning/index.md#configure-iac-scanning-manually), - that you can disable by creating a `SAST_DISABLED` environment variable and setting it to `1`. +- Trigger four pipeline stages: `test`, `validate`, `build`, and `deploy`. +- Run Terraform commands: `test`, `validate`, `plan`, and `plan-json`. It also runs the `apply` only on the default branch. +- Run the [Terraform SAST scanner](../../application_security/iac_scanning/index.md#configure-iac-scanning-manually). -You can override the values in the default template by updating your `.gitlab-ci.yml` file. +### Latest Terraform template -The latest template might contain breaking changes between major GitLab releases. -For a more stable template, we recommend: +The [latest template](https://gitlab.com/gitlab-org/gitlab/-/blob/master/lib/gitlab/ci/templates/Terraform.latest.gitlab-ci.yml) +is compatible with the most recent GitLab version. It provides the most recent +GitLab features, but can potentially include breaking changes. -- [A ready-to-use version](https://gitlab.com/gitlab-org/gitlab/-/blob/master/lib/gitlab/ci/templates/Terraform.gitlab-ci.yml) -- [A base template for customized setups](https://gitlab.com/gitlab-org/gitlab/-/blob/master/lib/gitlab/ci/templates/Terraform/Base.gitlab-ci.yml) +You can safely use the latest Terraform template: -This video from January 2021 walks you through all the GitLab Terraform integration features: +- If you use GitLab.com. +- If you use a self-managed instance updated with every new GitLab release. -<div class="video-fallback"> - See the video: <a href="https://www.youtube.com/watch?v=iGXjUrkkzDI">Terraform with GitLab</a>. -</div> -<figure class="video-container"> - <iframe src="https://www.youtube.com/embed/iGXjUrkkzDI" frameborder="0" allowfullscreen="true"> </iframe> -</figure> +### Stable and advanced Terraform templates -## GitLab-managed Terraform state +If you use earlier versions of GitLab, you might face incompatibility errors +between the GitLab version and the template version. In this case, you can opt +to use one of these templates: + +- [The stable template](https://gitlab.com/gitlab-org/gitlab/-/blob/master/lib/gitlab/ci/templates/Terraform.gitlab-ci.yml) with an skeleton that you can built on top of. +- [The advanced template](https://gitlab.com/gitlab-org/gitlab/-/blob/master/lib/gitlab/ci/templates/Terraform/Base.gitlab-ci.yml) to fully customize your setup. + +### Use a Terraform template + +To use a Terraform template: -[Terraform remote backends](https://www.terraform.io/language/settings/backends) -enable you to store the state file in a remote, shared store. GitLab uses the -[Terraform HTTP backend](https://www.terraform.io/language/settings/backends/http) -to securely store the state files in local storage (the default) or -[the remote store of your choice](../../../administration/terraform_state.md). +1. On the top bar, select **Menu > Projects** and find the project you want to integrate with Terraform. +1. On the left sidebar, select **Repository > Files**. +1. Edit your `.gitlab-ci.yml` file, use the `include` attribute to fetch the Terraform template: -The GitLab-managed Terraform state backend can store your Terraform state easily and -securely. It spares you from setting up additional remote resources like -Amazon S3 or Google Cloud Storage. Its features include: + ```yaml + include: + # To fetch the latest template, use: + - template: Terraform.latest.gitlab-ci.yml + # To fetch the stable template, use: + - template: Terraform/Base.gitlab-ci.yml + # To fetch the advanced template, use: + - template: Terraform/Base.latest.gitlab-ci.yml + ``` -- Supporting encryption of the state file both in transit and at rest. -- Locking and unlocking state. -- Remote Terraform plan and apply execution. +1. Add the variables as described below: -Read more about setting up and [using GitLab-managed Terraform states](terraform_state.md). + ```yaml + variables: + TF_STATE_NAME: default + TF_CACHE_KEY: default + # If your terraform files are in a subdirectory, set TF_ROOT accordingly. For example: + # TF_ROOT: terraform/production + ``` + +1. (Optional) Override in your `.gitlab-ci.yaml` file the attributes present +in the template you fetched to customize your configuration. + +## GitLab-managed Terraform state + +Use the [GitLab-managed Terraform state](terraform_state.md) to store state +files in local storage or in a remote store of your choice. ## Terraform module registry -GitLab can be used as a [Terraform module registry](../../packages/terraform_module_registry/index.md) -to create and publish Terraform modules to a private registry specific to your -top-level namespace. +Use GitLab as a [Terraform module registry](../../packages/terraform_module_registry/index.md) +to create and publish Terraform modules to a private registry. ## Terraform integration in merge requests -Collaborating around Infrastructure as Code (IaC) changes requires both code changes -and expected infrastructure changes to be checked and approved. GitLab provides a -solution to help collaboration around Terraform code changes and their expected -effects using the merge request pages. This way users don't have to build custom -tools or rely on 3rd party solutions to streamline their IaC workflows. - -Read more on setting up and [using the merge request integrations](mr_integration.md). +Use the [Terraform integration in merge requests](mr_integration.md) +to collaborate on Terraform code changes and Infrastructure-as-Code +workflows. ## The GitLab Terraform provider -WARNING: +NOTE: The GitLab Terraform provider is released separately from GitLab. -We are working on migrating the GitLab Terraform provider for GitLab.com. - -You can use the [GitLab Terraform provider](https://github.com/gitlabhq/terraform-provider-gitlab) -to manage various aspects of GitLab using Terraform. The provider is an open source project, -owned by GitLab, where everyone can contribute. +We are working on migrating the GitLab Terraform provider to GitLab.com. -The [documentation of the provider](https://registry.terraform.io/providers/gitlabhq/gitlab/latest/docs) -is available as part of the official Terraform provider documentation. +The [GitLab Terraform provider](https://github.com/gitlabhq/terraform-provider-gitlab) is a plugin for Terraform to facilitate +managing of GitLab resources such as users, groups, and projects. +Its documentation is available on [Terraform](https://registry.terraform.io/providers/gitlabhq/gitlab/latest/docs). ## Create a new cluster through IaC - Learn how to [create a new cluster on Amazon Elastic Kubernetes Service (EKS)](../clusters/connect/new_eks_cluster.md). - Learn how to [create a new cluster on Google Kubernetes Engine (GKE)](../clusters/connect/new_gke_cluster.md). -## Troubleshooting +## Related topics -See the [troubleshooting](troubleshooting.md) documentation. +- [Terraform images](https://gitlab.com/gitlab-org/terraform-images). +- [Troubleshooting](troubleshooting.md) issues with GitLab and Terraform. diff --git a/doc/user/infrastructure/iac/mr_integration.md b/doc/user/infrastructure/iac/mr_integration.md index bcad5c9279a..0fea05a3f03 100644 --- a/doc/user/infrastructure/iac/mr_integration.md +++ b/doc/user/infrastructure/iac/mr_integration.md @@ -23,7 +23,7 @@ recommend encrypting plan output or modifying the project visibility settings. ## Configure Terraform report artifacts -GitLab ships with a [pre-built CI template](index.md#integrate-your-project-with-terraform) that uses GitLab Managed Terraform state and integrates Terraform changes into merge requests. We recommend customizing the pre-built image and relying on the `gitlab-terraform` helper provided within for a quick setup. +GitLab [integrates with Terraform](index.md#integrate-your-project-with-terraform) through CI/CD templates that use GitLab-managed Terraform state and display Terraform changes on merge requests. We recommend customizing the pre-built image and relying on the `gitlab-terraform` helper provided within for a quick setup. To manually configure a GitLab Terraform Report artifact: diff --git a/doc/user/infrastructure/iac/terraform_state.md b/doc/user/infrastructure/iac/terraform_state.md index 39a57b60787..60f97f522cf 100644 --- a/doc/user/infrastructure/iac/terraform_state.md +++ b/doc/user/infrastructure/iac/terraform_state.md @@ -454,29 +454,6 @@ query ProjectTerraformStates { For those new to the GitLab GraphQL API, read [Getting started with GitLab GraphQL API](../../../api/graphql/getting_started.md). -## Troubleshooting +## Related topics -### Unable to lock Terraform state files in CI jobs for `terraform apply` using a plan created in a previous job - -When passing `-backend-config=` to `terraform init`, Terraform persists these values inside the plan -cache file. This includes the `password` value. - -As a result, to create a plan and later use the same plan in another CI job, you might get the error -`Error: Error acquiring the state lock` errors when using `-backend-config=password=$CI_JOB_TOKEN`. -This happens because the value of `$CI_JOB_TOKEN` is only valid for the duration of the current job. - -As a workaround, use [http backend configuration variables](https://www.terraform.io/docs/language/settings/backends/http.html#configuration-variables) in your CI job, -which is what happens behind the scenes when following the -[Get started using GitLab CI](#get-started-using-gitlab-ci) instructions. - -### Error: "address": required field is not set - -By default, we set `TF_ADDRESS` to `${CI_API_V4_URL}/projects/${CI_PROJECT_ID}/terraform/state/${TF_STATE_NAME}`. -If you don't set `TF_STATE_NAME` or `TF_ADDRESS` in your job, the job fails with the error message -`Error: "address": required field is not set`. - -To resolve this, ensure that either `TF_ADDRESS` or `TF_STATE_NAME` is accessible in the -job that returned the error: - -1. Configure the [CI/CD environment scope](../../../ci/variables/#add-a-cicd-variable-to-a-project) for the job. -1. Set the job's [environment](../../../ci/yaml/#environment), matching the environment scope from the previous step. +- [Troubleshooting GitLab-managed Terraform state](troubleshooting.md). diff --git a/doc/user/infrastructure/iac/troubleshooting.md b/doc/user/infrastructure/iac/troubleshooting.md index ecefa20db99..bc0aa39bc70 100644 --- a/doc/user/infrastructure/iac/troubleshooting.md +++ b/doc/user/infrastructure/iac/troubleshooting.md @@ -66,3 +66,30 @@ with better Terraform-specific names. To resolve the syntax error, you can: my-Terraform-job: extends: .terraform:init # The updated name. ``` + +## Troubleshooting Terraform state + +### Unable to lock Terraform state files in CI jobs for `terraform apply` using a plan created in a previous job + +When passing `-backend-config=` to `terraform init`, Terraform persists these values inside the plan +cache file. This includes the `password` value. + +As a result, to create a plan and later use the same plan in another CI job, you might get the error +`Error: Error acquiring the state lock` errors when using `-backend-config=password=$CI_JOB_TOKEN`. +This happens because the value of `$CI_JOB_TOKEN` is only valid for the duration of the current job. + +As a workaround, use [http backend configuration variables](https://www.terraform.io/docs/language/settings/backends/http.html#configuration-variables) in your CI job, +which is what happens behind the scenes when following the +[Get started using GitLab CI](terraform_state.md#get-started-using-gitlab-ci) instructions. + +### Error: "address": required field is not set + +By default, we set `TF_ADDRESS` to `${CI_API_V4_URL}/projects/${CI_PROJECT_ID}/terraform/state/${TF_STATE_NAME}`. +If you don't set `TF_STATE_NAME` or `TF_ADDRESS` in your job, the job fails with the error message +`Error: "address": required field is not set`. + +To resolve this, ensure that either `TF_ADDRESS` or `TF_STATE_NAME` is accessible in the +job that returned the error: + +1. Configure the [CI/CD environment scope](../../../ci/variables/#add-a-cicd-variable-to-a-project) for the job. +1. Set the job's [environment](../../../ci/yaml/#environment), matching the environment scope from the previous step. |