diff options
Diffstat (limited to 'doc/user/infrastructure/index.md')
| -rw-r--r-- | doc/user/infrastructure/index.md | 345 |
1 files changed, 149 insertions, 196 deletions
diff --git a/doc/user/infrastructure/index.md b/doc/user/infrastructure/index.md index 65237bf24e0..c17d1831b50 100644 --- a/doc/user/infrastructure/index.md +++ b/doc/user/infrastructure/index.md @@ -22,136 +22,146 @@ Amazon S3 or Google Cloud Storage. Its features include: - Locking and unlocking state. - Remote Terraform plan and apply execution. -To get started, there are two different options when using GitLab managed Terraform State. +To get started with a GitLab-managed Terraform State, there are two different options: -- Use a local machine -- Use GitLab CI +- [Use a local machine](#get-started-using-local-development). +- [Use GitLab CI](#get-started-using-gitlab-ci). -## Get Started using local development +## Get started using local development -If you are planning to only run `terraform plan` and `terraform apply` commands from your local machine, this is a simple way to get started. +If you plan to only run `terraform plan` and `terraform apply` commands from your +local machine, this is a simple way to get started: -First, create your project on your GitLab instance. +1. Create your project on your GitLab instance. +1. Navigate to **{settings}** **Settings > General** and note your **Project name** + and **Project ID**. +1. Define the Terraform backend in your Terraform project to be: -Next, define the Terraform backend in your Terraform project to be: - -```hcl -terraform { - backend "http" { - } -} -``` - -Finally, you need to run `terraform init` on your local machine and pass in the following options. The below example is using GitLab.com: - -```bash -terraform init \ - -backend-config="address=https://gitlab.com/api/v4/projects/<YOUR-PROJECT-ID>/terraform/state/<YOUR-PROJECT-NAME>" \ - -backend-config="lock_address=https://gitlab.com/api/v4/projects/<YOUR-PROJECT-ID>/terraform/state/<YOUR-PROJECT-NAME>/lock" \ - -backend-config="unlock_address=https://gitlab.com/api/v4/projects/<YOUR-PROJECT-ID>/terraform/state/<YOUR-PROJECT-NAME>/lock" \ - -backend-config="username=<YOUR-USERNAME>" \ - -backend-config="password=<YOUR-ACCESS-TOKEN>" \ - -backend-config="lock_method=POST" \ - -backend-config="unlock_method=DELETE" \ - -backend-config="retry_wait_min=5" -``` - -This will initialize your Terraform state and store that state within your GitLab project. - -NOTE: YOUR-PROJECT-ID and YOUR-PROJECT-NAME can be accessed from the project main page. - -## Get Started using a GitLab CI - -Another route is to leverage GitLab CI to run your `terraform plan` and `terraform apply` commands. - -### Configure the CI variables + ```hcl + terraform { + backend "http" { + } + } + ``` -To use the Terraform backend, [first create a Personal Access Token](../profile/personal_access_tokens.md) with the `api` scope. Keep in mind that the Terraform backend is restricted to tokens with [Maintainer access](../permissions.md) to the repository. +1. Create a [Personal Access Token](../profile/personal_access_tokens.md) with + the `api` scope. The Terraform backend is restricted to users with + [Maintainer access](../permissions.md) to the repository. + +1. On your local machine, run `terraform init`, passing in the following options, + replacing `<YOUR-PROJECT-NAME>`, `<YOUR-PROJECT-ID>`, `<YOUR-USERNAME>` and + `<YOUR-ACCESS-TOKEN>` with the relevant values. This command initializes your + Terraform state, and stores that state within your GitLab project. This example + uses `gitlab.com`: + + ```shell + terraform init \ + -backend-config="address=https://gitlab.com/api/v4/projects/<YOUR-PROJECT-ID>/terraform/state/<YOUR-PROJECT-NAME>" \ + -backend-config="lock_address=https://gitlab.com/api/v4/projects/<YOUR-PROJECT-ID>/terraform/state/<YOUR-PROJECT-NAME>/lock" \ + -backend-config="unlock_address=https://gitlab.com/api/v4/projects/<YOUR-PROJECT-ID>/terraform/state/<YOUR-PROJECT-NAME>/lock" \ + -backend-config="username=<YOUR-USERNAME>" \ + -backend-config="password=<YOUR-ACCESS-TOKEN>" \ + -backend-config="lock_method=POST" \ + -backend-config="unlock_method=DELETE" \ + -backend-config="retry_wait_min=5" + ``` -To keep the Personal Access Token secure, add it as a [CI/CD environment variable](../../ci/variables/README.md). In this example we set ours to the ENV: `GITLAB_TF_PASSWORD`. +Next, [configure the backend](#configure-the-backend). -If you are planning to use the ENV on a branch which is not protected, make sure to set the variable protection settings correctly. +## Get started using GitLab CI -### Configure the Terraform backend +If you don't want to start with local development, you can also use GitLab CI to +run your `terraform plan` and `terraform apply` commands. -Next we need to define the [http backend](https://www.terraform.io/docs/backends/types/http.html). In your Terraform project add the following code block in a `.tf` file such as `backend.tf` or wherever you desire to define the remote backend: +Next, [configure the backend](#configure-the-backend). -```hcl -terraform { - backend "http" { - } -} -``` +## Configure the backend -### Configure the CI YAML file +After executing the `terraform init` command, you must configure the Terraform backend +and the CI YAML file: -Finally, configure a `.gitlab-ci.yaml`, which lives in the root of your project repository. +CAUTION: **Important:** +The Terraform backend is restricted to users with [Maintainer access](../permissions.md) +to the repository. -In our case we are using a pre-built image: +1. In your Terraform project, define the [HTTP backend](https://www.terraform.io/docs/backends/types/http.html) + by adding the following code block in a `.tf` file (such as `backend.tf`) to + define the remote backend: -```yaml -image: - name: hashicorp/terraform:light - entrypoint: - - '/usr/bin/env' - - 'PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin' -``` - -We then define some environment variables to make life easier. `GITLAB_TF_ADDRESS` is the URL of the GitLab instance where this pipeline runs, and `TF_ROOT` is the directory where the Terraform commands must be executed. - -```yaml -variables: - GITLAB_TF_ADDRESS: ${CI_API_V4_URL}/projects/${CI_PROJECT_ID}/terraform/state/${CI_PROJECT_NAME} - TF_ROOT: ${CI_PROJECT_DIR}/environments/cloudflare/production - -cache: - paths: - - .terraform -``` + ```hcl + terraform { + backend "http" { + } + } + ``` -In a `before_script`, pass a `terraform init` call containing configuration parameters. -These parameters correspond to variables required by the -[http backend](https://www.terraform.io/docs/backends/types/http.html): +1. In the root directory of your project repository, configure a `.gitlab-ci.yaml` file. + This example uses a pre-built image: -```yaml -before_script: - - cd ${TF_ROOT} - - terraform --version - - terraform init -backend-config="address=${GITLAB_TF_ADDRESS}" -backend-config="lock_address=${GITLAB_TF_ADDRESS}/lock" -backend-config="unlock_address=${GITLAB_TF_ADDRESS}/lock" -backend-config="username=${GITLAB_USER_LOGIN}" -backend-config="password=${GITLAB_TF_PASSWORD}" -backend-config="lock_method=POST" -backend-config="unlock_method=DELETE" -backend-config="retry_wait_min=5" + ```yaml + image: + name: hashicorp/terraform:light + entrypoint: + - '/usr/bin/env' + - 'PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin' + ``` -stages: - - validate - - build - - test - - deploy +1. In the `.gitlab-ci.yaml` file, define some environment variables to ease development. In this + example, `GITLAB_TF_ADDRESS` is the URL of the GitLab instance where this pipeline + runs, and `TF_ROOT` is the directory where the Terraform commands must be executed: -validate: - stage: validate - script: - - terraform validate + ```yaml + variables: + GITLAB_TF_ADDRESS: ${CI_API_V4_URL}/projects/${CI_PROJECT_ID}/terraform/state/${CI_PROJECT_NAME} + TF_ROOT: ${CI_PROJECT_DIR}/environments/cloudflare/production -plan: - stage: build - script: - - terraform plan - - terraform show + cache: + paths: + - .terraform + ``` -apply: - stage: deploy - environment: - name: production - script: - - terraform apply - dependencies: - - plan - when: manual - only: - - master -``` +1. In a `before_script`, pass a `terraform init` call containing configuration parameters + corresponding to variables required by the + [HTTP backend](https://www.terraform.io/docs/backends/types/http.html): -### Push to GitLab + ```yaml + before_script: + - cd ${TF_ROOT} + - terraform --version + - terraform init -backend-config="address=${GITLAB_TF_ADDRESS}" -backend-config="lock_address=${GITLAB_TF_ADDRESS}/lock" -backend-config="unlock_address=${GITLAB_TF_ADDRESS}/lock" -backend-config="username=gitlab-ci-token" -backend-config="password=${CI_JOB_TOKEN}" -backend-config="lock_method=POST" -backend-config="unlock_method=DELETE" -backend-config="retry_wait_min=5" + + stages: + - validate + - build + - test + - deploy + + validate: + stage: validate + script: + - terraform validate + + plan: + stage: build + script: + - terraform plan + - terraform show + + apply: + stage: deploy + environment: + name: production + script: + - terraform apply + dependencies: + - plan + when: manual + only: + - master + ``` -Pushing your project to GitLab triggers a CI job pipeline, which runs the `terraform init`, `terraform validate`, and `terraform plan` commands automatically. +1. Push your project to GitLab, which triggers a CI job pipeline. This pipeline runs + the `terraform init`, `terraform validate`, and `terraform plan` commands. The output from the above `terraform` commands should be viewable in the job logs. @@ -161,14 +171,14 @@ See [this reference project](https://gitlab.com/nicholasklick/gitlab-terraform-a ## Output Terraform Plan information into a merge request -Using the [GitLab Terraform Report Artifact](../../ci/pipelines/job_artifacts.md#artifactsreportsterraform), +Using the [GitLab Terraform Report artifact](../../ci/pipelines/job_artifacts.md#artifactsreportsterraform), you can expose details from `terraform plan` runs directly into a merge request widget, enabling you to see statistics about the resources that Terraform will create, modify, or destroy. -Let's explore how to configure a GitLab Terraform Report Artifact: +Let's explore how to configure a GitLab Terraform Report artifact: -1. First, for simplicity, let's define a few reusable variables to allow us to +1. For simplicity, let's define a few reusable variables to allow us to refer to these files multiple times: ```yaml @@ -177,96 +187,39 @@ Let's explore how to configure a GitLab Terraform Report Artifact: PLAN_JSON: tfplan.json ``` -1. Next we need to install `jq`, a [lightweight and flexible command-line JSON processor](https://stedolan.github.io/jq/). We will also create an alias for a specific `jq` command that parses out the extact information we want to extract from the `terraform plan` output: - -```yaml -before_script: - - apk --no-cache add jq - - alias convert_report="jq -r '([.resource_changes[]?.change.actions?]|flatten)|{\"create\":(map(select(.==\"create\"))|length),\"update\":(map(select(.==\"update\"))|length),\"delete\":(map(select(.==\"delete\"))|length)}'" -``` - -1. Finally, we define a `script` that runs `terraform plan` and also a `terraform show` which pipes the output and converts the relevant bits into a store variable `PLAN_JSON`. This json is then leveraged to create a [GitLab Terraform Report Artifact](../../ci/pipelines/job_artifacts.md#artifactsreportsterraform). - -The terraform report obtains a Terraform tfplan.json file. The collected Terraform plan report will be uploaded to GitLab as an artifact and will be automatically shown in merge requests. - -```yaml -plan: - stage: build - script: - - terraform plan -out=$PLAN - - terraform show --json $PLAN | convert_report > $PLAN_JSON - artifacts: - name: plan - paths: - - $PLAN - reports: - terraform: $PLAN_JSON -``` - -A full `.gitlab-ci.yaml` file could look like this: - -```yaml -image: - name: hashicorp/terraform:light - entrypoint: - - '/usr/bin/env' - - 'PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin' - -# Default output file for Terraform plan -variables: - GITLAB_TF_ADDRESS: ${CI_API_V4_URL}/projects/${CI_PROJECT_ID}/terraform/state/${CI_PROJECT_NAME} - PLAN: plan.tfplan - PLAN_JSON: tfplan.json - TF_ROOT: ${CI_PROJECT_DIR} - -cache: - paths: - - .terraform - -before_script: - - apk --no-cache add jq - - alias convert_report="jq -r '([.resource_changes[]?.change.actions?]|flatten)|{\"create\":(map(select(.==\"create\"))|length),\"update\":(map(select(.==\"update\"))|length),\"delete\":(map(select(.==\"delete\"))|length)}'" - - cd ${TF_ROOT} - - terraform --version - - terraform init -backend-config="address=${GITLAB_TF_ADDRESS}" -backend-config="lock_address=${GITLAB_TF_ADDRESS}/lock" -backend-config="unlock_address=${GITLAB_TF_ADDRESS}/lock" -backend-config="username=${GITLAB_USER_LOGIN}" -backend-config="password=${GITLAB_TF_PASSWORD}" -backend-config="lock_method=POST" -backend-config="unlock_method=DELETE" -backend-config="retry_wait_min=5" - -stages: - - validate - - build - - deploy +1. Install `jq`, a + [lightweight and flexible command-line JSON processor](https://stedolan.github.io/jq/). +1. Create an alias for a specific `jq` command that parses out the information we + want to extract from the `terraform plan` output: -validate: - stage: validate - script: - - terraform validate + ```yaml + before_script: + - apk --no-cache add jq + - alias convert_report="jq -r '([.resource_changes[]?.change.actions?]|flatten)|{\"create\":(map(select(.==\"create\"))|length),\"update\":(map(select(.==\"update\"))|length),\"delete\":(map(select(.==\"delete\"))|length)}'" + ``` -plan: - stage: build - script: - - terraform plan -out=$PLAN - - terraform show --json $PLAN | convert_report > $PLAN_JSON - artifacts: - name: plan - paths: - - ${TF_ROOT}/plan.tfplan - reports: - terraform: ${TF_ROOT}/tfplan.json +1. Define a `script` that runs `terraform plan` and `terraform show`. These commands + pipe the output and convert the relevant bits into a store variable `PLAN_JSON`. + This JSON is used to create a + [GitLab Terraform Report artifact](../../ci/pipelines/job_artifacts.md#artifactsreportsterraform). + The Terraform report obtains a Terraform `tfplan.json` file. The collected + Terraform plan report is uploaded to GitLab as an artifact, and is shown in merge requests. -# Separate apply job for manual launching Terraform as it can be destructive -# action. -apply: - stage: deploy - environment: - name: production - script: - - terraform apply -input=false $PLAN - dependencies: - - plan - when: manual - only: - - master + ```yaml + plan: + stage: build + script: + - terraform plan -out=$PLAN + - terraform show --json $PLAN | convert_report > $PLAN_JSON + artifacts: + name: plan + paths: + - $PLAN + reports: + terraform: $PLAN_JSON + ``` -``` + For a full example, see [Example `.gitlab-ci.yaml` file](#example-gitlab-ciyaml-file). 1. Running the pipeline displays the widget in the merge request, like this: |