diff options
Diffstat (limited to 'doc/topics/autodevops')
-rw-r--r-- | doc/topics/autodevops/customize.md | 2 | ||||
-rw-r--r-- | doc/topics/autodevops/index.md | 17 | ||||
-rw-r--r-- | doc/topics/autodevops/stages.md | 17 | ||||
-rw-r--r-- | doc/topics/autodevops/upgrading_auto_deploy_dependencies.md | 262 | ||||
-rw-r--r-- | doc/topics/autodevops/upgrading_chart.md | 75 |
5 files changed, 287 insertions, 86 deletions
diff --git a/doc/topics/autodevops/customize.md b/doc/topics/autodevops/customize.md index 13aa8f7e035..45083dfeb64 100644 --- a/doc/topics/autodevops/customize.md +++ b/doc/topics/autodevops/customize.md @@ -332,7 +332,7 @@ applications. | `AUTO_DEVOPS_CHART_REPOSITORY_USERNAME` | From GitLab 11.11, used to set a username to connect to the Helm repository. Defaults to no credentials. Also set `AUTO_DEVOPS_CHART_REPOSITORY_PASSWORD`. | | `AUTO_DEVOPS_CHART_REPOSITORY_PASSWORD` | From GitLab 11.11, used to set a password to connect to the Helm repository. Defaults to no credentials. Also set `AUTO_DEVOPS_CHART_REPOSITORY_USERNAME`. | | `AUTO_DEVOPS_DEPLOY_DEBUG` | From GitLab 13.1, if this variable is present, Helm will output debug logs. | -| `AUTO_DEVOPS_ALLOW_TO_FORCE_DEPLOY_V<N>` | From [auto-deploy-image](https://gitlab.com/gitlab-org/cluster-integration/auto-deploy-image) v1.0.0, if this variable is present, a new major version of chart is forcibly deployed. [More details](upgrading_chart.md#ignore-warning-and-continue-deploying) | +| `AUTO_DEVOPS_ALLOW_TO_FORCE_DEPLOY_V<N>` | From [auto-deploy-image](https://gitlab.com/gitlab-org/cluster-integration/auto-deploy-image) v1.0.0, if this variable is present, a new major version of chart is forcibly deployed. For more information, see [Ignore warnings and continue deploying](upgrading_auto_deploy_dependencies.md#ignore-warnings-and-continue-deploying). | | `AUTO_DEVOPS_MODSECURITY_SEC_RULE_ENGINE` | From GitLab 12.5, used in combination with [ModSecurity feature flag](../../user/clusters/applications.md#web-application-firewall-modsecurity) to toggle [ModSecurity's `SecRuleEngine`](https://github.com/SpiderLabs/ModSecurity/wiki/Reference-Manual-(v2.x)#SecRuleEngine) behavior. Defaults to `DetectionOnly`. | | `BUILDPACK_URL` | Buildpack's full URL. Can point to either [a Git repository URL or a tarball URL](#custom-buildpacks). | | `CANARY_ENABLED` | From GitLab 11.0, used to define a [deploy policy for canary environments](#deploy-policy-for-canary-environments). | diff --git a/doc/topics/autodevops/index.md b/doc/topics/autodevops/index.md index a39f93a26e1..a7bd810abbe 100644 --- a/doc/topics/autodevops/index.md +++ b/doc/topics/autodevops/index.md @@ -3,10 +3,11 @@ > - [Introduced](https://gitlab.com/gitlab-org/gitlab-foss/-/issues/37115) in GitLab 10.0. > - Generally available on GitLab 11.0. -Auto DevOps provides pre-defined CI/CD configuration allowing you to automatically -detect, build, test, deploy, and monitor your applications. Leveraging CI/CD -best practices and tools, Auto DevOps aims to simplify the setup and execution -of a mature and modern software development lifecycle. +Auto DevOps are default CI/CD templates that auto-discover the source code you have. They +enable GitLab to automatically detect, build, test, deploy, and monitor your applications. +Leveraging [CI/CD best practices](../../ci/pipelines/pipeline_efficiency.md) and tools, +Auto DevOps aims to simplify the setup and execution of a mature and modern software +development lifecycle. ## Overview @@ -83,9 +84,9 @@ project in a simple and automatic way: 1. [Auto Build](stages.md#auto-build) 1. [Auto Test](stages.md#auto-test) -1. [Auto Code Quality](stages.md#auto-code-quality) **(STARTER)** -1. [Auto SAST (Static Application Security Testing)](stages.md#auto-sast) **(ULTIMATE)** -1. [Auto Secret Detection](stages.md#auto-secret-detection) **(ULTIMATE)** +1. [Auto Code Quality](stages.md#auto-code-quality) +1. [Auto SAST (Static Application Security Testing)](stages.md#auto-sast) +1. [Auto Secret Detection](stages.md#auto-secret-detection) 1. [Auto Dependency Scanning](stages.md#auto-dependency-scanning) **(ULTIMATE)** 1. [Auto License Compliance](stages.md#auto-license-compliance) **(ULTIMATE)** 1. [Auto Container Scanning](stages.md#auto-container-scanning) **(ULTIMATE)** @@ -317,7 +318,7 @@ metadata: name: gitlab-managed-apps-default-proxy namespace: gitlab-managed-apps spec: - env: + env: - name: http_proxy value: "PUT_YOUR_HTTP_PROXY_HERE" - name: https_proxy diff --git a/doc/topics/autodevops/stages.md b/doc/topics/autodevops/stages.md index b58c369714e..44eebb748a6 100644 --- a/doc/topics/autodevops/stages.md +++ b/doc/topics/autodevops/stages.md @@ -124,7 +124,10 @@ The supported buildpacks are: If your application needs a buildpack that is not in the above list, you might want to use a [custom buildpack](customize.md#custom-buildpacks). -## Auto Code Quality **(STARTER)** +## Auto Code Quality + +> - [Introduced](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/1984) in [GitLab Starter](https://about.gitlab.com/pricing/) 9.3. +> - Made [available in all tiers](https://gitlab.com/gitlab-org/gitlab/-/issues/212499) in GitLab 13.2. Auto Code Quality uses the [Code Quality image](https://gitlab.com/gitlab-org/ci-cd/codequality) to run @@ -133,9 +136,10 @@ report, it's uploaded as an artifact which you can later download and check out. The merge request widget also displays any [differences between the source and target branches](../../user/project/merge_requests/code_quality.md). -## Auto SAST **(ULTIMATE)** +## Auto SAST -> Introduced in [GitLab Ultimate](https://about.gitlab.com/pricing/) 10.3. +> - Introduced in [GitLab Ultimate](https://about.gitlab.com/pricing/) 10.3. +> - Select functionality made available in all tiers beginning in 13.1 Static Application Security Testing (SAST) uses the [SAST Docker image](https://gitlab.com/gitlab-org/security-products/sast) to run static @@ -151,9 +155,10 @@ warnings. To learn more about [how SAST works](../../user/application_security/sast/index.md), see the documentation. -## Auto Secret Detection **(ULTIMATE)** +## Auto Secret Detection -> Introduced in [GitLab Ultimate](https://about.gitlab.com/pricing/) 13.1. +> - Introduced in [GitLab Ultimate](https://about.gitlab.com/pricing/) 13.1. +> - [Select functionality made available in all tiers](../../user/application_security/secret_detection/#making-secret-detection-available-to-all-gitlab-tiers) in 13.3 Secret Detection uses the [Secret Detection Docker image](https://gitlab.com/gitlab-org/security-products/analyzers/secrets) to run Secret Detection on the current code, and checks for leaked secrets. The @@ -466,7 +471,7 @@ application runs. ### Upgrade auto-deploy-app Chart -You can upgrade auto-deploy-app chart by following the [upgrade guide](upgrading_chart.md). +You can upgrade the auto-deploy-app chart by following the [upgrade guide](upgrading_auto_deploy_dependencies.md). ### Workers diff --git a/doc/topics/autodevops/upgrading_auto_deploy_dependencies.md b/doc/topics/autodevops/upgrading_auto_deploy_dependencies.md new file mode 100644 index 00000000000..1aefb6b34df --- /dev/null +++ b/doc/topics/autodevops/upgrading_auto_deploy_dependencies.md @@ -0,0 +1,262 @@ +--- +stage: Release +group: Progressive Delivery +info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/engineering/ux/technical-writing/#designated-technical-writers +type: reference +--- + +# Upgrading deployments for newer Auto Deploy dependencies (Auto Deploy template, auto-deploy-image and auto-deploy-app chart) + +[Auto Deploy](stages.md#auto-deploy) is a feature that deploys your application to a Kubernetes cluster. +It consists of several dependencies: + +- [Auto Deploy template](https://gitlab.com/gitlab-org/gitlab/-/blob/master/lib/gitlab/ci/templates/Jobs/Deploy.gitlab-ci.yml) is a set of pipeline jobs and scripts that makes use of `auto-deploy-image`. +- [`auto-deploy-image`](https://gitlab.com/gitlab-org/cluster-integration/auto-deploy-image) is the executable image that communicates with the Kubernetes cluster. +- [`auto-deploy-app chart`](https://gitlab.com/gitlab-org/cluster-integration/auto-deploy-image/-/tree/master/assets/auto-deploy-app) is the Helm chart for deploying your application. + +The `auto-deploy-image` and `auto-deploy-app` charts use [Semantic Versioning](https://semver.org/). +By default, your Auto DevOps project keeps using the stable and non-breaking version. +However, these dependencies could be upgraded in a major version release of GitLab +with breaking changes requiring you to upgrade your deployments. + +This guide explains how to upgrade your deployments with newer or different major versions of Auto Deploy dependencies. + +## Verify dependency versions + +The process to check the current versions differs depending on which template you +are using. First verify which template is in use: + +- For self-managed instances, the [stable Auto Deploy template bundled with the GitLab package](https://gitlab.com/gitlab-org/gitlab/-/blob/master/lib/gitlab/ci/templates/Jobs/Deploy.gitlab-ci.yml) + is being used. +- [The GitLab.com stable Auto Deploy template](https://gitlab.com/gitlab-org/gitlab/-/blob/master/lib/gitlab/ci/templates/Jobs/Deploy.gitlab-ci.yml) + is being used if **one** of the following is true: + - Your Auto DevOps project doesn't have a `.gitlab-ci.yml` file. + - Your Auto DevOps project has a `.gitlab-ci.yml` and [includes](../../ci/yaml/README.md#includetemplate) + the `Auto-DevOps.gitlab-ci.yml` template. +- [The latest Auto Deploy template](https://gitlab.com/gitlab-org/gitlab/-/blob/master/lib/gitlab/ci/templates/Jobs/Deploy.latest.gitlab-ci.yml) + is being used if **both** of the following is true: + - Your Auto DevOps project has a `.gitlab-ci.yml` file and [includes](../../ci/yaml/README.md#includetemplate) + the `Auto-DevOps.gitlab-ci.yml` template. + - It also includes [the latest Auto Deploy template](#early-adopters) + +If you know what template is being used: + +- The `auto-deploy-image` version is in the template (for example `auto-deploy-image:v1.0.3`). +- The `auto-deploy-app` chart version is [in the auto-deploy-image repository](https://gitlab.com/gitlab-org/cluster-integration/auto-deploy-image/-/blob/v1.0.3/assets/auto-deploy-app/Chart.yaml) + (for example `version: 1.0.3`). + +## Compatibility + +The following table explains the version compatibility between GitLab and Auto Deploy dependencies: + +| GitLab version | `auto-deploy-image` version | Notes | +|------------------|-----------------------------|-------| +| v10.0 to v14.0 | v0.1.0 to v2.0.0 | v0 and v1 auto-deploy-image are backwards compatible. | +| v13.4 and higher | v2.0.0 and higher | v2 auto-deploy-image contains breaking changes, as explained in the [upgrade guide](#upgrade-deployments-to-the-v2-auto-deploy-image). | + +You can find the current stable version of auto-deploy-image in the [Auto Deploy stable template](https://gitlab.com/gitlab-org/gitlab/-/blob/master/lib/gitlab/ci/templates/Jobs/Deploy.gitlab-ci.yml). + +## Upgrade Guide + +Projects using Auto DevOps must use the unmodified chart managed by GitLab. +[Customized charts](customize.md#custom-helm-chart) are unsupported. + +### Upgrade deployments to the v1 `auto-deploy-image` + +The v1 chart is backward compatible with the v0 chart, so no configuration changes are needed. + +### Upgrade deployments to the v2 `auto-deploy-image` + +The v2 auto-deploy-image contains multiple dependency and architectural changes. +If your Auto DevOps project has an active environment deployed with the v1 `auto-deploy-image`, +please proceed with the following upgrade guide. Otherwise, you can skip this process. + +#### Helm 3 + +> [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/228609) in GitLab 13.4. + +The `auto-deploy-image` uses the Helm binary to manipulate the releases. +Previously, `auto-deploy-image` used Helm v2, which used Tiller in a cluster. +In the v2 `auto-deploy-image`, it uses Helm v3 that doesn't require Tiller anymore. + +If your Auto DevOps project has an active environment that was deployed with the v1 +`auto-deploy-image`, use the following steps to upgrade to v2, which uses Helm 3: + +1. Modify your `.gitlab-ci.yml` with: + + ```yaml + include: + - template: Auto-DevOps.gitlab-ci.yml + - remote: https://gitlab.com/hfyngvason/ci-templates/-/raw/master/Helm-2to3.gitlab-ci.yml + + variables: + # If this variable is not present, the migration jobs will not show up + MIGRATE_HELM_2TO3: "true" + + .auto-deploy: + image: registry.gitlab.com/gitlab-org/cluster-integration/auto-deploy-image:v2.0.0-beta.1 + variables: + AUTO_DEVOPS_FORCE_DEPLOY_V2: 1 + ``` + +1. Run the `<environment-name>:helm-2to3:migrate` job. +1. Deploy your environment as usual. This deployment uses Helm 3. +1. If the deployment succeeds, you can safely run `environment:helm-2to3:cleanup`. + This deletes all Helm 2 release data from the namespace. + + If you accidentally delete the Helm 2 releases before you are ready, the `<environment-name>:helm2to3:migrate` + job saves a backup for 1 week in a job artifact called `helm-2-release-backups`. + The backup is in a Kubernetes manifest file that can be restored using + `kubectl apply -f $backup`. +1. Remove the `MIGRATE_HELM_2TO3` variable. + +#### Traffic routing change for canary deployments and incremental rollouts + +> [Introduced](https://gitlab.com/gitlab-org/cluster-integration/auto-deploy-image/-/merge_requests/109) in GitLab 13.4. + +Auto Deploy supports advanced deployment strategies such as [canary deployments](customize.md#deploy-policy-for-canary-environments) +and [incremental rollouts](../../ci/environments/incremental_rollouts.md). + +Previously, `auto-deploy-image` created one service to balance the traffic between +unstable and stable tracks by changing the replica ratio. In the v2 `auto-deploy-image`, +it controls the traffic with [Canary Ingress](https://kubernetes.github.io/ingress-nginx/user-guide/nginx-configuration/annotations/#canary). + +For more details, see the [v2 `auto-deploy-app` chart resource architecture](#v2-chart-resource-architecture). + +If your Auto DevOps project has active `canary` or `rollout` track releases in the +`production` environment deployed with the v1 `auto-deploy-image`, use the following +steps to upgrade to v2: + +1. Verify your project is [using the v1 `auto-deploy-image`](#verify-dependency-versions). + If not, [specify the version](#use-a-specific-version-of-auto-deploy-dependencies). +1. If you're in the process of deploying `canary` or `rollout` deployments, promote + them to `production` first to delete the unstable tracks. +1. Verify your project is [using the v2 `auto-deploy-image`](#verify-dependency-versions). + If not, [specify the version](#use-a-specific-version-of-auto-deploy-dependencies). +1. Add an `AUTO_DEVOPS_FORCE_DEPLOY_V2` environment variable with a value of `true` + in the GitLab CI/CD settings. +1. Create a new pipeline and run the `production` job to renew the resource architecture + with the v2 `auto-deploy-app chart`. +1. Remove the `AUTO_DEVOPS_FORCE_DEPLOY_V2` environment variable. + +### Use a specific version of Auto Deploy dependencies + +To use a specifc version of Auto Deploy dependencies, specify the previous Auto Deploy +stable template that contains the [desired version of `auto-deploy-image` and `auto-deploy-app`](#verify-dependency-versions). + +For example, if the template is bundled in GitLab v13.3, change your `.gitlab-ci.yml` to: + +```yaml +include: + - template: Auto-DevOps.gitlab-ci.yml + - remote: https://gitlab.com/gitlab-org/gitlab/-/blob/v13.3.0-ee/lib/gitlab/ci/templates/Jobs/Deploy.gitlab-ci.yml +``` + +### Ignore warnings and continue deploying + +If you are certain that the new chart version is safe to be deployed, you can add +the `AUTO_DEVOPS_FORCE_DEPLOY_V<major-version-number>` [environment variable](customize.md#build-and-deployment) +to force the deployment to continue. + +For example, if you want to deploy the `v2.0.0` chart on a deployment that previously +used the `v0.17.0` chart, add `AUTO_DEVOPS_FORCE_DEPLOY_V2`. + +## Early adopters + +If you want to use the latest beta or unstable version of `auto-deploy-image`, include +the latest Auto Deploy template into your `.gitlab-ci.yml`: + +```yaml +include: + - template: Auto-DevOps.gitlab-ci.yml + - remote: https://gitlab.com/gitlab-org/gitlab/-/blob/master/lib/gitlab/ci/templates/Jobs/Deploy.latest.gitlab-ci.yml +``` + +CAUTION: **Warning:** +Using a beta or unstable `auto-deploy-image` could cause unrecoverable damage to +your environments. Do not test it with important projects or environments. + +The next stable template update is [planned for GitLab v14.0](https://gitlab.com/gitlab-org/gitlab/-/issues/232788). + +## Resource Architectures of the `auto-deploy-app` chart + +### v0 and v1 chart resource architecture + +```mermaid +graph TD; +subgraph gl-managed-app +Z[Nginx Ingress] +end +Z[Nginx Ingress] --> A(Ingress); +Z[Nginx Ingress] --> B(Ingress); +subgraph stg namespace +B[Ingress] --> H(...); +end +subgraph prd namespace +A[Ingress] --> D(Service); +D[Service] --> E(Deployment:Pods:app:stable); +D[Service] --> F(Deployment:Pods:app:canary); +D[Service] --> I(Deployment:Pods:app:rollout); +E(Deployment:Pods:app:stable)---id1[(Pods:Postgres)] +F(Deployment:Pods:app:canary)---id1[(Pods:Postgres)] +I(Deployment:Pods:app:rollout)---id1[(Pods:Postgres)] +end +``` + +### v2 chart resource architecture + +```mermaid +graph TD; +subgraph gl-managed-app +Z[Nginx Ingress] +end +Z[Nginx Ingress] --> A(Ingress); +Z[Nginx Ingress] --> B(Ingress); +Z[Nginx Ingress] --> |If canary is present or incremental rollout/|J(Canary Ingress); +subgraph stg namespace +B[Ingress] --> H(...); +end +subgraph prd namespace +subgraph stable track +A[Ingress] --> D[Service]; +D[Service] --> E(Deployment:Pods:app:stable); +end +subgraph canary track +J(Canary Ingress) --> K[Service] +K[Service] --> F(Deployment:Pods:app:canary); +end +E(Deployment:Pods:app:stable)---id1[(Pods:Postgres)] +F(Deployment:Pods:app:canary)---id1[(Pods:Postgres)] +end +``` + +## Troubleshooting + +### Major version mismatch warning + +If deploying a chart that has a major version that is different from the previous one, +the new chart might not be correctly deployed. This could be due to an architectural +change. If that happens, the deployment job fails with a message similar to: + +```plaintext +************************************************************************************* + [WARNING] +Detected a major version difference between the the chart that is currently deploying (auto-deploy-app-v0.7.0), and the previously deployed chart (auto-deploy-app-v1.0.0). +A new major version might not be backward compatible with the current release (production). The deployment could fail or be stuck in an unrecoverable status. +... +``` + +To clear this error message and resume deployments, you must do one of the following: + +- Manually [upgrade the chart version](#upgrade-guide). +- [Use a specific chart version](#use-a-specific-version-of-auto-deploy-dependencies). + +### Error: `missing key "app.kubernetes.io/managed-by": must be set to "Helm"` + +If your cluster has a deployment that was deployed with the v1 `auto-deploy-image`, +you might encounter the following error: + +- `Error: rendered manifests contain a resource that already exists. Unable to continue with install: Secret "production-postgresql" in namespace "<project-name>-production" exists and cannot be imported into the current release: invalid ownership metadata; label validation error: missing key "app.kubernetes.io/managed-by": must be set to "Helm"; annotation validation error: missing key "meta.helm.sh/release-name": must be set to "production-postgresql"; annotation validation error: missing key "meta.helm.sh/release-namespace": must be set to "<project-name>-production"` + +This is because the previous deployment was deployed with Helm2, which is not compatible with Helm3. +To resolve the problem, please follow the [upgrade guide](#upgrade-deployments-to-the-v2-auto-deploy-image). diff --git a/doc/topics/autodevops/upgrading_chart.md b/doc/topics/autodevops/upgrading_chart.md index ffa485f6d2c..e4fb84d4509 100644 --- a/doc/topics/autodevops/upgrading_chart.md +++ b/doc/topics/autodevops/upgrading_chart.md @@ -1,72 +1,5 @@ -# Upgrading auto-deploy-app chart for Auto DevOps +--- +redirect_to: 'upgrading_auto_deploy_dependencies.md' +--- -Auto DevOps provides the auto-deploy-app chart for deploying your application to the -Kubernetes cluster with Helm/Tiller. Major version changes of this chart could have -a significantly different resource architecture, and may not be backwards compatible. - -This guide provides instructions on how to upgrade your deployments to use the latest -chart and resource architecture. - -## Compatibility - -The following table lists the version compatibility between GitLab and [auto-deploy-image](https://gitlab.com/gitlab-org/cluster-integration/auto-deploy-image) (with the [auto-deploy-app chart](https://gitlab.com/gitlab-org/cluster-integration/auto-deploy-image/-/tree/master/assets/auto-deploy-app)). - -| GitLab version | auto-deploy-image version | Notes | -|------------------|---------------------------|--------------------------------------------| -| v10.0 and higher | v0.1.0 and higher | v0 and v1 charts are backwards compatible. | - -## Upgrade Guide - -The Auto DevOps project must use the unmodified chart managed by GitLab. -[Customized charts](customize.md#custom-helm-chart) are unsupported. - -### v1 chart - -The v1 chart is backward compatible with the v0 chart, so no configuration changes are needed. - -## Troubleshooting - -### Major version mismatch warning - -If deploying a chart that has a major version that is different from the previous one, -the new chart might not be correctly deployed. This could be due to an architectural -change. If that happens, the deployment job fails with a message similar to: - -```plaintext -************************************************************************************* - [WARNING] -Detected a major version difference between the the chart that is currently deploying (auto-deploy-app-v0.7.0), and the previously deployed chart (auto-deploy-app-v1.0.0). -A new major version might not be backward compatible with the current release (production). The deployment could fail or be stuck in an unrecoverable status. -... -``` - -To clear this error message and resume deployments, you must do one of the following: - -- Manually [upgrade the chart version](#upgrade-guide). -- [Use a specific chart version](#use-a-specific-chart-version). - -#### Use a specific chart version - -To use a specific chart version, you must specify a corresponding version of [auto-deploy-image](https://gitlab.com/gitlab-org/cluster-integration/auto-deploy-image). -Do this by [customizing the image in your `.gitlab-ci.yml`](customize.md#customizing-gitlab-ciyml). - -For example, create the following `.gitlab-ci.yml` file in the project. It configures Auto DevOps -to use [auto-deploy-image](https://gitlab.com/gitlab-org/cluster-integration/auto-deploy-image) version `v0.17.0` -for deployment jobs. It will download the chart from [chart repository](https://charts.gitlab.io/): - -```yaml -include: - - template: Auto-DevOps.gitlab-ci.yml - -.auto-deploy: - image: "registry.gitlab.com/gitlab-org/cluster-integration/auto-deploy-image:v0.17.0" -``` - -#### Ignore warning and continue deploying - -If you are certain that the new chart version is safe to be deployed, -you can add the `AUTO_DEVOPS_FORCE_DEPLOY_V<N>` [environment variable](customize.md#build-and-deployment) -to force the deployment to continue, where `<N>` is the major version. - -For example, if you want to deploy the v2.0.0 chart on a deployment that previously -used the v0.17.0 chart, add `AUTO_DEVOPS_FORCE_DEPLOY_V2`. +This document was moved to [another location](upgrading_auto_deploy_dependencies.md). |