diff options
author | GitLab Bot <gitlab-bot@gitlab.com> | 2020-10-21 10:08:36 +0300 |
---|---|---|
committer | GitLab Bot <gitlab-bot@gitlab.com> | 2020-10-21 10:08:36 +0300 |
commit | 48aff82709769b098321c738f3444b9bdaa694c6 (patch) | |
tree | e00c7c43e2d9b603a5a6af576b1685e400410dee /doc/topics | |
parent | 879f5329ee916a948223f8f43d77fba4da6cd028 (diff) |
Add latest changes from gitlab-org/gitlab@13-5-stable-eev13.5.0-rc42
Diffstat (limited to 'doc/topics')
-rw-r--r-- | doc/topics/autodevops/customize.md | 4 | ||||
-rw-r--r-- | doc/topics/autodevops/index.md | 66 | ||||
-rw-r--r-- | doc/topics/autodevops/stages.md | 27 | ||||
-rw-r--r-- | doc/topics/autodevops/upgrading_auto_deploy_dependencies.md | 262 | ||||
-rw-r--r-- | doc/topics/autodevops/upgrading_chart.md | 75 | ||||
-rw-r--r-- | doc/topics/autodevops/upgrading_postgresql.md | 7 | ||||
-rw-r--r-- | doc/topics/cron/index.md | 2 | ||||
-rw-r--r-- | doc/topics/git/git_rebase.md | 272 | ||||
-rw-r--r-- | doc/topics/git/img/git_rebase_v13_5.png | bin | 0 -> 49048 bytes | |||
-rw-r--r-- | doc/topics/git/index.md | 1 |
10 files changed, 624 insertions, 92 deletions
diff --git a/doc/topics/autodevops/customize.md b/doc/topics/autodevops/customize.md index 13aa8f7e035..0026cf4d18a 100644 --- a/doc/topics/autodevops/customize.md +++ b/doc/topics/autodevops/customize.md @@ -332,7 +332,7 @@ applications. | `AUTO_DEVOPS_CHART_REPOSITORY_USERNAME` | From GitLab 11.11, used to set a username to connect to the Helm repository. Defaults to no credentials. Also set `AUTO_DEVOPS_CHART_REPOSITORY_PASSWORD`. | | `AUTO_DEVOPS_CHART_REPOSITORY_PASSWORD` | From GitLab 11.11, used to set a password to connect to the Helm repository. Defaults to no credentials. Also set `AUTO_DEVOPS_CHART_REPOSITORY_USERNAME`. | | `AUTO_DEVOPS_DEPLOY_DEBUG` | From GitLab 13.1, if this variable is present, Helm will output debug logs. | -| `AUTO_DEVOPS_ALLOW_TO_FORCE_DEPLOY_V<N>` | From [auto-deploy-image](https://gitlab.com/gitlab-org/cluster-integration/auto-deploy-image) v1.0.0, if this variable is present, a new major version of chart is forcibly deployed. [More details](upgrading_chart.md#ignore-warning-and-continue-deploying) | +| `AUTO_DEVOPS_ALLOW_TO_FORCE_DEPLOY_V<N>` | From [auto-deploy-image](https://gitlab.com/gitlab-org/cluster-integration/auto-deploy-image) v1.0.0, if this variable is present, a new major version of chart is forcibly deployed. For more information, see [Ignore warnings and continue deploying](upgrading_auto_deploy_dependencies.md#ignore-warnings-and-continue-deploying). | | `AUTO_DEVOPS_MODSECURITY_SEC_RULE_ENGINE` | From GitLab 12.5, used in combination with [ModSecurity feature flag](../../user/clusters/applications.md#web-application-firewall-modsecurity) to toggle [ModSecurity's `SecRuleEngine`](https://github.com/SpiderLabs/ModSecurity/wiki/Reference-Manual-(v2.x)#SecRuleEngine) behavior. Defaults to `DetectionOnly`. | | `BUILDPACK_URL` | Buildpack's full URL. Can point to either [a Git repository URL or a tarball URL](#custom-buildpacks). | | `CANARY_ENABLED` | From GitLab 11.0, used to define a [deploy policy for canary environments](#deploy-policy-for-canary-environments). | @@ -391,6 +391,8 @@ The following table lists variables used to disable jobs. | `REVIEW_DISABLED` | From GitLab 11.0, used to disable the `review` and the manual `review:stop` job. If the variable is present, these jobs won't be created. | | `SAST_DISABLED` | From GitLab 11.0, used to disable the `sast` job. If the variable is present, the job won't be created. | | `TEST_DISABLED` | From GitLab 11.0, used to disable the `test` job. If the variable is present, the job won't be created. | +| `SECRET_DETECTION_DISABLED` | From GitLab 13.1, used to disable the `secret_detection` job. If the variable is present, the job won't be created. | +| `CODE_INTELLIGENCE_DISABLED` | From GitLab 13.6, used to disable the `code_intelligence` job. If the variable is present, the job won't be created. | ### Application secret variables diff --git a/doc/topics/autodevops/index.md b/doc/topics/autodevops/index.md index a39f93a26e1..1952fadc076 100644 --- a/doc/topics/autodevops/index.md +++ b/doc/topics/autodevops/index.md @@ -3,10 +3,11 @@ > - [Introduced](https://gitlab.com/gitlab-org/gitlab-foss/-/issues/37115) in GitLab 10.0. > - Generally available on GitLab 11.0. -Auto DevOps provides pre-defined CI/CD configuration allowing you to automatically -detect, build, test, deploy, and monitor your applications. Leveraging CI/CD -best practices and tools, Auto DevOps aims to simplify the setup and execution -of a mature and modern software development lifecycle. +Auto DevOps are default CI/CD templates that auto-discover the source code you have. They +enable GitLab to automatically detect, build, test, deploy, and monitor your applications. +Leveraging [CI/CD best practices](../../ci/pipelines/pipeline_efficiency.md) and tools, +Auto DevOps aims to simplify the setup and execution of a mature and modern software +development lifecycle. ## Overview @@ -83,9 +84,9 @@ project in a simple and automatic way: 1. [Auto Build](stages.md#auto-build) 1. [Auto Test](stages.md#auto-test) -1. [Auto Code Quality](stages.md#auto-code-quality) **(STARTER)** -1. [Auto SAST (Static Application Security Testing)](stages.md#auto-sast) **(ULTIMATE)** -1. [Auto Secret Detection](stages.md#auto-secret-detection) **(ULTIMATE)** +1. [Auto Code Quality](stages.md#auto-code-quality) +1. [Auto SAST (Static Application Security Testing)](stages.md#auto-sast) +1. [Auto Secret Detection](stages.md#auto-secret-detection) 1. [Auto Dependency Scanning](stages.md#auto-dependency-scanning) **(ULTIMATE)** 1. [Auto License Compliance](stages.md#auto-license-compliance) **(ULTIMATE)** 1. [Auto Container Scanning](stages.md#auto-container-scanning) **(ULTIMATE)** @@ -94,6 +95,7 @@ project in a simple and automatic way: 1. [Auto Deploy](stages.md#auto-deploy) 1. [Auto Browser Performance Testing](stages.md#auto-browser-performance-testing) **(PREMIUM)** 1. [Auto Monitoring](stages.md#auto-monitoring) +1. [Auto Code Intelligence](stages.md#auto-code-intelligence) As Auto DevOps relies on many different components, you should have a basic knowledge of the following: @@ -317,7 +319,7 @@ metadata: name: gitlab-managed-apps-default-proxy namespace: gitlab-managed-apps spec: - env: + env: - name: http_proxy value: "PUT_YOUR_HTTP_PROXY_HERE" - name: https_proxy @@ -417,6 +419,54 @@ DANGER: **Danger:** Setting `POSTGRES_ENABLED` to `false` permanently deletes any existing channel 1 database for your environment. +### Error: unable to recognize "": no matches for kind "Deployment" in version "extensions/v1beta1" + +After upgrading your Kubernetes cluster to [v1.16+](stages.md#kubernetes-116), +you may encounter this message when deploying with Auto DevOps: + +```plaintext +UPGRADE FAILED +Error: failed decoding reader into objects: unable to recognize "": no matches for kind "Deployment" in version "extensions/v1beta1" +``` + +This can occur if your current deployments on the environment namespace were deployed with a +deprecated/removed API that doesn't exist in Kubernetes v1.16+. For example, +if [your in-cluster PostgreSQL was installed in a legacy way](#detected-an-existing-postgresql-database), +the resource was created via the `extensions/v1beta1` API. However, the deployment resource +was moved to the `app/v1` API in v1.16. + +To recover such outdated resources, you must convert the current deployments by mapping legacy APIs +to newer APIs. There is a helper tool called [`mapkubeapis`](https://github.com/hickeyma/helm-mapkubeapis) +that works for this problem. Follow these steps to use the tool in Auto DevOps: + +1. Modify your `.gitlab-ci.yml` with: + + ```yaml + include: + - template: Auto-DevOps.gitlab-ci.yml + - remote: https://gitlab.com/shinya.maeda/ci-templates/-/raw/master/map-deprecated-api.gitlab-ci.yml + + variables: + HELM_VERSION_FOR_MAPKUBEAPIS: "v2" # If you're using auto-depoy-image v2 or above, please specify "v3". + ``` + +1. Run the job `<environment-name>:map-deprecated-api`. Ensure that this job succeeds before moving + to the next step. You should see something like the following output: + + ```shell + 2020/10/06 07:20:49 Found deprecated or removed Kubernetes API: + "apiVersion: extensions/v1beta1 + kind: Deployment" + Supported API equivalent: + "apiVersion: apps/v1 + kind: Deployment" + ``` + +1. Revert your `.gitlab-ci.yml` to the previous version. You no longer need to include the + supplemental template `map-deprecated-api`. + +1. Continue the deployments as usual. + ## Development guides [Development guide for Auto DevOps](../../development/auto_devops.md) diff --git a/doc/topics/autodevops/stages.md b/doc/topics/autodevops/stages.md index b58c369714e..d3f02889a2e 100644 --- a/doc/topics/autodevops/stages.md +++ b/doc/topics/autodevops/stages.md @@ -124,7 +124,10 @@ The supported buildpacks are: If your application needs a buildpack that is not in the above list, you might want to use a [custom buildpack](customize.md#custom-buildpacks). -## Auto Code Quality **(STARTER)** +## Auto Code Quality + +> - [Introduced](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/1984) in [GitLab Starter](https://about.gitlab.com/pricing/) 9.3. +> - Made [available in all tiers](https://gitlab.com/gitlab-org/gitlab/-/issues/212499) in GitLab 13.2. Auto Code Quality uses the [Code Quality image](https://gitlab.com/gitlab-org/ci-cd/codequality) to run @@ -133,9 +136,10 @@ report, it's uploaded as an artifact which you can later download and check out. The merge request widget also displays any [differences between the source and target branches](../../user/project/merge_requests/code_quality.md). -## Auto SAST **(ULTIMATE)** +## Auto SAST -> Introduced in [GitLab Ultimate](https://about.gitlab.com/pricing/) 10.3. +> - Introduced in [GitLab Ultimate](https://about.gitlab.com/pricing/) 10.3. +> - Select functionality made available in all tiers beginning in 13.1 Static Application Security Testing (SAST) uses the [SAST Docker image](https://gitlab.com/gitlab-org/security-products/sast) to run static @@ -151,9 +155,10 @@ warnings. To learn more about [how SAST works](../../user/application_security/sast/index.md), see the documentation. -## Auto Secret Detection **(ULTIMATE)** +## Auto Secret Detection -> Introduced in [GitLab Ultimate](https://about.gitlab.com/pricing/) 13.1. +> - Introduced in [GitLab Ultimate](https://about.gitlab.com/pricing/) 13.1. +> - [Select functionality made available in all tiers](../../user/application_security/secret_detection/#making-secret-detection-available-to-all-gitlab-tiers) in 13.3 Secret Detection uses the [Secret Detection Docker image](https://gitlab.com/gitlab-org/security-products/analyzers/secrets) to run Secret Detection on the current code, and checks for leaked secrets. The @@ -190,7 +195,7 @@ see the documentation. > Introduced in [GitLab Ultimate](https://about.gitlab.com/pricing/) 11.0. License Compliance uses the -[License Compliance Docker image](https://gitlab.com/gitlab-org/security-products/license-management) +[License Compliance Docker image](https://gitlab.com/gitlab-org/security-products/analyzers/license-finder) to search the project dependencies for their license. The Auto License Compliance stage is skipped on licenses other than [Ultimate](https://about.gitlab.com/pricing/). @@ -242,7 +247,7 @@ Helm, which you can [customize](customize.md#custom-helm-chart). The application into the [Kubernetes namespace](../../user/project/clusters/index.md#deployment-variables) for the environment. -Since GitLab 11.4, [local Tiller](https://gitlab.com/gitlab-org/gitlab-foss/-/merge_requests/22036) is +In GitLab 11.4 and later, [local Tiller](https://gitlab.com/gitlab-org/gitlab-foss/-/merge_requests/22036) is used. Previous versions of GitLab had a Tiller installed in the project namespace. @@ -360,7 +365,7 @@ chart to deploy the application into the [Kubernetes namespace](../../user/project/clusters/index.md#deployment-variables) for the environment. -Since GitLab 11.4, a +In GitLab 11.4 and later, a [local Tiller](https://gitlab.com/gitlab-org/gitlab-foss/-/merge_requests/22036) is used. Previous versions of GitLab had a Tiller installed in the project namespace. @@ -466,7 +471,7 @@ application runs. ### Upgrade auto-deploy-app Chart -You can upgrade auto-deploy-app chart by following the [upgrade guide](upgrading_chart.md). +You can upgrade the auto-deploy-app chart by following the [upgrade guide](upgrading_auto_deploy_dependencies.md). ### Workers @@ -660,3 +665,7 @@ To use Auto Monitoring: whole Kubernetes cluster, navigate to **Operations > Metrics**. ![Auto Metrics](img/auto_monitoring.png) + +## Auto Code Intelligence + +Code Intelligence is powered by [LSIF](https://lsif.dev/) and available for Go at this stage. We'll support more languages as they become available. diff --git a/doc/topics/autodevops/upgrading_auto_deploy_dependencies.md b/doc/topics/autodevops/upgrading_auto_deploy_dependencies.md new file mode 100644 index 00000000000..1aefb6b34df --- /dev/null +++ b/doc/topics/autodevops/upgrading_auto_deploy_dependencies.md @@ -0,0 +1,262 @@ +--- +stage: Release +group: Progressive Delivery +info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/engineering/ux/technical-writing/#designated-technical-writers +type: reference +--- + +# Upgrading deployments for newer Auto Deploy dependencies (Auto Deploy template, auto-deploy-image and auto-deploy-app chart) + +[Auto Deploy](stages.md#auto-deploy) is a feature that deploys your application to a Kubernetes cluster. +It consists of several dependencies: + +- [Auto Deploy template](https://gitlab.com/gitlab-org/gitlab/-/blob/master/lib/gitlab/ci/templates/Jobs/Deploy.gitlab-ci.yml) is a set of pipeline jobs and scripts that makes use of `auto-deploy-image`. +- [`auto-deploy-image`](https://gitlab.com/gitlab-org/cluster-integration/auto-deploy-image) is the executable image that communicates with the Kubernetes cluster. +- [`auto-deploy-app chart`](https://gitlab.com/gitlab-org/cluster-integration/auto-deploy-image/-/tree/master/assets/auto-deploy-app) is the Helm chart for deploying your application. + +The `auto-deploy-image` and `auto-deploy-app` charts use [Semantic Versioning](https://semver.org/). +By default, your Auto DevOps project keeps using the stable and non-breaking version. +However, these dependencies could be upgraded in a major version release of GitLab +with breaking changes requiring you to upgrade your deployments. + +This guide explains how to upgrade your deployments with newer or different major versions of Auto Deploy dependencies. + +## Verify dependency versions + +The process to check the current versions differs depending on which template you +are using. First verify which template is in use: + +- For self-managed instances, the [stable Auto Deploy template bundled with the GitLab package](https://gitlab.com/gitlab-org/gitlab/-/blob/master/lib/gitlab/ci/templates/Jobs/Deploy.gitlab-ci.yml) + is being used. +- [The GitLab.com stable Auto Deploy template](https://gitlab.com/gitlab-org/gitlab/-/blob/master/lib/gitlab/ci/templates/Jobs/Deploy.gitlab-ci.yml) + is being used if **one** of the following is true: + - Your Auto DevOps project doesn't have a `.gitlab-ci.yml` file. + - Your Auto DevOps project has a `.gitlab-ci.yml` and [includes](../../ci/yaml/README.md#includetemplate) + the `Auto-DevOps.gitlab-ci.yml` template. +- [The latest Auto Deploy template](https://gitlab.com/gitlab-org/gitlab/-/blob/master/lib/gitlab/ci/templates/Jobs/Deploy.latest.gitlab-ci.yml) + is being used if **both** of the following is true: + - Your Auto DevOps project has a `.gitlab-ci.yml` file and [includes](../../ci/yaml/README.md#includetemplate) + the `Auto-DevOps.gitlab-ci.yml` template. + - It also includes [the latest Auto Deploy template](#early-adopters) + +If you know what template is being used: + +- The `auto-deploy-image` version is in the template (for example `auto-deploy-image:v1.0.3`). +- The `auto-deploy-app` chart version is [in the auto-deploy-image repository](https://gitlab.com/gitlab-org/cluster-integration/auto-deploy-image/-/blob/v1.0.3/assets/auto-deploy-app/Chart.yaml) + (for example `version: 1.0.3`). + +## Compatibility + +The following table explains the version compatibility between GitLab and Auto Deploy dependencies: + +| GitLab version | `auto-deploy-image` version | Notes | +|------------------|-----------------------------|-------| +| v10.0 to v14.0 | v0.1.0 to v2.0.0 | v0 and v1 auto-deploy-image are backwards compatible. | +| v13.4 and higher | v2.0.0 and higher | v2 auto-deploy-image contains breaking changes, as explained in the [upgrade guide](#upgrade-deployments-to-the-v2-auto-deploy-image). | + +You can find the current stable version of auto-deploy-image in the [Auto Deploy stable template](https://gitlab.com/gitlab-org/gitlab/-/blob/master/lib/gitlab/ci/templates/Jobs/Deploy.gitlab-ci.yml). + +## Upgrade Guide + +Projects using Auto DevOps must use the unmodified chart managed by GitLab. +[Customized charts](customize.md#custom-helm-chart) are unsupported. + +### Upgrade deployments to the v1 `auto-deploy-image` + +The v1 chart is backward compatible with the v0 chart, so no configuration changes are needed. + +### Upgrade deployments to the v2 `auto-deploy-image` + +The v2 auto-deploy-image contains multiple dependency and architectural changes. +If your Auto DevOps project has an active environment deployed with the v1 `auto-deploy-image`, +please proceed with the following upgrade guide. Otherwise, you can skip this process. + +#### Helm 3 + +> [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/228609) in GitLab 13.4. + +The `auto-deploy-image` uses the Helm binary to manipulate the releases. +Previously, `auto-deploy-image` used Helm v2, which used Tiller in a cluster. +In the v2 `auto-deploy-image`, it uses Helm v3 that doesn't require Tiller anymore. + +If your Auto DevOps project has an active environment that was deployed with the v1 +`auto-deploy-image`, use the following steps to upgrade to v2, which uses Helm 3: + +1. Modify your `.gitlab-ci.yml` with: + + ```yaml + include: + - template: Auto-DevOps.gitlab-ci.yml + - remote: https://gitlab.com/hfyngvason/ci-templates/-/raw/master/Helm-2to3.gitlab-ci.yml + + variables: + # If this variable is not present, the migration jobs will not show up + MIGRATE_HELM_2TO3: "true" + + .auto-deploy: + image: registry.gitlab.com/gitlab-org/cluster-integration/auto-deploy-image:v2.0.0-beta.1 + variables: + AUTO_DEVOPS_FORCE_DEPLOY_V2: 1 + ``` + +1. Run the `<environment-name>:helm-2to3:migrate` job. +1. Deploy your environment as usual. This deployment uses Helm 3. +1. If the deployment succeeds, you can safely run `environment:helm-2to3:cleanup`. + This deletes all Helm 2 release data from the namespace. + + If you accidentally delete the Helm 2 releases before you are ready, the `<environment-name>:helm2to3:migrate` + job saves a backup for 1 week in a job artifact called `helm-2-release-backups`. + The backup is in a Kubernetes manifest file that can be restored using + `kubectl apply -f $backup`. +1. Remove the `MIGRATE_HELM_2TO3` variable. + +#### Traffic routing change for canary deployments and incremental rollouts + +> [Introduced](https://gitlab.com/gitlab-org/cluster-integration/auto-deploy-image/-/merge_requests/109) in GitLab 13.4. + +Auto Deploy supports advanced deployment strategies such as [canary deployments](customize.md#deploy-policy-for-canary-environments) +and [incremental rollouts](../../ci/environments/incremental_rollouts.md). + +Previously, `auto-deploy-image` created one service to balance the traffic between +unstable and stable tracks by changing the replica ratio. In the v2 `auto-deploy-image`, +it controls the traffic with [Canary Ingress](https://kubernetes.github.io/ingress-nginx/user-guide/nginx-configuration/annotations/#canary). + +For more details, see the [v2 `auto-deploy-app` chart resource architecture](#v2-chart-resource-architecture). + +If your Auto DevOps project has active `canary` or `rollout` track releases in the +`production` environment deployed with the v1 `auto-deploy-image`, use the following +steps to upgrade to v2: + +1. Verify your project is [using the v1 `auto-deploy-image`](#verify-dependency-versions). + If not, [specify the version](#use-a-specific-version-of-auto-deploy-dependencies). +1. If you're in the process of deploying `canary` or `rollout` deployments, promote + them to `production` first to delete the unstable tracks. +1. Verify your project is [using the v2 `auto-deploy-image`](#verify-dependency-versions). + If not, [specify the version](#use-a-specific-version-of-auto-deploy-dependencies). +1. Add an `AUTO_DEVOPS_FORCE_DEPLOY_V2` environment variable with a value of `true` + in the GitLab CI/CD settings. +1. Create a new pipeline and run the `production` job to renew the resource architecture + with the v2 `auto-deploy-app chart`. +1. Remove the `AUTO_DEVOPS_FORCE_DEPLOY_V2` environment variable. + +### Use a specific version of Auto Deploy dependencies + +To use a specifc version of Auto Deploy dependencies, specify the previous Auto Deploy +stable template that contains the [desired version of `auto-deploy-image` and `auto-deploy-app`](#verify-dependency-versions). + +For example, if the template is bundled in GitLab v13.3, change your `.gitlab-ci.yml` to: + +```yaml +include: + - template: Auto-DevOps.gitlab-ci.yml + - remote: https://gitlab.com/gitlab-org/gitlab/-/blob/v13.3.0-ee/lib/gitlab/ci/templates/Jobs/Deploy.gitlab-ci.yml +``` + +### Ignore warnings and continue deploying + +If you are certain that the new chart version is safe to be deployed, you can add +the `AUTO_DEVOPS_FORCE_DEPLOY_V<major-version-number>` [environment variable](customize.md#build-and-deployment) +to force the deployment to continue. + +For example, if you want to deploy the `v2.0.0` chart on a deployment that previously +used the `v0.17.0` chart, add `AUTO_DEVOPS_FORCE_DEPLOY_V2`. + +## Early adopters + +If you want to use the latest beta or unstable version of `auto-deploy-image`, include +the latest Auto Deploy template into your `.gitlab-ci.yml`: + +```yaml +include: + - template: Auto-DevOps.gitlab-ci.yml + - remote: https://gitlab.com/gitlab-org/gitlab/-/blob/master/lib/gitlab/ci/templates/Jobs/Deploy.latest.gitlab-ci.yml +``` + +CAUTION: **Warning:** +Using a beta or unstable `auto-deploy-image` could cause unrecoverable damage to +your environments. Do not test it with important projects or environments. + +The next stable template update is [planned for GitLab v14.0](https://gitlab.com/gitlab-org/gitlab/-/issues/232788). + +## Resource Architectures of the `auto-deploy-app` chart + +### v0 and v1 chart resource architecture + +```mermaid +graph TD; +subgraph gl-managed-app +Z[Nginx Ingress] +end +Z[Nginx Ingress] --> A(Ingress); +Z[Nginx Ingress] --> B(Ingress); +subgraph stg namespace +B[Ingress] --> H(...); +end +subgraph prd namespace +A[Ingress] --> D(Service); +D[Service] --> E(Deployment:Pods:app:stable); +D[Service] --> F(Deployment:Pods:app:canary); +D[Service] --> I(Deployment:Pods:app:rollout); +E(Deployment:Pods:app:stable)---id1[(Pods:Postgres)] +F(Deployment:Pods:app:canary)---id1[(Pods:Postgres)] +I(Deployment:Pods:app:rollout)---id1[(Pods:Postgres)] +end +``` + +### v2 chart resource architecture + +```mermaid +graph TD; +subgraph gl-managed-app +Z[Nginx Ingress] +end +Z[Nginx Ingress] --> A(Ingress); +Z[Nginx Ingress] --> B(Ingress); +Z[Nginx Ingress] --> |If canary is present or incremental rollout/|J(Canary Ingress); +subgraph stg namespace +B[Ingress] --> H(...); +end +subgraph prd namespace +subgraph stable track +A[Ingress] --> D[Service]; +D[Service] --> E(Deployment:Pods:app:stable); +end +subgraph canary track +J(Canary Ingress) --> K[Service] +K[Service] --> F(Deployment:Pods:app:canary); +end +E(Deployment:Pods:app:stable)---id1[(Pods:Postgres)] +F(Deployment:Pods:app:canary)---id1[(Pods:Postgres)] +end +``` + +## Troubleshooting + +### Major version mismatch warning + +If deploying a chart that has a major version that is different from the previous one, +the new chart might not be correctly deployed. This could be due to an architectural +change. If that happens, the deployment job fails with a message similar to: + +```plaintext +************************************************************************************* + [WARNING] +Detected a major version difference between the the chart that is currently deploying (auto-deploy-app-v0.7.0), and the previously deployed chart (auto-deploy-app-v1.0.0). +A new major version might not be backward compatible with the current release (production). The deployment could fail or be stuck in an unrecoverable status. +... +``` + +To clear this error message and resume deployments, you must do one of the following: + +- Manually [upgrade the chart version](#upgrade-guide). +- [Use a specific chart version](#use-a-specific-version-of-auto-deploy-dependencies). + +### Error: `missing key "app.kubernetes.io/managed-by": must be set to "Helm"` + +If your cluster has a deployment that was deployed with the v1 `auto-deploy-image`, +you might encounter the following error: + +- `Error: rendered manifests contain a resource that already exists. Unable to continue with install: Secret "production-postgresql" in namespace "<project-name>-production" exists and cannot be imported into the current release: invalid ownership metadata; label validation error: missing key "app.kubernetes.io/managed-by": must be set to "Helm"; annotation validation error: missing key "meta.helm.sh/release-name": must be set to "production-postgresql"; annotation validation error: missing key "meta.helm.sh/release-namespace": must be set to "<project-name>-production"` + +This is because the previous deployment was deployed with Helm2, which is not compatible with Helm3. +To resolve the problem, please follow the [upgrade guide](#upgrade-deployments-to-the-v2-auto-deploy-image). diff --git a/doc/topics/autodevops/upgrading_chart.md b/doc/topics/autodevops/upgrading_chart.md index ffa485f6d2c..e4fb84d4509 100644 --- a/doc/topics/autodevops/upgrading_chart.md +++ b/doc/topics/autodevops/upgrading_chart.md @@ -1,72 +1,5 @@ -# Upgrading auto-deploy-app chart for Auto DevOps +--- +redirect_to: 'upgrading_auto_deploy_dependencies.md' +--- -Auto DevOps provides the auto-deploy-app chart for deploying your application to the -Kubernetes cluster with Helm/Tiller. Major version changes of this chart could have -a significantly different resource architecture, and may not be backwards compatible. - -This guide provides instructions on how to upgrade your deployments to use the latest -chart and resource architecture. - -## Compatibility - -The following table lists the version compatibility between GitLab and [auto-deploy-image](https://gitlab.com/gitlab-org/cluster-integration/auto-deploy-image) (with the [auto-deploy-app chart](https://gitlab.com/gitlab-org/cluster-integration/auto-deploy-image/-/tree/master/assets/auto-deploy-app)). - -| GitLab version | auto-deploy-image version | Notes | -|------------------|---------------------------|--------------------------------------------| -| v10.0 and higher | v0.1.0 and higher | v0 and v1 charts are backwards compatible. | - -## Upgrade Guide - -The Auto DevOps project must use the unmodified chart managed by GitLab. -[Customized charts](customize.md#custom-helm-chart) are unsupported. - -### v1 chart - -The v1 chart is backward compatible with the v0 chart, so no configuration changes are needed. - -## Troubleshooting - -### Major version mismatch warning - -If deploying a chart that has a major version that is different from the previous one, -the new chart might not be correctly deployed. This could be due to an architectural -change. If that happens, the deployment job fails with a message similar to: - -```plaintext -************************************************************************************* - [WARNING] -Detected a major version difference between the the chart that is currently deploying (auto-deploy-app-v0.7.0), and the previously deployed chart (auto-deploy-app-v1.0.0). -A new major version might not be backward compatible with the current release (production). The deployment could fail or be stuck in an unrecoverable status. -... -``` - -To clear this error message and resume deployments, you must do one of the following: - -- Manually [upgrade the chart version](#upgrade-guide). -- [Use a specific chart version](#use-a-specific-chart-version). - -#### Use a specific chart version - -To use a specific chart version, you must specify a corresponding version of [auto-deploy-image](https://gitlab.com/gitlab-org/cluster-integration/auto-deploy-image). -Do this by [customizing the image in your `.gitlab-ci.yml`](customize.md#customizing-gitlab-ciyml). - -For example, create the following `.gitlab-ci.yml` file in the project. It configures Auto DevOps -to use [auto-deploy-image](https://gitlab.com/gitlab-org/cluster-integration/auto-deploy-image) version `v0.17.0` -for deployment jobs. It will download the chart from [chart repository](https://charts.gitlab.io/): - -```yaml -include: - - template: Auto-DevOps.gitlab-ci.yml - -.auto-deploy: - image: "registry.gitlab.com/gitlab-org/cluster-integration/auto-deploy-image:v0.17.0" -``` - -#### Ignore warning and continue deploying - -If you are certain that the new chart version is safe to be deployed, -you can add the `AUTO_DEVOPS_FORCE_DEPLOY_V<N>` [environment variable](customize.md#build-and-deployment) -to force the deployment to continue, where `<N>` is the major version. - -For example, if you want to deploy the v2.0.0 chart on a deployment that previously -used the v0.17.0 chart, add `AUTO_DEVOPS_FORCE_DEPLOY_V2`. +This document was moved to [another location](upgrading_auto_deploy_dependencies.md). diff --git a/doc/topics/autodevops/upgrading_postgresql.md b/doc/topics/autodevops/upgrading_postgresql.md index 2ebe362280f..bcbc1d914be 100644 --- a/doc/topics/autodevops/upgrading_postgresql.md +++ b/doc/topics/autodevops/upgrading_postgresql.md @@ -178,8 +178,11 @@ You can also PostgreSQL. 1. Set `AUTO_DEVOPS_POSTGRES_DELETE_V1` to a non-empty value. This flag is a safeguard to prevent accidental deletion of databases. -1. Set `POSTGRES_VERSION` to `11.7`. This is the minimum PostgreSQL - version supported. + <!-- DO NOT REPLACE when upgrading GitLab's supported version. This is NOT related to GitLab's PostgreSQL version support, but the one deployed by Auto DevOps. --> +1. If you have a `POSTGRES_VERSION` set, make sure it is set to `9.6.16` *or +higher*. This is the + minimum PostgreSQL version supported by Auto DevOps. See also the list of + [tags available](https://hub.docker.com/r/bitnami/postgresql/tags). 1. Set `PRODUCTION_REPLICAS` to `0`. For other environments, use `REPLICAS` with an [environment scope](../../ci/environments/index.md#scoping-environments-with-specs). 1. If you have set the `DB_INITIALIZE` or `DB_MIGRATE` variables, either diff --git a/doc/topics/cron/index.md b/doc/topics/cron/index.md index a3dd3b77c22..851dd6d3f77 100644 --- a/doc/topics/cron/index.md +++ b/doc/topics/cron/index.md @@ -24,7 +24,7 @@ Cron scheduling uses a series of five numbers, separated by spaces: # * * * * * <command to execute> ``` -[Source: [Wikipedia](https://en.wikipedia.org/wiki/Cron)] +(Source: [Wikipedia](https://en.wikipedia.org/wiki/Cron)) In cron syntax, the asterisk (`*`) means 'every,' so the following cron strings are valid: diff --git a/doc/topics/git/git_rebase.md b/doc/topics/git/git_rebase.md new file mode 100644 index 00000000000..6f50dea26dd --- /dev/null +++ b/doc/topics/git/git_rebase.md @@ -0,0 +1,272 @@ +--- +stage: Create +group: Source Code +info: "To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/engineering/ux/technical-writing/#designated-technical-writers" +type: concepts, howto +description: "Introduction to Git rebase, force-push, and resolving merge conflicts through the command line." +--- + +# Introduction to Git rebase, force-push, and merge conflicts + +This guide helps you to get started with rebasing, force-pushing, and fixing +merge conflicts locally. + +Before diving into this document, make sure you are familiar with using +[Git through the command line](../../gitlab-basics/start-using-git.md). + +## Git rebase + +[Rebasing](https://git-scm.com/docs/git-rebase) is a very common operation in +Git. There are the following rebase options: + +- [Regular rebase](#regular-rebase). +- [Interactive rebase](#interactive-rebase). + +### Before rebasing + +CAUTION: **Warning:** +`git rebase` rewrites the commit history. It **can be harmful** to do it in +shared branches. It can cause complex and hard to resolve merge conflicts. In +these cases, instead of rebasing your branch against the default branch, +consider pulling it instead (`git pull origin master`). It has a similar +effect without compromising the work of your contributors. + +It's safer to back up your branch before rebasing to make sure you don't lose +any changes. For example, consider a [feature branch](../../gitlab-basics/start-using-git.md#branching) +called `my-feature-branch`: + +1. Open your feature branch in the terminal: + + ```shell + git checkout my-feature-branch + ``` + +1. Checkout a new branch from it: + + ```shell + git checkout -b my-feature-branch-backup + ``` + +1. Go back to your original branch: + + ```shell + git checkout my-feature-branch + ``` + +Now you can safely rebase it. If anything goes wrong, you can recover your +changes by resetting `my-feature-branch` against `my-feature-branch-backup`: + +1. Make sure you're in the correct branch (`my-feature-branch`): + + ```shell + git checkout my-feature-branch + ``` + +1. Reset it against `my-feature-branch-backup`: + + ```shell + git reset --hard my-feature-branch-backup + ``` + +Note that if you added changes to `my-feature-branch` after creating the backup branch, +you will lose them when resetting. + +### Regular rebase + +With a regular rebase you can update your feature branch with the default +branch (or any other branch). +This is an important step for Git-based development strategies. You can +ensure that the changes you're adding to the codebase do not break any +existing changes added to the target branch _after_ you created your feature +branch. + +For example, to update your branch `my-feature-branch` with `master`: + +1. Fetch the latest changes from `master`: + + ```shell + git fetch origin master + ``` + +1. Checkout your feature branch: + + ```shell + git checkout my-feature-branch + ``` + +1. Rebase it against `master`: + + ```shell + git rebase origin/master + ``` + +1. [Force-push](#force-push) to your branch. + +When you rebase: + +1. Git imports all the commits submitted to `master` _after_ the + moment you created your feature branch until the present moment. +1. Git puts the commits you have in your feature branch on top of all + the commits imported from `master`: + +![Git rebase illustration](img/git_rebase_v13_5.png) + +You can replace `master` with any other branch you want to rebase against, for +example, `release-10-3`. You can also replace `origin` with other remote +repositories, for example, `upstream`. To check what remotes you have linked to your local +repository, you can run `git remote -v`. + +If there are [merge conflicts](#merge-conflicts), Git will prompt you to fix +them before continuing the rebase. + +To learn more, check Git's documentation on [rebasing](ttps://git-scm.com/book/en/v2/Git-Branching-Rebasing) +and [rebasing strategies](https://git-scm.com/book/en/v2/Git-Branching-Rebasing). + +### Interactive rebase + +You can use interactive rebase to modify commits. For example, amend a commit +message, squash (join multiple commits into one), edit, or delete +commits. It is handy for changing past commit messages, +as well as for organizing the commit history of your branch to keep it clean. + +TIP: **Tip:** +If you want to keep the default branch commit history clean, you don't need to +manually squash all your commits before merging every merge request; +with [Squash and Merge](../../user/project/merge_requests/squash_and_merge.md) +GitLab does it automatically. + +When you want to change anything in recent commits, use interactive +rebase by passing the flag `--interactive` (or `-i`) to the rebase command. + +For example, if you want to edit the last three commits in your branch +(`HEAD~3`), run: + +```shell +git rebase -i HEAD~3 +``` + +Git opens the last three commits in your terminal text editor and describes +all the interactive rebase options you can use. The default option is `pick`, +which maintains the commit unchanged. Replace the keyword `pick` according to +the operation you want to perform in each commit. To do so, you need to edit +the commits in your terminal's text editor. + +For example, if you're using [Vim](https://www.vim.org/) as the text editor in +a macOS's `ZSH` shell, and you want to **squash** all the three commits +(join them into one): + +1. Press <kbd>i</kbd> on your keyboard to switch to Vim's editing mode. +1. Navigate with your keyboard arrows to edit the **second** commit keyword + from `pick` to `squash` (or `s`). Do the same to the **third** commit. + The first commit should be left **unchanged** (`pick`) as we want to squash + the second and third into the first. +1. Press <kbd>Esc</kbd> to leave the editing mode. +1. Type `:wq` to "write" (save) and "quit". +1. Git outputs the commit message so you have a chance to edit it: + - All lines starting with `#` will be ignored and not included in the commit + message. Everything else will be included. + - To leave it as it is, type `:wq`. To edit the commit message: switch to the + editing mode, edit the commit message, and save it as you just did. +1. If you haven't pushed your commits to the remote branch before rebasing, + push your changes normally. If you had pushed these commits already, + [force-push](#force-push) instead. + +Note that the steps for editing through the command line can be slightly +different depending on your operating system and the shell you're using. + +See [Numerous undo possibilities in Git](numerous_undo_possibilities_in_git/index.md#with-history-modification) +for a deeper look into interactive rebase. + +## Force-push + +When you perform more complex operations, for example, squash commits, reset or +rebase your branch, you'll have to _force_ an update to the remote branch, +since these operations imply rewriting the commit history of the branch. +To force an update, pass the flag `--force` or `-f` to the `push` command. For +example: + +```shell +git push --force origin my-feature-branch +``` + +Forcing an update is **not** recommended when you're working on shared +branches. + +Alternatively, you can pass the flag [`--force-with-lease`](https://git-scm.com/docs/git-push#Documentation/git-push.txt---force-with-leaseltrefnamegt) +instead. It is safer, as it does not overwrite any work on the remote +branch if more commits were added to the remote branch by someone else: + +```shell +git push --force-with-lease origin my-feature-branch +``` + +If the branch you want to force-push is [protected](../../user/project/protected_branches.md), +you can't force-push to it unless you unprotect it first. Then you can +force-push and re-protect it. + +## Merge conflicts + +As Git is based on comparing versions of a file +line-by-line, whenever a line changed in your branch coincides with the same +line changed in the target branch (after the moment you created your feature branch from it), Git +identifies these changes as a merge conflict. To fix it, you need to choose +which version of that line you want to keep. + +Most conflicts can be [resolved through the GitLab UI](../../user/project/merge_requests/resolve_conflicts.md). + +For more complex cases, there are various methods for resolving them. There are +also [Git GUI apps](https://git-scm.com/downloads/guis) that can help by +visualizing the differences. + +To fix conflicts locally, you can use the following method: + +1. Open the terminal and checkout your feature branch, for example, `my-feature-branch`: + + ```shell + git checkout my-feature-branch + ``` + +1. [Rebase](#regular-rebase) your branch against the target branch so Git + prompts you with the conflicts: + + ```shell + git rebase origin/master + ``` + +1. Open the conflicting file in a code editor of your preference. +1. Look for the conflict block: + - It begins with the marker: `<<<<<<< HEAD`. + - Below, there is the content with your changes. + - The marker: `=======` indicates the end of your changes. + - Below, there's the content of the latest changes in the target branch. + - The marker `>>>>>>>` indicates the end of the conflict. +1. Edit the file: choose which version (before or after `=======`) you want to + keep, and then delete the portion of the content you don't want in the file. +1. Delete the markers. +1. Save the file. +1. Repeat the process if there are other conflicting files. +1. Stage your changes: + + ```shell + git add . + ``` + +1. Commit your changes: + + ```shell + git commit -m "Fix merge conflicts" + ``` + +1. Continue rebasing: + + ```shell + git rebase --continue + ``` + + CAUTION: **Caution:** + Up to this point, you can run `git rebase --abort` to stop the process. + Git aborts the rebase and rolls back the branch to the state you had before + running `git rebase`. + Once you run `git rebase --continue` the rebase **cannot** be aborted. + +1. [Force-push](#force-push) to your remote branch. diff --git a/doc/topics/git/img/git_rebase_v13_5.png b/doc/topics/git/img/git_rebase_v13_5.png Binary files differnew file mode 100644 index 00000000000..ff29fa97798 --- /dev/null +++ b/doc/topics/git/img/git_rebase_v13_5.png diff --git a/doc/topics/git/index.md b/doc/topics/git/index.md index 92181fb7bb0..cb2d7b74522 100644 --- a/doc/topics/git/index.md +++ b/doc/topics/git/index.md @@ -81,6 +81,7 @@ If you have problems with Git, the following may help: The following are advanced topics for those who want to get the most out of Git: +- [Introduction to Git rebase, force-push, and merge conflicts](git_rebase.md) - [Server Hooks](../../administration/server_hooks.md) - [Git Attributes](../../user/project/git_attributes.md) - Git Submodules: [Using Git submodules with GitLab CI](../../ci/git_submodules.md#using-git-submodules-with-gitlab-ci) |