diff options
| author | GitLab Bot <gitlab-bot@gitlab.com> | 2021-03-16 21:18:33 +0300 |
|---|---|---|
| committer | GitLab Bot <gitlab-bot@gitlab.com> | 2021-03-16 21:18:33 +0300 |
| commit | f64a639bcfa1fc2bc89ca7db268f594306edfd7c (patch) | |
| tree | a2c3c2ebcc3b45e596949db485d6ed18ffaacfa1 /doc/topics | |
| parent | bfbc3e0d6583ea1a91f627528bedc3d65ba4b10f (diff) | |
Add latest changes from gitlab-org/gitlab@13-10-stable-eev13.10.0-rc40
Diffstat (limited to 'doc/topics')
20 files changed, 673 insertions, 467 deletions
diff --git a/doc/topics/autodevops/customize.md b/doc/topics/autodevops/customize.md index b7f2a0768ef..7dcecb9af1e 100644 --- a/doc/topics/autodevops/customize.md +++ b/doc/topics/autodevops/customize.md @@ -17,7 +17,7 @@ staging and canary deployments, ## Custom buildpacks If the automatic buildpack detection fails for your project, or if you want to -use a custom buildpack, you can override the buildpack using a project variable +use a custom buildpack, you can override the buildpack using a project CI/CD variable or a `.buildpacks` file in your project: - **Project variable** - Create a project variable `BUILDPACK_URL` with the URL @@ -43,7 +43,7 @@ can't use the `.buildpacks` file. The buildpack in the backend to parse the `.buildpacks` file, does not provide the necessary commands `bin/test-compile` and `bin/test`. -If your goal is to use only a single custom buildpack, you should provide the project variable +If your goal is to use only a single custom buildpack, you should provide the project CI/CD variable `BUILDPACK_URL` instead. ## Custom `Dockerfile` @@ -55,13 +55,13 @@ builds a Docker image based on the Dockerfile, rather than using buildpacks. This can be much faster and result in smaller images, especially if your Dockerfile is based on [Alpine](https://hub.docker.com/_/alpine/). -If you set the `DOCKERFILE_PATH` CI variable, Auto Build looks for a Dockerfile there +If you set the `DOCKERFILE_PATH` CI/CD variable, Auto Build looks for a Dockerfile there instead. ## Passing arguments to `docker build` Arguments can be passed to the `docker build` command using the -`AUTO_DEVOPS_BUILD_IMAGE_EXTRA_ARGS` project variable. For example, to build a +`AUTO_DEVOPS_BUILD_IMAGE_EXTRA_ARGS` project CI/CD variable. For example, to build a Docker image based on based on the `ruby:alpine` instead of the default `ruby:latest`: 1. Set `AUTO_DEVOPS_BUILD_IMAGE_EXTRA_ARGS` to `--build-arg=RUBY_VERSION=alpine`. @@ -93,12 +93,12 @@ You can extend and manage your Auto DevOps configuration with GitLab APIs: - [Editing groups](../../api/groups.md#update-group). - [Editing projects](../../api/projects.md#edit-project). -## Forward CI variables to the build environment +## Forward CI/CD variables to the build environment > [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/25514) in GitLab 12.3, but available in versions 11.9 and above. -CI variables can be forwarded into the build environment using the -`AUTO_DEVOPS_BUILD_IMAGE_FORWARDED_CI_VARIABLES` CI variable. +CI/CD variables can be forwarded into the build environment using the +`AUTO_DEVOPS_BUILD_IMAGE_FORWARDED_CI_VARIABLES` CI/CD variable. The forwarded variables should be specified by name in a comma-separated list. For example, to forward the variables `CI_COMMIT_SHA` and `CI_ENVIRONMENT_NAME`, set `AUTO_DEVOPS_BUILD_IMAGE_FORWARDED_CI_VARIABLES` @@ -130,7 +130,7 @@ feature to use the `--secret` flag. Auto DevOps uses [Helm](https://helm.sh/) to deploy your application to Kubernetes. You can override the Helm chart used by bundling up a chart into your project -repository or by specifying a project variable: +repository or by specifying a project CI/CD variable: - **Bundled chart** - If your project has a `./chart` directory with a `Chart.yaml` file in it, Auto DevOps detects the chart and uses it instead of the @@ -151,17 +151,17 @@ You can override the default values in the `values.yaml` file in the - Adding a file named `.gitlab/auto-deploy-values.yaml` to your repository, which is automatically used, if found. - Adding a file with a different name or path to the repository, and setting the - `HELM_UPGRADE_VALUES_FILE` [environment variable](#environment-variables) with + `HELM_UPGRADE_VALUES_FILE` [CI/CD variable](#cicd-variables) with the path and name. NOTE: -For GitLab 12.5 and earlier, use the `HELM_UPGRADE_EXTRA_ARGS` environment variable +For GitLab 12.5 and earlier, use the `HELM_UPGRADE_EXTRA_ARGS` variable to override the default chart values by setting `HELM_UPGRADE_EXTRA_ARGS` to `--values <my-values.yaml>`. ## Customize the `helm upgrade` command You can customize the `helm upgrade` command used in the [auto-deploy-image](https://gitlab.com/gitlab-org/cluster-integration/auto-deploy-image) -by passing options to the command with the `HELM_UPGRADE_EXTRA_ARGS` variable. +by passing options to the command with the `HELM_UPGRADE_EXTRA_ARGS` CI/CD variable. For example, set the value of `HELM_UPGRADE_EXTRA_ARGS` to `--no-hooks` to disable pre-upgrade and post-upgrade hooks when the command is executed. @@ -170,7 +170,7 @@ list of options. ## Custom Helm chart per environment -You can specify the use of a custom Helm chart per environment by scoping the environment variable +You can specify the use of a custom Helm chart per environment by scoping the CI/CD variable to the desired environment. See [Limiting environment scopes of variables](../../ci/variables/README.md#limit-the-environment-scopes-of-cicd-variables). ## Customizing `.gitlab-ci.yml` @@ -204,7 +204,7 @@ into your project and edit it as needed. For clusters not managed by GitLab, you can customize the namespace in `.gitlab-ci.yml` by specifying -[`environment:kubernetes:namespace`](../../ci/environments/index.md#configuring-kubernetes-deployments). +[`environment:kubernetes:namespace`](../../ci/environments/index.md#configure-kubernetes-deployments). For example, the following configuration overrides the namespace used for `production` deployments: @@ -260,7 +260,7 @@ the [GitLab 12.10 based templates](https://gitlab.com/gitlab-org/auto-devops-v12 To support applications requiring a database, [PostgreSQL](https://www.postgresql.org/) is provisioned by default. The credentials to access the database are preconfigured, but can be customized by setting the associated -[variables](#environment-variables). You can use these credentials to define a `DATABASE_URL`: +[CI/CD variables](#cicd-variables). You can use these credentials to define a `DATABASE_URL`: ```yaml postgres://user:password@postgres-host:postgres-port/postgres-database @@ -269,7 +269,7 @@ postgres://user:password@postgres-host:postgres-port/postgres-database ### Upgrading PostgresSQL WARNING: -The variable `AUTO_DEVOPS_POSTGRES_CHANNEL` that controls default provisioned +The CI/CD variable `AUTO_DEVOPS_POSTGRES_CHANNEL` that controls default provisioned PostgreSQL was changed to `2` in [GitLab 13.0](https://gitlab.com/gitlab-org/gitlab/-/issues/210499). To keep using the old PostgreSQL, set the `AUTO_DEVOPS_POSTGRES_CHANNEL` variable to `1`. @@ -290,17 +290,17 @@ production environments, for some use cases, it may not be sufficiently secure o resilient, and you may want to use an external managed provider (such as AWS Relational Database Service) for PostgreSQL. -You must define environment-scoped variables for `POSTGRES_ENABLED` and +You must define environment-scoped CI/CD variables for `POSTGRES_ENABLED` and `DATABASE_URL` in your project's CI/CD settings: 1. Disable the built-in PostgreSQL installation for the required environments using - scoped [environment variables](../../ci/environments/index.md#scoping-environments-with-specs). + environment-scoped [CI/CD variables](../../ci/environments/index.md#scoping-environments-with-specs). For this use case, it's likely that only `production` must be added to this list. The built-in PostgreSQL setup for Review Apps and staging is sufficient.  -1. Define the `DATABASE_URL` CI variable as a scoped environment variable that is +1. Define the `DATABASE_URL` variable as an environment-scoped variable that is available to your application. This should be a URL in the following format: ```yaml @@ -310,7 +310,7 @@ You must define environment-scoped variables for `POSTGRES_ENABLED` and You must ensure that your Kubernetes cluster has network access to wherever PostgreSQL is hosted. -## Environment variables +## CI/CD variables The following variables can be used for setting up the Auto DevOps domain, providing a custom Helm chart, or scaling your application. PostgreSQL can @@ -318,10 +318,10 @@ also be customized, and you can use a [custom buildpack](#custom-buildpacks). ### Build and deployment -The following table lists variables related to building and deploying +The following table lists CI/CD variables related to building and deploying applications. -| **Variable** | **Description** | +| **CI/CD Variable** | **Description** | |-----------------------------------------|------------------------------------| | `ADDITIONAL_HOSTS` | Fully qualified domain names specified as a comma-separated list that are added to the Ingress hosts. | | `<ENVIRONMENT>_ADDITIONAL_HOSTS` | For a specific environment, the fully qualified domain names specified as a comma-separated list that are added to the Ingress hosts. This takes precedence over `ADDITIONAL_HOSTS`. | @@ -329,7 +329,7 @@ applications. | `AUTO_DEVOPS_BUILD_IMAGE_CNB_ENABLED` | When set to a non-empty value and no `Dockerfile` is present, Auto Build builds your application using Cloud Native Buildpacks instead of Herokuish. [More details](stages.md#auto-build-using-cloud-native-buildpacks-beta). | | `AUTO_DEVOPS_BUILD_IMAGE_CNB_BUILDER` | The builder used when building with Cloud Native Buildpacks. The default builder is `heroku/buildpacks:18`. [More details](stages.md#auto-build-using-cloud-native-buildpacks-beta). | | `AUTO_DEVOPS_BUILD_IMAGE_EXTRA_ARGS` | Extra arguments to be passed to the `docker build` command. Note that using quotes doesn't prevent word splitting. [More details](#passing-arguments-to-docker-build). | -| `AUTO_DEVOPS_BUILD_IMAGE_FORWARDED_CI_VARIABLES` | A [comma-separated list of CI variable names](#forward-ci-variables-to-the-build-environment) to be forwarded to the build environment (the buildpack builder or `docker build`). | +| `AUTO_DEVOPS_BUILD_IMAGE_FORWARDED_CI_VARIABLES` | A [comma-separated list of CI/CD variable names](#forward-cicd-variables-to-the-build-environment) to be forwarded to the build environment (the buildpack builder or `docker build`). | | `AUTO_DEVOPS_CHART` | Helm Chart used to deploy your apps. Defaults to the one [provided by GitLab](https://gitlab.com/gitlab-org/cluster-integration/auto-deploy-image/-/tree/master/assets/auto-deploy-app). | | `AUTO_DEVOPS_CHART_REPOSITORY` | Helm Chart repository used to search for charts. Defaults to `https://charts.gitlab.io`. | | `AUTO_DEVOPS_CHART_REPOSITORY_NAME` | From GitLab 11.11, used to set the name of the Helm repository. Defaults to `gitlab`. | @@ -367,9 +367,9 @@ Auto DevOps can undo your changes. ### Database -The following table lists variables related to the database. +The following table lists CI/CD variables related to the database. -| **Variable** | **Description** | +| **CI/CD Variable** | **Description** | |-----------------------------------------|------------------------------------| | `DB_INITIALIZE` | From GitLab 11.4, used to specify the command to run to initialize the application's PostgreSQL database. Runs inside the application pod. | | `DB_MIGRATE` | From GitLab 11.4, used to specify the command to run to migrate the application's PostgreSQL database. Runs inside the application pod. | @@ -383,7 +383,7 @@ The following table lists variables related to the database. The following table lists variables used to disable jobs. -| **Job Name** | **Variable** | **GitLab version** | **Description** | +| **Job Name** | **CI/CDVariable** | **GitLab version** | **Description** | |----------------------------------------|---------------------------------|-----------------------|-----------------| | `.fuzz_base` | `COVFUZZ_DISABLED` | [From GitLab 13.2](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/34984) | [Read more](../../user/application_security/coverage_fuzzing/) about how `.fuzz_base` provide capability for your own jobs. If the variable is present, your jobs aren't created. | | `apifuzzer_fuzz` | `API_FUZZING_DISABLED` | [From GitLab 13.3](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/39135) | If the variable is present, the job isn't created. | @@ -433,7 +433,7 @@ The following table lists variables used to disable jobs. > [Introduced](https://gitlab.com/gitlab-org/gitlab-foss/-/issues/49056) in GitLab 11.7. Some applications need to define secret variables that are accessible by the deployed -application. Auto DevOps detects variables starting with `K8S_SECRET_`, and makes +application. Auto DevOps detects CI/CD variables starting with `K8S_SECRET_`, and makes these prefixed variables available to the deployed application as environment variables. To configure your application variables: @@ -545,7 +545,7 @@ The normal behavior of Auto DevOps is to use continuous deployment, pushing automatically to the `production` environment every time a new pipeline is run on the default branch. However, there are cases where you might want to use a staging environment, and deploy to production manually. For this scenario, the -`STAGING_ENABLED` environment variable was introduced. +`STAGING_ENABLED` CI/CD variable was introduced. If you define `STAGING_ENABLED` with a non-empty value, then GitLab automatically deploys the application to a `staging` environment, and creates a `production_manual` job for @@ -584,7 +584,7 @@ are created: 1. `rollout 50%` 1. `rollout 100%` -The percentage is based on the `REPLICAS` variable, and defines the number of +The percentage is based on the `REPLICAS` CI/CD variable, and defines the number of pods you want to have for your deployment. If the value is `10`, and you run the `10%` rollout job, there is `1` new pod and `9` old ones. @@ -593,7 +593,7 @@ required to go from `10%` to `100%`, you can jump to whatever job you want. You can also scale down by running a lower percentage job, just before hitting `100%`. Once you get to `100%`, you can't scale down, and you'd have to roll back by redeploying the old version using the -[rollback button](../../ci/environments/index.md#retrying-and-rolling-back) in the +[rollback button](../../ci/environments/index.md#retry-or-roll-back-a-deployment) in the environment page. Below, you can see how the pipeline appears if the rollout or staging @@ -616,8 +616,8 @@ With `INCREMENTAL_ROLLOUT_MODE` set to `manual` and with `STAGING_ENABLED`  WARNING: -Before GitLab 11.4, the presence of the `INCREMENTAL_ROLLOUT_ENABLED` environment -variable enabled this feature. This configuration is deprecated, and is scheduled to be +Before GitLab 11.4, the presence of the `INCREMENTAL_ROLLOUT_ENABLED` CI/CD variable +enabled this feature. This configuration is deprecated, and is scheduled to be removed in the future. ### Timed incremental rollout to production **(PREMIUM)** @@ -632,7 +632,7 @@ This configuration is based on Everything behaves the same way, except: -- To enable it, set the `INCREMENTAL_ROLLOUT_MODE` variable to `timed`. +- To enable it, set the `INCREMENTAL_ROLLOUT_MODE` CI/CD variable to `timed`. - Instead of the standard `production` job, the following jobs are created with a 5 minute delay between each: @@ -651,7 +651,7 @@ permissions on new projects when Auto DevOps is not enabled: The banner can be disabled for: - A user, when they dismiss it themselves. -- A project, by explicitly [disabling Auto DevOps](index.md#enablingdisabling-auto-devops). +- A project, by explicitly [disabling Auto DevOps](index.md#enable-or-disable-auto-devops). - An entire GitLab instance: - By an administrator running the following in a Rails console: diff --git a/doc/topics/autodevops/img/alexj_autodevops_min_v13_8.png b/doc/topics/autodevops/img/alexj_autodevops_min_v13_8.png Binary files differdeleted file mode 100644 index 7dc37737835..00000000000 --- a/doc/topics/autodevops/img/alexj_autodevops_min_v13_8.png +++ /dev/null diff --git a/doc/topics/autodevops/img/kai_autodevops_min_v13_8.png b/doc/topics/autodevops/img/kai_autodevops_min_v13_8.png Binary files differdeleted file mode 100644 index fa3ab4c1820..00000000000 --- a/doc/topics/autodevops/img/kai_autodevops_min_v13_8.png +++ /dev/null diff --git a/doc/topics/autodevops/index.md b/doc/topics/autodevops/index.md index d95e5568e0b..dbb63275a06 100644 --- a/doc/topics/autodevops/index.md +++ b/doc/topics/autodevops/index.md @@ -6,69 +6,134 @@ info: To determine the technical writer assigned to the Stage/Group associated w # Auto DevOps **(FREE)** -> - [Introduced](https://gitlab.com/gitlab-org/gitlab-foss/-/issues/37115) in GitLab 10.0. -> - Generally available on GitLab 11.0. - -Auto DevOps are default CI/CD templates that auto-discover the source code you have. They -enable GitLab to automatically detect, build, test, deploy, and monitor your applications. -Leveraging [CI/CD best practices](../../ci/pipelines/pipeline_efficiency.md) and tools, -Auto DevOps aims to simplify the setup and execution of a mature and modern software -development lifecycle. - -## Overview - -You can spend a lot of effort to set up the workflow and processes required to -build, deploy, and monitor your project. It gets worse when your company has -hundreds, if not thousands, of projects to maintain. With new projects -constantly starting up, the entire software development process becomes -impossibly complex to manage. - -Auto DevOps provides you a seamless software development process by -automatically detecting all dependencies and language technologies required to -test, build, package, deploy, and monitor every project with minimal -configuration. Automation enables consistency across your projects, seamless -management of processes, and faster creation of new projects: push your code, -and GitLab does the rest, improving your productivity and efficiency. +> - Introduced in GitLab 11.0 for general availability. + +GitLab Auto DevOps helps to reduce the complexity of software delivery by +setting up pipelines and integrations for you. Instead of requiring you to +manually configure your entire GitLab environment, Auto DevOps configures +many of these areas for you, including security auditing and vulnerability +testing. + +Using Auto DevOps, you can: + +- Detect the language of your code. +- Automatically build, test, and measure code quality. +- Scan for potential vulnerabilities, security flaws, and licensing issues. +- Monitor in real-time. +- Deploy your application. + +The functionality of Auto DevOps is based on default CI/CD templates that +auto-discover your source code. These templates enable GitLab to provide +consistency across your projects, seamless management of processes, and faster +creation of new projects. Leveraging [CI/CD best practices](../../ci/pipelines/pipeline_efficiency.md) +and tools, Auto DevOps lets you push your code, with GitLab doing the rest, +improving your productivity and efficiency. <i class="fa fa-youtube-play youtube" aria-hidden="true"></i> -For an introduction to Auto DevOps, watch [AutoDevOps in GitLab 11.0](https://youtu.be/0Tc0YYBxqi4). +For an introduction to Auto DevOps, watch [AutoDevOps in GitLab 11.0](https://youtu.be/0Tc0YYBxqi4) or see this [overview](https://about.gitlab.com/stages-devops-lifecycle/auto-devops/). For requirements, read [Requirements for Auto DevOps](requirements.md) for more information. -For a developer's guide, read [Auto DevOps development guide](../../development/auto_devops.md). +For GitLab contributors, see the [Auto DevOps development guide](../../development/auto_devops.md). -### Share your feedback +## Enable or disable Auto DevOps -As Auto DevOps continues to gain popularity, and lowers the barrier to entry for -getting started with DevOps and CI/CD, see what our wider community is saying: +Auto DevOps is enabled by default for all projects in self-managed instances +(as of [GitLab 11.3](https://gitlab.com/gitlab-org/gitlab-foss/-/issues/41729)), +but not for GitLab SaaS instances. -From [AlexJonesax](https://twitter.com/AlexJonesax) and [KaiPMDH](https://twitter.com/KaiPMDH) on Twitter: +When first using Auto DevOps, review the [requirements](requirements.md) to +ensure all the necessary components to make full use of Auto DevOps are +available. First-time users should follow the [quick start guide](quick_start_guide.md). - +Depending on your instance type, you can enable or disable Auto DevOps at the +following levels: - +| Instance type | [Project](#at-the-project-level) | [Group](#at-the-group-level) | [Instance](#at-the-instance-level) (Admin Area) | +|---------------------|------------------------|------------------------|------------------------| +| GitLab SaaS | **{check-circle}** Yes | **{dotted-circle}** No | **{dotted-circle}** No | +| GitLab self-managed | **{check-circle}** Yes | **{check-circle}** Yes | **{check-circle}** Yes | -We welcome everyone to [share your experience by tagging GitLab on Twitter](https://twitter.com/gitlab). +When you enable AutoDevOps for your instance, it attempts to run on all +pipelines in each project, but will automatically disable itself for individual +projects on their first pipeline failure. An instance administrator can enable +or disable this default in the [Auto DevOps settings](../../user/admin_area/settings/continuous_integration.md#auto-devops). -## Enabled by default +Since [GitLab 12.7](https://gitlab.com/gitlab-org/gitlab/-/issues/26655), +Auto DevOps runs on pipelines automatically only if a [`Dockerfile` or matching buildpack](stages.md#auto-build) +exists. -> [Introduced](https://gitlab.com/gitlab-org/gitlab-foss/-/issues/41729) in GitLab 11.3. +If a [CI/CD configuration file](../../ci/yaml/README.md) is present in the +project, it isn't changed and won't be affected by Auto DevOps. -On self-managed instances, Auto DevOps is enabled by default for all projects. -It attempts to run on all pipelines in each project. An instance administrator can -enable or disable this default in the -[Auto DevOps settings](../../user/admin_area/settings/continuous_integration.md#auto-devops). -Auto DevOps automatically disables in individual projects on their first pipeline failure, +### At the project level -NOTE: -Auto DevOps is not enabled by default on GitLab.com. +When you enable Auto DevOps for a project, ensure that your project does not have a `.gitlab-ci.yml` present. If it exists, remove it before enabling Auto DevOps. -Since [GitLab 12.7](https://gitlab.com/gitlab-org/gitlab/-/issues/26655), Auto DevOps -runs on pipelines automatically only if a [`Dockerfile` or matching buildpack](stages.md#auto-build) -exists. +To enable it: + +1. Go to your project's **Settings > CI/CD > Auto DevOps**. +1. Select the **Default to Auto DevOps pipeline** checkbox to enable it. +1. (Optional, but recommended) When enabling, you can add in the + [base domain](#auto-devops-base-domain) Auto DevOps uses to + [deploy your application](stages.md#auto-deploy), + and choose the [deployment strategy](#deployment-strategy). +1. Click **Save changes** for the changes to take effect. + +After enabling the feature, an Auto DevOps pipeline is triggered on the `master` branch. + +### At the group level + +> - [Introduced](https://gitlab.com/gitlab-org/gitlab-foss/-/issues/52447) in GitLab 11.10. + +Only administrators and group owners can enable or disable Auto DevOps at the group level. + +When enabling or disabling Auto DevOps at group level, group configuration is +implicitly used for the subgroups and projects inside that group, unless Auto DevOps +is specifically enabled or disabled on the subgroup or project. + +To enable or disable Auto DevOps at the group level: + +1. Go to your group's **Settings > CI/CD > Auto DevOps** page. +1. Select the **Default to Auto DevOps pipeline** checkbox to enable it. +1. Click **Save changes** for the changes to take effect. + +### At the instance level **(FREE SELF)** + +Even when disabled at the instance level, group owners and project maintainers +can still enable Auto DevOps at the group and project level, respectively. + +1. As an administrator, go to **Admin Area > Settings > CI/CD > Continuous Integration and Deployment**. +1. Select **Default to Auto DevOps pipeline for all projects** to enable it. +1. (Optional) You can set up the Auto DevOps [base domain](#auto-devops-base-domain), + for Auto Deploy and Auto Review Apps to use. +1. Click **Save changes** for the changes to take effect. + +### Deployment strategy + +> - [Introduced](https://gitlab.com/gitlab-org/gitlab-foss/-/issues/38542) in GitLab 11.0. -If a [CI/CD configuration file](../../ci/yaml/README.md) is present in the project, -it continues to be used, whether or not Auto DevOps is enabled. +You can change the deployment strategy used by Auto DevOps by visiting your +project's **Settings > CI/CD > Auto DevOps**. The following options +are available: + +- **Continuous deployment to production**: Enables [Auto Deploy](stages.md#auto-deploy) + with `master` branch directly deployed to production. +- **Continuous deployment to production using timed incremental rollout**: Sets the + [`INCREMENTAL_ROLLOUT_MODE`](customize.md#timed-incremental-rollout-to-production) variable + to `timed`. Production deployments execute with a 5 minute delay between + each increment in rollout. +- **Automatic deployment to staging, manual deployment to production**: Sets the + [`STAGING_ENABLED`](customize.md#deploy-policy-for-staging-and-production-environments) and + [`INCREMENTAL_ROLLOUT_MODE`](customize.md#incremental-rollout-to-production) variables + to `1` and `manual`. This means: + + - `master` branch is directly deployed to staging. + - Manual actions are provided for incremental rollout to production. + +NOTE: +Use the [blue-green deployment](../../ci/environments/incremental_rollouts.md#blue-green-deployment) technique +to minimize downtime and risk. ## Quick start @@ -110,20 +175,20 @@ Depending on your target platform, some features might not be available to you. Comprised of a set of [stages](stages.md), Auto DevOps brings these best practices to your project in a simple and automatic way: -1. [Auto Build](stages.md#auto-build) -1. [Auto Test](stages.md#auto-test) -1. [Auto Code Quality](stages.md#auto-code-quality) -1. [Auto SAST (Static Application Security Testing)](stages.md#auto-sast) -1. [Auto Secret Detection](stages.md#auto-secret-detection) -1. [Auto Dependency Scanning](stages.md#auto-dependency-scanning) **(ULTIMATE)** -1. [Auto License Compliance](stages.md#auto-license-compliance) **(ULTIMATE)** -1. [Auto Container Scanning](stages.md#auto-container-scanning) **(ULTIMATE)** -1. [Auto Review Apps](stages.md#auto-review-apps) -1. [Auto DAST (Dynamic Application Security Testing)](stages.md#auto-dast) **(ULTIMATE)** -1. [Auto Deploy](stages.md#auto-deploy) -1. [Auto Browser Performance Testing](stages.md#auto-browser-performance-testing) **(PREMIUM)** -1. [Auto Monitoring](stages.md#auto-monitoring) -1. [Auto Code Intelligence](stages.md#auto-code-intelligence) +- [Auto Browser Performance Testing](stages.md#auto-browser-performance-testing) +- [Auto Build](stages.md#auto-build) +- [Auto Code Intelligence](stages.md#auto-code-intelligence) +- [Auto Code Quality](stages.md#auto-code-quality) +- [Auto Container Scanning](stages.md#auto-container-scanning) +- [Auto DAST (Dynamic Application Security Testing)](stages.md#auto-dast) +- [Auto Dependency Scanning](stages.md#auto-dependency-scanning) +- [Auto Deploy](stages.md#auto-deploy) +- [Auto License Compliance](stages.md#auto-license-compliance) +- [Auto Monitoring](stages.md#auto-monitoring) +- [Auto Review Apps](stages.md#auto-review-apps) +- [Auto SAST (Static Application Security Testing)](stages.md#auto-sast) +- [Auto Secret Detection](stages.md#auto-secret-detection) +- [Auto Test](stages.md#auto-test) As Auto DevOps relies on many different components, you should have a basic knowledge of the following: @@ -162,7 +227,7 @@ any of the following places: [groups](../../user/group/clusters/index.md#base-domain) - or at the project level as a variable: `KUBE_INGRESS_BASE_DOMAIN` - or at the group level as a variable: `KUBE_INGRESS_BASE_DOMAIN` -- or as an instance-wide fallback in **Admin Area > Settings** under the +- or as an instance-wide fallback in **Admin Area > Settings > CI/CD** under the **Continuous Integration and Delivery** section The base domain variable `KUBE_INGRESS_BASE_DOMAIN` follows the same order of precedence @@ -193,83 +258,6 @@ to the Kubernetes pods running your application. See [Auto DevOps requirements for Amazon ECS](requirements.md#auto-devops-requirements-for-amazon-ecs). -## Enabling/Disabling Auto DevOps - -When first using Auto DevOps, review the [requirements](requirements.md) to ensure -all the necessary components to make full use of Auto DevOps are available. First-time -users should follow the [quick start guide](quick_start_guide.md). - -GitLab.com users can enable or disable Auto DevOps only at the project level. -Self-managed users can enable or disable Auto DevOps at the project, group, or -instance level. - -### At the project level - -If enabling, check that your project does not have a `.gitlab-ci.yml`, or if one exists, remove it. - -1. Go to your project's **Settings > CI/CD > Auto DevOps**. -1. Select the **Default to Auto DevOps pipeline** checkbox to enable it. -1. (Optional, but recommended) When enabling, you can add in the - [base domain](#auto-devops-base-domain) Auto DevOps uses to - [deploy your application](stages.md#auto-deploy), - and choose the [deployment strategy](#deployment-strategy). -1. Click **Save changes** for the changes to take effect. - -After enabling the feature, an Auto DevOps pipeline is triggered on the `master` branch. - -### At the group level - -> [Introduced](https://gitlab.com/gitlab-org/gitlab-foss/-/issues/52447) in GitLab 11.10. - -Only administrators and group owners can enable or disable Auto DevOps at the group level. - -When enabling or disabling Auto DevOps at group level, group configuration is -implicitly used for the subgroups and projects inside that group, unless Auto DevOps -is specifically enabled or disabled on the subgroup or project. - -To enable or disable Auto DevOps at the group level: - -1. Go to your group's **Settings > CI/CD > Auto DevOps** page. -1. Select the **Default to Auto DevOps pipeline** checkbox to enable it. -1. Click **Save changes** for the changes to take effect. - -### At the instance level (Administrators only) - -Even when disabled at the instance level, group owners and project maintainers can still enable -Auto DevOps at the group and project level, respectively. - -1. Go to **Admin Area > Settings > Continuous Integration and Deployment**. -1. Select **Default to Auto DevOps pipeline for all projects** to enable it. -1. (Optional) You can set up the Auto DevOps [base domain](#auto-devops-base-domain), - for Auto Deploy and Auto Review Apps to use. -1. Click **Save changes** for the changes to take effect. - -### Deployment strategy - -> [Introduced](https://gitlab.com/gitlab-org/gitlab-foss/-/issues/38542) in GitLab 11.0. - -You can change the deployment strategy used by Auto DevOps by visiting your -project's **Settings > CI/CD > Auto DevOps**. The following options -are available: - -- **Continuous deployment to production**: Enables [Auto Deploy](stages.md#auto-deploy) - with `master` branch directly deployed to production. -- **Continuous deployment to production using timed incremental rollout**: Sets the - [`INCREMENTAL_ROLLOUT_MODE`](customize.md#timed-incremental-rollout-to-production) variable - to `timed`. Production deployments execute with a 5 minute delay between - each increment in rollout. -- **Automatic deployment to staging, manual deployment to production**: Sets the - [`STAGING_ENABLED`](customize.md#deploy-policy-for-staging-and-production-environments) and - [`INCREMENTAL_ROLLOUT_MODE`](customize.md#incremental-rollout-to-production) variables - to `1` and `manual`. This means: - - - `master` branch is directly deployed to staging. - - Manual actions are provided for incremental rollout to production. - -NOTE: -Use the [blue-green deployment](../../ci/environments/incremental_rollouts.md#blue-green-deployment) technique -to minimize downtime and risk. - ## Using multiple Kubernetes clusters When using Auto DevOps, you can deploy different environments to @@ -333,7 +321,7 @@ simplify configuration and prevent any unforeseen issues. The GitLab integration with Helm does not support installing applications when behind a proxy. Users who want to do so must inject their proxy settings into the installation pods at runtime, such as by using a -[`PodPreset`](https://kubernetes.io/docs/concepts/workloads/pods/podpreset/): +[`PodPreset`](https://v1-19.docs.kubernetes.io/docs/concepts/workloads/pods/podpreset/): ```yaml apiVersion: settings.k8s.io/v1alpha1 @@ -349,245 +337,5 @@ spec: value: "PUT_YOUR_HTTPS_PROXY_HERE" ``` -## Troubleshooting - -### Unable to select a buildpack - -Auto Build and Auto Test may fail to detect your language or framework with the -following error: - -```plaintext -Step 5/11 : RUN /bin/herokuish buildpack build - ---> Running in eb468cd46085 - -----> Unable to select a buildpack -The command '/bin/sh -c /bin/herokuish buildpack build' returned a non-zero code: 1 -``` - -The following are possible reasons: - -- Your application may be missing the key files the buildpack is looking for. - Ruby applications require a `Gemfile` to be properly detected, - even though it's possible to write a Ruby app without a `Gemfile`. -- No buildpack may exist for your application. Try specifying a - [custom buildpack](customize.md#custom-buildpacks). - -### Pipeline that extends Auto DevOps with only / except fails - -If your pipeline fails with the following message: - -```plaintext -Found errors in your .gitlab-ci.yml: - - jobs:test config key may not be used with `rules`: only -``` - -This error appears when the included job’s rules configuration has been overridden with the `only` or `except` syntax. -To fix this issue, you must either: - -- Transition your `only/except` syntax to rules. -- (Temporarily) Pin your templates to the [GitLab 12.10 based templates](https://gitlab.com/gitlab-org/auto-devops-v12-10). - -### Failure to create a Kubernetes namespace - -Auto Deploy fails if GitLab can't create a Kubernetes namespace and -service account for your project. For help debugging this issue, see -[Troubleshooting failed deployment jobs](../../user/project/clusters/index.md#troubleshooting). - -### Detected an existing PostgreSQL database - -After upgrading to GitLab 13.0, you may encounter this message when deploying -with Auto DevOps: - -```plaintext -Detected an existing PostgreSQL database installed on the -deprecated channel 1, but the current channel is set to 2. The default -channel changed to 2 in of GitLab 13.0. -[...] -``` - -Auto DevOps, by default, installs an in-cluster PostgreSQL database alongside -your application. The default installation method changed in GitLab 13.0, and -upgrading existing databases requires user involvement. The two installation -methods are: - -- **channel 1 (deprecated):** Pulls in the database as a dependency of the associated - Helm chart. Only supports Kubernetes versions up to version 1.15. -- **channel 2 (current):** Installs the database as an independent Helm chart. Required - for using the in-cluster database feature with Kubernetes versions 1.16 and greater. - -If you receive this error, you can do one of the following actions: - -- You can *safely* ignore the warning and continue using the channel 1 PostgreSQL - database by setting `AUTO_DEVOPS_POSTGRES_CHANNEL` to `1` and redeploying. - -- You can delete the channel 1 PostgreSQL database and install a fresh channel 2 - database by setting `AUTO_DEVOPS_POSTGRES_DELETE_V1` to a non-empty value and - redeploying. - - WARNING: - Deleting the channel 1 PostgreSQL database permanently deletes the existing - channel 1 database and all its data. See - [Upgrading PostgreSQL](upgrading_postgresql.md) - for more information on backing up and upgrading your database. - -- If you are not using the in-cluster database, you can set - `POSTGRES_ENABLED` to `false` and re-deploy. This option is especially relevant to - users of *custom charts without the in-chart PostgreSQL dependency*. - Database auto-detection is based on the `postgresql.enabled` Helm value for - your release. This value is set based on the `POSTGRES_ENABLED` CI variable - and persisted by Helm, regardless of whether or not your chart uses the - variable. - -WARNING: -Setting `POSTGRES_ENABLED` to `false` permanently deletes any existing -channel 1 database for your environment. - -### Error: unable to recognize "": no matches for kind "Deployment" in version "extensions/v1beta1" - -After upgrading your Kubernetes cluster to [v1.16+](stages.md#kubernetes-116), -you may encounter this message when deploying with Auto DevOps: - -```plaintext -UPGRADE FAILED -Error: failed decoding reader into objects: unable to recognize "": no matches for kind "Deployment" in version "extensions/v1beta1" -``` - -This can occur if your current deployments on the environment namespace were deployed with a -deprecated/removed API that doesn't exist in Kubernetes v1.16+. For example, -if [your in-cluster PostgreSQL was installed in a legacy way](#detected-an-existing-postgresql-database), -the resource was created via the `extensions/v1beta1` API. However, the deployment resource -was moved to the `app/v1` API in v1.16. - -To recover such outdated resources, you must convert the current deployments by mapping legacy APIs -to newer APIs. There is a helper tool called [`mapkubeapis`](https://github.com/hickeyma/helm-mapkubeapis) -that works for this problem. Follow these steps to use the tool in Auto DevOps: - -1. Modify your `.gitlab-ci.yml` with: - - ```yaml - include: - - template: Auto-DevOps.gitlab-ci.yml - - remote: https://gitlab.com/shinya.maeda/ci-templates/-/raw/master/map-deprecated-api.gitlab-ci.yml - - variables: - HELM_VERSION_FOR_MAPKUBEAPIS: "v2" # If you're using auto-depoy-image v2 or above, please specify "v3". - ``` - -1. Run the job `<environment-name>:map-deprecated-api`. Ensure that this job succeeds before moving - to the next step. You should see something like the following output: - - ```shell - 2020/10/06 07:20:49 Found deprecated or removed Kubernetes API: - "apiVersion: extensions/v1beta1 - kind: Deployment" - Supported API equivalent: - "apiVersion: apps/v1 - kind: Deployment" - ``` - -1. Revert your `.gitlab-ci.yml` to the previous version. You no longer need to include the - supplemental template `map-deprecated-api`. - -1. Continue the deployments as usual. - -### Error: error initializing: Looks like "https://kubernetes-charts.storage.googleapis.com" is not a valid chart repository or cannot be reached - -As [announced in the official CNCF blog post](https://www.cncf.io/blog/2020/10/07/important-reminder-for-all-helm-users-stable-incubator-repos-are-deprecated-and-all-images-are-changing-location/), -the stable Helm chart repository was deprecated and removed on November 13th, 2020. -You may encounter this error after that date. - -Some GitLab features had dependencies on the stable chart. To mitigate the impact, we changed them -to use new official repositories or the [Helm Stable Archive repository maintained by GitLab](https://gitlab.com/gitlab-org/cluster-integration/helm-stable-archive). -Auto Deploy contains [an example fix](https://gitlab.com/gitlab-org/cluster-integration/auto-deploy-image/-/merge_requests/127). - -In Auto Deploy, `v1.0.6+` of `auto-deploy-image` no longer adds the deprecated stable repository to -the `helm` command. If you use a custom chart and it relies on the deprecated stable repository, -specify an older `auto-deploy-image` like this example: - -```yaml -include: - - template: Auto-DevOps.gitlab-ci.yml - -.auto-deploy: - image: "registry.gitlab.com/gitlab-org/cluster-integration/auto-deploy-image:v1.0.5" -``` - -Keep in mind that this approach stops working when the stable repository is removed, -so you must eventually fix your custom chart. - -To fix your custom chart: - -1. In your chart directory, update the `repository` value in your `requirements.yaml` file from : - - ```yaml - repository: "https://kubernetes-charts.storage.googleapis.com/" - ``` - - to: - - ```yaml - repository: "https://charts.helm.sh/stable" - ``` - -1. In your chart directory, run `helm dep update .` using the same Helm major version as Auto DevOps. -1. Commit the changes for the `requirements.yaml` file. -1. If you previously had a `requirements.lock` file, commit the changes to the file. - If you did not previously have a `requirements.lock` file in your chart, - you do not need to commit the new one. This file is optional, but when present, - it's used to verify the integrity of the downloaded dependencies. - -You can find more information in -[issue #263778, "Migrate PostgreSQL from stable Helm repository"](https://gitlab.com/gitlab-org/gitlab/-/issues/263778). - -### Error: release .... failed: timed out waiting for the condition - -When getting started with Auto DevOps, you may encounter this error when first -deploying your application: - -```plaintext -INSTALL FAILED -PURGING CHART -Error: release staging failed: timed out waiting for the condition -``` - -This is most likely caused by a failed liveness (or readiness) probe attempted -during the deployment process. By default, these probes are run against the root -page of the deployed application on port 5000. If your application isn't configured -to serve anything at the root page, or is configured to run on a specific port -*other* than 5000, this check fails. - -If it fails, you should see these failures in the events for the relevant -Kubernetes namespace. These events look like the following example: - -```plaintext -LAST SEEN TYPE REASON OBJECT MESSAGE -3m20s Warning Unhealthy pod/staging-85db88dcb6-rxd6g Readiness probe failed: Get http://10.192.0.6:5000/: dial tcp 10.192.0.6:5000: connect: connection refused -3m32s Warning Unhealthy pod/staging-85db88dcb6-rxd6g Liveness probe failed: Get http://10.192.0.6:5000/: dial tcp 10.192.0.6:5000: connect: connection refused -``` - -To change the port used for the liveness checks, pass -[custom values to the Helm chart](customize.md#customize-values-for-helm-chart) -used by Auto DevOps: - -1. Create a directory and file at the root of your repository named `.gitlab/auto-deploy-values.yaml`. - -1. Populate the file with the following content, replacing the port values with - the actual port number your application is configured to use: - - ```yaml - service: - internalPort: <port_value> - externalPort: <port_value> - ``` - -1. Commit your changes. - -After committing your changes, subsequent probes should use the newly-defined ports. -The page that's probed can also be changed by overriding the `livenessProbe.path` -and `readinessProbe.path` values (shown in the -[default `values.yaml`](https://gitlab.com/gitlab-org/cluster-integration/auto-deploy-image/-/blob/master/assets/auto-deploy-app/values.yaml) -file) in the same fashion. - -## Development guides - -[Development guide for Auto DevOps](../../development/auto_devops.md) +<!-- DO NOT ADD TROUBLESHOOTING INFO HERE --> +<!-- Troubleshooting information has moved to troubleshooting.md --> diff --git a/doc/topics/autodevops/quick_start_guide.md b/doc/topics/autodevops/quick_start_guide.md index cd951e53bd1..631180f696c 100644 --- a/doc/topics/autodevops/quick_start_guide.md +++ b/doc/topics/autodevops/quick_start_guide.md @@ -222,7 +222,7 @@ you to common environment tasks: - **Terminal** (**{terminal}**) - Opens a [web terminal](../../ci/environments/index.md#web-terminals) session inside the container where the application is running - **Re-deploy to environment** (**{repeat}**) - For more information, see - [Retrying and rolling back](../../ci/environments/index.md#retrying-and-rolling-back) + [Retrying and rolling back](../../ci/environments/index.md#retry-or-roll-back-a-deployment) - **Stop environment** (**{stop}**) - For more information, see [Stopping an environment](../../ci/environments/index.md#stopping-an-environment) @@ -234,8 +234,8 @@ takes you to the pod's logs page. NOTE: The example shows only one pod hosting the application at the moment, but you can add -more pods by defining the [`REPLICAS` variable](customize.md#environment-variables) -in **Settings > CI/CD > Environment variables**. +more pods by defining the [`REPLICAS` CI/CD variable](customize.md#cicd-variables) +in **Settings > CI/CD > Variables**. ### Work with branches @@ -307,7 +307,7 @@ and customized to fit your workflow. Here are some helpful resources for further 1. [Auto DevOps](index.md) 1. [Multiple Kubernetes clusters](index.md#using-multiple-kubernetes-clusters) 1. [Incremental rollout to production](customize.md#incremental-rollout-to-production) **(PREMIUM)** -1. [Disable jobs you don't need with environment variables](customize.md#environment-variables) +1. [Disable jobs you don't need with CI/CD variables](customize.md#cicd-variables) 1. [Use a static IP for your cluster](../../user/clusters/applications.md#using-a-static-ip) 1. [Use your own buildpacks to build your application](customize.md#custom-buildpacks) 1. [Prometheus monitoring](../../user/project/integrations/prometheus.md) diff --git a/doc/topics/autodevops/requirements.md b/doc/topics/autodevops/requirements.md index 930938b7571..d536282cca4 100644 --- a/doc/topics/autodevops/requirements.md +++ b/doc/topics/autodevops/requirements.md @@ -101,7 +101,7 @@ If you don't have Kubernetes or Prometheus installed, then [Auto Deploy](stages.md#auto-deploy), and [Auto Monitoring](stages.md#auto-monitoring) are skipped. -After all requirements are met, you can [enable Auto DevOps](index.md#enablingdisabling-auto-devops). +After all requirements are met, you can [enable Auto DevOps](index.md#enable-or-disable-auto-devops). ## Auto DevOps requirements for Amazon ECS @@ -109,10 +109,10 @@ After all requirements are met, you can [enable Auto DevOps](index.md#enablingdi You can choose to target [AWS ECS](../../ci/cloud_deployment/index.md) as a deployment platform instead of using Kubernetes. -To get started on Auto DevOps to AWS ECS, you must add a specific Environment -Variable. To do so, follow these steps: +To get started on Auto DevOps to AWS ECS, you must add a specific CI/CD variable. +To do so, follow these steps: -1. In your project, go to **Settings > CI / CD** and expand the **Variables** +1. In your project, go to **Settings > CI/CD** and expand the **Variables** section. 1. Specify which AWS platform to target during the Auto DevOps deployment @@ -121,7 +121,7 @@ Variable. To do so, follow these steps: - `ECS` if you're not enforcing any launch type check when deploying to ECS. When you trigger a pipeline, if you have Auto DevOps enabled and if you have correctly -[entered AWS credentials as environment variables](../../ci/cloud_deployment/index.md#deploy-your-application-to-the-aws-elastic-container-service-ecs), +[entered AWS credentials as variables](../../ci/cloud_deployment/index.md#deploy-your-application-to-the-aws-elastic-container-service-ecs), your application is deployed to AWS ECS. [GitLab Managed Apps](../../user/clusters/applications.md) are not available when deploying to AWS ECS. @@ -145,7 +145,7 @@ own pipeline, as the override stops working when the name changes. You can target [AWS EC2](../../ci/cloud_deployment/index.md) as a deployment platform instead of Kubernetes. To use Auto DevOps with AWS EC2, you must add a -specific environment variable. +specific CI/CD variable. For more details, see [Custom build job for Auto DevOps](../../ci/cloud_deployment/index.md#custom-build-job-for-auto-devops) for deployments to AWS EC2. diff --git a/doc/topics/autodevops/stages.md b/doc/topics/autodevops/stages.md index f1244a1ad1b..913c221d8ec 100644 --- a/doc/topics/autodevops/stages.md +++ b/doc/topics/autodevops/stages.md @@ -53,7 +53,7 @@ For the requirements of other languages and frameworks, read the NOTE: If Auto Build fails despite the project meeting the buildpack requirements, set -a project variable `TRACE=true` to enable verbose logging, which may help you +a project CI/CD variable `TRACE=true` to enable verbose logging, which may help you troubleshoot. ### Auto Build using Cloud Native Buildpacks (beta) @@ -62,9 +62,9 @@ troubleshoot. Auto Build supports building your application using [Cloud Native Buildpacks](https://buildpacks.io) through the [`pack` command](https://github.com/buildpacks/pack). To use Cloud Native Buildpacks, -set the CI variable `AUTO_DEVOPS_BUILD_IMAGE_CNB_ENABLED` to a non-empty +set the CI/CD variable `AUTO_DEVOPS_BUILD_IMAGE_CNB_ENABLED` to a non-empty value. The default builder is `heroku/buildpacks:18` but a different builder -can be selected using the CI variable `AUTO_DEVOPS_BUILD_IMAGE_CNB_BUILDER`. +can be selected using the CI/CD variable `AUTO_DEVOPS_BUILD_IMAGE_CNB_BUILDER`. Cloud Native Buildpacks (CNBs) are an evolution of Heroku buildpacks, and GitLab expects them to eventually supersede Herokuish-based builds within Auto DevOps. For more @@ -103,7 +103,9 @@ NOTE: Not all buildpacks supported by [Auto Build](#auto-build) are supported by Auto Test. Auto Test uses [Herokuish](https://gitlab.com/gitlab-org/gitlab/-/issues/212689), *not* Cloud Native Buildpacks, and only buildpacks that implement the +<!-- vale gitlab.Spelling = NO --> [Testpack API](https://devcenter.heroku.com/articles/testpack-api) are supported. +<!-- vale gitlab.Spelling = YES --> ### Currently supported languages @@ -284,7 +286,7 @@ see the documentation. ### Overriding the DAST target To use a custom target instead of the auto-deployed review apps, -set a `DAST_WEBSITE` environment variable to the URL for DAST to scan. +set a `DAST_WEBSITE` CI/CD variable to the URL for DAST to scan. WARNING: If [DAST Full Scan](../../user/application_security/dast/index.md#full-scan) is @@ -297,10 +299,10 @@ data loss or corruption. You can disable DAST: -- On all branches by setting the `DAST_DISABLED` environment variable to `"true"`. +- On all branches by setting the `DAST_DISABLED` CI/CD variable to `"true"`. - Only on the default branch by setting the `DAST_DISABLED_FOR_DEFAULT_BRANCH` - environment variable to `"true"`. -- Only on feature branches by setting `REVIEW_DISABLED` environment variable to + variable to `"true"`. +- Only on feature branches by setting `REVIEW_DISABLED` variable to `"true"`. This also disables the Review App. ## Auto Browser Performance Testing **(PREMIUM)** @@ -336,15 +338,16 @@ uploads the report as an artifact. Some initial setup is required. A [k6](https://k6.io/) test needs to be written that's tailored to your specific application. The test also needs to be -configured so it can pick up the environment's dynamic URL via an environment variable. +configured so it can pick up the environment's dynamic URL via a CI/CD variable. Any load performance test result differences between the source and target branches are also [shown in the merge request widget](../../user/project/merge_requests/load_performance_testing.md). ## Auto Deploy -This is an optional step, since many projects don't have a Kubernetes cluster -available. If the [requirements](requirements.md) are not met, the job is skipped. +[Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/216008) in GitLab 13.6, you have the choice to deploy to [Amazon Elastic Compute Cloud (Amazon EC2)](https://aws.amazon.com/ec2/) in addition to a Kubernetes cluster. + +Auto Deploy is an optional step for Auto DevOps. If the [requirements](requirements.md) are not met, the job is skipped. After a branch or merge request is merged into the project's default branch (usually `master`), Auto Deploy deploys the application to a `production` environment in @@ -356,7 +359,7 @@ default, but the [Auto DevOps template](https://gitlab.com/gitlab-org/gitlab/blob/master/lib/gitlab/ci/templates/Auto-DevOps.gitlab-ci.yml) contains job definitions for these tasks if you want to enable them. -You can use [environment variables](customize.md#environment-variables) to automatically +You can use [CI/CD variables](customize.md#cicd-variables) to automatically scale your pod replicas, and to apply custom arguments to the Auto DevOps `helm upgrade` commands. This is an easy way to [customize the Auto Deploy Helm chart](customize.md#custom-helm-chart). @@ -440,7 +443,7 @@ On GitLab 12.9 and 12.10, opting into `AUTO_DEVOPS_POSTGRES_CHANNEL` version `2` deletes the version `1` PostgreSQL database. Follow the [guide to upgrading PostgreSQL](upgrading_postgresql.md) to back up and restore your database before opting into version `2` (On -GitLab 13.0, an additional variable is required to trigger the database +GitLab 13.0, an additional CI/CD variable is required to trigger the database deletion). ### Migrations @@ -448,7 +451,7 @@ deletion). > [Introduced](https://gitlab.com/gitlab-org/gitlab-foss/-/merge_requests/21955) in GitLab 11.4 You can configure database initialization and migrations for PostgreSQL to run -within the application pod by setting the project variables `DB_INITIALIZE` and +within the application pod by setting the project CI/CD variables `DB_INITIALIZE` and `DB_MIGRATE` respectively. If present, `DB_INITIALIZE` is run as a shell command within an application pod @@ -500,7 +503,7 @@ access to a Redis instance. Auto DevOps doesn't deploy this instance for you, so you must: - Maintain your own Redis instance. -- Set a CI variable `K8S_SECRET_REDIS_URL`, which is the URL of this instance, +- Set a CI/CD variable `K8S_SECRET_REDIS_URL`, which is the URL of this instance, to ensure it's passed into your deployments. After configuring your worker to respond to health checks, run a Sidekiq @@ -527,7 +530,8 @@ workers: ### Network Policy -> [Introduced](https://gitlab.com/gitlab-org/charts/auto-deploy-app/-/merge_requests/30) in GitLab 12.7. +- [Introduced](https://gitlab.com/gitlab-org/charts/auto-deploy-app/-/merge_requests/30) in GitLab 12.7. +- [Deprecated](https://gitlab.com/gitlab-org/cluster-integration/auto-deploy-image/-/merge_requests/184) in GitLab 13.9. By default, all Kubernetes pods are [non-isolated](https://kubernetes.io/docs/concepts/services-networking/network-policies/#isolated-and-non-isolated-pods), @@ -578,6 +582,76 @@ networkPolicy: For more information on installing Network Policies, see [Install Cilium using GitLab CI/CD](../../user/clusters/applications.md#install-cilium-using-gitlab-cicd). +### Cilium Network Policy + +> [Introduced](https://gitlab.com/gitlab-org/cluster-integration/auto-deploy-image/-/merge_requests/184) in GitLab 13.9. + +By default, all Kubernetes pods are +[non-isolated](https://kubernetes.io/docs/concepts/services-networking/network-policies/#isolated-and-non-isolated-pods), +and accept traffic to and from any source. You can use +[CiliumNetworkPolicy](https://docs.cilium.io/en/v1.8/concepts/kubernetes/policy/#ciliumnetworkpolicy) +to restrict connections to and from selected pods, namespaces, and the internet. + +#### Requirements + +As the default network plugin for Kubernetes (`kubenet`) +[does not implement](https://kubernetes.io/docs/concepts/extend-kubernetes/compute-storage-net/network-plugins/#kubenet) +support for it, you must have [Cilium](https://docs.cilium.io/en/v1.8/intro/) as your Kubernetes network plugin. + +The [Cilium](https://cilium.io/) network plugin can be +installed as a [cluster application](../../user/clusters/applications.md#install-cilium-using-gitlab-cicd) +to enable support for network policies. + +#### Configuration + +You can enable deployment of a network policy by setting the following +in the `.gitlab/auto-deploy-values.yaml` file: + +```yaml +ciliumNetworkPolicy: + enabled: true +``` + +The default policy deployed by the Auto Deploy pipeline allows +traffic within a local namespace, and from the `gitlab-managed-apps` +namespace. All other inbound connections are blocked. Outbound +traffic (for example, to the internet) is not affected by the default policy. + +You can also provide a custom [policy specification](https://docs.cilium.io/en/v1.8/policy/language/#simple-ingress-allow) +in the `.gitlab/auto-deploy-values.yaml` file, for example: + +```yaml +ciliumNetworkPolicy: + enabled: true + spec: + endpointSelector: + matchLabels: + app.gitlab.com/env: staging + ingress: + - fromEndpoints: + - matchLabels: + app.gitlab.com/managed_by: gitlab +``` + +#### Enabling Alerts + +You can also enable alerts. Network policies with alerts are considered only if +[GitLab Kubernetes Agent](../../user/clusters/agent/index.md) +has been integrated. + +You can enable alerts as follows: + +```yaml +ciliumNetworkPolicy: + enabled: true + alerts: + enabled: true + +``` + +For more information on installing Network Policies, see +[Install Cilium using GitLab CI/CD](../../user/clusters/applications.md#install-cilium-using-gitlab-cicd). + ### Web Application Firewall (ModSecurity) customization > [Introduced](https://gitlab.com/gitlab-org/charts/auto-deploy-app/-/merge_requests/44) in GitLab 12.8. @@ -664,7 +738,7 @@ GitLab provides some initial alerts for you after you install Prometheus: To use Auto Monitoring: 1. [Install and configure the Auto DevOps requirements](requirements.md). -1. [Enable Auto DevOps](index.md#enablingdisabling-auto-devops), if you haven't done already. +1. [Enable Auto DevOps](index.md#enable-or-disable-auto-devops), if you haven't done already. 1. Navigate to your project's **{rocket}** **CI/CD > Pipelines** and click **Run Pipeline**. 1. After the pipeline finishes successfully, open the [monitoring dashboard for a deployed environment](../../ci/environments/index.md#monitoring-environments) @@ -686,5 +760,5 @@ You can follow the [code intelligence epic](https://gitlab.com/groups/gitlab-org for updates. This stage is enabled by default. You can disable it by adding the -`CODE_INTELLIGENCE_DISABLED` environment variable. Read more about +`CODE_INTELLIGENCE_DISABLED` CI/CD variable. Read more about [disabling Auto DevOps jobs](../../topics/autodevops/customize.md#disable-jobs). diff --git a/doc/topics/autodevops/troubleshooting.md b/doc/topics/autodevops/troubleshooting.md new file mode 100644 index 00000000000..ed75dd4e262 --- /dev/null +++ b/doc/topics/autodevops/troubleshooting.md @@ -0,0 +1,247 @@ +--- +stage: Configure +group: Configure +info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/engineering/ux/technical-writing/#assignments +--- + +# Troubleshooting Auto DevOps **(FREE)** + +The information in this documentation page describes common errors when using +Auto DevOps, and any available workarounds. + +## Unable to select a buildpack + +Auto Build and Auto Test may fail to detect your language or framework with the +following error: + +```plaintext +Step 5/11 : RUN /bin/herokuish buildpack build + ---> Running in eb468cd46085 + -----> Unable to select a buildpack +The command '/bin/sh -c /bin/herokuish buildpack build' returned a non-zero code: 1 +``` + +The following are possible reasons: + +- Your application may be missing the key files the buildpack is looking for. + Ruby applications require a `Gemfile` to be properly detected, + even though it's possible to write a Ruby app without a `Gemfile`. +- No buildpack may exist for your application. Try specifying a + [custom buildpack](customize.md#custom-buildpacks). + +## Pipeline that extends Auto DevOps with only / except fails + +If your pipeline fails with the following message: + +```plaintext +Found errors in your .gitlab-ci.yml: + + jobs:test config key may not be used with `rules`: only +``` + +This error appears when the included job’s rules configuration has been overridden with the `only` or `except` syntax. +To fix this issue, you must either: + +- Transition your `only/except` syntax to rules. +- (Temporarily) Pin your templates to the [GitLab 12.10 based templates](https://gitlab.com/gitlab-org/auto-devops-v12-10). + +## Failure to create a Kubernetes namespace + +Auto Deploy fails if GitLab can't create a Kubernetes namespace and +service account for your project. For help debugging this issue, see +[Troubleshooting failed deployment jobs](../../user/project/clusters/index.md#troubleshooting). + +## Detected an existing PostgreSQL database + +After upgrading to GitLab 13.0, you may encounter this message when deploying +with Auto DevOps: + +```plaintext +Detected an existing PostgreSQL database installed on the +deprecated channel 1, but the current channel is set to 2. The default +channel changed to 2 in of GitLab 13.0. +[...] +``` + +Auto DevOps, by default, installs an in-cluster PostgreSQL database alongside +your application. The default installation method changed in GitLab 13.0, and +upgrading existing databases requires user involvement. The two installation +methods are: + +- **channel 1 (deprecated):** Pulls in the database as a dependency of the associated + Helm chart. Only supports Kubernetes versions up to version 1.15. +- **channel 2 (current):** Installs the database as an independent Helm chart. Required + for using the in-cluster database feature with Kubernetes versions 1.16 and greater. + +If you receive this error, you can do one of the following actions: + +- You can *safely* ignore the warning and continue using the channel 1 PostgreSQL + database by setting `AUTO_DEVOPS_POSTGRES_CHANNEL` to `1` and redeploying. + +- You can delete the channel 1 PostgreSQL database and install a fresh channel 2 + database by setting `AUTO_DEVOPS_POSTGRES_DELETE_V1` to a non-empty value and + redeploying. + + WARNING: + Deleting the channel 1 PostgreSQL database permanently deletes the existing + channel 1 database and all its data. See + [Upgrading PostgreSQL](upgrading_postgresql.md) + for more information on backing up and upgrading your database. + +- If you are not using the in-cluster database, you can set + `POSTGRES_ENABLED` to `false` and re-deploy. This option is especially relevant to + users of *custom charts without the in-chart PostgreSQL dependency*. + Database auto-detection is based on the `postgresql.enabled` Helm value for + your release. This value is set based on the `POSTGRES_ENABLED` CI/CD variable + and persisted by Helm, regardless of whether or not your chart uses the + variable. + +WARNING: +Setting `POSTGRES_ENABLED` to `false` permanently deletes any existing +channel 1 database for your environment. + +## Error: unable to recognize "": no matches for kind "Deployment" in version "extensions/v1beta1" + +After upgrading your Kubernetes cluster to [v1.16+](stages.md#kubernetes-116), +you may encounter this message when deploying with Auto DevOps: + +```plaintext +UPGRADE FAILED +Error: failed decoding reader into objects: unable to recognize "": no matches for kind "Deployment" in version "extensions/v1beta1" +``` + +This can occur if your current deployments on the environment namespace were deployed with a +deprecated/removed API that doesn't exist in Kubernetes v1.16+. For example, +if [your in-cluster PostgreSQL was installed in a legacy way](#detected-an-existing-postgresql-database), +the resource was created via the `extensions/v1beta1` API. However, the deployment resource +was moved to the `app/v1` API in v1.16. + +To recover such outdated resources, you must convert the current deployments by mapping legacy APIs +to newer APIs. There is a helper tool called [`mapkubeapis`](https://github.com/hickeyma/helm-mapkubeapis) +that works for this problem. Follow these steps to use the tool in Auto DevOps: + +1. Modify your `.gitlab-ci.yml` with: + + ```yaml + include: + - template: Auto-DevOps.gitlab-ci.yml + - remote: https://gitlab.com/shinya.maeda/ci-templates/-/raw/master/map-deprecated-api.gitlab-ci.yml + + variables: + HELM_VERSION_FOR_MAPKUBEAPIS: "v2" # If you're using auto-depoy-image v2 or above, please specify "v3". + ``` + +1. Run the job `<environment-name>:map-deprecated-api`. Ensure that this job succeeds before moving + to the next step. You should see something like the following output: + + ```shell + 2020/10/06 07:20:49 Found deprecated or removed Kubernetes API: + "apiVersion: extensions/v1beta1 + kind: Deployment" + Supported API equivalent: + "apiVersion: apps/v1 + kind: Deployment" + ``` + +1. Revert your `.gitlab-ci.yml` to the previous version. You no longer need to include the + supplemental template `map-deprecated-api`. + +1. Continue the deployments as usual. + +## Error: error initializing: Looks like "https://kubernetes-charts.storage.googleapis.com" is not a valid chart repository or cannot be reached + +As [announced in the official CNCF blog post](https://www.cncf.io/blog/2020/10/07/important-reminder-for-all-helm-users-stable-incubator-repos-are-deprecated-and-all-images-are-changing-location/), +the stable Helm chart repository was deprecated and removed on November 13th, 2020. +You may encounter this error after that date. + +Some GitLab features had dependencies on the stable chart. To mitigate the impact, we changed them +to use new official repositories or the [Helm Stable Archive repository maintained by GitLab](https://gitlab.com/gitlab-org/cluster-integration/helm-stable-archive). +Auto Deploy contains [an example fix](https://gitlab.com/gitlab-org/cluster-integration/auto-deploy-image/-/merge_requests/127). + +In Auto Deploy, `v1.0.6+` of `auto-deploy-image` no longer adds the deprecated stable repository to +the `helm` command. If you use a custom chart and it relies on the deprecated stable repository, +specify an older `auto-deploy-image` like this example: + +```yaml +include: + - template: Auto-DevOps.gitlab-ci.yml + +.auto-deploy: + image: "registry.gitlab.com/gitlab-org/cluster-integration/auto-deploy-image:v1.0.5" +``` + +Keep in mind that this approach stops working when the stable repository is removed, +so you must eventually fix your custom chart. + +To fix your custom chart: + +1. In your chart directory, update the `repository` value in your `requirements.yaml` file from : + + ```yaml + repository: "https://kubernetes-charts.storage.googleapis.com/" + ``` + + to: + + ```yaml + repository: "https://charts.helm.sh/stable" + ``` + +1. In your chart directory, run `helm dep update .` using the same Helm major version as Auto DevOps. +1. Commit the changes for the `requirements.yaml` file. +1. If you previously had a `requirements.lock` file, commit the changes to the file. + If you did not previously have a `requirements.lock` file in your chart, + you do not need to commit the new one. This file is optional, but when present, + it's used to verify the integrity of the downloaded dependencies. + +You can find more information in +[issue #263778, "Migrate PostgreSQL from stable Helm repository"](https://gitlab.com/gitlab-org/gitlab/-/issues/263778). + +## Error: release .... failed: timed out waiting for the condition + +When getting started with Auto DevOps, you may encounter this error when first +deploying your application: + +```plaintext +INSTALL FAILED +PURGING CHART +Error: release staging failed: timed out waiting for the condition +``` + +This is most likely caused by a failed liveness (or readiness) probe attempted +during the deployment process. By default, these probes are run against the root +page of the deployed application on port 5000. If your application isn't configured +to serve anything at the root page, or is configured to run on a specific port +*other* than 5000, this check fails. + +If it fails, you should see these failures in the events for the relevant +Kubernetes namespace. These events look like the following example: + +```plaintext +LAST SEEN TYPE REASON OBJECT MESSAGE +3m20s Warning Unhealthy pod/staging-85db88dcb6-rxd6g Readiness probe failed: Get http://10.192.0.6:5000/: dial tcp 10.192.0.6:5000: connect: connection refused +3m32s Warning Unhealthy pod/staging-85db88dcb6-rxd6g Liveness probe failed: Get http://10.192.0.6:5000/: dial tcp 10.192.0.6:5000: connect: connection refused +``` + +To change the port used for the liveness checks, pass +[custom values to the Helm chart](customize.md#customize-values-for-helm-chart) +used by Auto DevOps: + +1. Create a directory and file at the root of your repository named `.gitlab/auto-deploy-values.yaml`. + +1. Populate the file with the following content, replacing the port values with + the actual port number your application is configured to use: + + ```yaml + service: + internalPort: <port_value> + externalPort: <port_value> + ``` + +1. Commit your changes. + +After committing your changes, subsequent probes should use the newly-defined ports. +The page that's probed can also be changed by overriding the `livenessProbe.path` +and `readinessProbe.path` values (shown in the +[default `values.yaml`](https://gitlab.com/gitlab-org/cluster-integration/auto-deploy-image/-/blob/master/assets/auto-deploy-app/values.yaml) +file) in the same fashion. diff --git a/doc/topics/autodevops/upgrading_auto_deploy_dependencies.md b/doc/topics/autodevops/upgrading_auto_deploy_dependencies.md index 5f8dfcdfc05..0dabb80204a 100644 --- a/doc/topics/autodevops/upgrading_auto_deploy_dependencies.md +++ b/doc/topics/autodevops/upgrading_auto_deploy_dependencies.md @@ -5,7 +5,7 @@ info: To determine the technical writer assigned to the Stage/Group associated w type: reference --- -# Upgrading deployments for newer Auto Deploy dependencies (Auto Deploy template, auto-deploy-image and auto-deploy-app chart) +# Upgrading deployments for newer Auto Deploy dependencies [Auto Deploy](stages.md#auto-deploy) is a feature that deploys your application to a Kubernetes cluster. It consists of several dependencies: @@ -114,7 +114,7 @@ If your Auto DevOps project has an active environment that was deployed with the job saves a backup for 1 week in a job artifact called `helm-2-release-backups`. The backup is in a Kubernetes manifest file that can be restored using `kubectl apply -f $backup`. -1. Remove the `MIGRATE_HELM_2TO3` variable. +1. Remove the `MIGRATE_HELM_2TO3` CI/CD variable. #### In-Cluster PostgreSQL Channel 2 @@ -145,11 +145,11 @@ steps to upgrade to v2: them to `production` first to delete the unstable tracks. 1. Verify your project is [using the v2 `auto-deploy-image`](#verify-dependency-versions). If not, [specify the version](#use-a-specific-version-of-auto-deploy-dependencies). -1. Add an `AUTO_DEVOPS_FORCE_DEPLOY_V2` environment variable with a value of `true` +1. Add an `AUTO_DEVOPS_FORCE_DEPLOY_V2` CI/CD variable with a value of `true` in the GitLab CI/CD settings. 1. Create a new pipeline and run the `production` job to renew the resource architecture with the v2 `auto-deploy-app chart`. -1. Remove the `AUTO_DEVOPS_FORCE_DEPLOY_V2` environment variable. +1. Remove the `AUTO_DEVOPS_FORCE_DEPLOY_V2` variable. ### Use a specific version of Auto Deploy dependencies @@ -167,7 +167,7 @@ include: ### Ignore warnings and continue deploying If you are certain that the new chart version is safe to be deployed, you can add -the `AUTO_DEVOPS_FORCE_DEPLOY_V<major-version-number>` [environment variable](customize.md#build-and-deployment) +the `AUTO_DEVOPS_FORCE_DEPLOY_V<major-version-number>` [CI/CD variable](customize.md#build-and-deployment) to force the deployment to continue. For example, if you want to deploy the `v2.0.0` chart on a deployment that previously diff --git a/doc/topics/autodevops/upgrading_postgresql.md b/doc/topics/autodevops/upgrading_postgresql.md index a0c4a41f90d..35b9d2e055c 100644 --- a/doc/topics/autodevops/upgrading_postgresql.md +++ b/doc/topics/autodevops/upgrading_postgresql.md @@ -27,14 +27,14 @@ involves: ## Prerequisites 1. Install - [`kubectl`](https://kubernetes.io/docs/tasks/tools/install-kubectl/). + [`kubectl`](https://kubernetes.io/docs/tasks/tools/). 1. Ensure that you can access your Kubernetes cluster using `kubectl`. This varies based on Kubernetes providers. 1. Prepare for downtime. The steps below include taking the application offline so that the in-cluster database does not get modified after the database dump is created. 1. Ensure you have not set `POSTGRES_ENABLED` to `false`, as this setting deletes any existing channel 1 database. For more information, see - [Detected an existing PostgreSQL database](index.md#detected-an-existing-postgresql-database). + [Detected an existing PostgreSQL database](troubleshooting.md#detected-an-existing-postgresql-database). NOTE: If you have configured Auto DevOps to have staging, diff --git a/doc/topics/git/cherry_picking.md b/doc/topics/git/cherry_picking.md new file mode 100644 index 00000000000..5a0867371bb --- /dev/null +++ b/doc/topics/git/cherry_picking.md @@ -0,0 +1,53 @@ +--- +stage: Create +group: Source Code +info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/engineering/ux/technical-writing/#assignments +comments: false +--- + +# Cherry Pick + +Given an existing commit on one branch, apply the change to another branch. + +This can be useful for backporting bug fixes to previous release branches. Make +the commit on the default branch, and then cherry pick it into the release branch. + +## Sample workflow + +1. Check out a new `stable` branch from the default branch: + + ```shell + git checkout master + git checkout -b stable + ``` + +1. Change back to the default branch: + + ```shell + git checkout master + ``` + +1. Make any required changes, then commit the changes: + + ```shell + git add changed_file.rb + git commit -m 'Fix bugs in changed_file.rb' + ``` + +1. Review the commit log and copy the SHA of the latest commit: + + ```shell + git log + ``` + +1. Check out the `stable` branch: + + ```shell + git checkout stable + ``` + +1. Cherry pick the commit by using the SHA copied previously: + + ```shell + git cherry-pick <commit SHA> + ``` diff --git a/doc/topics/git/feature_branch_development.md b/doc/topics/git/feature_branch_development.md index 842c39f0bfd..5f4b5632f33 100644 --- a/doc/topics/git/feature_branch_development.md +++ b/doc/topics/git/feature_branch_development.md @@ -56,7 +56,7 @@ In this case, the feature branch would be `release-X-Y`. Assuming the `release-X  -1. After you click **Create merge request**, you'll see an option to **Change branches**. Select that option. +1. After you click **Create merge request**, an option to **Change branches** displays. Select that option. 1. In the **New Merge Request** screen, you can now select the **Source** and **Target** branches. In the screenshot shown, @@ -73,10 +73,10 @@ we have selected `test-branch` as the source, and `release-13-0` as the target. From test-branch into release-13-0 ``` - An entry like this confirms that your MR will **not** merge into master. + An entry like this confirms your merge request's destination. 1. Make any additional changes in the **New Merge Request** screen, and click **Submit merge request**. -1. In the new merge request, look for **Request to merge**. You'll see an entry similar to: +1. In the new merge request, look for **Request to merge**. An entry similar to this displays: ```plaintext Request to merge test-branch into release-13-0 diff --git a/doc/topics/git/how_to_install_git/index.md b/doc/topics/git/how_to_install_git/index.md index 17c5f31705f..8b097c4c1da 100644 --- a/doc/topics/git/how_to_install_git/index.md +++ b/doc/topics/git/how_to_install_git/index.md @@ -9,9 +9,9 @@ type: howto # Installing Git **(FREE)** To begin contributing to GitLab projects, -you will need to install the Git client on your computer. +you must install the Git client on your computer. -This article will show you how to install Git on macOS, Ubuntu Linux and Windows. +This article shows you how to install Git on macOS, Ubuntu Linux and Windows. Information on [installing Git](https://git-scm.com/book/en/v2/Getting-Started-Installing-Git) is also available at the official Git website. diff --git a/doc/topics/git/index.md b/doc/topics/git/index.md index 52e8a42de76..2c28bdcb0ed 100644 --- a/doc/topics/git/index.md +++ b/doc/topics/git/index.md @@ -14,7 +14,7 @@ large projects with speed and efficiency. [GitLab](https://about.gitlab.com) is a Git-based fully integrated platform for software development. Besides Git's functionalities, GitLab has a lot of powerful [features](https://about.gitlab.com/features/) to enhance your -[workflow](https://about.gitlab.com/blog/2016/10/25/gitlab-workflow-an-overview/). +[workflow](https://about.gitlab.com/topics/version-control/what-is-gitlab-workflow/). We've gathered some resources to help you to get the best from Git with GitLab. @@ -22,7 +22,7 @@ More information is also available on the [Git website](https://git-scm.com). ## Getting started -The following resources will help you get started with Git: +The following resources can help you get started with Git: - [Git-ing started with Git](https://www.youtube.com/watch?v=Ce5nz5n41z4), a video introduction to Git. @@ -42,7 +42,7 @@ The following resources will help you get started with Git: - [Git stash](../../university/training/topics/stash.md) - [Git file blame](../../user/project/repository/git_blame.md) - [Git file history](../../user/project/repository/git_history.md) -- [Git tags](../../university/training/user_training.md#tags) +- [Git tags](tags.md) ### Concepts @@ -84,7 +84,7 @@ The following are advanced topics for those who want to get the most out of Git: - [Introduction to Git rebase, force-push, and merge conflicts](git_rebase.md) - [Server Hooks](../../administration/server_hooks.md) - [Git Attributes](../../user/project/git_attributes.md) -- Git Submodules: [Using Git submodules with GitLab CI](../../ci/git_submodules.md#using-git-submodules-with-gitlab-ci) +- Git Submodules: [Using Git submodules with GitLab CI](../../ci/git_submodules.md) - [Partial Clone](partial_clone.md) ## API diff --git a/doc/topics/git/lfs/index.md b/doc/topics/git/lfs/index.md index 14bb28d2477..11c0fcc2373 100644 --- a/doc/topics/git/lfs/index.md +++ b/doc/topics/git/lfs/index.md @@ -269,3 +269,46 @@ You might choose to do this if you are using an appliance like a <!-- vale gitla GitLab can't verify LFS objects. Pushes then fail if you have GitLab LFS support enabled. To stop push failure, LFS support can be disabled in the [Project settings](../../../user/project/settings/index.md), which also disables GitLab LFS value-adds (Verifying LFS objects, UI integration for LFS). + +### Missing LFS objects + +An error about a missing LFS object may occur in either of these situations: + +- When migrating LFS objects from disk to object storage, with error messages like: + + ```plaintext + ERROR -- : Failed to transfer LFS object + 006622269c61b41bf14a22bbe0e43be3acf86a4a446afb4250c3794ea47541a7 + with error: No such file or directory @ rb_sysopen - + /var/opt/gitlab/gitlab-rails/shared/lfs-objects/00/66/22269c61b41bf14a22bbe0e43be3acf86a4a446afb4250c3794ea47541a7 + ``` + + (Line breaks have been added for legibility.) + +- When running the + [integrity check for LFS objects](../../../administration/raketasks/check.md#uploaded-files-integrity) + with the `VERBOSE=1` parameter. + +The database can have records for LFS objects which are not on disk. The database entry may +[prevent a new copy of the object being pushed](https://gitlab.com/gitlab-org/gitlab-foss/-/issues/49241). +To delete these references: + +1. [Start a rails console](../../../administration/operations/rails_console.md). +1. Query the object that's reported as missing in the rails console, to return a file path: + + ```ruby + lfs_object = LfsObject.find_by(oid: '006622269c61b41bf14a22bbe0e43be3acf86a4a446afb4250c3794ea47541a7') + lfs_object.file.path + ``` + +1. Check on disk if it exists: + + ```shell + ls -al /var/opt/gitlab/gitlab-rails/shared/lfs-objects/00/66/22269c61b41bf14a22bbe0e43be3acf86a4a446afb4250c3794ea47541a7 + ``` + +1. If the file is not present, remove the database record via the rails console: + + ```ruby + lfs_object.destroy + ``` diff --git a/doc/topics/git/numerous_undo_possibilities_in_git/index.md b/doc/topics/git/numerous_undo_possibilities_in_git/index.md index c263609125f..76fc9bc92b0 100644 --- a/doc/topics/git/numerous_undo_possibilities_in_git/index.md +++ b/doc/topics/git/numerous_undo_possibilities_in_git/index.md @@ -64,7 +64,7 @@ To avoid chaos with development workflows have to be followed. It depends on your internal workflow how certain changes or commits can be undone or changed. -[GitLab Flow](https://about.gitlab.com/blog/2014/09/29/gitlab-flow/) provides a good +[GitLab Flow](https://about.gitlab.com/topics/version-control/what-is-gitlab-flow/) provides a good balance between developers clashing with each other while developing the same feature and cooperating seamlessly. However, it does not enable joined development of the same feature by multiple developers by default. diff --git a/doc/topics/git/partial_clone.md b/doc/topics/git/partial_clone.md index d9c239486f5..7f2543f040a 100644 --- a/doc/topics/git/partial_clone.md +++ b/doc/topics/git/partial_clone.md @@ -23,13 +23,13 @@ Git 2.22.0 or later is required. > [Introduced](https://gitlab.com/gitlab-org/gitaly/-/issues/2553) in GitLab 12.10. Storing large binary files in Git is normally discouraged, because every large -file added will be downloaded by everyone who clones or fetches changes +file added is downloaded by everyone who clones or fetches changes thereafter. This is slow, if not a complete obstruction when working from a slow or unreliable internet connection. Using partial clone with a file size filter solves this problem, by excluding troublesome large files from clones and fetches. When Git encounters a missing -file, it will be downloaded on demand. +file, it's downloaded on demand. When cloning a repository, use the `--filter=blob:limit=<size>` argument. For example, to clone the repository excluding files larger than 1 megabyte: @@ -58,7 +58,7 @@ Updating files: 100% (13008/13008), done. Filtering content: 100% (3/3), 131.24 MiB | 4.65 MiB/s, done. ``` -The output will be longer because Git will first clone the repository excluding +The output is longer because Git first clones the repository excluding files larger than 1 megabyte, and second download any missing large files needed to checkout the `master` branch. @@ -109,8 +109,8 @@ For more details, see the Git documentation for ## Filter by file path WARNING: -Partial Clone using `sparse` filters is experimental, slow, and will -significantly increase Gitaly resource utilization when cloning and fetching. +Partial Clone using `sparse` filters is experimental, slow, and +significantly increases Gitaly resource utilization when cloning and fetching. Deeper integration between Partial Clone and Sparse Checkout is being explored through the `--filter=sparse:oid=<blob-ish>` filter spec, but this is highly @@ -143,7 +143,7 @@ For more details, see the Git documentation for ``` 1. **Create a new Git repository and fetch.** Support for `--filter=sparse:oid` - using the clone command is incomplete, so we will emulate the clone command + using the clone command is incomplete, so we emulate the clone command by hand, using `git init` and `git fetch`. Follow [issue tracking support for `--filter=sparse:oid`](https://gitlab.com/gitlab-org/git/-/issues/4) for updates. @@ -171,7 +171,7 @@ For more details, see the Git documentation for WARNING: Git integrations with `bash`, `zsh`, etc and editors that automatically - show Git status information often run `git fetch` which will fetch the + show Git status information often run `git fetch` which fetches the entire repository. You many need to disable or reconfigure these integrations. diff --git a/doc/topics/git/tags.md b/doc/topics/git/tags.md new file mode 100644 index 00000000000..76e3cff3edc --- /dev/null +++ b/doc/topics/git/tags.md @@ -0,0 +1,41 @@ +--- +stage: Create +group: Source Code +info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/engineering/ux/technical-writing/#assignments +--- + +# Tags + +Tags are useful for marking certain deployments and releases for later +reference. Git supports two types of tags: + +- Annotated tags: An unchangeable part of Git history. +- Lightweight (soft) tags: Tags that can be set and removed as needed. + +Many projects combine an annotated release tag with a stable branch. Consider +setting deployment or release tags automatically. + +## Tags sample workflow + +1. Create a lightweight tag. +1. Create an annotated tag. +1. Push the tags to the remote repository. + +```shell +git checkout master + +# Lightweight tag +git tag my_lightweight_tag + +# Annotated tag +git tag -a v1.0 -m ‘Version 1.0’ + +# Show list of the existing tags +git tag + +git push origin --tags +``` + +## Additional resources + +- [Tagging](https://git-scm.com/book/en/v2/Git-Basics-Tagging) Git reference page diff --git a/doc/topics/git/troubleshooting_git.md b/doc/topics/git/troubleshooting_git.md index 528a9a4ba00..be4903e2cb9 100644 --- a/doc/topics/git/troubleshooting_git.md +++ b/doc/topics/git/troubleshooting_git.md @@ -45,7 +45,7 @@ set to 50MB. The default is 1MB. **If pushing over SSH**, first check your SSH configuration as 'Broken pipe' errors can sometimes be caused by underlying issues with SSH (such as authentication). Make sure that SSH is correctly configured by following the -instructions in the [SSH troubleshooting](../../ssh/README.md#troubleshooting) documentation. +instructions in the [SSH troubleshooting](../../ssh/README.md#troubleshooting-ssh-connections) documentation. If you're a GitLab administrator and have access to the server, you can also prevent session timeouts by configuring SSH `keep alive` either on the client or on the server. diff --git a/doc/topics/gitlab_flow.md b/doc/topics/gitlab_flow.md index d02573a0e06..c238951be51 100644 --- a/doc/topics/gitlab_flow.md +++ b/doc/topics/gitlab_flow.md @@ -187,7 +187,7 @@ The name of a branch might be dictated by organizational standards. When you are done or want to discuss the code, open a merge request. A merge request is an online place to discuss the change and review the code. -If you open the merge request but do not assign it to anyone, it is a [draft merge request](../user/project/merge_requests/work_in_progress_merge_requests.md). +If you open the merge request but do not assign it to anyone, it is a [draft merge request](../user/project/merge_requests/drafts.md). These are used to discuss the proposed implementation but are not ready for inclusion in the `master` branch yet. Start the title of the merge request with `[Draft]`, `Draft:` or `(Draft)` to prevent it from being merged before it's ready. |
