Welcome to mirror list, hosted at ThFree Co, Russian Federation.

gitlab.com/gitlab-org/gitlab-foss.git - Unnamed repository; edit this file 'description' to name the repository.
summaryrefslogtreecommitdiff
path: root/doc
diff options
context:
space:
mode:
authorGitLab Bot <gitlab-bot@gitlab.com>2023-04-28 21:26:46 +0300
committerGitLab Bot <gitlab-bot@gitlab.com>2023-04-28 21:26:46 +0300
commit5509e479900ee537980a126287c20327c41a61d6 (patch)
tree8272f06bd58b1518eca38975f95656ffc5497bd2 /doc
parente0529f76a36026dc4bd51fbec1e5c52e7f3866e1 (diff)
Add latest changes from gitlab-org/gitlab@master
Diffstat (limited to 'doc')
-rw-r--r--doc/.vale/gitlab/spelling-exceptions.txt2
-rw-r--r--doc/administration/reference_architectures/index.md1
-rw-r--r--doc/ci/quick_start/index.md3
-rw-r--r--doc/ci/quick_start/tutorial.md504
-rw-r--r--doc/development/database/batched_background_migrations.md43
-rw-r--r--doc/tutorials/build_application.md1
-rw-r--r--doc/user/project/repository/code_suggestions.md54
7 files changed, 578 insertions, 30 deletions
diff --git a/doc/.vale/gitlab/spelling-exceptions.txt b/doc/.vale/gitlab/spelling-exceptions.txt
index b24608f1a1c..9345b4a7d79 100644
--- a/doc/.vale/gitlab/spelling-exceptions.txt
+++ b/doc/.vale/gitlab/spelling-exceptions.txt
@@ -292,6 +292,7 @@ Dockerfiles
Dockerize
Dockerized
Dockerizing
+Docusaurus
dogfood
dogfooding
dogfoods
@@ -436,6 +437,7 @@ hotfixed
hotfixes
hotfixing
hotspots
+HTMLHint
http
https
hyperparameter
diff --git a/doc/administration/reference_architectures/index.md b/doc/administration/reference_architectures/index.md
index f40c8fc3c67..a2b348fb642 100644
--- a/doc/administration/reference_architectures/index.md
+++ b/doc/administration/reference_architectures/index.md
@@ -332,6 +332,7 @@ Several database cloud provider services are known not to support the above or h
- [Amazon Aurora](https://aws.amazon.com/rds/aurora/) is incompatible and not supported. See [14.4.0](../../update/index.md#1440) for more details.
- [Azure Database for PostgreSQL Single Server](https://azure.microsoft.com/en-gb/products/postgresql/#overview) (Single / Flexible) is not supported for use due to notable performance / stability issues or missing functionality. See [Recommendation Notes for Azure](#recommendation-notes-for-azure) for more details.
+- Azure Database for PostgreSQL Flexible Server uses Microsoft Azure Active Directory (Azure AD) as authentication mechanism, which is incompatible with GitLab database integration.
- [Google AlloyDB](https://cloud.google.com/alloydb) and [Amazon RDS Multi-AZ DB cluster](https://docs.aws.amazon.com/AmazonRDS/latest/UserGuide/multi-az-db-clusters-concepts.html) have not been tested and are not recommended. Both solutions are specifically not expected to work with GitLab Geo.
- [Amazon RDS Multi-AZ DB instance](https://docs.aws.amazon.com/AmazonRDS/latest/UserGuide/Concepts.MultiAZSingleStandby.html) is a separate product and is supported.
diff --git a/doc/ci/quick_start/index.md b/doc/ci/quick_start/index.md
index 3e1cbca8bfa..ec58491e604 100644
--- a/doc/ci/quick_start/index.md
+++ b/doc/ci/quick_start/index.md
@@ -9,6 +9,9 @@ type: reference
This tutorial shows you how to configure and run your first CI/CD pipeline in GitLab.
+If you are already familiar with basic CI/CD concepts, you can learn about
+common keywords in [Tutorial: Create a complex pipeline](tutorial.md).
+
## Prerequisites
Before you start, make sure you have:
diff --git a/doc/ci/quick_start/tutorial.md b/doc/ci/quick_start/tutorial.md
new file mode 100644
index 00000000000..3e12c319069
--- /dev/null
+++ b/doc/ci/quick_start/tutorial.md
@@ -0,0 +1,504 @@
+---
+stage: Verify
+group: Pipeline Authoring
+info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments
+---
+
+# Tutorial: Create a complex pipeline
+
+This tutorial walks you through configuring a progressively more complex CI/CD pipeline
+through small, iterative steps. The pipeline is always fully functional,
+but it gains more functionality with each step.
+
+When you finish this tutorial, you will have a new project on GitLab.com and a working documentation site on
+[Docusaurus](https://docusaurus.io/).
+
+To complete this tutorial, you will:
+
+1. Create a project to hold the Docusaurus files
+1. Create the initial pipeline configuration file
+1. Add a job to build the site
+1. Add a job to deploy the site
+1. Add test jobs
+1. Start using merge request pipelines
+1. Reduce duplicated configuration
+
+## Prerequisites
+
+- You need an account on GitLab.com.
+- You should be familiar with Git.
+- Node.js must be installed on your local machine. For example, on macOS you can
+ [install node](https://formulae.brew.sh/formula/node) with `brew install node`.
+
+## Create a project to hold the Docusaurus files
+
+Before adding the pipeline configuration, you must first set up a Docusaurus project
+on GitLab.com:
+
+1. Create a new project under your username (not a group):
+ 1. On the top bar, select **Main menu > Projects > View all projects**.
+ 1. On the right of the page, select **New project**.
+ 1. Select **Create blank project**.
+ 1. Enter the project details:
+ - In the **Project name** field, enter the name of your project, for example `My Pipeline Tutorial Project`.
+ - Select **Initialize repository with a README**.
+ 1. Select **Create project**.
+1. On the right of the **Project Overview** page for your project, select **Clone**
+ to find the clone paths for your project. Copy the SSH or HTTP path and use the path
+ to clone the project locally.
+
+ For example, to clone with SSH into a `pipeline-tutorial` directory on your computer:
+
+ ```shell
+ git clone git@gitlab.com:my-username/my-pipeline-tutorial-project.git pipeline-tutorial
+ ```
+
+1. Change to the project's directory, then generate a new Docusaurus site:
+
+ ```shell
+ cd pipeline-tutorial
+ npm init docusaurus
+ ```
+
+ The Docusaurus initialization wizard prompts you with questions about the site.
+ Use all the default options.
+
+1. The initialization wizard sets up the site in `website/`, but the site should be in
+ the root of the project. Move the files up to the root and delete the old directory:
+
+ ```shell
+ mv website/* .
+ rm -r website
+ ```
+
+1. Update the Docusaurus configuration file with the details of your GitLab project.
+ In `docusaurus.config.js`:
+
+ - Set `url:` to a path with this format: `https://<my-username>.gitlab.io/`.
+ - Set `baseUrl:` to your project name, like `/my-pipeline-tutorial-project/`.
+
+1. Commit the changes, and push them to GitLab:
+
+ ```shell
+ git add .
+ git commit -m "Add simple generated Docusaurus site"
+ git push origin
+ ```
+
+## Create the initial CI/CD configuration file
+
+Start with the simplest possible pipeline configuration file to ensure CI/CD is enabled
+in the project and runners are available to run jobs.
+
+This step introduces:
+
+- [Jobs](../jobs/index.md): These are self-contained parts of a pipeline that run your commands.
+ Jobs run on [runners](../runners/index.md), separate from the GitLab instance.
+- [`script`](../yaml/index.md#script): This section of a job's configuration is
+ where you define the commands for jobs. If there are multiple commands (in an array),
+ they run in order. Each command executes as if it was run as a CLI command.
+ By default, if a command fails or returns an error, the job is flagged as failed
+ and no more commands run.
+
+In this step, create a `.gitlab-ci.yml` file in the root of the project with this configuration:
+
+```yaml
+test-job:
+ script:
+ - echo "This is my first job!"
+ - date
+```
+
+Commit and push this change to GitLab, then:
+
+1. Go to **Build > Pipelines** and make sure a pipeline runs in GitLab with this single job.
+1. Select the pipeline, then select the job to view the job's log and see the `This is my first job!` message
+ followed by the date.
+
+Now that you have a `.gitlab-ci.yml` file in your project, you can make all future changes
+to pipeline configuration with the [pipeline editor](../pipeline_editor/index.md).
+
+## Add a job to build the site
+
+A common task for a CI/CD pipeline is to build the code in the project then deploy it.
+Start by adding a job that builds the site.
+
+This step introduces:
+
+- [`image`](../yaml/index.md#image): Tell the runner which Docker
+ container to use to run the job in. The runner:
+ 1. Downloads the container image and starts it.
+ 1. Clones your GitLab project into the running container.
+ 1. Runs the `script` commands, one at a time.
+- [`artifacts`](../yaml/index.md#artifacts): Jobs are self-contained and do not share
+ resources with each other. If you want files generated in one job to be used in
+ another job, you must save them as artifacts first. Then later jobs can retrieve the
+ artifacts and use the generated files.
+
+In this step, replace `test-job` with `build-job`:
+
+- Use `image` to configure the job to run with the latest `node` image. Docusaurus
+ is a Node.js project and the `node` image has the needed `npm` commands built in.
+- Run `npm install` to install Docusaurus into the running `node` container, then run
+ `npm run build` to build the site.
+- Docusaurus saves the built site in `build/`, so save these files with `artifacts`.
+
+```yaml
+build-job:
+ image: node
+ script:
+ - npm install
+ - npm run build
+ artifacts:
+ paths:
+ - "build/"
+```
+
+Use the pipeline editor to commit this pipeline configuration to the default branch,
+and check the job log. You can:
+
+- See the `npm` commands run and build the site.
+- Verify that the artifacts are saved at the end.
+- Browse the contents of the artifacts file by selecting **Browse** to the right of the job log
+ after the job completes.
+
+## Add a job to deploy the site
+
+After verifying the Docusaurus site builds in `build-job`, you can add a job that deploys it.
+
+This step introduces:
+
+- [`stage`](../yaml/index.md#stage) and [`stages](../yaml/index.md#stage): The most common
+ pipeline configurations group jobs into stages. Jobs in the same stage can run in parallel,
+ while jobs in later stages wait for jobs in earlier stages to complete. If a job fails,
+ the whole stage is considered failed and jobs in later stages do not start running.
+- [GitLab Pages](../../user/project/pages/index.md): To host your static site, you
+ will use GitLab Pages.
+
+In this step:
+
+- Add a job that fetches the built site and deploys it. When using GitLab Pages,
+ the job is always named `pages`. The artifacts from the `build-job` are fetched automatically
+ and extracted into the job. Pages looks for the site in the `public/` directory though,
+ so add a `script` command to move the site to that directory.
+- Add a `stages` section, and define the stages for each job. `build-job` runs first
+ in the `build` stage, and `pages` runs after in the `deploy` stage.
+
+```yaml
+stages: # List of stages for jobs and their order of execution
+ - build
+ - deploy
+
+build-job:
+ stage: build # Set this job to run in the `build` stage
+ image: node
+ script:
+ - npm install
+ - npm run build
+ artifacts:
+ paths:
+ - "build/"
+
+pages:
+ stage: deploy # Set this new job to run in the `deploy` stage
+ script:
+ - mv build/ public/
+ artifacts:
+ paths:
+ - "public/"
+```
+
+Use the pipeline editor to commit this pipeline configuration to the default branch,
+and view the pipeline details from the **Pipelines** list. Verify that:
+
+- The two jobs run in different stages, `build` and `deploy`.
+- After the `pages` job completes a `pages-deploy` job appears, which is the GitLab process
+ that deploys the Pages site. When that job completes, you can visit your new Docusaurus
+ site. The Pages documentation explains [the URL formatting](../../user/project/pages/getting_started_part_one.md#gitlab-pages-default-domain-names),
+ which should be similar to `https://<my-username>.gitlab.io/<my-pipeline-tutorial-project>/`.
+
+## Add test jobs
+
+Now that the site builds and deploys as expected, you can add tests and linting.
+For example, a Ruby project might run RSpec test jobs. Docusaurus is a static site
+that uses Markdown and generated HTML, so this tutorial adds jobs to test the Markdown and HTML.
+
+This step introduces:
+
+- [`allow_failure`](../yaml/index.md#allow_failure): Jobs that fail intermittently,
+ or are expected to fail, can slow down productivity or be difficult to troubleshoot.
+ Use `allow_failure` to let jobs fail without halting pipeline execution.
+- [`dependencies`](../yaml/index.md#dependencies): Use `dependencies` to control
+ artifact downloads in individual jobs by listing which jobs to fetch artifacts from.
+
+In this step:
+
+- Add a new `test` stage that runs between `build` and `deploy`. These three stages
+ are the default stages when `stages` is undefined in the configuration.
+- Add a `lint-markdown` job to run [markdownlint](https://github.com/DavidAnson/markdownlint)
+ and check the Markdown in your project. markdownlint is a static analysis tool that
+ checks that your Markdown files follow formatting standards.
+ - The sample Markdown files Docusaurus generates are in `blog/` and `docs/`.
+ - This tool scans the original Markdown files only, and does not need the generated HTML
+ saved in the `build-job` artifacts. Speed up the job with `dependencies: []`
+ so that it fetches no artifacts.
+ - A few of the sample Markdown files violate default markdownlint rules, so add
+ `allow_failure: true` to let the pipeline continue despite the rule violations.
+- Add a `test-html` job to run [HTMLHint](https://htmlhint.com/) and check the generated HTML.
+ HTMLHint is a static analysis tool that scans generated HTML for known issues.
+- Both `test-html` and `pages` need the generated HTML found in the `build-job` artifacts.
+ Jobs fetch artifacts from all jobs in earlier stages by default, but add `dependencies:`
+ to make sure the jobs don't accidentally download other artifacts after future pipeline changes.
+
+```yaml
+stages:
+ - build
+ - test # Add a `test` stage for the test jobs
+ - deploy
+
+build-job:
+ stage: build
+ image: node
+ script:
+ - npm install
+ - npm run build
+ artifacts:
+ paths:
+ - "build/"
+
+lint-markdown:
+ stage: test
+ image: node
+ dependencies: [] # Don't fetch any artifacts
+ script:
+ - npm install markdownlint-cli2 --global # Install markdownlint into the container
+ - markdownlint-cli2 -v # Verify the version, useful for troubleshooting
+ - markdownlint-cli2 "blog/**/*.md" "docs/**/*.md" # Lint all markdown files in blog/ and docs/
+ allow_failure: true # This job fails right now, but don't let it stop the pipeline.
+
+test-html:
+ stage: test
+ image: node
+ dependencies:
+ - build-job # Only fetch artifacts from `build-job`
+ script:
+ - npm install --save-dev htmlhint # Install HTMLHint into the container
+ - npx htmlhint --version # Verify the version, useful for troubleshooting
+ - npx htmlhint build/ # Lint all markdown files in blog/ and docs/
+
+pages:
+ stage: deploy
+ dependencies:
+ - build-job # Only fetch artifacts from `build-job`
+ script:
+ - mv build/ public/
+ artifacts:
+ paths:
+ - "public/"
+```
+
+Commit this pipeline configuration to the default branch, and view the pipeline details.
+
+- The `test-markdown` job fails because the sample Markdown violates the default
+ markdownlint rules, but is allowed to fail. You can:
+ - Ignore the violations for now. They do not need to be fixed as part of the tutorial.
+ - Fix the Markdown file violations. Then you can change `allow_failure` to `false`,
+ or remove `allow_failure` completely because `allow_failure: false` is the default behavior
+ when not defined.
+ - Add a markdownlint configuration file to limit which rule violations to alert on.
+- You can also make changes to the Markdown file content and see the changes on the site
+ after the next deployment.
+
+## Start using merge request pipelines
+
+With the pipeline configurations above, the site deploys every time a pipeline completes
+successfully, but this is not an ideal development workflow. It's better to work from
+feature branches and merge requests, and only deploy the site when changes merge
+to the default branch.
+
+This step introduces:
+
+- [`rules`](../yaml/index.md#rules): Add rules to each job to configure in which
+ pipelines they run. You can configure jobs to run in [merge request pipelines](../pipelines/merge_request_pipelines.md),
+ [scheduled pipelines](../pipelines/schedules.md), or other specific situations.
+ Rules are evaluated from top to bottom, and if a rule matches, the job is
+ added to the pipeline.
+- [CI/CD variables](../variables/index.md): use these environment variables
+ to configure job behavior in the configuration file and in script commands.
+ [Predefined CI/CD variables](../variables/predefined_variables.md) are variables
+ that you do not need to manually define. They are automatically injected into pipelines
+ so you can use them to configure your pipeline. Variables are usually formatted as `$VARIABLE_NAME`.
+ and predefined variables are usually prefixed with `$CI_`.
+
+In this step:
+
+- Create a new feature branch and make the changes in the branch instead of the default branch.
+- Add `rules` to each job:
+ - The site should only deploy for changes to the default branch.
+ - The other jobs should run for all changes in merge requests or the default branch.
+- With this pipeline configuration, you can work from a feature branch without running any jobs,
+ which saves resources. When you are ready to validate your changes, create a merge request
+ and a pipeline runs with the jobs configured to run in merge requests.
+- When your merge request is accepted and the changes merge to the default branch,
+ a new pipeline runs which also contains the `pages` deployment job. The site deploys
+ if no jobs fail.
+
+```yaml
+stages:
+ - build
+ - test
+ - deploy
+
+build-job:
+ stage: build
+ image: node
+ script:
+ - npm install
+ - npm run build
+ artifacts:
+ paths:
+ - "build/"
+ rules:
+ - if: $CI_PIPELINE_SOURCE == 'merge_request_event' # Run for all changes to a merge request's source branch
+ - if: $CI_COMMIT_BRANCH == $CI_DEFAULT_BRANCH # Run for all changes to the default branch
+
+lint-markdown:
+ stage: test
+ image: node
+ dependencies: []
+ script:
+ - npm install markdownlint-cli2 --global
+ - markdownlint-cli2 -v
+ - markdownlint-cli2 "blog/**/*.md" "docs/**/*.md"
+ allow_failure: true
+ rules:
+ - if: $CI_PIPELINE_SOURCE == 'merge_request_event' # Run for all changes to a merge request's source branch
+ - if: $CI_COMMIT_BRANCH == $CI_DEFAULT_BRANCH # Run for all changes to the default branch
+
+test-html:
+ stage: test
+ image: node
+ dependencies:
+ - build-job
+ script:
+ - npm install --save-dev htmlhint
+ - npx htmlhint --version
+ - npx htmlhint build/
+ rules:
+ - if: $CI_PIPELINE_SOURCE == 'merge_request_event' # Run for all changes to a merge request's source branch
+ - if: $CI_COMMIT_BRANCH == $CI_DEFAULT_BRANCH # Run for all changes to the default branch
+
+pages:
+ stage: deploy
+ dependencies:
+ - build-job
+ script:
+ - mv build/ public/
+ artifacts:
+ paths:
+ - "public/"
+ rules:
+ - if: $CI_COMMIT_BRANCH == $CI_DEFAULT_BRANCH # Run for all changes to the default branch only
+```
+
+Merge the changes in your merge request. This action updates the default branch. Verify that
+the new pipeline contains the `pages` job that deploys the site.
+
+Be sure to use feature branches and merge requests for all future changes to pipeline configuration.
+Other project changes, like creating a Git tag or adding a pipeline schedule, do not
+trigger pipelines unless you add rules for those cases too.
+
+## Reduce duplicated configuration
+
+The pipeline now contains three jobs that all have identical `rules` and `image`
+configuration. Instead of repeating these rules, use `extends` and `default` to create
+single sources of truth.
+
+This step introduces:
+
+- [Hidden jobs](../jobs/index.md#hide-jobs): Jobs that start with `.` are never
+ added to a pipeline. Use them to hold configuration you want to reuse.
+- [`extends`](../yaml/index.md#extends): Use extends to repeat configuration in
+ multiple places, often from hidden jobs. If you update the hidden job's configuration,
+ all jobs extending the hidden job use the updated configuration.
+- [`default`](../yaml/index.md#default): Set keyword defaults that apply to all jobs
+ when not defined.
+- YAML overriding: When reusing configuration with `extends` or `default`, you can explicitly
+ define a keyword in the job to override the `extends` or `default` configuration.
+
+In this step:
+
+- Add a `.standard-rules` hidden job to hold the rules that are repeated in `build-job`,
+ `lint-markdown`, and `test-html`.
+- Use `extends` to reuse the `.standard-rules` configuration in the three jobs.
+- Add a `default` section to define the `image` default as `node`.
+- The `pages` deployment job does not need the default `node` image, so explicitly use
+ [`busybox`](https://hub.docker.com/_/busybox), an extremely tiny and fast image.
+
+```yaml
+stages:
+ - build
+ - test
+ - deploy
+
+default: # Add a default section to define the `image` keyword's default value
+ image: node
+
+.standard-rules: # Make a hidden job to hold the common rules
+ rules:
+ - if: $CI_PIPELINE_SOURCE == 'merge_request_event'
+ - if: $CI_COMMIT_BRANCH == $CI_DEFAULT_BRANCH
+
+build-job:
+ extends:
+ - .standard-rules # Reuse the configuration in `.standard-rules` here
+ stage: build
+ script:
+ - npm install
+ - npm run build
+ artifacts:
+ paths:
+ - "build/"
+
+lint-markdown:
+ stage: test
+ extends:
+ - .standard-rules # Reuse the configuration in `.standard-rules` here
+ dependencies: []
+ script:
+ - npm install markdownlint-cli2 --global
+ - markdownlint-cli2 -v
+ - markdownlint-cli2 "blog/**/*.md" "docs/**/*.md"
+ allow_failure: true
+
+test-html:
+ stage: test
+ extends:
+ - .standard-rules # Reuse the configuration in `.standard-rules` here
+ dependencies:
+ - build-job
+ script:
+ - npm install --save-dev htmlhint
+ - npx htmlhint --version
+ - npx htmlhint build/
+
+pages:
+ stage: deploy
+ image: busybox # Override the default `image` value with `busybox`
+ dependencies:
+ - build-job
+ script:
+ - mv build/ public/
+ artifacts:
+ paths:
+ - "public/"
+ rules:
+ - if: $CI_COMMIT_BRANCH == $CI_DEFAULT_BRANCH
+```
+
+Use a merge request to commit this pipeline configuration to the default branch.
+The file is simpler, but it should have the same behavior as the previous step.
+
+You've just created a full pipeline and streamlined it to be more efficient. Nice work!
+Now you can take this knowledge, learn about [the rest of the `.gitlab-ci.yml` keywords](../yaml/index.md),
+and build your own pipelines.
diff --git a/doc/development/database/batched_background_migrations.md b/doc/development/database/batched_background_migrations.md
index 326d2795558..6a6b43e52a0 100644
--- a/doc/development/database/batched_background_migrations.md
+++ b/doc/development/database/batched_background_migrations.md
@@ -148,6 +148,49 @@ Make sure the newly-created data is either migrated, or
saved in both the old and new version upon creation. Removals in
turn can be handled by defining foreign keys with cascading deletes.
+### Job retry mechanism
+
+The batched background migrations retry mechanism ensures that a job is executed again in case of failure.
+The following diagram shows the different stages of our retry mechanism:
+
+```plantuml
+@startuml
+hide empty description
+note as N1
+ can_split?:
+ the failure is due to a query timeout
+end note
+[*] --> Running
+Running --> Failed
+note on link
+ if number of retries <= MAX_ATTEMPTS
+end note
+Running --> Succeeded
+Failed --> Running
+note on link
+ if number of retries > MAX_ATTEMPTS
+ and can_split? == true
+ then two jobs with smaller
+ batch size will be created
+end note
+Failed --> [*]
+Succeeded --> [*]
+@enduml
+```
+
+- `MAX_ATTEMPTS` is defined in the [`Gitlab::Database::BackgroundMigration`](https://gitlab.com/gitlab-org/gitlab/blob/master/lib/gitlab/database/background_migration/batched_job.rb)
+class.
+- `can_split?` is defined in the [`Gitlab::Database::BatchedJob`](https://gitlab.com/gitlab-org/gitlab/-/blob/master/lib/gitlab/database/background_migration/batched_job.rb) class.
+
+### Failed batched background migrations
+
+The whole batched background migration is marked as `failed`
+(`/chatops run batched_background_migrations status MIGRATION_ID` will show
+the migration as `failed`) if any of the following are true:
+
+- There are no more jobs to consume, and there are failed jobs.
+- More than [half of the jobs failed since the background migration was started](https://gitlab.com/gitlab-org/gitlab/blob/master/lib/gitlab/database/background_migration/batched_migration.rb).
+
### Requeuing batched background migrations
If one of the batched background migrations contains a bug that is fixed in a patch
diff --git a/doc/tutorials/build_application.md b/doc/tutorials/build_application.md
index 685cf408e77..2e0130e46ca 100644
--- a/doc/tutorials/build_application.md
+++ b/doc/tutorials/build_application.md
@@ -13,6 +13,7 @@ Use CI/CD pipelines to automatically build, test, and deploy your code.
| Topic | Description | Good for beginners |
|-------|-------------|--------------------|
| [Create and run your first GitLab CI/CD pipeline](../ci/quick_start/index.md) | Create a `.gitlab-ci.yml` file and start a pipeline. | **{star}** |
+| [Create a complex pipeline](../ci/quick_start/tutorial.md) | Learn about the most commonly used GitLab CI/CD keywords by building an increasingly complex pipeline. | |
| <i class="fa fa-youtube-play youtube" aria-hidden="true"></i> [Get started: Learn about CI/CD](https://www.youtube.com/watch?v=sIegJaLy2ug) (9m 02s) | Learn about the `.gitlab-ci.yml` file and how it's used. | **{star}** |
| [GitLab CI/CD](https://levelup.gitlab.com/courses/continuous-integration-and-delivery-ci-cd-with-gitlab) | Learn about GitLab CI/CD and build a pipeline in this self-paced course. | **{star}** |
| <i class="fa fa-youtube-play youtube" aria-hidden="true"></i> [CI deep dive](https://www.youtube.com/watch?v=ZVUbmVac-m8&list=PL05JrBw4t0KorkxIFgZGnzzxjZRCGROt_&index=27) (22m 51s) | Take a closer look at pipelines and continuous integration concepts. | |
diff --git a/doc/user/project/repository/code_suggestions.md b/doc/user/project/repository/code_suggestions.md
index 6d3ce5267cf..857c75f2140 100644
--- a/doc/user/project/repository/code_suggestions.md
+++ b/doc/user/project/repository/code_suggestions.md
@@ -22,9 +22,11 @@ as you type. Depending on the cursor position, the extension either:
To accept a suggestion, press <kbd>Tab</kbd>.
-Code Suggestions are supported in Visual Studio Code with the GitLab Workflow extension.
+Code Suggestions are available in Visual Studio Code when you have the GitLab Workflow extension installed.
-Code Suggestions may produce [low-quality or incomplete suggestions](#model-accuracy-and-quality). Beta users should read about the [known limitations](#known-limitations). The best results from Code Suggestions are expected for these six languages:
+## Supported languages
+
+Code Suggestions may produce [low-quality or incomplete suggestions](#model-accuracy-and-quality). The best results from Code Suggestions are expected for these six languages:
- C
- C++
@@ -39,10 +41,6 @@ GitLab is continuously improving the model and expects to support an additional
Usage of Code Suggestions is governed by the [GitLab Testing Agreement](https://about.gitlab.com/handbook/legal/testing-agreement/). Learn about [data usage when using Code Suggestions](#code-suggestions-data-usage).
-## Group level setting
-
-[Group owners](../../permissions.md#group-members-permissions) can enable Code Suggestions for all projects in a group by using the [group level Code Suggestions setting](../../group/manage.md#group-code-suggestions).
-
## Enable Code Suggestions in VS Code
Prerequisites:
@@ -73,41 +71,43 @@ Start typing and receive suggestions for your GitLab projects.
<iframe src="https://www.youtube-nocookie.com/embed/WnxBYxN2-p4" frameborder="0" allowfullscreen> </iframe>
</figure>
-## Code Suggestions Data Usage
+## Code Suggestions data usage
-### Overview
+Code Suggestions is a generative artificial intelligence (AI) model hosted on GitLab.com.
-Code Suggestions is a generative artificial intelligence (AI) model hosted on GitLab.com that can empower your developers to code more efficiently by suggesting code as they type.
+Your personal access token enables a secure API connection to GitLab.com. This API connection securely transmits a context window from VS Code to the Code Suggestions ML model for inference, and the generated suggestion is transmitted back to VS Code.
-The personal access token enables a secure API connection to GitLab.com. This API connection securely transmits a context window from VS Code to the Code Suggestions ML model for inference, and the generated suggestion is transmitted back to VS Code.
+### Data privacy
-#### Progressive enhancement
+Code Suggestions operate completely in the GitLab.com infrastructure, providing the same level of [security](https://about.gitlab.com/security/) as any other feature of GitLab.com, and processing any personal data in accordance with our [Privacy Statement](https://about.gitlab.com/privacy/).
-This feature is designed as a progressive enhancement to the existing VS Code GitLab Workflow plugin. Code Suggestions offer a completion if the machine learning engine can generate a recommendation. In the event of a connection issue or model inference failure, the feature gracefully degrades. Code Suggestions do not prevent you from writing code in VS Code.
+No new additional data is collected to enable this feature. The content of your GitLab hosted source code is not used as training data. Source code inference against the Code Suggestions model is not used to re-train the model. Your data also never leaves GitLab.com. All training and inference is done in GitLab.com infrastructure.
-#### Off by default
+[Read more about the security of GitLab.com](https://about.gitlab.com/security/faq/).
-Code Suggestions are off by default and require a group owner to enable the feature with a [group-level setting](#group-level-setting).
+### Training data
-After the group level setting is enabled, Developers using Visual Studio Code with the [GitLab Workflow extension](https://marketplace.visualstudio.com/items?itemName=GitLab.gitlab-workflow) can connect to GitLab.com via a GitLab [personal access token](../../profile/personal_access_tokens.md#create-a-personal-access-token) with the `read_api` and `read_user` scopes.
+Code Suggestions uses open source pre-trained base models from the [CodeGen family](https://openreview.net/forum?id=iaYcJKpY2B_) including CodeGen-MULTI and CodeGen-NL. We then re-train and fine-tune these base models with a customized open source dataset to enable multi-language support and additional use cases. This customized dataset contains non-preprocessed open source code in 13 programming languages from [The Pile](https://pile.eleuther.ai/) and the [Google BigQuery source code dataset](https://cloud.google.com/blog/topics/public-datasets/github-on-bigquery-analyze-all-the-open-source-code). We then process this raw dataset against heuristics that aim to increase the quality of the dataset.
-#### Generating suggestions
+The Code Suggestions model is not trained on GitLab customer data.
-Once configured by a developer in VS Code. The personal access token enables a secure API connection to GitLab.com. This API connection securely transmits a context window from VS Code to the Code Suggestions ML model for inference, and the generated suggestion is transmitted back to VS Code.
+### Off by default
-Code Suggestions only work when you have internet connectivity and can access GitLab.com. Code Suggestions are not available for self-managed customers, nor customers operating within an air-gapped environment.
+Code Suggestions are off by default and require a group owner to enable the feature with a group-level setting.
-### Stability and performance
+After the group-level setting is enabled, developers using Visual Studio Code with the [GitLab Workflow extension](https://marketplace.visualstudio.com/items?itemName=GitLab.gitlab-workflow) can connect to GitLab.com by using a GitLab [personal access token](../../profile/personal_access_tokens.md#create-a-personal-access-token) with the `read_api` and `read_user` scopes.
-This feature is currently in [Beta](/ee/policy/alpha-beta-support.md#beta). While the Code Suggestions inference API operates completely within GitLab.com's enterprise infrastructure, we expect a high demand for this Beta feature, which may cause degraded performance or unexpected downtime of the feature. We have built this feature to gracefully degrade and have controls in place to allow us to mitigate abuse or misuse. GitLab may disable this feature for any or all customers at any time at our discretion.
+## Progressive enhancement
-## Data privacy
+This feature is designed as a progressive enhancement to the existing VS Code GitLab Workflow plugin. Code Suggestions offer a completion if the machine learning engine can generate a recommendation. In the event of a connection issue or model inference failure, the feature gracefully degrades. Code Suggestions do not prevent you from writing code in VS Code.
-Code Suggestions operate completely in the GitLab.com infrastructure, providing the same level of [security](https://about.gitlab.com/security/) as any other feature of GitLab.com, and processing any personal data in accordance with our [Privacy Statement](https://about.gitlab.com/privacy/).
+### Internet connectivity
-No new additional data is collected to enable this feature. The content of your GitLab hosted source code is not used as training data. Source code inference against the Code Suggestions model is not used to re-train the model. Your data also never leaves GitLab.com. All training and inference is done in GitLab.com infrastructure.
+Code Suggestions only work when you have internet connectivity and can access GitLab.com. Code Suggestions are not available for self-managed customers, nor customers operating within an air-gapped environment.
-[Read more about the security of GitLab.com](https://about.gitlab.com/security/faq/).
+### Stability and performance
+
+This feature is currently in [Beta](/ee/policy/alpha-beta-support.md#beta). While the Code Suggestions inference API operates completely within the GitLab.com enterprise infrastructure, we expect a high demand for this Beta feature, which may cause degraded performance or unexpected downtime of the feature. We have built this feature to gracefully degrade and have controls in place to allow us to mitigate abuse or misuse. GitLab may disable this feature for any or all customers at any time at our discretion.
### Model accuracy and quality
@@ -117,12 +117,6 @@ GitLab uses a customized open source dataset to fine-tune the model to support m
GitLab is actively refining these models to improve the quality of recommendations, add support for more languages, and add protections to limit personal data, insecure code, and other unwanted behavior that the model may have learned from training data.
-### Training data
-
-Code Suggestions uses open source pre-trained base models from the [CodeGen family](https://openreview.net/forum?id=iaYcJKpY2B_) including CodeGen-MULTI and CodeGen-NL. We then re-train and fine-tune these base models with a customized open source dataset to enable multi-language support and additional use cases. This customized dataset contains non-preprocessed open source code in 13 programming languages from [The Pile](https://pile.eleuther.ai/) and [Google's BigQuery source code dataset](https://cloud.google.com/blog/topics/public-datasets/github-on-bigquery-analyze-all-the-open-source-code). We then process this raw dataset against heuristics that aim to increase the quality of the dataset.
-
-The Code Suggestions model is not trained on GitLab customer data.
-
## Known limitations
While in Beta, we are working on improving the accuracy of overall generated content. However, Code Suggestions may generate suggestions that are: