From 7e5f8d08817c5672a0fe2f36609897f20d7e120e Mon Sep 17 00:00:00 2001 From: GitLab Bot Date: Mon, 18 May 2020 06:08:14 +0000 Subject: Add latest changes from gitlab-org/gitlab@master --- doc/administration/integration/plantuml.md | 13 ++ .../troubleshooting/gitlab_rails_cheat_sheet.md | 24 +--- doc/ci/jenkins/index.md | 4 + doc/ci/merge_request_pipelines/index.md | 134 +++------------------ doc/ci/yaml/README.md | 101 +++++++++------- doc/integration/jenkins.md | 4 +- doc/user/application_security/sast/index.md | 3 +- 7 files changed, 95 insertions(+), 188 deletions(-) (limited to 'doc') diff --git a/doc/administration/integration/plantuml.md b/doc/administration/integration/plantuml.md index 76655a945a3..682e47adc21 100644 --- a/doc/administration/integration/plantuml.md +++ b/doc/administration/integration/plantuml.md @@ -115,6 +115,19 @@ that, login with an Admin account and do following: - Check **Enable PlantUML** checkbox. - Set the PlantUML instance as `https://gitlab.example.com/-/plantuml/`. +NOTE: **Note:** If you are using a PlantUML server running v1.2020.9 and +above (for example, [plantuml.com](https://plantuml.com)), set the `PLANTUML_ENCODING` +environment variable to enable the `deflate` compression. On Omnibus, +this can be done set in `/etc/gitlab.rb`: + +```ruby +gitlab_rails['env'] = { 'PLANTUML_ENCODING' => 'deflate' } +``` + +From GitLab 13.1 and later, PlantUML integration now +[requires a header prefix in the URL](https://github.com/plantuml/plantuml/issues/117#issuecomment-6235450160) +to distinguish different encoding types. + ## Creating Diagrams With PlantUML integration enabled and configured, we can start adding diagrams to diff --git a/doc/administration/troubleshooting/gitlab_rails_cheat_sheet.md b/doc/administration/troubleshooting/gitlab_rails_cheat_sheet.md index 33a64176357..2cbc994fb4c 100644 --- a/doc/administration/troubleshooting/gitlab_rails_cheat_sheet.md +++ b/doc/administration/troubleshooting/gitlab_rails_cheat_sheet.md @@ -572,29 +572,7 @@ Ci::Pipeline.where(project_id: p.id).where(status: 'pending').count ### Remove artifacts more than a week old -The Latest version of these steps can be found in the [job artifacts documentation](../job_artifacts.md) - -```ruby -### SELECTING THE BUILDS TO CLEAR -# For a single project: -project = Project.find_by_full_path('') -builds_with_artifacts = project.builds.with_downloadable_artifacts - -# Instance-wide: -builds_with_artifacts = Ci::Build.with_downloadable_artifacts - -# Prior to 10.6 the above lines would be: -# builds_with_artifacts = project.builds.with_artifacts -# builds_with_artifacts = Ci::Build.with_artifacts - -### CLEAR THEM OUT -# Note that this will also erase artifacts that developers marked to "Keep" -builds_to_clear = builds_with_artifacts.where("finished_at < ?", 1.week.ago) -builds_to_clear.each do |build| - build.artifacts_expire_at = Time.now - build.erase_erasable_artifacts! -end -``` +This section has been moved to the [job artifacts troubleshooting documentation](../job_artifacts.md#delete-job-artifacts-from-jobs-completed-before-a-specific-date). ### Find reason failure (for when build trace is empty) (Introduced in 10.3.0) diff --git a/doc/ci/jenkins/index.md b/doc/ci/jenkins/index.md index c9235c62a84..c4346005138 100644 --- a/doc/ci/jenkins/index.md +++ b/doc/ci/jenkins/index.md @@ -13,6 +13,10 @@ First of all, our [Quick Start Guide](../quick_start/README.md) contains a good You may also be interested in [Auto DevOps](../../topics/autodevops/index.md) which can potentially be used to build, test, and deploy your applications with little to no configuration needed at all. +For an example of how to convert a Jenkins pipeline into a GitLab CI/CD pipeline, +or how to use Auto DevOps to test your code automatically, watch the +[Migrating from Jenkins to GitLab](https://www.youtube.com/watch?v=RlEVGOpYF5Y) video. + For advanced CI/CD teams, [templates](#templates) can enable the reuse of pipeline configurations. Otherwise, read on for important information that will help you get the ball rolling. Welcome diff --git a/doc/ci/merge_request_pipelines/index.md b/doc/ci/merge_request_pipelines/index.md index 9e9d4ee403b..a724bf416b6 100644 --- a/doc/ci/merge_request_pipelines/index.md +++ b/doc/ci/merge_request_pipelines/index.md @@ -37,79 +37,23 @@ To enable pipelines for merge requests: ## Configuring pipelines for merge requests -To configure pipelines for merge requests, configure your [CI/CD configuration file](../yaml/README.md). -There are a few different ways to do this. +To configure pipelines for merge requests you need to configure your [CI/CD configuration file](../yaml/README.md). +There are a few different ways to do this: -### Enable pipelines for merge requests for all jobs +### Use `rules` to run pipelines for merge requests -The recommended method for enabling pipelines for merge requests for all jobs in -a pipeline is to use [`workflow:rules`](../yaml/README.md#workflowrules). - -In this example, the pipeline always runs for all merge requests, as well as for all changes -to the master branch: - -```yaml -workflow: - rules: - - if: $CI_MERGE_REQUEST_ID # Execute jobs in merge request context - - if: $CI_COMMIT_BRANCH == 'master' # Execute jobs when a new commit is pushed to master branch - -build: - stage: build - script: ./build - -test: - stage: test - script: ./test - -deploy: - stage: deploy - script: ./deploy -``` - -### Enable pipelines for merge requests for specific jobs - -To enable pipelines for merge requests for specific jobs, you can use -[`rules`](../yaml/README.md#rules). - -In the following example: - -- The `build` job runs for all changes to the `master` branch, as well as for all merge requests. -- The `test` job runs for all merge requests. -- The `deploy` job runs for all changes to the `master` branch, but does *not* run - for merge requests. - -```yaml -build: - stage: build - script: ./build - rules: - - if: $CI_COMMIT_BRANCH == 'master' # Execute jobs when a new commit is pushed to master branch - - if: $CI_MERGE_REQUEST_ID # Execute jobs in merge request context - -test: - stage: test - script: ./test - rules: - - if: $CI_MERGE_REQUEST_ID # Execute jobs in merge request context - -deploy: - stage: deploy - script: ./deploy - rules: - - if: $CI_COMMIT_BRANCH == 'master' # Execute jobs when a new commit is pushed to master branch -``` +When using `rules`, which is the preferred method, we recommend starting with one +of the [`workflow:rules` templates](../yaml/README.md#workflowrules-templates) to ensure +your basic configuration is correct. Instructions on how to do this, as well as how +to customize, are available at that link. ### Use `only` or `except` to run pipelines for merge requests -NOTE: **Note**: -The [`only` / `except`](../yaml/README.md#onlyexcept-basic) keywords are going to be deprecated -and you should not use them. - -To enable pipelines for merge requests, you can use `only / except`. When you use this method, -you have to specify `only: - merge_requests` for each job. +If you want to continue using `only/except`, this is possible but please review the drawbacks +below. -In this example, the pipeline contains a `test` job that is configured to run on merge requests. +When you use this method, you have to specify `only: - merge_requests` for each job. In this +example, the pipeline contains a `test` job that is configured to run on merge requests. The `build` and `deploy` jobs don't have the `only: - merge_requests` parameter, so they will not run on merge requests. @@ -259,60 +203,16 @@ The variable names begin with the `CI_MERGE_REQUEST_` prefix. ### Two pipelines created when pushing to a merge request -If two pipelines are created when you push a new change to a merge request, -check your CI configuration file. - -For example, with this `.gitlab-ci.yml` configuration: - -```yaml -test: - script: ./test - rules: - - if: $CI_MERGE_REQUEST_ID # Include this job in pipelines for merge request - - if: $CI_COMMIT_BRANCH # Include this job in all branch pipelines - # Or, if you are using the `only:` keyword: - # only: - # - merge_requests - # - branches -``` - -Two pipelines are created when you push a commit to a branch that also has a pending -merge request: - -- A merge request pipeline that runs for the changes in the merge request. In - **CI/CD > Pipelines**, the merge request icon (**{merge-request}**) - and the merge request ID are displayed. If you hover over the ID, the merge request name is displayed. - - ![MR pipeline icon example](img/merge_request_pipelines_doubled_MR_v12_09.png) - -- A "branch" pipeline that runs for the commit pushed to the branch. In **CI/CD > Pipelines**, - the branch icon (**{branch}**) and branch name are displayed. This pipeline is - created even if no merge request exists. - - ![branch pipeline icon example](img/merge_request_pipelines_doubled_branch_v12_09.png) - -With the example configuration above, there is overlap between these two events. -When you push a commit to a branch that also has an open merge request pending, -both types of pipelines are created. - -To fix this overlap, you must explicitly define which job should run for which -purpose, for example: - -```yaml -test: - script: ./test - rules: - - if: $CI_MERGE_REQUEST_ID # Include this job in pipelines for merge request - - if: $CI_COMMIT_BRANCH == 'master' # Include this job in master branch pipelines -``` +If you are experiencing duplicated pipelines when using `rules`, take a look at +the [key details when using `rules`](../yaml/README.md#key-details-when-using-rules), +which will help you get your starting configuration correct. -Similar `rules:` should be added to all jobs to avoid any overlapping pipelines. Alternatively, -you can use the [`workflow:`](../yaml/README.md#exclude-jobs-with-rules-from-certain-pipelines) -parameter to add the same rules to all jobs globally. +If you are seeing two pipelines when using `only/except`, please see the caveats +related to using `only/except` above (or, consider moving to `rules`). ### Two pipelines created when pushing an invalid CI configuration file -Similar to above, pushing to a branch with an invalid CI configuration file can trigger +Pushing to a branch with an invalid CI configuration file can trigger the creation of two types of failed pipelines. One pipeline is a failed merge request pipeline, and the other is a failed branch pipeline, but both are caused by the same invalid configuration. diff --git a/doc/ci/yaml/README.md b/doc/ci/yaml/README.md index b1112e68486..64ee2940eee 100644 --- a/doc/ci/yaml/README.md +++ b/doc/ci/yaml/README.md @@ -286,7 +286,45 @@ determine whether or not a pipeline is created. It currently accepts a single `rules:` key that operates similarly to [`rules:` defined within jobs](#rules), enabling dynamic configuration of the pipeline. -The configuration options currently available for `workflow:rules` are:​ +#### `workflow:rules` templates + +> [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/217732) in GitLab 13.0. + +We provide pre-made templates for use with your pipelines that set up `workflow: rules` +for common scenarios. Usage of these will make things easier and prevent duplicate pipelines from running. + +The [`Branch-Pipelines` template](https://gitlab.com/gitlab-org/gitlab/-/tree/master/lib/gitlab/ci/templates/Workflows/Branch-Pipelines.gitlab-ci.yml) +makes your pipelines run for branches and tags. + +Branch pipeline status will be displayed within merge requests that use that branch +as a source, but this pipeline type does not support any features offered by +[Merge Request Pipelines](../merge_request_pipelines/) like +[Pipelines for Merge Results](../merge_request_pipelines/#pipelines-for-merged-results-premium) +or [Merge Trains](../merge_request_pipelines/pipelines_for_merged_results/merge_trains/). +Use this template if you are intentionally avoiding those features. + +It is [included](#include) as follows: + +```yaml +include: + - template: 'Workflows/Branch-Pipelines.gitlab-ci.yml' +``` + +The [`MergeRequest-Pipelines` include](https://gitlab.com/gitlab-org/gitlab/-/tree/master/lib/gitlab/ci/templates/Workflows/MergeRequest-Pipelines.gitlab-ci.yml) sets your pipelines to run for the default branch (usually `master`), tags, and +The [`MergeRequest-Pipelines` template](https://gitlab.com/gitlab-org/gitlab/-/tree/master/lib/gitlab/ci/templates/Workflows/MergeRequest-Pipelines.gitlab-ci.yml) +makes your pipelines run for the default branch (usually `master`), tags, and +all types of merge request pipelines. Use this template if you use any of the +the [Pipelines for Merge Requests features](../merge_request_pipelines/), as mentioned +above. + +It is [included](#include) as follows: + +```yaml +include: + - template: 'Workflows/MergeRequest-Pipelines.gitlab-ci.yml' +``` + +If you prefer to define your own rules, the configuration options currently available are:​ - [`if`](#rulesif): Define a rule. - [`when`](#when): May be set to `always` or `never` only. If not provided, the default value is `always`​. @@ -858,10 +896,26 @@ the `.template` job, and uses the `alpine` Docker image as defined in the local `rules` allows for a list of individual rule objects to be evaluated *in order*, until one matches and dynamically provides attributes to the job. -Note that `rules` can't be used in combination with `only/except` since it's intended -to replace that functionality. If you attempt to do this the linter will return a + +CAUTION: **Caution:** +`rules` can't be used in combination with `only/except` as it is a replacement for that functionality. If you attempt to do this, the linter will return a `key may not be used with rules` error. +#### Key details when using `rules` + +A very important difference between `rules` and `only/except`, is that jobs defined +with `rules` trigger merge request pipelines by default, but `only/except` jobs do not. +This may be surprising if migrating from `only` and `except`, so new users of `rules` +can use one of the [`workflow: rules` templates](#workflowrules-templates) to get started. +This will ensure that the behavior is more stable as you start adding additional `rules` +blocks, and will avoid issues like creating a duplicate, merge request (detached) pipeline. + +We don't recomment mixing `only/except` jobs with `rules` jobs in the same pipeline. +It may not cause YAML errors, but debugging the exact execution behavior can be complex +due to the different default behaviors of `only/except` and `rules`. + +### Rules clauses + Available rule clauses include: - [`if`](#rulesif) (similar to [`only:variables`](#onlyvariablesexceptvariables)) @@ -995,47 +1049,6 @@ job: In this example, if the first rule matches, then the job will have `when: manual` and `allow_failure: true`. -#### Exclude jobs with `rules:` from certain pipelines - -Jobs with `rules:` can cause two pipelines to be created unexpectedly: - -- One pipeline from pushing a commit to a branch. -- A second ["detached" pipeline for a merge request](../merge_request_pipelines/index.md). - -`only` and `except` jobs don't trigger merge request pipelines by default, but this -is not the case for jobs with `rules:`, which may be surprising if migrating from `only` -and `except` to `rules:`. - -If you're using `rules:` and you see two pipelines for commits to branches that have -a merge request, you have two options: - -- Individually exclude each job that uses `rules:` from merge request pipelines. The - example below will cause the job to **not** run in *pipelines for merge requests*, - but it **will** run in pipelines for *new tags and pipelines running on branch refs*: - - ```yaml - job: - rules: - - if: $CI_MERGE_REQUEST_ID - when: never - - when: manual - script: - - echo hello - ``` - -- Add a global [`workflow: rules`](#workflowrules) to allow pipelines in only certain - situations. The example below will only run pipelines for merge requests, new tags and - changes to master. It will **not** run any pipelines *on any branch except master*, but - it will run **detached merge request pipelines** for any merge request, targeting any branch: - - ```yaml - workflow: - rules: - - if: $CI_MERGE_REQUEST_ID - - if: $CI_COMMIT_TAG - - if: $CI_COMMIT_BRANCH == "master" - ``` - #### Complex rule clauses To conjoin `if`, `changes`, and `exists` clauses with an AND, use them in the diff --git a/doc/integration/jenkins.md b/doc/integration/jenkins.md index 0afa9cbad50..485ca67c9fc 100644 --- a/doc/integration/jenkins.md +++ b/doc/integration/jenkins.md @@ -9,11 +9,9 @@ From GitLab, you can trigger a Jenkins build when you push code to a repository, request is created. In return, Jenkins shows the pipeline status on merge requests widgets and on the GitLab project's home page. -To better understand GitLab's Jenkins integration, watch the following videos: +To better understand GitLab's Jenkins integration, watch the following video: - [GitLab workflow with Jira issues and Jenkins pipelines](https://youtu.be/Jn-_fyra7xQ) -- [Migrating from Jenkins to GitLab](https://www.youtube.com/watch?v=RlEVGOpYF5Y) - Use the Jenkins integration with GitLab when: - You plan to migrate your CI from Jenkins to [GitLab CI/CD](../ci/README.md) in the future, but diff --git a/doc/user/application_security/sast/index.md b/doc/user/application_security/sast/index.md index 69db0fe9d03..c932f434684 100644 --- a/doc/user/application_security/sast/index.md +++ b/doc/user/application_security/sast/index.md @@ -67,7 +67,8 @@ The following table shows which languages, package managers and frameworks are s | Language (package managers) / framework | Scan tool | Introduced in GitLab Version | |-----------------------------------------------------------------------------|----------------------------------------------------------------------------------------|------------------------------| -| .NET | [Security Code Scan](https://security-code-scan.github.io) | 11.0 | +| .NET Core | [Security Code Scan](https://security-code-scan.github.io) | 11.0 | +| .NET Framework | [Security Code Scan](https://security-code-scan.github.io) | 13.0 | | Any | [Gitleaks](https://github.com/zricethezav/gitleaks) and [TruffleHog](https://github.com/dxa4481/truffleHog) | 11.9 | | Apex (Salesforce) | [PMD](https://pmd.github.io/pmd/index.html) | 12.1 | | C/C++ | [Flawfinder](https://dwheeler.com/flawfinder/) | 10.7 | -- cgit v1.2.3