Add latest changes from gitlab-org/gitlab@master

7 files changed, 112 insertions, 39 deletions

diff --git a/.projections.json.example b/.projections.json.example
index 326e9544392..19ded7eba98 100644
--- a/.projections.json.example
+++ b/.projections.json.example
@@ -1,4 +1,12 @@
 {
+  "config/initializers/*.rb": {
+    "alternate": "spec/initializers/{}_spec.rb",
+    "type": "source"
+  },
+  "spec/initializers/*_spec.rb": {
+    "alternate": "config/initializers/{}.rb",
+    "type": "test"
+  },
   "app/*.rb": {
     "alternate": "spec/{}_spec.rb",
     "type": "source"
diff --git a/doc/administration/troubleshooting/img/sidekiq_flamegraph.png b/doc/administration/troubleshooting/img/sidekiq_flamegraph.png
new file mode 100644
index 00000000000..89d6e8da3ce
--- /dev/null
+++ b/doc/administration/troubleshooting/img/sidekiq_flamegraph.png
diff --git a/doc/administration/troubleshooting/sidekiq.md b/doc/administration/troubleshooting/sidekiq.md
index 7a8ac8c3dbe..a606a3712ba 100644
--- a/doc/administration/troubleshooting/sidekiq.md
+++ b/doc/administration/troubleshooting/sidekiq.md
@@ -85,6 +85,27 @@ several `WARN` level messages. Here's an example of a single thread's backtrace:
 In some cases Sidekiq may be hung and unable to respond to the `TTIN` signal.
 Move on to other troubleshooting methods if this happens.
 
+## Ruby profiling with `rbspy`
+
+[rbspy](https://rbspy.github.io) is an easy to use and low-overhead Ruby profiler that can be used to create
+flamegraph-style diagrams of CPU usage by Ruby processes.
+
+No changes to GitLab are required to use it and it has no dependencies. To install it:
+
+1. Download the binary from the [`rbspy` releases page](https://github.com/rbspy/rbspy/releases).
+1. Make the binary executable.
+
+To profile a Sidekiq worker for one minute, run:
+
+```shell
+sudo ./rbspy record --pid <sidekiq_pid> --duration 60 --file /tmp/sidekiq_profile.svg
+```
+
+![Example rbspy flamegraph](img/sidekiq_flamegraph.png)
+
+In this example of a flamegraph generated by `rbspy`, almost all of the Sidekiq process's time is spent in `rev_parse`, a native C
+function in Rugged. In the stack, we can see `rev_parse` is being called by the `ExpirePipelineCacheWorker`.
+
 ## Process profiling with `perf`
 
 Linux has a process profiling tool called `perf` that is helpful when a certain
@@ -358,3 +379,17 @@ has number of drawbacks, as mentioned in [Why Ruby's Timeout is dangerous (and T
 > - in any of your code, regardless of whether it could have possibly raised an exception before
 >
 > Nobody writes code to defend against an exception being raised on literally any line. That's not even possible. So Thread.raise is basically like a sneak attack on your code that could result in almost anything. It would probably be okay if it were pure-functional code that did not modify any state. But this is Ruby, so that's unlikely :)
+
+## Disable Rugged
+
+Calls into Rugged, Ruby bindings for `libgit2`, [lock the Sidekiq processes's GVL](https://silverhammermba.github.io/emberb/c/#c-in-ruby-threads),
+blocking all jobs on that worker from proceeding. If Rugged calls performed by Sidekiq are slow, this can cause significant delays in
+background task processing.
+
+By default, Rugged is used when Git repository data is stored on local storage or on an NFS mount.
+[Using Rugged is recommened when using NFS](../nfs.md#improving-nfs-performance-with-gitlab), but if
+you are using local storage, disabling Rugged can improve Sidekiq performance:
+
+```shell
+sudo gitlab-rake gitlab:features:disable_rugged
+```
diff --git a/doc/development/testing_guide/flaky_tests.md b/doc/development/testing_guide/flaky_tests.md
index 9489020de5d..d2e68ea7715 100644
--- a/doc/development/testing_guide/flaky_tests.md
+++ b/doc/development/testing_guide/flaky_tests.md
@@ -82,26 +82,19 @@ These flaky tests can fail depending on the order they run with other tests. For
 
 - <https://gitlab.com/gitlab-org/gitlab/-/issues/327668>
 
-To identify the tests that lead to such failure, we can use `rspec --bisect`,
+To identify the tests that lead to such failure, we can use `scripts/rspec_bisect_flaky`,
 which would give us the minimal test combination to reproduce the failure:
 
-```shell
-rspec --bisect ee/spec/services/ee/merge_requests/update_service_spec.rb ee/spec/services/ee/notes/quick_actions_service_spec.rb ee/spec/services/epic_links/create_service_spec.rb  ee/spec/services/ee/issuable/bulk_update_service_spec.rb
-Bisect started using options: "ee/spec/services/ee/merge_requests/update_service_spec.rb ee/spec/services/ee/notes/quick_actions_service_spec.rb ee/spec/services/epic_links/create_service_spec.rb ee/spec/services/ee/issuable/bulk_update_service_spec.rb"
-Running suite to find failures... (2 minutes 18.4 seconds)
-Starting bisect with 3 failing examples and 144 non-failing examples.
-Checking that failure(s) are order-dependent... failure appears to be order-dependent
-
-Round 1: bisecting over non-failing examples 1-144 . ignoring examples 1-72 (1 minute 11.33 seconds)
-...
-Round 7: bisecting over non-failing examples 132-133 . ignoring example 132 (43.78 seconds)
-Bisect complete! Reduced necessary non-failing examples from 144 to 1 in 8 minutes 31 seconds.
-
-The minimal reproduction command is:
-  rspec ./ee/spec/services/ee/issuable/bulk_update_service_spec.rb[1:2:1:1:1:1,1:2:1:2:1:1,1:2:1:3:1] ./ee/spec/services/epic_links/create_service_spec.rb[1:1:2:2:6:4]
-```
+1. First obtain the list of specs that ran before the flaky test. You can search
+   for the list under `Knapsack node specs:` in the CI job output log.
+1. Save the list of specs as a file, and run:
+
+    ```shell
+    cat knapsack_specs.txt | xargs scripts/rspec_bisect_flaky
+    ```
 
-We can reproduce the test failure with the reproduction command above. If we change the order of the tests, the test would pass.
+If there is an order-dependency issue, the script above will print the minimal
+reproduction.
 
 ### Time-sensitive flaky tests
 
diff --git a/doc/user/markdown.md b/doc/user/markdown.md
index a0fbc566833..b6c8bc19b64 100644
--- a/doc/user/markdown.md
+++ b/doc/user/markdown.md
@@ -249,7 +249,7 @@ the content. This data can be used by static site generators like [Jekyll](https
 When you view a Markdown file rendered by GitLab, front matter is displayed as-is,
 in a box at the top of the document. The HTML content displays after the front matter. To view an example,
 you can toggle between the source and rendered version of a
-[GitLab documentation file](https://gitlab.com/gitlab-org/gitlab/-/blob/master/doc/README.md).
+[GitLab documentation file](https://gitlab.com/gitlab-org/gitlab/-/blob/master/doc/index.md).
 
 In GitLab, front matter is used only in Markdown files and wiki pages, not the other
 places where Markdown formatting is supported. It must be at the very top of the document
diff --git a/doc/user/project/integrations/github.md b/doc/user/project/integrations/github.md
index 47a81594ca9..9f1ea3796c6 100644
--- a/doc/user/project/integrations/github.md
+++ b/doc/user/project/integrations/github.md
@@ -6,17 +6,17 @@ info: To determine the technical writer assigned to the Stage/Group associated w
 
 # GitHub project integration **(PREMIUM)**
 
-> [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/3836) in GitLab Premium 10.6.
+> [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/3836) in GitLab 10.6.
 
-GitLab provides an integration for updating the pipeline statuses on GitHub.
-This is especially useful if using GitLab for CI/CD only.
-
-This project integration is separate from the [instance wide GitHub integration](../import/github.md#mirror-a-repository-and-share-pipeline-status)
-and is automatically configured on [GitHub import](../../../integration/github.md).
+You can update GitHub with pipeline status updates from GitLab.
+This integration can help you if you use GitLab for CI/CD.
 
 ![Pipeline status update on GitHub](img/github_status_check_pipeline_update.png)
 
-## Configuration
+This project integration is separate from the [instance-wide GitHub integration](../import/github.md#mirror-a-repository-and-share-pipeline-status)
+and is automatically configured when you import a [GitHub project](../../../integration/github.md).
+
+## Configure the integration
 
 This integration requires a [GitHub API token](https://docs.github.com/en/authentication/keeping-your-account-and-data-secure/creating-a-personal-access-token)
 with `repo:status` access granted.
@@ -26,31 +26,39 @@ Complete these steps on GitHub:
 1. Go to your **Personal access tokens** page at <https://github.com/settings/tokens>.
 1. Select **Generate new token**.
 1. Under **Note**, enter a name for the new token.
-1. Ensure that `repo:status` is checked and select **Generate token**.
+1. Ensure `repo:status` is selected and select **Generate token**.
 1. Copy the generated token to use in GitLab.
 
 Complete these steps in GitLab:
 
-1. Go to the project you want to configure.
-1. Go to the [Integrations page](overview.md#accessing-integrations)
+1. On the top bar, select **Menu > Projects** and find your project.
+1. On the left sidebar, select **Settings > Integrations**.
 1. Select **GitHub**.
-1. Ensure that the **Active** toggle is enabled.
-1. Paste the token you generated on GitHub.
-1. Enter the path to your project on GitHub, such as `https://github.com/username/repository`.
-1. (Optional) To disable static status check names, clear the **Enable static status check names** checkbox.
-1. Select **Save changes** or optionally select **Test settings**.
+1. Ensure the **Active** checkbox is selected.
+1. In **Token**, paste the token you generated on GitHub.
+1. In **Repository URL**, enter the path to your project on GitHub, such as `https://github.com/username/repository`.
+1. Optional. To disable [static status check names](#static-or-dynamic-status-check-names), clear the **Enable static status check names** checkbox.
+1. Optional. Select **Test settings**.
+1. Select **Save changes**.
 
 After configuring the integration, see [Pipelines for external pull requests](../../../ci/ci_cd_for_external_repos/#pipelines-for-external-pull-requests)
 to configure pipelines to run for open pull requests.
 
 ### Static or dynamic status check names
 
-> - Introduced in GitLab 11.5: using static status check names as opt-in option.
-> - [In GitLab 12.4](https://gitlab.com/gitlab-org/gitlab/-/issues/9931), static status check names is default behavior for new projects.
+> - Introduced in GitLab 11.5 with static status check names as an opt-in option.
+> - [Changed](https://gitlab.com/gitlab-org/gitlab/-/issues/9931) in GitLab 12.4 to make static status check names the default behavior for new projects.
+
+A status check name can be static or dynamic:
+
+- **Static**: The hostname of your
+  GitLab instance is appended to the status check name.
 
-This makes it possible to mark these status checks as **Required** on GitHub.
+- **Dynamic**: The branch name is appended
+  to the status check name.
 
-When **Enable static status check names** is checked on the integration page, your
-GitLab instance host name is appended to a status check name.
+The **Enable static status check names** option enables you to configure
+required status checks in GitHub, which need a consistent (static) name to work correctly.
 
-When unchecked, it uses dynamic status check names and appends the branch name.
+If you [disable this option](#configure-the-integration),
+GitLab uses dynamic status check names instead.
diff --git a/scripts/rspec_bisect_flaky b/scripts/rspec_bisect_flaky
new file mode 100755
index 00000000000..efeb9bcb5a0
--- /dev/null
+++ b/scripts/rspec_bisect_flaky
@@ -0,0 +1,29 @@
+#!/usr/bin/env bash
+
+## Usage: scripts/rspec_bisect_flaky <files...>
+#
+# The files should be listed in order, with the last file being the file where
+# the flaky spec lives.
+
+if [ $# -eq 0 ]; then
+  echo "Usage: scripts/rspec_bisect_flaky <files...>"
+  exit
+fi
+
+files=( $@ )
+len=${#files[@]}
+target=${files[$len-1]}
+
+# Trap interrupts and exit instead of continuing the loop
+trap "echo Exited!; exit 2;" SIGINT SIGTERM
+
+# Show which set of specs are running
+set -x
+
+# Do the speedy case first, run each spec with our failing spec
+for file in "${files[@]}"; do
+  bin/rspec $file $target
+done
+
+# Do a full bisect given we did not find candidates with speedy cases 
+bin/rspec --bisect=verbose $@