Add latest changes from gitlab-org/gitlab@13-8-stable-eev13.8.0-rc42

4 files changed, 165 insertions, 5 deletions

diff --git a/doc/development/testing_guide/best_practices.md b/doc/development/testing_guide/best_practices.md
index d1b7883451f..ac5f1a47f9b 100644
--- a/doc/development/testing_guide/best_practices.md
+++ b/doc/development/testing_guide/best_practices.md
@@ -842,6 +842,41 @@ Example:
 expect(response).to have_gitlab_http_status(:ok)
 ```
 
+#### `match_schema` and `match_response_schema`
+
+The `match_schema` matcher allows validating that the subject matches a
+[JSON schema](https://json-schema.org/). The item inside `expect` can be
+a JSON string or a JSON-compatible data structure.
+
+`match_response_schema` is a convenience matcher for using with a
+response object. from a [request
+spec](testing_levels.md#integration-tests).
+
+Examples:
+
+```ruby
+# Matches against spec/fixtures/api/schemas/prometheus/additional_metrics_query_result.json
+expect(data).to match_schema('prometheus/additional_metrics_query_result')
+
+# Matches against ee/spec/fixtures/api/schemas/board.json
+expect(data).to match_schema('board', dir: 'ee')
+
+# Matches against a schema made up of Ruby data structures
+expect(data).to match_schema(Atlassian::Schemata.build_info)
+```
+
+#### `be_valid_json`
+
+`be_valid_json` allows validating that a string parses as JSON and gives
+a non-empty result. To combine it with the schema matching above, use
+`and`:
+
+```ruby
+expect(json_string).to be_valid_json
+
+expect(json_string).to be_valid_json.and match_schema(schema)
+```
+
 ### Testing query performance
 
 Testing query performance allows us to:
diff --git a/doc/development/testing_guide/end_to_end/running_tests_that_require_special_setup.md b/doc/development/testing_guide/end_to_end/running_tests_that_require_special_setup.md
index 8a49c333f9f..cd429a74a2a 100644
--- a/doc/development/testing_guide/end_to_end/running_tests_that_require_special_setup.md
+++ b/doc/development/testing_guide/end_to_end/running_tests_that_require_special_setup.md
@@ -403,3 +403,85 @@ Geo requires an EE license. To visit the Geo sites in your browser, you need a r
 - You can find the full image address from a pipeline by [following these instructions](https://about.gitlab.com/handbook/engineering/quality/guidelines/tips-and-tricks/#running-gitlab-qa-pipeline-against-a-specific-gitlab-release). You might be prompted to set the `GITLAB_QA_ACCESS_TOKEN` variable if you specify the full image address.
 - You can increase the wait time for replication by setting `GEO_MAX_FILE_REPLICATION_TIME` and `GEO_MAX_DB_REPLICATION_TIME`. The default is 120 seconds.
 - To save time during tests, create a Personal Access Token with API access on the Geo primary node, and pass that value in as `GITLAB_QA_ACCESS_TOKEN` and `GITLAB_QA_ADMIN_ACCESS_TOKEN`.
+
+## LDAP Tests
+
+Tests that are tagged with `:ldap_tls` and `:ldap_no_tls` meta are orchestrated tests where the sign-in happens via LDAP.
+
+These tests spin up a Docker container [(osixia/openldap)](https://hub.docker.com/r/osixia/openldap) running an instance of [OpenLDAP](https://www.openldap.org/).
+The container uses fixtures [checked into the GitLab-QA repo](https://gitlab.com/gitlab-org/gitlab-qa/-/tree/9ffb9ad3be847a9054967d792d6772a74220fb42/fixtures/ldap) to create
+base data such as users and groups including the admin group. The password for [all users](https://gitlab.com/gitlab-org/gitlab-qa/-/blob/9ffb9ad3be847a9054967d792d6772a74220fb42/fixtures/ldap/2_add_users.ldif) including [the `tanuki` user](https://gitlab.com/gitlab-org/gitlab-qa/-/blob/9ffb9ad3be847a9054967d792d6772a74220fb42/fixtures/ldap/tanuki.ldif) is `password`.
+
+A GitLab instance is also created in a Docker container based on our [General LDAP setup](../../../administration/auth/ldap/index.md#general-ldap-setup) documentation.
+
+Tests that are tagged `:ldap_tls` enable TLS on GitLab using the certificate [checked into the GitLab-QA repo](https://gitlab.com/gitlab-org/gitlab-qa/-/tree/9ffb9ad3be847a9054967d792d6772a74220fb42/tls_certificates/gitlab).
+
+The certificate was generated with openssl using this command:
+
+```shell
+openssl req -x509 -newkey rsa:4096 -keyout gitlab.test.key -out gitlab.test.crt -days 3650 -nodes -subj "/C=US/ST=CA/L=San Francisco/O=GitLab/OU=Org/CN=gitlab.test"
+```
+
+The OpenLDAP container also uses its [auto-generated TLS certificates](https://github.com/osixia/docker-openldap#use-auto-generated-certificate).
+
+### Running LDAP tests with TLS enabled
+
+To run the LDAP tests on your local with TLS enabled, follow these steps:
+
+1. Include the following entry in your `/etc/hosts` file:
+
+   `127.0.0.1    gitlab.test`
+
+   You can then run tests against GitLab in a Docker container on `https://gitlab.test`. Please note that the TLS certificate [checked into the GitLab-QA repo](https://gitlab.com/gitlab-org/gitlab-qa/-/tree/9ffb9ad3be847a9054967d792d6772a74220fb42/tls_certificates/gitlab) is configured for this domain.
+1. Run the OpenLDAP container with TLS enabled. Change the path to [`gitlab-qa/fixtures/ldap`](https://gitlab.com/gitlab-org/gitlab-qa/-/tree/9ffb9ad3be847a9054967d792d6772a74220fb42/fixtures/ldap) directory to your local checkout path:
+
+   ```shell
+   docker network create test && docker run --name ldap-server --net test --hostname ldap-server.test --volume /path/to/gitlab-qa/fixtures/ldap:/container/service/slapd/assets/config/bootstrap/ldif/custom:Z --env LDAP_TLS_CRT_FILENAME="ldap-server.test.crt" --env LDAP_TLS_KEY_FILENAME="ldap-server.test.key" --env LDAP_TLS_ENFORCE="true" --env LDAP_TLS_VERIFY_CLIENT="never" osixia/openldap:latest --copy-service
+   ```
+
+1. Run the GitLab container with TLS enabled. Change the path to [`gitlab-qa/tls_certificates/gitlab`](https://gitlab.com/gitlab-org/gitlab-qa/-/tree/9ffb9ad3be847a9054967d792d6772a74220fb42/tls_certificates/gitlab) directory to your local checkout path:
+
+   ```shell
+   sudo docker run \
+      --hostname gitlab.test \
+      --net test \
+      --publish 443:443 --publish 80:80 --publish 22:22 \
+      --name gitlab \
+      --volume /path/to/gitlab-qa/tls_certificates/gitlab:/etc/gitlab/ssl \
+      --env GITLAB_OMNIBUS_CONFIG="gitlab_rails['ldap_enabled'] = true; gitlab_rails['ldap_servers'] = {\"main\"=>{\"label\"=>\"LDAP\", \"host\"=>\"ldap-server.test\", \"port\"=>636, \"uid\"=>\"uid\", \"bind_dn\"=>\"cn=admin,dc=example,dc=org\", \"password\"=>\"admin\", \"encryption\"=>\"simple_tls\", \"verify_certificates\"=>false, \"base\"=>\"dc=example,dc=org\", \"user_filter\"=>\"\", \"group_base\"=>\"ou=Global Groups,dc=example,dc=org\", \"admin_group\"=>\"AdminGroup\", \"external_groups\"=>\"\", \"sync_ssh_keys\"=>false}}; letsencrypt['enable'] = false; external_url 'https://gitlab.test'; gitlab_rails['ldap_sync_worker_cron'] = '* * * * *'; gitlab_rails['ldap_group_sync_worker_cron'] = '* * * * *'; " \
+      gitlab/gitlab-ee:latest
+   ```
+
+1. Run an LDAP test from [`gitlab/qa`](https://gitlab.com/gitlab-org/gitlab/-/tree/d5447ebb5f99d4c72780681ddf4dc25b0738acba/qa) directory:
+
+   ```shell
+   GITLAB_LDAP_USERNAME="tanuki" GITLAB_LDAP_PASSWORD="password" QA_DEBUG=true CHROME_HEADLESS=false bin/qa Test::Instance::All https://gitlab.test qa/specs/features/browser_ui/1_manage/login/log_into_gitlab_via_ldap_spec.rb
+   ```
+
+### Running LDAP tests with TLS disabled
+
+To run the LDAP tests on your local with TLS disabled, follow these steps:
+
+1. Run OpenLDAP container with TLS disabled. Change the path to [`gitlab-qa/fixtures/ldap`](https://gitlab.com/gitlab-org/gitlab-qa/-/tree/9ffb9ad3be847a9054967d792d6772a74220fb42/fixtures/ldap) directory to your local checkout path:
+
+   ```shell
+   docker network create test && docker run --net test --publish 389:389 --publish 636:636 --name ldap-server --hostname ldap-server.test --volume /path/to/gitlab-qa/fixtures/ldap:/container/service/slapd/assets/config/bootstrap/ldif/custom:Z --env LDAP_TLS="false" osixia/openldap:latest --copy-service
+   ```
+
+1. Run the GitLab container:
+
+  ```shell
+  sudo docker run \
+    --hostname localhost \
+    --net test \
+    --publish 443:443 --publish 80:80 --publish 22:22 \
+    --name gitlab \
+    --env GITLAB_OMNIBUS_CONFIG="gitlab_rails['ldap_enabled'] = true; gitlab_rails['ldap_servers'] = {\"main\"=>{\"label\"=>\"LDAP\", \"host\"=>\"ldap-server.test\", \"port\"=>389, \"uid\"=>\"uid\", \"bind_dn\"=>\"cn=admin,dc=example,dc=org\", \"password\"=>\"admin\", \"encryption\"=>\"plain\", \"verify_certificates\"=>false, \"base\"=>\"dc=example,dc=org\", \"user_filter\"=>\"\", \"group_base\"=>\"ou=Global Groups,dc=example,dc=org\", \"admin_group\"=>\"AdminGroup\", \"external_groups\"=>\"\", \"sync_ssh_keys\"=>false}}; gitlab_rails['ldap_sync_worker_cron'] = '* * * * *'; gitlab_rails['ldap_group_sync_worker_cron'] = '* * * * *'; " \
+  gitlab/gitlab-ee:latest
+  ```
+
+1. Run an LDAP test from [`gitlab/qa`](https://gitlab.com/gitlab-org/gitlab/-/tree/d5447ebb5f99d4c72780681ddf4dc25b0738acba/qa) directory:
+
+   ```shell
+   GITLAB_LDAP_USERNAME="tanuki" GITLAB_LDAP_PASSWORD="password" QA_DEBUG=true CHROME_HEADLESS=false bin/qa Test::Instance::All http://localhost qa/specs/features/browser_ui/1_manage/login/log_into_gitlab_via_ldap_spec.rb
+   ```
diff --git a/doc/development/testing_guide/frontend_testing.md b/doc/development/testing_guide/frontend_testing.md
index d83d58d14dd..94bc80abcdb 100644
--- a/doc/development/testing_guide/frontend_testing.md
+++ b/doc/development/testing_guide/frontend_testing.md
@@ -89,7 +89,7 @@ If your test exceeds that time, it fails.
 
 If you cannot improve the performance of the tests, you can increase the timeout
 for a specific test using
-[`setTestTimeout`](https://gitlab.com/gitlab-org/gitlab/blob/master/spec/frontend/helpers/timeout.js).
+[`setTestTimeout`](https://gitlab.com/gitlab-org/gitlab/blob/master/spec/frontend/__helpers__/timeout.js).
 
 ```javascript
 import { setTestTimeout } from 'helpers/timeout';
@@ -834,12 +834,50 @@ The `response` variable gets automatically set if the test is marked as `type: :
 When creating a new fixture, it often makes sense to take a look at the corresponding tests for the
 endpoint in `(ee/)spec/controllers/` or `(ee/)spec/requests/`.
 
+##### GraphQL query fixtures
+
+You can create a fixture that represents the result of a GraphQL query using the `get_graphql_query_as_string`
+helper method. For example:
+
+```ruby
+# spec/frontend/fixtures/releases.rb
+
+describe GraphQL::Query, type: :request do
+  include GraphqlHelpers
+
+  all_releases_query_path = 'releases/queries/all_releases.query.graphql'
+  fragment_paths = ['releases/queries/release.fragment.graphql']
+
+  before(:all) do
+    clean_frontend_fixtures('graphql/releases/')
+  end
+
+  it "graphql/#{all_releases_query_path}.json" do
+    query = get_graphql_query_as_string(all_releases_query_path, fragment_paths)
+
+    post_graphql(query, current_user: admin, variables: { fullPath: project.full_path })
+
+    expect_graphql_errors_to_be_empty
+  end
+end
+```
+
+This will create a new fixture located at
+`tmp/tests/frontend/fixtures-ee/graphql/releases/queries/all_releases.query.graphql.json`.
+
+Note that you will need to provide the paths to all fragments used by the query.
+`get_graphql_query_as_string` reads all of the provided file paths and returns
+the result as a single, concatenated string.
+
+You can import the JSON fixture in a Jest test using the `getJSONFixture` method
+[as described below](#use-fixtures).
+
 ### Use fixtures
 
 Jest and Karma test suites import fixtures in different ways:
 
 - The Karma test suite are served by [jasmine-jquery](https://github.com/velesin/jasmine-jquery).
-- Jest use `spec/frontend/helpers/fixtures.js`.
+- Jest use `spec/frontend/__helpers__/fixtures.js`.
 
 The following are examples of tests that work for both Karma and Jest:
 
@@ -1024,6 +1062,9 @@ See also [Notes on testing Vue components](../fe_guide/vue.md#testing-vue-compon
 
 ## Test helpers
 
+Test helpers can be found in [`spec/frontend/__helpers__`](https://gitlab.com/gitlab-org/gitlab/blob/master/spec/frontend/__helpers__).
+If you introduce new helpers, please place them in that directory.  
+
 ### Vuex Helper: `testAction`
 
 We have a helper available to make testing actions easier, as per [official documentation](https://vuex.vuejs.org/guide/testing.html):
@@ -1065,7 +1106,7 @@ By doing so, the `wrapper` provides you with the ability to perform a `findByTes
 which is a shortcut to the more verbose `wrapper.find('[data-testid="my-test-id"]');`
 
 ```javascript
-import { extendedWrapper } from 'jest/helpers/vue_test_utils_helper';
+import { extendedWrapper } from 'helpers/vue_test_utils_helper';
 
 describe('FooComponent', () => {
   const wrapper = extendedWrapper(shallowMount({
diff --git a/doc/development/testing_guide/testing_levels.md b/doc/development/testing_guide/testing_levels.md
index 14d4ee82f75..abacb9a0c87 100644
--- a/doc/development/testing_guide/testing_levels.md
+++ b/doc/development/testing_guide/testing_levels.md
@@ -230,7 +230,7 @@ They're useful to test permissions, redirections, what view is rendered etc.
 
 | Code path | Tests path | Testing engine | Notes |
 | --------- | ---------- | -------------- | ----- |
-| `app/controllers/` | `spec/controllers/` | RSpec | For N+1 tests, use [request specs](../query_recorder.md#use-request-specs-instead-of-controller-specs) |
+| `app/controllers/` | `spec/requests/`, `spec/controllers` | RSpec | Request specs are preferred over legacy controller specs. |
 | `app/mailers/` | `spec/mailers/` | RSpec | |
 | `lib/api/` | `spec/requests/api/` | RSpec | |
 | `app/assets/javascripts/` | `spec/javascripts/`, `spec/frontend/` | Karma & Jest | [More details below](#frontend-integration-tests) |
@@ -310,6 +310,8 @@ graph RL
 
 ### About controller tests
 
+GitLab is [transitioning from controller specs to request specs](https://gitlab.com/groups/gitlab-org/-/epics/5076).
+
 In an ideal world, controllers should be thin. However, when this is not the
 case, it's acceptable to write a system or feature test without JavaScript instead
 of a controller test. Testing a fat controller usually involves a lot of stubbing, such as:
@@ -318,7 +320,7 @@ of a controller test. Testing a fat controller usually involves a lot of stubbin
 controller.instance_variable_set(:@user, user)
 ```
 
-and use methods which are deprecated in Rails 5 ([#23768](https://gitlab.com/gitlab-org/gitlab/-/issues/16260)).
+and use methods [deprecated in Rails 5](https://gitlab.com/gitlab-org/gitlab/-/issues/16260).
 
 ### About Karma