diff options
Diffstat (limited to 'doc/development/testing_guide')
4 files changed, 165 insertions, 5 deletions
diff --git a/doc/development/testing_guide/best_practices.md b/doc/development/testing_guide/best_practices.md index d1b7883451f..ac5f1a47f9b 100644 --- a/doc/development/testing_guide/best_practices.md +++ b/doc/development/testing_guide/best_practices.md @@ -842,6 +842,41 @@ Example: expect(response).to have_gitlab_http_status(:ok) ``` +#### `match_schema` and `match_response_schema` + +The `match_schema` matcher allows validating that the subject matches a +[JSON schema](https://json-schema.org/). The item inside `expect` can be +a JSON string or a JSON-compatible data structure. + +`match_response_schema` is a convenience matcher for using with a +response object. from a [request +spec](testing_levels.md#integration-tests). + +Examples: + +```ruby +# Matches against spec/fixtures/api/schemas/prometheus/additional_metrics_query_result.json +expect(data).to match_schema('prometheus/additional_metrics_query_result') + +# Matches against ee/spec/fixtures/api/schemas/board.json +expect(data).to match_schema('board', dir: 'ee') + +# Matches against a schema made up of Ruby data structures +expect(data).to match_schema(Atlassian::Schemata.build_info) +``` + +#### `be_valid_json` + +`be_valid_json` allows validating that a string parses as JSON and gives +a non-empty result. To combine it with the schema matching above, use +`and`: + +```ruby +expect(json_string).to be_valid_json + +expect(json_string).to be_valid_json.and match_schema(schema) +``` + ### Testing query performance Testing query performance allows us to: diff --git a/doc/development/testing_guide/end_to_end/running_tests_that_require_special_setup.md b/doc/development/testing_guide/end_to_end/running_tests_that_require_special_setup.md index 8a49c333f9f..cd429a74a2a 100644 --- a/doc/development/testing_guide/end_to_end/running_tests_that_require_special_setup.md +++ b/doc/development/testing_guide/end_to_end/running_tests_that_require_special_setup.md @@ -403,3 +403,85 @@ Geo requires an EE license. To visit the Geo sites in your browser, you need a r - You can find the full image address from a pipeline by [following these instructions](https://about.gitlab.com/handbook/engineering/quality/guidelines/tips-and-tricks/#running-gitlab-qa-pipeline-against-a-specific-gitlab-release). You might be prompted to set the `GITLAB_QA_ACCESS_TOKEN` variable if you specify the full image address. - You can increase the wait time for replication by setting `GEO_MAX_FILE_REPLICATION_TIME` and `GEO_MAX_DB_REPLICATION_TIME`. The default is 120 seconds. - To save time during tests, create a Personal Access Token with API access on the Geo primary node, and pass that value in as `GITLAB_QA_ACCESS_TOKEN` and `GITLAB_QA_ADMIN_ACCESS_TOKEN`. + +## LDAP Tests + +Tests that are tagged with `:ldap_tls` and `:ldap_no_tls` meta are orchestrated tests where the sign-in happens via LDAP. + +These tests spin up a Docker container [(osixia/openldap)](https://hub.docker.com/r/osixia/openldap) running an instance of [OpenLDAP](https://www.openldap.org/). +The container uses fixtures [checked into the GitLab-QA repo](https://gitlab.com/gitlab-org/gitlab-qa/-/tree/9ffb9ad3be847a9054967d792d6772a74220fb42/fixtures/ldap) to create +base data such as users and groups including the admin group. The password for [all users](https://gitlab.com/gitlab-org/gitlab-qa/-/blob/9ffb9ad3be847a9054967d792d6772a74220fb42/fixtures/ldap/2_add_users.ldif) including [the `tanuki` user](https://gitlab.com/gitlab-org/gitlab-qa/-/blob/9ffb9ad3be847a9054967d792d6772a74220fb42/fixtures/ldap/tanuki.ldif) is `password`. + +A GitLab instance is also created in a Docker container based on our [General LDAP setup](../../../administration/auth/ldap/index.md#general-ldap-setup) documentation. + +Tests that are tagged `:ldap_tls` enable TLS on GitLab using the certificate [checked into the GitLab-QA repo](https://gitlab.com/gitlab-org/gitlab-qa/-/tree/9ffb9ad3be847a9054967d792d6772a74220fb42/tls_certificates/gitlab). + +The certificate was generated with openssl using this command: + +```shell +openssl req -x509 -newkey rsa:4096 -keyout gitlab.test.key -out gitlab.test.crt -days 3650 -nodes -subj "/C=US/ST=CA/L=San Francisco/O=GitLab/OU=Org/CN=gitlab.test" +``` + +The OpenLDAP container also uses its [auto-generated TLS certificates](https://github.com/osixia/docker-openldap#use-auto-generated-certificate). + +### Running LDAP tests with TLS enabled + +To run the LDAP tests on your local with TLS enabled, follow these steps: + +1. Include the following entry in your `/etc/hosts` file: + + `127.0.0.1 gitlab.test` + + You can then run tests against GitLab in a Docker container on `https://gitlab.test`. Please note that the TLS certificate [checked into the GitLab-QA repo](https://gitlab.com/gitlab-org/gitlab-qa/-/tree/9ffb9ad3be847a9054967d792d6772a74220fb42/tls_certificates/gitlab) is configured for this domain. +1. Run the OpenLDAP container with TLS enabled. Change the path to [`gitlab-qa/fixtures/ldap`](https://gitlab.com/gitlab-org/gitlab-qa/-/tree/9ffb9ad3be847a9054967d792d6772a74220fb42/fixtures/ldap) directory to your local checkout path: + + ```shell + docker network create test && docker run --name ldap-server --net test --hostname ldap-server.test --volume /path/to/gitlab-qa/fixtures/ldap:/container/service/slapd/assets/config/bootstrap/ldif/custom:Z --env LDAP_TLS_CRT_FILENAME="ldap-server.test.crt" --env LDAP_TLS_KEY_FILENAME="ldap-server.test.key" --env LDAP_TLS_ENFORCE="true" --env LDAP_TLS_VERIFY_CLIENT="never" osixia/openldap:latest --copy-service + ``` + +1. Run the GitLab container with TLS enabled. Change the path to [`gitlab-qa/tls_certificates/gitlab`](https://gitlab.com/gitlab-org/gitlab-qa/-/tree/9ffb9ad3be847a9054967d792d6772a74220fb42/tls_certificates/gitlab) directory to your local checkout path: + + ```shell + sudo docker run \ + --hostname gitlab.test \ + --net test \ + --publish 443:443 --publish 80:80 --publish 22:22 \ + --name gitlab \ + --volume /path/to/gitlab-qa/tls_certificates/gitlab:/etc/gitlab/ssl \ + --env GITLAB_OMNIBUS_CONFIG="gitlab_rails['ldap_enabled'] = true; gitlab_rails['ldap_servers'] = {\"main\"=>{\"label\"=>\"LDAP\", \"host\"=>\"ldap-server.test\", \"port\"=>636, \"uid\"=>\"uid\", \"bind_dn\"=>\"cn=admin,dc=example,dc=org\", \"password\"=>\"admin\", \"encryption\"=>\"simple_tls\", \"verify_certificates\"=>false, \"base\"=>\"dc=example,dc=org\", \"user_filter\"=>\"\", \"group_base\"=>\"ou=Global Groups,dc=example,dc=org\", \"admin_group\"=>\"AdminGroup\", \"external_groups\"=>\"\", \"sync_ssh_keys\"=>false}}; letsencrypt['enable'] = false; external_url 'https://gitlab.test'; gitlab_rails['ldap_sync_worker_cron'] = '* * * * *'; gitlab_rails['ldap_group_sync_worker_cron'] = '* * * * *'; " \ + gitlab/gitlab-ee:latest + ``` + +1. Run an LDAP test from [`gitlab/qa`](https://gitlab.com/gitlab-org/gitlab/-/tree/d5447ebb5f99d4c72780681ddf4dc25b0738acba/qa) directory: + + ```shell + GITLAB_LDAP_USERNAME="tanuki" GITLAB_LDAP_PASSWORD="password" QA_DEBUG=true CHROME_HEADLESS=false bin/qa Test::Instance::All https://gitlab.test qa/specs/features/browser_ui/1_manage/login/log_into_gitlab_via_ldap_spec.rb + ``` + +### Running LDAP tests with TLS disabled + +To run the LDAP tests on your local with TLS disabled, follow these steps: + +1. Run OpenLDAP container with TLS disabled. Change the path to [`gitlab-qa/fixtures/ldap`](https://gitlab.com/gitlab-org/gitlab-qa/-/tree/9ffb9ad3be847a9054967d792d6772a74220fb42/fixtures/ldap) directory to your local checkout path: + + ```shell + docker network create test && docker run --net test --publish 389:389 --publish 636:636 --name ldap-server --hostname ldap-server.test --volume /path/to/gitlab-qa/fixtures/ldap:/container/service/slapd/assets/config/bootstrap/ldif/custom:Z --env LDAP_TLS="false" osixia/openldap:latest --copy-service + ``` + +1. Run the GitLab container: + + ```shell + sudo docker run \ + --hostname localhost \ + --net test \ + --publish 443:443 --publish 80:80 --publish 22:22 \ + --name gitlab \ + --env GITLAB_OMNIBUS_CONFIG="gitlab_rails['ldap_enabled'] = true; gitlab_rails['ldap_servers'] = {\"main\"=>{\"label\"=>\"LDAP\", \"host\"=>\"ldap-server.test\", \"port\"=>389, \"uid\"=>\"uid\", \"bind_dn\"=>\"cn=admin,dc=example,dc=org\", \"password\"=>\"admin\", \"encryption\"=>\"plain\", \"verify_certificates\"=>false, \"base\"=>\"dc=example,dc=org\", \"user_filter\"=>\"\", \"group_base\"=>\"ou=Global Groups,dc=example,dc=org\", \"admin_group\"=>\"AdminGroup\", \"external_groups\"=>\"\", \"sync_ssh_keys\"=>false}}; gitlab_rails['ldap_sync_worker_cron'] = '* * * * *'; gitlab_rails['ldap_group_sync_worker_cron'] = '* * * * *'; " \ + gitlab/gitlab-ee:latest + ``` + +1. Run an LDAP test from [`gitlab/qa`](https://gitlab.com/gitlab-org/gitlab/-/tree/d5447ebb5f99d4c72780681ddf4dc25b0738acba/qa) directory: + + ```shell + GITLAB_LDAP_USERNAME="tanuki" GITLAB_LDAP_PASSWORD="password" QA_DEBUG=true CHROME_HEADLESS=false bin/qa Test::Instance::All http://localhost qa/specs/features/browser_ui/1_manage/login/log_into_gitlab_via_ldap_spec.rb + ``` diff --git a/doc/development/testing_guide/frontend_testing.md b/doc/development/testing_guide/frontend_testing.md index d83d58d14dd..94bc80abcdb 100644 --- a/doc/development/testing_guide/frontend_testing.md +++ b/doc/development/testing_guide/frontend_testing.md @@ -89,7 +89,7 @@ If your test exceeds that time, it fails. If you cannot improve the performance of the tests, you can increase the timeout for a specific test using -[`setTestTimeout`](https://gitlab.com/gitlab-org/gitlab/blob/master/spec/frontend/helpers/timeout.js). +[`setTestTimeout`](https://gitlab.com/gitlab-org/gitlab/blob/master/spec/frontend/__helpers__/timeout.js). ```javascript import { setTestTimeout } from 'helpers/timeout'; @@ -834,12 +834,50 @@ The `response` variable gets automatically set if the test is marked as `type: : When creating a new fixture, it often makes sense to take a look at the corresponding tests for the endpoint in `(ee/)spec/controllers/` or `(ee/)spec/requests/`. +##### GraphQL query fixtures + +You can create a fixture that represents the result of a GraphQL query using the `get_graphql_query_as_string` +helper method. For example: + +```ruby +# spec/frontend/fixtures/releases.rb + +describe GraphQL::Query, type: :request do + include GraphqlHelpers + + all_releases_query_path = 'releases/queries/all_releases.query.graphql' + fragment_paths = ['releases/queries/release.fragment.graphql'] + + before(:all) do + clean_frontend_fixtures('graphql/releases/') + end + + it "graphql/#{all_releases_query_path}.json" do + query = get_graphql_query_as_string(all_releases_query_path, fragment_paths) + + post_graphql(query, current_user: admin, variables: { fullPath: project.full_path }) + + expect_graphql_errors_to_be_empty + end +end +``` + +This will create a new fixture located at +`tmp/tests/frontend/fixtures-ee/graphql/releases/queries/all_releases.query.graphql.json`. + +Note that you will need to provide the paths to all fragments used by the query. +`get_graphql_query_as_string` reads all of the provided file paths and returns +the result as a single, concatenated string. + +You can import the JSON fixture in a Jest test using the `getJSONFixture` method +[as described below](#use-fixtures). + ### Use fixtures Jest and Karma test suites import fixtures in different ways: - The Karma test suite are served by [jasmine-jquery](https://github.com/velesin/jasmine-jquery). -- Jest use `spec/frontend/helpers/fixtures.js`. +- Jest use `spec/frontend/__helpers__/fixtures.js`. The following are examples of tests that work for both Karma and Jest: @@ -1024,6 +1062,9 @@ See also [Notes on testing Vue components](../fe_guide/vue.md#testing-vue-compon ## Test helpers +Test helpers can be found in [`spec/frontend/__helpers__`](https://gitlab.com/gitlab-org/gitlab/blob/master/spec/frontend/__helpers__). +If you introduce new helpers, please place them in that directory. + ### Vuex Helper: `testAction` We have a helper available to make testing actions easier, as per [official documentation](https://vuex.vuejs.org/guide/testing.html): @@ -1065,7 +1106,7 @@ By doing so, the `wrapper` provides you with the ability to perform a `findByTes which is a shortcut to the more verbose `wrapper.find('[data-testid="my-test-id"]');` ```javascript -import { extendedWrapper } from 'jest/helpers/vue_test_utils_helper'; +import { extendedWrapper } from 'helpers/vue_test_utils_helper'; describe('FooComponent', () => { const wrapper = extendedWrapper(shallowMount({ diff --git a/doc/development/testing_guide/testing_levels.md b/doc/development/testing_guide/testing_levels.md index 14d4ee82f75..abacb9a0c87 100644 --- a/doc/development/testing_guide/testing_levels.md +++ b/doc/development/testing_guide/testing_levels.md @@ -230,7 +230,7 @@ They're useful to test permissions, redirections, what view is rendered etc. | Code path | Tests path | Testing engine | Notes | | --------- | ---------- | -------------- | ----- | -| `app/controllers/` | `spec/controllers/` | RSpec | For N+1 tests, use [request specs](../query_recorder.md#use-request-specs-instead-of-controller-specs) | +| `app/controllers/` | `spec/requests/`, `spec/controllers` | RSpec | Request specs are preferred over legacy controller specs. | | `app/mailers/` | `spec/mailers/` | RSpec | | | `lib/api/` | `spec/requests/api/` | RSpec | | | `app/assets/javascripts/` | `spec/javascripts/`, `spec/frontend/` | Karma & Jest | [More details below](#frontend-integration-tests) | @@ -310,6 +310,8 @@ graph RL ### About controller tests +GitLab is [transitioning from controller specs to request specs](https://gitlab.com/groups/gitlab-org/-/epics/5076). + In an ideal world, controllers should be thin. However, when this is not the case, it's acceptable to write a system or feature test without JavaScript instead of a controller test. Testing a fat controller usually involves a lot of stubbing, such as: @@ -318,7 +320,7 @@ of a controller test. Testing a fat controller usually involves a lot of stubbin controller.instance_variable_set(:@user, user) ``` -and use methods which are deprecated in Rails 5 ([#23768](https://gitlab.com/gitlab-org/gitlab/-/issues/16260)). +and use methods [deprecated in Rails 5](https://gitlab.com/gitlab-org/gitlab/-/issues/16260). ### About Karma |