Welcome to mirror list, hosted at ThFree Co, Russian Federation.

gitlab.com/gitlab-org/gitlab-foss.git - Unnamed repository; edit this file 'description' to name the repository.
summaryrefslogtreecommitdiff
diff options
context:
space:
mode:
authorGitLab Bot <gitlab-bot@gitlab.com>2022-12-20 17:22:11 +0300
committerGitLab Bot <gitlab-bot@gitlab.com>2022-12-20 17:22:11 +0300
commit0c872e02b2c822e3397515ec324051ff540f0cd5 (patch)
treece2fb6ce7030e4dad0f4118d21ab6453e5938cdd /doc/development/testing_guide/flaky_tests.md
parentf7e05a6853b12f02911494c4b3fe53d9540d74fc (diff)
Add latest changes from gitlab-org/gitlab@15-7-stable-eev15.7.0-rc42
Diffstat (limited to 'doc/development/testing_guide/flaky_tests.md')
-rw-r--r--doc/development/testing_guide/flaky_tests.md10
1 files changed, 6 insertions, 4 deletions
diff --git a/doc/development/testing_guide/flaky_tests.md b/doc/development/testing_guide/flaky_tests.md
index cc62a0ebf03..ef5b75d166f 100644
--- a/doc/development/testing_guide/flaky_tests.md
+++ b/doc/development/testing_guide/flaky_tests.md
@@ -31,7 +31,7 @@ it's reset to a pristine test after each test.
inconsistent state, so that following tests might not know about certain columns.
- [Example 2](https://gitlab.com/gitlab-org/gitlab/-/issues/368500): A test modifies data that is
used by a following test.
-- [Example 3](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/103434#note_1172316521): A test for a database query passes in a fresh database, but in a
+- [Example 3](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/103434#note_1172316521): A test for a database query passes in a fresh database, but in a
CI/CD pipeline where the database is used to process previous test sequences, the test fails. This likely
means that the query itself needs to be updated to work in a non-clean database.
@@ -56,6 +56,7 @@ the problem.
- [Example 1](https://gitlab.com/gitlab-org/gitlab-foss/-/merge_requests/10148/diffs): Without
specifying `ORDER BY`, database will not give deterministic ordering, or data race happening
in the tests.
+- [Example 2](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/106936/diffs).
### Dataset-specific
@@ -75,7 +76,7 @@ difficult to achieve locally.
any table has more than 500 columns. It could pass in the merge request, but fail later in
`master` if the order of tests changes.
- [Example 2](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/91016/diffs): A test asserts
- that trying to find a record with an unexisting ID retuns an error message. The test uses an
+ that trying to find a record with an nonexistent ID returns an error message. The test uses an
hardcoded ID that's supposed to not exist (e.g. `42`). If the test is run early in the test
suite, it might pass as not enough records were created before it, but as soon as it would run
later in the suite, there could be a record that actually has the ID `42`, hence the test would
@@ -104,6 +105,7 @@ or the app.
**Description:** The DOM selector used in the test is unreliable.
**Difficulty to reproduce:** Moderate to difficult. Depending on whether the DOM selector is duplicated, or appears after a delay etc.
+Adding a delay in API or controller could help reproducing the issue.
**Resolution:** It really depends on the problem here. It could be to wait for requests to finish, to scroll down the page etc.
@@ -207,10 +209,10 @@ The `rspec/flaky/report-suite.json` report is:
- [Sporadic RSpec failures due to `PG::UniqueViolation`](https://gitlab.com/gitlab-org/gitlab-foss/-/issues/28307#note_24958837): <https://gitlab.com/gitlab-org/gitlab-foss/-/merge_requests/9846>
- Follow-up: <https://gitlab.com/gitlab-org/gitlab-foss/-/merge_requests/10688>
- [Capybara.reset_session! should be called before requests are blocked](https://gitlab.com/gitlab-org/gitlab-foss/-/issues/33779): <https://gitlab.com/gitlab-org/gitlab-foss/-/merge_requests/12224>
-- FFaker generates funky data that tests are not ready to handle (and tests should be predictable so that's bad!):
+- ffaker generates funky data that tests are not ready to handle (and tests should be predictable so that's bad!):
- [Make `spec/mailers/notify_spec.rb` more robust](https://gitlab.com/gitlab-org/gitlab-foss/-/issues/20121): <https://gitlab.com/gitlab-org/gitlab-foss/-/merge_requests/10015>
- [Transient failure in `spec/requests/api/commits_spec.rb`](https://gitlab.com/gitlab-org/gitlab-foss/-/issues/27988#note_25342521): <https://gitlab.com/gitlab-org/gitlab-foss/-/merge_requests/9944>
- - [Replace FFaker factory data with sequences](https://gitlab.com/gitlab-org/gitlab-foss/-/issues/29643): <https://gitlab.com/gitlab-org/gitlab-foss/-/merge_requests/10184>
+ - [Replace ffaker factory data with sequences](https://gitlab.com/gitlab-org/gitlab-foss/-/issues/29643): <https://gitlab.com/gitlab-org/gitlab-foss/-/merge_requests/10184>
- [Transient failure in spec/finders/issues_finder_spec.rb](https://gitlab.com/gitlab-org/gitlab-foss/-/issues/30211#note_26707685): <https://gitlab.com/gitlab-org/gitlab-foss/-/merge_requests/10404>
### Order-dependent flaky tests