From f4b8ecbf9f8b2c47a7a8057f218c82ad77e94d66 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?R=C3=A9my=20Coutable?= Date: Tue, 6 Nov 2018 23:59:25 +0100 Subject: Add a manual job to run QA against a Review App MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit 1. Renames review to review-deploy. 2. Renames stop_review to review-stop. 3. Adds a build-qa-image job to the prepare stage and save the QA Docker image as artifact (example: https://gitlab.com/gitlab-org/gitlab-ce/-/jobs/120967019). 4. Adds a manual review-qa job to the test stage to run QA tests against the Review App (example: https://gitlab.com/gitlab-org/gitlab-ce/-/jobs/120967238). 5. Make the review-deploy job plays the review-qa job as soon as the Review App is deployed (example: https://gitlab.com/gitlab-org/gitlab-ce/-/jobs/120988775). 6. Updates the Review Apps documentation accordingly. Signed-off-by: Rémy Coutable --- doc/development/testing_guide/review_apps.md | 84 ++++++++++++++++------------ 1 file changed, 49 insertions(+), 35 deletions(-) (limited to 'doc/development') diff --git a/doc/development/testing_guide/review_apps.md b/doc/development/testing_guide/review_apps.md index 1830641431e..a6ed9e85a41 100644 --- a/doc/development/testing_guide/review_apps.md +++ b/doc/development/testing_guide/review_apps.md @@ -7,31 +7,36 @@ Review Apps are automatically deployed by each pipeline, both in ## How does it work? 1. On every [pipeline][gitlab-pipeline] during the `test` stage, the - [`review` job][review-job] is automatically started. -1. The `review` job [triggers a pipeline][cng-pipeline] in the - [`CNG-mirror`][cng-mirror] project. - - We use the `CNG-mirror` project so that the `CNG`, (**C**loud **N**ative - **G**itLab), project's registry is not overloaded with a lot of transient - Docker images. -1. The `CNG-mirror` pipeline creates the Docker images of each component (e.g. - `gitlab-rails-ee`, `gitlab-shell`, `gitaly` etc.) based on the commit from the - [GitLab pipeline][gitlab-pipeline] and store them in its - [registry][cng-mirror-registry]. -1. Once all images are built, the Review App is deployed using - [the official GitLab Helm chart][helm-chart] to the - [`review-apps-ee` Kubernetes cluster on GCP][review-apps-ee] - - The actual scripts used to deploy the Review App can be found at - [`scripts/review_apps/review-apps.sh`][review-apps.sh] - - These scripts are basically - [our official Auto DevOps scripts][Auto-DevOps.gitlab-ci.yml] where the - default CNG images are overridden with the images built and stored in the - [`CNG-mirror` project's registry][cng-mirror-registry]. - - Since we're using [the official GitLab Helm chart][helm-chart], this means - you get a dedicated environment for your branch that's very close to what it - would look in production. -1. Once the `review` job succeeds, you should be able to use your Review App - thanks to the direct link to it from the MR widget. The default username is - `root` and its password can be found in the 1Password secure note named + [`review-deploy`][review-deploy-job] job is automatically started. +1. The `review-deploy` job: + 1. Waits for the `gitlab:assets:compile` job to finish since the + [`CNG-mirror`][cng-mirror] pipeline triggerred in the following step + depends on it. + 1. [Triggers a pipeline][cng-pipeline] in the [`CNG-mirror`][cng-mirror] + project. + - We use the `CNG-mirror` project so that the `CNG`, (**C**loud + **N**ative **G**itLab), project's registry is not overloaded with a + lot of transient Docker images. + - The `CNG-mirror` pipeline creates the Docker images of each component + (e.g. `gitlab-rails-ee`, `gitlab-shell`, `gitaly` etc.) based on the + commit from the [GitLab pipeline][gitlab-pipeline] and store them in + its [registry][cng-mirror-registry]. + 1. Once all images are built by [`CNG-mirror`][cng-mirror], the Review App + is deployed using [the official GitLab Helm chart][helm-chart] to the + [`review-apps-ce`][review-apps-ce] / [`review-apps-ee`][review-apps-ee] + Kubernetes cluster on GCP. + - The actual scripts used to deploy the Review App can be found at + [`scripts/review_apps/review-apps.sh`][review-apps.sh]. + - These scripts are basically + [our official Auto DevOps scripts][Auto-DevOps.gitlab-ci.yml] where the + default CNG images are overridden with the images built and stored in the + [`CNG-mirror` project's registry][cng-mirror-registry]. + - Since we're using [the official GitLab Helm chart][helm-chart], this means + you get a dedicated environment for your branch that's very close to what + it would look in production. +1. Once the `review-deploy` job succeeds, you should be able to use your Review + App thanks to the direct link to it from the MR widget. The default username + is `root` and its password can be found in the 1Password secure note named **gitlab-{ce,ee} Review App's root password** (note that there's currently [a bug where the default password seems to be overridden][password-bug]). @@ -39,16 +44,23 @@ Review Apps are automatically deployed by each pipeline, both in - The Kubernetes cluster is connected to the `gitlab-{ce,ee}` projects using [GitLab's Kubernetes integration][gitlab-k8s-integration]. This basically - allows to have a link to the Review App directly from the merge request widget. -- The manual `stop_review` in the `test` stage can be used to stop a Review App - manually, and is also started by GitLab once a branch is deleted. -- Review Apps are cleaned up regularly using a pipeline schedule that runs - the [`scripts/review_apps/automated_cleanup.rb`][automated_cleanup.rb] script. + allows to have a link to the Review App directly from the merge request + widget. - If the Review App deployment fails, you can simply retry it (there's no need - to run the `stop_review` job first). -- If you're unable to log in using the `root` username and password, you may - encounter [this bug][password-bug]. Stop the Review App via the `stop_review` - manual job and then retry the `review` job to redeploy the Review App. + to run the [`review-stop`][gitlab-ci-yml] job first). +- The manual [`review-stop`][gitlab-ci-yml] in the `test` stage can be used to + stop a Review App manually, and is also started by GitLab once a branch is + deleted. +- Review Apps are cleaned up regularly using a pipeline schedule that runs + the [`schedule:review-cleanup`][gitlab-ci-yml] job. + +## QA runs + +On every [pipeline][gitlab-pipeline] during the `test` stage, the +`review-qa-smoke` job is automatically started: it runs the smoke QA suite. +You can also manually start the `review-qa-all`: it runs the full QA suite. + +Note that both jobs first wait for the `review-deploy` job to be finished. ## Frequently Asked Questions @@ -74,15 +86,17 @@ find a way to limit it to only us.** > This isn't enabled for forks. [gitlab-pipeline]: https://gitlab.com/gitlab-org/gitlab-ce/pipelines/35850709 -[review-job]: https://gitlab.com/gitlab-org/gitlab-ce/-/jobs/118076368 +[review-deploy-job]: https://gitlab.com/gitlab-org/gitlab-ce/-/jobs/118076368 [cng-mirror]: https://gitlab.com/gitlab-org/build/CNG-mirror [cng-pipeline]: https://gitlab.com/gitlab-org/build/CNG-mirror/pipelines/35883435 [cng-mirror-registry]: https://gitlab.com/gitlab-org/build/CNG-mirror/container_registry [helm-chart]: https://gitlab.com/charts/gitlab/ +[review-apps-ce]: https://console.cloud.google.com/kubernetes/clusters/details/us-central1-a/review-apps-ce?project=gitlab-review-apps [review-apps-ee]: https://console.cloud.google.com/kubernetes/clusters/details/us-central1-b/review-apps-ee?project=gitlab-review-apps [review-apps.sh]: https://gitlab.com/gitlab-org/gitlab-ee/blob/master/scripts/review_apps/review-apps.sh [automated_cleanup.rb]: https://gitlab.com/gitlab-org/gitlab-ee/blob/master/scripts/review_apps/automated_cleanup.rb [Auto-DevOps.gitlab-ci.yml]: https://gitlab.com/gitlab-org/gitlab-ce/blob/master/lib/gitlab/ci/templates/Auto-DevOps.gitlab-ci.yml +[gitlab-ci-yml]: https://gitlab.com/gitlab-org/gitlab-ce/blob/master/.gitlab-ci.yml [gitlab-k8s-integration]: https://docs.gitlab.com/ee/user/project/clusters/index.html [password-bug]: https://gitlab.com/gitlab-org/gitlab-ce/issues/53621 -- cgit v1.2.3