diff options
author | GitLab Bot <gitlab-bot@gitlab.com> | 2021-06-17 21:10:22 +0300 |
---|---|---|
committer | GitLab Bot <gitlab-bot@gitlab.com> | 2021-06-17 21:10:22 +0300 |
commit | 6f79cf2bd654a018387313de70b9dd45a373ce72 (patch) | |
tree | 1de67c412e2a97dcf036ec1a3b46648fc341e5de | |
parent | 5c5e86aa5c6e8be8424a92846cd0dfa8e72944e2 (diff) |
Add latest changes from gitlab-org/gitlab@master
72 files changed, 987 insertions, 436 deletions
diff --git a/.gitlab/issue_templates/Geo Replicate a new Git repository type.md b/.gitlab/issue_templates/Geo Replicate a new Git repository type.md index be6fef40f3a..3b4e6231882 100644 --- a/.gitlab/issue_templates/Geo Replicate a new Git repository type.md +++ b/.gitlab/issue_templates/Geo Replicate a new Git repository type.md @@ -1,6 +1,6 @@ <!-- -This template is based on a model named `CoolWidget`. +This template is based on a model named `CoolWidget`. To adapt this template, find and replace the following tokens: @@ -342,39 +342,6 @@ That's all of the required database changes. - [ ] Implement `CoolWidget.replicables_for_current_secondary` above. - [ ] Ensure `CoolWidget.replicables_for_current_secondary` is well-tested. Search the codebase for `replicables_for_current_secondary` to find examples of parameterized table specs. You may need to add more `FactoryBot` traits. -- [ ] If you are using a separate table `cool_widget_states` to track verification state on the Geo primary site, then: - - [ ] Do not include `::Gitlab::Geo::VerificationState` on the `CoolWidget` class. - - [ ] Add the following lines to the `cool_widget_state.rb` model: - - ```ruby - class CoolWidgetState < ApplicationRecord - ... - self.primary_key = :cool_widget_id - - include ::Gitlab::Geo::VerificationState - - belongs_to :cool_widget, inverse_of: :cool_widget_state - ... - end - ``` - - - [ ] Add the following lines to the `cool_widget` model: - - ```ruby - class CoolWidget < ApplicationRecord - ... - has_one :cool_widget_state, inverse_of: :cool_widget - - delegate :verification_retry_at, :verification_retry_at=, - :verified_at, :verified_at=, - :verification_checksum, :verification_checksum=, - :verification_failure, :verification_failure=, - :verification_retry_count, :verification_retry_count=, - to: :cool_widget_state - ... - end - ``` - - [ ] Create `ee/app/replicators/geo/cool_widget_replicator.rb`. Implement the `#repository` method which should return a `<Repository>` instance, and implement the class method `.model` to return the `CoolWidget` class: ```ruby @@ -542,6 +509,73 @@ That's all of the required database changes. end ``` +##### If you added verification state fields to a separate table (option 2 above), then you need to make additional model changes + +If you did not add verification state fields to a separate table, `cool_widget_states`, then skip to [Step 2. Implement metrics gathering](#step-2-implement-metrics-gathering). + +Otherwise, you can follow [the example of Merge Request Diffs](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/63309). + +- [ ] Add the following lines to the `cool_widget_state.rb` model: + + ``` ruby + class CoolWidgetState < ApplicationRecord + self.primary_key = :cool_widget_id + + belongs_to :cool_widget, inverse_of: :cool_widget_state + end + ``` + +- [ ] Add the following lines to the `cool_widget` model to accomplish some important tasks: + - Include the `::Gitlab::Geo::VerificationState` concern. + - Delegate verification related methods to the `cool_widget_state` model. + - Override some scopes to use the `cool_widget_states` table instead of the model table, for verification. + - Override some methods to use the `cool_widget_states` table in verification related queries. + + ```ruby + class CoolWidget < ApplicationRecord + ... + include ::Gitlab::Geo::VerificationState + + has_one :cool_widget_state, autosave: true, inverse_of: :cool_widget + + delegate :verification_retry_at, :verification_retry_at=, + :verified_at, :verified_at=, + :verification_checksum, :verification_checksum=, + :verification_failure, :verification_failure=, + :verification_retry_count, :verification_retry_count=, + :verification_state=, :verification_state, + :verification_started_at=, :verification_started_at, + to: :cool_widget_state + ... + + scope :with_verification_state, ->(state) { joins(:cool_widget_state).where(cool_widget_states: { verification_state: verification_state_value(state) }) } + scope :checksummed, -> { joins(:cool_widget_state).where.not(cool_widget_states: { verification_checksum: nil } ) } + scope :not_checksummed, -> { joins(:cool_widget_state).where(cool_widget_states: { verification_checksum: nil } ) } + + ... + + class_methods do + extend ::Gitlab::Utils::Override + ... + override :verification_state_table_name + def verification_state_table_name + 'cool_widget_states' + end + + override :verification_state_model_key + def verification_state_model_key + 'cool_widget_id' + end + + override :verification_arel_table + def verification_arel_table + CoolWidgetState.arel_table + end + end + ... + end + ``` + #### Step 2. Implement metrics gathering Metrics are gathered by `Geo::MetricsUpdateWorker`, persisted in `GeoNodeStatus` for display in the UI, and sent to Prometheus: diff --git a/.gitlab/issue_templates/Geo Replicate a new blob type.md b/.gitlab/issue_templates/Geo Replicate a new blob type.md index c7bbb6b937c..2fc50a81056 100644 --- a/.gitlab/issue_templates/Geo Replicate a new blob type.md +++ b/.gitlab/issue_templates/Geo Replicate a new blob type.md @@ -1,6 +1,6 @@ <!-- -This template is based on a model named `CoolWidget`. +This template is based on a model named `CoolWidget`. To adapt this template, find and replace the following tokens: @@ -331,39 +331,6 @@ That's all of the required database changes. - [ ] Implement `CoolWidget.replicables_for_current_secondary` above. - [ ] Ensure `CoolWidget.replicables_for_current_secondary` is well-tested. Search the codebase for `replicables_for_current_secondary` to find examples of parameterized table specs. You may need to add more `FactoryBot` traits. -- [ ] If you are using a separate table `cool_widget_states` to track verification state on the Geo primary site, then: - - [ ] Do not include `::Gitlab::Geo::VerificationState` on the `CoolWidget` class. - - [ ] Add the following lines to the `cool_widget_state.rb` model: - - ```ruby - class CoolWidgetState < ApplicationRecord - ... - self.primary_key = :cool_widget_id - - include ::Gitlab::Geo::VerificationState - - belongs_to :cool_widget, inverse_of: :cool_widget_state - ... - end - ``` - - - [ ] Add the following lines to the `cool_widget` model: - - ```ruby - class CoolWidget < ApplicationRecord - ... - has_one :cool_widget_state, inverse_of: :cool_widget - - delegate :verification_retry_at, :verification_retry_at=, - :verified_at, :verified_at=, - :verification_checksum, :verification_checksum=, - :verification_failure, :verification_failure=, - :verification_retry_count, :verification_retry_count=, - to: :cool_widget_state - ... - end - ``` - - [ ] Create `ee/app/replicators/geo/cool_widget_replicator.rb`. Implement the `#carrierwave_uploader` method which should return a `CarrierWave::Uploader`, and implement the class method `.model` to return the `CoolWidget` class: ```ruby @@ -508,6 +475,73 @@ That's all of the required database changes. end ``` +##### If you added verification state fields to a separate table (option 2 above), then you need to make additional model changes + +If you did not add verification state fields to a separate table, `cool_widget_states`, then skip to [Step 2. Implement metrics gathering](#step-2-implement-metrics-gathering). + +Otherwise, you can follow [the example of Merge Request Diffs](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/63309). + +- [ ] Add the following lines to the `cool_widget_state.rb` model: + + ``` ruby + class CoolWidgetState < ApplicationRecord + self.primary_key = :cool_widget_id + + belongs_to :cool_widget, inverse_of: :cool_widget_state + end + ``` + +- [ ] Add the following lines to the `cool_widget` model to accomplish some important tasks: + - Include the `::Gitlab::Geo::VerificationState` concern. + - Delegate verification related methods to the `cool_widget_state` model. + - Override some scopes to use the `cool_widget_states` table instead of the model table, for verification. + - Override some methods to use the `cool_widget_states` table in verification related queries. + + ```ruby + class CoolWidget < ApplicationRecord + ... + include ::Gitlab::Geo::VerificationState + + has_one :cool_widget_state, autosave: true, inverse_of: :cool_widget + + delegate :verification_retry_at, :verification_retry_at=, + :verified_at, :verified_at=, + :verification_checksum, :verification_checksum=, + :verification_failure, :verification_failure=, + :verification_retry_count, :verification_retry_count=, + :verification_state=, :verification_state, + :verification_started_at=, :verification_started_at, + to: :cool_widget_state + ... + + scope :with_verification_state, ->(state) { joins(:cool_widget_state).where(cool_widget_states: { verification_state: verification_state_value(state) }) } + scope :checksummed, -> { joins(:cool_widget_state).where.not(cool_widget_states: { verification_checksum: nil } ) } + scope :not_checksummed, -> { joins(:cool_widget_state).where(cool_widget_states: { verification_checksum: nil } ) } + + ... + + class_methods do + extend ::Gitlab::Utils::Override + ... + override :verification_state_table_name + def verification_state_table_name + 'cool_widget_states' + end + + override :verification_state_model_key + def verification_state_model_key + 'cool_widget_id' + end + + override :verification_arel_table + def verification_arel_table + CoolWidgetState.arel_table + end + end + ... + end + ``` + #### Step 2. Implement metrics gathering Metrics are gathered by `Geo::MetricsUpdateWorker`, persisted in `GeoNodeStatus` for display in the UI, and sent to Prometheus: diff --git a/.rubocop_manual_todo.yml b/.rubocop_manual_todo.yml index dae8084b656..0541a788c67 100644 --- a/.rubocop_manual_todo.yml +++ b/.rubocop_manual_todo.yml @@ -13,7 +13,6 @@ # WIP See https://gitlab.com/gitlab-org/gitlab/-/issues/322903 Graphql/Descriptions: Exclude: - - 'ee/app/graphql/types/epic_state_enum.rb' - 'ee/app/graphql/types/health_status_enum.rb' - 'ee/app/graphql/types/iteration_state_enum.rb' - 'ee/app/graphql/types/requirements_management/requirement_state_enum.rb' diff --git a/app/assets/javascripts/pipelines/components/graph/graph_component.vue b/app/assets/javascripts/pipelines/components/graph/graph_component.vue index 71ec81b8969..a999d935e13 100644 --- a/app/assets/javascripts/pipelines/components/graph/graph_component.vue +++ b/app/assets/javascripts/pipelines/components/graph/graph_component.vue @@ -207,6 +207,7 @@ export default { :source-job-hovered="hoveredSourceJobName" :pipeline-expanded="pipelineExpanded" :pipeline-id="pipeline.id" + :user-permissions="pipeline.userPermissions" @refreshPipelineGraph="$emit('refreshPipelineGraph')" @jobHover="setJob" @updateMeasurements="getMeasurements" diff --git a/app/assets/javascripts/pipelines/components/graph/stage_column_component.vue b/app/assets/javascripts/pipelines/components/graph/stage_column_component.vue index 81d59f1ef65..27b4b6aebc8 100644 --- a/app/assets/javascripts/pipelines/components/graph/stage_column_component.vue +++ b/app/assets/javascripts/pipelines/components/graph/stage_column_component.vue @@ -60,6 +60,10 @@ export default { required: false, default: '', }, + userPermissions: { + type: Object, + required: true, + }, }, titleClasses: [ 'gl-font-weight-bold', @@ -90,6 +94,9 @@ export default { hasAction() { return !isEmpty(this.action); }, + canUpdatePipeline() { + return this.userPermissions.updatePipeline; + }, }, errorCaptured(err, _vm, info) { reportToSentry('stage_column_component', `error: ${err}, info: ${info}`); @@ -132,7 +139,7 @@ export default { > <div>{{ formattedTitle }}</div> <action-component - v-if="hasAction" + v-if="hasAction && canUpdatePipeline" :action-icon="action.icon" :tooltip-text="action.title" :link="action.path" diff --git a/app/assets/javascripts/pipelines/components/pipelines_list/pipeline_url.vue b/app/assets/javascripts/pipelines/components/pipelines_list/pipeline_url.vue index 52c8ef2cf26..fc8f31c5b7e 100644 --- a/app/assets/javascripts/pipelines/components/pipelines_list/pipeline_url.vue +++ b/app/assets/javascripts/pipelines/components/pipelines_list/pipeline_url.vue @@ -60,19 +60,20 @@ export default { data-testid="pipeline-url-link" data-qa-selector="pipeline_url_link" > - <span class="pipeline-id">#{{ pipeline.id }}</span> + #{{ pipeline.id }} </gl-link> <div class="label-container"> - <gl-link v-if="isScheduled" :href="pipelineScheduleUrl" target="__blank"> - <gl-badge - v-gl-tooltip - :title="__('This pipeline was triggered by a schedule.')" - variant="info" - size="sm" - data-testid="pipeline-url-scheduled" - >{{ __('Scheduled') }}</gl-badge - > - </gl-link> + <gl-badge + v-if="isScheduled" + v-gl-tooltip + :href="pipelineScheduleUrl" + target="__blank" + :title="__('This pipeline was triggered by a schedule.')" + variant="info" + size="sm" + data-testid="pipeline-url-scheduled" + >{{ __('Scheduled') }}</gl-badge + > <gl-badge v-if="pipeline.flags.latest" v-gl-tooltip diff --git a/app/assets/javascripts/repository/components/blob_content_viewer.vue b/app/assets/javascripts/repository/components/blob_content_viewer.vue index 7fbf331d585..aa94f9cefdf 100644 --- a/app/assets/javascripts/repository/components/blob_content_viewer.vue +++ b/app/assets/javascripts/repository/components/blob_content_viewer.vue @@ -7,13 +7,13 @@ import { SIMPLE_BLOB_VIEWER, RICH_BLOB_VIEWER } from '~/blob/components/constant import createFlash from '~/flash'; import { __ } from '~/locale'; import blobInfoQuery from '../queries/blob_info.query.graphql'; -import BlobHeaderEdit from './blob_header_edit.vue'; +import BlobEdit from './blob_edit.vue'; import BlobReplace from './blob_replace.vue'; export default { components: { BlobHeader, - BlobHeaderEdit, + BlobEdit, BlobReplace, BlobContent, GlLoadingIcon, @@ -131,10 +131,7 @@ export default { @viewer-changed="switchViewer" > <template #actions> - <blob-header-edit - :edit-path="blobInfo.editBlobPath" - :web-ide-path="blobInfo.ideEditPath" - /> + <blob-edit :edit-path="blobInfo.editBlobPath" :web-ide-path="blobInfo.ideEditPath" /> <blob-replace v-if="isLoggedIn" :path="path" diff --git a/app/assets/javascripts/repository/components/blob_header_edit.vue b/app/assets/javascripts/repository/components/blob_edit.vue index 3d97ebe89e4..3d97ebe89e4 100644 --- a/app/assets/javascripts/repository/components/blob_header_edit.vue +++ b/app/assets/javascripts/repository/components/blob_edit.vue diff --git a/app/assets/javascripts/vue_shared/components/paginated_table_with_search_and_tabs/paginated_table_with_search_and_tabs.vue b/app/assets/javascripts/vue_shared/components/paginated_table_with_search_and_tabs/paginated_table_with_search_and_tabs.vue index d05e45e90b3..79a9e1fca8c 100644 --- a/app/assets/javascripts/vue_shared/components/paginated_table_with_search_and_tabs/paginated_table_with_search_and_tabs.vue +++ b/app/assets/javascripts/vue_shared/components/paginated_table_with_search_and_tabs/paginated_table_with_search_and_tabs.vue @@ -169,6 +169,12 @@ export default { methods: { filterItemsByStatus(tabIndex) { this.resetPagination(); + const activeStatusTab = this.statusTabs[tabIndex]; + + if (activeStatusTab == null) { + return; + } + const { filters, status } = this.statusTabs[tabIndex]; this.statusFilter = filters; this.filteredByStatus = status; diff --git a/app/assets/stylesheets/framework/header.scss b/app/assets/stylesheets/framework/header.scss index 8639b9a7f84..65d914e47cf 100644 --- a/app/assets/stylesheets/framework/header.scss +++ b/app/assets/stylesheets/framework/header.scss @@ -555,7 +555,8 @@ $top-nav-hover-bg: var(--indigo-900-alpha-008, $indigo-900-alpha-008) !important visibility: visible; } -.with-performance-bar .navbar-gitlab { +.with-performance-bar .navbar-gitlab, +.with-performance-bar .fixed-top { top: $performance-bar-height; } @@ -563,7 +564,7 @@ $top-nav-hover-bg: var(--indigo-900-alpha-008, $indigo-900-alpha-008) !important justify-content: center; height: $header-height; background: $white; - border-bottom: 1px solid $white-normal; + border-bottom: 1px solid $gray-100; .tanuki-logo, .brand-header-logo { diff --git a/app/assets/stylesheets/framework/system_messages.scss b/app/assets/stylesheets/framework/system_messages.scss index 437915d5034..1cb34bea069 100644 --- a/app/assets/stylesheets/framework/system_messages.scss +++ b/app/assets/stylesheets/framework/system_messages.scss @@ -60,7 +60,8 @@ // System Header &.with-performance-bar { // main navigation - header.navbar-gitlab { + header.navbar-gitlab, + .fixed-top { top: $performance-bar-height + $system-header-height; } diff --git a/app/assets/stylesheets/startup/startup-signin.scss b/app/assets/stylesheets/startup/startup-signin.scss index 81a87742850..c25b5b53070 100644 --- a/app/assets/stylesheets/startup/startup-signin.scss +++ b/app/assets/stylesheets/startup/startup-signin.scss @@ -542,7 +542,7 @@ label.label-bold { justify-content: center; height: 40px; background: #fff; - border-bottom: 1px solid #f0f0f0; + border-bottom: 1px solid #dbdbdb; } .navbar-empty .tanuki-logo, .navbar-empty .brand-header-logo { diff --git a/app/assets/stylesheets/utilities.scss b/app/assets/stylesheets/utilities.scss index cabbe5834cb..965e1806416 100644 --- a/app/assets/stylesheets/utilities.scss +++ b/app/assets/stylesheets/utilities.scss @@ -85,6 +85,10 @@ padding-bottom: $gl-spacing-scale-8; } +.gl-pt-11 { + padding-top: $gl-spacing-scale-11; +} + .gl-transition-property-stroke-opacity { transition-property: stroke-opacity; } diff --git a/app/controllers/registrations/experience_levels_controller.rb b/app/controllers/registrations/experience_levels_controller.rb index d04e8d296ed..3c94bce126c 100644 --- a/app/controllers/registrations/experience_levels_controller.rb +++ b/app/controllers/registrations/experience_levels_controller.rb @@ -2,7 +2,7 @@ module Registrations class ExperienceLevelsController < ApplicationController - layout 'signup_onboarding' + layout 'minimal' before_action :ensure_namespace_path_param diff --git a/app/controllers/registrations/welcome_controller.rb b/app/controllers/registrations/welcome_controller.rb index 34283cc8db7..303ee431a4d 100644 --- a/app/controllers/registrations/welcome_controller.rb +++ b/app/controllers/registrations/welcome_controller.rb @@ -2,7 +2,7 @@ module Registrations class WelcomeController < ApplicationController - layout 'welcome' + layout 'minimal' skip_before_action :authenticate_user!, :required_signup_info, :check_two_factor_requirement, only: [:show, :update] before_action :require_current_user diff --git a/app/graphql/queries/pipelines/get_pipeline_details.query.graphql b/app/graphql/queries/pipelines/get_pipeline_details.query.graphql index 873ecc81466..4e4caa1e27c 100644 --- a/app/graphql/queries/pipelines/get_pipeline_details.query.graphql +++ b/app/graphql/queries/pipelines/get_pipeline_details.query.graphql @@ -29,6 +29,9 @@ query getPipelineDetails($projectPath: ID!, $iid: ID!) { iid complete usesNeeds + userPermissions { + updatePipeline + } downstream { __typename nodes { diff --git a/app/services/merge_requests/push_options_handler_service.rb b/app/services/merge_requests/push_options_handler_service.rb index cc1e08e1606..ad0b033f082 100644 --- a/app/services/merge_requests/push_options_handler_service.rb +++ b/app/services/merge_requests/push_options_handler_service.rb @@ -142,6 +142,8 @@ module MergeRequests params[:add_assignee_ids] = params.delete(:assign).keys if params.has_key?(:assign) params[:remove_assignee_ids] = params.delete(:unassign).keys if params.has_key?(:unassign) + params[:milestone] = project.milestones&.find_by_name(push_options[:milestone]) if push_options[:milestone] + params end diff --git a/app/views/layouts/minimal.html.haml b/app/views/layouts/minimal.html.haml new file mode 100644 index 00000000000..c9b208de477 --- /dev/null +++ b/app/views/layouts/minimal.html.haml @@ -0,0 +1,17 @@ +- page_classes = page_class.push(@html_class).flatten.compact + +!!! 5 +%html{ lang: I18n.locale, class: page_classes } + = render "layouts/head" + %body{ data: body_data } + = header_message + = render 'peek/bar' + = render "layouts/header/empty" + .layout-page + .content-wrapper.content-wrapper-margin.gl-pt-11 + .alert-wrapper.gl-force-block-formatting-context + = render "layouts/broadcast" + .limit-container-width{ class: container_class } + %main#content-body.content + = yield + = footer_message diff --git a/app/views/layouts/welcome.html.haml b/app/views/layouts/welcome.html.haml deleted file mode 100644 index 944f524d692..00000000000 --- a/app/views/layouts/welcome.html.haml +++ /dev/null @@ -1,8 +0,0 @@ -!!! 5 -%html.subscriptions-layout-html{ lang: 'en' } - = render 'layouts/head' - %body.ui-indigo.gl-display-flex.vh-100 - = render "layouts/header/logo_with_title" - = render "layouts/broadcast" - .container.gl-display-flex.gl-flex-grow-1 - = yield diff --git a/app/views/registrations/welcome/show.html.haml b/app/views/registrations/welcome/show.html.haml index e85ce1ba6ac..9356b6ad49c 100644 --- a/app/views/registrations/welcome/show.html.haml +++ b/app/views/registrations/welcome/show.html.haml @@ -1,10 +1,11 @@ +- @html_class = "subscriptions-layout-html" - page_title _('Your profile') - add_page_specific_style 'page_bundles/signup' - gitlab_experience_text = _('To personalize your GitLab experience, we\'d like to know a bit more about you') .row.gl-flex-grow-1 - .d-flex.gl-flex-direction-column.gl-align-items-center.gl-w-full.gl-p-5 - .edit-profile.login-page.d-flex.flex-column.gl-align-items-center.pt-lg-3 + .d-flex.gl-flex-direction-column.gl-align-items-center.gl-w-full.gl-px-5.gl-pb-5 + .edit-profile.login-page.d-flex.flex-column.gl-align-items-center = render_if_exists "registrations/welcome/progress_bar" %h2.gl-text-center= html_escape(_('Welcome to GitLab,%{br_tag}%{name}!')) % { name: html_escape(current_user.first_name), br_tag: '<br/>'.html_safe } - if Gitlab.com? diff --git a/app/workers/pipeline_hooks_worker.rb b/app/workers/pipeline_hooks_worker.rb index 97e6adbbf18..40d138752b4 100644 --- a/app/workers/pipeline_hooks_worker.rb +++ b/app/workers/pipeline_hooks_worker.rb @@ -8,7 +8,7 @@ class PipelineHooksWorker # rubocop:disable Scalability/IdempotentWorker queue_namespace :pipeline_hooks worker_resource_boundary :cpu - data_consistency :delayed, feature_flag: :load_balancing_for_pipeline_hooks_worker + data_consistency :delayed # rubocop: disable CodeReuse/ActiveRecord def perform(pipeline_id) diff --git a/config/feature_flags/development/load_balancing_for_pipeline_hooks_worker.yml b/config/feature_flags/development/load_balancing_for_pipeline_hooks_worker.yml deleted file mode 100644 index 8e1794631f5..00000000000 --- a/config/feature_flags/development/load_balancing_for_pipeline_hooks_worker.yml +++ /dev/null @@ -1,8 +0,0 @@ ---- -name: load_balancing_for_pipeline_hooks_worker -introduced_by_url: https://gitlab.com/gitlab-org/gitlab/-/merge_requests/62104 -rollout_issue_url: https://gitlab.com/gitlab-org/gitlab/-/issues/331424 -milestone: '14.0' -type: development -group: group::memory -default_enabled: false diff --git a/doc/administration/geo/disaster_recovery/background_verification.md b/doc/administration/geo/disaster_recovery/background_verification.md index 8d3745130bd..c09daeec824 100644 --- a/doc/administration/geo/disaster_recovery/background_verification.md +++ b/doc/administration/geo/disaster_recovery/background_verification.md @@ -59,18 +59,18 @@ Feature.enable('geo_repository_verification') ## Repository verification Go to the **Admin Area > Geo** dashboard on the **primary** node and expand -the **Verification information** tab for that node to view automatic checksumming -status for repositories and wikis. Successes are shown in green, pending work +the **Verification information** section for that node to view automatic checksumming +status for each data type. Successes are shown in green, pending work in gray, and failures in red. -![Verification status](img/verification-status-primary.png) +![Verification status](img/verification_status_primary_v14_0.png) Go to the **Admin Area > Geo** dashboard on the **secondary** node and expand -the **Verification information** tab for that node to view automatic verification -status for repositories and wikis. As with checksumming, successes are shown in +the **Verification information** section for that node to view automatic verification +status for each data type. As with checksumming, successes are shown in green, pending work in gray, and failures in red. -![Verification status](img/verification-status-secondary.png) +![Verification status](img/verification_status_secondary_v14_0.png) ## Using checksums to compare Geo nodes diff --git a/doc/administration/geo/disaster_recovery/img/replication-status.png b/doc/administration/geo/disaster_recovery/img/replication-status.png Binary files differdeleted file mode 100644 index d7085927c75..00000000000 --- a/doc/administration/geo/disaster_recovery/img/replication-status.png +++ /dev/null diff --git a/doc/administration/geo/disaster_recovery/img/verification-status-primary.png b/doc/administration/geo/disaster_recovery/img/verification-status-primary.png Binary files differdeleted file mode 100644 index 2503408ec5d..00000000000 --- a/doc/administration/geo/disaster_recovery/img/verification-status-primary.png +++ /dev/null diff --git a/doc/administration/geo/disaster_recovery/img/verification-status-secondary.png b/doc/administration/geo/disaster_recovery/img/verification-status-secondary.png Binary files differdeleted file mode 100644 index 462274d8b14..00000000000 --- a/doc/administration/geo/disaster_recovery/img/verification-status-secondary.png +++ /dev/null diff --git a/doc/administration/geo/disaster_recovery/img/verification_status_primary_v14_0.png b/doc/administration/geo/disaster_recovery/img/verification_status_primary_v14_0.png Binary files differnew file mode 100644 index 00000000000..9d2537a18bf --- /dev/null +++ b/doc/administration/geo/disaster_recovery/img/verification_status_primary_v14_0.png diff --git a/doc/administration/geo/disaster_recovery/img/verification_status_secondary_v14_0.png b/doc/administration/geo/disaster_recovery/img/verification_status_secondary_v14_0.png Binary files differnew file mode 100644 index 00000000000..3b4ff9f393b --- /dev/null +++ b/doc/administration/geo/disaster_recovery/img/verification_status_secondary_v14_0.png diff --git a/doc/administration/geo/disaster_recovery/planned_failover.md b/doc/administration/geo/disaster_recovery/planned_failover.md index d50078da172..633d787473e 100644 --- a/doc/administration/geo/disaster_recovery/planned_failover.md +++ b/doc/administration/geo/disaster_recovery/planned_failover.md @@ -115,7 +115,7 @@ and there should be no failures (shown in red). If a large proportion of objects aren't yet replicated (shown in gray), consider giving the node more time to complete -![Replication status](img/replication-status.png) +![Replication status](../replication/img/geo_node_dashboard_v14_0.png) If any objects are failing to replicate, this should be investigated before scheduling the maintenance window. Following a planned failover, anything that diff --git a/doc/administration/geo/disaster_recovery/runbooks/planned_failover_multi_node.md b/doc/administration/geo/disaster_recovery/runbooks/planned_failover_multi_node.md index 3227fafca0f..e19aa671b89 100644 --- a/doc/administration/geo/disaster_recovery/runbooks/planned_failover_multi_node.md +++ b/doc/administration/geo/disaster_recovery/runbooks/planned_failover_multi_node.md @@ -69,7 +69,7 @@ and there should be no failures (shown in red). If a large proportion of objects aren't yet replicated (shown in gray), consider giving the node more time to complete. -![Replication status](../img/replication-status.png) +![Replication status](../../replication/img/geo_node_dashboard_v14_0.png) If any objects are failing to replicate, this should be investigated before scheduling the maintenance window. After a planned failover, anything that diff --git a/doc/administration/geo/disaster_recovery/runbooks/planned_failover_single_node.md b/doc/administration/geo/disaster_recovery/runbooks/planned_failover_single_node.md index 7f311d172ef..9b5c3f00040 100644 --- a/doc/administration/geo/disaster_recovery/runbooks/planned_failover_single_node.md +++ b/doc/administration/geo/disaster_recovery/runbooks/planned_failover_single_node.md @@ -57,7 +57,7 @@ and there should be no failures (shown in red). If a large proportion of objects aren't yet replicated (shown in gray), consider giving the node more time to complete. -![Replication status](../img/replication-status.png) +![Replication status](../../replication/img/geo_node_dashboard_v14_0.png) If any objects are failing to replicate, this should be investigated before scheduling the maintenance window. After a planned failover, anything that diff --git a/doc/administration/geo/replication/configuration.md b/doc/administration/geo/replication/configuration.md index 6d5f3e61ba0..8a1ea0ad3f2 100644 --- a/doc/administration/geo/replication/configuration.md +++ b/doc/administration/geo/replication/configuration.md @@ -269,7 +269,7 @@ The initial replication, or 'backfill', is probably still in progress. You can monitor the synchronization process on each Geo node from the **primary** node's **Geo Nodes** dashboard in your browser. -![Geo dashboard](img/geo_node_dashboard.png) +![Geo dashboard](img/geo_node_dashboard_v14_0.png) If your installation isn't working properly, check the [troubleshooting document](troubleshooting.md). diff --git a/doc/administration/geo/replication/img/geo_architecture.png b/doc/administration/geo/replication/img/geo_architecture.png Binary files differindex aac63be41ff..90272537f43 100644 --- a/doc/administration/geo/replication/img/geo_architecture.png +++ b/doc/administration/geo/replication/img/geo_architecture.png diff --git a/doc/administration/geo/replication/img/geo_node_dashboard.png b/doc/administration/geo/replication/img/geo_node_dashboard.png Binary files differdeleted file mode 100644 index 8b9aceba825..00000000000 --- a/doc/administration/geo/replication/img/geo_node_dashboard.png +++ /dev/null diff --git a/doc/administration/geo/replication/img/geo_node_dashboard_v14_0.png b/doc/administration/geo/replication/img/geo_node_dashboard_v14_0.png Binary files differnew file mode 100644 index 00000000000..6d183fc6bd2 --- /dev/null +++ b/doc/administration/geo/replication/img/geo_node_dashboard_v14_0.png diff --git a/doc/administration/geo/replication/img/geo_node_health_v14_0.png b/doc/administration/geo/replication/img/geo_node_health_v14_0.png Binary files differnew file mode 100644 index 00000000000..4c640522569 --- /dev/null +++ b/doc/administration/geo/replication/img/geo_node_health_v14_0.png diff --git a/doc/administration/geo/replication/troubleshooting.md b/doc/administration/geo/replication/troubleshooting.md index 1fd923dbaf1..7c1f7cf7a8d 100644 --- a/doc/administration/geo/replication/troubleshooting.md +++ b/doc/administration/geo/replication/troubleshooting.md @@ -35,7 +35,7 @@ to help identify if something is wrong: - Is the node's secondary tracking database connected? - Is the node's secondary tracking database up-to-date? -![Geo health check](img/geo_node_dashboard.png) +![Geo health check](img/geo_node_health_v14_0.png) For information on how to resolve common errors reported from the UI, see [Fixing Common Errors](#fixing-common-errors). diff --git a/doc/api/graphql/reference/index.md b/doc/api/graphql/reference/index.md index 2d775fcf83d..b51ae3e09f8 100644 --- a/doc/api/graphql/reference/index.md +++ b/doc/api/graphql/reference/index.md @@ -14245,9 +14245,9 @@ State of an epic. | Value | Description | | ----- | ----------- | -| <a id="epicstateall"></a>`all` | | -| <a id="epicstateclosed"></a>`closed` | | -| <a id="epicstateopened"></a>`opened` | | +| <a id="epicstateall"></a>`all` | All epics. | +| <a id="epicstateclosed"></a>`closed` | Closed epics. | +| <a id="epicstateopened"></a>`opened` | Open epics. | ### `EpicStateEvent` diff --git a/doc/ci/caching/index.md b/doc/ci/caching/index.md index 3c9a796a9f2..136c6c282df 100644 --- a/doc/ci/caching/index.md +++ b/doc/ci/caching/index.md @@ -57,55 +57,69 @@ For runners to work with caches efficiently, you must do one of the following: - Use multiple runners that have [distributed caching](https://docs.gitlab.com/runner/configuration/autoscale.html#distributed-runners-caching), where the cache is stored in S3 buckets. Shared runners on GitLab.com behave this way. These runners can be in autoscale mode, - but they don't have to be. + but they don't have to be. - Use multiple runners with the same architecture and have these runners share a common network-mounted directory to store the cache. This directory should use NFS or something similar. - These runners must be in autoscale mode. + These runners must be in autoscale mode. -### Share caches between jobs in the same branch - -To have jobs for each branch use the same cache, define a cache with the `key: ${CI_COMMIT_REF_SLUG}`: - -```yaml -cache: - key: ${CI_COMMIT_REF_SLUG} -``` +## Use multiple caches -This configuration prevents you from accidentally overwriting the cache. However, the -first pipeline for a merge request is slow. The next time a commit is pushed to the branch, the -cache is re-used and jobs run faster. +> - [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/32814) in GitLab 13.10. +> - [Feature Flag removed](https://gitlab.com/gitlab-org/gitlab/-/issues/321877), in GitLab 13.12. -To enable per-job and per-branch caching: +You can have a maximum of four caches: ```yaml -cache: - key: "$CI_JOB_NAME-$CI_COMMIT_REF_SLUG" +test-job: + stage: build + cache: + - key: + files: + - Gemfile.lock + paths: + - vendor/ruby + - key: + files: + - yarn.lock + paths: + - .yarn-cache/ + script: + - bundle install --path=vendor + - yarn install --cache-folder .yarn-cache + - echo Run tests... ``` -To enable per-stage and per-branch caching: +If multiple caches are combined with a [Fallback cache key](#fallback-cache-key), +the fallback cache is fetched every time a cache is not found. -```yaml -cache: - key: "$CI_JOB_STAGE-$CI_COMMIT_REF_SLUG" -``` +## Fallback cache key -### Share caches across jobs in different branches +> [Introduced](https://gitlab.com/gitlab-org/gitlab-runner/-/merge_requests/1534) in GitLab Runner 13.4. -To share a cache across all branches and all jobs, use the same key for everything: +You can use the `$CI_COMMIT_REF_SLUG` [predefined variable](../variables/predefined_variables.md) +to specify your [`cache:key`](../yaml/README.md#cachekey). For example, if your +`$CI_COMMIT_REF_SLUG` is `test` you can set a job to download cache that's tagged with `test`. -```yaml -cache: - key: one-key-to-rule-them-all -``` +If a cache with this tag is not found, you can use `CACHE_FALLBACK_KEY` to +specify a cache to use when none exists. -To share caches between branches, but have a unique cache for each job: +In the following example, if the `$CI_COMMIT_REF_SLUG` is not found, the job uses the key defined +by the `CACHE_FALLBACK_KEY` variable: ```yaml -cache: - key: ${CI_JOB_NAME} +variables: + CACHE_FALLBACK_KEY: fallback-key + +job1: + script: + - echo + cache: + key: "$CI_COMMIT_REF_SLUG" + paths: + - binaries/ ``` -### Disable cache for specific jobs +## Disable cache for specific jobs If you have defined the cache globally, it means that each job uses the same definition. You can override this behavior per-job, and if you want to @@ -116,7 +130,7 @@ job: cache: {} ``` -### Inherit global configuration, but override specific settings per job +## Inherit global configuration, but override specific settings per job You can override cache settings without overwriting the global cache by using [anchors](../yaml/README.md#anchors). For example, if you want to override the @@ -124,7 +138,7 @@ You can override cache settings without overwriting the global cache by using ```yaml cache: &global_cache - key: ${CI_COMMIT_REF_SLUG} + key: $CI_COMMIT_REF_SLUG paths: - node_modules/ - public/ @@ -150,6 +164,49 @@ PHP packages, Ruby gems, Python libraries, and others can all be cached. For more examples, check out our [GitLab CI/CD templates](https://gitlab.com/gitlab-org/gitlab/-/tree/master/lib/gitlab/ci/templates). +### Share caches between jobs in the same branch + +To have jobs for each branch use the same cache, define a cache with the `key: $CI_COMMIT_REF_SLUG`: + +```yaml +cache: + key: $CI_COMMIT_REF_SLUG +``` + +This configuration prevents you from accidentally overwriting the cache. However, the +first pipeline for a merge request is slow. The next time a commit is pushed to the branch, the +cache is re-used and jobs run faster. + +To enable per-job and per-branch caching: + +```yaml +cache: + key: "$CI_JOB_NAME-$CI_COMMIT_REF_SLUG" +``` + +To enable per-stage and per-branch caching: + +```yaml +cache: + key: "$CI_JOB_STAGE-$CI_COMMIT_REF_SLUG" +``` + +### Share caches across jobs in different branches + +To share a cache across all branches and all jobs, use the same key for everything: + +```yaml +cache: + key: one-key-to-rule-them-all +``` + +To share caches between branches, but have a unique cache for each job: + +```yaml +cache: + key: $CI_JOB_NAME +``` + ### Cache Node.js dependencies If your project is using [npm](https://www.npmjs.com/) to install the Node.js @@ -166,7 +223,7 @@ image: node:latest # Cache modules in between jobs cache: - key: ${CI_COMMIT_REF_SLUG} + key: $CI_COMMIT_REF_SLUG paths: - .npm/ @@ -193,7 +250,7 @@ image: php:7.2 # Cache libraries in between jobs cache: - key: ${CI_COMMIT_REF_SLUG} + key: $CI_COMMIT_REF_SLUG paths: - vendor/ @@ -262,7 +319,7 @@ image: ruby:2.6 # Cache gems in between builds cache: - key: ${CI_COMMIT_REF_SLUG} + key: $CI_COMMIT_REF_SLUG paths: - vendor/ruby @@ -287,7 +344,7 @@ cache: key: files: - Gemfile.lock - prefix: ${CI_JOB_NAME} + prefix: $CI_JOB_NAME paths: - vendor/ruby diff --git a/doc/ci/yaml/README.md b/doc/ci/yaml/README.md index 8e9cf00b160..b01fcaa5bc3 100644 --- a/doc/ci/yaml/README.md +++ b/doc/ci/yaml/README.md @@ -2351,250 +2351,215 @@ as Review Apps. You can see an example that uses Review Apps at Use `cache` to specify a list of files and directories to cache between jobs. You can only use paths that are in the local working copy. -If `cache` is defined outside the scope of jobs, it's set -globally and all jobs use that configuration. - Caching is shared between pipelines and jobs. Caches are restored before [artifacts](#artifacts). -Read how caching works and find out some good practices in the -[caching dependencies documentation](../caching/index.md). +Learn more about caches in [Caching in GitLab CI/CD](../caching/index.md). #### `cache:paths` -Use the `paths` directive to choose which files or directories to cache. Paths -are relative to the project directory (`$CI_PROJECT_DIR`) and can't directly link outside it. -You can use Wildcards that use [glob](https://en.wikipedia.org/wiki/Glob_(programming)) -patterns and: +Use the `cache:paths` keyword to choose which files or directories to cache. + +**Keyword type**: Job-specific. You can use it only as part of a job. + +**Possible inputs**: An array of paths relative to the project directory (`$CI_PROJECT_DIR`). +You can use wildcards that use [glob](https://en.wikipedia.org/wiki/Glob_(programming)) +patterns: - In [GitLab Runner 13.0](https://gitlab.com/gitlab-org/gitlab-runner/-/issues/2620) and later, [`doublestar.Glob`](https://pkg.go.dev/github.com/bmatcuk/doublestar@v1.2.2?tab=doc#Match). - In GitLab Runner 12.10 and earlier, [`filepath.Match`](https://pkg.go.dev/path/filepath#Match). +**Example of `cache:paths`**: + Cache all files in `binaries` that end in `.apk` and the `.config` file: ```yaml rspec: - script: test + script: + - echo "This job uses a cache." cache: + key: binaries-cache paths: - binaries/*.apk - .config ``` -Locally defined cache overrides globally defined options. The following `rspec` -job caches only `binaries/`: - -```yaml -cache: - paths: - - my/files - -rspec: - script: test - cache: - key: rspec - paths: - - binaries/ -``` +**Related topics**: -The cache is shared between jobs, so if you're using different -paths for different jobs, you should also set a different `cache:key`. -Otherwise cache content can be overwritten. +- See the [common `cache` use cases](../caching/index.md#common-use-cases) for more + `cache:paths` examples. #### `cache:key` -The `key` keyword defines the affinity of caching between jobs. -You can have a single cache for all jobs, cache per-job, cache per-branch, -or any other way that fits your workflow. You can fine tune caching, -including caching data between different jobs or even different branches. - -The `cache:key` variable can use any of the -[predefined variables](../variables/README.md). The default key, if not -set, is just literal `default`, which means everything is shared between -pipelines and jobs by default. - -For example, to enable per-branch caching: - -```yaml -cache: - key: "$CI_COMMIT_REF_SLUG" - paths: - - binaries/ -``` - -If you use **Windows Batch** to run your shell scripts you need to replace -`$` with `%`: +Use the `cache:key` keyword to give each cache a unique identifying key. All jobs +that use the same cache key use the same cache, including in different pipelines. -```yaml -cache: - key: "%CI_COMMIT_REF_SLUG%" - paths: - - binaries/ -``` +If not set, the default key is `default`. All jobs with the `cache:` keyword but +no `cache:key` share the `default` cache. -The `cache:key` variable can't contain the `/` character, or the equivalent -URI-encoded `%2F`. A value made only of dots (`.`, `%2E`) is also forbidden. - -You can specify a [fallback cache key](#fallback-cache-key) to use if the specified `cache:key` is not found. +**Keyword type**: Job-specific. You can use it only as part of a job. -##### Multiple caches +**Possible inputs**: -> - [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/32814) in GitLab 13.10. -> - [Feature Flag removed](https://gitlab.com/gitlab-org/gitlab/-/issues/321877), in GitLab 13.12. +- A string. +- A [predefined variables](../variables/README.md). +- A combination of both. -You can have a maximum of four caches: +**Example of `cache:key`**: ```yaml -test-job: - stage: build - cache: - - key: - files: - - Gemfile.lock - paths: - - vendor/ruby - - key: - files: - - yarn.lock - paths: - - .yarn-cache/ +cache-job: script: - - bundle install --path=vendor - - yarn install --cache-folder .yarn-cache - - echo Run tests... + - echo "This job uses a cache." + cache: + key: binaries-cache-$CI_COMMIT_REF_SLUG + paths: + - binaries/ ``` -If multiple caches are combined with a [Fallback cache key](#fallback-cache-key), -the fallback is fetched multiple times if multiple caches are not found. - -#### Fallback cache key - -> [Introduced](https://gitlab.com/gitlab-org/gitlab-runner/-/merge_requests/1534) in GitLab Runner 13.4. +**Additional details**: -You can use the `$CI_COMMIT_REF_SLUG` [variable](#variables) to specify your [`cache:key`](#cachekey). -For example, if your `$CI_COMMIT_REF_SLUG` is `test` you can set a job -to download cache that's tagged with `test`. +- If you use **Windows Batch** to run your shell scripts you need to replace + `$` with `%`. For example: `key: %CI_COMMIT_REF_SLUG%` +- The `cache:key` value can't contain: -If a cache with this tag is not found, you can use `CACHE_FALLBACK_KEY` to -specify a cache to use when none exists. + - The `/` character, or the equivalent URI-encoded `%2F`. + - Only the `.` character (any number), or the equivalent URI-encoded `%2E`. -In the following example, if the `$CI_COMMIT_REF_SLUG` is not found, the job uses the key defined -by the `CACHE_FALLBACK_KEY` variable: +- The cache is shared between jobs, so if you're using different + paths for different jobs, you should also set a different `cache:key`. + Otherwise cache content can be overwritten. -```yaml -variables: - CACHE_FALLBACK_KEY: fallback-key +**Related topics**: -cache: - key: "$CI_COMMIT_REF_SLUG" - paths: - - binaries/ -``` +- You can specify a [fallback cache key](../caching/index.md#fallback-cache-key) + to use if the specified `cache:key` is not found. +- You can [use multiple cache keys](../caching/index.md#use-multiple-caches) in a single job. +- See the [common `cache` use cases](../caching/index.md#common-use-cases) for more + `cache:key` examples. ##### `cache:key:files` > [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/18986) in GitLab v12.5. -The `cache:key:files` keyword extends the `cache:key` functionality by making it easier -to reuse some caches, and rebuild them less often, which speeds up subsequent pipeline -runs. +Use the `cache:key:files` keyword to generate a new key when one or two specific files +change. `cache:key:files` lets you reuse some caches, and rebuild them less often, +which speeds up subsequent pipeline runs. -When you include `cache:key:files`, you must also list the project files that are used to generate the key, up to a maximum of two files. -The cache `key` is a SHA checksum computed from the most recent commits (up to two, if two files are listed) -that changed the given files. If neither file is changed in any commits, -the fallback key is `default`. +**Keyword type**: Job-specific. You can use it only as part of a job. + +**Possible inputs**: An array of one or two file paths. + +**Example of `cache:key:files`**: ```yaml -cache: - key: - files: - - Gemfile.lock - - package.json - paths: - - vendor/ruby - - node_modules +cache-job: + script: + - echo "This job uses a cache." + cache: + key: + files: + - Gemfile.lock + - package.json + paths: + - vendor/ruby + - node_modules ``` -This example creates a cache for Ruby and Node.js dependencies that -is tied to current versions of the `Gemfile.lock` and `package.json` files. Whenever one of +This example creates a cache for Ruby and Node.js dependencies. The cache +is tied to the current versions of the `Gemfile.lock` and `package.json` files. When one of these files changes, a new cache key is computed and a new cache is created. Any future job runs that use the same `Gemfile.lock` and `package.json` with `cache:key:files` use the new cache, instead of rebuilding the dependencies. +**Additional details**: The cache `key` is a SHA computed from the most recent commits +that changed each listed file. If neither file is changed in any commits, the +fallback key is `default`. + ##### `cache:key:prefix` > [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/18986) in GitLab v12.5. -When you want to combine a prefix with the SHA computed for `cache:key:files`, -use the `prefix` keyword with `key:files`. -For example, if you add a `prefix` of `test`, the resulting key is: `test-feef9576d21ee9b6a32e30c5c79d0a0ceb68d1e5`. -If neither file is changed in any commits, the prefix is added to `default`, so the -key in the example would be `test-default`. +Use `cache:key:prefix` to combine a prefix with the SHA computed for [`cache:key:files`](#cachekeyfiles). -Like `cache:key`, `prefix` can use any of the [predefined variables](../variables/README.md), -but cannot include: +**Keyword type**: Job-specific. You can use it only as part of a job. -- the `/` character (or the equivalent URI-encoded `%2F`) -- a value made only of `.` (or the equivalent URI-encoded `%2E`) +**Possible inputs**: -```yaml -cache: - key: - files: - - Gemfile.lock - prefix: ${CI_JOB_NAME} - paths: - - vendor/ruby +- A string +- A [predefined variables](../variables/README.md) +- A combination of both. +**Example of `cache:key:prefix`**: + +```yaml rspec: script: - - bundle exec rspec + - echo "This rspec job uses a cache." + cache: + key: + files: + - Gemfile.lock + prefix: $CI_JOB_NAME + paths: + - vendor/ruby ``` -For example, adding a `prefix` of `$CI_JOB_NAME` -causes the key to look like: `rspec-feef9576d21ee9b6a32e30c5c79d0a0ceb68d1e5` and -the job cache is shared across different branches. If a branch changes -`Gemfile.lock`, that branch has a new SHA checksum for `cache:key:files`. A new cache key -is generated, and a new cache is created for that key. -If `Gemfile.lock` is not found, the prefix is added to -`default`, so the key in the example would be `rspec-default`. +For example, adding a `prefix` of `$CI_JOB_NAME` causes the key to look like `rspec-feef9576d21ee9b6a32e30c5c79d0a0ceb68d1e5`. +If a branch changes `Gemfile.lock`, that branch has a new SHA checksum for `cache:key:files`. +A new cache key is generated, and a new cache is created for that key. If `Gemfile.lock` +is not found, the prefix is added to `default`, so the key in the example would be `rspec-default`. + +**Additional details**: If no file in `cache:key:files` is changed in any commits, +the prefix is added to the `default` key. #### `cache:untracked` -Set `untracked: true` to cache all files that are untracked in your Git -repository: +Use `untracked: true` to cache all files that are untracked in your Git repository: -```yaml -rspec: - script: test - cache: - untracked: true -``` +**Keyword type**: Job-specific. You can use it only as part of a job. -Cache all Git untracked files and files in `binaries`: +**Possible inputs**: `true` or `false` (default). + +**Example of `cache:untracked`**: ```yaml rspec: script: test cache: untracked: true - paths: - - binaries/ ``` +**Additional details**: + +- You can combine `cache:untracked` with `cache:paths` to cache all untracked files + as well as files in the configured paths. For example: + + ```yaml + rspec: + script: test + cache: + untracked: true + paths: + - binaries/ + ``` + #### `cache:when` > [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/18969) in GitLab 13.5 and GitLab Runner v13.5.0. -`cache:when` defines when to save the cache, based on the status of the job. You can -set `cache:when` to: +Use `cache:when` to define when to save the cache, based on the status of the job. + +**Keyword type**: Job-specific. You can use it only as part of a job. + +**Possible inputs**: - `on_success` (default): Save the cache only when the job succeeds. - `on_failure`: Save the cache only when the job fails. - `always`: Always save the cache. -For example, to store a cache whether or not the job fails or succeeds: +**Example of `cache:untracked`**: ```yaml rspec: @@ -2605,32 +2570,47 @@ rspec: when: 'always' ``` +This example stores the cache whether or not the job fails or succeeds. + #### `cache:policy` -The default behavior of a caching job is to download the files at the start of -execution, and to re-upload them at the end. Any changes made by the -job are persisted for future runs. This behavior is known as the `pull-push` cache -policy. +To change the upload and download behavior of a cache, use the `cache:policy` keyword. +By default, the job downloads the cache when the job starts, and uploads changes +to the cache when the job ends. This is the `pull-push` policy (default). -If you know the job does not alter the cached files, you can skip the upload step -by setting `policy: pull` in the job specification. You can add an ordinary cache -job at an earlier stage to ensure the cache is updated from time to time: +To set a job to only download the cache when the job starts, but never upload changes +when the job finishes, use `cache:policy:pull`. -```yaml -stages: - - setup - - test +To set a job to only upload a cache when the job finishes, but never download the +cache when the job starts, use `cache:policy:push`. + +Use the `pull` policy when you have many jobs executing in parallel that use the same cache. +This policy speeds up job execution and reduces load on the cache server. You can +use a job with the `push` policy to build the cache. + +**Keyword type**: Job-specific. You can use it only as part of a job. + +**Possible inputs**: + +- `pull` +- `push` +- `pull-push` (default) -prepare: - stage: setup +**Example of `cache:policy`**: + +```yaml +prepare-dependencies-job: + stage: build cache: key: gems paths: - vendor/bundle + policy: push script: - - bundle install --deployment + - echo "This job only downloads dependencies and builds the cache." + - echo "Downloading dependencies..." -rspec: +faster-test-job: stage: test cache: key: gems @@ -2638,16 +2618,10 @@ rspec: - vendor/bundle policy: pull script: - - bundle exec rspec ... + - echo "This job script uses the cache, but does not update it." + - echo "Running tests..." ``` -Use the `pull` policy when you have many jobs executing in parallel that use caches. This -policy speeds up job execution and reduces load on the cache server. - -If you have a job that unconditionally recreates the cache without -referring to its previous contents, you can skip the download step. -To do so, add `policy: push` to the job. - ### `artifacts` Use `artifacts` to specify a list of files and directories that are diff --git a/doc/development/documentation/styleguide/index.md b/doc/development/documentation/styleguide/index.md index 8d65408a175..3d4c2654f83 100644 --- a/doc/development/documentation/styleguide/index.md +++ b/doc/development/documentation/styleguide/index.md @@ -1105,9 +1105,11 @@ Another example: An Admin Area example: -`1. On the top bar, select **Menu >** **{admin}** **Admin**.` +```markdown +1. On the top bar, select **Menu >** **{admin}** **Admin**. +``` -This text generates this HTML: +This text renders this output: 1. On the top bar, select **Menu >** **{admin}** **Admin**. @@ -1757,7 +1759,7 @@ badges and tooltips (`<span class="badge-trigger free">`). | _Only_ GitLab Ultimate SaaS (no self-managed instances) | `**(ULTIMATE SAAS)**` | Topics that mention the `gitlab.rb` file are referring to -self-managed instances of GitLab. To prevent confusion, include the relevant `TIER ONLY` +self-managed instances of GitLab. To prevent confusion, include the relevant `TIER SELF` tier badge on the highest applicable heading level on the page. diff --git a/doc/development/documentation/styleguide/word_list.md b/doc/development/documentation/styleguide/word_list.md index fd8766bbfb6..d3cde5e4e9c 100644 --- a/doc/development/documentation/styleguide/word_list.md +++ b/doc/development/documentation/styleguide/word_list.md @@ -99,7 +99,7 @@ Do not use first-person singular. Use **you**, **we**, or **us** instead. ([Vale ## Owner -When writing about the Owner role, use a capital "M." Do not use the phrase, "if you are an owner" +When writing about the Owner role, use a capital "O." Do not use the phrase, "if you are an owner" to mean someone who is assigned the Owner role. Instead, write it out. "If you are assigned the Owner role..." Do not use "Owner permissions." A user who is assigned the Owner role has a set of associated permissions. diff --git a/doc/development/pipelines.md b/doc/development/pipelines.md index 0dc1481f542..30a92181a7d 100644 --- a/doc/development/pipelines.md +++ b/doc/development/pipelines.md @@ -559,7 +559,7 @@ request, be sure to start the `dont-interrupt-me` job before pushing. - `.qa-cache` - `.yarn-cache` - `.assets-compile-cache` (the key includes `${NODE_ENV}` so it's actually two different caches). -1. These cache definitions are composed of [multiple atomic caches](../ci/yaml/README.md#multiple-caches). +1. These cache definitions are composed of [multiple atomic caches](../ci/caching/index.md#use-multiple-caches). 1. Only 6 specific jobs, running in 2-hourly scheduled pipelines, are pushing (i.e. updating) to the caches: - `update-setup-test-env-cache`, defined in [`.gitlab/ci/rails.gitlab-ci.yml`](https://gitlab.com/gitlab-org/gitlab/-/blob/master/.gitlab/ci/rails.gitlab-ci.yml). - `update-gitaly-binaries-cache`, defined in [`.gitlab/ci/rails.gitlab-ci.yml`](https://gitlab.com/gitlab-org/gitlab/-/blob/master/.gitlab/ci/rails.gitlab-ci.yml). diff --git a/doc/user/project/push_options.md b/doc/user/project/push_options.md index 728c682b009..46619c96f2f 100644 --- a/doc/user/project/push_options.md +++ b/doc/user/project/push_options.md @@ -66,6 +66,7 @@ time as pushing changes: | `merge_request.remove_source_branch` | Set the merge request to remove the source branch when it's merged. | [12.2](https://gitlab.com/gitlab-org/gitlab-foss/-/issues/64320) | | `merge_request.title="<title>"` | Set the title of the merge request. Ex: `git push -o merge_request.title="The title I want"`. | [12.2](https://gitlab.com/gitlab-org/gitlab-foss/-/issues/64320) | | `merge_request.description="<description>"` | Set the description of the merge request. Ex: `git push -o merge_request.description="The description I want"`. | [12.2](https://gitlab.com/gitlab-org/gitlab-foss/-/issues/64320) | +| `merge_request.milestone="<milestone>"` | Set the milestone of the merge request. Ex: `git push -o merge_request.milestone="3.0"`. | [14.1](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/63960) | | `merge_request.label="<label>"` | Add labels to the merge request. If the label does not exist, it is created. For example, for two labels: `git push -o merge_request.label="label1" -o merge_request.label="label2"`. | [12.3](https://gitlab.com/gitlab-org/gitlab-foss/-/merge_requests/31831) | | `merge_request.unlabel="<label>"` | Remove labels from the merge request. For example, for two labels: `git push -o merge_request.unlabel="label1" -o merge_request.unlabel="label2"`. | [12.3](https://gitlab.com/gitlab-org/gitlab-foss/-/merge_requests/31831) | | `merge_request.assign="<user>"` | Assign users to the merge request. For example, for two users: `git push -o merge_request.assign="user1" -o merge_request.assign="user2"`. | [13.10](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/25904) | diff --git a/geo_architecture.png b/geo_architecture.png Binary files differdeleted file mode 100644 index d1ad3d1d93d..00000000000 --- a/geo_architecture.png +++ /dev/null diff --git a/lib/api/api.rb b/lib/api/api.rb index 2a3033753f7..88343384f07 100644 --- a/lib/api/api.rb +++ b/lib/api/api.rb @@ -172,6 +172,7 @@ module API mount ::API::Features mount ::API::Files mount ::API::FreezePeriods + mount ::API::Geo mount ::API::GroupAvatar mount ::API::GroupBoards mount ::API::GroupClusters diff --git a/lib/api/geo.rb b/lib/api/geo.rb new file mode 100644 index 00000000000..9fc610c9b32 --- /dev/null +++ b/lib/api/geo.rb @@ -0,0 +1,29 @@ +# frozen_string_literal: true + +module API + class Geo < ::API::Base + feature_category :geo_replication + + helpers do + # Overridden in EE + def geo_proxy_response + {} + end + end + + resource :geo do + # Workhorse calls this to determine if it is a Geo site that should proxy + # requests. Workhorse doesn't know if it's in a FOSS/EE context. + get '/proxy' do + require_gitlab_workhorse! + + status :ok + content_type Gitlab::Workhorse::INTERNAL_API_CONTENT_TYPE + + geo_proxy_response + end + end + end +end + +API::Geo.prepend_mod diff --git a/lib/api/internal/base.rb b/lib/api/internal/base.rb index ee0ddccc8d4..a06b052847d 100644 --- a/lib/api/internal/base.rb +++ b/lib/api/internal/base.rb @@ -124,11 +124,6 @@ module API yield end end - - # Overridden in EE - def geo_proxy - {} - end end namespace 'internal' do @@ -320,12 +315,6 @@ module API two_factor_otp_check end - - # Workhorse calls this to determine if it is a Geo secondary site - # that should proxy requests. FOSS can quickly return empty data. - get '/geo_proxy', feature_category: :geo_replication do - geo_proxy - end end end end diff --git a/lib/gitlab/database/background_migration/batched_migration.rb b/lib/gitlab/database/background_migration/batched_migration.rb index 36e89023c86..fa59ea06f35 100644 --- a/lib/gitlab/database/background_migration/batched_migration.rb +++ b/lib/gitlab/database/background_migration/batched_migration.rb @@ -29,11 +29,16 @@ module Gitlab paused: 0, active: 1, finished: 3, - failed: 4 + failed: 4, + finalizing: 5 } attribute :pause_ms, :integer, default: 100 + def self.find_for_configuration(job_class_name, table_name, column_name, job_arguments) + for_configuration(job_class_name, table_name, column_name, job_arguments).first + end + def self.active_migration active.queue_order.first end diff --git a/lib/gitlab/database/background_migration/batched_migration_runner.rb b/lib/gitlab/database/background_migration/batched_migration_runner.rb index 67fe6c536e6..14e3919986e 100644 --- a/lib/gitlab/database/background_migration/batched_migration_runner.rb +++ b/lib/gitlab/database/background_migration/batched_migration_runner.rb @@ -4,6 +4,12 @@ module Gitlab module Database module BackgroundMigration class BatchedMigrationRunner + FailedToFinalize = Class.new(RuntimeError) + + def self.finalize(job_class_name, table_name, column_name, job_arguments) + new.finalize(job_class_name, table_name, column_name, job_arguments) + end + def initialize(migration_wrapper = BatchedMigrationWrapper.new) @migration_wrapper = migration_wrapper end @@ -37,10 +43,35 @@ module Gitlab raise 'this method is not intended for use in real environments' end - while migration.active? - run_migration_job(migration) + run_migration_while(migration, :active) + end - migration.reload_last_job + # Finalize migration for given configuration. + # + # If the migration is already finished, do nothing. Otherwise change its status to `finalizing` + # in order to prevent it being picked up by the background worker. Perform all pending jobs, + # then keep running until migration is finished. + def finalize(job_class_name, table_name, column_name, job_arguments) + migration = BatchedMigration.find_for_configuration(job_class_name, table_name, column_name, job_arguments) + + configuration = { + job_class_name: job_class_name, + table_name: table_name, + column_name: column_name, + job_arguments: job_arguments + } + + if migration.nil? + Gitlab::AppLogger.warn "Could not find batched background migration for the given configuration: #{configuration}" + elsif migration.finished? + Gitlab::AppLogger.warn "Batched background migration for the given configuration is already finished: #{configuration}" + else + migration.finalizing! + migration.batched_jobs.pending.each { |job| migration_wrapper.perform(job) } + + run_migration_while(migration, :finalizing) + + raise FailedToFinalize unless migration.finished? end end @@ -90,6 +121,14 @@ module Gitlab active_migration.finished! end end + + def run_migration_while(migration, status) + while migration.status == status.to_s + run_migration_job(migration) + + migration.reload_last_job + end + end end end end diff --git a/lib/gitlab/database/migration_helpers.rb b/lib/gitlab/database/migration_helpers.rb index d155abefdc8..e419ef76bb5 100644 --- a/lib/gitlab/database/migration_helpers.rb +++ b/lib/gitlab/database/migration_helpers.rb @@ -1106,7 +1106,11 @@ module Gitlab Gitlab::AppLogger.warn "Could not find batched background migration for the given configuration: #{configuration}" elsif !migration.finished? raise "Expected batched background migration for the given configuration to be marked as 'finished', " \ - "but it is '#{migration.status}': #{configuration}" + "but it is '#{migration.status}': #{configuration}" \ + "\n\n" \ + "Finalize it manualy by running" \ + "\n\n" \ + "\tgitlab-rake gitlab:background_migrations:finalize[#{job_class_name},#{table_name},#{column_name},'#{job_arguments.inspect.gsub(',', '\,')}']" end end diff --git a/lib/gitlab/error_tracking/processor/grpc_error_processor.rb b/lib/gitlab/error_tracking/processor/grpc_error_processor.rb index e2a9192806f..e835deeea2c 100644 --- a/lib/gitlab/error_tracking/processor/grpc_error_processor.rb +++ b/lib/gitlab/error_tracking/processor/grpc_error_processor.rb @@ -35,7 +35,12 @@ module Gitlab # Worse in new version, no setter! Have to poke at the # instance variable - exception.value = message if message + if message.present? + exceptions.each do |exception| + exception.value = message if valid_exception?(exception) + end + end + event.extra[:grpc_debug_error_string] = debug_str if debug_str end diff --git a/lib/gitlab/push_options.rb b/lib/gitlab/push_options.rb index ce9fced9465..9d954a74948 100644 --- a/lib/gitlab/push_options.rb +++ b/lib/gitlab/push_options.rb @@ -10,6 +10,7 @@ module Gitlab :description, :label, :merge_when_pipeline_succeeds, + :milestone, :remove_source_branch, :target, :title, diff --git a/lib/tasks/gitlab/background_migrations.rake b/lib/tasks/gitlab/background_migrations.rake new file mode 100644 index 00000000000..c978a2807ca --- /dev/null +++ b/lib/tasks/gitlab/background_migrations.rake @@ -0,0 +1,23 @@ +# frozen_string_literal: true + +namespace :gitlab do + namespace :background_migrations do + task :finalize, [:job_class_name, :table_name, :column_name, :job_arguments] => :environment do |_, args| + [:job_class_name, :table_name, :column_name, :job_arguments].each do |argument| + unless args[argument] + puts "Must specify #{argument} as an argument".color(:red) + exit 1 + end + end + + Gitlab::Database::BackgroundMigration::BatchedMigrationRunner.finalize( + args[:job_class_name], + args[:table_name], + args[:column_name], + Gitlab::Json.parse(args[:job_arguments]) + ) + + puts "Done.".color(:green) + end + end +end diff --git a/package.json b/package.json index 633cdeabadc..b4bd40a8fd8 100644 --- a/package.json +++ b/package.json @@ -56,7 +56,7 @@ "@gitlab/favicon-overlay": "2.0.0", "@gitlab/svgs": "1.199.0", "@gitlab/tributejs": "1.0.0", - "@gitlab/ui": "29.36.0", + "@gitlab/ui": "29.37.0", "@gitlab/visual-review-tools": "1.6.1", "@rails/actioncable": "6.1.3-2", "@rails/ujs": "6.1.3-2", diff --git a/spec/controllers/registrations/experience_levels_controller_spec.rb b/spec/controllers/registrations/experience_levels_controller_spec.rb index 6b8ab3ec715..ad145264bb8 100644 --- a/spec/controllers/registrations/experience_levels_controller_spec.rb +++ b/spec/controllers/registrations/experience_levels_controller_spec.rb @@ -24,7 +24,7 @@ RSpec.describe Registrations::ExperienceLevelsController do end it { is_expected.to have_gitlab_http_status(:ok) } - it { is_expected.to render_template('layouts/signup_onboarding') } + it { is_expected.to render_template('layouts/minimal') } it { is_expected.to render_template(:show) } end end diff --git a/spec/frontend/add_context_commits_modal/components/__snapshots__/add_context_commits_modal_spec.js.snap b/spec/frontend/add_context_commits_modal/components/__snapshots__/add_context_commits_modal_spec.js.snap index 1eb9ccc9c6c..10437c48f88 100644 --- a/spec/frontend/add_context_commits_modal/components/__snapshots__/add_context_commits_modal_spec.js.snap +++ b/spec/frontend/add_context_commits_modal/components/__snapshots__/add_context_commits_modal_spec.js.snap @@ -16,6 +16,7 @@ exports[`AddContextCommitsModal renders modal with 2 tabs 1`] = ` > <gl-tabs-stub contentclass="pt-0" + queryparamname="tab" theme="indigo" value="0" > diff --git a/spec/frontend/code_navigation/components/__snapshots__/popover_spec.js.snap b/spec/frontend/code_navigation/components/__snapshots__/popover_spec.js.snap index b59d1597a12..118d8ceceb9 100644 --- a/spec/frontend/code_navigation/components/__snapshots__/popover_spec.js.snap +++ b/spec/frontend/code_navigation/components/__snapshots__/popover_spec.js.snap @@ -13,7 +13,9 @@ exports[`Code navigation popover component renders popover 1`] = ` <gl-tabs-stub contentclass="gl-py-0" navclass="gl-hidden" + queryparamname="tab" theme="indigo" + value="0" > <gl-tab-stub title="Definition" diff --git a/spec/frontend/incidents_settings/components/__snapshots__/incidents_settings_tabs_spec.js.snap b/spec/frontend/incidents_settings/components/__snapshots__/incidents_settings_tabs_spec.js.snap index 4f70f908c4a..1e3c344ce65 100644 --- a/spec/frontend/incidents_settings/components/__snapshots__/incidents_settings_tabs_spec.js.snap +++ b/spec/frontend/incidents_settings/components/__snapshots__/incidents_settings_tabs_spec.js.snap @@ -39,7 +39,9 @@ exports[`IncidentsSettingTabs should render the component 1`] = ` class="settings-content" > <gl-tabs-stub + queryparamname="tab" theme="indigo" + value="0" > <!----> diff --git a/spec/frontend/pipelines/graph/mock_data.js b/spec/frontend/pipelines/graph/mock_data.js index 28fe3b67e7b..3812483766d 100644 --- a/spec/frontend/pipelines/graph/mock_data.js +++ b/spec/frontend/pipelines/graph/mock_data.js @@ -12,6 +12,10 @@ export const mockPipelineResponse = { usesNeeds: true, downstream: null, upstream: null, + userPermissions: { + __typename: 'PipelinePermissions', + updatePipeline: true, + }, stages: { __typename: 'CiStageConnection', nodes: [ @@ -573,6 +577,10 @@ export const wrappedPipelineReturn = { iid: '38', complete: true, usesNeeds: true, + userPermissions: { + __typename: 'PipelinePermissions', + updatePipeline: true, + }, downstream: { __typename: 'PipelineConnection', nodes: [], diff --git a/spec/frontend/pipelines/graph/stage_column_component_spec.js b/spec/frontend/pipelines/graph/stage_column_component_spec.js index f9f6c96a1a6..2a89dbd3fa5 100644 --- a/spec/frontend/pipelines/graph/stage_column_component_spec.js +++ b/spec/frontend/pipelines/graph/stage_column_component_spec.js @@ -31,6 +31,9 @@ const defaultProps = { name: 'Fish', groups: mockGroups, pipelineId: 159, + userPermissions: { + updatePipeline: true, + }, }; describe('stage column component', () => { @@ -152,36 +155,52 @@ describe('stage column component', () => { }); describe('with action', () => { - beforeEach(() => { + const defaults = { + groups: [ + { + id: 4259, + name: '<img src=x onerror=alert(document.domain)>', + status: { + icon: 'status_success', + label: 'success', + tooltip: '<img src=x onerror=alert(document.domain)>', + }, + jobs: [mockJob], + }, + ], + title: 'test', + hasTriggeredBy: false, + action: { + icon: 'play', + title: 'Play all', + path: 'action', + }, + }; + + it('renders action button if permissions are permitted', () => { createComponent({ method: mount, props: { - groups: [ - { - id: 4259, - name: '<img src=x onerror=alert(document.domain)>', - status: { - icon: 'status_success', - label: 'success', - tooltip: '<img src=x onerror=alert(document.domain)>', - }, - jobs: [mockJob], - }, - ], - title: 'test', - hasTriggeredBy: false, - action: { - icon: 'play', - title: 'Play all', - path: 'action', - }, + ...defaults, }, }); - }); - it('renders action button', () => { expect(findActionComponent().exists()).toBe(true); }); + + it('does not render action button if permissions are not permitted', () => { + createComponent({ + method: mount, + props: { + ...defaults, + userPermissions: { + updatePipeline: false, + }, + }, + }); + + expect(findActionComponent().exists()).toBe(false); + }); }); describe('without action', () => { diff --git a/spec/frontend/repository/components/blob_content_viewer_spec.js b/spec/frontend/repository/components/blob_content_viewer_spec.js index 495039b4ccb..5492fdd3065 100644 --- a/spec/frontend/repository/components/blob_content_viewer_spec.js +++ b/spec/frontend/repository/components/blob_content_viewer_spec.js @@ -4,7 +4,7 @@ import { nextTick } from 'vue'; import BlobContent from '~/blob/components/blob_content.vue'; import BlobHeader from '~/blob/components/blob_header.vue'; import BlobContentViewer from '~/repository/components/blob_content_viewer.vue'; -import BlobHeaderEdit from '~/repository/components/blob_header_edit.vue'; +import BlobEdit from '~/repository/components/blob_edit.vue'; import BlobReplace from '~/repository/components/blob_replace.vue'; let wrapper; @@ -78,7 +78,7 @@ const fullFactory = createFactory(mount); describe('Blob content viewer component', () => { const findLoadingIcon = () => wrapper.findComponent(GlLoadingIcon); const findBlobHeader = () => wrapper.findComponent(BlobHeader); - const findBlobHeaderEdit = () => wrapper.findComponent(BlobHeaderEdit); + const findBlobEdit = () => wrapper.findComponent(BlobEdit); const findBlobContent = () => wrapper.findComponent(BlobContent); const findBlobReplace = () => wrapper.findComponent(BlobReplace); @@ -177,7 +177,7 @@ describe('Blob content viewer component', () => { await nextTick(); - expect(findBlobHeaderEdit().props()).toMatchObject({ + expect(findBlobEdit().props()).toMatchObject({ editPath: editBlobPath, webIdePath: ideEditPath, }); @@ -194,7 +194,7 @@ describe('Blob content viewer component', () => { await nextTick(); - expect(findBlobHeaderEdit().props()).toMatchObject({ + expect(findBlobEdit().props()).toMatchObject({ editPath: editBlobPath, webIdePath: ideEditPath, }); diff --git a/spec/frontend/repository/components/blob_header_edit_spec.js b/spec/frontend/repository/components/blob_edit_spec.js index c0eb7c523c4..e6e69cd8549 100644 --- a/spec/frontend/repository/components/blob_header_edit_spec.js +++ b/spec/frontend/repository/components/blob_edit_spec.js @@ -1,6 +1,6 @@ import { GlButton } from '@gitlab/ui'; import { shallowMount } from '@vue/test-utils'; -import BlobHeaderEdit from '~/repository/components/blob_header_edit.vue'; +import BlobEdit from '~/repository/components/blob_edit.vue'; import WebIdeLink from '~/vue_shared/components/web_ide_link.vue'; const DEFAULT_PROPS = { @@ -8,11 +8,11 @@ const DEFAULT_PROPS = { webIdePath: 'some_file.js/ide/edit', }; -describe('BlobHeaderEdit component', () => { +describe('BlobEdit component', () => { let wrapper; const createComponent = (consolidatedEditButton = false, props = {}) => { - wrapper = shallowMount(BlobHeaderEdit, { + wrapper = shallowMount(BlobEdit, { propsData: { ...DEFAULT_PROPS, ...props, diff --git a/spec/lib/gitlab/database/background_migration/batched_migration_runner_spec.rb b/spec/lib/gitlab/database/background_migration/batched_migration_runner_spec.rb index 9f0493ab0d7..779e8e40c97 100644 --- a/spec/lib/gitlab/database/background_migration/batched_migration_runner_spec.rb +++ b/spec/lib/gitlab/database/background_migration/batched_migration_runner_spec.rb @@ -281,4 +281,152 @@ RSpec.describe Gitlab::Database::BackgroundMigration::BatchedMigrationRunner do end end end + + describe '#finalize' do + let(:migration_wrapper) { Gitlab::Database::BackgroundMigration::BatchedMigrationWrapper.new } + + let(:migration_helpers) { ActiveRecord::Migration.new } + let(:table_name) { :_batched_migrations_test_table } + let(:column_name) { :some_id } + let(:job_arguments) { [:some_id, :some_id_convert_to_bigint] } + + let(:migration_status) { :active } + + let!(:batched_migration) do + create( + :batched_background_migration, + status: migration_status, + max_value: 8, + batch_size: 2, + sub_batch_size: 1, + interval: 0, + table_name: table_name, + column_name: column_name, + job_arguments: job_arguments, + pause_ms: 0 + ) + end + + before do + migration_helpers.drop_table table_name, if_exists: true + migration_helpers.create_table table_name, id: false do |t| + t.integer :some_id, primary_key: true + t.integer :some_id_convert_to_bigint + end + + migration_helpers.execute("INSERT INTO #{table_name} VALUES (1, 1), (2, 2), (3, NULL), (4, NULL), (5, NULL), (6, NULL), (7, NULL), (8, NULL)") + end + + after do + migration_helpers.drop_table table_name, if_exists: true + end + + context 'when the migration is not yet completed' do + before do + common_attributes = { + batched_migration: batched_migration, + batch_size: 2, + sub_batch_size: 1, + pause_ms: 0 + } + + create(:batched_background_migration_job, common_attributes.merge(status: :succeeded, min_value: 1, max_value: 2)) + create(:batched_background_migration_job, common_attributes.merge(status: :pending, min_value: 3, max_value: 4)) + create(:batched_background_migration_job, common_attributes.merge(status: :failed, min_value: 5, max_value: 6, attempts: 1)) + end + + it 'completes the migration' do + expect(Gitlab::Database::BackgroundMigration::BatchedMigration).to receive(:find_for_configuration) + .with('CopyColumnUsingBackgroundMigrationJob', table_name, column_name, job_arguments) + .and_return(batched_migration) + + expect(batched_migration).to receive(:finalizing!).and_call_original + + expect do + runner.finalize( + batched_migration.job_class_name, + table_name, + column_name, + job_arguments + ) + end.to change { batched_migration.reload.status }.from('active').to('finished') + + expect(batched_migration.batched_jobs).to all(be_succeeded) + + not_converted = migration_helpers.execute("SELECT * FROM #{table_name} WHERE some_id_convert_to_bigint IS NULL") + expect(not_converted.to_a).to be_empty + end + + context 'when migration fails to complete' do + it 'raises an error' do + batched_migration.batched_jobs.failed.update_all(attempts: Gitlab::Database::BackgroundMigration::BatchedJob::MAX_ATTEMPTS) + + expect do + runner.finalize( + batched_migration.job_class_name, + table_name, + column_name, + job_arguments + ) + end.to raise_error described_class::FailedToFinalize + end + end + end + + context 'when the migration is already finished' do + let(:migration_status) { :finished } + + it 'is a no-op' do + expect(Gitlab::Database::BackgroundMigration::BatchedMigration).to receive(:find_for_configuration) + .with('CopyColumnUsingBackgroundMigrationJob', table_name, column_name, job_arguments) + .and_return(batched_migration) + + configuration = { + job_class_name: batched_migration.job_class_name, + table_name: table_name.to_sym, + column_name: column_name.to_sym, + job_arguments: job_arguments + } + + expect(Gitlab::AppLogger).to receive(:warn) + .with("Batched background migration for the given configuration is already finished: #{configuration}") + + expect(batched_migration).not_to receive(:finalizing!) + + runner.finalize( + batched_migration.job_class_name, + table_name, + column_name, + job_arguments + ) + end + end + + context 'when the migration does not exist' do + it 'is a no-op' do + expect(Gitlab::Database::BackgroundMigration::BatchedMigration).to receive(:find_for_configuration) + .with('CopyColumnUsingBackgroundMigrationJob', table_name, column_name, [:some, :other, :arguments]) + .and_return(nil) + + configuration = { + job_class_name: batched_migration.job_class_name, + table_name: table_name.to_sym, + column_name: column_name.to_sym, + job_arguments: [:some, :other, :arguments] + } + + expect(Gitlab::AppLogger).to receive(:warn) + .with("Could not find batched background migration for the given configuration: #{configuration}") + + expect(batched_migration).not_to receive(:finalizing!) + + runner.finalize( + batched_migration.job_class_name, + table_name, + column_name, + [:some, :other, :arguments] + ) + end + end + end end diff --git a/spec/lib/gitlab/database/background_migration/batched_migration_spec.rb b/spec/lib/gitlab/database/background_migration/batched_migration_spec.rb index d881390cd52..194d0243476 100644 --- a/spec/lib/gitlab/database/background_migration/batched_migration_spec.rb +++ b/spec/lib/gitlab/database/background_migration/batched_migration_spec.rb @@ -387,4 +387,22 @@ RSpec.describe Gitlab::Database::BackgroundMigration::BatchedMigration, type: :m expect(actual).to contain_exactly(migration) end end + + describe '.find_for_configuration' do + it 'returns nill if such migration does not exists' do + expect(described_class.find_for_configuration('MyJobClass', :projects, :id, [[:id], [:id_convert_to_bigint]])).to be_nil + end + + it 'returns the migration when it exists' do + migration = create( + :batched_background_migration, + job_class_name: 'MyJobClass', + table_name: :projects, + column_name: :id, + job_arguments: [[:id], [:id_convert_to_bigint]] + ) + + expect(described_class.find_for_configuration('MyJobClass', :projects, :id, [[:id], [:id_convert_to_bigint]])).to eq(migration) + end + end end diff --git a/spec/lib/gitlab/database/migration_helpers_spec.rb b/spec/lib/gitlab/database/migration_helpers_spec.rb index f0ea07646fb..d5e9de84538 100644 --- a/spec/lib/gitlab/database/migration_helpers_spec.rb +++ b/spec/lib/gitlab/database/migration_helpers_spec.rb @@ -2007,7 +2007,7 @@ RSpec.describe Gitlab::Database::MigrationHelpers do job_class_name: 'CopyColumnUsingBackgroundMigrationJob', table_name: :events, column_name: :id, - job_arguments: [[:id], [:id_convert_to_bigint]] + job_arguments: [["id"], ["id_convert_to_bigint"]] } end @@ -2017,7 +2017,11 @@ RSpec.describe Gitlab::Database::MigrationHelpers do create(:batched_background_migration, configuration.merge(status: :active)) expect { ensure_batched_background_migration_is_finished } - .to raise_error "Expected batched background migration for the given configuration to be marked as 'finished', but it is 'active': #{configuration}" + .to raise_error "Expected batched background migration for the given configuration to be marked as 'finished', but it is 'active': #{configuration}" \ + "\n\n" \ + "Finalize it manualy by running" \ + "\n\n" \ + "\tgitlab-rake gitlab:background_migrations:finalize[CopyColumnUsingBackgroundMigrationJob,events,id,'[[\"id\"]\\, [\"id_convert_to_bigint\"]]']" end it 'does not raise error when migration exists and is marked as finished' do diff --git a/spec/lib/gitlab/error_tracking/processor/grpc_error_processor_spec.rb b/spec/lib/gitlab/error_tracking/processor/grpc_error_processor_spec.rb index 6076e525f06..9acc7fd04be 100644 --- a/spec/lib/gitlab/error_tracking/processor/grpc_error_processor_spec.rb +++ b/spec/lib/gitlab/error_tracking/processor/grpc_error_processor_spec.rb @@ -15,6 +15,18 @@ RSpec.describe Gitlab::ErrorTracking::Processor::GrpcErrorProcessor do let(:event) { Raven::Event.from_exception(exception, required_options.merge(data)) } let(:result_hash) { described_class.call(event).to_hash } + let(:data) do + { + extra: { + caller: 'test' + }, + fingerprint: [ + 'GRPC::DeadlineExceeded', + '4:Deadline Exceeded. debug_error_string:{"created":"@1598938192.005782000","description":"Error received from peer unix:/home/git/gitalypraefect.socket","file":"src/core/lib/surface/call.cc","file_line":1055,"grpc_message":"Deadline Exceeded","grpc_status":4}' + ] + } + end + context 'when there is no GRPC exception' do let(:exception) { RuntimeError.new } let(:data) { { fingerprint: ['ArgumentError', 'Missing arguments'] } } @@ -24,19 +36,47 @@ RSpec.describe Gitlab::ErrorTracking::Processor::GrpcErrorProcessor do end end - context 'when there is a GPRC exception with a debug string' do + context 'when there is a GRPC exception with a debug string' do let(:exception) { GRPC::DeadlineExceeded.new('Deadline Exceeded', {}, '{"hello":1}') } - let(:data) do - { - extra: { - caller: 'test' - }, - fingerprint: [ - 'GRPC::DeadlineExceeded', - '4:Deadline Exceeded. debug_error_string:{"created":"@1598938192.005782000","description":"Error received from peer unix:/home/git/gitalypraefect.socket","file":"src/core/lib/surface/call.cc","file_line":1055,"grpc_message":"Deadline Exceeded","grpc_status":4}' - ] - } + it 'removes the debug error string and stores it as an extra field' do + expect(result_hash[:fingerprint]) + .to eq(['GRPC::DeadlineExceeded', '4:Deadline Exceeded.']) + + expect(result_hash[:exception][:values].first) + .to include(type: 'GRPC::DeadlineExceeded', value: '4:Deadline Exceeded.') + + expect(result_hash[:extra]) + .to include(caller: 'test', grpc_debug_error_string: '{"hello":1}') + end + + context 'with no custom fingerprint' do + let(:data) do + { extra: { caller: 'test' } } + end + + it 'removes the debug error string and stores it as an extra field' do + expect(result_hash).not_to include(:fingerprint) + + expect(result_hash[:exception][:values].first) + .to include(type: 'GRPC::DeadlineExceeded', value: '4:Deadline Exceeded.') + + expect(result_hash[:extra]) + .to include(caller: 'test', grpc_debug_error_string: '{"hello":1}') + end + end + end + + context 'when there is a wrapped GRPC exception with a debug string' do + let(:inner_exception) { GRPC::DeadlineExceeded.new('Deadline Exceeded', {}, '{"hello":1}') } + let(:exception) do + begin + raise inner_exception + rescue GRPC::DeadlineExceeded + raise StandardError.new, inner_exception.message + end + rescue StandardError => e + e end it 'removes the debug error string and stores it as an extra field' do @@ -46,6 +86,9 @@ RSpec.describe Gitlab::ErrorTracking::Processor::GrpcErrorProcessor do expect(result_hash[:exception][:values].first) .to include(type: 'GRPC::DeadlineExceeded', value: '4:Deadline Exceeded.') + expect(result_hash[:exception][:values].second) + .to include(type: 'StandardError', value: '4:Deadline Exceeded.') + expect(result_hash[:extra]) .to include(caller: 'test', grpc_debug_error_string: '{"hello":1}') end @@ -61,6 +104,9 @@ RSpec.describe Gitlab::ErrorTracking::Processor::GrpcErrorProcessor do expect(result_hash[:exception][:values].first) .to include(type: 'GRPC::DeadlineExceeded', value: '4:Deadline Exceeded.') + expect(result_hash[:exception][:values].second) + .to include(type: 'StandardError', value: '4:Deadline Exceeded.') + expect(result_hash[:extra]) .to include(caller: 'test', grpc_debug_error_string: '{"hello":1}') end diff --git a/spec/requests/api/geo_spec.rb b/spec/requests/api/geo_spec.rb new file mode 100644 index 00000000000..edbca5eb1c6 --- /dev/null +++ b/spec/requests/api/geo_spec.rb @@ -0,0 +1,30 @@ +# frozen_string_literal: true + +require 'spec_helper' + +RSpec.describe API::Geo do + include WorkhorseHelpers + + describe 'GET /geo/proxy' do + subject { get api('/geo/proxy'), headers: workhorse_headers } + + include_context 'workhorse headers' + + context 'with valid auth' do + it 'returns empty data' do + subject + + expect(response).to have_gitlab_http_status(:ok) + expect(json_response).to be_empty + end + end + + it 'rejects requests that bypassed gitlab-workhorse' do + workhorse_headers.delete(Gitlab::Workhorse::INTERNAL_API_REQUEST_HEADER) + + subject + + expect(response).to have_gitlab_http_status(:forbidden) + end + end +end diff --git a/spec/requests/api/internal/base_spec.rb b/spec/requests/api/internal/base_spec.rb index 631698554f9..3405d66f216 100644 --- a/spec/requests/api/internal/base_spec.rb +++ b/spec/requests/api/internal/base_spec.rb @@ -1378,29 +1378,6 @@ RSpec.describe API::Internal::Base do end end - describe 'GET /internal/geo_proxy' do - subject { get api('/internal/geo_proxy'), params: { secret_token: secret_token } } - - context 'with valid auth' do - it 'returns empty data' do - subject - - expect(response).to have_gitlab_http_status(:ok) - expect(json_response).to be_empty - end - end - - context 'with invalid auth' do - let(:secret_token) { 'invalid_token' } - - it 'returns unauthorized' do - subject - - expect(response).to have_gitlab_http_status(:unauthorized) - end - end - end - def lfs_auth_project(project) post( api("/internal/lfs_authenticate"), diff --git a/spec/services/merge_requests/push_options_handler_service_spec.rb b/spec/services/merge_requests/push_options_handler_service_spec.rb index 87c3fc6a2d8..c09435a70e2 100644 --- a/spec/services/merge_requests/push_options_handler_service_spec.rb +++ b/spec/services/merge_requests/push_options_handler_service_spec.rb @@ -10,6 +10,7 @@ RSpec.describe MergeRequests::PushOptionsHandlerService do let_it_be(:user2) { create(:user, developer_projects: [project]) } let_it_be(:user3) { create(:user, developer_projects: [project]) } let_it_be(:forked_project) { fork_project(project, user1, repository: true) } + let_it_be(:milestone) { create(:milestone, project: project, title: '1.0') } let(:service) { described_class.new(project: project, current_user: user1, changes: changes, push_options: push_options) } let(:source_branch) { 'fix' } @@ -59,6 +60,16 @@ RSpec.describe MergeRequests::PushOptionsHandlerService do end end + shared_examples_for 'a service that can set the milestone of a merge request' do + subject(:last_mr) { MergeRequest.last } + + it 'sets the milestone' do + service.execute + + expect(last_mr.milestone&.title).to eq(expected_milestone) + end + end + shared_examples_for 'a service that can set the merge request to merge when pipeline succeeds' do subject(:last_mr) { MergeRequest.last } @@ -514,6 +525,70 @@ RSpec.describe MergeRequests::PushOptionsHandlerService do it_behaves_like 'with the project default branch' end + describe '`milestone` push option' do + context 'with a valid milestone' do + let(:expected_milestone) { milestone.title } + let(:push_options) { { milestone: milestone.title } } + + context 'with a new branch' do + let(:changes) { new_branch_changes } + + it_behaves_like 'a service that does not create a merge request' + + it 'adds an error to the service' do + service.execute + + expect(service.errors).to include(error_mr_required) + end + + context 'when coupled with the `create` push option' do + let(:push_options) { { create: true, milestone: milestone.title } } + + it_behaves_like 'a service that can create a merge request' + it_behaves_like 'a service that can set the milestone of a merge request' + end + end + + context 'with an existing branch but no open MR' do + let(:changes) { existing_branch_changes } + + it_behaves_like 'a service that does not create a merge request' + + it 'adds an error to the service' do + service.execute + + expect(service.errors).to include(error_mr_required) + end + + context 'when coupled with the `create` push option' do + let(:push_options) { { create: true, milestone: milestone.title } } + + it_behaves_like 'a service that can create a merge request' + it_behaves_like 'a service that can set the milestone of a merge request' + end + end + + context 'with an existing branch that has a merge request open' do + let(:changes) { existing_branch_changes } + let!(:merge_request) { create(:merge_request, source_project: project, source_branch: source_branch)} + + it_behaves_like 'a service that does not create a merge request' + it_behaves_like 'a service that can set the milestone of a merge request' + end + + it_behaves_like 'with a deleted branch' + it_behaves_like 'with the project default branch' + end + + context 'with invalid milestone' do + let(:expected_milestone) { nil } + let(:changes) { new_branch_changes } + let(:push_options) { { create: true, milestone: 'invalid_milestone' } } + + it_behaves_like 'a service that can set the milestone of a merge request' + end + end + shared_examples 'with an existing branch that has a merge request open in foss' do let(:changes) { existing_branch_changes } let!(:merge_request) { create(:merge_request, source_project: project, source_branch: source_branch)} diff --git a/spec/workers/pipeline_hooks_worker_spec.rb b/spec/workers/pipeline_hooks_worker_spec.rb index 5957b355c8e..0ed00c0c66a 100644 --- a/spec/workers/pipeline_hooks_worker_spec.rb +++ b/spec/workers/pipeline_hooks_worker_spec.rb @@ -25,6 +25,5 @@ RSpec.describe PipelineHooksWorker do it_behaves_like 'worker with data consistency', described_class, - feature_flag: :load_balancing_for_pipeline_hooks_worker, data_consistency: :delayed end diff --git a/yarn.lock b/yarn.lock index f86c8d75736..46c6932fa68 100644 --- a/yarn.lock +++ b/yarn.lock @@ -908,10 +908,10 @@ resolved "https://registry.yarnpkg.com/@gitlab/tributejs/-/tributejs-1.0.0.tgz#672befa222aeffc83e7d799b0500a7a4418e59b8" integrity sha512-nmKw1+hB6MHvlmPz63yPwVs1qQkycHwsKgxpEbzmky16Y6mL4EJMk3w1b8QlOAF/AIAzjCERPhe/R4MJiohbZw== -"@gitlab/ui@29.36.0": - version "29.36.0" - resolved "https://registry.yarnpkg.com/@gitlab/ui/-/ui-29.36.0.tgz#a418c34c7ef768552b551807fa2a65deeaeba0bf" - integrity sha512-ZsaYpbp5cFN9hxVCf19E7avS9AmMaAyS4/Zwkwu2reHJUOkwyOY24eLr44u/Kbaq6SkFarQ2y+zU8vuhzXwQjQ== +"@gitlab/ui@29.37.0": + version "29.37.0" + resolved "https://registry.yarnpkg.com/@gitlab/ui/-/ui-29.37.0.tgz#ddfd4760562387f7c164756301f73e29c1a5cd13" + integrity sha512-DK+MRhCeAXs7RhbIq7k7z+jTvSoQFfziMgFidmFiyyLYsZRj0+ya2pF9SubxEzH9HKwhs2TNZFd28onO8i5upg== dependencies: "@babel/standalone" "^7.0.0" "@gitlab/vue-toasted" "^1.3.0" |