From 4e2bb4e5e7df1273a4d2fdd370b6c17a27c394d8 Mon Sep 17 00:00:00 2001 From: Stan Hu Date: Mon, 12 Aug 2019 15:31:58 -0700 Subject: Reduce Gitaly calls in PostReceive This commit reduces I/O load and memory utilization during PostReceive for the common case when no project hooks or services are set up. We saw a Gitaly N+1 issue in `CommitDelta` when many tags or branches are pushed. We can reduce this overhead in the common case because we observe that most new projects do not have any Web hooks or services, especially when they are first created. Previously, `BaseHooksService` unconditionally iterated through the last 20 commits of each ref to build the `push_data` structure. The `push_data` structured was used in numerous places: 1. Building the push payload in `EventCreateService` 2. Creating a CI pipeline 3. Executing project Web or system hooks 4. Executing project services 5. As the return value of `BaseHooksService#execute` 6. `BranchHooksService#invalidated_file_types` We only need to generate the full `push_data` for items 3, 4, and 6. Item 1: `EventCreateService` only needs the last commit and doesn't actually need the commit deltas. Item 2: In addition, `Ci::CreatePipelineService` only needed a subset of the parameters. Item 5: The return value of `BaseHooksService#execute` also wasn't being used anywhere. Item 6: This is only used when pushing to the default branch, so if many tags are pushed we can save significant I/O here. Closes https://gitlab.com/gitlab-org/gitlab-ce/issues/65878 Fic --- spec/lib/gitlab/data_builder/push_spec.rb | 34 +++++++++++++++++++++++++++++++ 1 file changed, 34 insertions(+) (limited to 'spec/lib/gitlab/data_builder') diff --git a/spec/lib/gitlab/data_builder/push_spec.rb b/spec/lib/gitlab/data_builder/push_spec.rb index cc31f88d365..e8a9f0b06a8 100644 --- a/spec/lib/gitlab/data_builder/push_spec.rb +++ b/spec/lib/gitlab/data_builder/push_spec.rb @@ -3,9 +3,43 @@ require 'spec_helper' describe Gitlab::DataBuilder::Push do + include RepoHelpers + let(:project) { create(:project, :repository) } let(:user) { build(:user, public_email: 'public-email@example.com') } + describe '.build' do + let(:sample) { RepoHelpers.sample_compare } + let(:commits) { project.repository.commits_between(sample.commits.first, sample.commits.last) } + let(:subject) do + described_class.build(project: project, + user: user, + ref: sample.target_branch, + commits: commits, + commits_count: commits.length, + message: 'test message', + with_changed_files: with_changed_files) + end + + context 'with changed files' do + let(:with_changed_files) { true } + + it 'returns commit hook data' do + expect(subject[:project]).to eq(project.hook_attrs) + expect(subject[:commits].first.keys).to include(*%i(added removed modified)) + end + end + + context 'without changed files' do + let(:with_changed_files) { false } + + it 'returns commit hook data without include deltas' do + expect(subject[:project]).to eq(project.hook_attrs) + expect(subject[:commits].first.keys).not_to include(*%i(added removed modified)) + end + end + end + describe '.build_sample' do let(:data) { described_class.build_sample(project, user) } -- cgit v1.2.3