Avoid process deadlock in popen by consuming input pipes

A process that spews a lot of output to stderr or stdout could stall out
due to the pipe buffer being full. As described in https://bugs.ruby-lang.org/issues/9082,
we can use the trick used in Ruby's capture3 function to read the pipes in separate
threads.

Closes https://gitlab.com/gitlab-org/gitlab-ee/issues/6895

1 files changed, 13 insertions, 0 deletions

diff --git a/spec/lib/gitlab/popen_spec.rb b/spec/lib/gitlab/popen_spec.rb
index 1dbead16d5b..c1b84e9f077 100644
--- a/spec/lib/gitlab/popen_spec.rb
+++ b/spec/lib/gitlab/popen_spec.rb
@@ -55,6 +55,19 @@ describe Gitlab::Popen do
     end
   end
 
+  context 'with a process that writes a lot of data to stderr' do
+    let(:test_string) { 'The quick brown fox jumped over the lazy dog' }
+    # The pipe buffer is typically 64K. This string is about 440K.
+    let(:spew_command) { ['bash', '-c', "for i in {1..10000}; do echo '#{test_string}' 1>&2; done"] }
+
+    it 'returns zero' do
+      output, status = @klass.new.popen(spew_command, path)
+
+      expect(output).to include(test_string)
+      expect(status).to eq(0)
+    end
+  end
+
   context 'without a directory argument' do
     before do
       @output, @status = @klass.new.popen(%w(ls))