Memoize GitLab logger to reduce open file descriptors

We see that in gitlab-org/gitlab-ee#3664 that if we log a lot of
data in Sidekiq workers, the number of open file descriptors
reaches over 1000. To avoid this, we can memoize the logger per
thread via RequestStore.

Closes gitlab-org/gitlab-ee#3664

1 files changed, 10 insertions, 2 deletions

diff --git a/lib/gitlab/logger.rb b/lib/gitlab/logger.rb
index 6bffd410ed0..a42e312b5d3 100644
--- a/lib/gitlab/logger.rb
+++ b/lib/gitlab/logger.rb
@@ -13,7 +13,7 @@ module Gitlab
     end
 
     def self.read_latest
-      path = Rails.root.join("log", file_name)
+      path = self.full_log_path
 
       return [] unless File.readable?(path)
 
@@ -22,7 +22,15 @@ module Gitlab
     end
 
     def self.build
-      new(Rails.root.join("log", file_name))
+      RequestStore[self.cache_key] ||= new(self.full_log_path)
+    end
+
+    def self.full_log_path
+      Rails.root.join("log", file_name)
+    end
+
+    def self.cache_key
+      'logger:'.freeze + self.full_log_path.to_s
     end
   end
 end