Fixed pagination of web hook logs

For reasons unknown, the logs of a web hook were paginated in memory.
This would result in the "Edit" page of a web hook timing out once it
has more than a few thousand log entries.

This commit makes the following changes:

1. We use LIMIT/OFFSET to paginate the data, instead of doing this in
   memory.

2. We limit the logs to the last two days, just like the documentation
   says (instead of retrieving everything).

3. We change the indexes on "web_hook_logs" so the query to get the data
   can perform a backwards index scan, without the need for a Filter.

These changes combined ensure that Projects::HooksController#edit no
longer times out.

2 files changed, 30 insertions, 1 deletions

diff --git a/db/migrate/20180628124813_alter_web_hook_logs_indexes.rb b/db/migrate/20180628124813_alter_web_hook_logs_indexes.rb
new file mode 100644
index 00000000000..1878e76811d
--- /dev/null
+++ b/db/migrate/20180628124813_alter_web_hook_logs_indexes.rb
@@ -0,0 +1,28 @@
+# frozen_string_literal: true
+
+# See http://doc.gitlab.com/ce/development/migration_style_guide.html
+# for more information on how to write migrations for GitLab.
+
+class AlterWebHookLogsIndexes < ActiveRecord::Migration
+  include Gitlab::Database::MigrationHelpers
+
+  # Set this constant to true if this migration requires downtime.
+  DOWNTIME = false
+
+  disable_ddl_transaction!
+
+  # "created_at" comes first so the Sidekiq worker pruning old webhook logs can
+  # use a composite index index.
+  #
+  # We leave the old standalone index on "web_hook_id" in place so future code
+  # that doesn't care about "created_at" can still use that index.
+  COLUMNS_TO_INDEX = %i[created_at web_hook_id]
+
+  def up
+    add_concurrent_index(:web_hook_logs, COLUMNS_TO_INDEX)
+  end
+
+  def down
+    remove_concurrent_index(:web_hook_logs, COLUMNS_TO_INDEX)
+  end
+end
diff --git a/db/schema.rb b/db/schema.rb
index 0112fc726d4..38c1710d73c 100644
--- a/db/schema.rb
+++ b/db/schema.rb
@@ -11,7 +11,7 @@
 #
 # It's strongly recommended that you check this file into your version control system.
 
-ActiveRecord::Schema.define(version: 20180626125654) do
+ActiveRecord::Schema.define(version: 20180628124813) do
 
   # These are extensions that must be enabled in order to support this database
   enable_extension "plpgsql"
@@ -2144,6 +2144,7 @@ ActiveRecord::Schema.define(version: 20180626125654) do
     t.datetime "updated_at", null: false
   end
 
+  add_index "web_hook_logs", ["created_at", "web_hook_id"], name: "index_web_hook_logs_on_created_at_and_web_hook_id", using: :btree
   add_index "web_hook_logs", ["web_hook_id"], name: "index_web_hook_logs_on_web_hook_id", using: :btree
 
   create_table "web_hooks", force: :cascade do |t|