Introduction of in-memory cache for reads distribution

With enabled distributed_reads feature each read operation leads to
a database query execution to get state of the storages for particular
repository. More read calls leads to more database access operations,
so the pressure to it increases in linear (or even worse).
To mitigate this problem it was decided to introduce an in-memory cache
added before accessing the database. Invalidation happens on receiving
notification events from the database. The events are send by the
triggers attached to the repositories (delete) and storage_repositories
(insert, delete, update) tables.

To monitor the cache a new counter was added: gitaly_praefect_uptodate_storages_cache_access_total.
It tracks amount of cache hits, misses and populates and evicts per virtual
repository. And to track an error rate of the notifications processing the
gitaly_praefect_uptodate_storages_errors_total was added with type set to one of:
retrieve, notification_decode.

Closes: https://gitlab.com/gitlab-org/gitaly/-/issues/3053

2 files changed, 21 insertions, 0 deletions

diff --git a/internal/praefect/config/config.go b/internal/praefect/config/config.go
index ebca60f43..a4564ea58 100644
--- a/internal/praefect/config/config.go
+++ b/internal/praefect/config/config.go
@@ -298,6 +298,9 @@ func (db DB) ToPQString(direct bool) string {
 	var portVal int
 
 	if direct {
+		if db.HostNoProxy == "" || db.PortNoProxy == 0 {
+			return ""
+		}
 		hostVal = db.HostNoProxy
 		portVal = db.PortNoProxy
 	} else {
diff --git a/internal/praefect/config/config_test.go b/internal/praefect/config/config_test.go
index 862dea798..9bcb23c4f 100644
--- a/internal/praefect/config/config_test.go
+++ b/internal/praefect/config/config_test.go
@@ -392,6 +392,24 @@ func TestToPQString(t *testing.T) {
 			out:    `port=2345 host=1.2.3.4 user=praefect-user password=secret dbname=praefect_production sslmode=require sslcert=/path/to/cert sslkey=/path/to/key sslrootcert=/path/to/root-cert binary_parameters=yes`,
 		},
 		{
+			desc: "direct connection host not set",
+			in: DB{
+				HostNoProxy: "",
+				PortNoProxy: 2345,
+			},
+			direct: true,
+			out:    "",
+		},
+		{
+			desc: "direct connection port not set",
+			in: DB{
+				HostNoProxy: "localhost",
+				PortNoProxy: 0,
+			},
+			direct: true,
+			out:    "",
+		},
+		{
 			desc: "with spaces and quotes",
 			in: DB{
 				Password: "secret foo'bar",