| Age | Commit message (Collapse) | Author |
|---|
|
Fix typos found by typos-cli(https://github.com/crate-ci/typos). Some
affected tests are adjusted.
There are a bunch of other typos are ignored, including
* CHANGELOG.md
* NOTICE
* internal/.../migrations/20201208163237_cleanup_notifications_payload.go
* other intended typos or false positives
Signed-off-by: Xing Xin <xingxin.xx@bytedance.com>
|
|
The `check` subcommand has been relocated from `gitaly-hooks` to the
main `gitaly` binary. References to the subcommand were updated to
reflect this change.
|
|
Previously attempting to destroy a cluster would result in the following errors:
```
Error: Error, failed to deleteuser praefect in instance <instance name>
```
```
Error: Error, failed to delete instance because deletion_protection
is set to true. Set it to false to proceed with instance deletion
```
For the `deleteuser` error,
https://github.com/hashicorp/terraform-provider-google/issues/7677 has
added the `ABANDON` flag, which according to
https://github.com/GoogleCloudPlatform/magic-modules/pull/4231:
```
Setting ABANDON allows the resource to be abandoned rather than
deleted. This is useful for Postgres, where users cannot be deleted from
the API if they have been granted SQL roles.
```
For the `delete_protection` error, as described in
https://stackoverflow.com/a/64911997, recent versions of Terraform sets
`delete_protection` on by default. We need to disable this to be able to
destroy this Cloud SQL instance.
Changelog: fixed
|
|
demo: Update Cloud SQL database to PostgreSQL 12
See merge request gitlab-org/gitaly!4074
|
|
This commit will cause PostgreSQL to log any database statements taking
longer than 500 ms. This is useful to flag any slow queries running in
Praefect.
|
|
We are shipping and using PostgreSQL 12 on GitLab.com, so let's use this
version to match production.
|
|
With introduction of database notifications listener we need
to configure direct connection between praefect and postgres
as PgBouncer can't serve LISTEN operation with transaction
scoped configuration.
The change adds praefect instances IPs to the list of allowed IPs
of the Postgres database. So each praefect is allowed to connect
directly to it.
Closes: https://gitlab.com/gitlab-org/gitaly/-/issues/3354
|
|
The terraform install uses `bin/check` from the vendorred `gitlab-shell`
code to check the hooks are configured correctly. By changing this to
`gitaly-hooks` we can prepare for a later situation where `gitlab-shell`
is removed and as such not available.
Currently the `gitaly-hooks check` command executes `gitlab-shell` under
the hood, so the results are identical.
|
|
After having deployed the cluster, the first connection to deployed
machines is currently going to fail because of unknown host keys. Let's
improve this situation by scanning deployed hosts and adding their keys
to the known_hosts file automatically.
|
|
With cluster deployment now being done via Ansible, let's change the
README to reflect this change.
|
|
Right now, creation of the demo cluster is performed via a set of Ruby
scripts. Let's convert them to use Ansible, too, so tasks become
idempotent and we have less of a mixture between Ansible and custom Ruby
logic.
|
|
We're about to introduce a second playbook for cluster creation, so
let's rename the generic "playbook.yml" to "configure.yml".
|
|
Both `destroy-demo-cluster` and `print-info` are simple wrappers around
terraform. Using Ruby for that is overkill.
Let's convert both scripts to shell onliners.
|
|
Setup the omnibus gitaly as a storage "internal"
|
|
Starting with commit 36437f18 (PgBouncer deployment with terraform,
2020-07-29), we've introduced a new dependency on the pgbouncer module
for Terraform. To download the module, the commit in question added a
call to `terraform get`, which causes us to download the module. But as
we're only executing `terraform init` in case no `.terraform` directory
exists and `terraform get` will unconditionally create that directory,
the result is that `terraform init` will never be executed.
Fix the issue by just removing the call to `terraform get`.
Initialization will fetch the pgbouncer module anyway, so there's no
need to do it explicitly.
|
|
Right now, our Terraform scripts only handle creation of machines in
GCP, but not their respective configuration. As it's a task we're doing
rather frequently which takes some time, this commit creates an Ansible
playbook which automates this task.
On creation of the cluster, our scripts now automatically generate a
`hosts.ini` file containing all necessary connection information. With
this file, the admin may now run `./configure-demo-cluster`, which will
invoke Ansible and automatically generate and apply configuration for
each of the nodes, restarting services as required. The task can be run
repeatedly in order to update configuration on the target nodes.
|
|
PgBouncer deployment with terraform
Closes #2975
See merge request gitlab-org/gitaly!2418
|
|
Resulting name of the IP for the SQL connection renamed
to praefect_pgbouncer_ip.
Closes: https://gitlab.com/gitlab-org/gitaly/-/issues/2975
|
|
Assignment of the dedicated IP address to PgBouncer instance
in order to narrow the set of public IP addresses allowed to
connect to PostgreSQL instance.
Closes: https://gitlab.com/gitlab-org/gitaly/-/issues/2975
|
|
Terraform requires a new module to be installed: pgbouncer
In order to install all required modules automatically
the `terraform get` command added to main script.
The command installs all missed dependencies recursively.
Closes: https://gitlab.com/gitlab-org/gitaly/-/issues/2975
|
|
As the target PostgeSQL version currently is 11 we should
use this version for tests and demo.
|
|
In order to verify usage of PgBouncer in front of Postgres
database PgBouncer included into terraform deployment.
It uses separate machine with internal IP that is accessible
to Praefect instances.
Cloud SQL authorized networks changed to '0.0.0.0/0' because
it is not possible to use PgBouncer IP for it, as PgBouncer
requires IP of Cloud SQL instance in setup (circular dependency).
The output of 'praefect_postgresql_ip' is a private IP of the
PgBouncer instance that should be used instead of a public Cloud
SQL instance to proxy SQL requests.
Closes: https://gitlab.com/gitlab-org/gitaly/-/issues/2975
|
|
|
|
Add load balancer and multiple Praefect nodes
Closes #2639
See merge request gitlab-org/gitaly!2098
|
|
In 86e90ec2711025a69ab3bb696783de324fdf9b51, support was added for PG
provisioning through terraform. The readme did still include the manual
steps however.
|
|
Praefect now stores state in SQL so that multiple nodes can be run
behind a load balancer. This updates the Terraform configuration to
reflect this.
|
|
This will help test multiple Praefect nodes to test SQL leader
elections. Currently `count` is set to 1; it can be increased
to 3 or more.
|
|
|
|
|
|
The based image default is 10GB which is insufficient for a single
large repository like www-gitlab-com to be repacked, or even moved to a
different shard since the host operating takes up space too.
The gitlab and gitaly-n nodes are now set to 250GB to provide
sufficient space to move a large repo from the default storage after
doing a simple Omnibus install to the HA Gitaly cluster.
|
|
|
