diff options
Diffstat (limited to 'doc/development/database')
4 files changed, 23 insertions, 23 deletions
diff --git a/doc/development/database/efficient_in_operator_queries.md b/doc/development/database/efficient_in_operator_queries.md index 4f18c937198..ef6ef232c9e 100644 --- a/doc/development/database/efficient_in_operator_queries.md +++ b/doc/development/database/efficient_in_operator_queries.md @@ -26,7 +26,7 @@ Pagination may be used to fetch subsequent records. Example tasks requiring querying nested domain objects from the group level: - Show first 20 issues by creation date or due date from the group `gitlab-org`. -- Show first 20 merge_requests by merged at date from the group `gitlab-com`. +- Show first 20 merge requests by merged at date from the group `gitlab-com`. Unfortunately, ordered group-level queries typically perform badly as their executions require heavy I/O, memory, and computations. @@ -877,7 +877,7 @@ this cursor would be (`2020-01-05`, `3`) for `project_id=9`. ### Initializing the recursive CTE query -For the initial recursive query, we'll need to produce exactly one row, we call this the +For the initial recursive query, we need to produce exactly one row, we call this the initializer query (`initializer_query`). Use `ARRAY_AGG` function to compact the initial result set into a single row diff --git a/doc/development/database/loose_foreign_keys.md b/doc/development/database/loose_foreign_keys.md index 6889b9123ca..1a6d995e78c 100644 --- a/doc/development/database/loose_foreign_keys.md +++ b/doc/development/database/loose_foreign_keys.md @@ -515,13 +515,13 @@ referenced child tables. ### Database structure The feature relies on triggers installed on the parent tables. When a parent record is deleted, -the trigger will automatically insert a new record into the `loose_foreign_keys_deleted_records` +the trigger automatically inserts a new record into the `loose_foreign_keys_deleted_records` database table. -The inserted record will store the following information about the deleted record: +The inserted record stores the following information about the deleted record: - `fully_qualified_table_name`: name of the database table where the record was located. -- `primary_key_value`: the ID of the record, the value will be present in the child tables as +- `primary_key_value`: the ID of the record, the value is present in the child tables as the foreign key value. At the moment, composite primary keys are not supported, the parent table must have an `id` column. - `status`: defaults to pending, represents the status of the cleanup process. @@ -532,7 +532,7 @@ several runs. #### Database decomposition -The `loose_foreign_keys_deleted_records` table will exist on both database servers (Ci and Main) +The `loose_foreign_keys_deleted_records` table exists on both database servers (`ci` and `main`) after the [database decomposition](https://gitlab.com/groups/gitlab-org/-/epics/6168). The worker ill determine which parent tables belong to which database by reading the `lib/gitlab/database/gitlab_schemas.yml` YAML file. @@ -547,10 +547,10 @@ Example: - `ci_builds` - `ci_pipelines` -When the worker is invoked for the Ci database, the worker will load deleted records only from the +When the worker is invoked for the `ci` database, the worker loads deleted records only from the `ci_builds` and `ci_pipelines` tables. During the cleanup process, `DELETE` and `UPDATE` queries -will mostly run on tables located in the Main database. In this example, one `UPDATE` query will -nullify the `merge_requests.head_pipeline_id` column. +mostly run on tables located in the Main database. In this example, one `UPDATE` query +nullifies the `merge_requests.head_pipeline_id` column. #### Database partitioning @@ -561,7 +561,7 @@ strategy was considered for the feature but due to the large data volume we deci new strategy. A deleted record is considered fully processed when all its direct children records have been -cleaned up. When this happens, the loose foreign key worker will update the `status` column of +cleaned up. When this happens, the loose foreign key worker updates the `status` column of the deleted record. After this step, the record is no longer needed. The sliding partitioning strategy provides an efficient way of cleaning up old, unused data by @@ -591,7 +591,7 @@ Partitions: gitlab_partitions_dynamic.loose_foreign_keys_deleted_records_84 FOR ``` The `partition` column controls the insert direction, the `partition` value determines which -partition will get the deleted rows inserted via the trigger. Notice that the default value of +partition gets the deleted rows inserted via the trigger. Notice that the default value of the `partition` table matches with the value of the list partition (84). In `INSERT` query within the trigger the value of the `partition` is omitted, the trigger always relies on the default value of the column. @@ -709,12 +709,12 @@ To mitigate these issues, several limits are applied when the worker runs. The limit rules are implemented in the `LooseForeignKeys::ModificationTracker` class. When one of the limits (record modification count, time limit) is reached the processing is stopped -immediately. After some time, the next scheduled worker will continue the cleanup process. +immediately. After some time, the next scheduled worker continues the cleanup process. #### Performance characteristics The database trigger on the parent tables will **decrease** the record deletion speed. Each -statement that removes rows from the parent table will invoke the trigger to insert records +statement that removes rows from the parent table invokes the trigger to insert records into the `loose_foreign_keys_deleted_records` table. The queries within the cleanup worker are fairly efficient index scans, with limits in place diff --git a/doc/development/database/migrations_for_multiple_databases.md b/doc/development/database/migrations_for_multiple_databases.md index 2500071f4cf..df9607f5672 100644 --- a/doc/development/database/migrations_for_multiple_databases.md +++ b/doc/development/database/migrations_for_multiple_databases.md @@ -260,7 +260,7 @@ the `database_tasks: false` set. `gitlab:db:validate_config` always runs before ## Validation -Validation in a nutshell uses [pg_query](https://github.com/pganalyze/pg_query) to analyze +Validation in a nutshell uses [`pg_query`](https://github.com/pganalyze/pg_query) to analyze each query and classify tables with information from [`gitlab_schema.yml`](multiple_databases.md#gitlab-schema). The migration is skipped if the specified `gitlab_schema` is outside of a list of schemas managed by a given database connection (`Gitlab::Database::gitlab_schemas_for_connection`). @@ -427,7 +427,7 @@ updating all `ci_pipelines`, you would set As with all DML migrations, you cannot query another database outside of `restrict_gitlab_migration` or `gitlab_shared`. If you need to query another database, -you'll likely need to separate these into two migrations somehow. +separate the migrations. Because the actual migration logic (not the queueing step) for background migrations runs in a Sidekiq worker, the logic can perform DML queries on diff --git a/doc/development/database/strings_and_the_text_data_type.md b/doc/development/database/strings_and_the_text_data_type.md index d764e54aa76..73e023f8d45 100644 --- a/doc/development/database/strings_and_the_text_data_type.md +++ b/doc/development/database/strings_and_the_text_data_type.md @@ -8,7 +8,7 @@ info: To determine the technical writer assigned to the Stage/Group associated w > [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/30453) in GitLab 13.0. -When adding new columns that will be used to store strings or other textual information: +When adding new columns to store strings or other textual information: 1. We always use the `text` data type instead of the `string` data type. 1. `text` columns should always have a limit set, either by using the `create_table` with @@ -142,8 +142,8 @@ instance of GitLab could have such records, so we would follow the same process We first add the limit as a `NOT VALID` check constraint to the table, which enforces consistency when new records are inserted or current records are updated. -In the example above, the existing issues with more than 1024 characters in their title will not be -affected and you'll be still able to update records in the `issues` table. However, when you'd try +In the example above, the existing issues with more than 1024 characters in their title are not +affected, and you are still able to update records in the `issues` table. However, when you'd try to update the `title_html` with a title that has more than 1024 characters, the constraint causes a database error. @@ -182,7 +182,7 @@ end #### Data migration to fix existing records (current release) The approach here depends on the data volume and the cleanup strategy. The number of records that must -be fixed on GitLab.com is a nice indicator that will help us decide whether to use a post-deployment +be fixed on GitLab.com is a nice indicator that helps us decide whether to use a post-deployment migration or a background data migration: - If the data volume is less than `1,000` records, then the data migration can be executed within the post-migration. @@ -233,7 +233,7 @@ You can find more information on the guide about [background migrations](backgro #### Validate the text limit (next release) -Validating the text limit will scan the whole table and make sure that each record is correct. +Validating the text limit scans the whole table, and makes sure that each record is correct. Still in our example, for the 13.1 milestone (next), we run the `validate_text_limit` migration helper in a final post-deployment migration, @@ -276,11 +276,11 @@ end ## Text limit constraints on large tables If you have to clean up a text column for a really [large table](https://gitlab.com/gitlab-org/gitlab/-/blob/master/rubocop/rubocop-migrations.yml#L3) -(for example, the `artifacts` in `ci_builds`), your background migration will go on for a while and -it will need an additional [background migration cleaning up](background_migrations.md#cleaning-up) +(for example, the `artifacts` in `ci_builds`), your background migration goes on for a while and +it needs an additional [background migration cleaning up](background_migrations.md#cleaning-up) in the release after adding the data migration. -In that rare case you will need 3 releases end-to-end: +In that rare case you need 3 releases end-to-end: 1. Release `N.M` - Add the text limit and the background migration to fix the existing records. 1. Release `N.M+1` - Cleanup the background migration. |