Welcome to mirror list, hosted at ThFree Co, Russian Federation.

gitlab.com/gitlab-org/gitlab-foss.git - Unnamed repository; edit this file 'description' to name the repository.
summaryrefslogtreecommitdiff
diff options
context:
space:
mode:
Diffstat (limited to 'doc/development/database/batched_background_migrations.md')
-rw-r--r--doc/development/database/batched_background_migrations.md39
1 files changed, 27 insertions, 12 deletions
diff --git a/doc/development/database/batched_background_migrations.md b/doc/development/database/batched_background_migrations.md
index 3a0fa77eff9..6d3d5fa7f92 100644
--- a/doc/development/database/batched_background_migrations.md
+++ b/doc/development/database/batched_background_migrations.md
@@ -1,6 +1,6 @@
---
type: reference, dev
-stage: Enablement
+stage: Data Stores
group: Database
info: "See the Technical Writers assigned to Development Guidelines: https://about.gitlab.com/handbook/engineering/ux/technical-writing/#assignments-to-development-guidelines"
---
@@ -152,9 +152,7 @@ When you start the second post-deployment migration, delete the
previously batched migration with the provided code:
```ruby
-Gitlab::Database::BackgroundMigration::BatchedMigration
- .for_configuration(MIGRATION_NAME, TABLE_NAME, COLUMN, JOB_ARGUMENTS)
- .delete_all
+delete_batched_background_migration(MIGRATION_NAME, TABLE_NAME, COLUMN, JOB_ARGUMENTS)
```
## Cleaning up
@@ -192,7 +190,7 @@ data to be in the new format.
The `routes` table has a `source_type` field that's used for a polymorphic relationship.
As part of a database redesign, we're removing the polymorphic relationship. One step of
-the work will be migrating data from the `source_id` column into a new singular foreign key.
+the work is migrating data from the `source_id` column into a new singular foreign key.
Because we intend to delete old rows later, there's no need to update them as part of the
background migration.
@@ -221,9 +219,9 @@ background migration.
NOTE:
Job classes must be subclasses of `BatchedMigrationJob` to be
correctly handled by the batched migration framework. Any subclass of
- `BatchedMigrationJob` will be initialized with necessary arguments to
+ `BatchedMigrationJob` is initialized with necessary arguments to
execute the batch, as well as a connection to the tracking database.
- Additional `job_arguments` set on the migration will be passed to the
+ Additional `job_arguments` set on the migration are passed to the
job's `perform` method.
1. Add a new trigger to the database to update newly created and updated routes,
@@ -245,12 +243,14 @@ background migration.
1. Create a post-deployment migration that queues the migration for existing data:
```ruby
- class QueueBackfillRoutesNamespaceId < Gitlab::Database::Migration[1.0]
+ class QueueBackfillRoutesNamespaceId < Gitlab::Database::Migration[2.0]
disable_ddl_transaction!
MIGRATION = 'BackfillRouteNamespaceId'
DELAY_INTERVAL = 2.minutes
+ restrict_gitlab_migration gitlab_schema: :gitlab_main
+
def up
queue_batched_background_migration(
MIGRATION,
@@ -261,12 +261,19 @@ background migration.
end
def down
- Gitlab::Database::BackgroundMigration::BatchedMigration
- .for_configuration(MIGRATION, :routes, :id, []).delete_all
+ delete_batched_background_migration(MIGRATION, :routes, :id, [])
end
end
```
+ NOTE:
+ When queuing a batched background migration, you need to restrict
+ the schema to the database where you make the actual changes.
+ In this case, we are updating `routes` records, so we set
+ `restrict_gitlab_migration gitlab_schema: :gitlab_main`. If, however,
+ you need to perform a CI data migration, you would set
+ `restrict_gitlab_migration gitlab_schema: :gitlab_ci`.
+
After deployment, our application:
- Continues using the data as before.
- Ensures that both existing and new data are migrated.
@@ -275,16 +282,19 @@ background migration.
that checks that the batched background migration is completed. For example:
```ruby
- class FinalizeBackfillRouteNamespaceId < Gitlab::Database::Migration[1.0]
+ class FinalizeBackfillRouteNamespaceId < Gitlab::Database::Migration[2.0]
MIGRATION = 'BackfillRouteNamespaceId'
disable_ddl_transaction!
+ restrict_gitlab_migration gitlab_schema: :gitlab_main
+
def up
ensure_batched_background_migration_is_finished(
job_class_name: MIGRATION,
table_name: :routes,
column_name: :id,
- job_arguments: []
+ job_arguments: [],
+ finalize: true
)
end
@@ -294,6 +304,11 @@ background migration.
end
```
+ NOTE:
+ If the batched background migration is not finished, the system will
+ execute the batched background migration inline. If you don't want
+ to see this behavior, you need to pass `finalize: false`.
+
If the application does not depend on the data being 100% migrated (for
instance, the data is advisory, and not mission-critical), then you can skip this
final step. This step confirms that the migration is completed, and all of the rows were migrated.