Welcome to mirror list, hosted at ThFree Co, Russian Federation.

gitlab.com/gitlab-org/gitlab-foss.git - Unnamed repository; edit this file 'description' to name the repository.
summaryrefslogtreecommitdiff
diff options
context:
space:
mode:
Diffstat (limited to 'doc/development/search/advanced_search_migration_styleguide.md')
-rw-r--r--doc/development/search/advanced_search_migration_styleguide.md86
1 files changed, 82 insertions, 4 deletions
diff --git a/doc/development/search/advanced_search_migration_styleguide.md b/doc/development/search/advanced_search_migration_styleguide.md
index 2f8cd036dcf..d7c0dddee7b 100644
--- a/doc/development/search/advanced_search_migration_styleguide.md
+++ b/doc/development/search/advanced_search_migration_styleguide.md
@@ -54,7 +54,7 @@ The following migration helpers are available in `ee/app/workers/concerns/elasti
Backfills a specific field in an index. In most cases, the mapping for the field should already be added.
-Requires the `index_name` and `field_name` methods.
+Requires the `index_name` and `field_name` methods to backfill a single field.
```ruby
class MigrationName < Elastic::Migration
@@ -72,6 +72,24 @@ class MigrationName < Elastic::Migration
end
```
+Requires the `index_name` and `field_names` methods to backfill multiple fields if any field is null.
+
+```ruby
+class MigrationName < Elastic::Migration
+ include Elastic::MigrationBackfillHelper
+
+ private
+
+ def index_name
+ Issue.__elasticsearch__.index_name
+ end
+
+ def field_names
+ %w[schema_version visibility_level]
+ end
+end
+```
+
#### `Elastic::MigrationUpdateMappingsHelper`
Updates a mapping in an index by calling `put_mapping` with the mapping specified.
@@ -152,6 +170,34 @@ class MigrationName < Elastic::Migration
end
```
+#### `Elastic::MigrationCreateIndex`
+
+Creates a new index.
+
+Requires:
+
+- The `target_class` and `document_type` methods
+- Mappings and index settings for the class in `ee/lib/elastic/latest/` and `ee/lib/elastic/v12p1/`
+
+WARNING:
+You must perform a follow-up migration to populate the index in the same milestone.
+
+```ruby
+class MigrationName < Elastic::Migration
+ include Elastic::MigrationCreateIndex
+
+ retry_on_failure
+
+ def document_type
+ :epic
+ end
+
+ def target_class
+ Epic
+ end
+end
+```
+
#### `Elastic::MigrationHelper`
Contains methods you can use when a migration doesn't fit the previous examples.
@@ -215,9 +261,22 @@ class BatchedMigrationName < Elastic::Migration
end
```
+## Avoiding downtime in migrations
+
+### Reverting a migration
+
+If a migration fails or is halted on GitLab.com, we prefer to revert the change that introduced the migration. This
+prevents self-managed customers from receiving a broken migration and reduces the need for backports.
+
+### When to merge
+
+We prefer not to merge migrations within 1 week of the release. This allows time for a revert if a migration fails or
+doesn't work as expected. Migrations still in development or review during the final week of the release should be pushed
+to the next milestone.
+
### Multi-version compatibility
-These advanced search migrations, like any other GitLab changes, need to support the case where
+Advanced search migrations, like any other GitLab changes, need to support the case where
[multiple versions of the application are running at the same time](../multi_version_compatibility.md).
Depending on the order of deployment, it's possible that the migration
@@ -225,7 +284,7 @@ has started or finished and there's still a server running the application code
migration. We need to take this into consideration until we can
[ensure all advanced search migrations start after the deployment has finished](https://gitlab.com/gitlab-org/gitlab/-/issues/321619).
-### Reverting a migration
+### High risk migrations
Because Elasticsearch does not support transactions, we always need to design our
migrations to accommodate a situation where the application
@@ -236,7 +295,26 @@ some data is moved) to a later merge request after the migrations have
completed successfully. To be safe, for self-managed customers we should also
defer it to another release if there is risk of important data loss.
-### Best practices for advanced search migrations
+## Calculating migration runtime
+
+It's important to understand how long a migration might take to run on GitLab.com. Derive the number of documents that
+will be processed by the migration. This number may come from querying the database or an existing Elasticsearch index.
+Use the following formula to calculate the runtime:
+
+```ruby
+> batch_size = 9_000
+=> 9000
+> throttle_delay = 1.minute
+=> 1 minute
+> number_of_documents = 15_536_906
+=> 15536906
+> (number_of_documents / batch_size) * throttle_delay
+=> 1726 minutes
+> (number_of_documents / batch_size) * throttle_delay / 1.hour
+=> 28
+```
+
+## Best practices for advanced search migrations
Follow these best practices for best results: