diff options
Diffstat (limited to 'doc/development/database/required_stops.md')
-rw-r--r-- | doc/development/database/required_stops.md | 56 |
1 files changed, 56 insertions, 0 deletions
diff --git a/doc/development/database/required_stops.md b/doc/development/database/required_stops.md index 46fabb5c1b4..e4f66f4424f 100644 --- a/doc/development/database/required_stops.md +++ b/doc/development/database/required_stops.md @@ -11,6 +11,62 @@ disruptive effect on customers. Before adding a required stop, consider if any alternative approaches exist to avoid a required stop. Sometimes a required stop is unavoidable. In those cases, follow the instructions below. +## Common scenarios that require stops + +### Long running migrations being finalized + +If a migration takes a long time, it could cause a large number of customers to encounter timeouts +during upgrades. The increased support volume may cause us to introduce a required stop. While any +background migration may cause these issues with particularly large customers, we typically only +introduce stops when the impact is widespread. + +- **Cause:** When an upgrade takes more than an hour, omnibus times out. +- **Mitigation:** Schedule finalization for the first minor version after the next required stop. + +### Improperly finalized background migrations + +You may need to introduce a required stop for mitigation when: + +- A background migration is not finalized, and +- A migration is written that depends on that background migration. + +- **Cause:** The dependent migration may fail if the background migration is incomplete. +- **Mitigation:** Ensure that all background migrations are finalized before authoring dependent migrations. + +### Remove a migration + +If a migration is removed, you may need to introduce a required stop to ensure customers +don't miss the required change. + +- **Cause:** Dependent migrations may fail, or the application may not function, because a required + migration was removed. +- **Mitigation:** Ensure migrations are only removed after they've been a part of a planned + required stop. + +### A migration timestamp is very old + +If a migration timestamp is very old (> 3 weeks, or after a before the last stop), +these scenarios may cause issues: + +- If the migration depends on another migration with a newer timestamp but introduced in a + previous release _after_ a required stop, then the new migration may run sequentially sooner + than the prerequisite migration, and thus fail. +- If the migration timestamp ID is before the last, it may be inadvertently squashed when the + team squashes other migrations from the required stop. + +- **Cause:** The migration may fail if it depends on a migration with a later timestamp introduced + in an earlier version. Or, the migration may be inadvertently squashed after a required stop. +- **Mitigation:** Aim for migration timestamps to fall inside the release dates and be sure that + they are not dated prior to the last required stop. + +### Bugs in migration related tooling + +In a few circumstances, bugs in migration related tooling has required us to introduce stops. While we aim +to prevent these in testing, sometimes they happen. + +- **Cause:** There have been a few different causes where we recognized these too late. +- **Mitigation:** Typically we try to backport fixes for migrations, but in some cases this is not possible. + ## Before the required stop is released Before releasing a known required stop, complete these steps. If the required stop |