1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
|
---
stage: Data Stores
group: Database
info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments
---
# Adding required stops
[Required stops](../../update/index.md#upgrade-paths) should only be added when it is deemed absolutely necessary, because of their
disruptive effect on customers. Before adding a required stop, consider if any
alternative approaches exist to avoid a required stop. Sometimes a required
stop is unavoidable. In those cases, follow the instructions below.
## Common scenarios that require stops
### Long running migrations being finalized
If a migration takes a long time, it could cause a large number of customers to encounter timeouts
during upgrades. The increased support volume may cause us to introduce a required stop. While any
background migration may cause these issues with particularly large customers, we typically only
introduce stops when the impact is widespread.
- **Cause:** When an upgrade takes more than an hour, omnibus times out.
- **Mitigation:** Schedule finalization for the first minor version after the next required stop.
### Improperly finalized background migrations
You may need to introduce a required stop for mitigation when:
- A background migration is not finalized, and
- A migration is written that depends on that background migration.
- **Cause:** The dependent migration may fail if the background migration is incomplete.
- **Mitigation:** Ensure that all background migrations are finalized before authoring dependent migrations.
### Remove a migration
If a migration is removed, you may need to introduce a required stop to ensure customers
don't miss the required change.
- **Cause:** Dependent migrations may fail, or the application may not function, because a required
migration was removed.
- **Mitigation:** Ensure migrations are only removed after they've been a part of a planned
required stop.
### A migration timestamp is very old
If a migration timestamp is very old (> 3 weeks, or after a before the last stop),
these scenarios may cause issues:
- If the migration depends on another migration with a newer timestamp but introduced in a
previous release _after_ a required stop, then the new migration may run sequentially sooner
than the prerequisite migration, and thus fail.
- If the migration timestamp ID is before the last, it may be inadvertently squashed when the
team squashes other migrations from the required stop.
- **Cause:** The migration may fail if it depends on a migration with a later timestamp introduced
in an earlier version. Or, the migration may be inadvertently squashed after a required stop.
- **Mitigation:** Aim for migration timestamps to fall inside the release dates and be sure that
they are not dated prior to the last required stop.
### Bugs in migration related tooling
In a few circumstances, bugs in migration related tooling has required us to introduce stops. While we aim
to prevent these in testing, sometimes they happen.
- **Cause:** There have been a few different causes where we recognized these too late.
- **Mitigation:** Typically we try to backport fixes for migrations, but in some cases this is not possible.
## Before the required stop is released
Before releasing a known required stop, complete these steps. If the required stop
is identified after release, the following steps must still be completed:
1. Update [upgrade paths](../../update/index.md#upgrade-paths) to include the new
required stop.
1. Communicate the changes with the customer Support and Release management teams.
1. File an issue with the Database group to squash migrations to that version in the
next release. Use this template for your issue:
```markdown
Title: `Squash migrations to <Required stop version>`
As a result of the required stop added for <required stop version> we should squash
migrations up to that version, and update the minimum schema version.
Deliverables:
- [ ] Migrations are squashed up to <required stop version>
- [ ] `Gitlab::Database::MIN_SCHEMA_VERSION` matches init_schema version
/label ~"group::database" ~"section::enablement" ~"devops::data_stores" ~"Category:Database" ~"type::maintenance"
/cc @gitlab-org/database-team/triage
```
## In the release following the required stop
1. Update `Gitlab::Database::MIN_SCHEMA_GITLAB_VERSION` in `lib/gitlab/database.rb` to the
new required stop versions. Do not change `Gitlab::Database::MIN_SCHEMA_VERSION`.
1. In the `charts` project, update the
[upgrade check hook](https://gitlab.com/gitlab-org/charts/gitlab/-/blame/master/templates/_runcheck.tpl#L32)
to the required stop version.
1. In the `omnibus-gitlab` project, update the
[pre-install version check](https://gitlab.com/gitlab-org/omnibus-gitlab/-/blob/master/config/templates/package-scripts/preinst.erb#L43)
to the required stop version.
|
