Add latest changes from gitlab-org/gitlab@master

6 files changed, 153 insertions, 46 deletions

diff --git a/doc/administration/job_artifacts.md b/doc/administration/job_artifacts.md
index bec1fa6ad05..5c99a4d32b7 100644
--- a/doc/administration/job_artifacts.md
+++ b/doc/administration/job_artifacts.md
@@ -370,42 +370,6 @@ steps below.
 If the `expire` directive is not set explicitly in your pipeline, artifacts expire per the
 default artifacts expiration setting, which you can find in the [CI/CD Administration settings](../user/admin_area/settings/continuous_integration.md).
 
-## Validation for dependencies
-
-> Introduced in GitLab 10.3.
-
-To disable [the dependencies validation](../ci/yaml/README.md#when-a-dependent-job-fails),
-you can enable the `ci_validate_build_dependencies_override` feature flag from a Rails console.
-
-**In Omnibus installations:**
-
-1. Enter the Rails console:
-
-   ```shell
-   sudo gitlab-rails console
-   ```
-
-1. Enable the feature flag to disable the validation:
-
-   ```ruby
-   Feature.enable(:ci_validate_build_dependencies_override)
-   ```
-
-**In installations from source:**
-
-1. Enter the Rails console:
-
-   ```shell
-   cd /home/git/gitlab
-   sudo -u git -H bundle exec rails console -e production
-   ```
-
-1. Enable the feature flag to disable the validation:
-
-   ```ruby
-   Feature.enable(:ci_validate_build_dependencies_override)
-   ```
-
 ## Set the maximum file size of the artifacts
 
 If artifacts are enabled, you can change the maximum file size of the
diff --git a/doc/ci/yaml/README.md b/doc/ci/yaml/README.md
index 858e738e750..48df512fb42 100644
--- a/doc/ci/yaml/README.md
+++ b/doc/ci/yaml/README.md
@@ -3264,10 +3264,6 @@ If the artifacts of the job that is set as a dependency are
 [deleted](../pipelines/job_artifacts.md#delete-job-artifacts), then
 the dependent job fails.
 
-You can ask your administrator to
-[flip this switch](../../administration/job_artifacts.md#validation-for-dependencies)
-and bring back the old behavior.
-
 #### `artifacts:exclude`
 
 > - [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/15122) in GitLab 13.1
diff --git a/doc/development/avoiding_downtime_in_migrations.md b/doc/development/avoiding_downtime_in_migrations.md
index d8981ce0999..6b6c47cfece 100644
--- a/doc/development/avoiding_downtime_in_migrations.md
+++ b/doc/development/avoiding_downtime_in_migrations.md
@@ -365,6 +365,12 @@ if the application no longer uses the table.
 Renaming tables requires downtime as an application may continue
 using the old table name during/after a database migration.
 
+If the table and the ActiveRecord model is not in use yet, removing the old
+table and creating a new one is the preferred way to "rename" the table.
+
+Renaming a table is possible without downtime by following our multi-release
+[rename table process](database/rename_database_tables.md#rename-table-without-downtime).
+
 ## Adding Foreign Keys
 
 Adding foreign keys usually works in 3 steps:
diff --git a/doc/development/database/rename_database_tables.md b/doc/development/database/rename_database_tables.md
new file mode 100644
index 00000000000..743558fae19
--- /dev/null
+++ b/doc/development/database/rename_database_tables.md
@@ -0,0 +1,140 @@
+---
+stage: Enablement
+group: Database
+info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/engineering/ux/technical-writing/#assignments
+---
+
+# Rename table without downtime
+
+> [Introduced](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/54354) in GitLab 13.12.
+
+With our database helper methods built into GitLab, it's possible to rename a database table without downtime.
+
+The technique builds on top of database views, using the following steps:
+
+1. Rename the database table.
+1. Create a database view using the old table name by pointing to the new table name.
+1. Add workaround for ActiveRecord's schema cache.
+
+For example, consider that we are renaming the `issues` table name to `tickets`. Run:
+
+```sql
+BEGIN;
+  ALTER TABLE issues RENAME TO tickets;
+  CREATE VIEW issues AS SELECT * FROM tickets;
+COMMIT;
+```
+
+As database views do not expose the underlying table schema (default values, not null
+constraints, and indexes), we need further steps to update the application to use the new
+table name. ActiveRecord heavily relies on this data, for example, to initialize new
+models.
+
+To work around this limitation, we need to tell ActiveRecord to acquire this information
+from a different table using the new table name.
+
+## Migration strategy breakdown
+
+### Release N.M: Mark the ActiveRecord model's table
+
+Consider the current release as "Release N.M".
+
+In this release, register the database table so that it instructs ActiveRecord to fetch the
+database table information (for `SchemaCache`) using the new table name (if it's present). Otherwise, fall back
+to the old table name. This is necessary to avoid errors during a zero-downtime deployment.
+
+1. Edit the `TABLES_TO_BE_RENAMED` constant in: `lib/gitlab/database.rb`
+
+   ```ruby
+   TABLES_TO_BE_RENAMED = {
+     'issues' => 'tickets'
+   }.freeze
+   ```
+
+Note that, in this release (N.M), the `tickets` database table does not exist yet. This step is preparing for the actual table rename in release N.M+1.
+
+### Release N.M+1: Rename the database table
+
+Consider the next release as "Release N.M".
+
+Execute a standard migration (not a post-migration):
+
+```ruby
+  include Gitlab::Database::MigrationHelpers
+
+  def up
+    rename_table_safely(:issues, :tickets)
+  end
+
+  def down
+    undo_rename_table_safely(:issues, :tickets)
+  end
+```
+
+**Important notes:**
+
+- Let other developers know that the table is going to be renamed.
+  - Ping the `@gl-database` group in your merge request.
+  - Add a note in the Engineering Week-in-Review document: `table_name` is going to be renamed in N.M. Modifications to this table are not allowed in release N.M and N.M+1.
+- The helper method uses the standard `rename_table` helper from Rails for renaming the table.
+- The helper renames the sequence and the indexes. Sometimes it diverges from the standard Rails convention
+when naming indexes, so there is a possibility that not all indexes are properly renamed. After running
+the migration locally, check if there are inconsistently named indexes (`db/structure.sql`). Those can be
+renamed manually in a separate migration, which can be also part of the release M.N+1.
+- Foreign key columns might still contain the old table name. For smaller tables, follow our [standard column 
+rename process](../avoiding_downtime_in_migrations.md#renaming-columns)
+- Avoid renaming database tables which are using with triggers.
+- Table modifications (add or remove columns) are not allowed during the rename process, please make sure that all changes to the table happen before the rename migration is started (or in the next release).
+- As the index names might change, verify that the model does not use bulk insert
+(for example, `insert_all` and `upsert_all`) with the `unique_by: index_name` option.
+Renaming an index while using these methods may break functionality.
+- Modify the model code to point to the new database table. Do this by
+renaming the model directly or setting the `self.table_name` variable.
+
+At this point, we don't have applications using the old database table name in their queries.
+
+1. Remove the database view through a post-migration:
+
+   ```ruby
+     include Gitlab::Database::MigrationHelpers
+
+     def up
+       finalize_table_rename(:issues, :tickets)
+     end
+
+     def down
+       undo_finalize_table_rename(:issues, :tickets)
+     end
+   ```
+
+1. Additionally the table definition from `TABLES_TO_BE_RENAMED` **must** be removed.
+
+To do so, edit the `TABLES_TO_BE_RENAMED` constant in `lib/gitlab/database.rb`:
+
+   From:
+
+   ```ruby
+   TABLES_TO_BE_RENAMED = {
+     'issues' => 'tickets'
+   }.freeze
+   ```
+
+   To:
+
+   ```ruby
+   TABLES_TO_BE_RENAMED = {}.freeze
+   ```
+
+#### Zero-downtime deployments
+
+When the application is upgraded without downtime, there can be application instances
+running the old code. The old code still references the old database table. The queries
+still function without any problems, because the backward-compatible database view is
+in place.
+
+In case the old version of the application needs to be restarted or reconnected to the
+database, ActiveRecord fetches the column information again. At this time, our previously
+marked table (`TABLES_TO_BE_RENAMED`) instructs ActiveRecord to use the new database table name
+when fetching the database table information.
+
+The new version of the application will use the new database table.
diff --git a/doc/development/elasticsearch.md b/doc/development/elasticsearch.md
index 3e466512c79..6b829faf74d 100644
--- a/doc/development/elasticsearch.md
+++ b/doc/development/elasticsearch.md
@@ -316,11 +316,11 @@ choose to remove the migration code and tests so that:
   anymore.
 
 To be extra safe, we will not delete migrations that were created in the last
-minor version before the major upgrade. So, if the we are upgrading to `%14.0`,
-we should not delete migrations that were only added in `%13.11`. This is an
+minor version before the major upgrade. So, if we are upgrading to `%14.0`,
+we should not delete migrations that were only added in `%13.12`. This is an
 extra safety net as we expect there are migrations that get merged that may
 take multiple weeks to finish on GitLab.com. It would be bad if we upgraded
-GitLab.com to `%14.0` before the migrations in `%13.11` were finished. Since
+GitLab.com to `%14.0` before the migrations in `%13.12` were finished. Since
 our deployments to GitLab.com are automated and we currently don't have
 automated checks to prevent this, the extra precaution is warranted.
 Additionally, even if we did have automated checks to prevent it, we wouldn't
@@ -340,7 +340,7 @@ being upgraded to, we do the following:
    def migrate
      log_raise "Migration has been deleted in the last major version upgrade." \
        "Migrations are supposed to be finished before upgrading major version https://docs.gitlab.com/ee/update/#upgrading-to-a-new-major-version ." \
-       "In order to correct this issue you will need to reacreate your index from scratch https://docs.gitlab.com/ee/integration/elasticsearch.html#last-resort-to-recreate-an-index ."
+       "To correct this issue, recreate your index from scratch: https://docs.gitlab.com/ee/integration/elasticsearch.html#last-resort-to-recreate-an-index."
    end
 
    def completed?
diff --git a/doc/raketasks/backup_restore.md b/doc/raketasks/backup_restore.md
index e182ecc48bc..74e254d864e 100644
--- a/doc/raketasks/backup_restore.md
+++ b/doc/raketasks/backup_restore.md
@@ -339,7 +339,8 @@ sudo -u git -H GITLAB_ASSUME_YES=1 bundle exec rake gitlab:backup:restore RAILS_
 
 > [Introduced](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/37158) in GitLab 13.3.
 
-Repositories can be backed up concurrently to help fully use CPU time. The
+When using [multiple repository storages](../administration/repository_storage_paths.md),
+repositories can be backed up concurrently to help fully use CPU time. The
 following variables are available to modify the default behavior of the Rake
 task:
 
@@ -349,7 +350,7 @@ task:
   back up at the same time on each storage. This allows the repository backups
   to be spread across storages. Defaults to `1`.
 
-For example, for Omnibus GitLab installations:
+For example, for Omnibus GitLab installations with 4 repository storages:
 
 ```shell
 sudo gitlab-backup create GITLAB_BACKUP_MAX_CONCURRENCY=4 GITLAB_BACKUP_MAX_STORAGE_CONCURRENCY=1