Add latest changes from gitlab-org/gitlab@master

3 files changed, 154 insertions, 12 deletions

diff --git a/doc/administration/geo/replication/datatypes.md b/doc/administration/geo/replication/datatypes.md
index 2bccb3e73dc..75ce7503c34 100644
--- a/doc/administration/geo/replication/datatypes.md
+++ b/doc/administration/geo/replication/datatypes.md
@@ -129,11 +129,11 @@ successfully, you must replicate their data using some other means.
 | Application data in PostgreSQL                      | **Yes**                  | **Yes**                     |                                             |
 | Project repository                                  | **Yes**                  | **Yes**                     |                                             |
 | Project wiki repository                             | **Yes**                  | **Yes**                     |                                             |
-| Project designs repository                          | **Yes**                  | [No][design-verification]   | Behind feature flag (*1*)                   |
-| Uploads                                             | **Yes**                  | [No][upload-verification]   | Verified only on transfer, or manually (*2*)|
-| LFS objects                                         | **Yes**                  | [No][lfs-verification]      | Verified only on transfer, or manually (*2*)|
-| CI job artifacts (other than traces)                | **Yes**                  | [No][artifact-verification] | Verified only manually (*2*)                |
-| Archived traces                                     | **Yes**                  | [No][artifact-verification] | Verified only on transfer, or manually (*2*)|
+| Project designs repository                          | **Yes**                  | [No][design-verification]   |                                             |
+| Uploads                                             | **Yes**                  | [No][upload-verification]   | Verified only on transfer, or manually (*1*)|
+| LFS objects                                         | **Yes**                  | [No][lfs-verification]      | Verified only on transfer, or manually (*1*)|
+| CI job artifacts (other than traces)                | **Yes**                  | [No][artifact-verification] | Verified only manually (*1*)                |
+| Archived traces                                     | **Yes**                  | [No][artifact-verification] | Verified only on transfer, or manually (*1*)|
 | Personal snippets                                   | **Yes**                  | **Yes**                     |                                             |
 | Project snippets                                    | **Yes**                  | **Yes**                     |                                             |
 | Object pools for forked project deduplication       | **Yes**                  | No                          |                                             |
@@ -147,13 +147,7 @@ successfully, you must replicate their data using some other means.
 | [External merge request diffs][merge-request-diffs] | [No][diffs-replication]  | No                          |                                             |
 | Content in object storage                           | **Yes**                  | No                          |                                             |
 
-- (*1*): Enable the `enable_geo_design_sync` feature flag by running the following in a Rails console:
-
-  ```ruby
-  Feature.disable(:enable_geo_design_sync)
-  ```
-
-- (*2*): The integrity can be verified manually using
+- (*1*): The integrity can be verified manually using
   [Integrity Check Rake Task](../../raketasks/check.md) on both nodes and comparing the output between them.
 
 [design-replication]: https://gitlab.com/groups/gitlab-org/-/epics/1633
diff --git a/doc/administration/job_artifacts.md b/doc/administration/job_artifacts.md
index 2cc2cb1751b..7a3d116ea58 100644
--- a/doc/administration/job_artifacts.md
+++ b/doc/administration/job_artifacts.md
@@ -294,3 +294,149 @@ memory and disk I/O.
 [reconfigure gitlab]: restart_gitlab.md#omnibus-gitlab-reconfigure "How to reconfigure Omnibus GitLab"
 [restart gitlab]: restart_gitlab.md#installations-from-source "How to restart GitLab"
 [gitlab workhorse]: https://gitlab.com/gitlab-org/gitlab-workhorse "GitLab Workhorse repository"
+
+## Troubleshooting
+
+### Job artifacts using too much disk space
+
+Job artifacts can fill up your disk space quicker than expected. Some possible
+reasons are:
+
+- Users have configured job artifacts expiration to be longer than necessary.
+- The number of jobs run, and hence artifacts generated, is higher than expected.
+- Job logs are larger than expected, and have accumulated over time.
+
+In these and other cases, you'll need to identify the projects most responsible
+for disk space usage, figure out what types of artifacts are using the most
+space, and in some cases, manually delete job artifacts to reclaim disk space.
+
+#### List projects by total size of job artifacts stored
+
+List the top 20 projects, sorted by the total size of job artifacts stored, by
+running the following code in the Rails console (`sudo gitlab-rails console`):
+
+```ruby
+include ActionView::Helpers::NumberHelper
+ProjectStatistics.order(build_artifacts_size: :desc).limit(20).each do |s|
+  puts "#{number_to_human_size(s.build_artifacts_size)} \t #{s.project.full_path}"
+end
+```
+
+You can change the number of projects listed by modifying `.limit(20)` to the
+number you want.
+
+#### List largest artifacts in a single project
+
+List the 50 largest job artifacts in a single project by running the following
+code in the Rails console (`sudo gitlab-rails console`):
+
+```ruby
+include ActionView::Helpers::NumberHelper
+project = Project.find_by_full_path('path/to/project')
+Ci::JobArtifact.where(project: project).order(size: :desc).limit(50).map { |a| puts "ID: #{a.id} - #{a.file_type}: #{number_to_human_size(a.size)}" }
+```
+
+You can change the number of job artifacts listed by modifying `.limit(50)` to
+the number you want.
+
+#### Delete job artifacts from jobs completed before a specific date
+
+CAUTION: **CAUTION:**
+These commands remove data permanently from the database and from disk. We
+highly recommend running them only under the guidance of a Support Engineer, or
+running them in a test environment with a backup of the instance ready to be
+restored, just in case.
+
+If you need to manually remove job artifacts associated with multiple jobs while
+**retaining their job logs**, this can be done from the Rails console (`sudo gitlab-rails console`):
+
+1. Select jobs to be deleted:
+
+   To select all jobs with artifacts for a single project:
+
+   ```ruby
+   project = Project.find_by_full_path('path/to/project')
+   builds_with_artifacts =  project.builds.with_artifacts_archive
+   ```
+
+   To select all jobs with artifacts across the entire GitLab instance:
+
+   ```ruby
+   builds_with_artifacts = Ci::Build.with_artifacts_archive
+   ```
+
+1. Delete job artifacts older than a specific date:
+
+   NOTE: **NOTE:**
+   This step will also erase artifacts that users have chosen to
+   ["keep"](../user/project/pipelines/job_artifacts.html#browsing-artifacts).
+
+   ```ruby
+   builds_to_clear = builds_with_artifacts.where("finished_at < ?", 1.week.ago)
+   builds_to_clear.find_each do |build|
+     build.artifacts_expire_at = Time.now
+     build.erase_erasable_artifacts!
+   end
+   ```
+
+   `1.week.ago` is a Rails `ActiveSupport::Duration` method which calculates a new
+   date or time in the past. Other valid examples are:
+
+   - `7.days.ago`
+   - `3.months.ago`
+   - `1.year.ago`
+
+#### Delete job artifacts and logs from jobs completed before a specific date
+
+CAUTION: **CAUTION:**
+These commands remove data permanently from the database and from disk. We
+highly recommend running them only under the guidance of a Support Engineer, or
+running them in a test environment with a backup of the instance ready to be
+restored, just in case.
+
+If you need to manually remove ALL job artifacts associated with multiple jobs,
+**including job logs**, this can be done from the Rails console (`sudo gitlab-rails console`):
+
+1. Select jobs to be deleted:
+
+   To select jobs with artifacts for a single project:
+
+   ```ruby
+   project = Project.find_by_full_path('path/to/project')
+   builds_with_artifacts =  project.builds.with_existing_job_artifacts
+   ```
+
+   To select jobs with artifacts across the entire GitLab instance:
+
+   ```ruby
+   builds_with_artifacts = Ci::Build.with_existing_job_artifacts
+   ```
+
+1. Select the user which will be mentioned in the web UI as erasing the job:
+
+   ```ruby
+   admin_user = User.find_by(username: 'username')
+   ```
+
+1. Erase job artifacts and logs older than a specific date:
+
+   ```ruby
+   builds_to_clear = builds_with_artifacts.where("finished_at < ?", 1.week.ago)
+   builds_to_clear.find_each do |build|
+     print "Ci::Build ID #{build.id}... "
+
+     if build.erasable?
+       build.erase(erased_by: admin_user)
+       puts "Erased"
+     else
+       puts "Skipped (Nothing to erase or not erasable)"
+     end
+   end
+   ```
+
+   `1.week.ago` is a Rails `ActiveSupport::Duration` method which calculates a new
+   date or time in the past. Other valid examples are:
+
+   - `7.days.ago`
+   - `3.months.ago`
+   - `1.year.ago`
diff --git a/doc/administration/troubleshooting/gitlab_rails_cheat_sheet.md b/doc/administration/troubleshooting/gitlab_rails_cheat_sheet.md
index cb0b24ae026..0ab8d629b61 100644
--- a/doc/administration/troubleshooting/gitlab_rails_cheat_sheet.md
+++ b/doc/administration/troubleshooting/gitlab_rails_cheat_sheet.md
@@ -743,6 +743,8 @@ Namespace.find_by_full_path("user/proj").namespace_statistics.update(shared_runn
 
 ### Remove artifacts more than a week old
 
+The Latest version of these steps can be found in the [job artifacts documentation](../job_artifacts.md)
+
 ```ruby
 ### SELECTING THE BUILDS TO CLEAR
 # For a single project: