diff options
Diffstat (limited to 'doc/administration/raketasks')
-rw-r--r-- | doc/administration/raketasks/check.md | 34 | ||||
-rw-r--r-- | doc/administration/raketasks/github_import.md | 4 | ||||
-rw-r--r-- | doc/administration/raketasks/incoming_email.md | 149 | ||||
-rw-r--r-- | doc/administration/raketasks/maintenance.md | 9 | ||||
-rw-r--r-- | doc/administration/raketasks/project_import_export.md | 2 | ||||
-rw-r--r-- | doc/administration/raketasks/service_desk_email.md | 149 |
6 files changed, 342 insertions, 5 deletions
diff --git a/doc/administration/raketasks/check.md b/doc/administration/raketasks/check.md index 2660caa80b3..2cb664b0859 100644 --- a/doc/administration/raketasks/check.md +++ b/doc/administration/raketasks/check.md @@ -381,3 +381,37 @@ To delete these references to missing local and/or remote artifacts (`job.log` f If `gitlab-rake gitlab:lfs:check VERBOSE=1` detects LFS objects that exist in the database but not on disk, [follow the procedure in the LFS documentation](../lfs/index.md#missing-lfs-objects) to remove the database entries. + +### Update dangling object storage references + +If you have [migrated from object storage to local storage](../job_artifacts.md#migrating-from-object-storage-to-local-storage) and files were missing, then dangling database references remain. + +This is visible in the migration logs with errors like the following: + +```shell +W, [2022-11-28T13:14:09.283833 #10025] WARN -- : Failed to transfer Ci::JobArtifact ID 11 with error: undefined method `body' for nil:NilClass +W, [2022-11-28T13:14:09.296911 #10025] WARN -- : Failed to transfer Ci::JobArtifact ID 12 with error: undefined method `body' for nil:NilClass +``` + +Attempting to [delete references to missing artifacts](check.md#delete-references-to-missing-artifacts) after you have disabled object storage, results in the following error: + +```shell +RuntimeError (Object Storage is not enabled for JobArtifactUploader) +``` + +To update these references to point to local storage: + +1. Open the [GitLab Rails Console](../operations/rails_console.md#starting-a-rails-console-session). +1. Run the following Ruby code: + + ```ruby + artifacts_updated = 0 + ::Ci::JobArtifact.find_each do |artifact| ### Iterate artifacts + next if artifact.file_store != 2 ### Skip if file_store already points to local storage + artifacts_updated += 1 + # artifact.update(file_store: 1) ### Uncomment to actually update + end + puts "Updated file_store count: #{artifacts_updated}" + ``` + +The script to [delete references to missing artifacts](check.md#delete-references-to-missing-artifacts) now functions correctly and cleans up the database. diff --git a/doc/administration/raketasks/github_import.md b/doc/administration/raketasks/github_import.md index 2e6a88a77e2..d089682f78e 100644 --- a/doc/administration/raketasks/github_import.md +++ b/doc/administration/raketasks/github_import.md @@ -6,6 +6,8 @@ info: To determine the technical writer assigned to the Stage/Group associated w # GitHub import Rake task **(FREE SELF)** +> [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/390690) in GitLab 15.9, Rake task no longer automatically creates namespaces or groups that don't exist. + To retrieve and import GitHub repositories, you need a [GitHub personal access token](https://github.com/settings/tokens). A username should be passed as the second argument to the Rake task, which becomes the owner of the project. You can resume an import @@ -39,7 +41,7 @@ bundle exec rake "import:github[access_token,root,foo/bar]" RAILS_ENV=production In this case, `access_token` is your GitHub personal access token, `root` is your GitLab username, and `foo/bar` is the new GitLab namespace/project -created from your GitHub project. Subgroups are also possible: `foo/foo/bar`. The importer creates any missing intermediate namespaces (groups) if they do not exist. +created from your GitHub project. Subgroups are also possible: `foo/foo/bar`. ## Importing a single project diff --git a/doc/administration/raketasks/incoming_email.md b/doc/administration/raketasks/incoming_email.md new file mode 100644 index 00000000000..6b9c27ed144 --- /dev/null +++ b/doc/administration/raketasks/incoming_email.md @@ -0,0 +1,149 @@ +--- +stage: Systems +group: Distribution +info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments +--- + +# Incoming email Rake tasks **(FREE SELF)** + +> [Introduced](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/108279) in GitLab 15.9. + +The following are Incoming email-related Rake tasks. + +## Secrets + +GitLab can use [Incoming email](../incoming_email.md) secrets read from an encrypted file instead of storing them in plaintext in the file system. The following Rake tasks are provided for updating the contents of the encrypted file. + +### Show secret + +Show the contents of the current Incoming email secrets. + +::Tabs + +:::TabTitle Linux package (Omnibus) + +```shell +sudo gitlab-rake gitlab:incoming_email:secret:show +``` + +:::TabTitle Helm chart (Kubernetes) + +Use a Kubernetes secret to store the incoming email password. For more information, +read about [Helm IMAP secrets](https://docs.gitlab.com/charts/installation/secrets.html#imap-password-for-incoming-emails). + +:::TabTitle Docker + +```shell +sudo docker exec -t <container name> gitlab:incoming_email:secret:show +``` + +:::TabTitle Self-compiled (source) + +```shell +bundle exec rake gitlab:incoming_email:secret:show RAILS_ENV=production +``` + +::EndTabs + +#### Example output + +```plaintext +password: 'examplepassword' +user: 'incoming-email@mail.example.com' +``` + +### Edit secret + +Opens the secret contents in your editor, and writes the resulting content to the encrypted secret file when you exit. + +::Tabs + +:::TabTitle Linux package (Omnibus) + +```shell +sudo gitlab-rake gitlab:incoming_email:secret:edit EDITOR=vim +``` + +:::TabTitle Helm chart (Kubernetes) + +Use a Kubernetes secret to store the incoming email password. For more information, +read about [Helm IMAP secrets](https://docs.gitlab.com/charts/installation/secrets.html#imap-password-for-incoming-emails). + +:::TabTitle Docker + +```shell +sudo docker exec -t <container name> gitlab:incoming_email:secret:edit EDITOR=editor +``` + +:::TabTitle Self-compiled (source) + +```shell +bundle exec rake gitlab:incoming_email:secret:edit RAILS_ENV=production EDITOR=vim +``` + +::EndTabs + +### Write raw secret + +Write new secret content by providing it on `STDIN`. + +::Tabs + +:::TabTitle Linux package (Omnibus) + +```shell +echo -e "password: 'examplepassword'" | sudo gitlab-rake gitlab:incoming_email:secret:write +``` + +:::TabTitle Helm chart (Kubernetes) + +Use a Kubernetes secret to store the incoming email password. For more information, +read about [Helm IMAP secrets](https://docs.gitlab.com/charts/installation/secrets.html#imap-password-for-incoming-emails). + +:::TabTitle Docker + +```shell +sudo docker exec -t <container name> /bin/bash +echo -e "password: 'examplepassword'" | gitlab-rake gitlab:incoming_email:secret:write +``` + +:::TabTitle Self-compiled (source) + +```shell +echo -e "password: 'examplepassword'" | bundle exec rake gitlab:incoming_email:secret:write RAILS_ENV=production +``` + +::EndTabs + +### Secrets examples + +**Editor example** + +The write task can be used in cases where the edit command does not work with your editor: + +```shell +# Write the existing secret to a plaintext file +sudo gitlab-rake gitlab:incoming_email:secret:show > incoming_email.yaml +# Edit the incoming_email file in your editor +... +# Re-encrypt the file +cat incoming_email.yaml | sudo gitlab-rake gitlab:incoming_email:secret:write +# Remove the plaintext file +rm incoming_email.yaml +``` + +**KMS integration example** + +It can also be used as a receiving application for content encrypted with a KMS: + +```shell +gcloud kms decrypt --key my-key --keyring my-test-kms --plaintext-file=- --ciphertext-file=my-file --location=us-west1 | sudo gitlab-rake gitlab:incoming_email:secret:write +``` + +**Google Cloud secret integration example** + +It can also be used as a receiving application for secrets out of Google Cloud: + +```shell +gcloud secrets versions access latest --secret="my-test-secret" > $1 | sudo gitlab-rake gitlab:incoming_email:secret:write +``` diff --git a/doc/administration/raketasks/maintenance.md b/doc/administration/raketasks/maintenance.md index ba095b33bf5..5c258d73fdb 100644 --- a/doc/administration/raketasks/maintenance.md +++ b/doc/administration/raketasks/maintenance.md @@ -234,8 +234,11 @@ sudo -u git -H bundle exec rake cache:clear RAILS_ENV=production Sometimes during version upgrades you might end up with some wrong CSS or missing some icons. In that case, try to precompile the assets again. -This only applies to source installations and does not apply to -Omnibus packages. +This Rake task only applies to source installations. [Read more](../../update/package/index.md#missing-asset-files) +about troubleshooting this problem when running the Omnibus GitLab package. +The guidance for Omnibus GitLab might be applicable for Kubernetes and Docker Omnibus +deployments of GitLab, though in general, container-based installations +don't have issues with missing assets. **Source Installation** @@ -374,7 +377,7 @@ The following index types are not supported: Optionally, this Rake task sends annotations to a Grafana (4.6 or later) endpoint. Use the following custom environment variables to enable annotations: -1. `GRAFANA_API_URL` - Grafana's base URL, for example `http://some-host:3000`. +1. `GRAFANA_API_URL` - The base URL for Grafana, for example `http://some-host:3000`. 1. `GRAFANA_API_KEY` - Grafana API key with at least `Editor role`. You can also [enable reindexing as a regular cron job](https://docs.gitlab.com/omnibus/settings/database.html#automatic-database-reindexing). diff --git a/doc/administration/raketasks/project_import_export.md b/doc/administration/raketasks/project_import_export.md index e43fbac25e9..4694af18af2 100644 --- a/doc/administration/raketasks/project_import_export.md +++ b/doc/administration/raketasks/project_import_export.md @@ -42,7 +42,7 @@ bundle exec rake gitlab:import_export:data RAILS_ENV=production Note the following: - Importing is only possible if the version of the import and export GitLab instances are - compatible as described in the [Version history](../../user/project/settings/import_export.md#version-history). + [compatible](../../user/project/settings/import_export.md#compatibility). - The project import option must be enabled: 1. On the top bar, select **Main menu > Admin**. diff --git a/doc/administration/raketasks/service_desk_email.md b/doc/administration/raketasks/service_desk_email.md new file mode 100644 index 00000000000..10de379b1cd --- /dev/null +++ b/doc/administration/raketasks/service_desk_email.md @@ -0,0 +1,149 @@ +--- +stage: Systems +group: Distribution +info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments +--- + +# Service Desk email Rake tasks **(FREE SELF)** + +> [Introduced](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/108279) in GitLab 15.9. + +The following are Service Desk email-related Rake tasks. + +## Secrets + +GitLab can use [Service Desk email](../../user/project/service_desk.md#configuring-a-custom-mailbox) secrets read from an encrypted file instead of storing them in plaintext in the file system. The following Rake tasks are provided for updating the contents of the encrypted file. + +### Show secret + +Show the contents of the current Service Desk email secrets. + +::Tabs + +:::TabTitle Linux package (Omnibus) + +```shell +sudo gitlab-rake gitlab:service_desk_email:secret:show +``` + +:::TabTitle Helm chart (Kubernetes) + +Use a Kubernetes secret to store the Service Desk email password. For more information, +read about [Helm IMAP secrets](https://docs.gitlab.com/charts/installation/secrets.html#imap-password-for-service-desk-emails). + +:::TabTitle Docker + +```shell +sudo docker exec -t <container name> gitlab:service_desk_email:secret:show +``` + +:::TabTitle Self-compiled (source) + +```shell +bundle exec rake gitlab:service_desk_email:secret:show RAILS_ENV=production +``` + +::EndTabs + +#### Example output + +```plaintext +password: 'examplepassword' +user: 'service-desk-email@mail.example.com' +``` + +### Edit secret + +Opens the secret contents in your editor, and writes the resulting content to the encrypted secret file when you exit. + +::Tabs + +:::TabTitle Linux package (Omnibus) + +```shell +sudo gitlab-rake gitlab:service_desk_email:secret:edit EDITOR=vim +``` + +:::TabTitle Helm chart (Kubernetes) + +Use a Kubernetes secret to store the Service Desk email password. For more information, +read about [Helm IMAP secrets](https://docs.gitlab.com/charts/installation/secrets.html#imap-password-for-service-desk-emails). + +:::TabTitle Docker + +```shell +sudo docker exec -t <container name> gitlab:service_desk_email:secret:edit EDITOR=editor +``` + +:::TabTitle Self-compiled (source) + +```shell +bundle exec rake gitlab:service_desk_email:secret:edit RAILS_ENV=production EDITOR=vim +``` + +::EndTabs + +### Write raw secret + +Write new secret content by providing it on `STDIN`. + +::Tabs + +:::TabTitle Linux package (Omnibus) + +```shell +echo -e "password: 'examplepassword'" | sudo gitlab-rake gitlab:service_desk_email:secret:write +``` + +:::TabTitle Helm chart (Kubernetes) + +Use a Kubernetes secret to store the Service Desk email password. For more information, +read about [Helm IMAP secrets](https://docs.gitlab.com/charts/installation/secrets.html#imap-password-for-service-desk-emails). + +:::TabTitle Docker + +```shell +sudo docker exec -t <container name> /bin/bash +echo -e "password: 'examplepassword'" | gitlab-rake gitlab:service_desk_email:secret:write +``` + +:::TabTitle Self-compiled (source) + +```shell +echo -e "password: 'examplepassword'" | bundle exec rake gitlab:service_desk_email:secret:write RAILS_ENV=production +``` + +::EndTabs + +### Secrets examples + +**Editor example** + +The write task can be used in cases where the edit command does not work with your editor: + +```shell +# Write the existing secret to a plaintext file +sudo gitlab-rake gitlab:service_desk_email:secret:show > service_desk_email.yaml +# Edit the service_desk_email file in your editor +... +# Re-encrypt the file +cat service_desk_email.yaml | sudo gitlab-rake gitlab:service_desk_email:secret:write +# Remove the plaintext file +rm service_desk_email.yaml +``` + +**KMS integration example** + +It can also be used as a receiving application for content encrypted with a KMS: + +```shell +gcloud kms decrypt --key my-key --keyring my-test-kms --plaintext-file=- --ciphertext-file=my-file --location=us-west1 | sudo gitlab-rake gitlab:service_desk_email:secret:write +``` + +**Google Cloud secret integration example** + +It can also be used as a receiving application for secrets out of Google Cloud: + +```shell +gcloud secrets versions access latest --secret="my-test-secret" > $1 | sudo gitlab-rake gitlab:service_desk_email:secret:write +``` |