diff options
Diffstat (limited to 'doc/administration/operations')
-rw-r--r-- | doc/administration/operations/extra_sidekiq_processes.md | 8 | ||||
-rw-r--r-- | doc/administration/operations/fast_ssh_key_lookup.md | 2 | ||||
-rw-r--r-- | doc/administration/operations/filesystem_benchmarking.md | 14 | ||||
-rw-r--r-- | doc/administration/operations/index.md | 8 | ||||
-rw-r--r-- | doc/administration/operations/moving_repositories.md | 49 | ||||
-rw-r--r-- | doc/administration/operations/puma.md | 2 | ||||
-rw-r--r-- | doc/administration/operations/speed_up_ssh.md | 8 |
7 files changed, 42 insertions, 49 deletions
diff --git a/doc/administration/operations/extra_sidekiq_processes.md b/doc/administration/operations/extra_sidekiq_processes.md index 1b7c0edab04..d07afb3bb14 100644 --- a/doc/administration/operations/extra_sidekiq_processes.md +++ b/doc/administration/operations/extra_sidekiq_processes.md @@ -4,7 +4,7 @@ group: Distribution info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/engineering/ux/technical-writing/#assignments --- -# Run multiple Sidekiq processes **(CORE ONLY)** +# Run multiple Sidekiq processes **(FREE SELF)** GitLab allows you to start multiple Sidekiq processes. These processes can be used to consume a dedicated set @@ -27,7 +27,7 @@ can be started. ## Start multiple processes > - [Introduced](https://gitlab.com/gitlab-org/omnibus-gitlab/-/merge_requests/4006) in GitLab 12.10, starting multiple processes with Sidekiq cluster. -> - [Sidekiq cluster moved](https://gitlab.com/groups/gitlab-com/gl-infra/-/epics/181) to GitLab [Core](https://about.gitlab.com/pricing/#self-managed) in GitLab 12.10. +> - [Sidekiq cluster moved](https://gitlab.com/groups/gitlab-com/gl-infra/-/epics/181) to GitLab Free in 12.10. > - [Sidekiq cluster became default](https://gitlab.com/gitlab-org/omnibus-gitlab/-/merge_requests/4140) in GitLab 13.0. To start multiple processes: @@ -112,8 +112,8 @@ you list: ## Queue selector -> - [Introduced](https://gitlab.com/gitlab-com/gl-infra/scalability/-/issues/45) in [GitLab Starter](https://about.gitlab.com/pricing/) 12.8. -> - [Sidekiq cluster including queue selector moved](https://gitlab.com/groups/gitlab-com/gl-infra/-/epics/181) to GitLab [Core](https://about.gitlab.com/pricing/#self-managed) in GitLab 12.10. +> - [Introduced](https://gitlab.com/gitlab-com/gl-infra/scalability/-/issues/45) in GitLab 12.8. +> - [Sidekiq cluster, including queue selector, moved](https://gitlab.com/groups/gitlab-com/gl-infra/-/epics/181) to GitLab Free in 12.10. > - [Renamed from `experimental_queue_selector` to `queue_selector`](https://gitlab.com/gitlab-com/gl-infra/scalability/-/issues/147) in GitLab 13.6. In addition to selecting queues by name, as above, the `queue_selector` diff --git a/doc/administration/operations/fast_ssh_key_lookup.md b/doc/administration/operations/fast_ssh_key_lookup.md index b93af074795..330bd9a43da 100644 --- a/doc/administration/operations/fast_ssh_key_lookup.md +++ b/doc/administration/operations/fast_ssh_key_lookup.md @@ -224,5 +224,5 @@ the database. The following instructions can be used to build OpenSSH 7.5: GitLab supports `authorized_keys` database lookups with [SELinux](https://en.wikipedia.org/wiki/Security-Enhanced_Linux). Because the SELinux policy is static, GitLab doesn't support the ability to change -internal Unicorn ports at the moment. Admins would have to create a special `.te` +internal Unicorn ports at the moment. Administrators would have to create a special `.te` file for the environment, since it isn't generated dynamically. diff --git a/doc/administration/operations/filesystem_benchmarking.md b/doc/administration/operations/filesystem_benchmarking.md index 9072214eb02..6119fcdae45 100644 --- a/doc/administration/operations/filesystem_benchmarking.md +++ b/doc/administration/operations/filesystem_benchmarking.md @@ -4,15 +4,15 @@ group: Distribution info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/engineering/ux/technical-writing/#assignments --- -# Filesystem Performance Benchmarking +# File system Performance Benchmarking -Filesystem performance has a big impact on overall GitLab performance, +File system performance has a big impact on overall GitLab performance, especially for actions that read or write to Git repositories. This information -will help benchmark filesystem performance against known good and bad real-world +will help benchmark file system performance against known good and bad real-world systems. -Normally when talking about filesystem performance the biggest concern is -with Network Filesystems (NFS). However, even some local disks can have slow +Normally when talking about file system performance the biggest concern is +with Network File Systems (NFS). However, even some local disks can have slow I/O. The information on this page can be used for either scenario. ## Executing benchmarks @@ -77,7 +77,7 @@ available on the system. It's possible to receive good results on this test but still have poor performance due to read speed and various other factors. -The following one-line commands provide a quick benchmark for filesystem write and read +The following one-line commands provide a quick benchmark for file system write and read performance. This will write 1,000 small files to the directory in which it is executed, and then read the same 1,000 files. @@ -125,4 +125,4 @@ sys 0m1.663s ``` From experience with multiple customers, this task should take under 10 -seconds to indicate good filesystem performance. +seconds to indicate good file system performance. diff --git a/doc/administration/operations/index.md b/doc/administration/operations/index.md index b15417ea8d9..708861d8529 100644 --- a/doc/administration/operations/index.md +++ b/doc/administration/operations/index.md @@ -13,24 +13,24 @@ Keep your GitLab instance up and running smoothly. you have been running a large GitLab server (thousands of users) since before GitLab 7.3 we recommend cleaning up stale sessions to compact the Redis database after you upgrade to GitLab 7.3. -- [Rake tasks](../../raketasks/README.md): Tasks for common administration and operational processes such as +- [Rake tasks](../../raketasks/index.md): Tasks for common administration and operational processes such as [cleaning up unneeded items from GitLab instance](../../raketasks/cleanup.md), integrity checks, and more. - [Moving repositories](moving_repositories.md): Moving all repositories managed by GitLab to another file system or another server. - [Sidekiq MemoryKiller](sidekiq_memory_killer.md): Configure Sidekiq MemoryKiller to restart Sidekiq. -- [Multiple Sidekiq processes](extra_sidekiq_processes.md): Configure multiple Sidekiq processes to ensure certain queues always have dedicated workers, no matter the number of jobs that need to be processed. **(CORE ONLY)** +- [Multiple Sidekiq processes](extra_sidekiq_processes.md): Configure multiple Sidekiq processes to ensure certain queues always have dedicated workers, no matter the number of jobs that need to be processed. **(FREE SELF)** - [Puma](puma.md): Understand Puma and puma-worker-killer. - [Unicorn](unicorn.md): Understand Unicorn and unicorn-worker-killer. - Speed up SSH operations by [Authorizing SSH users via a fast, indexed lookup to the GitLab database](fast_ssh_key_lookup.md), and/or by [doing away with user SSH keys stored on GitLab entirely in favor of SSH certificates](ssh_certificates.md). -- [Filesystem Performance Benchmarking](filesystem_benchmarking.md): Filesystem +- [File System Performance Benchmarking](filesystem_benchmarking.md): File system performance can have a big impact on GitLab performance, especially for actions that read or write Git repositories. This information will help benchmark - filesystem performance against known good and bad real-world systems. + file system performance against known good and bad real-world systems. - [The Rails Console](rails_console.md): Provides a way to interact with your GitLab instance from the command line. Used for troubleshooting a problem or retrieving some data that can only be done through direct access to GitLab. - [ChatOps Scripts](https://gitlab.com/gitlab-com/chatops): The GitLab.com Infrastructure team uses this repository to house diff --git a/doc/administration/operations/moving_repositories.md b/doc/administration/operations/moving_repositories.md index 029f3bb01ed..da3b3dbbc15 100644 --- a/doc/administration/operations/moving_repositories.md +++ b/doc/administration/operations/moving_repositories.md @@ -5,7 +5,7 @@ info: To determine the technical writer assigned to the Stage/Group associated w type: reference --- -# Moving repositories managed by GitLab **(CORE ONLY)** +# Moving repositories managed by GitLab **(FREE SELF)** Sometimes you need to move all repositories managed by GitLab to another file system or another server. @@ -20,25 +20,23 @@ The GitLab API is the recommended way to move Git repositories: For more information, see: -- [Configuring additional storage for Gitaly](../gitaly/index.md#network-architecture). Within this - example, additional storage called `storage1` and `storage2` is configured. +- [Configuring additional storage for Gitaly](../gitaly/index.md#network-architecture). This + example configures additional storage called `storage1` and `storage2`. - [The API documentation](../../api/project_repository_storage_moves.md) details the endpoints for querying and scheduling project repository moves. - [The API documentation](../../api/snippet_repository_storage_moves.md) details the endpoints for querying and scheduling snippet repository moves. +- [The API documentation](../../api/group_repository_storage_moves.md) details the endpoints for + querying and scheduling group repository moves **(PREMIUM SELF)**. - [Migrate existing repositories to Gitaly Cluster](../gitaly/praefect.md#migrate-existing-repositories-to-gitaly-cluster). -### Limitations - -Read more in the [API documentation for projects](../../api/project_repository_storage_moves.md#limitations) and the [API documentation for snippets](../../api/snippet_repository_storage_moves.md#limitations). - ## Migrating to another GitLab instance [Using the API](#moving-data-within-a-gitlab-instance) isn't an option if you are migrating to a new GitLab environment, for example: - From a single-node GitLab to a scaled-out architecture. -- From a GitLab instance in your private datacenter to a cloud provider. +- From a GitLab instance in your private data center to a cloud provider. The rest of the document looks at some of the ways you can copy all your repositories from @@ -103,8 +101,8 @@ Using `rsync` to migrate Git data can cause data loss and repository corruption. If the target directory already contains a partial / outdated copy of the repositories it may be wasteful to copy all the data again with `tar`. In this scenario it is better to use `rsync`. This utility -is either already installed on your system or easily installable -via `apt`, `yum`, and so on. +is either already installed on your system, or installable +by using `apt` or `yum`. ```shell sudo -u git sh -c 'rsync -a --delete /var/opt/gitlab/git-data/repositories/. \ @@ -112,7 +110,7 @@ sudo -u git sh -c 'rsync -a --delete /var/opt/gitlab/git-data/repositories/. \ ``` The `/.` in the command above is very important, without it you can -easily get the wrong directory structure in the target directory. +get the wrong directory structure in the target directory. If you want to see progress, replace `-a` with `-av`. #### Single `rsync` to another server @@ -135,20 +133,23 @@ WARNING: Using `rsync` to migrate Git data can cause data loss and repository corruption. [These instructions are being reviewed](https://gitlab.com/gitlab-org/gitlab/-/issues/270422). -Every time you start an `rsync` job it has to inspect all files in -the source directory, all files in the target directory, and then -decide what files to copy or not. If the source or target directory -has many contents this startup phase of `rsync` can become a burden -for your GitLab server. In cases like this you can make `rsync`'s -life easier by dividing its work in smaller pieces, and sync one -repository at a time. +Every time you start an `rsync` job it must: + +- Inspect all files in the source directory. +- Inspect all files in the target directory. +- Decide whether or not to copy files. + +If the source or target directory +has many contents, this startup phase of `rsync` can become a burden +for your GitLab server. You can reduce the workload of `rsync` by dividing its +work in smaller pieces, and sync one repository at a time. In addition to `rsync` we use [GNU Parallel](http://www.gnu.org/software/parallel/). -This utility is not included in GitLab so you need to install it yourself with `apt` -or `yum`. Also note that the GitLab scripts we used below were added in GitLab 8.1. +This utility is not included in GitLab, so you must install it yourself with `apt` +or `yum`. -**This process does not clean up repositories at the target location that no -longer exist at the source.** +This process does not clean up repositories at the target location that no +longer exist at the source. #### Parallel `rsync` for all repositories known to GitLab @@ -218,8 +219,8 @@ Using `rsync` to migrate Git data can cause data loss and repository corruption. [These instructions are being reviewed](https://gitlab.com/gitlab-org/gitlab/-/issues/270422). Suppose you have already done one sync that started after 2015-10-1 12:00 UTC. -Then you might only want to sync repositories that were changed via GitLab -_after_ that time. You can use the `SINCE` variable to tell `rake +Then you might only want to sync repositories that were changed by using GitLab +after that time. You can use the `SINCE` variable to tell `rake gitlab:list_repos` to only print repositories with recent activity. ```shell diff --git a/doc/administration/operations/puma.md b/doc/administration/operations/puma.md index 44ac014650e..3b676010bfe 100644 --- a/doc/administration/operations/puma.md +++ b/doc/administration/operations/puma.md @@ -31,7 +31,7 @@ Unicorn in GitLab 14.0. When switching to Puma, Unicorn server configuration will _not_ carry over automatically, due to differences between the two application servers. For Omnibus-based deployments, see [Configuring Puma Settings](https://docs.gitlab.com/omnibus/settings/puma.html#configuring-puma-settings). -For Helm based deployments, see the [Webservice Chart documentation](https://docs.gitlab.com/charts/charts/gitlab/webservice/index.html). +For Helm based deployments, see the [`webservice` chart documentation](https://docs.gitlab.com/charts/charts/gitlab/webservice/index.html). Additionally we strongly recommend that multi-node deployments [configure their load balancers to use the readiness check](../load_balancer.md#readiness-check) due to a difference between Unicorn and Puma in how they handle connections during a restart of the service. diff --git a/doc/administration/operations/speed_up_ssh.md b/doc/administration/operations/speed_up_ssh.md deleted file mode 100644 index 2f3cf40a222..00000000000 --- a/doc/administration/operations/speed_up_ssh.md +++ /dev/null @@ -1,8 +0,0 @@ ---- -redirect_to: 'fast_ssh_key_lookup.md' ---- - -This document was moved to [another location](fast_ssh_key_lookup.md). - -<!-- This redirect file can be deleted after February 1, 2021. --> -<!-- Before deletion, see: https://docs.gitlab.com/ee/development/documentation/#move-or-rename-a-page --> |