diff options
Diffstat (limited to 'doc/administration')
23 files changed, 337 insertions, 366 deletions
diff --git a/doc/administration/auth/ldap/index.md b/doc/administration/auth/ldap/index.md index 466ae8e108c..4180c57e3c7 100644 --- a/doc/administration/auth/ldap/index.md +++ b/doc/administration/auth/ldap/index.md @@ -180,9 +180,16 @@ production: | `allow_username_or_email_login` | If enabled, GitLab ignores everything after the first `@` in the LDAP username submitted by the user on sign-in. If you are using `uid: 'userPrincipalName'` on ActiveDirectory you need to disable this setting, because the userPrincipalName contains an `@`. | no | boolean | | `block_auto_created_users` | To maintain tight control over the number of billable users on your GitLab installation, enable this setting to keep new users blocked until they have been cleared by an administrator (default: false). | no | boolean | | `base` | Base where we can search for users. | yes | `'ou=people,dc=gitlab,dc=example'` or `'DC=mydomain,DC=com'` | -| `user_filter` | Filter LDAP users. Format: [RFC 4515](https://tools.ietf.org/search/rfc4515) Note: GitLab does not support `omniauth-ldap`'s custom filter syntax. | no | `'(employeeType=developer)'` or `'(&(objectclass=user)(|(samaccountname=momo)(samaccountname=toto)))'` | +| `user_filter` | Filter LDAP users. Format: [RFC 4515](https://tools.ietf.org/search/rfc4515) Note: GitLab does not support `omniauth-ldap`'s custom filter syntax. | no | For examples, read [Examples of user filters](#examples-of-user-filters). | | `lowercase_usernames` | If lowercase_usernames is enabled, GitLab converts the name to lower case. | no | boolean | +#### Examples of user filters + +Some examples of the `user_filter` field syntax: + +- `'(employeeType=developer)'` +- `'(&(objectclass=user)(|(samaccountname=momo)(samaccountname=toto)))'` + ### SSL Configuration Settings **(FREE SELF)** | Setting | Description | Required | Examples | diff --git a/doc/administration/consul.md b/doc/administration/consul.md index dfc859e30c2..926267a414a 100644 --- a/doc/administration/consul.md +++ b/doc/administration/consul.md @@ -146,7 +146,7 @@ sudo gitlab-ctl restart consul ### Consul nodes unable to communicate By default, Consul will attempt to -[bind](https://www.consul.io/docs/agent/options.html#_bind) to `0.0.0.0`, but +[bind](https://www.consul.io/docs/agent/options#_bind) to `0.0.0.0`, but it will advertise the first private IP address on the node for other Consul nodes to communicate with it. If the other nodes cannot communicate with a node on this address, then the cluster will have a failed status. diff --git a/doc/administration/gitaly/index.md b/doc/administration/gitaly/index.md index f02b9b8fc1a..7bb99bd1018 100644 --- a/doc/administration/gitaly/index.md +++ b/doc/administration/gitaly/index.md @@ -816,7 +816,7 @@ exceed 1, and the concurrency limiter has no effect. ## Background Repository Optimization -Empty directories and unneeded config settings may accumulate in a repository and +Empty directories and unneeded configuration settings may accumulate in a repository and slow down Git operations. Gitaly can schedule a daily background task with a maximum duration to clean up these items and improve performance. diff --git a/doc/administration/integration/plantuml.md b/doc/administration/integration/plantuml.md index dbbe17cccc8..834f4047fdd 100644 --- a/doc/administration/integration/plantuml.md +++ b/doc/administration/integration/plantuml.md @@ -5,22 +5,94 @@ info: "To determine the technical writer assigned to the Stage/Group associated type: reference, howto --- -# PlantUML & GitLab **(FREE)** - -> [Introduced](https://gitlab.com/gitlab-org/gitlab-foss/-/merge_requests/8537) in GitLab 8.16. +# PlantUML and GitLab **(FREE)** When [PlantUML](https://plantuml.com) integration is enabled and configured in -GitLab you can create diagrams in AsciiDoc and Markdown documents -created in snippets, wikis, and repositories. +GitLab, you can create diagrams in snippets, wikis, and repositories. To set up +the integration, you must: + +1. [Configure your PlantUML server](#configure-your-plantuml-server). +1. [Configure local PlantUML access](#configure-local-plantuml-access). +1. [Configure PlantUML security](#configure-plantuml-security). +1. [Enable the integration](#enable-plantuml-integration). + +After completing the integration, PlantUML converts `plantuml` +blocks to an HTML image tag, with the source pointing to the PlantUML instance. The PlantUML +diagram delimiters `@startuml`/`@enduml` aren't required, as these are replaced +by the `plantuml` block: + +- **Markdown** + + ````markdown + ```plantuml + Bob -> Alice : hello + Alice -> Bob : hi + ``` + ```` + +- **AsciiDoc** + + ```plaintext + [plantuml, format="png", id="myDiagram", width="200px"] + ---- + Bob->Alice : hello + Alice -> Bob : hi + ---- + ``` + +- **reStructuredText** -## PlantUML Server + ```plaintext + .. plantuml:: + :caption: Caption with **bold** and *italic* + + Bob -> Alice: hello + Alice -> Bob: hi + ``` -Before you can enable PlantUML in GitLab; set up your own PlantUML -server to generate the diagrams. + Although you can use the `uml::` directive for compatibility with + [`sphinxcontrib-plantuml`](https://pypi.org/project/sphinxcontrib-plantuml/), + GitLab supports only the `caption` option. + +If the PlantUML server is correctly configured, these examples should render a +diagram instead of the code block: + +```plantuml +Bob -> Alice : hello +Alice -> Bob : hi +``` + +Inside the block you can add any of the diagrams PlantUML supports, such as: + +- [Activity](https://plantuml.com/activity-diagram-legacy) +- [Class](https://plantuml.com/class-diagram) +- [Component](https://plantuml.com/component-diagram) +- [Object](https://plantuml.com/object-diagram) +- [Sequence](https://plantuml.com/sequence-diagram) +- [State](https://plantuml.com/state-diagram) +- [Use Case](https://plantuml.com/use-case-diagram) + +You can add parameters to block definitions: + +- `format`: Can be either `png` (default) or `svg`. Use `svg` with care, as it's + not supported by all browsers, and isn't supported by Markdown. +- `id`: A CSS ID added to the diagram HTML tag. +- `width`: Width attribute added to the image tag. +- `height`: Height attribute added to the image tag. + +Markdown does not support any parameters, and always uses PNG format. + +## Configure your PlantUML server + +Before you can enable PlantUML in GitLab, set up your own PlantUML +server to generate the diagrams: + +- [In Docker](#docker). +- [In Debian/Ubuntu](#debianubuntu). ### Docker -With Docker, you can just run a container like this: +To run a PlantUML container in Docker, run this command: ```shell docker run -d --name plantuml -p 8080:8080 plantuml/plantuml-server:tomcat @@ -29,8 +101,9 @@ docker run -d --name plantuml -p 8080:8080 plantuml/plantuml-server:tomcat The **PlantUML URL** is the hostname of the server running the container. When running GitLab in Docker, it must have access to the PlantUML container. -You can achieve that by using [Docker Compose](https://docs.docker.com/compose/). -A basic `docker-compose.yml` file could contain: +To achieve that, use [Docker Compose](https://docs.docker.com/compose/). +In this basic `docker-compose.yml` file, PlantUML is accessible to GitLab at the URL +`http://plantuml:8080/`: ```yaml version: "3" @@ -46,37 +119,32 @@ services: container_name: plantuml ``` -In this scenario, PlantUML is accessible to GitLab at the URL -`http://plantuml:8080/`. - ### Debian/Ubuntu -You can also install and configure a PlantUML server in Debian/Ubuntu distributions using Tomcat. +You can install and configure a PlantUML server in Debian/Ubuntu distributions +using Tomcat: -First you need to create a `plantuml.war` file from the source code: +1. Run these commands to create a `plantuml.war` file from the source code: -```shell -sudo apt-get install graphviz openjdk-8-jdk git-core maven -git clone https://github.com/plantuml/plantuml-server.git -cd plantuml-server -mvn package -``` + ```shell + sudo apt-get install graphviz openjdk-8-jdk git-core maven + git clone https://github.com/plantuml/plantuml-server.git + cd plantuml-server + mvn package + ``` -The above sequence of commands generates a `.war` file you can deploy with Tomcat: +1. Deploy the `.war` file from the previous step with these commands: -```shell -sudo apt-get install tomcat8 -sudo cp target/plantuml.war /var/lib/tomcat8/webapps/plantuml.war -sudo chown tomcat8:tomcat8 /var/lib/tomcat8/webapps/plantuml.war -sudo service tomcat8 restart -``` + ```shell + sudo apt-get install tomcat8 + sudo cp target/plantuml.war /var/lib/tomcat8/webapps/plantuml.war + sudo chown tomcat8:tomcat8 /var/lib/tomcat8/webapps/plantuml.war + sudo service tomcat8 restart + ``` -After the Tomcat service restarts, the PlantUML service is ready and -listening for requests on port 8080: - -```plaintext -http://localhost:8080/plantuml -``` +The Tomcat service should restart. After the restart is complete, the +PlantUML service is ready and listening for requests on port 8080: +`http://localhost:8080/plantuml` To change these defaults, edit the `/etc/tomcat8/server.xml` file. @@ -85,40 +153,44 @@ The default URL is different when using this approach. The Docker-based image makes the service available at the root URL, with no relative path. Adjust the configuration below accordingly. -### Making local PlantUML accessible using custom GitLab setup +## Configure local PlantUML access -The PlantUML server runs locally on your server, so it is not accessible -externally by default. As such, it is necessary to catch external PlantUML -calls and redirect them to the local server. +The PlantUML server runs locally on your server, so it can't be accessed +externally by default. Your server must catch external PlantUML +calls to `https://gitlab.example.com/-/plantuml/` and redirect them to the +local PlantUML server. Depending on your setup, the URL is either of the +following: -The idea is to redirect each call to `https://gitlab.example.com/-/plantuml/` -to the local PlantUML server `http://plantuml:8080/` or `http://localhost:8080/plantuml/`, depending on your setup. +- `http://plantuml:8080/` +- `http://localhost:8080/plantuml/` -To enable the redirection, add the following line in `/etc/gitlab/gitlab.rb`: +If you're running [GitLab with TLS](https://docs.gitlab.com/omnibus/settings/ssl.html) +you must configure this redirection, because PlantUML uses the insecure HTTP protocol. +Newer browsers such as [Google Chrome 86+](https://www.chromestatus.com/feature/4926989725073408) +don't load insecure HTTP resources on pages served over HTTPS. -```ruby -# Docker deployment -nginx['custom_gitlab_server_config'] = "location /-/plantuml/ { \n proxy_cache off; \n proxy_pass http://plantuml:8080/; \n}\n" +To enable this redirection: -# Built from source -nginx['custom_gitlab_server_config'] = "location /-/plantuml { \n rewrite ^/-/(plantuml.*) /$1 break;\n proxy_cache off; \n proxy_pass http://localhost:8080/plantuml; \n}\n" -``` +1. Add the following line in `/etc/gitlab/gitlab.rb`, depending on your setup method: -To activate the changes, run the following command: + ```ruby + # Docker deployment + nginx['custom_gitlab_server_config'] = "location /-/plantuml/ { \n proxy_cache off; \n proxy_pass http://plantuml:8080/; \n}\n" -```shell -sudo gitlab-ctl reconfigure -``` + # Built from source + nginx['custom_gitlab_server_config'] = "location /-/plantuml { \n rewrite ^/-/(plantuml.*) /$1 break;\n proxy_cache off; \n proxy_pass http://localhost:8080/plantuml; \n}\n" + ``` -Note that the redirection through GitLab must be configured -when running [GitLab with TLS](https://docs.gitlab.com/omnibus/settings/ssl.html) -due to PlantUML's use of the insecure HTTP protocol. Newer browsers such -as [Google Chrome 86+](https://www.chromestatus.com/feature/4926989725073408) -do not load insecure HTTP resources on a page served over HTTPS. +1. To activate the changes, run the following command: -### Security + ```shell + sudo gitlab-ctl reconfigure + ``` -PlantUML has features that allow fetching network resources. +### Configure PlantUML security + +PlantUML has features that allow fetching network resources. If you self-host the +PlantUML server, put network controls in place to isolate it. ```plaintext @startuml @@ -129,98 +201,29 @@ stop; @enduml ``` -**If you self-host the PlantUML server, network controls should be put in place to isolate it.** - -## GitLab +## Enable PlantUML integration -You need to enable PlantUML integration from Settings under Admin Area. To do -that, sign in with an Administrator account, and then do following: +After configuring your local PlantUML server, you're ready to enable the PlantUML integration: -1. In GitLab, go to **Admin Area > Settings > General**. -1. Expand the **PlantUML** section. +1. Sign in to GitLab as an [Administrator](../../user/permissions.md) user. +1. In the top menu, click **{admin}** **Admin Area**. +1. In the left sidebar, go to **Settings > General** and expand the **PlantUML** section. 1. Select the **Enable PlantUML** check box. -1. Set the PlantUML instance as `https://gitlab.example.com/-/plantuml/`. - -NOTE: -If you are using a PlantUML server running v1.2020.9 and -above (for example, [plantuml.com](https://plantuml.com)), set the `PLANTUML_ENCODING` -environment variable to enable the `deflate` compression. On Omnibus GitLab, -this can be set in `/etc/gitlab.rb`: - -```ruby -gitlab_rails['env'] = { 'PLANTUML_ENCODING' => 'deflate' } -``` - -From GitLab 13.1 and later, PlantUML integration now -[requires a header prefix in the URL](https://github.com/plantuml/plantuml/issues/117#issuecomment-6235450160) -to distinguish different encoding types. - -## Creating Diagrams - -With PlantUML integration enabled and configured, we can start adding diagrams to -our AsciiDoc snippets, wikis, and repositories using delimited blocks: - -- **Markdown** - - ````markdown - ```plantuml - Bob -> Alice : hello - Alice -> Bob : hi - ``` - ```` - -- **AsciiDoc** - - ```plaintext - [plantuml, format="png", id="myDiagram", width="200px"] - ---- - Bob->Alice : hello - Alice -> Bob : hi - ---- - ``` - -- **reStructuredText** - - ```plaintext - .. plantuml:: - :caption: Caption with **bold** and *italic* - - Bob -> Alice: hello - Alice -> Bob: hi - ``` +1. Set the PlantUML instance as `https://gitlab.example.com/-/plantuml/`, + and click **Save changes**. - You can also use the `uml::` directive for compatibility with - [`sphinxcontrib-plantuml`](https://pypi.org/project/sphinxcontrib-plantuml/), - but GitLab only supports the `caption` option. +Depending on your PlantUML and GitLab version numbers, you may also need to take +these steps: -The above blocks are converted to an HTML image tag with source pointing to the -PlantUML instance. If the PlantUML server is correctly configured, this should -render a nice diagram instead of the block: +- For PlantUML servers running v1.2020.9 and above, such as [plantuml.com](https://plantuml.com), + you must set the `PLANTUML_ENCODING` environment variable to enable the `deflate` + compression. In Omnibus GitLab, you can set this value in `/etc/gitlab.rb` with + this command: -```plantuml -Bob -> Alice : hello -Alice -> Bob : hi -``` - -Inside the block you can add any of the diagrams PlantUML supports, such as: - -- [Sequence](https://plantuml.com/sequence-diagram) -- [Use Case](https://plantuml.com/use-case-diagram) -- [Class](https://plantuml.com/class-diagram) -- [Activity](https://plantuml.com/activity-diagram-legacy) -- [Component](https://plantuml.com/component-diagram) -- [State](https://plantuml.com/state-diagram), -- [Object](https://plantuml.com/object-diagram) - -You do not need to use the PlantUML -diagram delimiters `@startuml`/`@enduml`, as these are replaced by the AsciiDoc `plantuml` block. - -Some parameters can be added to the AsciiDoc block definition: - -- `format`: Can be either `png` or `svg`. Note that `svg` is not supported by - all browsers so use with care. The default is `png`. -- `id`: A CSS ID added to the diagram HTML tag. -- `width`: Width attribute added to the image tag. -- `height`: Height attribute added to the image tag. + ```ruby + gitlab_rails['env'] = { 'PLANTUML_ENCODING' => 'deflate' } + ``` -Markdown does not support any parameters and always uses PNG format. +- For GitLab versions 13.1 and later, PlantUML integration now + [requires a header prefix in the URL](https://github.com/plantuml/plantuml/issues/117#issuecomment-6235450160) + to distinguish different encoding types. diff --git a/doc/administration/integration/terminal.md b/doc/administration/integration/terminal.md index 0f26eb83d17..644e2d905ae 100644 --- a/doc/administration/integration/terminal.md +++ b/doc/administration/integration/terminal.md @@ -4,7 +4,7 @@ group: Ecosystem info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/engineering/ux/technical-writing/#assignments --- -# Web terminals +# Web terminals **(FREE)** > [Introduced](https://gitlab.com/gitlab-org/gitlab-foss/-/merge_requests/7690) in GitLab 8.15. diff --git a/doc/administration/invalidate_markdown_cache.md b/doc/administration/invalidate_markdown_cache.md index 211316534ee..7a880c81843 100644 --- a/doc/administration/invalidate_markdown_cache.md +++ b/doc/administration/invalidate_markdown_cache.md @@ -1,6 +1,6 @@ --- -stage: Create -group: Source Code +stage: Plan +group: Project Management info: "To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/engineering/ux/technical-writing/#assignments" type: reference --- diff --git a/doc/administration/issue_closing_pattern.md b/doc/administration/issue_closing_pattern.md index 32d367bb15e..6b1922fe142 100644 --- a/doc/administration/issue_closing_pattern.md +++ b/doc/administration/issue_closing_pattern.md @@ -1,6 +1,6 @@ --- stage: Create -group: Source Code +group: Code Review info: "To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/engineering/ux/technical-writing/#assignments" type: reference --- diff --git a/doc/administration/monitoring/index.md b/doc/administration/monitoring/index.md index ac09acf2d2e..d2e5f40b171 100644 --- a/doc/administration/monitoring/index.md +++ b/doc/administration/monitoring/index.md @@ -11,9 +11,19 @@ Explore our features to monitor your GitLab instance: - [GitLab self-monitoring](gitlab_self_monitoring_project/index.md): The GitLab instance administration project helps to monitor the GitLab instance and take action on alerts. -- [Performance monitoring](performance/index.md): GitLab Performance Monitoring makes it possible to measure a wide variety of statistics of your instance. -- [Prometheus](prometheus/index.md): Prometheus is a powerful time-series monitoring service, providing a flexible platform for monitoring GitLab and other software products. -- [GitHub imports](github_imports.md): Monitor the health and progress of the GitHub importer with various Prometheus metrics. -- [Monitoring uptime](../../user/admin_area/monitoring/health_check.md): Check the server status using the health check endpoint. - - [IP whitelists](ip_whitelist.md): Configure GitLab for monitoring endpoints that provide health check information when probed. -- [`nginx_status`](https://docs.gitlab.com/omnibus/settings/nginx.html#enablingdisabling-nginx_status): Monitor your NGINX server status +- [Performance monitoring](performance/index.md): GitLab Performance Monitoring + makes it possible to measure a wide variety of statistics of your instance. +- [Prometheus](prometheus/index.md): Prometheus is a powerful time-series monitoring + service, providing a flexible platform for monitoring GitLab and other software + products. +- [GitHub imports](github_imports.md): Monitor the health and progress of the GitHub + importer with various Prometheus metrics. +- [Monitoring uptime](../../user/admin_area/monitoring/health_check.md): Check the + server status using the health check endpoint. + - [IP whitelists](ip_whitelist.md): Configure GitLab for monitoring endpoints that + provide health check information when probed. +- [`nginx_status`](https://docs.gitlab.com/omnibus/settings/nginx.html#enablingdisabling-nginx_status): + Monitor your NGINX server status. +- [Auto Monitoring](../../topics/autodevops/stages.md#auto-monitoring): Automated + monitoring for your application’s server and response metrics, provided by + [Auto DevOps](../../topics/autodevops/index.md). diff --git a/doc/administration/monitoring/performance/performance_bar.md b/doc/administration/monitoring/performance/performance_bar.md index 0516cbc2f8b..ff1bdbe3bd8 100644 --- a/doc/administration/monitoring/performance/performance_bar.md +++ b/doc/administration/monitoring/performance/performance_bar.md @@ -6,6 +6,8 @@ info: To determine the technical writer assigned to the Stage/Group associated w # Performance Bar **(FREE SELF)** +> The **Stats** field [introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/271551) in GitLab SaaS 13.9. + You can display the GitLab Performance Bar to see statistics for the performance of a page. When activated, it looks as follows: @@ -53,6 +55,8 @@ From left to right, it displays: - **Request Selector**: a select box displayed on the right-hand side of the Performance Bar which enables you to view these metrics for any requests made while the current page was open. Only the first two requests per unique URL are captured. +- **Stats** (optional): if the `GITLAB_PERFORMANCE_BAR_STATS_URL` environment variable is set, + this URL is displayed in the bar. In GitLab 13.9 and later, used only in GitLab SaaS. ## Request warnings diff --git a/doc/administration/object_storage.md b/doc/administration/object_storage.md index 3cad18dc497..a14f902db91 100644 --- a/doc/administration/object_storage.md +++ b/doc/administration/object_storage.md @@ -570,13 +570,15 @@ See the following additional guides: ## Warnings, limitations, and known issues -### Separate buckets required when using Helm +### Use separate buckets -Generally, using the same bucket for your Object Storage is fine to do -for convenience. +Using separate buckets for each data type is the recommended approach for GitLab. +This ensures there are no collisions across the various types of data GitLab stores. +There are plans to [enable the use of a single bucket](https://gitlab.com/gitlab-org/gitlab/-/issues/292958) +in the future. -However, if you're using or planning to use Helm, separate buckets will -be required as there is a [known limitation with restorations of Helm chart backups](https://docs.gitlab.com/charts/advanced/external-object-storage/#lfs-artifacts-uploads-packages-external-diffs-pseudonymizer). +Helm-based installs require separate buckets to +[handle backup restorations](https://docs.gitlab.com/charts/advanced/external-object-storage/#lfs-artifacts-uploads-packages-external-diffs-pseudonymizer) ### S3 API compatibility issues diff --git a/doc/administration/packages/dependency_proxy.md b/doc/administration/packages/dependency_proxy.md index 16cfdc8c784..720734bf344 100644 --- a/doc/administration/packages/dependency_proxy.md +++ b/doc/administration/packages/dependency_proxy.md @@ -47,6 +47,10 @@ To enable the dependency proxy feature: Since Puma is already the default web server for installations from source as of GitLab 12.9, no further changes are needed. +**Multi-node GitLab installations** + +Follow the steps for **Omnibus GitLab installation** for each Web and Sidekiq nodes. + ## Changing the storage path By default, the dependency proxy files are stored locally, but you can change the default diff --git a/doc/administration/packages/index.md b/doc/administration/packages/index.md index b1d2a82e31a..012afd6a419 100644 --- a/doc/administration/packages/index.md +++ b/doc/administration/packages/index.md @@ -43,14 +43,14 @@ guides you through the process. | CocoaPods | [#36890](https://gitlab.com/gitlab-org/gitlab/-/issues/36890) | | Conda | [#36891](https://gitlab.com/gitlab-org/gitlab/-/issues/36891) | | CRAN | [#36892](https://gitlab.com/gitlab-org/gitlab/-/issues/36892) | -| Debian | [WIP: Merge Request](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/44746) | +| Debian | [Draft: Merge Request](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/50438) | | Opkg | [#36894](https://gitlab.com/gitlab-org/gitlab/-/issues/36894) | | P2 | [#36895](https://gitlab.com/gitlab-org/gitlab/-/issues/36895) | | Puppet | [#36897](https://gitlab.com/gitlab-org/gitlab/-/issues/36897) | | RPM | [#5932](https://gitlab.com/gitlab-org/gitlab/-/issues/5932) | | RubyGems | [#803](https://gitlab.com/gitlab-org/gitlab/-/issues/803) | | SBT | [#36898](https://gitlab.com/gitlab-org/gitlab/-/issues/36898) | -| Terraform | [WIP: Merge Request](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/18834) | +| Terraform | [Draft: Merge Request](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/18834) | | Vagrant | [#36899](https://gitlab.com/gitlab-org/gitlab/-/issues/36899) | <!-- vale gitlab.Spelling = YES --> diff --git a/doc/administration/pages/index.md b/doc/administration/pages/index.md index 655e35c3e60..4febd8648d1 100644 --- a/doc/administration/pages/index.md +++ b/doc/administration/pages/index.md @@ -178,13 +178,16 @@ outside world. pages_external_url 'https://example.io' pages_nginx['redirect_http_to_https'] = true + ``` + +1. If you haven’t named your certificate and key `example.io.crt` and `example.io.key` +then you'll need to also add the full paths as shown below: + + ```ruby pages_nginx['ssl_certificate'] = "/etc/gitlab/ssl/pages-nginx.crt" pages_nginx['ssl_certificate_key'] = "/etc/gitlab/ssl/pages-nginx.key" ``` - where `pages-nginx.crt` and `pages-nginx.key` are the SSL cert and key, - respectively. - 1. [Reconfigure GitLab](../restart_gitlab.md#omnibus-gitlab-reconfigure). ### Additional configuration for Docker container @@ -720,7 +723,7 @@ preferred source is `gitlab`, which uses [API-based configuration](#gitlab-api-b For more details see this [blog post](https://about.gitlab.com/blog/2020/08/03/how-gitlab-pages-uses-the-gitlab-api-to-serve-content/). -### Deprecated domain_config_source +### Deprecated `domain_config_source` WARNING: The flag `gitlab_pages['domain_config_source']` is deprecated for use in [GitLab 13.9](https://gitlab.com/gitlab-org/gitlab/-/issues/217913), diff --git a/doc/administration/raketasks/github_import.md b/doc/administration/raketasks/github_import.md index c29865be56c..0338732e886 100644 --- a/doc/administration/raketasks/github_import.md +++ b/doc/administration/raketasks/github_import.md @@ -19,8 +19,8 @@ before/after the brackets. Also, some shells (for example, `zsh`) can interpret ## Caveats -If the GitHub [rate limit](https://docs.github.com/v3/#rate-limiting) is reached while importing, -the importing process waits (`sleep()`) until it can continue importing. +If the GitHub [rate limit](https://docs.github.com/en/rest/reference/rate-limit) is reached while +importing, the importing process waits (`sleep()`) until it can continue importing. ## Importing multiple projects diff --git a/doc/administration/reference_architectures/10k_users.md b/doc/administration/reference_architectures/10k_users.md index d4d522ab1b8..d6a38e1b713 100644 --- a/doc/administration/reference_architectures/10k_users.md +++ b/doc/administration/reference_architectures/10k_users.md @@ -17,23 +17,23 @@ full list of reference architectures, see | Service | Nodes | Configuration | GCP | AWS | Azure | |--------------------------------------------|-------------|-------------------------|-----------------|-------------|----------| -| External load balancing node | 1 | 2 vCPU, 1.8 GB memory | n1-highcpu-2 | c5.large | F2s v2 | -| Consul | 3 | 2 vCPU, 1.8 GB memory | n1-highcpu-2 | c5.large | F2s v2 | -| PostgreSQL | 3 | 8 vCPU, 30 GB memory | n1-standard-8 | m5.2xlarge | D8s v3 | -| PgBouncer | 3 | 2 vCPU, 1.8 GB memory | n1-highcpu-2 | c5.large | F2s v2 | -| Internal load balancing node | 1 | 2 vCPU, 1.8 GB memory | n1-highcpu-2 | c5.large | F2s v2 | -| Redis - Cache | 3 | 4 vCPU, 15 GB memory | n1-standard-4 | m5.xlarge | D4s v3 | -| Redis - Queues / Shared State | 3 | 4 vCPU, 15 GB memory | n1-standard-4 | m5.xlarge | D4s v3 | -| Redis Sentinel - Cache | 3 | 1 vCPU, 1.7 GB memory | g1-small | t3.small | B1MS | -| Redis Sentinel - Queues / Shared State | 3 | 1 vCPU, 1.7 GB memory | g1-small | t3.small | B1MS | -| Gitaly Cluster | 3 | 16 vCPU, 60 GB memory | n1-standard-16 | m5.4xlarge | D16s v3 | -| Praefect | 3 | 2 vCPU, 1.8 GB memory | n1-highcpu-2 | c5.large | F2s v2 | -| Praefect PostgreSQL | 1+* | 2 vCPU, 1.8 GB memory | n1-highcpu-2 | c5.large | F2s v2 | -| Sidekiq | 4 | 4 vCPU, 15 GB memory | n1-standard-4 | m5.xlarge | D4s v3 | -| GitLab Rails | 3 | 32 vCPU, 28.8 GB memory | n1-highcpu-32 | c5.9xlarge | F32s v2 | -| Monitoring node | 1 | 4 vCPU, 3.6 GB memory | n1-highcpu-4 | c5.xlarge | F4s v2 | +| External load balancing node | 1 | 2 vCPU, 1.8 GB memory | n1-highcpu-2 | `c5.large` | F2s v2 | +| Consul | 3 | 2 vCPU, 1.8 GB memory | n1-highcpu-2 | `c5.large` | F2s v2 | +| PostgreSQL | 3 | 8 vCPU, 30 GB memory | n1-standard-8 | `m5.2xlarge` | D8s v3 | +| PgBouncer | 3 | 2 vCPU, 1.8 GB memory | n1-highcpu-2 | `c5.large` | F2s v2 | +| Internal load balancing node | 1 | 2 vCPU, 1.8 GB memory | n1-highcpu-2 | `c5.large` | F2s v2 | +| Redis - Cache | 3 | 4 vCPU, 15 GB memory | n1-standard-4 | `m5.xlarge` | D4s v3 | +| Redis - Queues / Shared State | 3 | 4 vCPU, 15 GB memory | n1-standard-4 | `m5.xlarge` | D4s v3 | +| Redis Sentinel - Cache | 3 | 1 vCPU, 1.7 GB memory | g1-small | `t3.small` | B1MS | +| Redis Sentinel - Queues / Shared State | 3 | 1 vCPU, 1.7 GB memory | g1-small | `t3.small` | B1MS | +| Gitaly Cluster | 3 | 16 vCPU, 60 GB memory | n1-standard-16 | `m5.4xlarge` | D16s v3 | +| Praefect | 3 | 2 vCPU, 1.8 GB memory | n1-highcpu-2 | `c5.large` | F2s v2 | +| Praefect PostgreSQL | 1+* | 2 vCPU, 1.8 GB memory | n1-highcpu-2 | `c5.large` | F2s v2 | +| Sidekiq | 4 | 4 vCPU, 15 GB memory | n1-standard-4 | `m5.xlarge` | D4s v3 | +| GitLab Rails | 3 | 32 vCPU, 28.8 GB memory | n1-highcpu-32 | `c5.9xlarge` | F32s v2 | +| Monitoring node | 1 | 4 vCPU, 3.6 GB memory | n1-highcpu-4 | `c5.xlarge` | F4s v2 | | Object storage | n/a | n/a | n/a | n/a | n/a | -| NFS server | 1 | 4 vCPU, 3.6 GB memory | n1-highcpu-4 | c5.xlarge | F4s v2 | +| NFS server | 1 | 4 vCPU, 3.6 GB memory | n1-highcpu-4 | `c5.xlarge` | F4s v2 | ```plantuml @startuml 10k @@ -1508,7 +1508,7 @@ Praefect requires several secret tokens to secure communications across the Clus Gitaly Cluster nodes are configured in Praefect via a `virtual storage`. Each storage contains the details of each Gitaly node that makes up the cluster. Each storage is also given a name -and this name is used in several areas of the config. In this guide, the name of the storage will be +and this name is used in several areas of the configuration. In this guide, the name of the storage will be `default`. Also, this guide is geared towards new installs, if upgrading an existing environment to use Gitaly Cluster, you may need to use a different name. Refer to the [Praefect documentation](../gitaly/praefect.md#praefect) for more info. @@ -1756,7 +1756,7 @@ Note the following: necessary. - The Internal Load Balancer will also access to the certificates and need to be configured - to allow for TLS passthrough. + to allow for TLS pass-through. Refer to the load balancers documentation on how to configure this. To configure Praefect with TLS: @@ -1940,13 +1940,13 @@ To configure the Sidekiq nodes, on each one: 'google_project' => '<gcp-project-name>', 'google_json_key_location' => '<path-to-gcp-service-account-key>' } - gitlab_rails['object_store']['objects']['artifacts']['bucket'] = "<gcp-bucket-name>" - gitlab_rails['object_store']['objects']['external_diffs']['bucket'] = "<gcp-bucket-name>" - gitlab_rails['object_store']['objects']['lfs']['bucket'] = "<gcp-bucket-name>" - gitlab_rails['object_store']['objects']['uploads']['bucket'] = "<gcp-bucket-name>" - gitlab_rails['object_store']['objects']['packages']['bucket'] = "<gcp-bucket-name>" - gitlab_rails['object_store']['objects']['dependency_proxy']['bucket'] = "<gcp-bucket-name>" - gitlab_rails['object_store']['objects']['terraform_state']['bucket'] = "<gcp-bucket-name>" + gitlab_rails['object_store']['objects']['artifacts']['bucket'] = "<gcp-artifacts-bucket-name>" + gitlab_rails['object_store']['objects']['external_diffs']['bucket'] = "<gcp-external-diffs-bucket-name>" + gitlab_rails['object_store']['objects']['lfs']['bucket'] = "<gcp-lfs-bucket-name>" + gitlab_rails['object_store']['objects']['uploads']['bucket'] = "<gcp-uploads-bucket-name>" + gitlab_rails['object_store']['objects']['packages']['bucket'] = "<gcp-packages-bucket-name>" + gitlab_rails['object_store']['objects']['dependency_proxy']['bucket'] = "<gcp-dependency-proxy-bucket-name>" + gitlab_rails['object_store']['objects']['terraform_state']['bucket'] = "<gcp-terraform-state-bucket-name>" ``` 1. Copy the `/etc/gitlab/gitlab-secrets.json` file from your Consul server, and replace @@ -2069,13 +2069,13 @@ On each node perform the following: 'google_project' => '<gcp-project-name>', 'google_json_key_location' => '<path-to-gcp-service-account-key>' } - gitlab_rails['object_store']['objects']['artifacts']['bucket'] = "<gcp-bucket-name>" - gitlab_rails['object_store']['objects']['external_diffs']['bucket'] = "<gcp-bucket-name>" - gitlab_rails['object_store']['objects']['lfs']['bucket'] = "<gcp-bucket-name>" - gitlab_rails['object_store']['objects']['uploads']['bucket'] = "<gcp-bucket-name>" - gitlab_rails['object_store']['objects']['packages']['bucket'] = "<gcp-bucket-name>" - gitlab_rails['object_store']['objects']['dependency_proxy']['bucket'] = "<gcp-bucket-name>" - gitlab_rails['object_store']['objects']['terraform_state']['bucket'] = "<gcp-bucket-name>" + gitlab_rails['object_store']['objects']['artifacts']['bucket'] = "<gcp-artifacts-bucket-name>" + gitlab_rails['object_store']['objects']['external_diffs']['bucket'] = "<gcp-external-diffs-bucket-name>" + gitlab_rails['object_store']['objects']['lfs']['bucket'] = "<gcp-lfs-bucket-name>" + gitlab_rails['object_store']['objects']['uploads']['bucket'] = "<gcp-uploads-bucket-name>" + gitlab_rails['object_store']['objects']['packages']['bucket'] = "<gcp-packages-bucket-name>" + gitlab_rails['object_store']['objects']['dependency_proxy']['bucket'] = "<gcp-dependency-proxy-bucket-name>" + gitlab_rails['object_store']['objects']['terraform_state']['bucket'] = "<gcp-terraform-state-bucket-name>" ``` 1. Save the file and [reconfigure GitLab](../restart_gitlab.md#omnibus-gitlab-reconfigure). @@ -2303,20 +2303,9 @@ on what features you intend to use: | [Terraform state files](../terraform_state.md#using-object-storage) | Yes | Using separate buckets for each data type is the recommended approach for GitLab. - -A limitation of our configuration is that each use of object storage is separately configured. -[We have an issue for improving this](https://gitlab.com/gitlab-org/gitlab/-/issues/23345) -and easily using one bucket with separate folders is one improvement that this might bring. - -There is at least one specific issue with using the same bucket: -when GitLab is deployed with the Helm chart restore from backup -[will not properly function](https://docs.gitlab.com/charts/advanced/external-object-storage/#lfs-artifacts-uploads-packages-external-diffs-pseudonymizer) -unless separate buckets are used. - -One risk of using a single bucket would be if your organization decided to -migrate GitLab to the Helm deployment in the future. GitLab would run, but the situation with -backups might not be realized until the organization had a critical requirement for the backups to -work. +This ensures there are no collisions across the various types of data GitLab stores. +There are plans to [enable the use of a single bucket](https://gitlab.com/gitlab-org/gitlab/-/issues/292958) +in the future. <div align="right"> <a type="button" class="btn btn-default" href="#setup-components"> diff --git a/doc/administration/reference_architectures/25k_users.md b/doc/administration/reference_architectures/25k_users.md index 48c72bb930d..d02f7ea66ac 100644 --- a/doc/administration/reference_architectures/25k_users.md +++ b/doc/administration/reference_architectures/25k_users.md @@ -1637,13 +1637,13 @@ To configure the Sidekiq nodes, on each one: 'google_project' => '<gcp-project-name>', 'google_json_key_location' => '<path-to-gcp-service-account-key>' } - gitlab_rails['object_store']['objects']['artifacts']['bucket'] = "<gcp-bucket-name>" - gitlab_rails['object_store']['objects']['external_diffs']['bucket'] = "<gcp-bucket-name>" - gitlab_rails['object_store']['objects']['lfs']['bucket'] = "<gcp-bucket-name>" - gitlab_rails['object_store']['objects']['uploads']['bucket'] = "<gcp-bucket-name>" - gitlab_rails['object_store']['objects']['packages']['bucket'] = "<gcp-bucket-name>" - gitlab_rails['object_store']['objects']['dependency_proxy']['bucket'] = "<gcp-bucket-name>" - gitlab_rails['object_store']['objects']['terraform_state']['bucket'] = "<gcp-bucket-name>" + gitlab_rails['object_store']['objects']['artifacts']['bucket'] = "<gcp-artifacts-bucket-name>" + gitlab_rails['object_store']['objects']['external_diffs']['bucket'] = "<gcp-external-diffs-bucket-name>" + gitlab_rails['object_store']['objects']['lfs']['bucket'] = "<gcp-lfs-bucket-name>" + gitlab_rails['object_store']['objects']['uploads']['bucket'] = "<gcp-uploads-bucket-name>" + gitlab_rails['object_store']['objects']['packages']['bucket'] = "<gcp-packages-bucket-name>" + gitlab_rails['object_store']['objects']['dependency_proxy']['bucket'] = "<gcp-dependency-proxy-bucket-name>" + gitlab_rails['object_store']['objects']['terraform_state']['bucket'] = "<gcp-terraform-state-bucket-name>" ``` 1. Copy the `/etc/gitlab/gitlab-secrets.json` file from your Consul server, and replace @@ -1769,13 +1769,13 @@ On each node perform the following: 'google_project' => '<gcp-project-name>', 'google_json_key_location' => '<path-to-gcp-service-account-key>' } - gitlab_rails['object_store']['objects']['artifacts']['bucket'] = "<gcp-bucket-name>" - gitlab_rails['object_store']['objects']['external_diffs']['bucket'] = "<gcp-bucket-name>" - gitlab_rails['object_store']['objects']['lfs']['bucket'] = "<gcp-bucket-name>" - gitlab_rails['object_store']['objects']['uploads']['bucket'] = "<gcp-bucket-name>" - gitlab_rails['object_store']['objects']['packages']['bucket'] = "<gcp-bucket-name>" - gitlab_rails['object_store']['objects']['dependency_proxy']['bucket'] = "<gcp-bucket-name>" - gitlab_rails['object_store']['objects']['terraform_state']['bucket'] = "<gcp-bucket-name>" + gitlab_rails['object_store']['objects']['artifacts']['bucket'] = "<gcp-artifacts-bucket-name>" + gitlab_rails['object_store']['objects']['external_diffs']['bucket'] = "<gcp-external-diffs-bucket-name>" + gitlab_rails['object_store']['objects']['lfs']['bucket'] = "<gcp-lfs-bucket-name>" + gitlab_rails['object_store']['objects']['uploads']['bucket'] = "<gcp-uploads-bucket-name>" + gitlab_rails['object_store']['objects']['packages']['bucket'] = "<gcp-packages-bucket-name>" + gitlab_rails['object_store']['objects']['dependency_proxy']['bucket'] = "<gcp-dependency-proxy-bucket-name>" + gitlab_rails['object_store']['objects']['terraform_state']['bucket'] = "<gcp-terraform-state-bucket-name>" ``` 1. Save the file and [reconfigure GitLab](../restart_gitlab.md#omnibus-gitlab-reconfigure). @@ -2002,20 +2002,9 @@ on what features you intend to use: | [Terraform state files](../terraform_state.md#using-object-storage) | Yes | Using separate buckets for each data type is the recommended approach for GitLab. - -A limitation of our configuration is that each use of object storage is separately configured. -[We have an issue for improving this](https://gitlab.com/gitlab-org/gitlab/-/issues/23345) -and easily using one bucket with separate folders is one improvement that this might bring. - -There is at least one specific issue with using the same bucket: -when GitLab is deployed with the Helm chart restore from backup -[will not properly function](https://docs.gitlab.com/charts/advanced/external-object-storage/#lfs-artifacts-uploads-packages-external-diffs-pseudonymizer) -unless separate buckets are used. - -One risk of using a single bucket would be if your organization decided to -migrate GitLab to the Helm deployment in the future. GitLab would run, but the situation with -backups might not be realized until the organization had a critical requirement for the backups to -work. +This ensures there are no collisions across the various types of data GitLab stores. +There are plans to [enable the use of a single bucket](https://gitlab.com/gitlab-org/gitlab/-/issues/292958) +in the future. <div align="right"> <a type="button" class="btn btn-default" href="#setup-components"> diff --git a/doc/administration/reference_architectures/2k_users.md b/doc/administration/reference_architectures/2k_users.md index 9ad6054104a..62f0c993213 100644 --- a/doc/administration/reference_architectures/2k_users.md +++ b/doc/administration/reference_architectures/2k_users.md @@ -668,13 +668,13 @@ On each node perform the following: 'google_project' => '<gcp-project-name>', 'google_json_key_location' => '<path-to-gcp-service-account-key>' } - gitlab_rails['object_store']['objects']['artifacts']['bucket'] = "<gcp-bucket-name>" - gitlab_rails['object_store']['objects']['external_diffs']['bucket'] = "<gcp-bucket-name>" - gitlab_rails['object_store']['objects']['lfs']['bucket'] = "<gcp-bucket-name>" - gitlab_rails['object_store']['objects']['uploads']['bucket'] = "<gcp-bucket-name>" - gitlab_rails['object_store']['objects']['packages']['bucket'] = "<gcp-bucket-name>" - gitlab_rails['object_store']['objects']['dependency_proxy']['bucket'] = "<gcp-bucket-name>" - gitlab_rails['object_store']['objects']['terraform_state']['bucket'] = "<gcp-bucket-name>" + gitlab_rails['object_store']['objects']['artifacts']['bucket'] = "<gcp-artifacts-bucket-name>" + gitlab_rails['object_store']['objects']['external_diffs']['bucket'] = "<gcp-external-diffs-bucket-name>" + gitlab_rails['object_store']['objects']['lfs']['bucket'] = "<gcp-lfs-bucket-name>" + gitlab_rails['object_store']['objects']['uploads']['bucket'] = "<gcp-uploads-bucket-name>" + gitlab_rails['object_store']['objects']['packages']['bucket'] = "<gcp-packages-bucket-name>" + gitlab_rails['object_store']['objects']['dependency_proxy']['bucket'] = "<gcp-dependency-proxy-bucket-name>" + gitlab_rails['object_store']['objects']['terraform_state']['bucket'] = "<gcp-terraform-state-bucket-name>" ## Uncomment and edit the following options if you have set up NFS ## @@ -920,18 +920,9 @@ on what features you intend to use: | [Terraform state files](../terraform_state.md#using-object-storage) | Yes | Using separate buckets for each data type is the recommended approach for GitLab. - -A limitation of our configuration is that each use of object storage is -separately configured. We have an [issue](https://gitlab.com/gitlab-org/gitlab/-/issues/23345) -for improving this, which would allow for one bucket with separate folders. - -Using a single bucket when GitLab is deployed with the Helm chart causes -restoring from a backup to -[not function properly](https://docs.gitlab.com/charts/advanced/external-object-storage/#lfs-artifacts-uploads-packages-external-diffs-pseudonymizer). -Although you may not be using a Helm deployment right now, if you migrate -GitLab to a Helm deployment later, GitLab would still work, but you may not -realize backups aren't working correctly until a critical requirement for -functioning backups is encountered. +This ensures there are no collisions across the various types of data GitLab stores. +There are plans to [enable the use of a single bucket](https://gitlab.com/gitlab-org/gitlab/-/issues/292958) +in the future. <div align="right"> <a type="button" class="btn btn-default" href="#setup-components"> diff --git a/doc/administration/reference_architectures/3k_users.md b/doc/administration/reference_architectures/3k_users.md index 175c4318d78..593bfaf7282 100644 --- a/doc/administration/reference_architectures/3k_users.md +++ b/doc/administration/reference_architectures/3k_users.md @@ -1322,13 +1322,14 @@ To configure the Sidekiq nodes, one each one: 'google_project' => '<gcp-project-name>', 'google_json_key_location' => '<path-to-gcp-service-account-key>' } - gitlab_rails['object_store']['objects']['artifacts']['bucket'] = "<gcp-bucket-name>" - gitlab_rails['object_store']['objects']['external_diffs']['bucket'] = "<gcp-bucket-name>" - gitlab_rails['object_store']['objects']['lfs']['bucket'] = "<gcp-bucket-name>" - gitlab_rails['object_store']['objects']['uploads']['bucket'] = "<gcp-bucket-name>" - gitlab_rails['object_store']['objects']['packages']['bucket'] = "<gcp-bucket-name>" - gitlab_rails['object_store']['objects']['dependency_proxy']['bucket'] = "<gcp-bucket-name>" - gitlab_rails['object_store']['objects']['terraform_state']['bucket'] = "<gcp-bucket-name>" + gitlab_rails['object_store']['objects']['artifacts']['bucket'] = "<gcp-artifacts-bucket-name>" + gitlab_rails['object_store']['objects']['external_diffs']['bucket'] = "<gcp-external-diffs-bucket-name>" + gitlab_rails['object_store']['objects']['lfs']['bucket'] = "<gcp-lfs-bucket-name>" + gitlab_rails['object_store']['objects']['uploads']['bucket'] = "<gcp-uploads-bucket-name>" + gitlab_rails['object_store']['objects']['packages']['bucket'] = "<gcp-packages-bucket-name>" + gitlab_rails['object_store']['objects']['dependency_proxy']['bucket'] = "<gcp-dependency-proxy-bucket-name>" + gitlab_rails['object_store']['objects']['terraform_state']['bucket'] = "<gcp-terraform-state-bucket-name>" + ``` 1. Save the file and [reconfigure GitLab](../restart_gitlab.md#omnibus-gitlab-reconfigure). @@ -1489,13 +1490,13 @@ On each node perform the following: 'google_project' => '<gcp-project-name>', 'google_json_key_location' => '<path-to-gcp-service-account-key>' } - gitlab_rails['object_store']['objects']['artifacts']['bucket'] = "<gcp-bucket-name>" - gitlab_rails['object_store']['objects']['external_diffs']['bucket'] = "<gcp-bucket-name>" - gitlab_rails['object_store']['objects']['lfs']['bucket'] = "<gcp-bucket-name>" - gitlab_rails['object_store']['objects']['uploads']['bucket'] = "<gcp-bucket-name>" - gitlab_rails['object_store']['objects']['packages']['bucket'] = "<gcp-bucket-name>" - gitlab_rails['object_store']['objects']['dependency_proxy']['bucket'] = "<gcp-bucket-name>" - gitlab_rails['object_store']['objects']['terraform_state']['bucket'] = "<gcp-bucket-name>" + gitlab_rails['object_store']['objects']['artifacts']['bucket'] = "<gcp-artifacts-bucket-name>" + gitlab_rails['object_store']['objects']['external_diffs']['bucket'] = "<gcp-external-diffs-bucket-name>" + gitlab_rails['object_store']['objects']['lfs']['bucket'] = "<gcp-lfs-bucket-name>" + gitlab_rails['object_store']['objects']['uploads']['bucket'] = "<gcp-uploads-bucket-name>" + gitlab_rails['object_store']['objects']['packages']['bucket'] = "<gcp-packages-bucket-name>" + gitlab_rails['object_store']['objects']['dependency_proxy']['bucket'] = "<gcp-dependency-proxy-bucket-name>" + gitlab_rails['object_store']['objects']['terraform_state']['bucket'] = "<gcp-terraform-state-bucket-name>" ``` 1. If you're using [Gitaly with TLS support](#gitaly-tls-support), make sure the @@ -1697,20 +1698,9 @@ on what features you intend to use: | [Terraform state files](../terraform_state.md#using-object-storage) | Yes | Using separate buckets for each data type is the recommended approach for GitLab. - -A limitation of our configuration is that each use of object storage is separately configured. -[We have an issue for improving this](https://gitlab.com/gitlab-org/gitlab/-/issues/23345) -and easily using one bucket with separate folders is one improvement that this might bring. - -There is at least one specific issue with using the same bucket: -when GitLab is deployed with the Helm chart restore from backup -[will not properly function](https://docs.gitlab.com/charts/advanced/external-object-storage/#lfs-artifacts-uploads-packages-external-diffs-pseudonymizer) -unless separate buckets are used. - -One risk of using a single bucket would be if your organization decided to -migrate GitLab to the Helm deployment in the future. GitLab would run, but the situation with -backups might not be realized until the organization had a critical requirement for the backups to -work. +This ensures there are no collisions across the various types of data GitLab stores. +There are plans to [enable the use of a single bucket](https://gitlab.com/gitlab-org/gitlab/-/issues/292958) +in the future. <div align="right"> <a type="button" class="btn btn-default" href="#setup-components"> diff --git a/doc/administration/reference_architectures/50k_users.md b/doc/administration/reference_architectures/50k_users.md index 0b22a2d3602..04ce39046d8 100644 --- a/doc/administration/reference_architectures/50k_users.md +++ b/doc/administration/reference_architectures/50k_users.md @@ -1637,13 +1637,13 @@ To configure the Sidekiq nodes, on each one: 'google_project' => '<gcp-project-name>', 'google_json_key_location' => '<path-to-gcp-service-account-key>' } - gitlab_rails['object_store']['objects']['artifacts']['bucket'] = "<gcp-bucket-name>" - gitlab_rails['object_store']['objects']['external_diffs']['bucket'] = "<gcp-bucket-name>" - gitlab_rails['object_store']['objects']['lfs']['bucket'] = "<gcp-bucket-name>" - gitlab_rails['object_store']['objects']['uploads']['bucket'] = "<gcp-bucket-name>" - gitlab_rails['object_store']['objects']['packages']['bucket'] = "<gcp-bucket-name>" - gitlab_rails['object_store']['objects']['dependency_proxy']['bucket'] = "<gcp-bucket-name>" - gitlab_rails['object_store']['objects']['terraform_state']['bucket'] = "<gcp-bucket-name>" + gitlab_rails['object_store']['objects']['artifacts']['bucket'] = "<gcp-artifacts-bucket-name>" + gitlab_rails['object_store']['objects']['external_diffs']['bucket'] = "<gcp-external-diffs-bucket-name>" + gitlab_rails['object_store']['objects']['lfs']['bucket'] = "<gcp-lfs-bucket-name>" + gitlab_rails['object_store']['objects']['uploads']['bucket'] = "<gcp-uploads-bucket-name>" + gitlab_rails['object_store']['objects']['packages']['bucket'] = "<gcp-packages-bucket-name>" + gitlab_rails['object_store']['objects']['dependency_proxy']['bucket'] = "<gcp-dependency-proxy-bucket-name>" + gitlab_rails['object_store']['objects']['terraform_state']['bucket'] = "<gcp-terraform-state-bucket-name>" ``` 1. Copy the `/etc/gitlab/gitlab-secrets.json` file from your Consul server, and replace @@ -1769,13 +1769,13 @@ On each node perform the following: 'google_project' => '<gcp-project-name>', 'google_json_key_location' => '<path-to-gcp-service-account-key>' } - gitlab_rails['object_store']['objects']['artifacts']['bucket'] = "<gcp-bucket-name>" - gitlab_rails['object_store']['objects']['external_diffs']['bucket'] = "<gcp-bucket-name>" - gitlab_rails['object_store']['objects']['lfs']['bucket'] = "<gcp-bucket-name>" - gitlab_rails['object_store']['objects']['uploads']['bucket'] = "<gcp-bucket-name>" - gitlab_rails['object_store']['objects']['packages']['bucket'] = "<gcp-bucket-name>" - gitlab_rails['object_store']['objects']['dependency_proxy']['bucket'] = "<gcp-bucket-name>" - gitlab_rails['object_store']['objects']['terraform_state']['bucket'] = "<gcp-bucket-name>" + gitlab_rails['object_store']['objects']['artifacts']['bucket'] = "<gcp-artifacts-bucket-name>" + gitlab_rails['object_store']['objects']['external_diffs']['bucket'] = "<gcp-external-diffs-bucket-name>" + gitlab_rails['object_store']['objects']['lfs']['bucket'] = "<gcp-lfs-bucket-name>" + gitlab_rails['object_store']['objects']['uploads']['bucket'] = "<gcp-uploads-bucket-name>" + gitlab_rails['object_store']['objects']['packages']['bucket'] = "<gcp-packages-bucket-name>" + gitlab_rails['object_store']['objects']['dependency_proxy']['bucket'] = "<gcp-dependency-proxy-bucket-name>" + gitlab_rails['object_store']['objects']['terraform_state']['bucket'] = "<gcp-terraform-state-bucket-name>" ``` 1. Save the file and [reconfigure GitLab](../restart_gitlab.md#omnibus-gitlab-reconfigure). @@ -2002,20 +2002,9 @@ on what features you intend to use: | [Terraform state files](../terraform_state.md#using-object-storage) | Yes | Using separate buckets for each data type is the recommended approach for GitLab. - -A limitation of our configuration is that each use of object storage is separately configured. -[We have an issue for improving this](https://gitlab.com/gitlab-org/gitlab/-/issues/23345) -and easily using one bucket with separate folders is one improvement that this might bring. - -There is at least one specific issue with using the same bucket: -when GitLab is deployed with the Helm chart restore from backup -[will not properly function](https://docs.gitlab.com/charts/advanced/external-object-storage/#lfs-artifacts-uploads-packages-external-diffs-pseudonymizer) -unless separate buckets are used. - -One risk of using a single bucket would be if your organization decided to -migrate GitLab to the Helm deployment in the future. GitLab would run, but the situation with -backups might not be realized until the organization had a critical requirement for the backups to -work. +This ensures there are no collisions across the various types of data GitLab stores. +There are plans to [enable the use of a single bucket](https://gitlab.com/gitlab-org/gitlab/-/issues/292958) +in the future. <div align="right"> <a type="button" class="btn btn-default" href="#setup-components"> diff --git a/doc/administration/reference_architectures/5k_users.md b/doc/administration/reference_architectures/5k_users.md index 37d35c299fa..37e67b0ab73 100644 --- a/doc/administration/reference_architectures/5k_users.md +++ b/doc/administration/reference_architectures/5k_users.md @@ -1319,13 +1319,13 @@ To configure the Sidekiq nodes, one each one: 'google_project' => '<gcp-project-name>', 'google_json_key_location' => '<path-to-gcp-service-account-key>' } - gitlab_rails['object_store']['objects']['artifacts']['bucket'] = "<gcp-bucket-name>" - gitlab_rails['object_store']['objects']['external_diffs']['bucket'] = "<gcp-bucket-name>" - gitlab_rails['object_store']['objects']['lfs']['bucket'] = "<gcp-bucket-name>" - gitlab_rails['object_store']['objects']['uploads']['bucket'] = "<gcp-bucket-name>" - gitlab_rails['object_store']['objects']['packages']['bucket'] = "<gcp-bucket-name>" - gitlab_rails['object_store']['objects']['dependency_proxy']['bucket'] = "<gcp-bucket-name>" - gitlab_rails['object_store']['objects']['terraform_state']['bucket'] = "<gcp-bucket-name>" + gitlab_rails['object_store']['objects']['artifacts']['bucket'] = "<gcp-artifacts-bucket-name>" + gitlab_rails['object_store']['objects']['external_diffs']['bucket'] = "<gcp-external-diffs-bucket-name>" + gitlab_rails['object_store']['objects']['lfs']['bucket'] = "<gcp-lfs-bucket-name>" + gitlab_rails['object_store']['objects']['uploads']['bucket'] = "<gcp-uploads-bucket-name>" + gitlab_rails['object_store']['objects']['packages']['bucket'] = "<gcp-packages-bucket-name>" + gitlab_rails['object_store']['objects']['dependency_proxy']['bucket'] = "<gcp-dependency-proxy-bucket-name>" + gitlab_rails['object_store']['objects']['terraform_state']['bucket'] = "<gcp-terraform-state-bucket-name>" ``` 1. Save the file and [reconfigure GitLab](../restart_gitlab.md#omnibus-gitlab-reconfigure). @@ -1471,13 +1471,13 @@ On each node perform the following: 'google_project' => '<gcp-project-name>', 'google_json_key_location' => '<path-to-gcp-service-account-key>' } - gitlab_rails['object_store']['objects']['artifacts']['bucket'] = "<gcp-bucket-name>" - gitlab_rails['object_store']['objects']['external_diffs']['bucket'] = "<gcp-bucket-name>" - gitlab_rails['object_store']['objects']['lfs']['bucket'] = "<gcp-bucket-name>" - gitlab_rails['object_store']['objects']['uploads']['bucket'] = "<gcp-bucket-name>" - gitlab_rails['object_store']['objects']['packages']['bucket'] = "<gcp-bucket-name>" - gitlab_rails['object_store']['objects']['dependency_proxy']['bucket'] = "<gcp-bucket-name>" - gitlab_rails['object_store']['objects']['terraform_state']['bucket'] = "<gcp-bucket-name>" + gitlab_rails['object_store']['objects']['artifacts']['bucket'] = "<gcp-artifacts-bucket-name>" + gitlab_rails['object_store']['objects']['external_diffs']['bucket'] = "<gcp-external-diffs-bucket-name>" + gitlab_rails['object_store']['objects']['lfs']['bucket'] = "<gcp-lfs-bucket-name>" + gitlab_rails['object_store']['objects']['uploads']['bucket'] = "<gcp-uploads-bucket-name>" + gitlab_rails['object_store']['objects']['packages']['bucket'] = "<gcp-packages-bucket-name>" + gitlab_rails['object_store']['objects']['dependency_proxy']['bucket'] = "<gcp-dependency-proxy-bucket-name>" + gitlab_rails['object_store']['objects']['terraform_state']['bucket'] = "<gcp-terraform-state-bucket-name>" ## Uncomment and edit the following options if you have set up NFS ## @@ -1694,20 +1694,9 @@ on what features you intend to use: | [Terraform state files](../terraform_state.md#using-object-storage) | Yes | Using separate buckets for each data type is the recommended approach for GitLab. - -A limitation of our configuration is that each use of object storage is separately configured. -[We have an issue for improving this](https://gitlab.com/gitlab-org/gitlab/-/issues/23345) -and easily using one bucket with separate folders is one improvement that this might bring. - -There is at least one specific issue with using the same bucket: -when GitLab is deployed with the Helm chart restore from backup -[will not properly function](https://docs.gitlab.com/charts/advanced/external-object-storage/#lfs-artifacts-uploads-packages-external-diffs-pseudonymizer) -unless separate buckets are used. - -One risk of using a single bucket would be if your organization decided to -migrate GitLab to the Helm deployment in the future. GitLab would run, but the situation with -backups might not be realized until the organization had a critical requirement for the backups to -work. +This ensures there are no collisions across the various types of data GitLab stores. +There are plans to [enable the use of a single bucket](https://gitlab.com/gitlab-org/gitlab/-/issues/292958) +in the future. <div align="right"> <a type="button" class="btn btn-default" href="#setup-components"> diff --git a/doc/administration/repository_checks.md b/doc/administration/repository_checks.md index 404a7bd77c1..059acbd12f8 100644 --- a/doc/administration/repository_checks.md +++ b/doc/administration/repository_checks.md @@ -5,7 +5,7 @@ info: "To determine the technical writer assigned to the Stage/Group associated type: reference --- -# Repository checks +# Repository checks **(FREE)** > [Introduced](https://gitlab.com/gitlab-org/gitlab-foss/-/merge_requests/3232) in GitLab 8.7. diff --git a/doc/administration/repository_storage_paths.md b/doc/administration/repository_storage_paths.md index 5884a8b3283..6f373089bf2 100644 --- a/doc/administration/repository_storage_paths.md +++ b/doc/administration/repository_storage_paths.md @@ -23,7 +23,7 @@ several mount points. For example: ```ruby git_data_dirs({ 'default' => { 'gitaly_address' => 'tcp://gitaly1.internal:8075' }, - 'storage1' => { 'gitaly_address' => 'tcp://gitaly2.internal:8075' }, + 'storage2' => { 'gitaly_address' => 'tcp://gitaly2.internal:8075' }, }) ``` @@ -31,18 +31,23 @@ several mount points. For example: ```plaintext default: - path: /mnt/git-storage-1 + gitaly_address: tcp://gitaly1.example:8075 storage2: - path: /mnt/git-storage-2 + gitaly_address: tcp://gitaly2.example:8075 ``` -For more information on +For more information on: - Configuring Gitaly, see [Configure Gitaly](gitaly/index.md#configure-gitaly). - Configuring direct repository access, see the following section below. ## Configure repository storage paths +WARNING: +The following information is for configuring GitLab to directly access repositories. This +configuration option is deprecated in favor of using [Gitaly](gitaly/index.md) and is scheduled to +[be removed in GitLab 14.0](https://gitlab.com/gitlab-org/gitaly/-/issues/1690). + To configure repository storage paths: 1. Edit the necessary configuration files: @@ -96,10 +101,6 @@ For compatibility reasons `gitlab.yml` has a different structure than Omnibus Gi example. Then Omnibus GitLab creates a `repositories` directory under that path to use with `gitlab.yml`. -NOTE: -This example uses NFS. We do not recommend using EFS for storage as it may impact GitLab performance. -Read the [relevant documentation](nfs.md#avoid-using-awss-elastic-file-system-efs) for more details. - **For installations from source** 1. Edit `gitlab.yml` and add the storage paths: @@ -111,10 +112,10 @@ Read the [relevant documentation](nfs.md#avoid-using-awss-elastic-file-system-ef storages: # You must have at least a 'default' repository storage path. default: path: /home/git/repositories - nfs_1: - path: /mnt/nfs1/repositories - nfs_2: - path: /mnt/nfs2/repositories + storage1: + path: /mnt/storage1/repositories + storage2: + path: /mnt/storage2/repositories ``` 1. [Restart GitLab](restart_gitlab.md#installations-from-source) for the changes to take effect. @@ -126,8 +127,8 @@ Edit `/etc/gitlab/gitlab.rb` by appending the rest of the paths to the default o ```ruby git_data_dirs({ "default" => { "path" => "/var/opt/gitlab/git-data" }, - "nfs_1" => { "path" => "/mnt/nfs1/git-data" }, - "nfs_2" => { "path" => "/mnt/nfs2/git-data" } + "storage1" => { "path" => "/mnt/storage1/git-data" }, + "storage2" => { "path" => "/mnt/storage2/git-data" } }) ``` diff --git a/doc/administration/static_objects_external_storage.md b/doc/administration/static_objects_external_storage.md index 0f3dab743e9..5d26ff7cad6 100644 --- a/doc/administration/static_objects_external_storage.md +++ b/doc/administration/static_objects_external_storage.md @@ -5,7 +5,7 @@ info: "To determine the technical writer assigned to the Stage/Group associated type: reference --- -# Static objects external storage +# Static objects external storage **(FREE)** > [Introduced](https://gitlab.com/gitlab-org/gitlab-foss/-/merge_requests/31025) in GitLab 12.3. |