diff options
Diffstat (limited to 'doc/administration/monitoring')
8 files changed, 54 insertions, 60 deletions
diff --git a/doc/administration/monitoring/gitlab_self_monitoring_project/index.md b/doc/administration/monitoring/gitlab_self_monitoring_project/index.md index 8d3c8555660..b1ec74c2f40 100644 --- a/doc/administration/monitoring/gitlab_self_monitoring_project/index.md +++ b/doc/administration/monitoring/gitlab_self_monitoring_project/index.md @@ -20,7 +20,8 @@ GitLab instance. All administrators at the time of creation of the project and group are added as maintainers of the group and project, and as an administrator, you can -add new members to the group to give them maintainer access to the project. +add new members to the group to give them the [Maintainer role](../../../user/permissions.md) for +the project. This project is used to self monitor your GitLab instance. The metrics dashboard of the project shows some basic resource usage charts, such as CPU and memory usage @@ -32,9 +33,12 @@ metrics exposed by the [GitLab exporter](../prometheus/gitlab_metrics.md#metrics ## Creating the self monitoring project -1. Go to **Admin Area > Settings > Metrics and profiling** and expand the **Self monitoring** section. +1. On the top bar, select **Menu >** **{admin}** **Admin**. +1. On the left sidebar, select **Settings > Metrics and profiling** and expand **Self monitoring**. 1. Toggle the **Create Project** button on. -1. Once your GitLab instance creates the project, GitLab displays a link to the project in the text above the **Create Project** toggle. You can also find it under **Projects > Your projects**. +1. After your GitLab instance creates the project, GitLab displays a link to the + project in the text above the **Create Project** toggle. You can also find it + from the top bar by selecting **Menu > Project**, then selecting **Your projects**. ## Deleting the self monitoring project @@ -42,7 +46,8 @@ WARNING: Deleting the self monitoring project removes any changes made to the project. If you create the project again, it's created in its default state. -1. Go to **Admin Area > Settings > Metrics and profiling** and expand the **Self monitoring** section. +1. On the top bar, select **Menu >** **{admin}** **Admin**. +1. On the left sidebar, go to **Settings > Metrics and profiling** and expand **Self monitoring**. 1. Toggle the **Create Project** button off. 1. In the confirmation dialog that opens, click **Delete project**. It can take a few seconds for it to be deleted. diff --git a/doc/administration/monitoring/performance/gitlab_configuration.md b/doc/administration/monitoring/performance/gitlab_configuration.md index 6e7557854ad..e8abe2708c7 100644 --- a/doc/administration/monitoring/performance/gitlab_configuration.md +++ b/doc/administration/monitoring/performance/gitlab_configuration.md @@ -9,7 +9,8 @@ info: To determine the technical writer assigned to the Stage/Group associated w GitLab Performance Monitoring is disabled by default. To enable it and change any of its settings: -1. Go to **Admin Area > Settings > Metrics and profiling** +1. On the top bar, select **Menu >** **{admin}** **Admin**. +1. On the left sidebar, select **Settings > Metrics and profiling** (`/admin/application_settings/metrics_and_profiling`). 1. Add the necessary configuration changes. 1. Restart all GitLab for the changes to take effect: diff --git a/doc/administration/monitoring/performance/grafana_configuration.md b/doc/administration/monitoring/performance/grafana_configuration.md index ac322f3e1ef..3b82b0e4bb8 100644 --- a/doc/administration/monitoring/performance/grafana_configuration.md +++ b/doc/administration/monitoring/performance/grafana_configuration.md @@ -62,8 +62,9 @@ repository. After setting up Grafana, you can enable a link to access it easily from the GitLab sidebar: -1. Go to the **Admin Area > Settings > Metrics and profiling**. -1. Expand **Metrics - Grafana**. +1. On the top bar, select **Menu >** **{admin}** **Admin**. +1. On the left sidebar, select **Settings > Metrics and profiling** + and expand **Metrics - Grafana**. 1. Check the **Enable access to Grafana** checkbox. 1. Configure the **Grafana URL**: - *If Grafana is enabled through Omnibus GitLab and on the same server,* @@ -71,7 +72,7 @@ GitLab sidebar: - *Otherwise,* enter the full URL of the Grafana instance. 1. Click **Save changes**. -GitLab displays your link in the **Admin Area > Monitoring > Metrics Dashboard**. +GitLab displays your link in the **Menu > Admin > Monitoring > Metrics Dashboard**. ## Security Update diff --git a/doc/administration/monitoring/performance/index.md b/doc/administration/monitoring/performance/index.md index 15c54a36f6c..f3db6ac9f03 100644 --- a/doc/administration/monitoring/performance/index.md +++ b/doc/administration/monitoring/performance/index.md @@ -69,6 +69,5 @@ The following environment variables are recognized: - `DATABASE_SAMPLER_INTERVAL_SECONDS` - `ACTION_CABLE_SAMPLER_INTERVAL_SECONDS` - `PUMA_SAMPLER_INTERVAL_SECONDS` -- `UNICORN_SAMPLER_INTERVAL_SECONDS` - `THREADS_SAMPLER_INTERVAL_SECONDS` - `GLOBAL_SEARCH_SAMPLER_INTERVAL_SECONDS` diff --git a/doc/administration/monitoring/performance/performance_bar.md b/doc/administration/monitoring/performance/performance_bar.md index dd43c7d6fbb..1125547f13f 100644 --- a/doc/administration/monitoring/performance/performance_bar.md +++ b/doc/administration/monitoring/performance/performance_bar.md @@ -6,7 +6,8 @@ info: To determine the technical writer assigned to the Stage/Group associated w # Performance Bar **(FREE SELF)** -> The **Stats** field [introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/271551) in GitLab SaaS 13.9. +> The **Stats** field [introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/271551) in GitLab 13.9. +> The **Memory** field [introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/330736) in GitLab 14.0. You can display the GitLab Performance Bar to see statistics for the performance of a page. When activated, it looks as follows: @@ -40,9 +41,11 @@ From left to right, it displays: Time until something was visible to the user. - [**DomContentLoaded**](https://developers.google.com/web/fundamentals/performance/critical-rendering-path/measure-crp) Event. - **Total number of requests** the page loaded. -- **Trace**: If Jaeger is integrated, **Trace** links to a Jaeger tracing page +- **Memory**: the amount of memory consumed and objects allocated during the selected request. + Select it to display a window with more details. +- **Trace**: if Jaeger is integrated, **Trace** links to a Jaeger tracing page with the current request's `correlation_id` included. -- **+**: A link to add a request's details to the performance bar. The request +- **+**: a link to add a request's details to the performance bar. The request can be added by its full URL (authenticated as the current user), or by the value of its `X-Request-Id` header. - **Download**: a link to download the raw JSON used to generate the Performance Bar reports. @@ -52,6 +55,11 @@ From left to right, it displays: - **Stats** (optional): if the `GITLAB_PERFORMANCE_BAR_STATS_URL` environment variable is set, this URL is displayed in the bar. In GitLab 13.9 and later, used only in GitLab SaaS. +NOTE: +Not all indicators are available in all environments. For instance, the memory view +requires to run Ruby with [specific patches](https://gitlab.com/gitlab-org/gitlab-build-images/-/blob/master/patches/ruby/2.7.2/thread-memory-allocations-2.7.patch) applied. +When running GitLab locally using the GDK this is typically not the case and the memory view cannot be used. + ## Request warnings Requests that exceed predefined limits display a warning **{warning}** icon and @@ -68,12 +76,13 @@ appears next to requests with warnings. ## Enable the Performance Bar via the Admin Area -The GitLab Performance Bar is disabled by default. To enable it for a given group: +The GitLab Performance Bar is disabled by default for non-administrators. To enable it +for a given group: 1. Sign in as a user with Administrator [permissions](../../../user/permissions.md). -1. In the menu bar, click **Admin Area**. -1. Go to **Settings > Metrics and profiling** - (`admin/application_settings/metrics_and_profiling`), and expand the section +1. On the top bar, select **Menu >** **{admin}** **Admin**. +1. On the left sidebar, select **Settings > Metrics and profiling** + (`admin/application_settings/metrics_and_profiling`), and expand **Profiling - Performance bar**. 1. Click **Enable access to the Performance Bar**. 1. In the **Allowed group** field, provide the full path of the group allowed diff --git a/doc/administration/monitoring/performance/request_profiling.md b/doc/administration/monitoring/performance/request_profiling.md index 15a58456e05..ebdca8d3960 100644 --- a/doc/administration/monitoring/performance/request_profiling.md +++ b/doc/administration/monitoring/performance/request_profiling.md @@ -8,7 +8,7 @@ info: To determine the technical writer assigned to the Stage/Group associated w To profile a request: -1. Sign in to GitLab as a user with Administrator or Maintainer [permissions](../../../user/permissions.md). +1. Sign in to GitLab as an Administrator or a user with the [Maintainer role](../../../user/permissions.md). 1. In the navigation bar, click **Admin area**. 1. Go to **Monitoring > Requests Profiles**. 1. In the **Requests Profiles** section, copy the token. diff --git a/doc/administration/monitoring/prometheus/gitlab_metrics.md b/doc/administration/monitoring/prometheus/gitlab_metrics.md index f29db9ead38..7e72f6ed7df 100644 --- a/doc/administration/monitoring/prometheus/gitlab_metrics.md +++ b/doc/administration/monitoring/prometheus/gitlab_metrics.md @@ -9,7 +9,8 @@ info: To determine the technical writer assigned to the Stage/Group associated w To enable the GitLab Prometheus metrics: 1. Log into GitLab as a user with [administrator permissions](../../../user/permissions.md). -1. Go to **Admin Area > Settings > Metrics and profiling**. +1. On the top bar, select **Menu >** **{admin}** **Admin**. +1. On the left sidebar, select **Settings > Metrics and profiling**. 1. Find the **Metrics - Prometheus** section, and click **Enable Prometheus Metrics**. 1. [Restart GitLab](../../restart_gitlab.md#omnibus-gitlab-restart) for the changes to take effect. @@ -58,7 +59,7 @@ The following metrics are available: | `gitlab_transaction_cache_duration_total` | Counter | 10.2 | Counter for total time (seconds) spent in Rails cache calls (aggregate) | | | `gitlab_transaction_cache_read_hit_count_total` | Counter | 10.2 | Counter for cache hits for Rails cache calls | `controller`, `action` | | `gitlab_transaction_cache_read_miss_count_total` | Counter | 10.2 | Counter for cache misses for Rails cache calls | `controller`, `action` | -| `gitlab_transaction_duration_seconds` | Histogram | 10.2 | Duration for all transactions (`gitlab_transaction_*` metrics) | `controller`, `action` | +| `gitlab_transaction_duration_seconds` | Histogram | 10.2 | Duration for successful requests (`gitlab_transaction_*` metrics) | `controller`, `action` | | `gitlab_transaction_event_build_found_total` | Counter | 9.4 | Counter for build found for API /jobs/request | | | `gitlab_transaction_event_build_invalid_total` | Counter | 9.4 | Counter for build invalid due to concurrency conflict for API /jobs/request | | | `gitlab_transaction_event_build_not_found_cached_total` | Counter | 9.4 | Counter for cached response of build not found for API /jobs/request | | @@ -91,7 +92,7 @@ The following metrics are available: | `gitlab_transaction_view_duration_total` | Counter | 9.4 | Duration for views | `controller`, `action`, `view` | | `gitlab_view_rendering_duration_seconds` | Histogram | 10.2 | Duration for views (histogram) | `controller`, `action`, `view` | | `http_requests_total` | Counter | 9.4 | Rack request count | `method`, `status` | -| `http_request_duration_seconds` | Histogram | 9.4 | HTTP response time from rack middleware | `method` | +| `http_request_duration_seconds` | Histogram | 9.4 | HTTP response time from rack middleware for successful requests | `method` | | `gitlab_transaction_db_count_total` | Counter | 13.1 | Counter for total number of SQL calls | `controller`, `action` | | `gitlab_transaction_db_<role>_count_total` | Counter | 13.10 | Counter for total number of SQL calls, grouped by database roles (primary/replica) | `controller`, `action` | | `gitlab_transaction_db_write_count_total` | Counter | 13.1 | Counter for total number of write SQL calls | `controller`, `action` | @@ -130,6 +131,8 @@ The following metrics are available: | `gitlab_ci_pipeline_security_orchestration_policy_processing_duration_seconds` | Histogram | 13.12 | Time in seconds it takes to process Security Policies in CI/CD pipeline | | | `gitlab_ci_difference_live_vs_actual_minutes` | Histogram | 13.12 | Difference between CI minute consumption counted while jobs were running (live) vs when jobs are complete (actual). Used to enforce CI minute consumption limits on long running jobs. | `plan` | | `gitlab_spamcheck_request_duration_seconds` | Histogram | 13.12 | The duration for requests between Rails and the anti-spam engine | | +| `service_desk_thank_you_email` | Counter | 14.0 | Total number of email responses to new service desk emails | | +| `service_desk_new_note_email` | Counter | 14.0 | Total number of email notifications on new service desk comment | | ## Metrics controlled by a feature flag @@ -227,11 +230,15 @@ configuration option in `gitlab.yml`. These metrics are served from the | `global_search_bulk_cron_queue_size` | Gauge | 12.10 | Number of database records waiting to be synchronized to Elasticsearch | | | `global_search_awaiting_indexing_queue_size` | Gauge | 13.2 | Number of database updates waiting to be synchronized to Elasticsearch while indexing is paused | | | `geo_merge_request_diffs` | Gauge | 13.4 | Number of merge request diffs on primary | `url` | -| `geo_merge_request_diffs_checksummed` | Gauge | 13.4 | Number of merge request diffs checksummed on primary | `url` | +| `geo_merge_request_diffs_checksum_total` | Gauge | 13.12 | Number of merge request diffs tried to checksum on primary | `url` | +| `geo_merge_request_diffs_checksummed` | Gauge | 13.4 | Number of merge request diffs successfully checksummed on primary | `url` | | `geo_merge_request_diffs_checksum_failed` | Gauge | 13.4 | Number of merge request diffs failed to calculate the checksum on primary | `url` | | `geo_merge_request_diffs_synced` | Gauge | 13.4 | Number of syncable merge request diffs synced on secondary | `url` | | `geo_merge_request_diffs_failed` | Gauge | 13.4 | Number of syncable merge request diffs failed to sync on secondary | `url` | | `geo_merge_request_diffs_registry` | Gauge | 13.4 | Number of merge request diffs in the registry | `url` | +| `geo_merge_request_diffs_verification_total` | Gauge | 13.12 | Number of merge request diffs verifications tried on secondary | `url` | +| `geo_merge_request_diffs_verified` | Gauge | 13.12 | Number of merge request diffs verified on secondary | `url` | +| `geo_merge_request_diffs_verification_failed` | Gauge | 13.12 | Number of merge request diffs verifications failed on secondary | `url` | | `geo_snippet_repositories` | Gauge | 13.4 | Number of snippets on primary | `url` | | `geo_snippet_repositories_checksummed` | Gauge | 13.4 | Number of snippets checksummed on primary | `url` | | `geo_snippet_repositories_checksum_failed` | Gauge | 13.4 | Number of snippets failed to calculate the checksum on primary | `url` | @@ -308,20 +315,8 @@ Some basic Ruby runtime metrics are available: | `ruby_process_proportional_memory_bytes` | Gauge | 13.0 | Memory usage by process (PSS/Proportional Set Size) | | `ruby_process_start_time_seconds` | Gauge | 12.0 | UNIX timestamp of process start time | -## Unicorn Metrics - -Unicorn specific metrics, when Unicorn is used. - -| Metric | Type | Since | Description | -|:-----------------------------|:------|:------|:---------------------------------------------------| -| `unicorn_active_connections` | Gauge | 11.0 | The number of active Unicorn connections (workers) | -| `unicorn_queued_connections` | Gauge | 11.0 | The number of queued Unicorn connections | -| `unicorn_workers` | Gauge | 12.0 | The number of Unicorn workers | - ## Puma Metrics -When Puma is used instead of Unicorn, the following metrics are available: - | Metric | Type | Since | Description | |:--------------------------------- |:------- |:----- |:----------- | | `puma_workers` | Gauge | 12.0 | Total number of workers | @@ -352,8 +347,8 @@ instance (`cache`, `shared_state` etc.). ## Metrics shared directory The GitLab Prometheus client requires a directory to store metrics data shared between multi-process services. -Those files are shared among all instances running under Unicorn server. -The directory must be accessible to all running Unicorn's processes, or +Those files are shared among all instances running under Puma server. +The directory must be accessible to all running Puma's processes, or metrics can't function correctly. This directory's location is configured using environment variable `prometheus_multiproc_dir`. diff --git a/doc/administration/monitoring/prometheus/index.md b/doc/administration/monitoring/prometheus/index.md index 035c5b3ee7e..dd402f800e3 100644 --- a/doc/administration/monitoring/prometheus/index.md +++ b/doc/administration/monitoring/prometheus/index.md @@ -122,44 +122,28 @@ The steps below are the minimum necessary to configure a Monitoring node running 1. Edit `/etc/gitlab/gitlab.rb` and add the contents: ```ruby + roles ['monitoring_role'] + external_url 'http://gitlab.example.com' - # Enable Prometheus - prometheus['enable'] = true + # Prometheus prometheus['listen_address'] = '0.0.0.0:9090' prometheus['monitor_kubernetes'] = false - # Enable Login form - grafana['disable_login_form'] = false - - # Enable Grafana + # Grafana grafana['enable'] = true grafana['admin_password'] = 'toomanysecrets' + grafana['disable_login_form'] = false # Enable service discovery for Prometheus consul['enable'] = true consul['monitoring_service_discovery'] = true - - # The addresses can be IPs or FQDNs - consul['configuration'] = { - retry_join: %w(10.0.0.1 10.0.0.2 10.0.0.3), + consul['configuration'] = { + retry_join: %w(10.0.0.1 10.0.0.2 10.0.0.3), # The addresses can be IPs or FQDNs } - # Disable all other services - gitlab_rails['auto_migrate'] = false - alertmanager['enable'] = false - gitaly['enable'] = false - gitlab_exporter['enable'] = false - gitlab_workhorse['enable'] = false + # Nginx - For Grafana access nginx['enable'] = true - postgres_exporter['enable'] = false - postgresql['enable'] = false - redis['enable'] = false - redis_exporter['enable'] = false - sidekiq['enable'] = false - puma['enable'] = false - node_exporter['enable'] = false - gitlab_exporter['enable'] = false ``` 1. Run `sudo gitlab-ctl reconfigure` to compile the configuration. @@ -227,7 +211,7 @@ To use an external Prometheus server: gitlab_rails['monitoring_whitelist'] = ['127.0.0.0/8', '192.168.0.1'] ``` -1. On **all** GitLab Rails(Puma/Unicorn, Sidekiq) servers, set the Prometheus server IP address and listen port. For example: +1. On **all** GitLab Rails(Puma, Sidekiq) servers, set the Prometheus server IP address and listen port. For example: ```ruby gitlab_rails['prometheus_address'] = '192.168.0.1:9090' |