diff options
Diffstat (limited to 'doc/administration/monitoring')
18 files changed, 106 insertions, 23 deletions
diff --git a/doc/administration/monitoring/github_imports.md b/doc/administration/monitoring/github_imports.md index e91483eb79d..e16e9bb0336 100644 --- a/doc/administration/monitoring/github_imports.md +++ b/doc/administration/monitoring/github_imports.md @@ -1,6 +1,6 @@ --- stage: Monitor -group: Monitor +group: Respond info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/engineering/ux/technical-writing/#assignments --- diff --git a/doc/administration/monitoring/gitlab_self_monitoring_project/index.md b/doc/administration/monitoring/gitlab_self_monitoring_project/index.md index 1cf4e5a25ba..911d76a89e3 100644 --- a/doc/administration/monitoring/gitlab_self_monitoring_project/index.md +++ b/doc/administration/monitoring/gitlab_self_monitoring_project/index.md @@ -1,13 +1,17 @@ --- stage: Monitor -group: Monitor +group: Respond info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/engineering/ux/technical-writing/#assignments --- -# Self monitoring project **(FREE SELF)** +# Self-monitoring project (DEPRECATED) **(FREE SELF)** -> - [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/32351) in GitLab 12.7, behind a disabled feature flag (`self_monitoring_project`). -> - The feature flag was removed and the self monitoring project was [made generally available](https://gitlab.com/gitlab-org/gitlab/-/issues/198511) in GitLab 12.8. +> - [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/32351) in GitLab 12.7 [with a flag](../../feature_flags.md) named `self_monitoring_project`. Disabled by default. +> - Generally available in GitLab 12.8. [Feature flag `self_monitoring_project`](https://gitlab.com/gitlab-org/gitlab/-/issues/198511) removed. +> - [Deprecated](https://gitlab.com/gitlab-org/gitlab/-/issues/348909) in GitLab 14.8. + +WARNING: +This feature was [deprecated](https://gitlab.com/gitlab-org/gitlab/-/issues/348909) in GitLab 14.8. GitLab provides administrators insights into the health of their GitLab instance. diff --git a/doc/administration/monitoring/index.md b/doc/administration/monitoring/index.md index 4c49efd6bd5..df655053723 100644 --- a/doc/administration/monitoring/index.md +++ b/doc/administration/monitoring/index.md @@ -1,6 +1,6 @@ --- stage: Monitor -group: Monitor +group: Respond info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/engineering/ux/technical-writing/#assignments --- diff --git a/doc/administration/monitoring/ip_whitelist.md b/doc/administration/monitoring/ip_whitelist.md index 75b09f8a366..b8347ba3f0d 100644 --- a/doc/administration/monitoring/ip_whitelist.md +++ b/doc/administration/monitoring/ip_whitelist.md @@ -1,6 +1,6 @@ --- stage: Monitor -group: Monitor +group: Respond info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/engineering/ux/technical-writing/#assignments --- diff --git a/doc/administration/monitoring/performance/gitlab_configuration.md b/doc/administration/monitoring/performance/gitlab_configuration.md index f316a75a868..128ddad6555 100644 --- a/doc/administration/monitoring/performance/gitlab_configuration.md +++ b/doc/administration/monitoring/performance/gitlab_configuration.md @@ -1,6 +1,6 @@ --- stage: Monitor -group: Monitor +group: Respond info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/engineering/ux/technical-writing/#assignments --- diff --git a/doc/administration/monitoring/performance/grafana_configuration.md b/doc/administration/monitoring/performance/grafana_configuration.md index c37a264938e..79612145327 100644 --- a/doc/administration/monitoring/performance/grafana_configuration.md +++ b/doc/administration/monitoring/performance/grafana_configuration.md @@ -1,6 +1,6 @@ --- stage: Monitor -group: Monitor +group: Respond info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/engineering/ux/technical-writing/#assignments --- diff --git a/doc/administration/monitoring/performance/index.md b/doc/administration/monitoring/performance/index.md index f3db6ac9f03..20fad8baf91 100644 --- a/doc/administration/monitoring/performance/index.md +++ b/doc/administration/monitoring/performance/index.md @@ -1,6 +1,6 @@ --- stage: Monitor -group: Monitor +group: Respond info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/engineering/ux/technical-writing/#assignments --- diff --git a/doc/administration/monitoring/performance/performance_bar.md b/doc/administration/monitoring/performance/performance_bar.md index 14a560223f9..e93dbd3a2d6 100644 --- a/doc/administration/monitoring/performance/performance_bar.md +++ b/doc/administration/monitoring/performance/performance_bar.md @@ -1,6 +1,6 @@ --- stage: Monitor -group: Monitor +group: Respond info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/engineering/ux/technical-writing/#assignments --- @@ -108,7 +108,7 @@ appears next to requests with warnings. The performance bar is disabled by default for non-administrators. To enable it for a given group: -1. Sign in as a user with Administrator [role](../../../user/permissions.md). +1. Sign in as a user with administrator access. 1. On the top bar, select **Menu > Admin**. 1. On the left sidebar, select **Settings > Metrics and profiling** (`admin/application_settings/metrics_and_profiling`), and expand diff --git a/doc/administration/monitoring/performance/request_profiling.md b/doc/administration/monitoring/performance/request_profiling.md index ebdca8d3960..f8a66f6f48f 100644 --- a/doc/administration/monitoring/performance/request_profiling.md +++ b/doc/administration/monitoring/performance/request_profiling.md @@ -1,10 +1,16 @@ --- stage: Monitor -group: Monitor +group: Respond info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/engineering/ux/technical-writing/#assignments --- -# Request Profiling **(FREE SELF)** +# Request profiling (DEPRECATED) **(FREE SELF)** + +> [Deprecated](https://gitlab.com/gitlab-org/gitlab/-/issues/352488) in GitLab 14.8, and planned for removal in GitLab 15.0. + +WARNING: +This feature is in its end-of-life process. It is [deprecated](https://gitlab.com/gitlab-org/gitlab/-/issues/352488) +for use in GitLab 14.8, and is planned for removal in GitLab 15.0. To profile a request: diff --git a/doc/administration/monitoring/prometheus/gitlab_exporter.md b/doc/administration/monitoring/prometheus/gitlab_exporter.md index d9852524aec..15ec880533e 100644 --- a/doc/administration/monitoring/prometheus/gitlab_exporter.md +++ b/doc/administration/monitoring/prometheus/gitlab_exporter.md @@ -1,6 +1,6 @@ --- stage: Monitor -group: Monitor +group: Respond info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/engineering/ux/technical-writing/#assignments --- diff --git a/doc/administration/monitoring/prometheus/gitlab_metrics.md b/doc/administration/monitoring/prometheus/gitlab_metrics.md index c5b87afd94b..a1bfae678c7 100644 --- a/doc/administration/monitoring/prometheus/gitlab_metrics.md +++ b/doc/administration/monitoring/prometheus/gitlab_metrics.md @@ -1,6 +1,6 @@ --- stage: Monitor -group: Monitor +group: Respond info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/engineering/ux/technical-writing/#assignments --- @@ -8,7 +8,7 @@ info: To determine the technical writer assigned to the Stage/Group associated w To enable the GitLab Prometheus metrics: -1. Log in to GitLab as a user with Administrator [role](../../../user/permissions.md). +1. Log in to GitLab as a user with administrator access. 1. On the top bar, select **Menu > Admin**. 1. On the left sidebar, select **Settings > Metrics and profiling**. 1. Find the **Metrics - Prometheus** section, and select **Add link to Prometheus**. @@ -48,6 +48,7 @@ The following metrics are available: | `gitlab_database_transaction_seconds` | Histogram | 12.1 | Time spent in database transactions, in seconds | | | `gitlab_method_call_duration_seconds` | Histogram | 10.2 | Method calls real duration | `controller`, `action`, `module`, `method` | | `gitlab_page_out_of_bounds` | Counter | 12.8 | Counter for the PageLimiter pagination limit being hit | `controller`, `action`, `bot` | +| `gitlab_rails_boot_time_seconds` | Gauge | 14.8 | Time elapsed for Rails primary process to finish startup | | | `gitlab_rails_queue_duration_seconds` | Histogram | 9.4 | Measures latency between GitLab Workhorse forwarding a request to Rails | | | `gitlab_sql_duration_seconds` | Histogram | 10.2 | SQL execution time, excluding `SCHEMA` operations and `BEGIN` / `COMMIT` | | | `gitlab_sql_<role>_duration_seconds` | Histogram | 13.10 | SQL execution time, excluding `SCHEMA` operations and `BEGIN` / `COMMIT`, grouped by database roles (primary/replica) | | @@ -269,6 +270,16 @@ configuration option in `gitlab.yml`. These metrics are served from the | `geo_pages_deployments_verification_total` | Gauge | 14.6 | Number of pages deployments verifications tried on secondary | `url` | | `geo_pages_deployments_verified` | Gauge | 14.6 | Number of pages deployments verified on secondary | `url` | | `geo_pages_deployments_verification_failed` | Gauge | 14.6 | Number of pages deployments verifications failed on secondary | `url` | +| `geo_job_artifacts` | Gauge | 14.8 | Number of job artifacts on primary | `url` | +| `geo_job_artifacts_checksum_total` | Gauge | 14.8 | Number of job artifacts tried to checksum on primary | `url` | +| `geo_job_artifacts_checksummed` | Gauge | 14.8 | Number of job artifacts successfully checksummed on primary | `url` | +| `geo_job_artifacts_checksum_failed` | Gauge | 14.8 | Number of job artifacts failed to calculate the checksum on primary | `url` | +| `geo_job_artifacts_synced` | Gauge | 14.8 | Number of syncable job artifacts synced on secondary | `url` | +| `geo_job_artifacts_failed` | Gauge | 14.8 | Number of syncable job artifacts failed to sync on secondary | `url` | +| `geo_job_artifacts_registry` | Gauge | 14.8 | Number of job artifacts in the registry | `url` | +| `geo_job_artifacts_verification_total` | Gauge | 14.8 | Number of job artifacts verifications tried on secondary | `url` | +| `geo_job_artifacts_verified` | Gauge | 14.8 | Number of job artifacts verified on secondary | `url` | +| `geo_job_artifacts_verification_failed` | Gauge | 14.8 | Number of job artifacts verifications failed on secondary | `url` | | `limited_capacity_worker_running_jobs` | Gauge | 13.5 | Number of running jobs | `worker` | | `limited_capacity_worker_max_running_jobs` | Gauge | 13.5 | Maximum number of running jobs | `worker` | | `limited_capacity_worker_remaining_work_count` | Gauge | 13.5 | Number of jobs waiting to be enqueued | `worker` | diff --git a/doc/administration/monitoring/prometheus/index.md b/doc/administration/monitoring/prometheus/index.md index acdcdb41dca..9d90c4f3cc7 100644 --- a/doc/administration/monitoring/prometheus/index.md +++ b/doc/administration/monitoring/prometheus/index.md @@ -295,6 +295,36 @@ To use an external Prometheus server: 1. Reload the Prometheus server. +### Configure the storage retention size + +Prometheus has several custom flags to configure local storage: + +- `storage.tsdb.retention.time`: when to remove old data. Defaults to `15d`. Overrides + `storage.tsdb.retention` if this flag is set to anything other than the default. +- `storage.tsdb.retention.size`: [EXPERIMENTAL] the maximum number of bytes of storage blocks to + retain. The oldest data is removed first. Defaults to `0` (disabled). This flag is experimental + and may change in future releases. Units supported: `B`, `KB`, `MB`, `GB`, `TB`, `PB`, `EB`. For + example, `512MB`. + +To configure the storage retention size: + +1. Edit `/etc/gitlab/gitlab.rb`: + + ```ruby + prometheus['flags'] = { + 'storage.tsdb.path' => "/var/opt/gitlab/prometheus/data", + 'storage.tsdb.retention.time' => "7d", + 'storage.tsdb.retention.size' => "2GB", + 'config.file' => "/var/opt/gitlab/prometheus/prometheus.yml" + } + ``` + +1. Reconfigure GitLab: + + ```shell + sudo gitlab-ctl reconfigure + ``` + ## Viewing performance metrics You can visit `http://localhost:9090` for the dashboard that Prometheus offers by default. @@ -402,3 +432,35 @@ To disable the monitoring of Kubernetes: 1. Save the file and [reconfigure GitLab](../../restart_gitlab.md#omnibus-gitlab-reconfigure) for the changes to take effect. + +## Troubleshooting + +### `/var/opt/gitlab/prometheus` consumes too much disk space + +If you are **not** using Prometheus monitoring: + +1. [Disable Prometheus](index.md#configuring-prometheus). +1. Delete the data under `/var/opt/gitlab/prometheus`. + +If you are using Prometheus monitoring: + +1. Stop Prometheus (deleting data while it's running can cause data corruption): + + ```shell + gitlab-ctl stop prometheus + ``` + +1. Delete the data under `/var/opt/gitlab/prometheus/data`. +1. Start the service again: + + ```shell + gitlab-ctl start prometheus + ``` + +1. Verify the service is up and running: + + ```shell + gitlab-ctl status prometheus + ``` + +1. Optional. [Configure the storage retention size](index.md#configure-the-storage-retention-size). diff --git a/doc/administration/monitoring/prometheus/node_exporter.md b/doc/administration/monitoring/prometheus/node_exporter.md index 68d997d7596..d7a4a96cd9a 100644 --- a/doc/administration/monitoring/prometheus/node_exporter.md +++ b/doc/administration/monitoring/prometheus/node_exporter.md @@ -1,6 +1,6 @@ --- stage: Monitor -group: Monitor +group: Respond info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/engineering/ux/technical-writing/#assignments --- diff --git a/doc/administration/monitoring/prometheus/pgbouncer_exporter.md b/doc/administration/monitoring/prometheus/pgbouncer_exporter.md index aba1561500a..979a6bcd232 100644 --- a/doc/administration/monitoring/prometheus/pgbouncer_exporter.md +++ b/doc/administration/monitoring/prometheus/pgbouncer_exporter.md @@ -1,6 +1,6 @@ --- stage: Monitor -group: Monitor +group: Respond info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/engineering/ux/technical-writing/#assignments --- diff --git a/doc/administration/monitoring/prometheus/postgres_exporter.md b/doc/administration/monitoring/prometheus/postgres_exporter.md index 8a851afe35b..95a6540bd19 100644 --- a/doc/administration/monitoring/prometheus/postgres_exporter.md +++ b/doc/administration/monitoring/prometheus/postgres_exporter.md @@ -1,6 +1,6 @@ --- stage: Monitor -group: Monitor +group: Respond info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/engineering/ux/technical-writing/#assignments --- diff --git a/doc/administration/monitoring/prometheus/puma_exporter.md b/doc/administration/monitoring/prometheus/puma_exporter.md index 804c4243cfa..794e2c10b25 100644 --- a/doc/administration/monitoring/prometheus/puma_exporter.md +++ b/doc/administration/monitoring/prometheus/puma_exporter.md @@ -1,6 +1,6 @@ --- stage: Monitor -group: Monitor +group: Respond info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/engineering/ux/technical-writing/#assignments --- diff --git a/doc/administration/monitoring/prometheus/redis_exporter.md b/doc/administration/monitoring/prometheus/redis_exporter.md index 6cc262842a1..a5f12bbc52f 100644 --- a/doc/administration/monitoring/prometheus/redis_exporter.md +++ b/doc/administration/monitoring/prometheus/redis_exporter.md @@ -1,6 +1,6 @@ --- stage: Monitor -group: Monitor +group: Respond info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/engineering/ux/technical-writing/#assignments --- diff --git a/doc/administration/monitoring/prometheus/registry_exporter.md b/doc/administration/monitoring/prometheus/registry_exporter.md index 3a2acd47338..f4fa35c206e 100644 --- a/doc/administration/monitoring/prometheus/registry_exporter.md +++ b/doc/administration/monitoring/prometheus/registry_exporter.md @@ -1,6 +1,6 @@ --- stage: Monitor -group: Monitor +group: Respond info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/engineering/ux/technical-writing/#assignments --- |