diff options
Diffstat (limited to 'doc/operations/metrics')
| -rw-r--r-- | doc/operations/metrics/alerts.md | 32 | ||||
| -rw-r--r-- | doc/operations/metrics/dashboards/img/metrics_dashboard_template_selection_v13_3.png | bin | 31905 -> 9033 bytes | |||
| -rw-r--r-- | doc/operations/metrics/dashboards/index.md | 48 | ||||
| -rw-r--r-- | doc/operations/metrics/dashboards/settings.md | 4 | ||||
| -rw-r--r-- | doc/operations/metrics/embed_grafana.md | 2 | ||||
| -rw-r--r-- | doc/operations/metrics/index.md | 34 |
6 files changed, 74 insertions, 46 deletions
diff --git a/doc/operations/metrics/alerts.md b/doc/operations/metrics/alerts.md index 09cfea06198..16cfb05ad9a 100644 --- a/doc/operations/metrics/alerts.md +++ b/doc/operations/metrics/alerts.md @@ -25,7 +25,7 @@ For managed Prometheus instances using auto configuration, you can [configure alerts for metrics](index.md#adding-custom-metrics) directly in the [metrics dashboard](index.md). To set an alert: -1. In your project, navigate to **Operations > Metrics**, +1. In your project, navigate to **Monitor > Metrics**, 1. Identify the metric you want to create the alert for, and click the **ellipsis** **{ellipsis_v}** icon in the top right corner of the metric. 1. Choose **Alerts**. @@ -39,7 +39,15 @@ To remove the alert, click back on the alert icon for the desired metric, and cl ### Link runbooks to alerts -> Runbook URLs [introduced](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/39315) in GitLab 13.3. +> - [Introduced](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/39315) in GitLab 13.3. +> - [Deprecated](https://gitlab.com/groups/gitlab-org/-/epics/5877) in GitLab 13.11. +> - [Removed](https://gitlab.com/groups/gitlab-org/-/epics/4280) in GitLab 14.0. + +WARNING: +Linking runbooks to alerts through the alerts UI is [deprecated](https://gitlab.com/groups/gitlab-org/-/epics/5877) +and scheduled for [removal in GitLab 14.0](https://gitlab.com/groups/gitlab-org/-/epics/4280). +However, you can still add runbooks to your alert payload. They show up in the alert UI when the +alert is triggered. When creating alerts from the metrics dashboard for [managed Prometheus instances](#managed-prometheus-instances), you can also link a runbook. When the alert triggers, the @@ -72,14 +80,16 @@ section of your Prometheus Alertmanager configuration: ```yaml receivers: - name: gitlab - webhook_configs: - - http_config: - bearer_token: 9e1cbfcd546896a9ea8be557caf13a76 - send_resolved: true - url: http://192.168.178.31:3001/root/manual_prometheus/prometheus/alerts/notify.json - # Rest of configuration omitted - # ... + - name: gitlab + webhook_configs: + - http_config: + authorization: + type: Bearer + credentials: 9e1cbfcd546896a9ea8be557caf13a76 + send_resolved: true + url: http://192.168.178.31:3001/root/manual_prometheus/prometheus/alerts/notify.json + # Rest of configuration omitted + # ... ``` For GitLab to associate your alerts with an [environment](../../ci/environments/index.md), @@ -100,7 +110,7 @@ Prometheus server to use the Alerts can be used to trigger actions, like opening an issue automatically (disabled by default since `13.1`). To configure the actions: -1. Navigate to your project's **Settings > Operations > Incidents**. +1. Navigate to your project's **Settings > Monitor > Alerts**. 1. Enable the option to create issues. 1. Choose the [issue template](../../user/project/description_templates.md) to create the issue from. 1. Optionally, select whether to send an email notification to the developers of the project. diff --git a/doc/operations/metrics/dashboards/img/metrics_dashboard_template_selection_v13_3.png b/doc/operations/metrics/dashboards/img/metrics_dashboard_template_selection_v13_3.png Binary files differindex cad075ca421..1571ab9de90 100644 --- a/doc/operations/metrics/dashboards/img/metrics_dashboard_template_selection_v13_3.png +++ b/doc/operations/metrics/dashboards/img/metrics_dashboard_template_selection_v13_3.png diff --git a/doc/operations/metrics/dashboards/index.md b/doc/operations/metrics/dashboards/index.md index 7b056020a99..d7b748034d5 100644 --- a/doc/operations/metrics/dashboards/index.md +++ b/doc/operations/metrics/dashboards/index.md @@ -20,14 +20,14 @@ or [duplicate a GitLab-defined Prometheus dashboard](#duplicate-a-gitlab-defined You can configure a custom dashboard by adding a new YAML file into your project's `.gitlab/dashboards/` directory. For the dashboard to display on your project's -**Operations > Metrics** page, the files must have a `.yml` +**Monitor > Metrics** page, the files must have a `.yml` extension and be present in your project's **default** branch. To create a new dashboard from the GitLab user interface: 1. Sign in to GitLab as a user with Maintainer or Owner [permissions](../../../user/permissions.md#project-members-permissions). -1. Navigate to your dashboard at **Operations > Metrics**. +1. Navigate to your dashboard at **Monitor > Metrics**. 1. In the top-right corner of your dashboard, click the **{ellipsis_v}** **More actions** menu, and select **Create new**:  @@ -60,7 +60,7 @@ To create a new dashboard from the command line: ``` 1. Save the file, commit, and push to your repository. The file must be present in your **default** branch. -1. Navigate to your project's **Operations > Metrics** and choose the custom +1. Navigate to your project's **Monitor > Metrics** and choose the custom dashboard from the dropdown. Your custom dashboard is available at `https://example.com/project/-/metrics/custom_dashboard_name.yml`. @@ -217,8 +217,46 @@ links: ## Troubleshooting -When troubleshooting issues with a managed Prometheus app, it is often useful to -[view the Prometheus UI](../../../user/project/integrations/prometheus.md#access-the-ui-of-a-prometheus-managed-application-in-kubernetes). +### Accessing the UI of Prometheus in Kubernetes + +When troubleshooting issues with an in-cluster Prometheus, it can help to +view the Prometheus UI. In the example below, we assume the Prometheus +server to be the pod `prometheus-prometheus-server` in the `gitlab-managed-apps` +namespace: + +1. Find the name of the Prometheus pod in the user interface of your Kubernetes + provider, such as GKE, or by running the following `kubectl` command in your + terminal. For example: + + ```shell + kubectl get pods -n gitlab-managed-apps | grep 'prometheus-prometheus-server' + ``` + + The command should return a result like the following example, where + `prometheus-prometheus-server-55b4bd64c9-dpc6b` is the name of the Prometheus pod: + + ```plaintext + gitlab-managed-apps prometheus-prometheus-server-55b4bd64c9-dpc6b 2/2 Running 0 71d + ``` + +1. Run a `kubectl port-forward` command. In the following example, `9090` is the + Prometheus server's listening port: + + ```shell + kubectl port-forward prometheus-prometheus-server-55b4bd64c9-dpc6b 9090:9090 -n gitlab-managed-apps + ``` + + The `port-forward` command forwards all requests sent to your system's `9090` port + to the `9090` port of the Prometheus pod. If the `9090` port on your system is used + by another application, you can change the port number before the colon to your + desired port. For example, to forward port `8080` of your local system, change the + command to: + + ```shell + kubectl port-forward prometheus-prometheus-server-55b4bd64c9-dpc6b 8080:9090 -n gitlab-managed-apps + ``` + +1. Open `localhost:9090` in your browser to display the Prometheus user interface. ### "No data found" error on Metrics dashboard page diff --git a/doc/operations/metrics/dashboards/settings.md b/doc/operations/metrics/dashboards/settings.md index e998db6b51b..0e467623e05 100644 --- a/doc/operations/metrics/dashboards/settings.md +++ b/doc/operations/metrics/dashboards/settings.md @@ -21,7 +21,7 @@ time zone, but you can display dates and times in UTC format. To change the time zone: 1. Sign in as a user with Manage Project Operations [permissions](../../../user/permissions.md). -1. Navigate to **Settings > Operations**. +1. Navigate to **Settings > Monitor**. 1. Scroll to **Metrics Dashboard** and click **Expand**. 1. In the **Dashboard timezone** select box, select *User's local timezone* or *UTC*: @@ -37,7 +37,7 @@ You can add a button on your monitoring dashboard that links directly to your existing external dashboards: 1. Sign in as a user with Manage Project Operations [permissions](../../../user/permissions.md). -1. Navigate to **Settings > Operations**. +1. Navigate to **Settings > Monitor**. 1. Scroll to **Metrics Dashboard** and click **Expand**. 1. In **External dashboard URL**, provide the URL to your external dashboard: diff --git a/doc/operations/metrics/embed_grafana.md b/doc/operations/metrics/embed_grafana.md index ff92997e44b..473b335d4c5 100644 --- a/doc/operations/metrics/embed_grafana.md +++ b/doc/operations/metrics/embed_grafana.md @@ -47,7 +47,7 @@ format. To embed panels from a Grafana instance, the data source must be: To set up the Grafana API in Grafana: 1. In Grafana, [generate an Admin-level API Token](https://grafana.com/docs/grafana/latest/http_api/auth/#create-api-token). -1. In your GitLab project, go to **Settings > Operations** and expand the **Grafana authentication** +1. In your GitLab project, go to **Settings > Monitor** and expand the **Grafana authentication** section. 1. To enable the integration, check the **Active** checkbox. 1. For **Grafana URL**, enter the base URL of the Grafana instance. diff --git a/doc/operations/metrics/index.md b/doc/operations/metrics/index.md index 8ff45ac4015..09bc3237cb6 100644 --- a/doc/operations/metrics/index.md +++ b/doc/operations/metrics/index.md @@ -16,10 +16,10 @@ critical. For GitLab to display your information in charts, you must: For an overview, see [How to instrument Prometheus metrics in GitLab](https://www.youtube.com/watch?v=tuI2oJ3TTB4). 1. **Expose metrics for capture** - Make logs, metrics, and traces available for capture. 1. [**Configure Prometheus to gather metrics**](#configure-prometheus-to-gather-metrics) - - Deploy managed applications like Elasticsearch, Prometheus, and Jaeger to gather + Use applications like Elasticsearch, Prometheus, and Jaeger to gather the data you've exposed. 1. **GitLab collects metrics** - GitLab uses Prometheus to scrape the data you've - captured in your managed apps, and prepares the data for display. To learn more, read + captured in your applications, and prepares the data for display. To learn more, read [Collect and process metrics](#collect-and-process-metrics). 1. **Display charts in the GitLab user interface** - GitLab converts your metrics into easy-to-read charts on a default dashboard. You can create as many custom charts @@ -34,30 +34,10 @@ your Prometheus integration depends on where your apps are running: - **For manually-configured Prometheus** - [Specify your Prometheus server](../../user/project/integrations/prometheus.md#manual-configuration-of-prometheus), and define at least one environment. -- **For GitLab-managed Prometheus** - GitLab can - [deploy and manage Prometheus](../../user/project/integrations/prometheus.md#managed-prometheus-on-kubernetes) for you. - You must also complete a code deployment, as described in - [Deploy code with GitLab-managed Prometheus](#deploy-code-with-gitlab-managed-prometheus), - for the **Operations > Metrics** page to contain data. - -### Deploy code with GitLab-managed Prometheus - -For GitLab-managed Prometheus, you can set up [Auto DevOps](../../topics/autodevops/index.md) -to quickly create a deployment: - -1. Navigate to your project's **Operations > Kubernetes** page. -1. Ensure that, in addition to Prometheus, you also have GitLab Runner and Ingress - installed. -1. After installing Ingress, copy its endpoint. -1. Navigate to your project's **Settings > CI/CD** page. In the - **Auto DevOps** section, select a deployment strategy and save your changes. -1. On the same page, in the **Variables** section, add a variable named - `KUBE_INGRESS_BASE_DOMAIN` with the value of the Ingress endpoint you - copied previously. Leave the type as **Variable**. -1. Navigate to your project's **{rocket}** **CI/CD > Pipelines** page, and run a - pipeline on any branch. -1. When the pipeline has run successfully, graphs are available on the - **Operations > Metrics** page. +- **For a cluster integrated Prometheus** - GitLab can query + [an in-cluster Prometheus](../../user/clusters/integrations.md#prometheus-cluster-integration). + You must also complete a code deployment to your cluster for the **Monitor > Metrics** + page to contain data. You can do this using [Auto DevOps](../../topics/autodevops/quick_start_guide.md).  @@ -77,7 +57,7 @@ To view the [default metrics dashboard](dashboards/default.md) for an environmen 1. *If the metrics dashboard is only visible to project members,* sign in to GitLab as a member of a project. Learn more about [metrics dashboard visibility](#metrics-dashboard-visibility). -1. In your project, navigate to **Operations > Metrics**. +1. In your project, navigate to **Monitor > Metrics**. GitLab displays the [default metrics dashboard](dashboards/default.md) for the environment, like the following example: |
