Welcome to mirror list, hosted at ThFree Co, Russian Federation.

gitlab.com/gitlab-org/gitlab-foss.git - Unnamed repository; edit this file 'description' to name the repository.
summaryrefslogtreecommitdiff
path: root/doc/user/project/integrations/prometheus.md
diff options
context:
space:
mode:
Diffstat (limited to 'doc/user/project/integrations/prometheus.md')
-rw-r--r--doc/user/project/integrations/prometheus.md161
1 files changed, 135 insertions, 26 deletions
diff --git a/doc/user/project/integrations/prometheus.md b/doc/user/project/integrations/prometheus.md
index 6c40e5b9696..0d17ff51372 100644
--- a/doc/user/project/integrations/prometheus.md
+++ b/doc/user/project/integrations/prometheus.md
@@ -25,7 +25,7 @@ Once enabled, GitLab will automatically detect metrics from known services in th
### Managed Prometheus on Kubernetes
-> [Introduced](https://gitlab.com/gitlab-org/gitlab-foss/issues/28916) in GitLab 10.5.
+> [Introduced](https://gitlab.com/gitlab-org/gitlab-foss/-/issues/28916) in GitLab 10.5.
GitLab can seamlessly deploy and manage Prometheus on a [connected Kubernetes cluster](../clusters/index.md), making monitoring of your apps easy.
@@ -61,7 +61,7 @@ will help you to quickly create a deployment:
1. Navigate to your project's **CI/CD > Pipelines** page, and run a pipeline on any branch.
1. When the pipeline has run successfully, graphs will be available on the **Operations > Metrics** page.
-![Monitoring Dashboard](img/prometheus_monitoring_dashboard_v12_8.png)
+![Monitoring Dashboard](img/prometheus_monitoring_dashboard_v13_1.png)
#### Using the Metrics Dashboard
@@ -209,10 +209,19 @@ GitLab supports a limited set of [CI variables](../../../ci/variables/README.md)
- `ci_project_namespace`
- `ci_project_path`
- `ci_environment_name`
+- `__range`
NOTE: **Note:**
Variables for Prometheus queries must be lowercase.
+###### __range
+
+The `__range` variable is useful in Prometheus
+[range vector selectors](https://prometheus.io/docs/prometheus/latest/querying/basics/#range-vector-selectors).
+Its value is the total number of seconds in the dashboard's time range.
+For example, if the dashboard time range is set to 8 hours, the value of
+`__range` is `28800s`.
+
##### User-defined variables
[Variables can be defined](#templating-templating-properties) in a custom dashboard YAML file.
@@ -244,7 +253,7 @@ http://gitlab.com/<user>/<project>/-/environments/<environment_id>/metrics?dashb
#### Editing additional metrics from the dashboard
-> [Introduced](https://gitlab.com/gitlab-org/gitlab/issues/208976) in GitLab 12.9.
+> [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/208976) in GitLab 12.9.
You can edit existing additional custom metrics by clicking the **{ellipsis_v}** **More actions** dropdown and selecting **Edit metric**.
@@ -252,7 +261,7 @@ You can edit existing additional custom metrics by clicking the **{ellipsis_v}**
### Defining custom dashboards per project
-> [Introduced](https://gitlab.com/gitlab-org/gitlab-foss/issues/59974) in GitLab 12.1.
+> [Introduced](https://gitlab.com/gitlab-org/gitlab-foss/-/issues/59974) in GitLab 12.1.
By default, all projects include a GitLab-defined Prometheus dashboard, which
includes a few key metrics, but you can also define your own custom dashboards.
@@ -308,8 +317,8 @@ supported and will not be available in the UI.
#### Duplicating a GitLab-defined dashboard
-> - [Introduced](https://gitlab.com/gitlab-org/gitlab/issues/37238) in GitLab 12.7.
-> - From [GitLab 12.8 onwards](https://gitlab.com/gitlab-org/gitlab/issues/39505), custom metrics are also duplicated when you duplicate a dashboard.
+> - [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/37238) in GitLab 12.7.
+> - From [GitLab 12.8 onwards](https://gitlab.com/gitlab-org/gitlab/-/issues/39505), custom metrics are also duplicated when you duplicate a dashboard.
You can save a complete copy of a GitLab defined dashboard along with all custom metrics added to it.
Resulting `.yml` file can be customized and adapted to your project.
@@ -326,6 +335,42 @@ new branch.
If you select your **default** branch, the new dashboard becomes immediately available.
If you select another branch, this branch should be merged to your **default** branch first.
+#### Dashboard YAML syntax validation
+
+> [Introduced](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/33202) in GitLab 13.1.
+
+To confirm your dashboard definition contains valid YAML syntax:
+
+1. Navigate to **{doc-text}** **Repository > Files**.
+1. Navigate to your dashboard file in your repository.
+1. Review the information pane about the file, displayed above the file contents.
+
+Files with valid syntax display **Metrics Dashboard YAML definition is valid**,
+and files with invalid syntax display **Metrics Dashboard YAML definition is invalid**.
+
+![Metrics Dashboard_YAML_syntax_validation](img/prometheus_dashboard_yaml_validation_v13_1.png)
+
+When **Metrics Dashboard YAML definition is invalid** at least one of the following messages is displayed:
+
+1. `dashboard: can't be blank` [learn more](#dashboard-top-level-properties)
+1. `panel_groups: can't be blank` [learn more](#dashboard-top-level-properties)
+1. `group: can't be blank` [learn more](#panel-group-panel_groups-properties)
+1. `panels: can't be blank` [learn more](#panel-group-panel_groups-properties)
+1. `metrics: can't be blank` [learn more](#panel-panels-properties)
+1. `title: can't be blank` [learn more](#panel-panels-properties)
+1. `query: can't be blank` [learn more](#metrics-metrics-properties)
+1. `query_range: can't be blank` [learn more](#metrics-metrics-properties)
+1. `unit: can't be blank` [learn more](#metrics-metrics-properties)
+1. `YAML syntax: The parsed YAML is too big`
+
+ This is displayed when the YAML file is larger than 1 MB.
+
+1. `YAML syntax: Invalid configuration format`
+
+ This is displayed when the YAML file is empty or does not contain valid YAML.
+
+Metrics Dashboard YAML definition validation information is also available as a [GraphQL API field](../../../api/graphql/reference/index.md#metricsdashboard)
+
#### Dashboard YAML properties
Dashboards have several components:
@@ -342,16 +387,27 @@ The following tables outline the details of expected properties.
| ------ | ------ | ------ | ------ |
| `dashboard` | string | yes | Heading for the dashboard. Only one dashboard should be defined per file. |
| `panel_groups` | array | yes | The panel groups which should be on the dashboard. |
-| `templating` | Hash | no | Top level key under which templating related options can be added. |
+| `templating` | hash | no | Top level key under which templating related options can be added. |
+| `links` | array | no | Add links to display on the dashboard. |
##### **Templating (`templating`) properties**
| Property | Type | Required | Description |
| -------- | ---- | -------- | ----------- |
-| `variables` | Hash | no | Variables can be defined here. |
+| `variables` | hash | yes | Variables can be defined here. |
Read the documentation on [templating](#templating-variables-for-metrics-dashboards).
+##### **Links (`links`) properties**
+
+| Property | Type | Required | Description |
+| -------- | ---- | -------- | ----------- |
+| `url` | string | yes | The address of the link. |
+| `title` | string | no | Display title for the link. |
+| `type` | string | no | Type of the link. Specifies the link type, can be: `grafana` |
+
+Read the documentation on [links](#add-related-links-to-custom-dashboards).
+
##### **Panel group (`panel_groups`) properties**
| Property | Type | Required | Description |
@@ -371,6 +427,7 @@ Read the documentation on [templating](#templating-variables-for-metrics-dashboa
| `max_value` | number | no | Denominator value used for calculating [percentile based results](#percentile-based-results) |
| `weight` | number | no, defaults to order in file | Order to appear within the grouping. Lower number means higher priority, which will be higher on the page. Numbers do not need to be consecutive. |
| `metrics` | array | yes | The metrics which should be displayed in the panel. Any number of metrics can be displayed when `type` is `area-chart` or `line-chart`, whereas only 3 can be displayed when `type` is `anomaly-chart`. |
+| `links` | array | no | Add links to display on the chart's [context menu](#chart-context-menu). |
##### **Axis (`panels[].y_axis`) properties**
@@ -472,7 +529,7 @@ Note the following properties:
![area panel chart](img/prometheus_dashboard_area_panel_type_v12_8.png)
-Starting in [version 12.8](https://gitlab.com/gitlab-org/gitlab/issues/202696), the y-axis values will automatically scale according to the data. Previously, it always started from 0.
+Starting in [version 12.8](https://gitlab.com/gitlab-org/gitlab/-/issues/202696), the y-axis values will automatically scale according to the data. Previously, it always started from 0.
##### Anomaly chart
@@ -574,7 +631,7 @@ Note the following properties:
##### Stacked column
-> [Introduced](https://gitlab.com/gitlab-org/gitlab/issues/30583) in GitLab 12.8.
+> [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/30583) in GitLab 12.8.
To add a stacked column panel type to a dashboard, look at the following sample dashboard file:
@@ -639,7 +696,7 @@ Note the following properties:
###### Percentile based results
-> [Introduced](https://gitlab.com/gitlab-org/gitlab/issues/201946) in GitLab 12.8.
+> [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/201946) in GitLab 12.8.
Query results sometimes need to be represented as a percentage value out of 100. You can use the `max_value` property at the root of the panel definition:
@@ -662,7 +719,7 @@ For example, if you have a query value of `53.6`, adding `%` as the unit results
##### Heatmaps
-> [Introduced](https://gitlab.com/gitlab-org/gitlab/issues/30581) in GitLab 12.5.
+> [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/30581) in GitLab 12.5.
To add a heatmap panel type to a dashboard, look at the following sample dashboard file:
@@ -699,7 +756,7 @@ Templating variables can be used to make your metrics dashboard more versatile.
[dashboard YAML](#dashboard-top-level-properties).
Define your variables in the `variables` key, under `templating`. The value of
the `variables` key should be a hash, and each key under `variables`
-defines a templating variable on the dashboard.
+defines a templating variable on the dashboard, and may contain alphanumeric and underscore characters.
A variable can be used in a Prometheus query in the same dashboard using the syntax
described [here](#using-variables).
@@ -766,7 +823,7 @@ templating:
###### Full syntax
-This example creates a variable called `variable1`, with a default value of `var1_option_2`.
+This example creates a variable called `variable1`, with a default value of `value_option_2`.
The label for the text box on the UI will be the value of the `label` key.
The dashboard UI will display a dropdown with `Option 1` and `Option 2`
as the choices.
@@ -789,9 +846,50 @@ templating:
default: true # (Optional) This option should be the default value of this variable.
```
+### Add related links to custom dashboards
+
+> [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/216385) in GitLab 13.1.
+
+You can embed links to other dashboards or external services in your custom
+dashboard by adding **Related links** to your dashboard's YAML file. Related links
+open in the same tab as the dashboard. Related links can be displayed in the
+following locations on your dashboard:
+
+- At the top of your dashboard as the top level [`links` dashboard property](#dashboard-top-level-properties).
+- In charts context menus as the [`links` property of a panel](#panel-panels-properties).
+
+Related links can contain the following attributes:
+
+- `url`: The full URL to the link. Required.
+- `title`: A phrase describing the link. Optional. If this attribute is not set,
+ the full URL is used for the link title.
+- `type`: A string declaring the type of link. Optional. If set to `grafana`, the
+ dashboard's time range values are converted to Grafana's time range format and
+ appended to the `url`.
+
+The dashboard's time range is appended to the `url` as URL parameters.
+
+The following example shows two related links (`GitLab.com` and `GitLab Documentation`)
+added to a dashboard:
+
+![Links UI](img/related_links_v13_1.png)
+
+#### Links Syntax
+
+```yaml
+links:
+ - title: GitLab.com
+ url: https://gitlab.com
+ - title: GitLab Documentation
+ url: https://docs.gitlab.com
+ - title: Public Grafana playground dashboard
+ url: https://play.grafana.org/d/000000012/grafana-play-home?orgId=1
+ type: grafana
+```
+
### View and edit the source file of a custom dashboard
-> [Introduced](https://gitlab.com/gitlab-org/gitlab/issues/34779) in GitLab 12.5.
+> [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/34779) in GitLab 12.5.
When viewing a custom dashboard of a project, you can view the original
`.yml` file by clicking on the **Edit dashboard** button.
@@ -812,7 +910,7 @@ The options are:
### Dashboard Annotations
-> - [Introduced](https://gitlab.com/gitlab-org/gitlab/issues/211330) in GitLab 12.10 (enabled by feature flag `metrics_dashboard_annotations`).
+> - [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/211330) in GitLab 12.10 (enabled by feature flag `metrics_dashboard_annotations`).
> - [Feature flag removed](https://gitlab.com/gitlab-org/gitlab/-/issues/215224) in GitLab 13.0.
You can use **Metrics Dashboard Annotations** to mark any important events on
@@ -827,6 +925,14 @@ You can create annotations by making requests to the
![Annotations UI](img/metrics_dashboard_annotations_ui_v13.0.png)
+#### Retention policy
+
+> [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/211433) in GitLab 13.01.
+
+To avoid excessive storage space consumption by stale annotations, records attached
+to time periods older than two weeks are removed daily. This recurring background
+job runs at 1:00 a.m. local server time.
+
### Expand panel
> [Introduced](https://gitlab.com/groups/gitlab-org/-/epics/3100) in GitLab 13.0.
@@ -839,7 +945,7 @@ browser, or pressing the <kbd>Esc</kbd> key.
### View Logs **(ULTIMATE)**
-> [Introduced](https://gitlab.com/gitlab-org/gitlab/issues/122013) in GitLab 12.8.
+> [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/122013) in GitLab 12.8.
If you have [Logs](../clusters/kubernetes_pod_logs.md) enabled,
you can navigate from the charts in the dashboard to view Logs by
@@ -881,8 +987,8 @@ To remove the alert, click back on the alert icon for the desired metric, and cl
#### External Prometheus instances
->- [Introduced](https://gitlab.com/gitlab-org/gitlab/issues/9258) in [GitLab Ultimate](https://about.gitlab.com/pricing/) 11.8.
->- [Moved](https://gitlab.com/gitlab-org/gitlab/issues/42640) to [GitLab Core](https://about.gitlab.com/pricing/) in 12.10.
+>- [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/9258) in [GitLab Ultimate](https://about.gitlab.com/pricing/) 11.8.
+>- [Moved](https://gitlab.com/gitlab-org/gitlab/-/issues/42640) to [GitLab Core](https://about.gitlab.com/pricing/) in 12.10.
For manually configured Prometheus servers, a notify endpoint is provided to use with Prometheus webhooks. If you have manual configuration enabled, an **Alerts** section is added to **Settings > Integrations > Prometheus**. This contains the *URL* and *Authorization Key*. The **Reset Key** button will invalidate the key and generate a new one.
@@ -903,12 +1009,15 @@ receivers:
In order for GitLab to associate your alerts with an [environment](../../../ci/environments/index.md), you need to configure a `gitlab_environment_name` label on the alerts you set up in Prometheus. The value of this should match the name of your Environment in GitLab.
+NOTE: **Note**
+In GitLab versions 13.1 and greater, you can configure your manually configured Prometheus server to use the [Generic alerts integration](generic_alerts.md).
+
### Taking action on incidents **(ULTIMATE)**
->- [Introduced](https://gitlab.com/gitlab-org/gitlab/issues/4925) in [GitLab Ultimate](https://about.gitlab.com/pricing/) 11.11.
->- [From GitLab Ultimate 12.5](https://gitlab.com/gitlab-org/gitlab/issues/13401), when GitLab receives a recovery alert, it will automatically close the associated issue.
+>- [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/4925) in [GitLab Ultimate](https://about.gitlab.com/pricing/) 11.11.
+>- [From GitLab Ultimate 12.5](https://gitlab.com/gitlab-org/gitlab/-/issues/13401), when GitLab receives a recovery alert, it will automatically close the associated issue.
-Alerts can be used to trigger actions, like opening an issue automatically (enabled by default since `12.1`). To configure the actions:
+Alerts can be used to trigger actions, like opening an issue automatically (disabled by default since `13.1`). To configure the actions:
1. Navigate to your project's **Settings > Operations > Incidents**.
1. Enable the option to create issues.
@@ -931,14 +1040,14 @@ When GitLab receives a **Recovery Alert**, it will automatically close the assoc
To further customize the issue, you can add labels, mentions, or any other supported [quick action](../quick_actions.md) in the selected issue template, which will apply to all incidents. To limit quick actions or other information to only specific types of alerts, use the `annotations/gitlab_incident_markdown` field.
-Since [version 12.2](https://gitlab.com/gitlab-org/gitlab-foss/issues/63373), GitLab will tag each incident issue with the `incident` label automatically. If the label does not yet exist, it will be created automatically as well.
+Since [version 12.2](https://gitlab.com/gitlab-org/gitlab-foss/-/issues/63373), GitLab will tag each incident issue with the `incident` label automatically. If the label does not yet exist, it will be created automatically as well.
If the metric exceeds the threshold of the alert for over 5 minutes, an email will be sent to all [Maintainers and Owners](../../permissions.md#project-members-permissions) of the project.
## Determining the performance impact of a merge
> - [Introduced](https://gitlab.com/gitlab-org/gitlab-foss/-/merge_requests/10408) in GitLab 9.2.
-> - GitLab 9.3 added the [numeric comparison](https://gitlab.com/gitlab-org/gitlab-foss/issues/27439) of the 30 minute averages.
+> - GitLab 9.3 added the [numeric comparison](https://gitlab.com/gitlab-org/gitlab-foss/-/issues/27439) of the 30 minute averages.
Developers can view the performance impact of their changes within the merge
request workflow.
@@ -1031,7 +1140,7 @@ For [manually configured Prometheus instances](#manual-configuration-of-promethe
### Embedding Cluster Health Charts **(ULTIMATE)**
-> [Introduced](<https://gitlab.com/gitlab-org/gitlab/issues/40997>) in [GitLab Ultimate](https://about.gitlab.com/pricing/) 12.9.
+> [Introduced](<https://gitlab.com/gitlab-org/gitlab/-/issues/40997>) in [GitLab Ultimate](https://about.gitlab.com/pricing/) 12.9.
[Cluster Health Metrics](../clusters/index.md#monitoring-your-kubernetes-cluster-ultimate) can also be embedded in [GitLab-flavored Markdown](../../markdown.md).
@@ -1075,7 +1184,7 @@ This will render like so:
#### Embedding charts via integration with Grafana HTTP API
-> [Introduced](https://gitlab.com/gitlab-org/gitlab/issues/31376) in GitLab 12.5.
+> [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/31376) in GitLab 12.5.
Each project can support integration with one Grafana instance. This configuration allows a user to copy a link to a panel in Grafana, then paste it into a GitLab Markdown field. The chart will be rendered in the GitLab chart format.
generated by cgit v1.2.3 (git 2.25.1) at 2024-09-01 08:35:41 +0300