diff options
Diffstat (limited to 'doc/user/analytics')
| -rw-r--r-- | doc/user/analytics/ci_cd_analytics.md | 6 | ||||
| -rw-r--r-- | doc/user/analytics/code_review_analytics.md | 2 | ||||
| -rw-r--r-- | doc/user/analytics/img/product_analytics_commits_per_mr_v14_4.png | bin | 0 -> 135480 bytes | |||
| -rw-r--r-- | doc/user/analytics/img/productivity_analytics_time_to_merge_v14_4.png | bin | 0 -> 96144 bytes | |||
| -rw-r--r-- | doc/user/analytics/img/productivity_analytics_trendline_v14_4.png | bin | 0 -> 47250 bytes | |||
| -rw-r--r-- | doc/user/analytics/index.md | 132 | ||||
| -rw-r--r-- | doc/user/analytics/merge_request_analytics.md | 2 | ||||
| -rw-r--r-- | doc/user/analytics/productivity_analytics.md | 88 | ||||
| -rw-r--r-- | doc/user/analytics/repository_analytics.md | 2 |
9 files changed, 147 insertions, 85 deletions
diff --git a/doc/user/analytics/ci_cd_analytics.md b/doc/user/analytics/ci_cd_analytics.md index 4a42133c308..e949f968c2b 100644 --- a/doc/user/analytics/ci_cd_analytics.md +++ b/doc/user/analytics/ci_cd_analytics.md @@ -21,6 +21,12 @@ View pipeline duration history:  +Pipeline statistics are gathered by collecting all available pipelines for the +project regardless of status. The data available for each individual day is based +on when the pipeline was created. The total pipeline calculation includes child +pipelines and pipelines that failed with invalid YAML. If you are interested in +filtering pipelines based on other attributes, consider using the [Pipelines API](../../api/pipelines.md#list-project-pipelines). + ## DevOps Research and Assessment (DORA) key metrics **(ULTIMATE)** > - [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/275991) in GitLab 13.7. diff --git a/doc/user/analytics/code_review_analytics.md b/doc/user/analytics/code_review_analytics.md index fe091fc9899..496e768456f 100644 --- a/doc/user/analytics/code_review_analytics.md +++ b/doc/user/analytics/code_review_analytics.md @@ -22,7 +22,7 @@ Initially, no data appears. Data is populated as users comment on open merge req ## Overview -Code Review Analytics is available to users with Reporter access and above, and displays a table of open merge requests that have at least one non-author comment. The review time is measured from the time the first non-author comment was submitted. +Code Review Analytics is available to users with at least the Reporter [role](../permissions.md), and displays a table of open merge requests that have at least one non-author comment. The review time is measured from the time the first non-author comment was submitted. To access Code Review Analytics, from your project's menu, go to **Analytics > Code Review**. diff --git a/doc/user/analytics/img/product_analytics_commits_per_mr_v14_4.png b/doc/user/analytics/img/product_analytics_commits_per_mr_v14_4.png Binary files differnew file mode 100644 index 00000000000..649416c02c0 --- /dev/null +++ b/doc/user/analytics/img/product_analytics_commits_per_mr_v14_4.png diff --git a/doc/user/analytics/img/productivity_analytics_time_to_merge_v14_4.png b/doc/user/analytics/img/productivity_analytics_time_to_merge_v14_4.png Binary files differnew file mode 100644 index 00000000000..b5d0dd4d2ea --- /dev/null +++ b/doc/user/analytics/img/productivity_analytics_time_to_merge_v14_4.png diff --git a/doc/user/analytics/img/productivity_analytics_trendline_v14_4.png b/doc/user/analytics/img/productivity_analytics_trendline_v14_4.png Binary files differnew file mode 100644 index 00000000000..da5d3aec957 --- /dev/null +++ b/doc/user/analytics/img/productivity_analytics_trendline_v14_4.png diff --git a/doc/user/analytics/index.md b/doc/user/analytics/index.md index c01788f4fc4..0cc21e3f390 100644 --- a/doc/user/analytics/index.md +++ b/doc/user/analytics/index.md @@ -6,65 +6,6 @@ info: To determine the technical writer assigned to the Stage/Group associated w # Analyze GitLab usage **(FREE)** -## Definitions - -When we describe GitLab analytics, we use the following terms: - -- **Cycle time:** The duration of only the execution work. Cycle time is often displayed in combination with the lead time, which is longer than the cycle time. GitLab measures cycle time from the earliest commit of a [linked issue's merge request](../project/issues/crosslinking_issues.md) to when that issue is closed. The cycle time approach underestimates the lead time because merge request creation is always later than commit time. GitLab displays cycle time in [group-level Value Stream Analytics](../group/value_stream_analytics/index.md) and [project-level Value Stream Analytics](../analytics/value_stream_analytics.md). -- **Deploys:** The total number of successful deployments to production in the given time frame (across all applicable projects). GitLab displays deploys in [group-level Value Stream Analytics](../group/value_stream_analytics/index.md) and [project-level Value Stream Analytics](value_stream_analytics.md). -- **DORA (DevOps Research and Assessment)** ["Four Keys"](https://cloud.google.com/blog/products/devops-sre/using-the-four-keys-to-measure-your-devops-performance): - - **Speed/Velocity** - - - **Deployment frequency:** The relative frequency of successful deployments to production - (hourly, daily, weekly, monthly, or yearly). - This effectively measures how often you are delivering value to end users. A higher deployment - frequency means you are able to get feedback and iterate more quickly in delivering - improvements and features faster. GitLab measures this as the number of deployments to a - [production environment](../../ci/environments/index.md#deployment-tier-of-environments) in - the given time period. - GitLab displays deployment frequency in [group-level Value Stream Analytics](../group/value_stream_analytics/index.md) and [project-level Value Stream Analytics](value_stream_analytics.md). - - **Lead Time for Changes:** The time it takes for a commit to get into production. (1) GitLab - measures this as the median duration between merge request merge and deployment to a - [production environment](../../ci/environments/index.md#deployment-tier-of-environments) for - all MRs deployed in the given time period. This measure under-estimates lead time because - merge time is always later than commit time. The - [standard definition](https://github.com/GoogleCloudPlatform/fourkeys/blob/main/METRICS.md#lead-time-for-changes) uses median commit time. - [An issue exists](https://gitlab.com/gitlab-org/gitlab/-/issues/328459) to start - measuring from "issue first commit" as a better proxy, although still imperfect. - - - **Stability** - - **Change Failure Rate:** The percentage of deployments causing a failure in production. - GitLab measures this as the number of [incidents](../../operations/incident_management/incidents.md) - divided by the number of deployments to a - [production environment](../../ci/environments/index.md#deployment-tier-of-environments) in - the given time period. This assumes: - - - All incidents are related to a production environment. - - Incidents and deployments have a strictly one-to-one relationship (meaning any incident is - related to only one production deployment, and any production deployment is related to no - more than one incident). - - - **Time to Restore Service:** How long it takes an organization to recover from a failure in - production. GitLab measures this as the average time required to close the - [incidents](../../operations/incident_management/incidents.md) in the given time period. - This assumes: - - - All incidents are related to a [production environment](../../ci/environments/index.md#deployment-tier-of-environments). - - Incidents and deployments have a strictly one-to-one relationship (meaning any incident is related to only one production deployment, and any production deployment is related to no more than one incident). - -- **Lead time:** The duration of your value stream, from start to finish. Different from [Lead time for changes](#lead-time-for-changes). Often displayed in combination with "cycle time," which is shorter. GitLab measures lead time from issue creation to issue close. GitLab displays lead time in [group-level Value Stream Analytics](../group/value_stream_analytics/index.md). -- **MTTC (Mean Time to Change):** The average duration between idea and delivery. GitLab measures MTTC from issue creation to the issue's latest related merge request's deployment to production. -- **MTTD (Mean Time to Detect):** The average duration that a bug goes undetected in production. GitLab measures MTTD from deployment of bug to issue creation. -- **MTTM (Mean Time To Merge):** The average lifespan of a merge request. GitLab measures MTTM from merge request creation to merge request merge (and closed/un-merged merge requests are excluded). For more information, see [Merge Request Analytics](merge_request_analytics.md). -- **MTTR (Mean Time to Recover/Repair/Resolution/Resolve/Restore):** The average duration that a bug is not fixed in production. GitLab measures MTTR from deployment of bug to deployment of fix. -- **Throughput:** The number of issues closed or merge requests merged (not closed) in some period of time. Often measured per sprint. GitLab displays merge request throughput in [Merge Request Analytics](merge_request_analytics.md). -- **Value Stream:** The entire work process that is followed to deliver value to customers. For example, the [DevOps lifecycle](https://about.gitlab.com/stages-devops-lifecycle/) is a value stream that starts with "plan" and ends with "monitor". GitLab helps you track your value stream using [Value Stream Analytics](value_stream_analytics.md). -- **Velocity:** The total issue burden completed in some period of time. The burden is usually measured in points or weight, often per sprint. For example, your velocity may be "30 points per sprint". GitLab measures velocity as the total points/weight of issues closed in a given period of time. - -### Lead time for changes - -"Lead Time for Changes" differs from ordinary "Lead Time" because it "focuses on measuring only the time to deliver a feature once it has been developed", as described in ([Measuring DevOps Performance](https://devops.com/measuring-devops-performance/)). - ## Instance-level analytics > [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/12077) in GitLab 12.2. @@ -112,3 +53,76 @@ The following analytics features are available for users to create personalized - [Application Security](../application_security/security_dashboard/#security-center) Be sure to review the documentation page for this feature for GitLab tier requirements. + +## Definitions + +We use the following terms to describe GitLab analytics: + +- **Cycle time:** The duration of only the execution work. Cycle time is often displayed in combination with the lead time, which is longer than the cycle time. GitLab measures cycle time from the earliest commit of a [linked issue's merge request](../project/issues/crosslinking_issues.md) to when that issue is closed. The cycle time approach underestimates the lead time because merge request creation is always later than commit time. GitLab displays cycle time in [group-level Value Stream Analytics](../group/value_stream_analytics/index.md) and [project-level Value Stream Analytics](../analytics/value_stream_analytics.md). +- **Deploys:** The total number of successful deployments to production in the given time frame (across all applicable projects). GitLab displays deploys in [group-level Value Stream Analytics](../group/value_stream_analytics/index.md) and [project-level Value Stream Analytics](value_stream_analytics.md). +- **DORA (DevOps Research and Assessment)** ["Four Keys"](https://cloud.google.com/blog/products/devops-sre/using-the-four-keys-to-measure-your-devops-performance): + - **Speed/Velocity** + + - **Deployment frequency:** The relative frequency of successful deployments to production + (hourly, daily, weekly, monthly, or yearly). + This measures how often you are delivering value to end users. A higher deployment + frequency means you are able to get feedback and iterate faster to deliver + improvements and features. GitLab measures this as the number of deployments to a + [production environment](../../ci/environments/index.md#deployment-tier-of-environments) in + the given time period. + GitLab displays deployment frequency in [group-level Value Stream Analytics](../group/value_stream_analytics/index.md) and [project-level Value Stream Analytics](value_stream_analytics.md). + - **Lead Time for Changes:** The time it takes for a commit to get into production. GitLab + measures this as the median duration between merge request merge and deployment to a + [production environment](../../ci/environments/index.md#deployment-tier-of-environments) for + all MRs deployed in the given time period. This measure under estimates lead time because + merge time is always later than commit time. The + [standard definition](https://github.com/GoogleCloudPlatform/fourkeys/blob/main/METRICS.md#lead-time-for-changes) uses median commit time. + [An issue exists](https://gitlab.com/gitlab-org/gitlab/-/issues/328459) to start + measuring from "issue first commit" as a better proxy, although still imperfect. + + - **Stability** + - **Change Failure Rate:** The percentage of deployments causing a failure in production. + GitLab measures this as the number of [incidents](../../operations/incident_management/incidents.md) + divided by the number of deployments to a + [production environment](../../ci/environments/index.md#deployment-tier-of-environments) in + the given time period. This assumes: + + - All incidents are related to a production environment. + - Incidents and deployments have a strictly one-to-one relationship (meaning any incident is + related to only one production deployment, and any production deployment is related to no + more than one incident). + + - **Time to Restore Service:** How long it takes an organization to recover from a failure in + production. GitLab measures this as the average time required to close the + [incidents](../../operations/incident_management/incidents.md) in the given time period. + This assumes: + + - All incidents are related to a [production environment](../../ci/environments/index.md#deployment-tier-of-environments). + - Incidents and deployments have a strictly one-to-one relationship (meaning any incident is related to only one production deployment, and any production deployment is related to no more than one incident). + +- **Lead time:** The duration of your value stream, from start to finish. Different to +[Lead time for changes](#lead-time-for-changes). Often displayed in combination with "cycle time," +which is shorter. GitLab measures lead time from issue creation to issue close. GitLab displays lead +time in [group-level Value Stream Analytics](../group/value_stream_analytics/index.md). +- **Mean Time to Change (MTTC):** The average duration between idea and delivery. GitLab measures +MTTC from issue creation to the issue's latest related merge request's deployment to production. +- **Mean Time to Detect (MTTD):** The average duration that a bug goes undetected in production. +GitLab measures MTTD from deployment of bug to issue creation. +- **Mean Time To Merge (MTTM):** The average lifespan of a merge request. GitLab measures MTTM from +merge request creation to merge request merge (and closed/un-merged merge requests are excluded). +For more information, see [Merge Request Analytics](merge_request_analytics.md). +- **Mean Time to Recover/Repair/Resolution/Resolve/Restore (MTTR):** The average duration that a bug +is not fixed in production. GitLab measures MTTR from deployment of bug to deployment of fix. +- **Throughput:** The number of issues closed or merge requests merged (not closed) in a period of +time. Often measured per sprint. GitLab displays merge request throughput in [Merge Request Analytics](merge_request_analytics.md). +- **Value Stream:** The entire work process that is followed to deliver value to customers. For example, +the [DevOps lifecycle](https://about.gitlab.com/stages-devops-lifecycle/) is a value stream that starts +with "plan" and ends with "monitor". GitLab helps you track your value stream using [Value Stream Analytics](value_stream_analytics.md). +- **Velocity:** The total issue burden completed in some period of time. The burden is usually measured +in points or weight, often per sprint. For example, your velocity may be "30 points per sprint". GitLab +measures velocity as the total points or weight of issues closed in a given period of time. + +## Lead time for changes + +"Lead Time for Changes" differs from "Lead Time" because it "focuses on measuring only the time to +deliver a feature once it has been developed", as described in ([Measuring DevOps Performance](https://devops.com/measuring-devops-performance/)). diff --git a/doc/user/analytics/merge_request_analytics.md b/doc/user/analytics/merge_request_analytics.md index 2ff15c00a3f..bb110ab495f 100644 --- a/doc/user/analytics/merge_request_analytics.md +++ b/doc/user/analytics/merge_request_analytics.md @@ -116,4 +116,4 @@ bookmark for those preferred settings in your browser. The **Merge Request Analytics** feature can be accessed only: - On [GitLab Premium](https://about.gitlab.com/pricing/) and above. -- By users with [Reporter access](../permissions.md) and above. +- By users with at least the Reporter [role](../permissions.md). diff --git a/doc/user/analytics/productivity_analytics.md b/doc/user/analytics/productivity_analytics.md index dbe71d5f743..da55a0f093c 100644 --- a/doc/user/analytics/productivity_analytics.md +++ b/doc/user/analytics/productivity_analytics.md @@ -16,37 +16,79 @@ long, on average, it takes to deliver features is an enormous endeavor. While [Value Stream Analytics](../analytics/value_stream_analytics.md) focuses on the entire Software Development Life Cycle (SDLC) process, Productivity Analytics provides a way for Engineering Management to drill down in a systematic way to uncover patterns and causes for success or failure at an individual, project, or group level. -Productivity can slow down for many reasons ranging from degrading codebase to quickly growing teams. In order to investigate, department or team leaders can start by visualizing the time it takes for merge requests to be merged. +Productivity can slow down for many reasons ranging from degrading codebase to quickly growing teams. To investigate, department or team leaders can start by visualizing the time it takes for merge requests to be merged. -## Supported features +## Visualizations and metrics -Productivity Analytics allows GitLab users to: +With Productivity Analytics, GitLab users can: -- Visualize typical merge request (MR) lifetime and statistics. Use a histogram that shows the distribution of the time elapsed between creating and merging merge requests. -- Drill down into the most time consuming merge requests, select a number of outliers, and filter down all subsequent charts to investigate potential causes. +- Visualize typical merge request (MR) lifetime and statistics. A histogram shows the distribution of the time elapsed between creating and merging merge requests. +- Drill down into the most time consuming merge requests, select outliers, and filter subsequent charts to investigate potential causes. - Filter by group, project, author, label, milestone, or a specific date range. For example, filter down to the merge requests of a specific author in a group or project during a milestone or specific date range. -- Measure velocity over time. Visualize the trends of each metric from the charts above over time in order to observe progress. Zoom in on a particular date range if you notice outliers. +- Measure velocity over time. To observe progress, visualize the trends of each metric from the charts over time. Zoom in on a particular date range if you notice outliers. -## Accessing metrics and visualizations +## Metrics charts -To access the chart, navigate to a group's sidebar and select **Analytics > Productivity Analytics**. +To access the charts, navigate to a group's sidebar and select **Analytics > Productivity Analytics**. +Metrics and visualizations of **merged** merge requests are available on a project or group level. -The following metrics and visualizations are available on a project or group level - currently only covering **merged** merge requests: +### Time to merge -- Histogram showing the number of merge request that took a specified number of days to merge after creation. Select a specific column to filter down subsequent charts. -- Histogram showing a breakdown of the time taken (in hours) to merge a merge request. The following intervals are available: - - Time from first commit to first comment. - - Time from first comment until last commit. - - Time from last commit to merge. -- Histogram showing the size or complexity of a merge request, using the following: - - Number of commits per merge request. - - Number of lines of code per commit. - - Number of files touched. -- Scatterplot showing all MRs merged on a certain date, together with the days it took to complete the action and a 30 day rolling median. -- Table showing the list of merge requests with their respective time duration metrics. - - Users can sort by any of the above metrics. +The **Time to merge** histogram shows the number of merge requests and the number +of days it took to merge after creation. Select a column to filter subsequent charts. -## Date ranges + + +### Trendline + +The **Trendline** scatterplot shows all merge requests on a certain date, +and the days it took to complete the action and a 30 day rolling median. Select the dropdown to view: + +- Time from first commit to first comment. +- Time from first comment until last commit. +- Time from last commit to merge. +- Number of commits per merge request. +- Number of lines of code (LOC) per commit. +- Number of files touched. + + + +### Commits and merge request size + +Under the **Trendline** scatterplot, the left-side histogram shows +the time taken (in hours) between commits and comments until the merge +request is merged. Select the dropdown to view: + +- Time from first commit to first comment. +- Time from first comment until last commit. +- Time from last commit to merge. + +The right-side histogram shows the size or complexity of a merge request. +Select the dropdown to view: + +- Number of commits per merge request. +- Number of lines of code (LOC) per commit. +- Number of files touched. + + + +### Merge request list + +The **List** table shows a list of merge requests with their respective time duration metrics. + +Sort metrics by: + +- Time from first commit to first comment. +- Time from first comment until last commit. +- Time from last commit to merge. + +Filter metrics by: + +- Number of commits per merge request. +- Number of lines of code per commit. +- Number of files touched. + +## Filter by date range > [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/13188) in GitLab 12.4. @@ -61,4 +103,4 @@ You can filter analytics based on a date range. To filter results: The **Productivity Analytics** dashboard can be accessed only: - On [GitLab Premium](https://about.gitlab.com/pricing/) and above. -- By users with [Reporter access](../permissions.md) and above. +- By users with at least the Reporter [role](../permissions.md). diff --git a/doc/user/analytics/repository_analytics.md b/doc/user/analytics/repository_analytics.md index e8f8acf5661..936504187b4 100644 --- a/doc/user/analytics/repository_analytics.md +++ b/doc/user/analytics/repository_analytics.md @@ -29,7 +29,7 @@ Commits in a project's [wiki](../project/wiki/index.md#track-wiki-events) are no ### Charts -The data in the charts are updated soon after each commit in the default branch. +The data in the charts are queued. Background workers update the charts 10 minutes after each commit in the default branch. Depending on the size of the GitLab installation, it may take longer for data to refresh due to variations in the size of background job queues. Available charts: |
