Add latest changes from gitlab-org/gitlab@master

9 files changed, 108 insertions, 31 deletions

diff --git a/doc/api/group_milestones.md b/doc/api/group_milestones.md
index d16c0329f72..e157655a713 100644
--- a/doc/api/group_milestones.md
+++ b/doc/api/group_milestones.md
@@ -54,7 +54,6 @@ Example Response:
     "state": "active",
     "updated_at": "2013-10-02T09:24:18Z",
     "created_at": "2013-10-02T09:24:18Z",
-    "expired": false,
     "web_url": "https://gitlab.com/groups/gitlab-org/-/milestones/42"
   }
 ]
diff --git a/doc/api/metrics_user_starred_dashboards.md b/doc/api/metrics_user_starred_dashboards.md
index dd9144d1319..1ed6b15c660 100644
--- a/doc/api/metrics_user_starred_dashboards.md
+++ b/doc/api/metrics_user_starred_dashboards.md
@@ -15,6 +15,7 @@ Parameters:
 
 | Attribute      | Type           | Required | Description                                                                  |
 |:---------------|:---------------|:---------|:-----------------------------------------------------------------------------|
+| `id` | integer/string | yes | The ID or [URL-encoded path of the project](README.md#namespaced-path-encoding) |
 | `dashboard_path` | string        | yes      | URL-encoded path to file defining the dashboard which should be marked as favorite.   |
 
 ```shell
@@ -45,6 +46,7 @@ Parameters:
 
 | Attribute      | Type           | Required | Description                                                                  |
 |:---------------|:---------------|:---------|:-----------------------------------------------------------------------------|
+| `id` | integer/string | yes | The ID or [URL-encoded path of the project](README.md#namespaced-path-encoding) |
 | `dashboard_path` | string        | no      | URL-encoded path to file defining the dashboard which should no longer be marked as favorite. When not supplied all dashboards within given projects will be removed from favorites.   |
 
 ```shell
diff --git a/doc/api/milestones.md b/doc/api/milestones.md
index d736d00779c..b5702c7d6e0 100644
--- a/doc/api/milestones.md
+++ b/doc/api/milestones.md
@@ -51,8 +51,7 @@ Example Response:
     "start_date": "2013-11-10",
     "state": "active",
     "updated_at": "2013-10-02T09:24:18Z",
-    "created_at": "2013-10-02T09:24:18Z",
-    "expired": false
+    "created_at": "2013-10-02T09:24:18Z"
   }
 ]
 ```
diff --git a/doc/api/project_snippets.md b/doc/api/project_snippets.md
index e5dd85fb3bb..fd8cbd6e256 100644
--- a/doc/api/project_snippets.md
+++ b/doc/api/project_snippets.md
@@ -183,6 +183,28 @@ curl "https://gitlab.com/api/v4/projects/:id/snippets/:snippet_id/raw" \
      --header "PRIVATE-TOKEN: <your_access_token>"
 ```
 
+## Snippet repository file content
+
+Returns the raw file content as plain text.
+
+```plaintext
+GET /projects/:id/snippets/:snippet_id/files/:ref/:file_path/raw
+```
+
+Parameters:
+
+- `id` (required) - The ID or [URL-encoded path of the project](README.md#namespaced-path-encoding) owned by the authenticated user
+- `snippet_id` (required) - The ID of a project's snippet
+- `ref` (required) - The name of a branch, tag or commit e.g. master
+- `file_path` (required) - The URL-encoded path to the file, e.g. snippet%2Erb
+
+Example request:
+
+```shell
+curl "https://gitlab.com/api/v4/projects/1/snippets/2/files/master/snippet%2Erb/raw" \
+     --header "PRIVATE-TOKEN: <your_access_token>"
+```
+
 ## Get user agent details
 
 > [Introduced](https://gitlab.com/gitlab-org/gitlab-foss/-/issues/29508) in GitLab 9.4.
diff --git a/doc/api/snippets.md b/doc/api/snippets.md
index 1aa3eecfd29..db94716c2d4 100644
--- a/doc/api/snippets.md
+++ b/doc/api/snippets.md
@@ -150,6 +150,34 @@ Example response:
 Hello World snippet
 ```
 
+## Snippet repository file content
+
+Returns the raw file content as plain text.
+
+```plaintext
+GET /snippets/:id/files/:ref/:file_path/raw
+```
+
+Parameters:
+
+| Attribute   | Type    | Required | Description                                                        |
+|:------------|:--------|:---------|:-------------------------------------------------------------------|
+| `id`        | integer | yes      | ID of snippet to retrieve                                          |
+| `ref`       | string  | yes      | Reference to a tag, branch or commit                               |
+| `file_path` | string  | yes      | URL-encoded path to the file                                       |
+
+Example request:
+
+```shell
+curl --header "PRIVATE-TOKEN: <your_access_token>" "https://gitlab.example.com/api/v4/snippets/1/files/master/snippet%2Erb/raw"
+```
+
+Example response:
+
+```plaintext
+Hello World snippet
+```
+
 ## Create new snippet
 
 Create a new snippet.
diff --git a/doc/development/geo/framework.md b/doc/development/geo/framework.md
index f9e793bbd8c..e8e38372664 100644
--- a/doc/development/geo/framework.md
+++ b/doc/development/geo/framework.md
@@ -385,36 +385,58 @@ Widgets should now be replicated by Geo!
    end
    ```
 
-1. Add fields `widget_count`, `widget_checksummed_count`, `widget_checksum_failed_count`,
-   `widget_synced_count` and `widget_failed_count`
-   to `GeoNodeStatus#RESOURCE_STATUS_FIELDS` array in `ee/app/models/geo_node_status.rb`.
+To do: Add verification on secondaries. This should be done as part of
+[Geo: Self Service Framework - First Implementation for Package File verification](https://gitlab.com/groups/gitlab-org/-/epics/1817)
+
+Widgets should now be verified by Geo!
+
+#### Metrics
+
+Metrics are gathered by `Geo::MetricsUpdateWorker`, persisted in
+`GeoNodeStatus` for display in the UI, and sent to Prometheus.
+
+1. Add fields `widget_count`, `widget_checksummed_count`,
+   `widget_checksum_failed_count`, `widget_synced_count`,
+   `widget_failed_count`, and `widget_registry_count` to
+   `GeoNodeStatus#RESOURCE_STATUS_FIELDS` array in
+   `ee/app/models/geo_node_status.rb`.
 1. Add the same fields to `GeoNodeStatus#PROMETHEUS_METRICS` hash in
    `ee/app/models/geo_node_status.rb`.
 1. Add the same fields to `Sidekiq metrics` table in
    `doc/administration/monitoring/prometheus/gitlab_metrics.md`.
-1. Add the same fields to `GET /geo_nodes/status` example response in `doc/api/geo_nodes.md`.
-1. Modify `GeoNodeStatus#load_verification_data` to make sure the fields mantioned above
-   are set:
+1. Add the same fields to `GET /geo_nodes/status` example response in
+   `doc/api/geo_nodes.md`.
+1. Add the same fields to `ee/spec/models/geo_node_status_spec.rb` and
+   `ee/spec/factories/geo_node_statuses.rb`.
+1. Set `widget_count` in `GeoNodeStatus#load_data_from_current_node`:
 
    ```ruby
-     self.widget_count = Geo::WidgetReplicator.model.count
-     self.widget_checksummed_count = Geo::WidgetReplicator.checksummed.count
-     self.widget_checksum_failed_count = Geo::WidgetReplicator.checksum_failed.count
+   self.widget_count = Geo::WidgetReplicator.primary_total_count
+   ```
+
+1. Add `GeoNodeStatus#load_widgets_data` to set `widget_synced_count`,
+   `widget_failed_count`, and `widget_registry_count`:
+
+   ```ruby
+   def load_widget_data
      self.widget_synced_count = Geo::WidgetReplicator.synced_count
      self.widget_failed_count = Geo::WidgetReplicator.failed_count
+     self.widget_registry_count = Geo::WidgetReplicator.registry_count
+   end
    ```
 
-1. Make sure `Widget` model has `checksummed` and `checksum_failed` scopes.
-1. Update `ee/spec/fixtures/api/schemas/public_api/v4/geo_node_status.json` with new fields.
-1. Update `GeoNodeStatus#PROMETHEUS_METRICS` hash in `ee/app/models/geo_node_status.rb` with new fields.
-1. Update `Sidekiq metrics` table in `doc/administration/monitoring/prometheus/gitlab_metrics.md` with new fields.
-1. Update `GET /geo_nodes/status` example response in `doc/api/geo_nodes.md` with new fields.
-1. Update `ee/spec/models/geo_node_status_spec.rb` and `ee/spec/factories/geo_node_statuses.rb` with new fields.
+1. Call `GeoNodeStatus#load_widgets_data` in
+   `GeoNodeStatus#load_secondary_data`.
 
-To do: Add verification on secondaries. This should be done as part of
-[Geo: Self Service Framework - First Implementation for Package File verification](https://gitlab.com/groups/gitlab-org/-/epics/1817)
+1. Set `widget_checksummed_count` and `widget_checksum_failed_count` in
+   `GeoNodeStatus#load_verification_data`:
 
-Widgets should now be verified by Geo!
+   ```ruby
+   self.widget_checksummed_count = Geo::WidgetReplicator.checksummed_count   self.widget_checksum_failed_count = Geo::WidgetReplicator.checksum_failed_count
+   ```
+
+Widget replication and verification metrics should now be available in the API,
+the Admin Area UI, and Prometheus!
 
 #### GraphQL API
 
diff --git a/doc/development/pipelines.md b/doc/development/pipelines.md
index 05b80cdb4a6..fdccba8e6f8 100644
--- a/doc/development/pipelines.md
+++ b/doc/development/pipelines.md
@@ -370,18 +370,17 @@ and are as follows:
 | MRs | 11 |
 | `master` (non-scheduled pipelines) | 11 |
 | 2-hourly scheduled pipelines | 11 |
+| `nightly` scheduled pipelines | 11, 12 |
 
 #### Long-term plan
 
 We follow the [PostgreSQL versions shipped with Omnibus GitLab](https://docs.gitlab.com/omnibus/package-information/postgresql_versions.html):
 
-| PostgreSQL version | 12.10 (April 2020) | 13.0 (May 2020) | 13.1 (June 2020) | 13.2 (July 2020) | 13.3 (August 2020) | 13.4, 13.5   | 13.6 (November 2020) | 14.0 (May 2021?) |
-| ------ | ------------------ | --------------- | ---------------- | ---------------- | ------------------ | ------------ | -------------------- | ---------------- |
-| PG9.6  | MRs/`master`/`2-hour`/`nightly` | -               | -                | -                | -                  | -            | -                    | -                |
-| PG10   | `nightly`           | -               | -                | -                | -                  | -            | -                    | -                |
-| PG11   | `master`/`2-hour` | MRs/`master`/`2-hour`/`nightly` | MRs/`master`/`2-hour`/`nightly` | MRs/`master`/`2-hour`/`nightly` | MRs/`master`/`2-hour`/`nightly` | MRs/`master`/`2-hour`/`nightly` | `nightly`              | -                |
-| PG12   | -                  | -               | -                | -                | `master`/`2-hour` | `master`/`2-hour` | MRs/`master`/`2-hour`/`nightly`         | `master`/`2-hour` |
-| PG13   | -                  | -               | -                | -                | -                  | -            | -                    | MRs/`master`/`2-hour`/`nightly`     |
+| PostgreSQL version | 13.0 (May 2020) | 13.1 (June 2020) | 13.2 (July 2020) | 13.3 (August 2020) | 13.4, 13.5   | 13.6 (November 2020) | 14.0 (May 2021?) |
+| ------ | --------------- | ---------------- | ---------------- | ------------------ | ------------ | -------------------- | ---------------- |
+| PG11   | MRs/`master`/`2-hour`/`nightly` | MRs/`master`/`2-hour`/`nightly` | MRs/`master`/`2-hour`/`nightly` | MRs/`master`/`2-hour`/`nightly` | MRs/`master`/`2-hour`/`nightly` | `nightly` | - |
+| PG12   | - | - | `nightly` | `2-hour`/`nightly` | `2-hour`/`nightly` | MRs/`2-hour`/`nightly` | `2-hour`/`nightly`     |
+| PG13   | - | - | -         | -                  | -                  | -                      | MRs/`2-hour`/`nightly` |
 
 ### Test jobs
 
diff --git a/doc/development/telemetry/usage_ping.md b/doc/development/telemetry/usage_ping.md
index 409f9ff2a5e..0ca1018352c 100644
--- a/doc/development/telemetry/usage_ping.md
+++ b/doc/development/telemetry/usage_ping.md
@@ -708,6 +708,7 @@ appear to be associated to any of the services running, since they all appear to
 | `nodes`                                                   | `topology`                           | `enablement`  |                  |         | The list of server nodes on which GitLab components are running            |
 | `node_memory_total_bytes`                                 | `topology > nodes`                   | `enablement`  |                  |         | The total available memory of this node                                    |
 | `node_cpus`                                               | `topology > nodes`                   | `enablement`  |                  |         | The number of CPU cores of this node                                       |
+| `node_uname_info`                                         | `topology > nodes`                   | `enablement`  |                  |         | The basic hardware architecture and OS release information on this node    |
 | `node_services`                                           | `topology > nodes`                   | `enablement`  |                  |         | The list of GitLab services running on this node                           |
 | `name`                                                    | `topology > nodes > node_services`   | `enablement`  |                  |         | The name of the GitLab service running on this node                        |
 | `process_count`                                           | `topology > nodes > node_services`   | `enablement`  |                  |         | The number of processes running for this service                           |
@@ -920,6 +921,11 @@ The following is example content of the Usage Ping payload.
       {
         "node_memory_total_bytes": 33269903360,
         "node_cpus": 16,
+        "node_uname_info": {
+          "machine": "x86_64",
+          "sysname": "Linux",
+          "release": "4.19.76-linuxkit"
+        },
         "node_services": [
           {
             "name": "web",
diff --git a/doc/integration/openid_connect_provider.md b/doc/integration/openid_connect_provider.md
index 3da17347e91..b66262772da 100644
--- a/doc/integration/openid_connect_provider.md
+++ b/doc/integration/openid_connect_provider.md
@@ -37,11 +37,11 @@ Currently the following user information is shared with clients:
 | `auth_time`      | `integer` | The timestamp for the user's last authentication
 | `name`           | `string`  | The user's full name
 | `nickname`       | `string`  | The user's GitLab username
-| `email`          | `string`  | The user's public email address
-| `email_verified` | `boolean` | Whether the user's public email address was verified
+| `email`          | `string`  | The user's email address<br>This is the user's *primary* email address if the application has access to the `email` claim and the user's *public* email address otherwise
+| `email_verified` | `boolean` | Whether the user's email address was verified
 | `website`        | `string`  | URL for the user's website
 | `profile`        | `string`  | URL for the user's GitLab profile
 | `picture`        | `string`  | URL for the user's GitLab avatar
 | `groups`         | `array`   | Names of the groups the user is a member of
 
-Only the `sub` and `sub_legacy` claims are included in the ID token, all other claims are available from the `/oauth/userinfo` endpoint used by OIDC clients.
+The claims `sub`, `sub_legacy`, `email` and `email_verified` are included in the ID token, all other claims are available from the `/oauth/userinfo` endpoint used by OIDC clients.