Add latest changes from gitlab-org/gitlab@master

4 files changed, 182 insertions, 147 deletions

diff --git a/doc/development/architecture.md b/doc/development/architecture.md
index 2bb5ab6bca6..984ac49c514 100644
--- a/doc/development/architecture.md
+++ b/doc/development/architecture.md
@@ -378,6 +378,7 @@ Component statuses are linked to configuration documentation for each component.
 | [Runner](#gitlab-runner)                              | Executes GitLab CI/CD jobs                                           |       ⤓        |       ⤓        |      ✅       |        ⚙         |     ✅      |   ⚙    |  ⚙  | CE & EE |
 | [Sentry integration](#sentry)                         | Error tracking for deployed apps                                     |       ⤓        |       ⤓        |      ⤓       |        ⤓         |     ⤓      |   ⤓    |  ⤓  | CE & EE |
 | [Sidekiq](#sidekiq)                                   | Background jobs processor                                            |       ✅       |       ✅        |      ✅       |        ✅         |     ✅      |   ✅    |  ✅  | CE & EE |
+| [Token Revocation API](sec/token_revocation_api.md)   | Receives and revokes leaked secrets                                  |       ❌       |       ❌        |      ❌       |        ❌         |     ✅      |   ❌    |  ❌  | EE Only |
 
 ### Component details
 
diff --git a/doc/development/sec/token_revocation_api.md b/doc/development/sec/token_revocation_api.md
new file mode 100644
index 00000000000..15d1d2d0ef3
--- /dev/null
+++ b/doc/development/sec/token_revocation_api.md
@@ -0,0 +1,118 @@
+---
+stage: Secure
+group: Static Analysis
+info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/engineering/ux/technical-writing/#assignments
+---
+
+# Token Revocation API
+
+The Token Revocation API is an externally-deployed HTTP API that interfaces with GitLab
+to receive and revoke API tokens and other secrets detected by GitLab Secret Detection.
+See the [high-level architecture](../../user/application_security/secret_detection/post_processing.md)
+to understand the Secret Detection post-processing and revocation flow.
+
+GitLab.com uses the internally-maintained [Secret Revocation Service](https://gitlab.com/gitlab-com/gl-security/engineering-and-research/automation-team/secret-revocation-service)
+(team-members only) as its Token Revocation API. For GitLab self-managed, you can create
+your own API and configure GitLab to use it.
+
+## Implement a Token Revocation API for self-managed
+
+GitLab self-managed instances interested in using the revocation capabilities must:
+
+- Implement and deploy your own Token Revocation API.
+- Configure the GitLab instance to use the Token Revocation API.
+
+Your service must:
+
+- Match the API specification below.
+- Provide two endpoints:
+  - Fetching revocable token types.
+  - Revoking leaked tokens.
+- Be rate-limited and idempotent.
+
+Requests to the documented endpoints are authenticated using API tokens passed in
+the `Authorization` header. Request and response bodies, if present, are
+expected to have the content type `application/json`.
+
+All endpoints may return these responses:
+
+- `401 Unauthorized`
+- `405 Method Not Allowed`
+- `500 Internal Server Error`
+
+### `GET /v1/revocable_token_types`
+
+Returns the valid `type` values for use in the `revoke_tokens` endpoint.
+
+NOTE:
+These values match the concatenation of [the `secrets` analyzer's](../../user/application_security/secret_detection/index.md)
+[primary identifier](../integrations/secure.md#identifiers) by means
+of concatenating the `primary_identifier.type` and `primary_identifier.value`.
+For example, the value `gitleaks_rule_id_gitlab_personal_access_token` matches the following finding identifier:
+
+```json
+{"type": "gitleaks_rule_id", "name": "Gitleaks rule ID GitLab Personal Access Token", "value": "GitLab Personal Access Token"}
+```
+
+| Status Code | Description |
+| ----- | --- |
+| `200` | The response body contains the valid token `type` values. |
+
+Example response body:
+
+```json
+{
+    "types": ["gitleaks_rule_id_gitlab_personal_access_token"]
+}
+```
+
+### `POST /v1/revoke_tokens`
+
+Accepts a list of tokens to be revoked by the appropriate provider. Your service is responsible for communicating
+with each provider to revoke the token.
+
+| Status Code | Description |
+| ----- | --- |
+| `204` | All submitted tokens have been accepted for eventual revocation. |
+| `400` | The request body is invalid or one of the submitted token types is not supported. The request should not be retried. |
+| `429` | The provider has received too many requests. The request should be retried later. |
+
+Example request body:
+
+```json
+[{
+    "type": "gitleaks_rule_id_gitlab_personal_access_token",
+    "token": "glpat--8GMtG8Mf4EnMJzmAWDU",
+    "location": "https://example.com/some-repo/blob/abcdefghijklmnop/compromisedfile1.java"
+},
+{
+    "type": "gitleaks_rule_id_gitlab_personal_access_token",
+    "token": "glpat--tG84EGK33nMLLDE70zU",
+    "location": "https://example.com/some-repo/blob/abcdefghijklmnop/compromisedfile2.java"
+}]
+```
+
+### Configure GitLab to interface with the Token Revocation API
+
+You must configure the following database settings in the GitLab instance:
+
+| Setting | Type | Description |
+| ------- | ---- | ----------- |
+| `secret_detection_token_revocation_enabled` | Boolean | Whether automatic token revocation is enabled |
+| `secret_detection_token_revocation_url` | String | A fully-qualified URL to the `/v1/revoke_tokens` endpoint of the Token Revocation API |
+| `secret_detection_revocation_token_types_url` | String | A fully-qualified URL to the `/v1/revocable_token_types` endpoint of the Token Revocation API |
+| `secret_detection_token_revocation_token` | String | A pre-shared token to authenticate requests to the Token Revocation API |
+
+For example, to configure these values in the
+[Rails console](../../administration/operations/rails_console.md#starting-a-rails-console-session):
+
+```ruby
+::Gitlab::CurrentSettings.update!(secret_detection_token_revocation_token: 'MYSECRETTOKEN')
+::Gitlab::CurrentSettings.update!(secret_detection_token_revocation_url: 'https://example.gitlab.com/revocation_service/v1/revoke_tokens')
+::Gitlab::CurrentSettings.update!(secret_detection_revocation_token_types_url: 'https://example.gitlab.com/revocation_service/v1/revocable_token_types')
+::Gitlab::CurrentSettings.update!(secret_detection_token_revocation_enabled: true)
+```
+
+After you configure these values, the Token Revocation API will be called according to the
+[high-level architecture](../../user/application_security/secret_detection/post_processing.md#high-level-architecture)
+diagram.
diff --git a/doc/user/application_security/secret_detection/index.md b/doc/user/application_security/secret_detection/index.md
index d316ed7a9de..d6aab71a2c6 100644
--- a/doc/user/application_security/secret_detection/index.md
+++ b/doc/user/application_security/secret_detection/index.md
@@ -195,7 +195,12 @@ Pipelines now include a Secret Detection job.
 
 ## Responding to a leaked secret
 
-If the scanner detects a secret you should rotate it immediately. [Purging a file from the repository's history](../../project/repository/reducing_the_repo_size_using_git.md#purge-files-from-repository-history) may not be effective in removing all references to the file. Also, the secret remains in any forks of the repository.
+Secrets detected by the analyzer should be immediately rotated.
+[Purging a file from the repository's history](../../project/repository/reducing_the_repo_size_using_git.md#purge-files-from-repository-history)
+may not be effective in removing all references to the file. Additionally, the secret will remain in any existing
+forks or clones of the repository.
+
+GitLab will attempt to [automatically revoke](post_processing.md) some types of leaked secrets.
 
 ## Pinning to specific analyzer version
 
diff --git a/doc/user/application_security/secret_detection/post_processing.md b/doc/user/application_security/secret_detection/post_processing.md
index 6e9b77426ad..10742114b90 100644
--- a/doc/user/application_security/secret_detection/post_processing.md
+++ b/doc/user/application_security/secret_detection/post_processing.md
@@ -10,26 +10,28 @@ info: To determine the technical writer assigned to the Stage/Group associated w
 > - [Disabled by default for GitLab personal access tokens](https://gitlab.com/gitlab-org/gitlab/-/issues/371658) in GitLab 15.6 [with a flag](../../../administration/feature_flags.md) named `gitlab_pat_auto_revocation`. Available to GitLab.com only.
 > - [Enabled by default for GitLab personal access tokens](https://gitlab.com/gitlab-org/gitlab/-/issues/371658) in GitLab 15.9
 
-GitLab supports running post-processing hooks after detecting a secret. These
-hooks can perform actions, like notifying the cloud service that issued the secret.
-The cloud provider can then confirm the credentials and take remediation actions, like:
+GitLab.com and self-managed supports running post-processing hooks after detecting a secret. These
+hooks can perform actions, like notifying the vendor that issued the secret.
+The vendor can then confirm the credentials and take remediation actions, like:
 
 - Revoking a secret.
 - Reissuing a secret.
 - Notifying the creator of the secret.
 
-GitLab SaaS supports post-processing for [GitLab personal access tokens](../../profile/personal_access_tokens.md) and Amazon Web Services (AWS).
-Post-processing workflows vary by supported cloud providers.
+GitLab supports post-processing for the following vendors and secrets:
 
-Post-processing is limited to a project's default branch. The epic
-[Post-processing of leaked secrets](https://gitlab.com/groups/gitlab-org/-/epics/4639).
-contains:
+| Vendor | Secret | GitLab.com | Self-managed |
+| ----- | --- | --- | --- |
+| GitLab | [Personal access tokens](../../profile/personal_access_tokens.md) | ✅ | ✅ 15.9 and later |
+| Amazon Web Services (AWS) | [IAM access keys](https://docs.aws.amazon.com/IAM/latest/UserGuide/id_credentials_access-keys.html) | ✅ | ⚙ |
 
-- Technical details of post-processing secrets.
-- Discussions of efforts to support additional branches.
+**Component legend**
+
+- ✅ - Available by default
+- ⚙ - Requires manual integration using a [Token Revocation API](../../../development/sec/token_revocation_api.md)
 
 NOTE:
-Post-processing is currently limited to a project's default branch
+Post-processing is limited to a project's default branch.
 
 ## High-level architecture
 
@@ -40,142 +42,51 @@ sequenceDiagram
     autonumber
     GitLab Rails->>+Sidekiq: gl-secret-detection-report.json
     Sidekiq-->+Sidekiq: StoreSecurityReportsWorker
-    Sidekiq-->+RevocationAPI: GET revocable keys types
-    RevocationAPI-->>-Sidekiq: OK
-    Sidekiq->>+RevocationAPI: POST revoke revocable keys
-    RevocationAPI-->>-Sidekiq: ACCEPTED
-    RevocationAPI-->>+Cloud Vendor: revoke revocable keys
-    Cloud Vendor-->>+RevocationAPI: ACCEPTED
-```
-
-## Integrate your cloud provider service with GitLab SaaS
-
-Third party cloud and SaaS providers can [express integration interest by filling out this form](https://forms.gle/wWpvrtLRK21Q2WJL9).
-
-### Implement a vendor revocation receiver service
-
-A vendor revocation receiver service integrates with a GitLab instance to receive
-a web notification and respond to leaked token requests.
-
-To implement a receiver service to revoke leaked tokens:
-
-1. Create a publicly accessible HTTP service matching the corresponding API contract
-   below. Your service should be idempotent and rate-limited.
-1. When a pipeline corresponding to its revocable token type (in the example, `my_api_token`)
-   completes, GitLab sends a request to your receiver service.
-1. The included URL should be publicly accessible, and contain the commit where the
-   leaked token can be found. For example:
-
-    ```plaintext
-    POST / HTTP/2
-    Accept: */*
-    Content-Type: application/json
-    X-Gitlab-Token: MYSECRETTOKEN
-
-    [
-      {"type": "my_api_token", "token":"XXXXXXXXXXXXXXXX","url": "https://example.com/some-repo/blob/abcdefghijklmnop/compromisedfile1.java"}
-    ]
-    ```
-
-## Implement a revocation service for self-managed
-
-Self-managed instances interested in using the revocation capabilities must:
-
-- Deploy the [RevocationAPI](#high-level-architecture).
-- Configure the GitLab instance to use the RevocationAPI.
-
-A RevocationAPI must:
-
-- Match a minimal API specification.
-- Provide two endpoints:
-  - Fetching revocable token types.
-  - Revoking leaked tokens.
-- Be rate-limited and idempotent.
-
-Requests to the documented endpoints are authenticated via API tokens passed in
-the `Authorization` header. Request and response bodies, if present, are
-expected to have the content type `application/json`.
-
-All endpoints may return these responses:
-
-- `401 Unauthorized`
-- `405 Method Not Allowed`
-- `500 Internal Server Error`
-
-### `GET /v1/revocable_token_types`
-
-Returns the valid `type` values for use in the `revoke_tokens` endpoint.
-
-NOTE:
-These values match the concatenation of [the `secrets` analyzer's](index.md)
-[primary identifier](../../../development/integrations/secure.md#identifiers) by means
-of concatenating the `primary_identifier.type` and `primary_identifier.value`.
-In the case below, a finding identifier matches:
-
-```json
-{"type": "gitleaks_rule_id", "name": "Gitleaks rule ID GitLab Personal Access Token", "value": "GitLab Personal Access Token"}
+    Sidekiq-->+Token Revocation API: GET revocable keys types
+    Token Revocation API-->>-Sidekiq: OK
+    Sidekiq->>+Token Revocation API: POST revoke revocable keys
+    Token Revocation API-->>-Sidekiq: ACCEPTED
+    Token Revocation API-->>+Receiver Service: revoke revocable keys
+    Receiver Service-->>+Token Revocation API: ACCEPTED
 ```
 
-| Status Code | Description |
-| ----- | --- |
-| `200` | The response body contains the valid token `type` values. |
-
-Example response body:
-
-```json
-{
-    "types": ["gitleaks_rule_id_gitlab_personal_access_token"]
-}
+1. A pipeline with a Secret Detection job completes on the project's default branch, producing a scan
+   report (**1**).
+1. The report is processed (**2**) by an asynchronous worker, which communicates with an externally
+   deployed HTTP service (**3** and **4**) to determine which kinds of secrets can be automatically
+   revoked.
+1. The worker sends (**5** and **6**) the list of detected secrets which the Token Revocation API is able to
+   revoke.
+1. The Token Revocation API sends (**7** and **8**) each revocable token to their respective vendor's [receiver service](#integrate-your-cloud-provider-service-with-gitlabcom).
+
+See the [Token Revocation API](../../../development/sec/token_revocation_api.md) documentation for more
+information.
+
+## Integrate your cloud provider service with GitLab.com
+
+Third-party cloud and SaaS vendors interested in automated token revocation can
+[express integration interest by filling out this form](https://forms.gle/wWpvrtLRK21Q2WJL9).
+Vendors must [implement a revocation receiver service](#implement-a-revocation-receiver-service)
+which will be called by the Token Revocation API.
+
+### Implement a revocation receiver service
+
+A revocation receiver service integrates with a GitLab instance's Token Revocation API to receive and respond
+to leaked token revocation requests. The service should be a publicly accessible HTTP API that is
+idempotent and rate-limited. Requests to your service from the Token Revocation API will follow the example
+below:
+
+```plaintext
+POST / HTTP/2
+Accept: */*
+Content-Type: application/json
+X-Gitlab-Token: MYSECRETTOKEN
+
+[
+  {"type": "my_api_token", "token":"XXXXXXXXXXXXXXXX","url": "https://example.com/some-repo/~/raw/abcdefghijklmnop/compromisedfile1.java"}
+]
 ```
 
-### `POST /v1/revoke_tokens`
-
-Accepts a list of tokens to be revoked by the appropriate provider.
-
-| Status Code | Description |
-| ----- | --- |
-| `204` | All submitted tokens have been accepted for eventual revocation. |
-| `400` | The request body is invalid or one of the submitted token types is not supported. The request should not be retried. |
-| `429` | The provider has received too many requests. The request should be retried later. |
-
-Example request body:
-
-```json
-[{
-    "type": "gitleaks_rule_id_gitlab_personal_access_token",
-    "token": "glpat--8GMtG8Mf4EnMJzmAWDU",
-    "location": "https://example.com/some-repo/blob/abcdefghijklmnop/compromisedfile1.java"
-},
-{
-    "type": "gitleaks_rule_id_gitlab_personal_access_token",
-    "token": "glpat--tG84EGK33nMLLDE70zU",
-    "location": "https://example.com/some-repo/blob/abcdefghijklmnop/compromisedfile2.java"
-}]
-```
-
-### Configure GitLab to interface with RevocationAPI
-
-You must configure the following database settings in the GitLab instance:
-
-- `secret_detection_token_revocation_enabled`
-- `secret_detection_token_revocation_url`
-- `secret_detection_token_revocation_token`
-- `secret_detection_revocation_token_types_url`
-
-For example, to configure these values in the
-[Rails console](../../../administration/operations/rails_console.md#starting-a-rails-console-session):
-
-```ruby
-::Gitlab::CurrentSettings.update!(secret_detection_token_revocation_token: 'MYSECRETTOKEN')
-::Gitlab::CurrentSettings.update!(secret_detection_token_revocation_url: 'https://example.gitlab.com/revocation_service/v1/revoke_tokens')
-::Gitlab::CurrentSettings.update!(secret_detection_revocation_token_types_url: 'https://example.gitlab.com/revocation_service/v1/revocable_token_types')
-::Gitlab::CurrentSettings.update!(secret_detection_token_revocation_enabled: true)
-```
-
-After you configure these values, completing a pipeline performs these actions:
-
-1. The revocation service is triggered once.
-1. A request is made to `secret_detection_revocation_token_types_url` to fetch a
-   list of revocable tokens.
-1. Any Secret Detection findings matching the results of the `token_types` request
-   are included in the subsequent revocation request.
+In this example, Secret Detection has determined that an instance of `my_api_token` has been leaked. The
+value of the token is provided to you, in addition to a publicly accessible URL to the raw content of the
+file containing the leaked token.