Add latest changes from gitlab-org/gitlab@masterHEADmaster

11 files changed, 168 insertions, 120 deletions

diff --git a/doc/api/graphql/reference/index.md b/doc/api/graphql/reference/index.md
index b21e82c9b54..92363f5dad8 100644
--- a/doc/api/graphql/reference/index.md
+++ b/doc/api/graphql/reference/index.md
@@ -20240,6 +20240,7 @@ four standard [pagination arguments](#connection-pagination-arguments):
 | <a id="groupprojectshascodecoverage"></a>`hasCodeCoverage` | [`Boolean`](#boolean) | Returns only the projects which have code coverage. |
 | <a id="groupprojectshasvulnerabilities"></a>`hasVulnerabilities` | [`Boolean`](#boolean) | Returns only the projects which have vulnerabilities. |
 | <a id="groupprojectsids"></a>`ids` | [`[ID!]`](#id) | Filter projects by IDs. |
+| <a id="groupprojectsincludearchived"></a>`includeArchived` | [`Boolean`](#boolean) | Include also archived projects. |
 | <a id="groupprojectsincludesubgroups"></a>`includeSubgroups` | [`Boolean`](#boolean) | Include also subgroup projects. |
 | <a id="groupprojectsnotaimedfordeletion"></a>`notAimedForDeletion` | [`Boolean`](#boolean) | Include projects that are not aimed for deletion. |
 | <a id="groupprojectssbomcomponentid"></a>`sbomComponentId` | [`ID`](#id) | Return only the projects related to the specified SBOM component. |
@@ -23297,6 +23298,7 @@ four standard [pagination arguments](#connection-pagination-arguments):
 | <a id="namespaceprojectshascodecoverage"></a>`hasCodeCoverage` | [`Boolean`](#boolean) | Returns only the projects which have code coverage. |
 | <a id="namespaceprojectshasvulnerabilities"></a>`hasVulnerabilities` | [`Boolean`](#boolean) | Returns only the projects which have vulnerabilities. |
 | <a id="namespaceprojectsids"></a>`ids` | [`[ID!]`](#id) | Filter projects by IDs. |
+| <a id="namespaceprojectsincludearchived"></a>`includeArchived` | [`Boolean`](#boolean) | Include also archived projects. |
 | <a id="namespaceprojectsincludesubgroups"></a>`includeSubgroups` | [`Boolean`](#boolean) | Include also subgroup projects. |
 | <a id="namespaceprojectsnotaimedfordeletion"></a>`notAimedForDeletion` | [`Boolean`](#boolean) | Include projects that are not aimed for deletion. |
 | <a id="namespaceprojectssbomcomponentid"></a>`sbomComponentId` | [`ID`](#id) | Return only the projects related to the specified SBOM component. |
diff --git a/doc/architecture/blueprints/capacity_planning/images/dedicated-capacity-planning-as-a-service.png b/doc/architecture/blueprints/capacity_planning/images/dedicated-capacity-planning-as-a-service.png
new file mode 100644
index 00000000000..c30fa2970eb
--- /dev/null
+++ b/doc/architecture/blueprints/capacity_planning/images/dedicated-capacity-planning-as-a-service.png
diff --git a/doc/architecture/blueprints/capacity_planning/images/dedicated-capacity-planning-forecasting.png b/doc/architecture/blueprints/capacity_planning/images/dedicated-capacity-planning-forecasting.png
new file mode 100644
index 00000000000..25d959560e2
--- /dev/null
+++ b/doc/architecture/blueprints/capacity_planning/images/dedicated-capacity-planning-forecasting.png
diff --git a/doc/architecture/blueprints/capacity_planning/images/dedicated-capacity-planning-reporting.png b/doc/architecture/blueprints/capacity_planning/images/dedicated-capacity-planning-reporting.png
new file mode 100644
index 00000000000..9d9a207571c
--- /dev/null
+++ b/doc/architecture/blueprints/capacity_planning/images/dedicated-capacity-planning-reporting.png
diff --git a/doc/architecture/blueprints/capacity_planning/images/tamland-as-a-service.png b/doc/architecture/blueprints/capacity_planning/images/tamland-as-a-service.png
deleted file mode 100644
index fa8f1223917..00000000000
--- a/doc/architecture/blueprints/capacity_planning/images/tamland-as-a-service.png
+++ /dev/null
diff --git a/doc/architecture/blueprints/capacity_planning/images/tamland-as-part-of-stack.png b/doc/architecture/blueprints/capacity_planning/images/tamland-as-part-of-stack.png
deleted file mode 100644
index 0b47d91e133..00000000000
--- a/doc/architecture/blueprints/capacity_planning/images/tamland-as-part-of-stack.png
+++ /dev/null
diff --git a/doc/architecture/blueprints/capacity_planning/index.md b/doc/architecture/blueprints/capacity_planning/index.md
index 31740d50368..0d1ad84c914 100644
--- a/doc/architecture/blueprints/capacity_planning/index.md
+++ b/doc/architecture/blueprints/capacity_planning/index.md
@@ -3,7 +3,7 @@ status: proposed
 creation-date: "2023-09-11"
 authors: [ "@abrandl" ]
 coach: "@andrewn"
-approvers: [ "@swiskow", "@rnienaber", "@o-lluch" ]
+approvers: [ "@swiskow", "@lmcandrew", "@o-lluch" ]
 ---
 
 <!-- Blueprints often contain forward-looking statements -->
@@ -19,7 +19,7 @@ We make use of [Tamland](https://gitlab.com/gitlab-com/gl-infra/tamland), a tool
 We propose to include Tamland as a part of the GitLab Dedicated stack and execute forecasting from within the tenant environments.
 
 Tamland predicts SLO violations and their respective dates, which need to be reviewed and acted upon.
-In terms of team organisation, the Dedicated team is proposed to own the tenant-side setup for Tamland and to own the predicted SLO violations, with the help and guidance of the Scalability::Projections team, which drives further development, documentation and overall guidance for capacity planning, including for Dedicated.
+In terms of team organisation, the Dedicated team is proposed to own the tenant-side setup for Tamland and to own the predicted SLO violations, with the help and guidance of the Scalability::Observability team, which drives further development, documentation and overall guidance for capacity planning, including for Dedicated.
 
 With this setup, we aim to turn Tamland into a more generic tool, which can be used in various environments including but not limited to Dedicated tenants.
 Long-term, we think of including Tamland in self-managed installations and think of Tamland as a candidate for open source release.
@@ -32,8 +32,8 @@ Long-term, we think of including Tamland in self-managed installations and think
 It implements [capacity planning](https://about.gitlab.com/handbook/engineering/infrastructure/capacity-planning/) for GitLab.com, which is a [controlled activity covered by SOC 2](https://gitlab.com/gitlab-com/gl-security/security-assurance/security-compliance-commercial-and-dedicated/observation-management/-/issues/604).
 As of today, it is used exclusively for GitLab.com to predict upcoming SLO violations across hundreds of monitored infrastructure components.
 
-Tamland produces a [report](https://gitlab-com.gitlab.io/gl-infra/tamland/intro.html) (hosted on GitLab Pages) containing forecast plots, information around predicted violations and other information around the components monitored.
-Any predicted SLO violation result in a capacity warning issue being created in the [issue tracker for capacity planning](https://gitlab.com/gitlab-com/gl-infra/capacity-planning/-/boards/2816983) on GitLab.com.
+Tamland produces a [report](https://gitlab-com.gitlab.io/gl-infra/tamland/intro.html) (internal link, hosted on GitLab Pages) containing forecast plots, information around predicted violations and other information around the components monitored.
+Any predicted SLO violation results in a capacity warning issue being created in the [issue tracker for capacity planning](https://gitlab.com/gitlab-com/gl-infra/capacity-planning/-/boards/2816983) on GitLab.com.
 
 At present, Tamland is quite tailor made and specific for GitLab.com:
 
@@ -44,13 +44,13 @@ At present, Tamland is quite tailor made and specific for GitLab.com:
 
 For illustration, we can see a saturation forecast plot below for the `disk_space` resource for a PostgreSQL service called `patroni-ci`.
 Within the 90 days forecast horizon, we predict a violation of the `soft` SLO (set at 85% saturation) and this resulted in the creation of a [capacity planning issue](https://gitlab.com/gitlab-com/gl-infra/capacity-planning/-/issues/1219) for further review and potential actions.
-At present, the Scalability::Projections group reviews those issues and engages with the respective DRI for the service in question to remedy a saturation concern.
+At present, the Scalability::Observability group reviews those issues and engages with the respective DRI for the service in question to remedy a saturation concern.
 
 <img src="images/image-20230911144743188.png" alt="image-20230911144743188" style="zoom:67%;" />
 
 For GitLab.com capacity planning, we operate Tamland from a scheduled CI pipeline with access to the central Thanos, which provides saturation and utilization metrics for GitLab.com.
 The CI pipeline produces the desired report, exposes it on GitLab Pages and also creates capacity planning issues.
-Scalability::Projections runs a capacity planning triage rotation which entails reviewing and prioritizing any open issues and their respective saturation concerns.
+Scalability::Observability runs a capacity planning triage rotation which entails reviewing and prioritizing any open issues and their respective saturation concerns.
 
 ### Problem Statement
 
@@ -62,7 +62,7 @@ These metrics are standardized in the [metrics catalog](https://gitlab.com/gitla
 
 In order to provide capacity planning and forecasts for saturation metrics for each tenant, we'd like to get Tamland set up for GitLab Dedicated.
 
-While Tamland is developed by the Scalability::Projections and this team also owns the capacity planning process for GitLab.com, they don't have access to any of the Dedicated infrastructure as we have strong isolation implemented for Dedicated environments.
+While Tamland is developed by the Scalability::Observability and this team also owns the capacity planning process for GitLab.com, they don't have access to any of the Dedicated infrastructure as we have strong isolation implemented for Dedicated environments.
 As such, the technical design choices are going to affect how those teams interact and vice versa. We include this consideration into this documentation as we think the organisational aspect is a crucial part of it.
 
 ### Key questions
@@ -79,21 +79,27 @@ As such, the technical design choices are going to affect how those teams intera
 1. Tamland's output is forecasting data only (plots, SLO violation dates, etc. - no report, no issue management - see below)
 1. Tamland stores the output data in a S3 bucket for further inspection
 
-#### Non-goals
+### Goals: Iteration 1
 
-##### Reporting
+In Iteration 0, we've integrated Tamland into GitLab Dedicated environments and started to generate forecasting data for each tenant regularly.
 
-As of today, it's not quite clear yet how we'd like to consume forecasting data across tenants.
-In contrast to GitLab.com, we generate forecasts across a potentially large number of tenants.
-At this point, we suspect that we're more interested in an aggregate report across tenants rather than individual, very detailed saturation forecasts.
-As such, this is subject to refinement in a further iteration once we have the underlying data available and gathered practical insight in how we consume this information.
+In order to consume this data and make it actionable, this iteration is about providing reporting functionality for GitLab Dedicated:
+We generate a GitLab Pages deployed static site that contains individual Tamland reports for all tenants.
 
-##### Issue management
+We use the default Tamland report to generate the per-tenant report.
+In a future iteration, we may want to provide another type of report specifically tailored for GitLab Dedicated needs.
 
-While each predicted SLO violation results in the creation of a GitLab issue, this may not be the right mode of raising awareness for Dedicated.
-Similar to the reporting side, this is subject to further discussion once we have data to look at.
+### Goals: Iteration 2
 
-##### Customizing forecasting models
+In order to raise awareness for a predicted SLO violation, Tamland has functionality to manage a GitLab issue tracker and create an issue for a capacity warning.
+We use this, for example, to manage capacity warnings for GitLab.com using the [`gitlab-com` capacity planning tracker](https://gitlab.com/gitlab-com/gl-infra/capacity-planning-trackers/gitlab-com/-/issues).
+
+For GitLab Dedicated tenants, we suggest to use the `gitlab-dedicated` capacity planning tracker in a similar fashion:
+For each predicted SLO violation with reasonable confidence, we create a capacity warning issue on this tracker and use a scoped label to distinguish warnings for different tenants (see below for more details).
+
+### Non-goals
+
+#### Customizing forecasting models
 
 Forecasting models can and should be tuned and informed with domain knowledge to produce accurate forecasts.
 This information is a part of the Tamland manifest.
@@ -105,9 +111,11 @@ Dedicated environments are fully isolated and run their own Prometheus instance
 Tamland will run from each individual Dedicated tenant environment, consume metrics from Prometheus and store the resulting data in S3.
 From there, we consume forecast data and act on it.
 
-![tamland-as-part-of-stack](images/tamland-as-part-of-stack.png)
+![dedicated-capacity-planning-forecasting](images/dedicated-capacity-planning-forecasting.png)
 
-### Storage for output and cache
+### Generating forecasts
+
+#### Storage for output and cache
 
 Any data Tamland relies on is stored in a S3 bucket.
 We use one bucket per tenant to clearly separate data between tenants.
@@ -117,7 +125,7 @@ We use one bucket per tenant to clearly separate data between tenants.
 
 There is no need for a persistent state across Tamland runs aside from the S3 bucket.
 
-### Benefits of executing inside tenant environments
+#### Benefits of executing inside tenant environments
 
 Each Tamland run for a single environment (tenant) can take a few hours to execute.
 With the number of tenants expected to increase significantly, we need to consider scaling the execution environment for Tamland.
@@ -125,11 +133,11 @@ With the number of tenants expected to increase significantly, we need to consid
 In this design, Tamland becomes a part of the Dedicated stack and a component of the individual tenant environment.
 As such, scaling the execution environment for Tamland is solved by design, because tenant forecasts execute inherently parallel in their respective environments.
 
-### Distribution model: Docker
+#### Distribution model: Docker + Helm chart
 
-Tamland is released as a Docker image, see [Tamland's README](https://gitlab.com/gitlab-com/gl-infra/tamland/-/blob/main/README.md) for further details.
+Tamland is released as a Docker image along with a Helm chart, see [Tamland's README](https://gitlab.com/gitlab-com/gl-infra/tamland/-/blob/main/README.md) for further details.
 
-### Tamland manifest
+#### Tamland Manifest
 
 The manifest contains information about which saturation metrics to forecast on (see this [manifest example](https://gitlab.com/gitlab-com/gl-infra/tamland/-/blob/62854e1afbc2ed3160a55a738ea587e0cf7f994f/saturation.json) for GitLab.com).
 This will be generated from the metrics catalog and will be the same for all tenants for starters.
@@ -139,10 +147,33 @@ On a regular basis, a scheduled pipeline grabs the metrics catalog, generates th
 
 On the Dedicated tenants, we download the latest version of the committed JSON manifest from `tamland-dedicated` and use this as input to execute Tamland.
 
-### Acting on forecast insights
+### Capacity planning reports and Capacity Warnings
+
+Based on Tamland's forecasting data, we generate reports to display forecasting information and enable teams to act on capacity warnings by creating capacity warnings in a GitLab issue tracker.
+
+![dedicated-capacity-planning-reporting](images/dedicated-capacity-planning-reporting.png)
+
+The Scalability::Observability team maintains an [internal GitLab project called `gitlab-dedicated`](https://gitlab.com/gitlab-com/gl-infra/capacity-planning-trackers/gitlab-dedicated).
+This project contains a scheduled CI pipeline to regularly produce a [static site deployed to GitLab Pages (only available internally)](https://gitlab-com.gitlab.io/gl-infra/capacity-planning-trackers/gitlab-dedicated/).
+It also contains functionality to create and manage capacity warnings in the issue tracker of this project.
+
+CI configuration for this project contains a list of tenants along with their respective metadata (e.g. AWS account, codename, etc.).
+
+For each configured tenant, the CI pipeline uses a central IAM role in the amp account.
+With this role, a tenant-specific IAM role can be assumed, which has read-only access to the respective S3 bucket containing the tenant's forecasting data.
+
+The CI pipeline produces a standard Tamland report for each tenant and integrates all individual reports into a single static site.
+This site provides unified access to capacity forecasting insights across tenant environments.
+
+Along with the report, the CI pipeline also reacts to predicted SLO violations and creates a capacity warning issue in the project's issue tracker.
+As the tracker is being used for *all* GitLab Dedicated tenants, we employ a `~tenant:CN` label to distinguish tenant environments (e.g. we use `~tenant:C1` for the tenant with codename C1).
+These issues contain further information about the tenant and component affected, along with forecasts and status information.
+The intention here is to create visibility into predicted SLO violations and provide a way for the Dedicated team to engage with capacity warnings directly (e.g. for discussion, work scheduling etc.).
+
+Overall, the Dedicated teams and operators use the Tamland report and issue tracker to act on capacity warnings.
 
-When Tamland forecast data is available for a tenant, the Dedicated teams consume this data and act on it accordingly.
-The Scalability::Observability group is going to support and guide this process to get started and help interpret data, along with implementing Tamland features required to streamline this process for Dedicated in further iterations.
+In order to get started, we suggest the Dedicated group to take a regular pass across the capacity warnings and triage those.
+For additional visibility, we may want to consider enabling getting Slack updates sent out for new capacity warnings created.
 
 ## Alternative Solution
 
@@ -150,7 +181,7 @@ The Scalability::Observability group is going to support and guide this process
 
 An alternative design, we don't consider an option at this point, is to setup Tamland as a Service and run it fully **outside** of tenant environments.
 
-![tamland-as-a-service](images/tamland-as-a-service.png)
+![dedicated-capacity-planning-as-a-service](images/dedicated-capacity-planning-as-a-service.png)
 
 In this design, a central Prometheus/Thanos instance is needed to provide the metrics data for Tamland.
 Dedicated tenants use remote-write to push their Prometheus data to the central Thanos instance.
diff --git a/doc/ci/runners/saas/macos_saas_runner.md b/doc/ci/runners/saas/macos_saas_runner.md
index dcf81158e82..55deced783a 100644
--- a/doc/ci/runners/saas/macos_saas_runner.md
+++ b/doc/ci/runners/saas/macos_saas_runner.md
@@ -38,8 +38,8 @@ in your `.gitlab-ci.yml` file. Each image runs a specific version of macOS and X
 | VM image                   | Status |              |
 |----------------------------|--------|--------------|
 | `macos-12-xcode-14`        | `Deprecated` | (Removal in GitLab 16.10) |
-| `macos-13-xcode-14`        | `GA`   | [Preinstalled Software](https://gitlab.com/gitlab-org/ci-cd/shared-runners/images/job-images/-/blob/main/toolchain/macos-13.yml) |
-| `macos-14-xcode-15`        | `GA`   | [Preinstalled Software](https://gitlab.com/gitlab-org/ci-cd/shared-runners/images/job-images/-/blob/main/toolchain/macos-14.yml) |
+| `macos-13-xcode-14`        | `GA`   | [Preinstalled Software](https://gitlab.com/gitlab-org/ci-cd/shared-runners/images/job-images/-/blob/36d443841732f2d4f7e3de1bce63f530edef1676/toolchain/macos-13.yml) |
+| `macos-14-xcode-15`        | `GA`   | [Preinstalled Software](https://gitlab.com/gitlab-org/ci-cd/shared-runners/images/job-images/-/blob/36d443841732f2d4f7e3de1bce63f530edef1676/toolchain/macos-14.yml) |
 
 If no image is specified, the macOS runner uses `macos-13-xcode-14`.
 
diff --git a/doc/development/ee_features.md b/doc/development/ee_features.md
index 78177612aa9..fd00b8b86cb 100644
--- a/doc/development/ee_features.md
+++ b/doc/development/ee_features.md
@@ -63,6 +63,29 @@ Each SaaS feature is defined in a separate YAML file consisting of a number of f
 | `milestone`         | no       | Milestone in which the SaaS feature was created.                                                             |
 | `group`             | no       | The [group](https://about.gitlab.com/handbook/product/categories/#devops-stages) that owns the feature flag. |
 
+#### Create a new SaaS feature file definition
+
+The GitLab codebase provides [`bin/saas-feature.rb`](https://gitlab.com/gitlab-org/gitlab/-/blob/master/bin/saas-feature.rb),
+a dedicated tool to create new SaaS feature definitions.
+The tool asks various questions about the new SaaS feature, then creates
+a YAML definition in `ee/config/saas_features`.
+
+Only SaaS features that have a YAML definition file can be used when running the development or testing environments.
+
+```shell
+❯ bin/saas-feature my_saas_feature
+You picked the group 'group::acquisition'
+
+>> URL of the MR introducing the SaaS feature (enter to skip and let Danger provide a suggestion directly in the MR):
+?> https://gitlab.com/gitlab-org/gitlab/-/merge_requests/38602
+create ee/config/saas_features/my_saas_feature.yml
+---
+name: my_saas_feature
+introduced_by_url: https://gitlab.com/gitlab-org/gitlab/-/merge_requests/38602
+milestone: '16.8'
+group: group::acquisition
+```
+
 ### Opting out of a SaaS-only feature on another SaaS instance (JiHu)
 
 Prepend the `ee/lib/ee/gitlab/saas.rb` module and override the `Gitlab::Saas.feature_available?` method.
diff --git a/doc/development/permissions/custom_roles.md b/doc/development/permissions/custom_roles.md
index 7a0d325d09a..457fe5a5d8b 100644
--- a/doc/development/permissions/custom_roles.md
+++ b/doc/development/permissions/custom_roles.md
@@ -168,74 +168,87 @@ For example, you see in `GroupPolicy` that there is an ability called
 `read_project_security_dashboard`. You'd like to make both customizable. Rather
 than adding a row to the `member_roles` table for each ability, consider
 renaming them to `read_security_dashboard` and adding `read_security_dashboard`
-to the `member_roles` table. This is more expected because it means that
-enabling `read_security_dashboard` on the parent group will enable the custom role.
-For example, `GroupPolicy` has an ability called `read_group_security_dashboard` and `ProjectPolicy` has an ability
-called `read_project_security_dashboard`. If you would like to make both customizable, rather than adding a row to the
-`member_roles` table for each ability, consider renaming them to `read_security_dashboard` and adding
-`read_security_dashboard` to the `member_roles` table. This convention means that enabling `read_security_dashboard` on
+to the `member_roles` table. Enabling `read_security_dashboard` on
 the parent group will allow the custom role to access the group security dashboard and the project security dashboard
 for each project in that group. Enabling the same permission on a specific project will allow access to that projects'
 security dashboard.
 
 ### Implement a new ability
 
-To add a new ability to a custom role:
+#### Step 1. Generate a configuration file
 
-- Generate YAML file by running `./ee/bin/custom-ability` generator
-- Add a new column to `member_roles` table, either manually or by running `custom_roles:code`  generator, eg. by running `rails generate gitlab:custom_roles:code --ability new_ability_name`. The ability parameter is case sensitive and has to exactly match the permission name from the YAML file.
-- Add the ability to the respective Policy for example in [this change in merge request 114734](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/114734/diffs#diff-content-edcbe28bdecbd848d4d9efdc5b5e9bddd2a7299e).
-- Update the specs. Don't forget to add a spec to `ee/spec/requests/custom_roles` - the spec template file was pre-generated if you used the code generator
-- Compile the documentation by running `bundle exec rake gitlab:custom_roles:compile_docs`
-- Update the GraphQL documentation by running `bundle exec rake gitlab:graphql:compile_docs`
-
-Examples of merge requests adding new abilities to custom roles:
+- Run `./ee/bin/custom-ability <ABILITY_NAME>` to generate a configuration file for the new ability.
+- This will generate a YAML file in `ee/config/custom_abilities` which follows the following schema:
 
-- [Read code](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/106256)
-- [Read vulnerability](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/114734)
-- [Admin vulnerability](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/121534)
+| Field | Required | Description |
+| ----- | -------- |--------------|
+| `name` | yes     | Unique, lowercase and underscored name describing the custom ability. Must match the filename. |
+| `description` | yes | Human-readable description of the custom ability. |
+| `feature_category` | yes | Name of the feature category. For example, `vulnerability_management`. |
+| `introduced_by_issue` | yes | Issue URL that proposed the addition of this custom ability. |
+| `introduced_by_mr` | yes | MR URL that added this custom ability. |
+| `milestone` | yes | Milestone in which this custom ability was added. |
+| `group_ability` | yes | Boolean value to indicate whether this ability is checked on group level. |
+| `project_ability` | yes | Boolean value to whether this ability is checked on project level. |
+| `requirements` | no | The list of custom permissions this ability is dependent on. For instance `admin_vulnerability` is dependent on `read_vulnerability`. If none, then enter `[]`  |
 
-The above merge requests don't use YAML files and code generators. Some of the changes are not needed anymore. We will update the documentation once we have a permission implemented using the generators.
+#### Step 2: Create a migration file
 
-If you have any concerns, put the new ability behind a feature flag.
+- Run `bundle exec rails generate gitlab:custom_roles:code --ability <ABILITY_NAME>` which will generate a migration file to add the ability as a boolean column to the `member_roles` table.
 
-#### Documenting handling the feature flag
+#### Step 3: Update policies
 
-- When you introduce a new custom ability under a feature flag, add the `feature_flag` attribute to the appropriate ability YAML file.
-- When you enable the ability by default, add the `feature_flag_enabled_milestone` and `feature_flag_enabled_mr` attributes to the appropriate ability YAML file and regenerate the documentation.
-- You do not have to include these attributes in the YAML file if the feature flag is enabled by default in the same release as the ability is introduced.
+- If the ability is checked on a group level, add rule(s) to GroupPolicy to enable the ability.
+- For example: if the ability we would like to add is `read_dependency`, then an update to `ee/app/policies/ee/group_policy.rb` would look like as follows:
 
-#### Testing
+```ruby
+desc "Custom role on group that enables read dependency"
+condition(:role_enables_read_dependency) do
+  ::Auth::MemberRoleAbilityLoader.new(
+    user: @user,
+    resource: @subject,
+    ability: :read_dependency
+  ).has_ability?
+end
+
+rule { custom_roles_allowed & role_enables_read_dependency }.policy do
+  enable :read_dependency
+end
+```
 
-Unit tests are preferred to test out changes to any policies affected by the
-addition of new custom permissions. Custom Roles is an Ultimate tier feature so
-these tests can be found in the `ee/spec/policies` directory. The [spec file](https://gitlab.com/gitlab-org/gitlab/-/blob/13baa4e8c92a56260591a5bf0a58d3339890ee10/ee/spec/policies/project_policy_spec.rb#L2726-2740) for
-the `ProjectPolicy` contains shared examples that can be used to test out the
-following conditions:
+- Similarly, If the ability is checked on a project level, add rule(s) to ProjectPolicy to enable the ability.
+- For example: if the ability we would like to add is `read_dependency`, then an update to `ee/app/policies/ee/project_policy.rb` would look like as follows:
 
-- when the `custom_roles` licensed feature is not enabled
-- when the `custom_roles` licensed feature is enabled
-  - when a user is a member of a custom role via an inherited group member
-  - when a user is a member of a custom role via a direct group member
-  - when a user is a member of a custom role via a direct project membership
+```ruby
+desc "Custom role on project that enables read dependency"
+condition(:role_enables_read_dependency) do
+  ::Auth::MemberRoleAbilityLoader.new(
+    user: @user,
+    resource: @subject,
+    ability: :read_dependency
+  ).has_ability?
+end
+
+rule { custom_roles_allowed & role_enables_read_dependency }.policy do
+  enable :read_dependency
+end
+```
 
-Below is an example for testing out `ProjectPolicy` related changes.
+- Not all abilities need to be enabled on both levels, for instance `admin_terraform_state` allows users to manage a project's terraform state. It only needs to be enabled on the project level and not the group level, and thus only needs to be configured in `ee/app/policies/ee/project_policy.rb`.
 
-```ruby
-  context 'for a role with `custom_permission` enabled' do
-    let(:member_role_abilities) { { custom_permission: true } }
-    let(:allowed_abilities) { [:custom_permission] }
+#### Step 4: Verify
 
-    it_behaves_like 'custom roles abilities'
-  end
-```
+- Ensure SaaS mode is enabled with `GITLAB_SIMULATE_SAAS=1`.
+- Navigate to any Group that you are an owner of, then go to `Settings -> Roles and Permissions`.
+- Click on `Add new role` and create a custom role with the permission you have just created.
+- Navigate to the Group's `Manage -> Members` page and assign a member to this newly created custom role.
+- Next, log-in as that member and ensure that you are able to access the page that the custom ability is intended for.
 
-Request specs are preferred to test out any endpoint that allow access via a custom role permission.
-This includes controllers, REST API, and GraphQL. Examples of request specs can be found in `ee/spec/requests/custom_roles/`. In this directory you will find a sub-directory named after each permission that can be enabled via a custom role.
-The `custom_roles` licensed feature must be enabled to test this functionality.
+#### Step 5: Add specs
 
-Below is an example of the typical setup that is required to test out a
-Rails Controller endpoint.
+- Add the ability as a trait in the `MemberRoles` factory, `ee/spec/factories/member_roles.rb`.
+- Add tests to `ee/spec/requests/custom_roles/<ABILITY_NAME>/request_spec.rb` to ensure that once the user has been assigned the custom ability, they can successfully access the controllers, REST API endpoints and GraphQL API endpoints.
+- Below is an example of the typical setup that is required to test a Rails Controller endpoint.
 
 ```ruby
   let_it_be(:user) { create(:user) }
@@ -245,6 +258,7 @@ Rails Controller endpoint.
 
   before do
     stub_licensed_features(custom_roles: true)
+    
     sign_in(user)
   end
 
@@ -260,8 +274,7 @@ Rails Controller endpoint.
   end
 ```
 
-Below is an example of the typical setup that is required to test out a GraphQL
-mutation.
+- Below is an example of the typical setup that is required to test a GraphQL mutation.
 
 ```ruby
   let_it_be(:user) { create(:user) }
@@ -271,57 +284,36 @@ mutation.
 
   before do
     stub_licensed_features(custom_roles: true)
+
+    sign_in(user)
   end
 
   describe MyMutation do
     include GraphqlHelpers
 
     describe '#show' do
-      it 'allows access' do
-        post_graphql_mutation(graphql_mutation(:my_mutation, {
-          example: "Example"
-        }), current_user: user)
-
-        expect(response).to have_gitlab_http_status(:success)
-        mutation_response = graphql_mutation_response(:my_mutation)
-        expect(mutation_response).to be_present
-        expect(mutation_response["errors"]).to be_empty
-      end
+      let(:mutation) { graphql_mutation(:my_mutation) } 
+
+      it_behaves_like 'a working graphql query'
     end
   end
 ```
 
-[`GITLAB_DEBUG_POLICIES=true`](#finding-existing-abilities-checks) can be used
-to troubleshoot runtime policy decisions.
-
-## Custom abilities definition
-
-All new custom abilities must have a type definition stored in `ee/config/custom_abilities` that contains a single source of truth for every ability that is part of custom roles feature.
+- Add tests to `ProjectPolicy` and/or `GroupPolicy`. Below is an example for testing `ProjectPolicy` related changes.
 
-### Add a new custom ability definition
-
-To add a new custom ability:
+```ruby
+  context 'for a member role with read_dependency true' do
+    let(:member_role_abilities) { { read_dependency: true } }
+    let(:allowed_abilities) { [:read_dependency] }
 
-1. Create the YAML definition. You can either:
-   - Use the `ee/bin/custom-ability` CLI to create the YAML definition automatically.
-   - Perform manual steps to create a new file in `ee/config/custom_abilities/` with the filename matching the name of the ability name.
-1. Add contents to the file that conform to the [schema](#schema) defined in `ee/config/custom_abilities/types/type_schema.json`.
-1. Add [tests](#testing) for the new ability in `ee/spec/requests/custom_roles/` with a new directory named after the ability name.
+    it_behaves_like 'custom roles abilities'
+  end
+```
 
-### Schema
+#### Step 6: Update documentation
 
-| Field | Required | Description |
-| ----- | -------- |--------------|
-| `name` | yes     | Unique, lowercase and underscored name describing the custom ability. Must match the filename. |
-| `description` | yes | Human-readable description of the custom ability. |
-| `feature_category` | yes | Name of the feature category. For example, `vulnerability_management`. |
-| `introduced_by_issue` | yes | Issue URL that proposed the addition of this custom ability. |
-| `introduced_by_mr` | no | MR URL that added this custom ability. |
-| `milestone` | yes | Milestone in which this custom ability was added. |
-| `group_ability` | yes | Indicate whether this ability is checked on group level. |
-| `project_ability` | yes | Indicate whether this ability is checked on project level. |
-| `requirements` | no | The custom abilities that need to be enabled for this ability. |
-| `skip_seat_consumption` | yes | Indicate wheter this ability should be skiped when counting licensed users. |
+- Update the list of custom abilities by running `bundle exec rake gitlab:custom_roles:compile_docs`
+- Update the GraphQL documentation by running `bundle exec rake gitlab:graphql:compile_docs`
 
 ### Privilege escalation consideration
 
diff --git a/doc/development/secure_coding_guidelines.md b/doc/development/secure_coding_guidelines.md
index d8fad6deb9c..75df9a67aff 100644
--- a/doc/development/secure_coding_guidelines.md
+++ b/doc/development/secure_coding_guidelines.md
@@ -258,7 +258,7 @@ the mitigations for a new feature.
 
 #### URL blocker & validation libraries
 
-[`Gitlab::UrlBlocker`](https://gitlab.com/gitlab-org/gitlab/-/blob/master/lib/gitlab/url_blocker.rb) can be used to validate that a
+[`Gitlab::HTTP_V2::UrlBlocker`](https://gitlab.com/gitlab-org/gitlab/-/blob/master/gems/gitlab-http/lib/gitlab/http_v2/url_blocker.rb) can be used to validate that a
 provided URL meets a set of constraints. Importantly, when `dns_rebind_protection` is `true`, the method returns a known-safe URI where the hostname
 has been replaced with an IP address. This prevents DNS rebinding attacks, because the DNS record has been resolved. However, if we ignore this returned
 value, we **will not** be protected against DNS rebinding.
@@ -1234,7 +1234,7 @@ These types of bugs are often seen in environments which allow multi-threading a
 
 **Example 1:** you have a model which accepts a URL as input. When the model is created you verify that the URL host resolves to a public IP address, to prevent attackers making internal network calls. But DNS records can change ([DNS rebinding](#server-side-request-forgery-ssrf)]). An attacker updates the DNS record to `127.0.0.1`, and when your code resolves those URL host it results in sending a potentially malicious request to a server on the internal network. The property was valid at the "time of check", but invalid and malicious at "time of use".
 
-GitLab-specific example can be found in [this issue](https://gitlab.com/gitlab-org/gitlab/-/issues/214401) where, although `Gitlab::UrlBlocker.validate!` was called, the returned value was not used. This made it vulnerable to TOCTOU bug and SSRF protection bypass through [DNS rebinding](#server-side-request-forgery-ssrf). The fix was to [use the validated IP address](https://gitlab.com/gitlab-org/gitlab/-/commit/7af8abd4df9a98f7a1ae7c4ec9840d0a7a8c684d).
+GitLab-specific example can be found in [this issue](https://gitlab.com/gitlab-org/gitlab/-/issues/214401) where, although `Gitlab::HTTP_V2::UrlBlocker.validate!` was called, the returned value was not used. This made it vulnerable to TOCTOU bug and SSRF protection bypass through [DNS rebinding](#server-side-request-forgery-ssrf). The fix was to [use the validated IP address](https://gitlab.com/gitlab-org/gitlab/-/commit/85c6a73598e72ab104ab29b72bf83661cd961646).
 
 **Example 2:** you have a feature which schedules jobs. When the user schedules the job, they have permission to do so. But imagine if, between the time they schedule the job and the time it is run, their permissions are restricted. Unless you re-check permissions at time of use, you could inadvertently allow unauthorized activity.
 
@@ -1264,9 +1264,9 @@ end
 - Use your framework's validations and database features to impose constraints and atomic reads and writes.
 - Read about [Server Side Request Forgery (SSRF) and DNS rebinding](#server-side-request-forgery-ssrf)
 
-An example of well implemented `Gitlab::UrlBlocker.validate!` call that prevents TOCTOU bug:
+An example of well implemented `Gitlab::HTTP_V2::UrlBlocker.validate!` call that prevents TOCTOU bug:
 
-1. [Preventing DNS rebinding in Gitea importer](https://gitlab.com/gitlab-org/gitlab/-/commit/7af8abd4df9a98f7a1ae7c4ec9840d0a7a8c684d)
+1. [Preventing DNS rebinding in Gitea importer](https://gitlab.com/gitlab-org/gitlab/-/commit/85c6a73598e72ab104ab29b72bf83661cd961646)
 
 ### Resources