diff options
author | GitLab Bot <gitlab-bot@gitlab.com> | 2020-05-06 15:09:36 +0300 |
---|---|---|
committer | GitLab Bot <gitlab-bot@gitlab.com> | 2020-05-06 15:09:36 +0300 |
commit | 4279f24a19836d3e74e4aae8bea7acc2dd8222cc (patch) | |
tree | 76e4b3cf4d6bd85ff50e40bf011e7f9bc350441a /doc | |
parent | 51c20446a0dcf2f5f4a0254230876bd472a254e7 (diff) |
Add latest changes from gitlab-org/gitlab@master
Diffstat (limited to 'doc')
-rw-r--r-- | doc/administration/reference_architectures/index.md | 27 | ||||
-rw-r--r-- | doc/api/graphql/reference/gitlab_schema.graphql | 71 | ||||
-rw-r--r-- | doc/api/graphql/reference/gitlab_schema.json | 235 | ||||
-rw-r--r-- | doc/api/graphql/reference/index.md | 15 | ||||
-rw-r--r-- | doc/ci/pipelines/index.md | 12 | ||||
-rw-r--r-- | doc/install/aws/index.md | 54 |
6 files changed, 389 insertions, 25 deletions
diff --git a/doc/administration/reference_architectures/index.md b/doc/administration/reference_architectures/index.md index 88a4f2c97a6..79d4a3328b1 100644 --- a/doc/administration/reference_architectures/index.md +++ b/doc/administration/reference_architectures/index.md @@ -62,11 +62,20 @@ This solution is appropriate for many teams that have a single server at their d You can also optionally configure GitLab to use an [external PostgreSQL service](../external_database.md) or an [external object storage service](../high_availability/object_storage.md) for added performance and reliability at a relatively low complexity cost. -<!-- ## Up to 2,000 users -For up to 2,000 users, defining the reference architecture is [being worked on](https://gitlab.com/gitlab-org/quality/performance/-/issues/223). ---> +> - **Supported users (approximate):** 2,000 +> - **High Availability:** False +> - **Test RPS rates:** API: 40 RPS, Web: 4 RPS, Git: 4 RPS + +| Service | Nodes | Configuration ([8](#footnotes)) | GCP type | AWS type ([9](#footnotes)) | +|--------------------------------------------------------------|-------|---------------------------------|---------------|----------------------------| +| GitLab Rails, Sidekiq, Consul ([1](#footnotes)) | 2 | 8 vCPU, 7.2GB Memory | n1-highcpu-8 | c5.2xlarge | +| PostgreSQL | 1 | 2 vCPU, 7.5GB Memory | n1-standard-2 | m5.large | +| Gitaly ([2](#footnotes)) ([5](#footnotes)) ([7](#footnotes)) | X | 4 vCPU, 15GB Memory | n1-standard-4 | m5.xlarge | +| Cloud Object Storage ([4](#footnotes)) | - | - | - | - | +| NFS Server ([5](#footnotes)) ([7](#footnotes)) | 1 | 4 vCPU, 3.6GB Memory | n1-highcpu-4 | c5.xlarge | +| External load balancing node ([6](#footnotes)) | 1 | 2 vCPU, 1.8GB Memory | n1-highcpu-2 | c5.large | ## Up to 3,000 users @@ -79,7 +88,8 @@ server, a PostgreSQL server and a Redis server. A reference architecture with this alternative in mind is [being worked on](https://gitlab.com/gitlab-org/quality/performance/-/issues/223). > - **Supported users (approximate):** 3,000 -> - **Test RPS rates:** API: 40 RPS, Web: 4 RPS, Git: 4 RPS +> - **High Availability:** True +> - **Test RPS rates:** API: 60 RPS, Web: 6 RPS, Git: 6 RPS | Service | Nodes | Configuration ([8](#footnotes)) | GCP type | AWS type ([9](#footnotes)) | |--------------------------------------------------------------|-------|---------------------------------|---------------|----------------------------| @@ -99,6 +109,7 @@ this alternative in mind is [being worked on](https://gitlab.com/gitlab-org/qual ## Up to 5,000 users > - **Supported users (approximate):** 5,000 +> - **High Availability:** True > - **Test RPS rates:** API: 100 RPS, Web: 10 RPS, Git: 10 RPS | Service | Nodes | Configuration ([8](#footnotes)) | GCP type | AWS type ([9](#footnotes)) | @@ -119,6 +130,7 @@ this alternative in mind is [being worked on](https://gitlab.com/gitlab-org/qual ## Up to 10,000 users > - **Supported users (approximate):** 10,000 +> - **High Availability:** True > - **Test RPS rates:** API: 200 RPS, Web: 20 RPS, Git: 20 RPS | Service | Nodes | GCP Configuration ([8](#footnotes)) | GCP type | AWS type ([9](#footnotes)) | @@ -142,6 +154,7 @@ this alternative in mind is [being worked on](https://gitlab.com/gitlab-org/qual ## Up to 25,000 users > - **Supported users (approximate):** 25,000 +> - **High Availability:** True > - **Test RPS rates:** API: 500 RPS, Web: 50 RPS, Git: 50 RPS | Service | Nodes | Configuration ([8](#footnotes)) | GCP type | AWS type ([9](#footnotes)) | @@ -165,6 +178,7 @@ this alternative in mind is [being worked on](https://gitlab.com/gitlab-org/qual ## Up to 50,000 users > - **Supported users (approximate):** 50,000 +> - **High Availability:** True > - **Test RPS rates:** API: 1000 RPS, Web: 100 RPS, Git: 100 RPS | Service | Nodes | Configuration ([8](#footnotes)) | GCP type | AWS type ([9](#footnotes)) | @@ -288,7 +302,10 @@ column. ## Footnotes 1. In our architectures we run each GitLab Rails node using the Puma webserver - and have its number of workers set to 90% of available CPUs along with four threads. + and have its number of workers set to 90% of available CPUs along with four threads. For + nodes that are running Rails with other components the worker value should be reduced + accordingly where we've found 50% achieves a good balance but this is dependent + on workload. 1. Gitaly node requirements are dependent on customer data, specifically the number of projects and their sizes. We recommend two nodes as an absolute minimum for HA environments diff --git a/doc/api/graphql/reference/gitlab_schema.graphql b/doc/api/graphql/reference/gitlab_schema.graphql index b4cfd50422a..69ca80015a7 100644 --- a/doc/api/graphql/reference/gitlab_schema.graphql +++ b/doc/api/graphql/reference/gitlab_schema.graphql @@ -143,6 +143,21 @@ Describes an alert from the project's Alert Management """ type AlertManagementAlert { """ + Timestamp the alert was created + """ + createdAt: Time + + """ + Description of the alert + """ + description: String + + """ + Alert details + """ + details: JSON + + """ Timestamp the alert ended """ endedAt: Time @@ -153,6 +168,11 @@ type AlertManagementAlert { eventCount: Int """ + List of hosts the alert came from + """ + hosts: [String!] + + """ Internal ID of the alert """ iid: ID! @@ -186,6 +206,11 @@ type AlertManagementAlert { Title of the alert """ title: String + + """ + Timestamp the alert was last updated + """ + updatedAt: Time } """ @@ -6074,6 +6099,7 @@ type Mutation { todoRestoreMany(input: TodoRestoreManyInput!): TodoRestoreManyPayload todosMarkAllDone(input: TodosMarkAllDoneInput!): TodosMarkAllDonePayload toggleAwardEmoji(input: ToggleAwardEmojiInput!): ToggleAwardEmojiPayload + updateAlertStatus(input: UpdateAlertStatusInput!): UpdateAlertStatusPayload updateEpic(input: UpdateEpicInput!): UpdateEpicPayload """ @@ -9729,6 +9755,51 @@ enum TypeEnum { project } +""" +Autogenerated input type of UpdateAlertStatus +""" +input UpdateAlertStatusInput { + """ + A unique identifier for the client performing the mutation. + """ + clientMutationId: String + + """ + The iid of the alert to mutate + """ + iid: String! + + """ + The project the alert to mutate is in + """ + projectPath: ID! + + """ + The status to set the alert + """ + status: AlertManagementStatus! +} + +""" +Autogenerated return type of UpdateAlertStatus +""" +type UpdateAlertStatusPayload { + """ + The alert after mutation + """ + alert: AlertManagementAlert + + """ + A unique identifier for the client performing the mutation. + """ + clientMutationId: String + + """ + Reasons why the mutation failed. + """ + errors: [String!]! +} + input UpdateDiffImagePositionInput { """ Total height of the image diff --git a/doc/api/graphql/reference/gitlab_schema.json b/doc/api/graphql/reference/gitlab_schema.json index 7a4bc3f6360..4ea50ac353e 100644 --- a/doc/api/graphql/reference/gitlab_schema.json +++ b/doc/api/graphql/reference/gitlab_schema.json @@ -395,6 +395,48 @@ "description": "Describes an alert from the project's Alert Management", "fields": [ { + "name": "createdAt", + "description": "Timestamp the alert was created", + "args": [ + + ], + "type": { + "kind": "SCALAR", + "name": "Time", + "ofType": null + }, + "isDeprecated": false, + "deprecationReason": null + }, + { + "name": "description", + "description": "Description of the alert", + "args": [ + + ], + "type": { + "kind": "SCALAR", + "name": "String", + "ofType": null + }, + "isDeprecated": false, + "deprecationReason": null + }, + { + "name": "details", + "description": "Alert details", + "args": [ + + ], + "type": { + "kind": "SCALAR", + "name": "JSON", + "ofType": null + }, + "isDeprecated": false, + "deprecationReason": null + }, + { "name": "endedAt", "description": "Timestamp the alert ended", "args": [ @@ -423,6 +465,28 @@ "deprecationReason": null }, { + "name": "hosts", + "description": "List of hosts the alert came from", + "args": [ + + ], + "type": { + "kind": "LIST", + "name": null, + "ofType": { + "kind": "NON_NULL", + "name": null, + "ofType": { + "kind": "SCALAR", + "name": "String", + "ofType": null + } + } + }, + "isDeprecated": false, + "deprecationReason": null + }, + { "name": "iid", "description": "Internal ID of the alert", "args": [ @@ -523,6 +587,20 @@ }, "isDeprecated": false, "deprecationReason": null + }, + { + "name": "updatedAt", + "description": "Timestamp the alert was last updated", + "args": [ + + ], + "type": { + "kind": "SCALAR", + "name": "Time", + "ofType": null + }, + "isDeprecated": false, + "deprecationReason": null } ], "inputFields": null, @@ -18130,6 +18208,33 @@ "deprecationReason": null }, { + "name": "updateAlertStatus", + "description": null, + "args": [ + { + "name": "input", + "description": null, + "type": { + "kind": "NON_NULL", + "name": null, + "ofType": { + "kind": "INPUT_OBJECT", + "name": "UpdateAlertStatusInput", + "ofType": null + } + }, + "defaultValue": null + } + ], + "type": { + "kind": "OBJECT", + "name": "UpdateAlertStatusPayload", + "ofType": null + }, + "isDeprecated": false, + "deprecationReason": null + }, + { "name": "updateEpic", "description": null, "args": [ @@ -29250,6 +29355,136 @@ }, { "kind": "INPUT_OBJECT", + "name": "UpdateAlertStatusInput", + "description": "Autogenerated input type of UpdateAlertStatus", + "fields": null, + "inputFields": [ + { + "name": "projectPath", + "description": "The project the alert to mutate is in", + "type": { + "kind": "NON_NULL", + "name": null, + "ofType": { + "kind": "SCALAR", + "name": "ID", + "ofType": null + } + }, + "defaultValue": null + }, + { + "name": "iid", + "description": "The iid of the alert to mutate", + "type": { + "kind": "NON_NULL", + "name": null, + "ofType": { + "kind": "SCALAR", + "name": "String", + "ofType": null + } + }, + "defaultValue": null + }, + { + "name": "status", + "description": "The status to set the alert", + "type": { + "kind": "NON_NULL", + "name": null, + "ofType": { + "kind": "ENUM", + "name": "AlertManagementStatus", + "ofType": null + } + }, + "defaultValue": null + }, + { + "name": "clientMutationId", + "description": "A unique identifier for the client performing the mutation.", + "type": { + "kind": "SCALAR", + "name": "String", + "ofType": null + }, + "defaultValue": null + } + ], + "interfaces": null, + "enumValues": null, + "possibleTypes": null + }, + { + "kind": "OBJECT", + "name": "UpdateAlertStatusPayload", + "description": "Autogenerated return type of UpdateAlertStatus", + "fields": [ + { + "name": "alert", + "description": "The alert after mutation", + "args": [ + + ], + "type": { + "kind": "OBJECT", + "name": "AlertManagementAlert", + "ofType": null + }, + "isDeprecated": false, + "deprecationReason": null + }, + { + "name": "clientMutationId", + "description": "A unique identifier for the client performing the mutation.", + "args": [ + + ], + "type": { + "kind": "SCALAR", + "name": "String", + "ofType": null + }, + "isDeprecated": false, + "deprecationReason": null + }, + { + "name": "errors", + "description": "Reasons why the mutation failed.", + "args": [ + + ], + "type": { + "kind": "NON_NULL", + "name": null, + "ofType": { + "kind": "LIST", + "name": null, + "ofType": { + "kind": "NON_NULL", + "name": null, + "ofType": { + "kind": "SCALAR", + "name": "String", + "ofType": null + } + } + } + }, + "isDeprecated": false, + "deprecationReason": null + } + ], + "inputFields": null, + "interfaces": [ + + ], + "enumValues": null, + "possibleTypes": null + }, + { + "kind": "INPUT_OBJECT", "name": "UpdateDiffImagePositionInput", "description": null, "fields": null, diff --git a/doc/api/graphql/reference/index.md b/doc/api/graphql/reference/index.md index 8b47fbda02e..185df0c2fe4 100644 --- a/doc/api/graphql/reference/index.md +++ b/doc/api/graphql/reference/index.md @@ -52,8 +52,12 @@ Describes an alert from the project's Alert Management | Name | Type | Description | | --- | ---- | ---------- | +| `createdAt` | Time | Timestamp the alert was created | +| `description` | String | Description of the alert | +| `details` | JSON | Alert details | | `endedAt` | Time | Timestamp the alert ended | | `eventCount` | Int | Number of events of this alert | +| `hosts` | String! => Array | List of hosts the alert came from | | `iid` | ID! | Internal ID of the alert | | `monitoringTool` | String | Monitoring tool the alert came from | | `service` | String | Service the alert came from | @@ -61,6 +65,7 @@ Describes an alert from the project's Alert Management | `startedAt` | Time | Timestamp the alert was raised | | `status` | AlertManagementStatus | Status of the alert | | `title` | String | Title of the alert | +| `updatedAt` | Time | Timestamp the alert was last updated | ## AwardEmoji @@ -1515,6 +1520,16 @@ Represents a directory | `type` | EntryType! | Type of tree entry | | `webUrl` | String | Web URL for the tree entry (directory) | +## UpdateAlertStatusPayload + +Autogenerated return type of UpdateAlertStatus + +| Name | Type | Description | +| --- | ---- | ---------- | +| `alert` | AlertManagementAlert | The alert after mutation | +| `clientMutationId` | String | A unique identifier for the client performing the mutation. | +| `errors` | String! => Array | Reasons why the mutation failed. | + ## UpdateEpicPayload Autogenerated return type of UpdateEpic diff --git a/doc/ci/pipelines/index.md b/doc/ci/pipelines/index.md index d2632dfe719..d4fa41ca19e 100644 --- a/doc/ci/pipelines/index.md +++ b/doc/ci/pipelines/index.md @@ -549,15 +549,3 @@ To illustrate its life cycle: even if the commit history of the `example` branch has been overwritten by force-push. 1. GitLab Runner fetches the persistent pipeline ref and gets source code from the checkout-SHA. 1. When the pipeline finished, its persistent ref is cleaned up in a background process. - -NOTE: **NOTE**: At this moment, this feature is on by default and can be manually disabled -by disabling `depend_on_persistent_pipeline_ref` feature flag. If you're interested in -manually disabling this behavior, please ask the administrator -to execute the following commands in rails console. - -```shell -> sudo gitlab-rails console # Login to Rails console of GitLab instance. -> project = Project.find_by_full_path('namespace/project-name') # Get the project instance. -> Feature.disable(:depend_on_persistent_pipeline_ref, project) # Disable the feature flag for specific project -> Feature.disable(:depend_on_persistent_pipeline_ref) # Disable the feature flag system-wide -``` diff --git a/doc/install/aws/index.md b/doc/install/aws/index.md index 8daa9b40111..05906520c1c 100644 --- a/doc/install/aws/index.md +++ b/doc/install/aws/index.md @@ -63,17 +63,52 @@ Here's a list of the AWS services we will use, with links to pricing information NOTE: **Note:** Please note that while we will be using EBS for storage, we do not recommend using EFS as it may negatively impact GitLab's performance. You can review the [relevant documentation](../../administration/high_availability/nfs.md#avoid-using-awss-elastic-file-system-efs) for more details. -## Creating an IAM EC2 instance role and profile +## Create an IAM EC2 instance role and profile + +As we'll be using [Amazon S3 object storage](#amazon-s3-object-storage), our EC2 instances need to have read, write, and list permissions for our S3 buckets. To avoid embedding AWS keys in our GitLab config, we'll make use of an [IAM Role](https://docs.aws.amazon.com/IAM/latest/UserGuide/id_roles.html) to allow our GitLab instance with this access. We'll need to create an IAM policy to attach to our IAM role: + +### Create an IAM Policy + +1. Navigate to the IAM dashboard and click on **Policies** in the left menu. +1. Click **Create policy**, select the `JSON` tab, and add a policy. We want to [follow security best practices and grant _least privilege_](https://docs.aws.amazon.com/IAM/latest/UserGuide/best-practices.html#grant-least-privilege), giving our role only the permissions needed to perform the required actions. + 1. Assuming you prefix the S3 bucket names with `gl-` as shown in the diagram, add the following policy: + +```json +{ + "Version": "2012-10-17", + "Statement": [ + { + "Effect": "Allow", + "Action": [ + "s3:AbortMultipartUpload", + "s3::CompleteMultipartUpload", + "s3:ListBucket", + "s3:PutObject", + "s3:GetObject", + "s3:DeleteObject", + "s3:PutObjectAcl" + ], + "Resource": [ + "arn:aws:s3:::gl-*/*" + ] + } + ] +} +``` + +1. Click **Review policy**, give your policy a name (we'll use `gl-s3-policy`), and click **Create policy**. -To minimize the permissions of the user, we'll create a new [IAM](https://docs.aws.amazon.com/IAM/latest/UserGuide/introduction.html) -role with limited access: +### Create an IAM Role -1. Navigate to the IAM dashboard <https://console.aws.amazon.com/iam/home>, click on **Roles** in the left menu, and +1. Still on the IAM dashboard, click on **Roles** in the left menu, and click **Create role**. 1. Create a new role by selecting **AWS service > EC2**, then click **Next: Permissions**. -1. Choose **AmazonEC2FullAccess** and **AmazonS3FullAccess**, click **Tags** and add tags if needed. -1. Click **Review**, give your role the name (we'll use `GitLabAdmin`), and click **Create role**. +1. In the policy filter, search for the `gl-s3-policy` we created above, select it, and click **Tags**. +1. Add tags if needed and click **Review**. +1. Give the role a name (we'll use `GitLabS3Access`) and click **Create Role**. + +We'll use this role when we [create a launch configuration](#create-a-launch-configuration) later on. ## Configuring the network @@ -575,7 +610,10 @@ HostKey /etc/ssh_static/ssh_host_ed25519_key #### Amazon S3 object storage -Since we're not using NFS for shared storage, we will use [Amazon S3](https://aws.amazon.com/s3/) buckets to store backups, artifacts, LFS objects, uploads, merge request diffs, container registry images, and more. Our [documentation includes configuration instructions](../../administration/object_storage.md) for each of these, and other information about using object storage with GitLab. +Since we're not using NFS for shared storage, we will use [Amazon S3](https://aws.amazon.com/s3/) buckets to store backups, artifacts, LFS objects, uploads, merge request diffs, container registry images, and more. Our documentation includes [instructions on how to configure object storage](../../administration/object_storage.md) for each of these data types, and other information about using object storage with GitLab. + +NOTE: **Note:** +Since we are using the [AWS IAM profile](#create-an-iam-role) we created earlier, be sure to omit the AWS access key and secret access key/value pairs when configuring object storage. Instead, use `'use_iam_profile' => true` in your configuration as shown in the object storage documentation linked above. Remember to run `sudo gitlab-ctl reconfigure` after saving the changes to the `gitlab.rb` file. @@ -611,7 +649,7 @@ From the EC2 dashboard: 1. Select an instance type best suited for your needs (at least a `c5.xlarge`) and click **Configure details**. 1. Enter a name for your launch configuration (we'll use `gitlab-ha-launch-config`). 1. **Do not** check **Request Spot Instance**. -1. From the **IAM Role** dropdown, pick the `GitLabAdmin` instance role we [created earlier](#creating-an-iam-ec2-instance-role-and-profile). +1. From the **IAM Role** dropdown, pick the `GitLabAdmin` instance role we [created earlier](#create-an-iam-ec2-instance-role-and-profile). 1. Leave the rest as defaults and click **Add Storage**. 1. The root volume is 8GiB by default and should be enough given that we won’t store any data there. Click **Configure Security Group**. 1. Check **Select and existing security group** and select the `gitlab-loadbalancer-sec-group` we created earlier. |