diff options
Diffstat (limited to 'doc')
99 files changed, 2599 insertions, 1876 deletions
diff --git a/doc/README.md b/doc/README.md index a56f17e3bf0..5ab8653dc35 100644 --- a/doc/README.md +++ b/doc/README.md @@ -1,126 +1,8 @@ --- -stage: none -group: unassigned -info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/engineering/ux/technical-writing/#assignments -comments: false -description: 'Learn how to use and administer GitLab, the most scalable Git-based fully integrated platform for software development.' +redirect_to: 'index.md' --- -<div class="d-none"> - <h3>Visit <a href="https://docs.gitlab.com/ee/">docs.gitlab.com</a> for the latest version - of this help information with enhanced navigation, discoverability, and readability.</h3> -</div> -<!-- the div above will not display on the docs site but will display on /help --> +This document was moved to [another location](index.md). -# GitLab Docs - -Welcome to [GitLab](https://about.gitlab.com/) documentation. - -Here you can access the complete documentation for GitLab, the single application for the -[entire DevOps lifecycle](#the-entire-devops-lifecycle). - -## Overview - -No matter how you use GitLab, we have documentation for you. - -| Essential documentation | Essential documentation | -|:------------------------|:------------------------| -| [**User documentation**](user/index.md)<br>Discover features and concepts for GitLab users. | [**Administrator documentation**](administration/index.md)<br/>Everything GitLab self-managed administrators need to know. | -| [**Contributing to GitLab**](#contributing-to-gitlab)<br/>At GitLab, everyone can contribute! | [**New to Git and GitLab?**](#new-to-git-and-gitlab)<br/>We have the resources to get you started. | -| [**Build an integration with GitLab**](#build-an-integration-with-gitlab)<br/>Consult our integration documentation. | [**Coming to GitLab from another platform?**](#coming-to-gitlab-from-another-platform)<br/>Consult our guides. | -| [**Install GitLab**](https://about.gitlab.com/install/)<br/>Installation options for different platforms. | [**Customers**](subscriptions/index.md)<br/>Information for new and existing customers. | -| [**Update GitLab**](update/index.md)<br/>Update your GitLab self-managed instance to the latest version. | [**Reference Architectures**](administration/reference_architectures/index.md)<br/>GitLab reference architectures. | -| [**GitLab releases**](https://about.gitlab.com/releases/)<br/>What's new in GitLab. | | - -## Popular topics - -Have a look at some of our most popular topics: - -| Popular topic | Description | -|:-------------------------------------------------------------------------------------------|:------------| -| [Two-factor authentication](user/profile/account/two_factor_authentication.md) | Improve the security of your GitLab account. | -| [GitLab groups](user/group/index.md) | Manage projects together. | -| [GitLab CI/CD pipeline configuration reference](ci/yaml/README.md) | Available configuration options for `.gitlab-ci.yml` files. | -| [Activate GitLab EE with a license](user/admin_area/license.md) | Activate GitLab Enterprise Edition functionality with a license. | -| [Back up and restore GitLab](raketasks/backup_restore.md) | Rake tasks for backing up and restoring GitLab self-managed instances. | -| [GitLab release and maintenance policy](policy/maintenance.md) | Policies for version naming and cadence, and also upgrade recommendations. | -| [Elasticsearch integration](integration/elasticsearch.md) | Integrate Elasticsearch with GitLab to enable advanced searching. | -| [Omnibus GitLab database settings](https://docs.gitlab.com/omnibus/settings/database.html) | Database settings for Omnibus GitLab self-managed instances. | -| [Omnibus GitLab NGINX settings](https://docs.gitlab.com/omnibus/settings/nginx.html) | NGINX settings for Omnibus GitLab self-managed instances. | -| [Omnibus GitLab SSL configuration](https://docs.gitlab.com/omnibus/settings/ssl.html) | SSL settings for Omnibus GitLab self-managed instances. | -| [GitLab.com settings](user/gitlab_com/index.md) | Settings used for GitLab.com. | - -## The entire DevOps lifecycle - -GitLab is the first single application for software development, security, -and operations that enables [Concurrent DevOps](https://about.gitlab.com/topics/concurrent-devops/). -GitLab makes the software lifecycle faster and radically improves the speed of business. - -GitLab provides solutions for [each of the stages of the DevOps lifecycle](https://about.gitlab.com/stages-devops-lifecycle/). - -## New to Git and GitLab? - -Working with new systems can be daunting. - -We have the following documentation to rapidly uplift your GitLab knowledge: - -| Topic | Description | -|:--------------------------------------------------------------------------------------------------|:------------| -| [GitLab basics guides](gitlab-basics/index.md) | Start working on the command line and with GitLab. | -| [GitLab workflow overview](https://about.gitlab.com/topics/version-control/what-is-gitlab-workflow/) | Enhance your workflow with the best of GitLab Workflow. | -| [Get started with GitLab CI/CD](ci/quick_start/index.md) | Quickly implement GitLab CI/CD. | -| [Auto DevOps](topics/autodevops/index.md) | Learn more about Auto DevOps in GitLab. | -| [GitLab Markdown](user/markdown.md) | Advanced formatting system (GitLab Flavored Markdown). | - -### User account - -Learn more about GitLab account management: - -| Topic | Description | -|:-----------------------------------------------------------|:------------| -| [User account](user/profile/index.md) | Manage your account. | -| [Authentication](topics/authentication/index.md) | Account security with two-factor authentication, set up your SSH keys, and deploy keys for secure access to your projects. | -| [User settings](user/profile/index.md#access-your-user-settings) | Manage your user settings, two factor authentication, and more. | -| [User permissions](user/permissions.md) | Learn what each role in a project can do. | - -### Git and GitLab - -Learn more about using Git, and using Git with GitLab: - -| Topic | Description | -|:-----------------------------------------------------------------------------|:------------| -| [Git](topics/git/index.md) | Getting started with Git, branching strategies, Git LFS, and advanced use. | -| [Git cheat sheet](https://about.gitlab.com/images/press/git-cheat-sheet.pdf) | Download a PDF describing the most used Git operations. | -| [GitLab Flow](topics/gitlab_flow.md) | Explore the best of Git with the GitLab Flow strategy. | - -## Coming to GitLab from another platform - -If you are coming to GitLab from another platform, the following information is useful: - -| Topic | Description | -|:----------------------------------------------------|:------------| -| [Importing to GitLab](user/project/import/index.md) | Import your projects from GitHub, Bitbucket, GitLab.com, FogBugz, and SVN into GitLab. | -| [Migrating from SVN](user/project/import/svn.md) | Convert a SVN repository to Git and GitLab. | - -## Build an integration with GitLab - -There are many ways to integrate with GitLab, including: - -| Topic | Description | -|:-------------------------------------------|:------------| -| [GitLab REST API](api/README.md) | Integrate with GitLab using our REST API. | -| [GitLab GraphQL API](api/graphql/index.md) | Integrate with GitLab using our GraphQL API. | -| [Integrations](integration/index.md) | Integrations with third-party products. | - -## Contributing to GitLab - -GitLab Community Edition is [open source](https://gitlab.com/gitlab-org/gitlab-foss/) -and GitLab Enterprise Edition is [open-core](https://gitlab.com/gitlab-org/gitlab/). - -Learn how to contribute to GitLab with the following resources: - -| Topic | Description | -|:------------------------------------------------------------|:------------| -| [Development](development/README.md) | How to contribute to GitLab development. | -| [Legal](legal/index.md) | Contributor license agreements. | -| [Writing documentation](development/documentation/index.md) | How to contribute to GitLab Docs. | +<!-- This redirect file can be deleted after 2021-09-28. --> +<!-- Before deletion, see: https://docs.gitlab.com/ee/development/documentation/#move-or-rename-a-page --> diff --git a/doc/administration/auth/README.md b/doc/administration/auth/README.md index a072cc73c43..5ab8653dc35 100644 --- a/doc/administration/auth/README.md +++ b/doc/administration/auth/README.md @@ -1,52 +1,8 @@ --- -comments: false -type: index -stage: Manage -group: Access -info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/engineering/ux/technical-writing/#assignments +redirect_to: 'index.md' --- -# GitLab authentication and authorization **(FREE SELF)** +This document was moved to [another location](index.md). -GitLab integrates with the following external authentication and authorization -providers: - -- [Atlassian](atlassian.md) -- [Auth0](../../integration/auth0.md) -- [Authentiq](authentiq.md) -- [AWS Cognito](cognito.md) -- [Azure](../../integration/azure.md) -- [Bitbucket Cloud](../../integration/bitbucket.md) -- [CAS](../../integration/cas.md) -- [Crowd](crowd.md) -- [Facebook](../../integration/facebook.md) -- [GitHub](../../integration/github.md) -- [GitLab.com](../../integration/gitlab.md) -- [Google OAuth](../../integration/google.md) -- [JWT](jwt.md) -- [Kerberos](../../integration/kerberos.md) -- [LDAP](ldap/index.md): Includes Active Directory, Apple Open Directory, Open LDAP, - and 389 Server. - - [Google Secure LDAP](ldap/google_secure_ldap.md) -- [Salesforce](../../integration/salesforce.md) -- [SAML](../../integration/saml.md) -- [SAML for GitLab.com groups](../../user/group/saml_sso/index.md) **(PREMIUM SAAS)** -- [Shibboleth](../../integration/shibboleth.md) -- [Smartcard](smartcard.md) **(PREMIUM SELF)** -- [Twitter](../../integration/twitter.md) - -NOTE: -UltraAuth has removed their software which supports OmniAuth integration. We have therefore removed all references to UltraAuth integration. - -## SaaS vs Self-Managed Comparison - -The external authentication and authorization providers may support the following capabilities. -For more information, see the links shown on this page for each external provider. - -| Capability | SaaS | Self-Managed | -|-------------------------------------------------|-----------------------------------------|------------------------------------| -| **User Provisioning** | SCIM<br>JIT Provisioning | LDAP Sync | -| **User Detail Updating** (not group management) | Not Available | LDAP Sync | -| **Authentication** | SAML at top-level group (1 provider) | LDAP (multiple providers)<br>Generic OAuth2<br>SAML (only 1 permitted per unique provider)<br>Kerberos<br>JWT<br>Smartcard<br>OmniAuth Providers (only 1 permitted per unique provider) | -| **Provider-to-GitLab Role Sync** | SAML Group Sync | LDAP Group Sync | -| **User Removal** | SCIM (remove user from top-level group) | LDAP (Blocking User from Instance) | +<!-- This redirect file can be deleted after 2021-09-28. --> +<!-- Before deletion, see: https://docs.gitlab.com/ee/development/documentation/#move-or-rename-a-page --> diff --git a/doc/administration/auth/index.md b/doc/administration/auth/index.md new file mode 100644 index 00000000000..a072cc73c43 --- /dev/null +++ b/doc/administration/auth/index.md @@ -0,0 +1,52 @@ +--- +comments: false +type: index +stage: Manage +group: Access +info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/engineering/ux/technical-writing/#assignments +--- + +# GitLab authentication and authorization **(FREE SELF)** + +GitLab integrates with the following external authentication and authorization +providers: + +- [Atlassian](atlassian.md) +- [Auth0](../../integration/auth0.md) +- [Authentiq](authentiq.md) +- [AWS Cognito](cognito.md) +- [Azure](../../integration/azure.md) +- [Bitbucket Cloud](../../integration/bitbucket.md) +- [CAS](../../integration/cas.md) +- [Crowd](crowd.md) +- [Facebook](../../integration/facebook.md) +- [GitHub](../../integration/github.md) +- [GitLab.com](../../integration/gitlab.md) +- [Google OAuth](../../integration/google.md) +- [JWT](jwt.md) +- [Kerberos](../../integration/kerberos.md) +- [LDAP](ldap/index.md): Includes Active Directory, Apple Open Directory, Open LDAP, + and 389 Server. + - [Google Secure LDAP](ldap/google_secure_ldap.md) +- [Salesforce](../../integration/salesforce.md) +- [SAML](../../integration/saml.md) +- [SAML for GitLab.com groups](../../user/group/saml_sso/index.md) **(PREMIUM SAAS)** +- [Shibboleth](../../integration/shibboleth.md) +- [Smartcard](smartcard.md) **(PREMIUM SELF)** +- [Twitter](../../integration/twitter.md) + +NOTE: +UltraAuth has removed their software which supports OmniAuth integration. We have therefore removed all references to UltraAuth integration. + +## SaaS vs Self-Managed Comparison + +The external authentication and authorization providers may support the following capabilities. +For more information, see the links shown on this page for each external provider. + +| Capability | SaaS | Self-Managed | +|-------------------------------------------------|-----------------------------------------|------------------------------------| +| **User Provisioning** | SCIM<br>JIT Provisioning | LDAP Sync | +| **User Detail Updating** (not group management) | Not Available | LDAP Sync | +| **Authentication** | SAML at top-level group (1 provider) | LDAP (multiple providers)<br>Generic OAuth2<br>SAML (only 1 permitted per unique provider)<br>Kerberos<br>JWT<br>Smartcard<br>OmniAuth Providers (only 1 permitted per unique provider) | +| **Provider-to-GitLab Role Sync** | SAML Group Sync | LDAP Group Sync | +| **User Removal** | SCIM (remove user from top-level group) | LDAP (Blocking User from Instance) | diff --git a/doc/administration/auth/ldap/index.md b/doc/administration/auth/ldap/index.md index bc6a854c518..a9d59ca0983 100644 --- a/doc/administration/auth/ldap/index.md +++ b/doc/administration/auth/ldap/index.md @@ -174,6 +174,7 @@ production: | `base` | Base where we can search for users. | **{check-circle}** Yes | `'ou=people,dc=gitlab,dc=example'` or `'DC=mydomain,DC=com'` | | `user_filter` | Filter LDAP users. Format: [RFC 4515](https://tools.ietf.org/search/rfc4515) Note: GitLab does not support `omniauth-ldap`'s custom filter syntax. | **{dotted-circle}** No | For examples, read [Examples of user filters](#examples-of-user-filters). | | `lowercase_usernames` | If enabled, GitLab converts the name to lower case. | **{dotted-circle}** No | boolean | +| `retry_empty_result_with_codes` | An array of LDAP query response code that will attempt to retrying the operation if the result/content is empty. | **{dotted-circle}** No | `[80]` | #### Examples of user filters diff --git a/doc/administration/compliance.md b/doc/administration/compliance.md index b381cee5c2d..4f90d049ed0 100644 --- a/doc/administration/compliance.md +++ b/doc/administration/compliance.md @@ -10,7 +10,7 @@ You can configure the following GitLab features to help ensure that your GitLab instance meets common compliance standards. Click a feature name for additional documentation. -The [security features](../security/README.md) in GitLab may also help you meet +The [security features](../security/index.md) in GitLab may also help you meet relevant compliance standards. | Feature | GitLab tier | GitLab SaaS | Product level | diff --git a/doc/administration/configure.md b/doc/administration/configure.md index 12a8f721ccf..73fbf527fe1 100644 --- a/doc/administration/configure.md +++ b/doc/administration/configure.md @@ -9,7 +9,7 @@ type: reference Customize and configure your self-managed GitLab installation. -- [Authentication](auth/README.md) +- [Authentication](auth/index.md) - [Configuration](../user/admin_area/index.md) - [Repository storage](repository_storage_paths.md) - [Geo](geo/index.md) diff --git a/doc/administration/geo/replication/usage.md b/doc/administration/geo/replication/usage.md index 1491aa3427e..7fe8eec467e 100644 --- a/doc/administration/geo/replication/usage.md +++ b/doc/administration/geo/replication/usage.md @@ -27,7 +27,7 @@ Everything up-to-date ``` NOTE: -If you're using HTTPS instead of [SSH](../../../ssh/README.md) to push to the secondary, +If you're using HTTPS instead of [SSH](../../../ssh/index.md) to push to the secondary, you can't store credentials in the URL like `user:password@URL`. Instead, you can use a [`.netrc` file](https://www.gnu.org/software/inetutils/manual/html_node/The-_002enetrc-file.html) for Unix-like operating systems or `_netrc` for Windows. In that case, the credentials diff --git a/doc/administration/index.md b/doc/administration/index.md index 69e8689c589..b53b9754ee2 100644 --- a/doc/administration/index.md +++ b/doc/administration/index.md @@ -43,7 +43,7 @@ Learn how to install, configure, update, and maintain your GitLab instance. - [Adjust your instance's timezone](timezone.md): Customize the default time zone of GitLab. - [System hooks](../system_hooks/system_hooks.md): Notifications when users, projects and keys are changed. -- [Security](../security/README.md): Learn what you can do to further secure your GitLab instance. +- [Security](../security/index.md): Learn what you can do to further secure your GitLab instance. - [Usage statistics, version check, and usage ping](../user/admin_area/settings/usage_statistics.md): Enable or disable information about your instance to be sent to GitLab, Inc. - [Global user settings](user_settings.md): Configure instance-wide user permissions. - [Polling](polling.md): Configure how often the GitLab UI polls for updates. @@ -122,7 +122,7 @@ Learn how to install, configure, update, and maintain your GitLab instance. - [Libravatar](libravatar.md): Use Libravatar instead of Gravatar for user avatars. - [Sign-up restrictions](../user/admin_area/settings/sign_up_restrictions.md): block email addresses of specific domains, or whitelist only specific domains. - [Access restrictions](../user/admin_area/settings/visibility_and_access_controls.md#enabled-git-access-protocols): Define which Git access protocols can be used to talk to GitLab (SSH, HTTP, HTTPS). -- [Authentication and Authorization](auth/README.md): Configure external authentication with LDAP, SAML, CAS, and additional providers. +- [Authentication and Authorization](auth/index.md): Configure external authentication with LDAP, SAML, CAS, and additional providers. - [Sync LDAP](auth/ldap/index.md) - [Kerberos authentication](../integration/kerberos.md) - See also other [authentication](../topics/authentication/index.md#gitlab-administrators) topics (for example, enforcing 2FA). @@ -241,7 +241,7 @@ who are aware of the risks. - [GitLab Rails console commands](troubleshooting/gitlab_rails_cheat_sheet.md) (for Support Engineers) - [Troubleshooting SSL](troubleshooting/ssl.md) - Related links: - - [GitLab Developer Documentation](../development/README.md) + - [GitLab Developer Documentation](../development/index.md) - [Repairing and recovering broken Git repositories](https://git.seveas.net/repairing-and-recovering-broken-git-repositories.html) - [Testing with OpenSSL](https://www.feistyduck.com/library/openssl-cookbook/online/ch-testing-with-openssl.html) - [`strace` zine](https://wizardzines.com/zines/strace/) diff --git a/doc/administration/pages/index.md b/doc/administration/pages/index.md index c0111209133..9e3f41a1ec8 100644 --- a/doc/administration/pages/index.md +++ b/doc/administration/pages/index.md @@ -64,7 +64,7 @@ Before proceeding with the Pages configuration, you must: 1. Configure a **wildcard DNS record**. 1. (Optional) Have a **wildcard certificate** for that domain if you decide to serve Pages under HTTPS. -1. (Optional but recommended) Enable [Shared runners](../../ci/runners/README.md) +1. (Optional but recommended) Enable [Shared runners](../../ci/runners/index.md) so that your users don't have to bring their own. 1. (Only for custom domains) Have a **secondary IP**. diff --git a/doc/administration/pages/source.md b/doc/administration/pages/source.md index 1427713b9a4..4aaf430db97 100644 --- a/doc/administration/pages/source.md +++ b/doc/administration/pages/source.md @@ -61,7 +61,7 @@ Before proceeding with the Pages configuration, make sure that: Pages artifacts. 1. (Optional) You have a **wildcard certificate** for the Pages domain if you decide to serve Pages (`*.example.io`) under HTTPS. -1. (Optional but recommended) You have configured and enabled the [shared runners](../../ci/runners/README.md) +1. (Optional but recommended) You have configured and enabled the [shared runners](../../ci/runners/index.md) so that your users don't have to bring their own. ### DNS configuration diff --git a/doc/administration/troubleshooting/postgresql.md b/doc/administration/troubleshooting/postgresql.md index 341c6bfbc65..994c194c6db 100644 --- a/doc/administration/troubleshooting/postgresql.md +++ b/doc/administration/troubleshooting/postgresql.md @@ -55,7 +55,7 @@ This section is for links to information elsewhere in the GitLab documentation. - Including [troubleshooting](../postgresql/replication_and_failover.md#troubleshooting) `gitlab-ctl patroni check-leader` and PgBouncer errors. -- [Developer database documentation](../../development/README.md#database-guides), +- [Developer database documentation](../../development/index.md#database-guides), some of which is absolutely not for production use. Including: - Understanding EXPLAIN plans. diff --git a/doc/api/README.md b/doc/api/README.md index b05e0efa455..ae80fd51902 100644 --- a/doc/api/README.md +++ b/doc/api/README.md @@ -236,7 +236,7 @@ To make sure that this token doesn't leak, GitLab: - Grants permissions to the job token only when the job is running. To make sure that this token doesn't leak, you should also configure -your [runners](../ci/runners/README.md) to be secure. Avoid: +your [runners](../ci/runners/index.md) to be secure. Avoid: - Using Docker's `privileged` mode if the machines are re-used. - Using the [`shell` executor](https://docs.gitlab.com/runner/executors/shell.html) when jobs diff --git a/doc/api/graphql/reference/index.md b/doc/api/graphql/reference/index.md index abede2bb13c..a9c61d8f6e3 100644 --- a/doc/api/graphql/reference/index.md +++ b/doc/api/graphql/reference/index.md @@ -3612,6 +3612,25 @@ Input type: `ScanExecutionPolicyCommitInput` | <a id="mutationscanexecutionpolicycommitclientmutationid"></a>`clientMutationId` | [`String`](#string) | A unique identifier for the client performing the mutation. | | <a id="mutationscanexecutionpolicycommiterrors"></a>`errors` | [`[String!]!`](#string) | Errors encountered during execution of the mutation. | +### `Mutation.securityPolicyProjectAssign` + +Input type: `SecurityPolicyProjectAssignInput` + +#### Arguments + +| Name | Type | Description | +| ---- | ---- | ----------- | +| <a id="mutationsecuritypolicyprojectassignclientmutationid"></a>`clientMutationId` | [`String`](#string) | A unique identifier for the client performing the mutation. | +| <a id="mutationsecuritypolicyprojectassignprojectpath"></a>`projectPath` | [`ID!`](#id) | Full path of the project. | +| <a id="mutationsecuritypolicyprojectassignsecuritypolicyprojectid"></a>`securityPolicyProjectId` | [`ProjectID!`](#projectid) | ID of the security policy project. | + +#### Fields + +| Name | Type | Description | +| ---- | ---- | ----------- | +| <a id="mutationsecuritypolicyprojectassignclientmutationid"></a>`clientMutationId` | [`String`](#string) | A unique identifier for the client performing the mutation. | +| <a id="mutationsecuritypolicyprojectassignerrors"></a>`errors` | [`[String!]!`](#string) | Errors encountered during execution of the mutation. | + ### `Mutation.terraformStateDelete` Input type: `TerraformStateDeleteInput` diff --git a/doc/api/job_artifacts.md b/doc/api/job_artifacts.md index 54404559577..596b41bdfdb 100644 --- a/doc/api/job_artifacts.md +++ b/doc/api/job_artifacts.md @@ -20,7 +20,7 @@ GET /projects/:id/jobs/:job_id/artifacts |-------------|----------------|----------|--------------------------------------------------------------------------------------------------------------| | `id` | integer/string | yes | ID or [URL-encoded path of the project](README.md#namespaced-path-encoding) owned by the authenticated user. | | `job_id` | integer | yes | ID of a job. | -| `job_token` **(PREMIUM)** | string | no | To be used with [triggers](../ci/triggers/README.md#when-a-pipeline-depends-on-the-artifacts-of-another-pipeline) for multi-project pipelines. It should be invoked only inside `.gitlab-ci.yml`. Its value is always `$CI_JOB_TOKEN`. | +| `job_token` **(PREMIUM)** | string | no | To be used with [triggers](../ci/triggers/index.md#when-a-pipeline-depends-on-the-artifacts-of-another-pipeline) for multi-project pipelines. It should be invoked only inside `.gitlab-ci.yml`. Its value is always `$CI_JOB_TOKEN`. | Example request using the `PRIVATE-TOKEN` header: @@ -85,7 +85,7 @@ Parameters | `id` | integer/string | yes | ID or [URL-encoded path of the project](README.md#namespaced-path-encoding) owned by the authenticated user. | | `ref_name` | string | yes | Branch or tag name in repository. HEAD or SHA references are not supported. | | `job` | string | yes | The name of the job. | -| `job_token` **(PREMIUM)** | string | no | To be used with [triggers](../ci/triggers/README.md#when-a-pipeline-depends-on-the-artifacts-of-another-pipeline) for multi-project pipelines. It should be invoked only inside `.gitlab-ci.yml`. Its value is always `$CI_JOB_TOKEN`. | +| `job_token` **(PREMIUM)** | string | no | To be used with [triggers](../ci/triggers/index.md#when-a-pipeline-depends-on-the-artifacts-of-another-pipeline) for multi-project pipelines. It should be invoked only inside `.gitlab-ci.yml`. Its value is always `$CI_JOB_TOKEN`. | Example request using the `PRIVATE-TOKEN` header: @@ -146,7 +146,7 @@ Parameters | `id` | integer/string | yes | ID or [URL-encoded path of the project](README.md#namespaced-path-encoding) owned by the authenticated user. | | `job_id` | integer | yes | The unique job identifier. | | `artifact_path` | string | yes | Path to a file inside the artifacts archive. | -| `job_token` **(PREMIUM)** | string | no | To be used with [triggers](../ci/triggers/README.md#when-a-pipeline-depends-on-the-artifacts-of-another-pipeline) for multi-project pipelines. It should be invoked only inside `.gitlab-ci.yml`. Its value is always `$CI_JOB_TOKEN`. | +| `job_token` **(PREMIUM)** | string | no | To be used with [triggers](../ci/triggers/index.md#when-a-pipeline-depends-on-the-artifacts-of-another-pipeline) for multi-project pipelines. It should be invoked only inside `.gitlab-ci.yml`. Its value is always `$CI_JOB_TOKEN`. | Example request: @@ -188,7 +188,7 @@ Parameters: | `ref_name` | string | yes | Branch or tag name in repository. `HEAD` or `SHA` references are not supported. | | `artifact_path` | string | yes | Path to a file inside the artifacts archive. | | `job` | string | yes | The name of the job. | -| `job_token` **(PREMIUM)** | string | no | To be used with [triggers](../ci/triggers/README.md#when-a-pipeline-depends-on-the-artifacts-of-another-pipeline) for multi-project pipelines. It should be invoked only inside `.gitlab-ci.yml`. Its value is always `$CI_JOB_TOKEN`. | +| `job_token` **(PREMIUM)** | string | no | To be used with [triggers](../ci/triggers/index.md#when-a-pipeline-depends-on-the-artifacts-of-another-pipeline) for multi-project pipelines. It should be invoked only inside `.gitlab-ci.yml`. Its value is always `$CI_JOB_TOKEN`. | Example request: diff --git a/doc/api/pipeline_triggers.md b/doc/api/pipeline_triggers.md index 94122a40b2d..bd21c510df1 100644 --- a/doc/api/pipeline_triggers.md +++ b/doc/api/pipeline_triggers.md @@ -6,7 +6,7 @@ info: To determine the technical writer assigned to the Stage/Group associated w # Pipeline triggers API **(FREE)** -You can read more about [triggering pipelines through the API](../ci/triggers/README.md). +You can read more about [triggering pipelines through the API](../ci/triggers/index.md). ## List project triggers diff --git a/doc/api/runners.md b/doc/api/runners.md index 951e72edcb5..31879ab9042 100644 --- a/doc/api/runners.md +++ b/doc/api/runners.md @@ -14,7 +14,7 @@ There are two tokens to take into account when connecting a runner with GitLab. | Token | Description | | ----- | ----------- | -| Registration token | Token used to [register the runner](https://docs.gitlab.com/runner/register/). It can be [obtained through GitLab](../ci/runners/README.md). | +| Registration token | Token used to [register the runner](https://docs.gitlab.com/runner/register/). It can be [obtained through GitLab](../ci/runners/index.md). | | Authentication token | Token used to authenticate the runner with the GitLab instance. It is obtained either automatically when [registering a runner](https://docs.gitlab.com/runner/register/), or manually when [registering the runner via the Runner API](#register-a-new-runner). | Here's an example of how the two tokens are used in runner registration: diff --git a/doc/ci/README.md b/doc/ci/README.md index c58e02d08d7..c880c1bbefa 100644 --- a/doc/ci/README.md +++ b/doc/ci/README.md @@ -73,7 +73,7 @@ GitLab CI/CD supports numerous configuration options: | [Custom path for `.gitlab-ci.yml`](pipelines/settings.md#custom-cicd-configuration-file) | Define a custom path for the CI/CD configuration file. | | [Git submodules for CI/CD](git_submodules.md) | Configure jobs for using Git submodules. | | [SSH keys for CI/CD](ssh_keys/index.md) | Using SSH keys in your CI pipelines. | -| [Pipeline triggers](triggers/README.md) | Trigger pipelines through the API. | +| [Pipeline triggers](triggers/index.md) | Trigger pipelines through the API. | | [Pipelines for Merge Requests](merge_request_pipelines/index.md) | Design a pipeline structure for running a pipeline in merge requests. | | [Integrate with Kubernetes clusters](../user/project/clusters/index.md) | Connect your project to Google Kubernetes Engine (GKE) or an existing Kubernetes cluster. | | [Optimize GitLab and GitLab Runner for large repositories](large_repositories/index.md) | Recommended strategies for handling large repositories. | @@ -123,7 +123,7 @@ Its feature set is listed on the table below according to DevOps stages. ## Examples Find example project code and tutorials for using GitLab CI/CD with a variety of app frameworks, languages, and platforms -on the [CI Examples](examples/README.md) page. +on the [CI Examples](examples/index.md) page. ## Administration **(FREE SELF)** diff --git a/doc/ci/cloud_deployment/ecs/quick_start_guide.md b/doc/ci/cloud_deployment/ecs/quick_start_guide.md index a801be549df..700a922a74d 100644 --- a/doc/ci/cloud_deployment/ecs/quick_start_guide.md +++ b/doc/ci/cloud_deployment/ecs/quick_start_guide.md @@ -14,7 +14,7 @@ In this guide, you begin by creating an ECS cluster manually using the AWS conso deploy a simple application that you create from a GitLab template. These instructions work for both SaaS and self-managed GitLab instances. -Ensure your own [runners are configured](../../runners/README.md). +Ensure your own [runners are configured](../../runners/index.md). ## Prerequisites diff --git a/doc/ci/docker/using_docker_build.md b/doc/ci/docker/using_docker_build.md index 496ac484a0e..4b7a6b114eb 100644 --- a/doc/ci/docker/using_docker_build.md +++ b/doc/ci/docker/using_docker_build.md @@ -26,7 +26,7 @@ If you don't want to execute a runner in privileged mode, but want to use `docker build`, you can also [use kaniko](using_kaniko.md). If you are using shared runners on GitLab.com, -[learn more about how these runners are configured](../runners/README.md). +[learn more about how these runners are configured](../runners/index.md). ### Use the shell executor @@ -90,7 +90,7 @@ The Docker image has all of the `docker` tools installed and can run the job script in context of the image in privileged mode. We recommend you use [Docker-in-Docker with TLS enabled](#docker-in-docker-with-tls-enabled), -which is supported by [GitLab.com shared runners](../runners/README.md). +which is supported by [GitLab.com shared runners](../runners/index.md). You should always specify a specific version of the image, like `docker:19.03.12`. If you use a tag like `docker:stable`, you have no control over which version is used. diff --git a/doc/ci/docker/using_kaniko.md b/doc/ci/docker/using_kaniko.md index 5e294d14f04..2feca97b37b 100644 --- a/doc/ci/docker/using_kaniko.md +++ b/doc/ci/docker/using_kaniko.md @@ -133,7 +133,7 @@ The [Least Privilege Container Builds with Kaniko on GitLab](https://www.youtube video is a walkthrough of the [Kaniko Docker Build](https://gitlab.com/guided-explorations/containers/kaniko-docker-build) Guided Exploration project pipeline. It was tested on: -- [GitLab.com shared runners](../runners/README.md) +- [GitLab.com shared runners](../runners/index.md) - [The Kubernetes runner executor](https://docs.gitlab.com/runner/executors/kubernetes.html) The example can be copied to your own group or instance for testing. More details diff --git a/doc/ci/enable_or_disable_ci.md b/doc/ci/enable_or_disable_ci.md index 268be535591..95f1fa9fb33 100644 --- a/doc/ci/enable_or_disable_ci.md +++ b/doc/ci/enable_or_disable_ci.md @@ -11,7 +11,7 @@ To effectively use GitLab CI/CD, you need: - A valid [`.gitlab-ci.yml`](yaml/README.md) file present at the root directory of your project. -- A [runner](runners/README.md) properly set up. +- A [runner](runners/index.md) properly set up. You can read our [quick start guide](quick_start/index.md) to get you started. diff --git a/doc/ci/examples/README.md b/doc/ci/examples/README.md index 90273190697..5ab8653dc35 100644 --- a/doc/ci/examples/README.md +++ b/doc/ci/examples/README.md @@ -1,182 +1,8 @@ --- -stage: Verify -group: Pipeline Execution -info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/engineering/ux/technical-writing/#assignments -comments: false -type: index +redirect_to: 'index.md' --- -# GitLab CI/CD Examples +This document was moved to [another location](index.md). -This page contains links to a variety of examples that can help you understand how to -implement [GitLab CI/CD](../README.md) for your specific use case. - -Examples are available in several forms. As a collection of: - -- `.gitlab-ci.yml` [template files](#cicd-templates) maintained in GitLab, for many - common frameworks and programming languages. -- Repositories with [example projects](https://gitlab.com/gitlab-examples) for various languages. You can fork and adjust them to your own needs. Projects include an example of using [Review Apps with a static site served by NGINX](https://gitlab.com/gitlab-examples/review-apps-nginx/). -- Examples and [other resources](#other-resources) listed below. - -## CI/CD examples - -The following table lists examples with step-by-step tutorials that are contained in this section: - -| Use case | Resource | -|-------------------------------|----------| -| Browser performance testing | [Browser Performance Testing with the Sitespeed.io container](../../user/project/merge_requests/browser_performance_testing.md). | -| Deployment with Dpl | [Using `dpl` as deployment tool](deployment/README.md). | -| GitLab Pages | See the [GitLab Pages](../../user/project/pages/index.md) documentation for a complete example of deploying a static site. | -| End-to-end testing | [End-to-end testing with GitLab CI/CD and WebdriverIO](end_to_end_testing_webdriverio/index.md). | -| Load performance testing | [Load Performance Testing with the k6 container](../../user/project/merge_requests/load_performance_testing.md). | -| Multi project pipeline | [Build, test deploy using multi project pipeline](https://gitlab.com/gitlab-examples/upstream-project). | -| npm with semantic-release | [Publish npm packages to the GitLab Package Registry using semantic-release](semantic-release.md). | -| PHP with Laravel, Envoy | [Test and deploy Laravel applications with GitLab CI/CD and Envoy](laravel_with_gitlab_and_envoy/index.md). | -| PHP with npm, SCP | [Running Composer and npm scripts with deployment via SCP in GitLab CI/CD](deployment/composer-npm-deploy.md). | -| PHP with PHPunit, `atoum` | [Testing PHP projects](php.md). | -| Secrets management with Vault | [Authenticating and Reading Secrets With HashiCorp Vault](authenticating-with-hashicorp-vault/index.md). | - -### Contributed examples - -You can help people that use your favorite programming language by submitting a link -to a guide for that language. These contributed guides are hosted externally or in -separate example projects: - -| Use case | Resource | -|-------------------------------|----------| -| Clojure | [Test a Clojure application with GitLab CI/CD](https://gitlab.com/gitlab-examples/clojure-web-application). | -| Game development | [DevOps and Game Development with GitLab CI/CD](https://gitlab.com/gitlab-examples/gitlab-game-demo/). | -| Java with Maven | [How to deploy Maven projects to Artifactory with GitLab CI/CD](https://gitlab.com/gitlab-examples/maven/simple-maven-example). | -| Java with Spring Boot | [Deploy a Spring Boot application to Cloud Foundry with GitLab CI/CD](https://gitlab.com/gitlab-examples/spring-gitlab-cf-deploy-demo). | -| Parallel testing Ruby & JS | [GitLab CI/CD parallel jobs testing for Ruby & JavaScript projects](https://docs.knapsackpro.com/2019/how-to-run-parallel-jobs-for-rspec-tests-on-gitlab-ci-pipeline-and-speed-up-ruby-javascript-testing). | -| Python on Heroku | [Test and deploy a Python application with GitLab CI/CD](https://gitlab.com/gitlab-examples/python-getting-started). | -| Ruby on Heroku | [Test and deploy a Ruby application with GitLab CI/CD](https://gitlab.com/gitlab-examples/ruby-getting-started). | -| Scala on Heroku | [Test and deploy a Scala application to Heroku](https://gitlab.com/gitlab-examples/scala-sbt). | - -## CI/CD templates - -Get started with GitLab CI/CD and your favorite programming language or framework by using a -`.gitlab-ci.yml` [template](https://gitlab.com/gitlab-org/gitlab/-/tree/master/lib/gitlab/ci/templates). - -When you create a `gitlab-ci.yml` file in the UI, you can -choose one of these templates: - -- [Android (`Android.gitlab-ci.yml`)](https://gitlab.com/gitlab-org/gitlab/-/blob/master/lib/gitlab/ci/templates/Android.gitlab-ci.yml) -- [Android with fastlane (`Android-Fastlane.gitlab-ci.yml`)](https://gitlab.com/gitlab-org/gitlab/-/blob/master/lib/gitlab/ci/templates/Android-Fastlane.gitlab-ci.yml) -- [Bash (`Bash.gitlab-ci.yml`)](https://gitlab.com/gitlab-org/gitlab/-/blob/master/lib/gitlab/ci/templates/Bash.gitlab-ci.yml) -- [C++ (`C++.gitlab-ci.yml`)](https://gitlab.com/gitlab-org/gitlab/-/blob/master/lib/gitlab/ci/templates/C++.gitlab-ci.yml) -- [Chef (`Chef.gitlab-ci.yml`)](https://gitlab.com/gitlab-org/gitlab/-/blob/master/lib/gitlab/ci/templates/Chef.gitlab-ci.yml) -- [Clojure (`Clojure.gitlab-ci.yml`)](https://gitlab.com/gitlab-org/gitlab/-/blob/master/lib/gitlab/ci/templates/Clojure.gitlab-ci.yml) -- [Composer `Composer.gitlab-ci.yml`](https://gitlab.com/gitlab-org/gitlab/-/blob/master/lib/gitlab/ci/templates/Composer.gitlab-ci.yml) -- [Crystal (`Crystal.gitlab-ci.yml`)](https://gitlab.com/gitlab-org/gitlab/-/blob/master/lib/gitlab/ci/templates/Crystal.gitlab-ci.yml) -- [Dart (`Dart.gitlab-ci.yml`)](https://gitlab.com/gitlab-org/gitlab/-/blob/master/lib/gitlab/ci/templates/Dart.gitlab-ci.yml) -- [Django (`Django.gitlab-ci.yml`)](https://gitlab.com/gitlab-org/gitlab/-/blob/master/lib/gitlab/ci/templates/Django.gitlab-ci.yml) -- [Docker (`Docker.gitlab-ci.yml`)](https://gitlab.com/gitlab-org/gitlab/-/blob/master/lib/gitlab/ci/templates/Docker.gitlab-ci.yml) -- [dotNET (`dotNET.gitlab-ci.yml`)](https://gitlab.com/gitlab-org/gitlab/-/blob/master/lib/gitlab/ci/templates/dotNET.gitlab-ci.yml) -- [dotNET Core (`dotNET-Core.yml`)](https://gitlab.com/gitlab-org/gitlab/-/blob/master/lib/gitlab/ci/templates/dotNET-Core.yml) -- [Elixir (`Elixir.gitlab-ci.yml`)](https://gitlab.com/gitlab-org/gitlab/-/blob/master/lib/gitlab/ci/templates/Elixir.gitlab-ci.yml) -- [Flutter (`Flutter.gitlab-ci.yml`)](https://gitlab.com/gitlab-org/gitlab/-/blob/master/lib/gitlab/ci/templates/Flutter.gitlab-ci.yml) -- [goLang (`Go.gitlab-ci.yml`)](https://gitlab.com/gitlab-org/gitlab/-/blob/master/lib/gitlab/ci/templates/Go.gitlab-ci.yml) -- [Gradle (`Gradle.gitlab-ci.yml`)](https://gitlab.com/gitlab-org/gitlab/-/blob/master/lib/gitlab/ci/templates/Gradle.gitlab-ci.yml) -- [Grails (`Grails.gitlab-ci.yml`)](https://gitlab.com/gitlab-org/gitlab/-/blob/master/lib/gitlab/ci/templates/Grails.gitlab-ci.yml) -- [iOS with fastlane (`iOS-Fastlane.gitlab-ci.yml`)](https://gitlab.com/gitlab-org/gitlab/-/blob/master/lib/gitlab/ci/templates/iOS-Fastlane.gitlab-ci.yml) -- [Julia (`Julia.gitlab-ci.yml`)](https://gitlab.com/gitlab-org/gitlab/-/blob/master/lib/gitlab/ci/templates/Julia.gitlab-ci.yml) -- [Laravel (`Laravel.gitlab-ci.yml`)](https://gitlab.com/gitlab-org/gitlab/-/blob/master/lib/gitlab/ci/templates/Laravel.gitlab-ci.yml) -- [LaTeX (`LaTeX.gitlab-ci.yml`)](https://gitlab.com/gitlab-org/gitlab/-/blob/master/lib/gitlab/ci/templates/LaTeX.gitlab-ci.yml) -- [Maven (`Maven.gitlab-ci.yml`)](https://gitlab.com/gitlab-org/gitlab/-/blob/master/lib/gitlab/ci/templates/Maven.gitlab-ci.yml) -- [Mono (`Mono.gitlab-ci.yml`)](https://gitlab.com/gitlab-org/gitlab/-/blob/master/lib/gitlab/ci/templates/Mono.gitlab-ci.yml) -- [npm (`npm.gitlab-ci.yml`)](https://gitlab.com/gitlab-org/gitlab/-/blob/master/lib/gitlab/ci/templates/npm.gitlab-ci.yml) -- [Node.js (`Nodejs.gitlab-ci.yml`)](https://gitlab.com/gitlab-org/gitlab/-/blob/master/lib/gitlab/ci/templates/Nodejs.gitlab-ci.yml) -- [OpenShift (`OpenShift.gitlab-ci.yml`)](https://gitlab.com/gitlab-org/gitlab/-/blob/master/lib/gitlab/ci/templates/OpenShift.gitlab-ci.yml) -- [Packer (`Packer.gitlab-ci.yml`)](https://gitlab.com/gitlab-org/gitlab/-/blob/master/lib/gitlab/ci/templates/Packer.gitlab-ci.yml) -- [PHP (`PHP.gitlab-ci.yml`)](https://gitlab.com/gitlab-org/gitlab/-/blob/master/lib/gitlab/ci/templates/PHP.gitlab-ci.yml) -- [Python (`Python.gitlab-ci.yml`)](https://gitlab.com/gitlab-org/gitlab/-/blob/master/lib/gitlab/ci/templates/Python.gitlab-ci.yml) -- [Ruby (`Ruby.gitlab-ci.yml`)](https://gitlab.com/gitlab-org/gitlab/-/blob/master/lib/gitlab/ci/templates/Ruby.gitlab-ci.yml) -- [Rust (`Rust.gitlab-ci.yml`)](https://gitlab.com/gitlab-org/gitlab/-/blob/master/lib/gitlab/ci/templates/Rust.gitlab-ci.yml) -- [Scala (`Scala.gitlab-ci.yml`)](https://gitlab.com/gitlab-org/gitlab/-/blob/master/lib/gitlab/ci/templates/Scala.gitlab-ci.yml) -- [Swift (`Swift.gitlab-ci.yml`)](https://gitlab.com/gitlab-org/gitlab/-/blob/master/lib/gitlab/ci/templates/Swift.gitlab-ci.yml) -- [Terraform (`Terraform.latest.gitlab-ci.yml`)](https://gitlab.com/gitlab-org/gitlab/-/blob/master/lib/gitlab/ci/templates/Terraform.latest.gitlab-ci.yml) - -If a programming language or framework template is not in this list, you can contribute -one. To create a template, submit a merge request -to [the templates list](https://gitlab.com/gitlab-org/gitlab/-/tree/master/lib/gitlab/ci/templates). - -### Adding templates to your GitLab installation **(PREMIUM SELF)** - -You can add custom examples and templates to your self-managed GitLab instance. -Your GitLab administrator can [designate an instance template repository](../../user/admin_area/settings/instance_template_repository.md) -that contains examples and templates specific to your organization. - -## Other resources - -This section provides further resources to help you get familiar with various uses of GitLab CI/CD. -Note that older articles and videos may not reflect the state of the latest GitLab release. - -### CI/CD in the cloud - -For examples of setting up GitLab CI/CD for cloud-based environments, see: - -- [How to set up multi-account AWS SAM deployments with GitLab CI](https://about.gitlab.com/blog/2019/02/04/multi-account-aws-sam-deployments-with-gitlab-ci/) -- [Automating Kubernetes Deployments with GitLab CI/CD](https://www.youtube.com/watch?v=wEDRfAz6_Uw) -- [How to autoscale continuous deployment with GitLab Runner on DigitalOcean](https://about.gitlab.com/blog/2018/06/19/autoscale-continuous-deployment-gitlab-runner-digital-ocean/) -- [How to create a CI/CD pipeline with Auto Deploy to Kubernetes using GitLab and Helm](https://about.gitlab.com/blog/2017/09/21/how-to-create-ci-cd-pipeline-with-autodeploy-to-kubernetes-using-gitlab-and-helm/) -- [Demo - Deploying from GitLab to OpenShift Container Cluster](https://youtu.be/EwbhA53Jpp4) - -See also the following video overviews: - -- [Kubernetes, GitLab, and Cloud Native](https://www.youtube.com/watch?v=d-9awBxEbvQ). -- [Deploying to IBM Cloud with GitLab CI/CD](https://www.youtube.com/watch?v=6ZF4vgKMd-g). - -### Customer stories - -For some customer experiences with GitLab CI/CD, see: - -- [How Verizon Connect reduced data center deploys from 30 days to under 8 hours with GitLab](https://about.gitlab.com/blog/2019/02/14/verizon-customer-story/) -- [How Wag! cut their release process from 40 minutes to just 6](https://about.gitlab.com/blog/2019/01/16/wag-labs-blog-post/) -- [How Jaguar Land Rover embraced CI to speed up their software lifecycle](https://about.gitlab.com/blog/2018/07/23/chris-hill-devops-enterprise-summit-talk/) - -### Getting started - -For some examples to help get you started, see: - -- [GitLab CI/CD's 2018 highlights](https://about.gitlab.com/blog/2019/01/21/gitlab-ci-cd-features-improvements/) -- [A beginner's guide to continuous integration](https://about.gitlab.com/blog/2018/01/22/a-beginners-guide-to-continuous-integration/) - -### Implementing GitLab CI/CD - -For examples of others who have implemented GitLab CI/CD, see: - -- [How to streamline interactions between multiple repositories with multi-project pipelines](https://about.gitlab.com/blog/2018/10/31/use-multiproject-pipelines-with-gitlab-cicd/) -- [How we used GitLab CI to build GitLab faster](https://about.gitlab.com/blog/2018/05/02/using-gitlab-ci-to-build-gitlab-faster/) -- [Test all the things in GitLab CI with Docker by example](https://about.gitlab.com/blog/2018/02/05/test-all-the-things-gitlab-ci-docker-examples/) -- [A Craftsman looks at continuous integration](https://about.gitlab.com/blog/2018/01/17/craftsman-looks-at-continuous-integration/) -- [Go tools and GitLab: How to do continuous integration like a boss](https://about.gitlab.com/blog/2017/11/27/go-tools-and-gitlab-how-to-do-continuous-integration-like-a-boss/) -- [GitBot – automating boring Git operations with CI](https://about.gitlab.com/blog/2017/11/02/automating-boring-git-operations-gitlab-ci/) -- [How to use GitLab CI for Vue.js](https://about.gitlab.com/blog/2017/09/12/vuejs-app-gitlab/) -- Video: [GitLab CI/CD Deep Dive](https://youtu.be/pBe4t1CD8Fc?t=195) -- [Dockerizing GitLab Review Apps](https://about.gitlab.com/blog/2017/07/11/dockerizing-review-apps/) -- [Fast and natural continuous integration with GitLab CI](https://about.gitlab.com/blog/2017/05/22/fast-and-natural-continuous-integration-with-gitlab-ci/) -- [Demo: CI/CD with GitLab in action](https://about.gitlab.com/blog/2017/03/13/ci-cd-demo/) - -### Migrating to GitLab from third-party CI tools - -- [Migrating from Jenkins to GitLab](https://youtu.be/RlEVGOpYF5Y) - -### Integrating GitLab CI/CD with other systems - -To see how you can integrate GitLab CI/CD with third-party systems, see: - -- [Streamline and shorten error remediation with Sentry's new GitLab integration](https://about.gitlab.com/blog/2019/01/25/sentry-integration-blog-post/) -- [How to simplify your smart home configuration with GitLab CI/CD](https://about.gitlab.com/blog/2018/08/02/using-the-gitlab-ci-slash-cd-for-smart-home-configuration-management/) -- [Demo: GitLab + Jira + Jenkins](https://about.gitlab.com/blog/2018/07/30/gitlab-workflow-with-jira-jenkins/) -- [Introducing Auto Breakfast from GitLab (sort of)](https://about.gitlab.com/blog/2018/06/29/introducing-auto-breakfast-from-gitlab/) - -### Mobile development - -For help with using GitLab CI/CD for mobile application development, see: - -- [How to publish Android apps to the Google Play Store with GitLab and fastlane](https://about.gitlab.com/blog/2019/01/28/android-publishing-with-gitlab-and-fastlane/) -- [Setting up GitLab CI for Android projects](https://about.gitlab.com/blog/2018/10/24/setting-up-gitlab-ci-for-android-projects/) -- [Working with YAML in GitLab CI from the Android perspective](https://about.gitlab.com/blog/2017/11/20/working-with-yaml-gitlab-ci-android/) -- [How to use GitLab CI and MacStadium to build your macOS or iOS projects](https://about.gitlab.com/blog/2017/05/15/how-to-use-macstadium-and-gitlab-ci-to-build-your-macos-or-ios-projects/) -- [Setting up GitLab CI for iOS projects](https://about.gitlab.com/blog/2016/03/10/setting-up-gitlab-ci-for-ios-projects/) +<!-- This redirect file can be deleted after 2021-09-28. --> +<!-- Before deletion, see: https://docs.gitlab.com/ee/development/documentation/#move-or-rename-a-page --> diff --git a/doc/ci/examples/deployment/README.md b/doc/ci/examples/deployment/README.md index 4d2c22a17f0..5ab8653dc35 100644 --- a/doc/ci/examples/deployment/README.md +++ b/doc/ci/examples/deployment/README.md @@ -1,131 +1,8 @@ --- -stage: Release -group: Release -info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/engineering/ux/technical-writing/#assignments -type: tutorial +redirect_to: 'index.md' --- -# Using Dpl as deployment tool +This document was moved to [another location](index.md). -[Dpl](https://github.com/travis-ci/dpl) (pronounced like the letters D-P-L) is a deploy tool made for -continuous deployment that's developed and used by Travis CI, but can also be -used with GitLab CI/CD. - -Dpl can be used to deploy to any of the [supported providers](https://github.com/travis-ci/dpl#supported-providers). - -## Requirements - -To use Dpl you need at least Ruby 1.9.3 with ability to install gems. - -## Basic usage - -Dpl can be installed on any machine with: - -```shell -gem install dpl -``` - -This allows you to test all commands from your local terminal, rather than -having to test it on a CI server. - -If you don't have Ruby installed you can do it on Debian-compatible Linux with: - -```shell -apt-get update -apt-get install ruby-dev -``` - -The Dpl provides support for vast number of services, including: Heroku, Cloud Foundry, AWS/S3, and more. -To use it simply define provider and any additional parameters required by the provider. - -For example if you want to use it to deploy your application to Heroku, you need to specify `heroku` as provider, specify `api_key` and `app`. -All possible parameters can be found in the [Heroku API section](https://github.com/travis-ci/dpl#heroku-api). - -```yaml -staging: - stage: deploy - script: - - gem install dpl - - dpl --provider=heroku --app=my-app-staging --api_key=$HEROKU_STAGING_API_KEY -``` - -In the above example we use Dpl to deploy `my-app-staging` to Heroku server with API key stored in `HEROKU_STAGING_API_KEY` secure variable. - -To use different provider take a look at long list of [Supported Providers](https://github.com/travis-ci/dpl#supported-providers). - -## Using Dpl with Docker - -In most cases, you configured [GitLab Runner](https://docs.gitlab.com/runner/) to use your server's shell commands. -This means that all commands are run in the context of local user (e.g. `gitlab_runner` or `gitlab_ci_multi_runner`). -It also means that most probably in your Docker container you don't have the Ruby runtime installed. -You must install it: - -```yaml -staging: - stage: deploy - script: - - apt-get update -yq - - apt-get install -y ruby-dev - - gem install dpl - - dpl --provider=heroku --app=my-app-staging --api_key=$HEROKU_STAGING_API_KEY - only: - - master -``` - -The first line `apt-get update -yq` updates the list of available packages, -where second `apt-get install -y ruby-dev` installs the Ruby runtime on system. -The above example is valid for all Debian-compatible systems. - -## Usage in staging and production - -It's pretty common in the development workflow to have staging (development) and -production environments - -Let's consider the following example: we would like to deploy the `master` -branch to `staging` and all tags to the `production` environment. -The final `.gitlab-ci.yml` for that setup would look like this: - -```yaml -staging: - stage: deploy - script: - - gem install dpl - - dpl --provider=heroku --app=my-app-staging --api_key=$HEROKU_STAGING_API_KEY - only: - - master - -production: - stage: deploy - script: - - gem install dpl - - dpl --provider=heroku --app=my-app-production --api_key=$HEROKU_PRODUCTION_API_KEY - only: - - tags -``` - -We created two deploy jobs that are executed on different events: - -1. `staging` is executed for all commits that were pushed to `master` branch, -1. `production` is executed for all pushed tags. - -We also use two secure variables: - -1. `HEROKU_STAGING_API_KEY` - Heroku API key used to deploy staging app, -1. `HEROKU_PRODUCTION_API_KEY` - Heroku API key used to deploy production app. - -## Storing API keys - -To add secure variables, navigate to your project's -**Settings > CI/CD > Variables**. The variables that are defined -in the project settings are sent along with the build script to the runner. -The secure variables are stored out of the repository. Never store secrets in -your project's `.gitlab-ci.yml`. It is also important that the secret's value -is hidden in the job log. - -You access added variable by prefixing it's name with `$` (on non-Windows runners) -or `%` (for Windows Batch runners): - -1. `$VARIABLE` - use it for non-Windows runners -1. `%VARIABLE%` - use it for Windows Batch runners - -Read more about the [CI/CD variables](../../variables/README.md). +<!-- This redirect file can be deleted after 2021-09-28. --> +<!-- Before deletion, see: https://docs.gitlab.com/ee/development/documentation/#move-or-rename-a-page --> diff --git a/doc/ci/examples/deployment/index.md b/doc/ci/examples/deployment/index.md new file mode 100644 index 00000000000..4d2c22a17f0 --- /dev/null +++ b/doc/ci/examples/deployment/index.md @@ -0,0 +1,131 @@ +--- +stage: Release +group: Release +info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/engineering/ux/technical-writing/#assignments +type: tutorial +--- + +# Using Dpl as deployment tool + +[Dpl](https://github.com/travis-ci/dpl) (pronounced like the letters D-P-L) is a deploy tool made for +continuous deployment that's developed and used by Travis CI, but can also be +used with GitLab CI/CD. + +Dpl can be used to deploy to any of the [supported providers](https://github.com/travis-ci/dpl#supported-providers). + +## Requirements + +To use Dpl you need at least Ruby 1.9.3 with ability to install gems. + +## Basic usage + +Dpl can be installed on any machine with: + +```shell +gem install dpl +``` + +This allows you to test all commands from your local terminal, rather than +having to test it on a CI server. + +If you don't have Ruby installed you can do it on Debian-compatible Linux with: + +```shell +apt-get update +apt-get install ruby-dev +``` + +The Dpl provides support for vast number of services, including: Heroku, Cloud Foundry, AWS/S3, and more. +To use it simply define provider and any additional parameters required by the provider. + +For example if you want to use it to deploy your application to Heroku, you need to specify `heroku` as provider, specify `api_key` and `app`. +All possible parameters can be found in the [Heroku API section](https://github.com/travis-ci/dpl#heroku-api). + +```yaml +staging: + stage: deploy + script: + - gem install dpl + - dpl --provider=heroku --app=my-app-staging --api_key=$HEROKU_STAGING_API_KEY +``` + +In the above example we use Dpl to deploy `my-app-staging` to Heroku server with API key stored in `HEROKU_STAGING_API_KEY` secure variable. + +To use different provider take a look at long list of [Supported Providers](https://github.com/travis-ci/dpl#supported-providers). + +## Using Dpl with Docker + +In most cases, you configured [GitLab Runner](https://docs.gitlab.com/runner/) to use your server's shell commands. +This means that all commands are run in the context of local user (e.g. `gitlab_runner` or `gitlab_ci_multi_runner`). +It also means that most probably in your Docker container you don't have the Ruby runtime installed. +You must install it: + +```yaml +staging: + stage: deploy + script: + - apt-get update -yq + - apt-get install -y ruby-dev + - gem install dpl + - dpl --provider=heroku --app=my-app-staging --api_key=$HEROKU_STAGING_API_KEY + only: + - master +``` + +The first line `apt-get update -yq` updates the list of available packages, +where second `apt-get install -y ruby-dev` installs the Ruby runtime on system. +The above example is valid for all Debian-compatible systems. + +## Usage in staging and production + +It's pretty common in the development workflow to have staging (development) and +production environments + +Let's consider the following example: we would like to deploy the `master` +branch to `staging` and all tags to the `production` environment. +The final `.gitlab-ci.yml` for that setup would look like this: + +```yaml +staging: + stage: deploy + script: + - gem install dpl + - dpl --provider=heroku --app=my-app-staging --api_key=$HEROKU_STAGING_API_KEY + only: + - master + +production: + stage: deploy + script: + - gem install dpl + - dpl --provider=heroku --app=my-app-production --api_key=$HEROKU_PRODUCTION_API_KEY + only: + - tags +``` + +We created two deploy jobs that are executed on different events: + +1. `staging` is executed for all commits that were pushed to `master` branch, +1. `production` is executed for all pushed tags. + +We also use two secure variables: + +1. `HEROKU_STAGING_API_KEY` - Heroku API key used to deploy staging app, +1. `HEROKU_PRODUCTION_API_KEY` - Heroku API key used to deploy production app. + +## Storing API keys + +To add secure variables, navigate to your project's +**Settings > CI/CD > Variables**. The variables that are defined +in the project settings are sent along with the build script to the runner. +The secure variables are stored out of the repository. Never store secrets in +your project's `.gitlab-ci.yml`. It is also important that the secret's value +is hidden in the job log. + +You access added variable by prefixing it's name with `$` (on non-Windows runners) +or `%` (for Windows Batch runners): + +1. `$VARIABLE` - use it for non-Windows runners +1. `%VARIABLE%` - use it for Windows Batch runners + +Read more about the [CI/CD variables](../../variables/README.md). diff --git a/doc/ci/examples/index.md b/doc/ci/examples/index.md new file mode 100644 index 00000000000..9d6a4f53123 --- /dev/null +++ b/doc/ci/examples/index.md @@ -0,0 +1,182 @@ +--- +stage: Verify +group: Pipeline Execution +info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/engineering/ux/technical-writing/#assignments +comments: false +type: index +--- + +# GitLab CI/CD Examples + +This page contains links to a variety of examples that can help you understand how to +implement [GitLab CI/CD](../README.md) for your specific use case. + +Examples are available in several forms. As a collection of: + +- `.gitlab-ci.yml` [template files](#cicd-templates) maintained in GitLab, for many + common frameworks and programming languages. +- Repositories with [example projects](https://gitlab.com/gitlab-examples) for various languages. You can fork and adjust them to your own needs. Projects include an example of using [Review Apps with a static site served by NGINX](https://gitlab.com/gitlab-examples/review-apps-nginx/). +- Examples and [other resources](#other-resources) listed below. + +## CI/CD examples + +The following table lists examples with step-by-step tutorials that are contained in this section: + +| Use case | Resource | +|-------------------------------|----------| +| Browser performance testing | [Browser Performance Testing with the Sitespeed.io container](../../user/project/merge_requests/browser_performance_testing.md). | +| Deployment with Dpl | [Using `dpl` as deployment tool](deployment/index.md). | +| GitLab Pages | See the [GitLab Pages](../../user/project/pages/index.md) documentation for a complete example of deploying a static site. | +| End-to-end testing | [End-to-end testing with GitLab CI/CD and WebdriverIO](end_to_end_testing_webdriverio/index.md). | +| Load performance testing | [Load Performance Testing with the k6 container](../../user/project/merge_requests/load_performance_testing.md). | +| Multi project pipeline | [Build, test deploy using multi project pipeline](https://gitlab.com/gitlab-examples/upstream-project). | +| npm with semantic-release | [Publish npm packages to the GitLab Package Registry using semantic-release](semantic-release.md). | +| PHP with Laravel, Envoy | [Test and deploy Laravel applications with GitLab CI/CD and Envoy](laravel_with_gitlab_and_envoy/index.md). | +| PHP with npm, SCP | [Running Composer and npm scripts with deployment via SCP in GitLab CI/CD](deployment/composer-npm-deploy.md). | +| PHP with PHPunit, `atoum` | [Testing PHP projects](php.md). | +| Secrets management with Vault | [Authenticating and Reading Secrets With HashiCorp Vault](authenticating-with-hashicorp-vault/index.md). | + +### Contributed examples + +You can help people that use your favorite programming language by submitting a link +to a guide for that language. These contributed guides are hosted externally or in +separate example projects: + +| Use case | Resource | +|-------------------------------|----------| +| Clojure | [Test a Clojure application with GitLab CI/CD](https://gitlab.com/gitlab-examples/clojure-web-application). | +| Game development | [DevOps and Game Development with GitLab CI/CD](https://gitlab.com/gitlab-examples/gitlab-game-demo/). | +| Java with Maven | [How to deploy Maven projects to Artifactory with GitLab CI/CD](https://gitlab.com/gitlab-examples/maven/simple-maven-example). | +| Java with Spring Boot | [Deploy a Spring Boot application to Cloud Foundry with GitLab CI/CD](https://gitlab.com/gitlab-examples/spring-gitlab-cf-deploy-demo). | +| Parallel testing Ruby & JS | [GitLab CI/CD parallel jobs testing for Ruby & JavaScript projects](https://docs.knapsackpro.com/2019/how-to-run-parallel-jobs-for-rspec-tests-on-gitlab-ci-pipeline-and-speed-up-ruby-javascript-testing). | +| Python on Heroku | [Test and deploy a Python application with GitLab CI/CD](https://gitlab.com/gitlab-examples/python-getting-started). | +| Ruby on Heroku | [Test and deploy a Ruby application with GitLab CI/CD](https://gitlab.com/gitlab-examples/ruby-getting-started). | +| Scala on Heroku | [Test and deploy a Scala application to Heroku](https://gitlab.com/gitlab-examples/scala-sbt). | + +## CI/CD templates + +Get started with GitLab CI/CD and your favorite programming language or framework by using a +`.gitlab-ci.yml` [template](https://gitlab.com/gitlab-org/gitlab/-/tree/master/lib/gitlab/ci/templates). + +When you create a `gitlab-ci.yml` file in the UI, you can +choose one of these templates: + +- [Android (`Android.gitlab-ci.yml`)](https://gitlab.com/gitlab-org/gitlab/-/blob/master/lib/gitlab/ci/templates/Android.gitlab-ci.yml) +- [Android with fastlane (`Android-Fastlane.gitlab-ci.yml`)](https://gitlab.com/gitlab-org/gitlab/-/blob/master/lib/gitlab/ci/templates/Android-Fastlane.gitlab-ci.yml) +- [Bash (`Bash.gitlab-ci.yml`)](https://gitlab.com/gitlab-org/gitlab/-/blob/master/lib/gitlab/ci/templates/Bash.gitlab-ci.yml) +- [C++ (`C++.gitlab-ci.yml`)](https://gitlab.com/gitlab-org/gitlab/-/blob/master/lib/gitlab/ci/templates/C++.gitlab-ci.yml) +- [Chef (`Chef.gitlab-ci.yml`)](https://gitlab.com/gitlab-org/gitlab/-/blob/master/lib/gitlab/ci/templates/Chef.gitlab-ci.yml) +- [Clojure (`Clojure.gitlab-ci.yml`)](https://gitlab.com/gitlab-org/gitlab/-/blob/master/lib/gitlab/ci/templates/Clojure.gitlab-ci.yml) +- [Composer `Composer.gitlab-ci.yml`](https://gitlab.com/gitlab-org/gitlab/-/blob/master/lib/gitlab/ci/templates/Composer.gitlab-ci.yml) +- [Crystal (`Crystal.gitlab-ci.yml`)](https://gitlab.com/gitlab-org/gitlab/-/blob/master/lib/gitlab/ci/templates/Crystal.gitlab-ci.yml) +- [Dart (`Dart.gitlab-ci.yml`)](https://gitlab.com/gitlab-org/gitlab/-/blob/master/lib/gitlab/ci/templates/Dart.gitlab-ci.yml) +- [Django (`Django.gitlab-ci.yml`)](https://gitlab.com/gitlab-org/gitlab/-/blob/master/lib/gitlab/ci/templates/Django.gitlab-ci.yml) +- [Docker (`Docker.gitlab-ci.yml`)](https://gitlab.com/gitlab-org/gitlab/-/blob/master/lib/gitlab/ci/templates/Docker.gitlab-ci.yml) +- [dotNET (`dotNET.gitlab-ci.yml`)](https://gitlab.com/gitlab-org/gitlab/-/blob/master/lib/gitlab/ci/templates/dotNET.gitlab-ci.yml) +- [dotNET Core (`dotNET-Core.yml`)](https://gitlab.com/gitlab-org/gitlab/-/blob/master/lib/gitlab/ci/templates/dotNET-Core.yml) +- [Elixir (`Elixir.gitlab-ci.yml`)](https://gitlab.com/gitlab-org/gitlab/-/blob/master/lib/gitlab/ci/templates/Elixir.gitlab-ci.yml) +- [Flutter (`Flutter.gitlab-ci.yml`)](https://gitlab.com/gitlab-org/gitlab/-/blob/master/lib/gitlab/ci/templates/Flutter.gitlab-ci.yml) +- [goLang (`Go.gitlab-ci.yml`)](https://gitlab.com/gitlab-org/gitlab/-/blob/master/lib/gitlab/ci/templates/Go.gitlab-ci.yml) +- [Gradle (`Gradle.gitlab-ci.yml`)](https://gitlab.com/gitlab-org/gitlab/-/blob/master/lib/gitlab/ci/templates/Gradle.gitlab-ci.yml) +- [Grails (`Grails.gitlab-ci.yml`)](https://gitlab.com/gitlab-org/gitlab/-/blob/master/lib/gitlab/ci/templates/Grails.gitlab-ci.yml) +- [iOS with fastlane (`iOS-Fastlane.gitlab-ci.yml`)](https://gitlab.com/gitlab-org/gitlab/-/blob/master/lib/gitlab/ci/templates/iOS-Fastlane.gitlab-ci.yml) +- [Julia (`Julia.gitlab-ci.yml`)](https://gitlab.com/gitlab-org/gitlab/-/blob/master/lib/gitlab/ci/templates/Julia.gitlab-ci.yml) +- [Laravel (`Laravel.gitlab-ci.yml`)](https://gitlab.com/gitlab-org/gitlab/-/blob/master/lib/gitlab/ci/templates/Laravel.gitlab-ci.yml) +- [LaTeX (`LaTeX.gitlab-ci.yml`)](https://gitlab.com/gitlab-org/gitlab/-/blob/master/lib/gitlab/ci/templates/LaTeX.gitlab-ci.yml) +- [Maven (`Maven.gitlab-ci.yml`)](https://gitlab.com/gitlab-org/gitlab/-/blob/master/lib/gitlab/ci/templates/Maven.gitlab-ci.yml) +- [Mono (`Mono.gitlab-ci.yml`)](https://gitlab.com/gitlab-org/gitlab/-/blob/master/lib/gitlab/ci/templates/Mono.gitlab-ci.yml) +- [npm (`npm.gitlab-ci.yml`)](https://gitlab.com/gitlab-org/gitlab/-/blob/master/lib/gitlab/ci/templates/npm.gitlab-ci.yml) +- [Node.js (`Nodejs.gitlab-ci.yml`)](https://gitlab.com/gitlab-org/gitlab/-/blob/master/lib/gitlab/ci/templates/Nodejs.gitlab-ci.yml) +- [OpenShift (`OpenShift.gitlab-ci.yml`)](https://gitlab.com/gitlab-org/gitlab/-/blob/master/lib/gitlab/ci/templates/OpenShift.gitlab-ci.yml) +- [Packer (`Packer.gitlab-ci.yml`)](https://gitlab.com/gitlab-org/gitlab/-/blob/master/lib/gitlab/ci/templates/Packer.gitlab-ci.yml) +- [PHP (`PHP.gitlab-ci.yml`)](https://gitlab.com/gitlab-org/gitlab/-/blob/master/lib/gitlab/ci/templates/PHP.gitlab-ci.yml) +- [Python (`Python.gitlab-ci.yml`)](https://gitlab.com/gitlab-org/gitlab/-/blob/master/lib/gitlab/ci/templates/Python.gitlab-ci.yml) +- [Ruby (`Ruby.gitlab-ci.yml`)](https://gitlab.com/gitlab-org/gitlab/-/blob/master/lib/gitlab/ci/templates/Ruby.gitlab-ci.yml) +- [Rust (`Rust.gitlab-ci.yml`)](https://gitlab.com/gitlab-org/gitlab/-/blob/master/lib/gitlab/ci/templates/Rust.gitlab-ci.yml) +- [Scala (`Scala.gitlab-ci.yml`)](https://gitlab.com/gitlab-org/gitlab/-/blob/master/lib/gitlab/ci/templates/Scala.gitlab-ci.yml) +- [Swift (`Swift.gitlab-ci.yml`)](https://gitlab.com/gitlab-org/gitlab/-/blob/master/lib/gitlab/ci/templates/Swift.gitlab-ci.yml) +- [Terraform (`Terraform.latest.gitlab-ci.yml`)](https://gitlab.com/gitlab-org/gitlab/-/blob/master/lib/gitlab/ci/templates/Terraform.latest.gitlab-ci.yml) + +If a programming language or framework template is not in this list, you can contribute +one. To create a template, submit a merge request +to [the templates list](https://gitlab.com/gitlab-org/gitlab/-/tree/master/lib/gitlab/ci/templates). + +### Adding templates to your GitLab installation **(PREMIUM SELF)** + +You can add custom examples and templates to your self-managed GitLab instance. +Your GitLab administrator can [designate an instance template repository](../../user/admin_area/settings/instance_template_repository.md) +that contains examples and templates specific to your organization. + +## Other resources + +This section provides further resources to help you get familiar with various uses of GitLab CI/CD. +Note that older articles and videos may not reflect the state of the latest GitLab release. + +### CI/CD in the cloud + +For examples of setting up GitLab CI/CD for cloud-based environments, see: + +- [How to set up multi-account AWS SAM deployments with GitLab CI](https://about.gitlab.com/blog/2019/02/04/multi-account-aws-sam-deployments-with-gitlab-ci/) +- [Automating Kubernetes Deployments with GitLab CI/CD](https://www.youtube.com/watch?v=wEDRfAz6_Uw) +- [How to autoscale continuous deployment with GitLab Runner on DigitalOcean](https://about.gitlab.com/blog/2018/06/19/autoscale-continuous-deployment-gitlab-runner-digital-ocean/) +- [How to create a CI/CD pipeline with Auto Deploy to Kubernetes using GitLab and Helm](https://about.gitlab.com/blog/2017/09/21/how-to-create-ci-cd-pipeline-with-autodeploy-to-kubernetes-using-gitlab-and-helm/) +- [Demo - Deploying from GitLab to OpenShift Container Cluster](https://youtu.be/EwbhA53Jpp4) + +See also the following video overviews: + +- [Kubernetes, GitLab, and Cloud Native](https://www.youtube.com/watch?v=d-9awBxEbvQ). +- [Deploying to IBM Cloud with GitLab CI/CD](https://www.youtube.com/watch?v=6ZF4vgKMd-g). + +### Customer stories + +For some customer experiences with GitLab CI/CD, see: + +- [How Verizon Connect reduced data center deploys from 30 days to under 8 hours with GitLab](https://about.gitlab.com/blog/2019/02/14/verizon-customer-story/) +- [How Wag! cut their release process from 40 minutes to just 6](https://about.gitlab.com/blog/2019/01/16/wag-labs-blog-post/) +- [How Jaguar Land Rover embraced CI to speed up their software lifecycle](https://about.gitlab.com/blog/2018/07/23/chris-hill-devops-enterprise-summit-talk/) + +### Getting started + +For some examples to help get you started, see: + +- [GitLab CI/CD's 2018 highlights](https://about.gitlab.com/blog/2019/01/21/gitlab-ci-cd-features-improvements/) +- [A beginner's guide to continuous integration](https://about.gitlab.com/blog/2018/01/22/a-beginners-guide-to-continuous-integration/) + +### Implementing GitLab CI/CD + +For examples of others who have implemented GitLab CI/CD, see: + +- [How to streamline interactions between multiple repositories with multi-project pipelines](https://about.gitlab.com/blog/2018/10/31/use-multiproject-pipelines-with-gitlab-cicd/) +- [How we used GitLab CI to build GitLab faster](https://about.gitlab.com/blog/2018/05/02/using-gitlab-ci-to-build-gitlab-faster/) +- [Test all the things in GitLab CI with Docker by example](https://about.gitlab.com/blog/2018/02/05/test-all-the-things-gitlab-ci-docker-examples/) +- [A Craftsman looks at continuous integration](https://about.gitlab.com/blog/2018/01/17/craftsman-looks-at-continuous-integration/) +- [Go tools and GitLab: How to do continuous integration like a boss](https://about.gitlab.com/blog/2017/11/27/go-tools-and-gitlab-how-to-do-continuous-integration-like-a-boss/) +- [GitBot – automating boring Git operations with CI](https://about.gitlab.com/blog/2017/11/02/automating-boring-git-operations-gitlab-ci/) +- [How to use GitLab CI for Vue.js](https://about.gitlab.com/blog/2017/09/12/vuejs-app-gitlab/) +- Video: [GitLab CI/CD Deep Dive](https://youtu.be/pBe4t1CD8Fc?t=195) +- [Dockerizing GitLab Review Apps](https://about.gitlab.com/blog/2017/07/11/dockerizing-review-apps/) +- [Fast and natural continuous integration with GitLab CI](https://about.gitlab.com/blog/2017/05/22/fast-and-natural-continuous-integration-with-gitlab-ci/) +- [Demo: CI/CD with GitLab in action](https://about.gitlab.com/blog/2017/03/13/ci-cd-demo/) + +### Migrating to GitLab from third-party CI tools + +- [Migrating from Jenkins to GitLab](https://youtu.be/RlEVGOpYF5Y) + +### Integrating GitLab CI/CD with other systems + +To see how you can integrate GitLab CI/CD with third-party systems, see: + +- [Streamline and shorten error remediation with Sentry's new GitLab integration](https://about.gitlab.com/blog/2019/01/25/sentry-integration-blog-post/) +- [How to simplify your smart home configuration with GitLab CI/CD](https://about.gitlab.com/blog/2018/08/02/using-the-gitlab-ci-slash-cd-for-smart-home-configuration-management/) +- [Demo: GitLab + Jira + Jenkins](https://about.gitlab.com/blog/2018/07/30/gitlab-workflow-with-jira-jenkins/) +- [Introducing Auto Breakfast from GitLab (sort of)](https://about.gitlab.com/blog/2018/06/29/introducing-auto-breakfast-from-gitlab/) + +### Mobile development + +For help with using GitLab CI/CD for mobile application development, see: + +- [How to publish Android apps to the Google Play Store with GitLab and fastlane](https://about.gitlab.com/blog/2019/01/28/android-publishing-with-gitlab-and-fastlane/) +- [Setting up GitLab CI for Android projects](https://about.gitlab.com/blog/2018/10/24/setting-up-gitlab-ci-for-android-projects/) +- [Working with YAML in GitLab CI from the Android perspective](https://about.gitlab.com/blog/2017/11/20/working-with-yaml-gitlab-ci-android/) +- [How to use GitLab CI and MacStadium to build your macOS or iOS projects](https://about.gitlab.com/blog/2017/05/15/how-to-use-macstadium-and-gitlab-ci-to-build-your-macos-or-ios-projects/) +- [Setting up GitLab CI for iOS projects](https://about.gitlab.com/blog/2016/03/10/setting-up-gitlab-ci-for-ios-projects/) diff --git a/doc/ci/examples/laravel_with_gitlab_and_envoy/index.md b/doc/ci/examples/laravel_with_gitlab_and_envoy/index.md index 7080ce82b2c..2e019e35f78 100644 --- a/doc/ci/examples/laravel_with_gitlab_and_envoy/index.md +++ b/doc/ci/examples/laravel_with_gitlab_and_envoy/index.md @@ -106,7 +106,7 @@ sudo apt install acl ### Add SSH key -Let's suppose we want to deploy our app to the production server from a private repository on GitLab. First, we need to [generate a new SSH key pair **with no passphrase**](../../../ssh/README.md) for the deployer user. +Let's suppose we want to deploy our app to the production server from a private repository on GitLab. First, we need to [generate a new SSH key pair **with no passphrase**](../../../ssh/index.md) for the deployer user. After that, we need to copy the private key, which will be used to connect to our server as the deployer user with SSH, to be able to automate our deployment process: @@ -530,7 +530,7 @@ That's a lot to take in, isn't it? Let's run through it step by step. #### Image and Services -[Runners](../../runners/README.md) run the script defined by `.gitlab-ci.yml`. +[Runners](../../runners/index.md) run the script defined by `.gitlab-ci.yml`. The `image` keyword tells the runners which image to use. The `services` keyword defines additional images [that are linked to the main image](../../services/index.md). Here we use the container image we created before as our main image and also use MySQL 5.7 as a service. diff --git a/doc/ci/examples/php.md b/doc/ci/examples/php.md index 9e0ff992d40..8184879a738 100644 --- a/doc/ci/examples/php.md +++ b/doc/ci/examples/php.md @@ -270,7 +270,7 @@ gitlab-runner exec shell test:app We have set up an [Example PHP Project](https://gitlab.com/gitlab-examples/php) for your convenience that runs on [GitLab.com](https://gitlab.com) using our publicly available -[shared runners](../runners/README.md). +[shared runners](../runners/index.md). Want to hack on it? Simply fork it, commit, and push your changes. Within a few moments the changes are picked by a public runner and the job begins. diff --git a/doc/ci/interactive_web_terminal/index.md b/doc/ci/interactive_web_terminal/index.md index cbf92438488..46abcf4a21d 100644 --- a/doc/ci/interactive_web_terminal/index.md +++ b/doc/ci/interactive_web_terminal/index.md @@ -16,7 +16,7 @@ is deployed, some [security precautions](../../administration/integration/termin taken to protect the users. NOTE: -[Shared runners on GitLab.com](../runners/README.md) do not +[Shared runners on GitLab.com](../runners/index.md) do not provide an interactive web terminal. Follow [this issue](https://gitlab.com/gitlab-org/gitlab/-/issues/24674) for progress on adding support. For groups and projects hosted on GitLab.com, interactive web diff --git a/doc/ci/jobs/index.md b/doc/ci/jobs/index.md index f359eecf01b..09c5463d165 100644 --- a/doc/ci/jobs/index.md +++ b/doc/ci/jobs/index.md @@ -29,7 +29,7 @@ jobs, where each of the jobs executes a different command. Of course a command can execute code directly (`./configure;make;make install`) or run a script (`test.sh`) in the repository. -Jobs are picked up by [runners](../runners/README.md) and executed in the +Jobs are picked up by [runners](../runners/index.md) and executed in the environment of the runner. What is important is that each job is run independently from each other. diff --git a/doc/ci/jobs/job_control.md b/doc/ci/jobs/job_control.md index 27eda577b46..1ca1bce979d 100644 --- a/doc/ci/jobs/job_control.md +++ b/doc/ci/jobs/job_control.md @@ -225,7 +225,7 @@ check the value of the `$CI_PIPELINE_SOURCE` variable: | `pipeline` | For [multi-project pipelines](../multi_project_pipelines.md) created by [using the API with `CI_JOB_TOKEN`](../multi_project_pipelines.md#create-multi-project-pipelines-by-using-the-api), or the [`trigger`](../yaml/README.md#trigger) keyword. | | `push` | For pipelines triggered by a `git push` event, including for branches and tags. | | `schedule` | For [scheduled pipelines](../pipelines/schedules.md). | -| `trigger` | For pipelines created by using a [trigger token](../triggers/README.md#trigger-token). | +| `trigger` | For pipelines created by using a [trigger token](../triggers/index.md#trigger-token). | | `web` | For pipelines created by using **Run pipeline** button in the GitLab UI, from the project's **CI/CD > Pipelines** section. | | `webide` | For pipelines created by using the [WebIDE](../../user/project/web_ide/index.md). | @@ -306,7 +306,7 @@ to control when to add jobs to pipelines. In the following example, `job` runs only for: - Git tags -- [Triggers](../triggers/README.md#trigger-token) +- [Triggers](../triggers/index.md#trigger-token) - [Scheduled pipelines](../pipelines/schedules.md) ```yaml diff --git a/doc/ci/migration/jenkins.md b/doc/ci/migration/jenkins.md index 812f1caa5d1..328f4b38aad 100644 --- a/doc/ci/migration/jenkins.md +++ b/doc/ci/migration/jenkins.md @@ -17,7 +17,7 @@ that were able to quickly complete this migration: 1. Start by reading the GitLab CI/CD [Quick Start Guide](../quick_start/index.md) and [important product differences](#important-product-differences). 1. Learn the importance of [managing the organizational transition](#managing-the-organizational-transition). -1. [Add runners](../runners/README.md) to your GitLab instance. +1. [Add runners](../runners/index.md) to your GitLab instance. 1. Educate and enable your developers to independently perform the following steps in their projects: 1. Review the [Quick Start Guide](../quick_start/index.md) and [Pipeline Configuration Reference](../yaml/README.md). 1. Use the [Jenkins Wrapper](#jenkinsfile-wrapper) to temporarily maintain fragile Jenkins jobs. @@ -77,8 +77,8 @@ There are some high level differences between the products worth mentioning: - on push - on [schedule](../pipelines/schedules.md) - from the [GitLab UI](../pipelines/index.md#run-a-pipeline-manually) - - by [API call](../triggers/README.md) - - by [webhook](../triggers/README.md#triggering-a-pipeline-from-a-webhook) + - by [API call](../triggers/index.md) + - by [webhook](../triggers/index.md#triggering-a-pipeline-from-a-webhook) - by [ChatOps](../chatops/index.md) - You can control which jobs run in which cases, depending on how they are triggered, @@ -122,7 +122,7 @@ There are some high level differences between the products worth mentioning: ## Agents vs. runners Both Jenkins agents and GitLab runners are the hosts that run jobs. To convert the -Jenkins agent, simply uninstall it and then [install and register the runner](../runners/README.md). +Jenkins agent, simply uninstall it and then [install and register the runner](../runners/index.md). Runners do not require much overhead, so you can size them similarly to the Jenkins agents you were using. @@ -137,7 +137,7 @@ There are some important differences in the way runners work in comparison to ag Use autoscaling to provision runners only when needed, and scale down when not needed. This is similar to ephemeral agents in Jenkins. -If you are using `gitlab.com`, you can take advantage of our [shared runner fleet](../runners/README.md) +If you are using `gitlab.com`, you can take advantage of our [shared runner fleet](../runners/index.md) to run jobs without provisioning your own runners. We are investigating making them [available for self-managed instances](https://gitlab.com/groups/gitlab-org/-/epics/835) as well. @@ -227,7 +227,7 @@ and is meant to be a mapping of concepts there to concepts in GitLab. #### `agent` -The agent section is used to define how a pipeline executes. For GitLab, we use [runners](../runners/README.md) +The agent section is used to define how a pipeline executes. For GitLab, we use [runners](../runners/index.md) to provide this capability. You can configure your own runners in Kubernetes or on any host, or take advantage of our shared runner fleet (note that the shared runner fleet is only available for GitLab.com users). We also support using [tags](../runners/configure_runners.md#use-tags-to-limit-the-number-of-jobs-using-the-runner) to direct different jobs diff --git a/doc/ci/multi_project_pipelines.md b/doc/ci/multi_project_pipelines.md index abdf01521db..ac800e1baee 100644 --- a/doc/ci/multi_project_pipelines.md +++ b/doc/ci/multi_project_pipelines.md @@ -258,7 +258,7 @@ upstream_bridge: > [Moved](https://gitlab.com/gitlab-org/gitlab/-/issues/31573) to GitLab Free in 12.4. -When you use the [`CI_JOB_TOKEN` to trigger pipelines](triggers/README.md#ci-job-token), +When you use the [`CI_JOB_TOKEN` to trigger pipelines](triggers/index.md#ci-job-token), GitLab recognizes the source of the job token. The pipelines become related, so you can visualize their relationships on pipeline graphs. diff --git a/doc/ci/pipelines/index.md b/doc/ci/pipelines/index.md index f499e4077f4..f9a58c04fa2 100644 --- a/doc/ci/pipelines/index.md +++ b/doc/ci/pipelines/index.md @@ -20,7 +20,7 @@ Pipelines comprise: - Jobs, which define *what* to do. For example, jobs that compile or test code. - Stages, which define *when* to run the jobs. For example, stages that run tests after stages that compile the code. -Jobs are executed by [runners](../runners/README.md). Multiple jobs in the same stage are executed in parallel, +Jobs are executed by [runners](../runners/index.md). Multiple jobs in the same stage are executed in parallel, if there are enough concurrent runners. If *all* jobs in a stage succeed, the pipeline moves on to the next stage. @@ -441,5 +441,5 @@ GitLab provides API endpoints to: - Perform basic functions. For more information, see [Pipelines API](../../api/pipelines.md). - Maintain pipeline schedules. For more information, see [Pipeline schedules API](../../api/pipeline_schedules.md). - Trigger pipeline runs. For more information, see: - - [Triggering pipelines through the API](../triggers/README.md). + - [Triggering pipelines through the API](../triggers/index.md). - [Pipeline triggers API](../../api/pipeline_triggers.md). diff --git a/doc/ci/pipelines/pipeline_efficiency.md b/doc/ci/pipelines/pipeline_efficiency.md index 5bb435dddf6..f0b0b8cf3d5 100644 --- a/doc/ci/pipelines/pipeline_efficiency.md +++ b/doc/ci/pipelines/pipeline_efficiency.md @@ -33,7 +33,7 @@ heavily influenced by the: - The ["critical path"](#directed-acyclic-graphs-dag-visualization), which represents the minimum and maximum pipeline duration. -Additional points to pay attention relate to [GitLab Runners](../runners/README.md): +Additional points to pay attention relate to [GitLab Runners](../runners/index.md): - Availability of the runners and the resources they are provisioned with. - Build dependencies and their installation time. @@ -62,7 +62,7 @@ It's important to understand and document the pipeline workflows, and discuss po actions and changes. Refactoring pipelines may need careful interaction between teams in the DevSecOps lifecycle. -Pipeline analysis can help identify issues with cost efficiency. For example, [runners](../runners/README.md) +Pipeline analysis can help identify issues with cost efficiency. For example, [runners](../runners/index.md) hosted with a paid cloud service may be provisioned with: - More resources than needed for CI/CD pipelines, wasting money. diff --git a/doc/ci/review_apps/index.md b/doc/ci/review_apps/index.md index a64efd50f6f..d23118f7c38 100644 --- a/doc/ci/review_apps/index.md +++ b/doc/ci/review_apps/index.md @@ -22,7 +22,7 @@ Review Apps: - Provide an automatic live preview of changes made in a feature branch by spinning up a dynamic environment for your merge requests. - Allow designers and product managers to see your changes without needing to check out your branch and run your changes in a sandbox environment. -- Are fully integrated with the [GitLab DevOps LifeCycle](../../README.md#the-entire-devops-lifecycle). +- Are fully integrated with the [GitLab DevOps LifeCycle](../../index.md#the-entire-devops-lifecycle). - Allow you to deploy your changes wherever you want.  diff --git a/doc/ci/runners/README.md b/doc/ci/runners/README.md index 45113c480c7..5ab8653dc35 100644 --- a/doc/ci/runners/README.md +++ b/doc/ci/runners/README.md @@ -1,289 +1,8 @@ --- -stage: Verify -group: Runner -info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/engineering/ux/technical-writing/#assignments -type: reference +redirect_to: 'index.md' --- -# GitLab SaaS runners +This document was moved to [another location](index.md). -If you are using self-managed GitLab or you want to use your own runners on GitLab.com, you can -[install and configure your own runners](https://docs.gitlab.com/runner/install/). - -If you are using GitLab SaaS (GitLab.com), your CI jobs automatically run on shared runners. No configuration is required. -Your jobs can run on [Linux](#linux-shared-runners) or [Windows](#windows-shared-runners-beta). - -The number of minutes you can use on these shared runners depends on your -[quota](../../user/admin_area/settings/continuous_integration.md#shared-runners-pipeline-minutes-quota), -which depends on your [subscription plan](../../subscriptions/gitlab_com/index.md#ci-pipeline-minutes). - -## Linux shared runners - -Linux shared runners on GitLab.com run in autoscale mode and are powered by Google Cloud Platform. - -Autoscaling means reduced queue times to spin up CI/CD jobs, and isolated VMs for each job, thus maximizing security. These shared runners are available for users and customers on GitLab.com. - -GitLab offers Ultimate tier capabilities and included CI/CD minutes per group per month for our [Open Source](https://about.gitlab.com/solutions/open-source/join/), [Education](https://about.gitlab.com/solutions/education/), and [Startups](https://about.gitlab.com/solutions/startups/) programs. For private projects, GitLab offers various [plans](https://about.gitlab.com/pricing/), starting with a Free tier. - -All your CI/CD jobs run on [n1-standard-1 instances](https://cloud.google.com/compute/docs/machine-types) with 3.75GB of RAM, CoreOS and the latest Docker Engine -installed. Instances provide 1 vCPU and 25GB of HDD disk space. The default -region of the VMs is US East1. -Each instance is used only for one job, this ensures any sensitive data left on the system can't be accessed by other people their CI jobs. - -The `gitlab-shared-runners-manager-X.gitlab.com` fleet of runners are dedicated for GitLab projects as well as community forks of them. They use a slightly larger machine type (n1-standard-2) and have a bigger SSD disk size. They don't run untagged jobs and unlike the general fleet of shared runners, the instances are re-used up to 40 times. - -Jobs handled by the shared runners on GitLab.com (`shared-runners-manager-X.gitlab.com`), -**time out after 3 hours**, regardless of the timeout configured in a -project. Check the issues [4010](https://gitlab.com/gitlab-com/infrastructure/-/issues/4010) and [4070](https://gitlab.com/gitlab-com/infrastructure/-/issues/4070) for the reference. - -Below are the shared runners settings. - -| Setting | GitLab.com | Default | -| ----------- | ----------------- | ---------- | -| [GitLab Runner](https://gitlab.com/gitlab-org/gitlab-runner) | [Runner versions dashboard](https://dashboards.gitlab.com/d/000000159/ci?from=now-1h&to=now&refresh=5m&orgId=1&panelId=12&fullscreen&theme=light) | - | -| Executor | `docker+machine` | - | -| Default Docker image | `ruby:2.5` | - | -| `privileged` (run [Docker in Docker](https://hub.docker.com/_/docker/)) | `true` | `false` | - -### Pre-clone script - -Linux shared runners on GitLab.com provide a way to run commands in a CI -job before the runner attempts to run `git init` and `git fetch` to -download a GitLab repository. The -[`pre_clone_script`](https://docs.gitlab.com/runner/configuration/advanced-configuration.html#the-runners-section) -can be used for: - -- Seeding the build directory with repository data -- Sending a request to a server -- Downloading assets from a CDN -- Any other commands that must run before the `git init` - -To use this feature, define a [CI/CD variable](../../ci/variables/README.md#custom-cicd-variables) called -`CI_PRE_CLONE_SCRIPT` that contains a bash script. - -[This example](../../development/pipelines.md#pre-clone-step) -demonstrates how you might use a pre-clone step to seed the build -directory. - -### `config.toml` - -The full contents of our `config.toml` are: - -NOTE: -Settings that are not public are shown as `X`. - -**Google Cloud Platform** - -```toml -concurrent = X -check_interval = 1 -metrics_server = "X" -sentry_dsn = "X" - -[[runners]] - name = "docker-auto-scale" - request_concurrency = X - url = "https://gitlab.com/" - token = "SHARED_RUNNER_TOKEN" - pre_clone_script = "eval \"$CI_PRE_CLONE_SCRIPT\"" - executor = "docker+machine" - environment = [ - "DOCKER_DRIVER=overlay2", - "DOCKER_TLS_CERTDIR=" - ] - limit = X - [runners.docker] - image = "ruby:2.5" - privileged = true - volumes = [ - "/certs/client", - "/dummy-sys-class-dmi-id:/sys/class/dmi/id:ro" # Make kaniko builds work on GCP. - ] - [runners.machine] - IdleCount = 50 - IdleTime = 3600 - MaxBuilds = 1 # For security reasons we delete the VM after job has finished so it's not reused. - MachineName = "srm-%s" - MachineDriver = "google" - MachineOptions = [ - "google-project=PROJECT", - "google-disk-size=25", - "google-machine-type=n1-standard-1", - "google-username=core", - "google-tags=gitlab-com,srm", - "google-use-internal-ip", - "google-zone=us-east1-d", - "engine-opt=mtu=1460", # Set MTU for container interface, for more information check https://gitlab.com/gitlab-org/gitlab-runner/-/issues/3214#note_82892928 - "google-machine-image=PROJECT/global/images/IMAGE", - "engine-opt=ipv6", # This will create IPv6 interfaces in the containers. - "engine-opt=fixed-cidr-v6=fc00::/7", - "google-operation-backoff-initial-interval=2" # Custom flag from forked docker-machine, for more information check https://github.com/docker/machine/pull/4600 - ] - [[runners.machine.autoscaling]] - Periods = ["* * * * * sat,sun *"] - Timezone = "UTC" - IdleCount = 70 - IdleTime = 3600 - [[runners.machine.autoscaling]] - Periods = ["* 30-59 3 * * * *", "* 0-30 4 * * * *"] - Timezone = "UTC" - IdleCount = 700 - IdleTime = 3600 - [runners.cache] - Type = "gcs" - Shared = true - [runners.cache.gcs] - CredentialsFile = "/path/to/file" - BucketName = "bucket-name" -``` - -## Windows shared runners (beta) - -The Windows shared runners are in [beta](https://about.gitlab.com/handbook/product/gitlab-the-product/#beta) -and shouldn't be used for production workloads. - -During this beta period, the [shared runner pipeline quota](../../user/admin_area/settings/continuous_integration.md#shared-runners-pipeline-minutes-quota) -applies for groups and projects in the same manner as Linux runners. This may -change when the beta period ends, as discussed in this [related issue](https://gitlab.com/gitlab-org/gitlab/-/issues/30834). - -Windows shared runners on GitLab.com autoscale by launching virtual machines on -the Google Cloud Platform. This solution uses an -[autoscaling driver](https://gitlab.com/gitlab-org/ci-cd/custom-executor-drivers/autoscaler/tree/master/docs/readme.md) -developed by GitLab for the [custom executor](https://docs.gitlab.com/runner/executors/custom.html). -Windows shared runners execute your CI/CD jobs on `n1-standard-2` instances with -2 vCPUs and 7.5 GB RAM. You can find a full list of available Windows packages in -the [package documentation](https://gitlab.com/gitlab-org/ci-cd/shared-runners/images/gcp/windows-containers/blob/master/cookbooks/preinstalled-software/README.md). - -We want to keep iterating to get Windows shared runners in a stable state and -[generally available](https://about.gitlab.com/handbook/product/gitlab-the-product/#generally-available-ga). -You can follow our work towards this goal in the -[related epic](https://gitlab.com/groups/gitlab-org/-/epics/2162). - -### Configuration - -The full contents of our `config.toml` are: - -NOTE: -Settings that aren't public are shown as `X`. - -```toml -concurrent = X -check_interval = 3 - -[[runners]] - name = "windows-runner" - url = "https://gitlab.com/" - token = "TOKEN" - executor = "custom" - builds_dir = "C:\\GitLab-Runner\\builds" - cache_dir = "C:\\GitLab-Runner\\cache" - shell = "powershell" - [runners.custom] - config_exec = "C:\\GitLab-Runner\\autoscaler\\autoscaler.exe" - config_args = ["--config", "C:\\GitLab-Runner\\autoscaler\\config.toml", "custom", "config"] - prepare_exec = "C:\\GitLab-Runner\\autoscaler\\autoscaler.exe" - prepare_args = ["--config", "C:\\GitLab-Runner\\autoscaler\\config.toml", "custom", "prepare"] - run_exec = "C:\\GitLab-Runner\\autoscaler\\autoscaler.exe" - run_args = ["--config", "C:\\GitLab-Runner\\autoscaler\\config.toml", "custom", "run"] - cleanup_exec = "C:\\GitLab-Runner\\autoscaler\\autoscaler.exe" - cleanup_args = ["--config", "C:\\GitLab-Runner\\autoscaler\\config.toml", "custom", "cleanup"] -``` - -The full contents of our `autoscaler/config.toml` are: - -```toml -Provider = "gcp" -Executor = "winrm" -OS = "windows" -LogLevel = "info" -LogFormat = "text" -LogFile = "C:\\GitLab-Runner\\autoscaler\\autoscaler.log" -VMTag = "windows" - -[GCP] - ServiceAccountFile = "PATH" - Project = "some-project-df9323" - Zone = "us-east1-c" - MachineType = "n1-standard-2" - Image = "IMAGE" - DiskSize = 50 - DiskType = "pd-standard" - Subnetwork = "default" - Network = "default" - Tags = ["TAGS"] - Username = "gitlab_runner" - -[WinRM] - MaximumTimeout = 3600 - ExecutionMaxRetries = 0 - -[ProviderCache] - Enabled = true - Directory = "C:\\GitLab-Runner\\autoscaler\\machines" -``` - -### Example - -Below is a simple `.gitlab-ci.yml` file to show how to start using the -Windows shared runners: - -```yaml -.shared_windows_runners: - tags: - - shared-windows - - windows - - windows-1809 - -stages: - - build - - test - -before_script: - - Set-Variable -Name "time" -Value (date -Format "%H:%m") - - echo ${time} - - echo "started by ${GITLAB_USER_NAME}" - -build: - extends: - - .shared_windows_runners - stage: build - script: - - echo "running scripts in the build job" - -test: - extends: - - .shared_windows_runners - stage: test - script: - - echo "running scripts in the test job" -``` - -### Limitations and known issues - -- All the limitations mentioned in our [beta - definition](https://about.gitlab.com/handbook/product/#beta). -- The average provisioning time for a new Windows VM is 5 minutes. - This means that you may notice slower build start times - on the Windows shared runner fleet during the beta. In a future - release we intend to update the autoscaler to enable - the pre-provisioning of virtual machines. This is intended to significantly reduce - the time it takes to provision a VM on the Windows fleet. You can - follow along in the [related issue](https://gitlab.com/gitlab-org/ci-cd/custom-executor-drivers/autoscaler/-/issues/32). -- The Windows shared runner fleet may be unavailable occasionally - for maintenance or updates. -- The Windows shared runner virtual machine instances do not use the - GitLab Docker executor. This means that you can't specify - [`image`](../../ci/yaml/README.md#image) or [`services`](../../ci/yaml/README.md#services) in - your pipeline configuration. -- For the beta release, we have included a set of software packages in - the base VM image. If your CI job requires additional software that's - not included in this list, then you must add installation - commands to [`before_script`](../../ci/yaml/README.md#before_script) or [`script`](../../ci/yaml/README.md#script) to install the required - software. Note that each job runs on a new VM instance, so the - installation of additional software packages needs to be repeated for - each job in your pipeline. -- The job may stay in a pending state for longer than the - Linux shared runners. -- There is the possibility that we introduce breaking changes which will - require updates to pipelines that are using the Windows shared runner - fleet. +<!-- This redirect file can be deleted after 2021-09-28. --> +<!-- Before deletion, see: https://docs.gitlab.com/ee/development/documentation/#move-or-rename-a-page --> diff --git a/doc/ci/runners/index.md b/doc/ci/runners/index.md new file mode 100644 index 00000000000..45113c480c7 --- /dev/null +++ b/doc/ci/runners/index.md @@ -0,0 +1,289 @@ +--- +stage: Verify +group: Runner +info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/engineering/ux/technical-writing/#assignments +type: reference +--- + +# GitLab SaaS runners + +If you are using self-managed GitLab or you want to use your own runners on GitLab.com, you can +[install and configure your own runners](https://docs.gitlab.com/runner/install/). + +If you are using GitLab SaaS (GitLab.com), your CI jobs automatically run on shared runners. No configuration is required. +Your jobs can run on [Linux](#linux-shared-runners) or [Windows](#windows-shared-runners-beta). + +The number of minutes you can use on these shared runners depends on your +[quota](../../user/admin_area/settings/continuous_integration.md#shared-runners-pipeline-minutes-quota), +which depends on your [subscription plan](../../subscriptions/gitlab_com/index.md#ci-pipeline-minutes). + +## Linux shared runners + +Linux shared runners on GitLab.com run in autoscale mode and are powered by Google Cloud Platform. + +Autoscaling means reduced queue times to spin up CI/CD jobs, and isolated VMs for each job, thus maximizing security. These shared runners are available for users and customers on GitLab.com. + +GitLab offers Ultimate tier capabilities and included CI/CD minutes per group per month for our [Open Source](https://about.gitlab.com/solutions/open-source/join/), [Education](https://about.gitlab.com/solutions/education/), and [Startups](https://about.gitlab.com/solutions/startups/) programs. For private projects, GitLab offers various [plans](https://about.gitlab.com/pricing/), starting with a Free tier. + +All your CI/CD jobs run on [n1-standard-1 instances](https://cloud.google.com/compute/docs/machine-types) with 3.75GB of RAM, CoreOS and the latest Docker Engine +installed. Instances provide 1 vCPU and 25GB of HDD disk space. The default +region of the VMs is US East1. +Each instance is used only for one job, this ensures any sensitive data left on the system can't be accessed by other people their CI jobs. + +The `gitlab-shared-runners-manager-X.gitlab.com` fleet of runners are dedicated for GitLab projects as well as community forks of them. They use a slightly larger machine type (n1-standard-2) and have a bigger SSD disk size. They don't run untagged jobs and unlike the general fleet of shared runners, the instances are re-used up to 40 times. + +Jobs handled by the shared runners on GitLab.com (`shared-runners-manager-X.gitlab.com`), +**time out after 3 hours**, regardless of the timeout configured in a +project. Check the issues [4010](https://gitlab.com/gitlab-com/infrastructure/-/issues/4010) and [4070](https://gitlab.com/gitlab-com/infrastructure/-/issues/4070) for the reference. + +Below are the shared runners settings. + +| Setting | GitLab.com | Default | +| ----------- | ----------------- | ---------- | +| [GitLab Runner](https://gitlab.com/gitlab-org/gitlab-runner) | [Runner versions dashboard](https://dashboards.gitlab.com/d/000000159/ci?from=now-1h&to=now&refresh=5m&orgId=1&panelId=12&fullscreen&theme=light) | - | +| Executor | `docker+machine` | - | +| Default Docker image | `ruby:2.5` | - | +| `privileged` (run [Docker in Docker](https://hub.docker.com/_/docker/)) | `true` | `false` | + +### Pre-clone script + +Linux shared runners on GitLab.com provide a way to run commands in a CI +job before the runner attempts to run `git init` and `git fetch` to +download a GitLab repository. The +[`pre_clone_script`](https://docs.gitlab.com/runner/configuration/advanced-configuration.html#the-runners-section) +can be used for: + +- Seeding the build directory with repository data +- Sending a request to a server +- Downloading assets from a CDN +- Any other commands that must run before the `git init` + +To use this feature, define a [CI/CD variable](../../ci/variables/README.md#custom-cicd-variables) called +`CI_PRE_CLONE_SCRIPT` that contains a bash script. + +[This example](../../development/pipelines.md#pre-clone-step) +demonstrates how you might use a pre-clone step to seed the build +directory. + +### `config.toml` + +The full contents of our `config.toml` are: + +NOTE: +Settings that are not public are shown as `X`. + +**Google Cloud Platform** + +```toml +concurrent = X +check_interval = 1 +metrics_server = "X" +sentry_dsn = "X" + +[[runners]] + name = "docker-auto-scale" + request_concurrency = X + url = "https://gitlab.com/" + token = "SHARED_RUNNER_TOKEN" + pre_clone_script = "eval \"$CI_PRE_CLONE_SCRIPT\"" + executor = "docker+machine" + environment = [ + "DOCKER_DRIVER=overlay2", + "DOCKER_TLS_CERTDIR=" + ] + limit = X + [runners.docker] + image = "ruby:2.5" + privileged = true + volumes = [ + "/certs/client", + "/dummy-sys-class-dmi-id:/sys/class/dmi/id:ro" # Make kaniko builds work on GCP. + ] + [runners.machine] + IdleCount = 50 + IdleTime = 3600 + MaxBuilds = 1 # For security reasons we delete the VM after job has finished so it's not reused. + MachineName = "srm-%s" + MachineDriver = "google" + MachineOptions = [ + "google-project=PROJECT", + "google-disk-size=25", + "google-machine-type=n1-standard-1", + "google-username=core", + "google-tags=gitlab-com,srm", + "google-use-internal-ip", + "google-zone=us-east1-d", + "engine-opt=mtu=1460", # Set MTU for container interface, for more information check https://gitlab.com/gitlab-org/gitlab-runner/-/issues/3214#note_82892928 + "google-machine-image=PROJECT/global/images/IMAGE", + "engine-opt=ipv6", # This will create IPv6 interfaces in the containers. + "engine-opt=fixed-cidr-v6=fc00::/7", + "google-operation-backoff-initial-interval=2" # Custom flag from forked docker-machine, for more information check https://github.com/docker/machine/pull/4600 + ] + [[runners.machine.autoscaling]] + Periods = ["* * * * * sat,sun *"] + Timezone = "UTC" + IdleCount = 70 + IdleTime = 3600 + [[runners.machine.autoscaling]] + Periods = ["* 30-59 3 * * * *", "* 0-30 4 * * * *"] + Timezone = "UTC" + IdleCount = 700 + IdleTime = 3600 + [runners.cache] + Type = "gcs" + Shared = true + [runners.cache.gcs] + CredentialsFile = "/path/to/file" + BucketName = "bucket-name" +``` + +## Windows shared runners (beta) + +The Windows shared runners are in [beta](https://about.gitlab.com/handbook/product/gitlab-the-product/#beta) +and shouldn't be used for production workloads. + +During this beta period, the [shared runner pipeline quota](../../user/admin_area/settings/continuous_integration.md#shared-runners-pipeline-minutes-quota) +applies for groups and projects in the same manner as Linux runners. This may +change when the beta period ends, as discussed in this [related issue](https://gitlab.com/gitlab-org/gitlab/-/issues/30834). + +Windows shared runners on GitLab.com autoscale by launching virtual machines on +the Google Cloud Platform. This solution uses an +[autoscaling driver](https://gitlab.com/gitlab-org/ci-cd/custom-executor-drivers/autoscaler/tree/master/docs/readme.md) +developed by GitLab for the [custom executor](https://docs.gitlab.com/runner/executors/custom.html). +Windows shared runners execute your CI/CD jobs on `n1-standard-2` instances with +2 vCPUs and 7.5 GB RAM. You can find a full list of available Windows packages in +the [package documentation](https://gitlab.com/gitlab-org/ci-cd/shared-runners/images/gcp/windows-containers/blob/master/cookbooks/preinstalled-software/README.md). + +We want to keep iterating to get Windows shared runners in a stable state and +[generally available](https://about.gitlab.com/handbook/product/gitlab-the-product/#generally-available-ga). +You can follow our work towards this goal in the +[related epic](https://gitlab.com/groups/gitlab-org/-/epics/2162). + +### Configuration + +The full contents of our `config.toml` are: + +NOTE: +Settings that aren't public are shown as `X`. + +```toml +concurrent = X +check_interval = 3 + +[[runners]] + name = "windows-runner" + url = "https://gitlab.com/" + token = "TOKEN" + executor = "custom" + builds_dir = "C:\\GitLab-Runner\\builds" + cache_dir = "C:\\GitLab-Runner\\cache" + shell = "powershell" + [runners.custom] + config_exec = "C:\\GitLab-Runner\\autoscaler\\autoscaler.exe" + config_args = ["--config", "C:\\GitLab-Runner\\autoscaler\\config.toml", "custom", "config"] + prepare_exec = "C:\\GitLab-Runner\\autoscaler\\autoscaler.exe" + prepare_args = ["--config", "C:\\GitLab-Runner\\autoscaler\\config.toml", "custom", "prepare"] + run_exec = "C:\\GitLab-Runner\\autoscaler\\autoscaler.exe" + run_args = ["--config", "C:\\GitLab-Runner\\autoscaler\\config.toml", "custom", "run"] + cleanup_exec = "C:\\GitLab-Runner\\autoscaler\\autoscaler.exe" + cleanup_args = ["--config", "C:\\GitLab-Runner\\autoscaler\\config.toml", "custom", "cleanup"] +``` + +The full contents of our `autoscaler/config.toml` are: + +```toml +Provider = "gcp" +Executor = "winrm" +OS = "windows" +LogLevel = "info" +LogFormat = "text" +LogFile = "C:\\GitLab-Runner\\autoscaler\\autoscaler.log" +VMTag = "windows" + +[GCP] + ServiceAccountFile = "PATH" + Project = "some-project-df9323" + Zone = "us-east1-c" + MachineType = "n1-standard-2" + Image = "IMAGE" + DiskSize = 50 + DiskType = "pd-standard" + Subnetwork = "default" + Network = "default" + Tags = ["TAGS"] + Username = "gitlab_runner" + +[WinRM] + MaximumTimeout = 3600 + ExecutionMaxRetries = 0 + +[ProviderCache] + Enabled = true + Directory = "C:\\GitLab-Runner\\autoscaler\\machines" +``` + +### Example + +Below is a simple `.gitlab-ci.yml` file to show how to start using the +Windows shared runners: + +```yaml +.shared_windows_runners: + tags: + - shared-windows + - windows + - windows-1809 + +stages: + - build + - test + +before_script: + - Set-Variable -Name "time" -Value (date -Format "%H:%m") + - echo ${time} + - echo "started by ${GITLAB_USER_NAME}" + +build: + extends: + - .shared_windows_runners + stage: build + script: + - echo "running scripts in the build job" + +test: + extends: + - .shared_windows_runners + stage: test + script: + - echo "running scripts in the test job" +``` + +### Limitations and known issues + +- All the limitations mentioned in our [beta + definition](https://about.gitlab.com/handbook/product/#beta). +- The average provisioning time for a new Windows VM is 5 minutes. + This means that you may notice slower build start times + on the Windows shared runner fleet during the beta. In a future + release we intend to update the autoscaler to enable + the pre-provisioning of virtual machines. This is intended to significantly reduce + the time it takes to provision a VM on the Windows fleet. You can + follow along in the [related issue](https://gitlab.com/gitlab-org/ci-cd/custom-executor-drivers/autoscaler/-/issues/32). +- The Windows shared runner fleet may be unavailable occasionally + for maintenance or updates. +- The Windows shared runner virtual machine instances do not use the + GitLab Docker executor. This means that you can't specify + [`image`](../../ci/yaml/README.md#image) or [`services`](../../ci/yaml/README.md#services) in + your pipeline configuration. +- For the beta release, we have included a set of software packages in + the base VM image. If your CI job requires additional software that's + not included in this list, then you must add installation + commands to [`before_script`](../../ci/yaml/README.md#before_script) or [`script`](../../ci/yaml/README.md#script) to install the required + software. Note that each job runs on a new VM instance, so the + installation of additional software packages needs to be repeated for + each job in your pipeline. +- The job may stay in a pending state for longer than the + Linux shared runners. +- There is the possibility that we introduce breaking changes which will + require updates to pipelines that are using the Windows shared runner + fleet. diff --git a/doc/ci/services/gitlab.md b/doc/ci/services/gitlab.md index 81a17300e55..1f6ea828b9c 100644 --- a/doc/ci/services/gitlab.md +++ b/doc/ci/services/gitlab.md @@ -11,7 +11,7 @@ Many applications need to access JSON APIs, so application tests might need acce to APIs too. The following example shows how to use GitLab as a microservice to give tests access to the GitLab API. -1. Configure a [runner](../runners/README.md) with the Docker or Kubernetes executor. +1. Configure a [runner](../runners/index.md) with the Docker or Kubernetes executor. 1. In your `.gitlab-ci.yml` add: ```yaml diff --git a/doc/ci/services/mysql.md b/doc/ci/services/mysql.md index 1e0762ca010..f1dc3cb43b8 100644 --- a/doc/ci/services/mysql.md +++ b/doc/ci/services/mysql.md @@ -12,7 +12,7 @@ need it for your tests to run. ## Use MySQL with the Docker executor -If you want to use a MySQL container, you can use [GitLab Runner](../runners/README.md) with the Docker executor. +If you want to use a MySQL container, you can use [GitLab Runner](../runners/index.md) with the Docker executor. This example shows you how to set a username and password that GitLab uses to access the MySQL container. If you do not set a username and password, you must use `root`. @@ -130,5 +130,5 @@ GitLab Runner with the Shell executor. ## Example project To view a MySQL example, create a fork of this [sample project](https://gitlab.com/gitlab-examples/mysql). -This project uses publicly-available [shared runners](../runners/README.md) on [GitLab.com](https://gitlab.com). +This project uses publicly-available [shared runners](../runners/index.md) on [GitLab.com](https://gitlab.com). Update the README.md file, commit your changes, and view the CI/CD pipeline to see it in action. diff --git a/doc/ci/services/postgres.md b/doc/ci/services/postgres.md index 8451d56a71c..c5513934bc9 100644 --- a/doc/ci/services/postgres.md +++ b/doc/ci/services/postgres.md @@ -13,7 +13,7 @@ do this with the Docker and Shell executors of GitLab Runner. ## Use PostgreSQL with the Docker executor -If you're using [GitLab Runner](../runners/README.md) with the Docker executor, +If you're using [GitLab Runner](../runners/index.md) with the Docker executor, you basically have everything set up already. First, in your `.gitlab-ci.yml` add: @@ -121,7 +121,7 @@ Database: nice_marmot We have set up an [Example PostgreSQL Project](https://gitlab.com/gitlab-examples/postgres) for your convenience that runs on [GitLab.com](https://gitlab.com) using our publicly -available [shared runners](../runners/README.md). +available [shared runners](../runners/index.md). Want to hack on it? Fork it, commit, and push your changes. Within a few moments the changes are picked by a public runner and the job begins. diff --git a/doc/ci/services/redis.md b/doc/ci/services/redis.md index 71d3ffb1c60..d8c7b805864 100644 --- a/doc/ci/services/redis.md +++ b/doc/ci/services/redis.md @@ -13,7 +13,7 @@ do this with the Docker and Shell executors of GitLab Runner. ## Use Redis with the Docker executor -If you are using [GitLab Runner](../runners/README.md) with the Docker executor +If you are using [GitLab Runner](../runners/index.md) with the Docker executor you basically have everything set up already. First, in your `.gitlab-ci.yml` add: @@ -67,7 +67,7 @@ Host: localhost We have set up an [Example Redis Project](https://gitlab.com/gitlab-examples/redis) for your convenience that runs on [GitLab.com](https://gitlab.com) using our publicly available -[shared runners](../runners/README.md). +[shared runners](../runners/index.md). Want to hack on it? Simply fork it, commit and push your changes. Within a few moments the changes are picked by a public runner and the job begins. diff --git a/doc/ci/ssh_keys/index.md b/doc/ci/ssh_keys/index.md index 85755f9a179..c80389adb82 100644 --- a/doc/ci/ssh_keys/index.md +++ b/doc/ci/ssh_keys/index.md @@ -48,7 +48,7 @@ contained) and you want to deploy your code in a private server, you need a way to access it. This is where an SSH key pair comes in handy. 1. You first need to create an SSH key pair. For more information, follow - the instructions to [generate an SSH key](../../ssh/README.md#generate-an-ssh-key-pair). + the instructions to [generate an SSH key](../../ssh/index.md#generate-an-ssh-key-pair). **Do not** add a passphrase to the SSH key, or the `before_script` will prompt for it. @@ -124,7 +124,7 @@ on, and use that key for all projects that are run on this machine. ``` 1. Generate the SSH key pair as described in the instructions to - [generate an SSH key](../../ssh/README.md#generate-an-ssh-key-pair). + [generate an SSH key](../../ssh/index.md#generate-an-ssh-key-pair). **Do not** add a passphrase to the SSH key, or the `before_script` will prompt for it. @@ -207,7 +207,7 @@ before_script: We have set up an [Example SSH Project](https://gitlab.com/gitlab-examples/ssh-private-key/) for your convenience that runs on [GitLab.com](https://gitlab.com) using our publicly available -[shared runners](../runners/README.md). +[shared runners](../runners/index.md). Want to hack on it? Simply fork it, commit and push your changes. Within a few moments the changes is picked by a public runner and the job starts. diff --git a/doc/ci/triggers/README.md b/doc/ci/triggers/README.md index b8d0df44598..5ab8653dc35 100644 --- a/doc/ci/triggers/README.md +++ b/doc/ci/triggers/README.md @@ -1,288 +1,8 @@ --- -stage: Verify -group: Pipeline Execution -info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/engineering/ux/technical-writing/#assignments -type: tutorial +redirect_to: 'index.md' --- -# Triggering pipelines through the API **(FREE)** +This document was moved to [another location](index.md). -Triggers can be used to force a pipeline rerun of a specific `ref` (branch or -tag) with an API call. - -## Authentication tokens - -The following methods of authentication are supported: - -- [Trigger token](#trigger-token) -- [CI job token](#ci-job-token) - -If using the `$CI_PIPELINE_SOURCE` [predefined CI/CD variable](../variables/predefined_variables.md) -to limit which jobs run in a pipeline, the value could be either `pipeline` or `trigger`, -depending on which trigger method is used. - -| `$CI_PIPELINE_SOURCE` value | Trigger method | -|-----------------------------|----------------| -| `pipeline` | Using the `trigger:` keyword in the CI/CD configuration file, or using the trigger API with `$CI_JOB_TOKEN`. | -| `trigger` | Using the trigger API using a generated trigger token | - -This also applies when using the `pipelines` or `triggers` keywords with the legacy [`only/except` basic syntax](../yaml/README.md#only--except). - -### Trigger token - -A unique trigger token can be obtained when [adding a new trigger](#adding-a-new-trigger). - -WARNING: -Passing plain text tokens in public projects is a security issue. Potential -attackers can impersonate the user that exposed their trigger token publicly in -their `.gitlab-ci.yml` file. Use [CI/CD variables](../variables/README.md) -to protect trigger tokens. - -### CI job token - -You can use the `CI_JOB_TOKEN` [CI/CD variable](../variables/README.md#predefined-cicd-variables) (used to authenticate -with the [GitLab Container Registry](../../user/packages/container_registry/index.md)) in the following cases. - -#### When used with multi-project pipelines - -> - Use of `CI_JOB_TOKEN` for multi-project pipelines was [introduced](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/2017) in [GitLab Premium](https://about.gitlab.com/pricing/) 9.3. -> - Use of `CI_JOB_TOKEN` for multi-project pipelines was [made available](https://gitlab.com/gitlab-org/gitlab/-/issues/31573) in all tiers in GitLab 12.4. - -This way of triggering can only be used when invoked inside `.gitlab-ci.yml`, -and it creates a dependent pipeline relation visible on the -[pipeline graph](../multi_project_pipelines.md). For example: - -```yaml -trigger_pipeline: - stage: deploy - script: - - curl --request POST --form "token=$CI_JOB_TOKEN" --form ref=main "https://gitlab.example.com/api/v4/projects/9/trigger/pipeline" - rules: - - if: $CI_COMMIT_TAG -``` - -Pipelines triggered that way also expose a special variable: -`CI_PIPELINE_SOURCE=pipeline`. - -Read more about the [pipelines trigger API](../../api/pipeline_triggers.md). - -#### When a pipeline depends on the artifacts of another pipeline **(PREMIUM)** - -> The use of `CI_JOB_TOKEN` in the artifacts download API was [introduced](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/2346) in [GitLab Premium](https://about.gitlab.com/pricing/) 9.5. - -With the introduction of dependencies between different projects, one of -them may need to access artifacts created by a previous one. This process -must be granted for authorized accesses, and it can be done using the -`CI_JOB_TOKEN` variable that identifies a specific job. For example: - -```yaml -build_submodule: - image: debian - stage: test - script: - - apt update && apt install -y unzip - - curl --location --output artifacts.zip "https://gitlab.example.com/api/v4/projects/1/jobs/artifacts/main/download?job=test&job_token=$CI_JOB_TOKEN" - - unzip artifacts.zip - rules: - - if: $CI_COMMIT_TAG -``` - -This allows you to use that for multi-project pipelines and download artifacts -from any project to which you have access as this follows the same principles -with the [permission model](../../user/permissions.md#job-permissions). - -Read more about the [jobs API](../../api/job_artifacts.md#download-the-artifacts-archive). - -## Adding a new trigger - -Go to your -**Settings > CI/CD** under **Triggers** to add a new trigger. The **Add trigger** button creates -a new token which you can then use to trigger a rerun of this -particular project's pipeline. - -Every new trigger you create, gets assigned a different token which you can -then use inside your scripts or `.gitlab-ci.yml`. You also have a nice -overview of the time the triggers were last used. - - - -## Revoking a trigger - -You can revoke a trigger any time by going at your project's -**Settings > CI/CD** under **Triggers** and hitting the **Revoke** button. -The action is irreversible. - -## Triggering a pipeline - -To trigger a job you need to send a `POST` request to the GitLab API endpoint: - -```plaintext -POST /projects/:id/trigger/pipeline -``` - -The required parameters are the [trigger's `token`](#authentication-tokens) -and the Git `ref` on which the trigger is performed. Valid refs are -branches or tags. The `:id` of a project can be found by -[querying the API](../../api/projects.md) or by visiting the **CI/CD** -settings page which provides self-explanatory examples. - -When a rerun of a pipeline is triggered, jobs are marked as triggered `by API` in -**CI/CD > Jobs**. - -You can see which trigger caused a job to run by visiting the single job page. -A part of the trigger's token is exposed in the UI as you can see from the image -below. - - - -By using cURL you can trigger a pipeline rerun with minimal effort, for example: - -```shell -curl --request POST \ - --form token=TOKEN \ - --form ref=main \ - "https://gitlab.example.com/api/v4/projects/9/trigger/pipeline" -``` - -In this case, the pipeline for the project with ID `9` runs on the `main` branch. - -Alternatively, you can pass the `token` and `ref` arguments in the query string: - -```shell -curl --request POST \ - "https://gitlab.example.com/api/v4/projects/9/trigger/pipeline?token=TOKEN&ref=main" -``` - -You can also benefit by using triggers in your `.gitlab-ci.yml`. Let's say that -you have two projects, A and B, and you want to trigger a pipeline on the `main` -branch of project B whenever a tag on project A is created. This is the job you -need to add in project A's `.gitlab-ci.yml`: - -```yaml -trigger_pipeline: - stage: deploy - script: - - 'curl --request POST --form token=TOKEN --form ref=main "https://gitlab.example.com/api/v4/projects/9/trigger/pipeline"' - rules: - - if: $CI_COMMIT_TAG -``` - -This means that whenever a new tag is pushed on project A, the job runs and the -`trigger_pipeline` job is executed, triggering the pipeline for project B. The -`stage: deploy` ensures that this job runs only after all jobs with -`stage: test` complete successfully. - -## Triggering a pipeline from a webhook - -To trigger a job from a webhook of another project you need to add the following -webhook URL for Push and Tag events (change the project ID, ref and token): - -```plaintext -https://gitlab.example.com/api/v4/projects/9/ref/main/trigger/pipeline?token=TOKEN -``` - -You should pass `ref` as part of the URL, to take precedence over `ref` from -the webhook body that designates the branch ref that fired the trigger in the -source repository. Be sure to URL-encode `ref` if it contains slashes. - -### Using webhook payload in the triggered pipeline - -> - [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/31197) in GitLab 13.9. -> - [Feature flag removed](https://gitlab.com/gitlab-org/gitlab/-/issues/321027) in GitLab 13.11. - -If you trigger a pipeline by using a webhook, you can access the webhook payload with -the `TRIGGER_PAYLOAD` [predefined CI/CD variable](../variables/predefined_variables.md). -The payload is exposed as a [file-type variable](../variables/README.md#cicd-variable-types), -so you can access the data with `cat $TRIGGER_PAYLOAD` or a similar command. - -## Making use of trigger variables - -You can pass any number of arbitrary variables in the trigger API call and they -are available in GitLab CI/CD so that they can be used in your `.gitlab-ci.yml` -file. The parameter is of the form: - -```plaintext -variables[key]=value -``` - -This information is also exposed in the UI. Please note that _values_ are only viewable by Owners and Maintainers. - - - -Using trigger variables can be proven useful for a variety of reasons: - -- Identifiable jobs. Since the variable is exposed in the UI you can know - why the pipeline was triggered if you pass a variable that explains the - purpose. -- Conditional job processing. You can have conditional jobs that run whenever - a certain variable is present. - -Consider the following `.gitlab-ci.yml` where we set three -[stages](../yaml/README.md#stages) and the `upload_package` job is run only -when all jobs from the test and build stages pass. When the `UPLOAD_TO_S3` -variable is non-zero, `make upload` is run. - -```yaml -stages: - - test - - build - - package - -run_tests: - stage: test - script: - - make test - -build_package: - stage: build - script: - - make build - -upload_package: - stage: package - script: - - if [ -n "${UPLOAD_TO_S3}" ]; then make upload; fi -``` - -You can then trigger a pipeline while you pass the `UPLOAD_TO_S3` variable -and the script of the `upload_package` job is run: - -```shell -curl --request POST \ - --form token=TOKEN \ - --form ref=main \ - --form "variables[UPLOAD_TO_S3]=true" \ - "https://gitlab.example.com/api/v4/projects/9/trigger/pipeline" -``` - -Trigger variables have the [highest priority](../variables/README.md#cicd-variable-precedence) -of all types of variables. - -## Using cron to trigger nightly pipelines - -Whether you craft a script or just run cURL directly, you can trigger jobs -in conjunction with cron. The example below triggers a job on the `main` branch -of project with ID `9` every night at `00:30`: - -```shell -30 0 * * * curl --request POST --form token=TOKEN --form ref=main "https://gitlab.example.com/api/v4/projects/9/trigger/pipeline" -``` - -This behavior can also be achieved through the GitLab UI with -[pipeline schedules](../pipelines/schedules.md). - -## Legacy triggers - -Old triggers, created before GitLab 9.0 are marked as legacy. - -Triggers with the legacy label do not have an associated user and only have -access to the current project. They are considered deprecated and might be -removed with one of the future versions of GitLab. - -## Troubleshooting - -### '404 not found' when triggering a pipeline - -A response of `{"message":"404 Not Found"}` when triggering a pipeline might be caused -by using a Personal Access Token instead of a trigger token. [Add a new trigger](#adding-a-new-trigger) -and use that token to authenticate when triggering a pipeline. +<!-- This redirect file can be deleted after 2021-09-28. --> +<!-- Before deletion, see: https://docs.gitlab.com/ee/development/documentation/#move-or-rename-a-page --> diff --git a/doc/ci/triggers/index.md b/doc/ci/triggers/index.md new file mode 100644 index 00000000000..b8d0df44598 --- /dev/null +++ b/doc/ci/triggers/index.md @@ -0,0 +1,288 @@ +--- +stage: Verify +group: Pipeline Execution +info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/engineering/ux/technical-writing/#assignments +type: tutorial +--- + +# Triggering pipelines through the API **(FREE)** + +Triggers can be used to force a pipeline rerun of a specific `ref` (branch or +tag) with an API call. + +## Authentication tokens + +The following methods of authentication are supported: + +- [Trigger token](#trigger-token) +- [CI job token](#ci-job-token) + +If using the `$CI_PIPELINE_SOURCE` [predefined CI/CD variable](../variables/predefined_variables.md) +to limit which jobs run in a pipeline, the value could be either `pipeline` or `trigger`, +depending on which trigger method is used. + +| `$CI_PIPELINE_SOURCE` value | Trigger method | +|-----------------------------|----------------| +| `pipeline` | Using the `trigger:` keyword in the CI/CD configuration file, or using the trigger API with `$CI_JOB_TOKEN`. | +| `trigger` | Using the trigger API using a generated trigger token | + +This also applies when using the `pipelines` or `triggers` keywords with the legacy [`only/except` basic syntax](../yaml/README.md#only--except). + +### Trigger token + +A unique trigger token can be obtained when [adding a new trigger](#adding-a-new-trigger). + +WARNING: +Passing plain text tokens in public projects is a security issue. Potential +attackers can impersonate the user that exposed their trigger token publicly in +their `.gitlab-ci.yml` file. Use [CI/CD variables](../variables/README.md) +to protect trigger tokens. + +### CI job token + +You can use the `CI_JOB_TOKEN` [CI/CD variable](../variables/README.md#predefined-cicd-variables) (used to authenticate +with the [GitLab Container Registry](../../user/packages/container_registry/index.md)) in the following cases. + +#### When used with multi-project pipelines + +> - Use of `CI_JOB_TOKEN` for multi-project pipelines was [introduced](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/2017) in [GitLab Premium](https://about.gitlab.com/pricing/) 9.3. +> - Use of `CI_JOB_TOKEN` for multi-project pipelines was [made available](https://gitlab.com/gitlab-org/gitlab/-/issues/31573) in all tiers in GitLab 12.4. + +This way of triggering can only be used when invoked inside `.gitlab-ci.yml`, +and it creates a dependent pipeline relation visible on the +[pipeline graph](../multi_project_pipelines.md). For example: + +```yaml +trigger_pipeline: + stage: deploy + script: + - curl --request POST --form "token=$CI_JOB_TOKEN" --form ref=main "https://gitlab.example.com/api/v4/projects/9/trigger/pipeline" + rules: + - if: $CI_COMMIT_TAG +``` + +Pipelines triggered that way also expose a special variable: +`CI_PIPELINE_SOURCE=pipeline`. + +Read more about the [pipelines trigger API](../../api/pipeline_triggers.md). + +#### When a pipeline depends on the artifacts of another pipeline **(PREMIUM)** + +> The use of `CI_JOB_TOKEN` in the artifacts download API was [introduced](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/2346) in [GitLab Premium](https://about.gitlab.com/pricing/) 9.5. + +With the introduction of dependencies between different projects, one of +them may need to access artifacts created by a previous one. This process +must be granted for authorized accesses, and it can be done using the +`CI_JOB_TOKEN` variable that identifies a specific job. For example: + +```yaml +build_submodule: + image: debian + stage: test + script: + - apt update && apt install -y unzip + - curl --location --output artifacts.zip "https://gitlab.example.com/api/v4/projects/1/jobs/artifacts/main/download?job=test&job_token=$CI_JOB_TOKEN" + - unzip artifacts.zip + rules: + - if: $CI_COMMIT_TAG +``` + +This allows you to use that for multi-project pipelines and download artifacts +from any project to which you have access as this follows the same principles +with the [permission model](../../user/permissions.md#job-permissions). + +Read more about the [jobs API](../../api/job_artifacts.md#download-the-artifacts-archive). + +## Adding a new trigger + +Go to your +**Settings > CI/CD** under **Triggers** to add a new trigger. The **Add trigger** button creates +a new token which you can then use to trigger a rerun of this +particular project's pipeline. + +Every new trigger you create, gets assigned a different token which you can +then use inside your scripts or `.gitlab-ci.yml`. You also have a nice +overview of the time the triggers were last used. + + + +## Revoking a trigger + +You can revoke a trigger any time by going at your project's +**Settings > CI/CD** under **Triggers** and hitting the **Revoke** button. +The action is irreversible. + +## Triggering a pipeline + +To trigger a job you need to send a `POST` request to the GitLab API endpoint: + +```plaintext +POST /projects/:id/trigger/pipeline +``` + +The required parameters are the [trigger's `token`](#authentication-tokens) +and the Git `ref` on which the trigger is performed. Valid refs are +branches or tags. The `:id` of a project can be found by +[querying the API](../../api/projects.md) or by visiting the **CI/CD** +settings page which provides self-explanatory examples. + +When a rerun of a pipeline is triggered, jobs are marked as triggered `by API` in +**CI/CD > Jobs**. + +You can see which trigger caused a job to run by visiting the single job page. +A part of the trigger's token is exposed in the UI as you can see from the image +below. + + + +By using cURL you can trigger a pipeline rerun with minimal effort, for example: + +```shell +curl --request POST \ + --form token=TOKEN \ + --form ref=main \ + "https://gitlab.example.com/api/v4/projects/9/trigger/pipeline" +``` + +In this case, the pipeline for the project with ID `9` runs on the `main` branch. + +Alternatively, you can pass the `token` and `ref` arguments in the query string: + +```shell +curl --request POST \ + "https://gitlab.example.com/api/v4/projects/9/trigger/pipeline?token=TOKEN&ref=main" +``` + +You can also benefit by using triggers in your `.gitlab-ci.yml`. Let's say that +you have two projects, A and B, and you want to trigger a pipeline on the `main` +branch of project B whenever a tag on project A is created. This is the job you +need to add in project A's `.gitlab-ci.yml`: + +```yaml +trigger_pipeline: + stage: deploy + script: + - 'curl --request POST --form token=TOKEN --form ref=main "https://gitlab.example.com/api/v4/projects/9/trigger/pipeline"' + rules: + - if: $CI_COMMIT_TAG +``` + +This means that whenever a new tag is pushed on project A, the job runs and the +`trigger_pipeline` job is executed, triggering the pipeline for project B. The +`stage: deploy` ensures that this job runs only after all jobs with +`stage: test` complete successfully. + +## Triggering a pipeline from a webhook + +To trigger a job from a webhook of another project you need to add the following +webhook URL for Push and Tag events (change the project ID, ref and token): + +```plaintext +https://gitlab.example.com/api/v4/projects/9/ref/main/trigger/pipeline?token=TOKEN +``` + +You should pass `ref` as part of the URL, to take precedence over `ref` from +the webhook body that designates the branch ref that fired the trigger in the +source repository. Be sure to URL-encode `ref` if it contains slashes. + +### Using webhook payload in the triggered pipeline + +> - [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/31197) in GitLab 13.9. +> - [Feature flag removed](https://gitlab.com/gitlab-org/gitlab/-/issues/321027) in GitLab 13.11. + +If you trigger a pipeline by using a webhook, you can access the webhook payload with +the `TRIGGER_PAYLOAD` [predefined CI/CD variable](../variables/predefined_variables.md). +The payload is exposed as a [file-type variable](../variables/README.md#cicd-variable-types), +so you can access the data with `cat $TRIGGER_PAYLOAD` or a similar command. + +## Making use of trigger variables + +You can pass any number of arbitrary variables in the trigger API call and they +are available in GitLab CI/CD so that they can be used in your `.gitlab-ci.yml` +file. The parameter is of the form: + +```plaintext +variables[key]=value +``` + +This information is also exposed in the UI. Please note that _values_ are only viewable by Owners and Maintainers. + + + +Using trigger variables can be proven useful for a variety of reasons: + +- Identifiable jobs. Since the variable is exposed in the UI you can know + why the pipeline was triggered if you pass a variable that explains the + purpose. +- Conditional job processing. You can have conditional jobs that run whenever + a certain variable is present. + +Consider the following `.gitlab-ci.yml` where we set three +[stages](../yaml/README.md#stages) and the `upload_package` job is run only +when all jobs from the test and build stages pass. When the `UPLOAD_TO_S3` +variable is non-zero, `make upload` is run. + +```yaml +stages: + - test + - build + - package + +run_tests: + stage: test + script: + - make test + +build_package: + stage: build + script: + - make build + +upload_package: + stage: package + script: + - if [ -n "${UPLOAD_TO_S3}" ]; then make upload; fi +``` + +You can then trigger a pipeline while you pass the `UPLOAD_TO_S3` variable +and the script of the `upload_package` job is run: + +```shell +curl --request POST \ + --form token=TOKEN \ + --form ref=main \ + --form "variables[UPLOAD_TO_S3]=true" \ + "https://gitlab.example.com/api/v4/projects/9/trigger/pipeline" +``` + +Trigger variables have the [highest priority](../variables/README.md#cicd-variable-precedence) +of all types of variables. + +## Using cron to trigger nightly pipelines + +Whether you craft a script or just run cURL directly, you can trigger jobs +in conjunction with cron. The example below triggers a job on the `main` branch +of project with ID `9` every night at `00:30`: + +```shell +30 0 * * * curl --request POST --form token=TOKEN --form ref=main "https://gitlab.example.com/api/v4/projects/9/trigger/pipeline" +``` + +This behavior can also be achieved through the GitLab UI with +[pipeline schedules](../pipelines/schedules.md). + +## Legacy triggers + +Old triggers, created before GitLab 9.0 are marked as legacy. + +Triggers with the legacy label do not have an associated user and only have +access to the current project. They are considered deprecated and might be +removed with one of the future versions of GitLab. + +## Troubleshooting + +### '404 not found' when triggering a pipeline + +A response of `{"message":"404 Not Found"}` when triggering a pipeline might be caused +by using a Personal Access Token instead of a trigger token. [Add a new trigger](#adding-a-new-trigger) +and use that token to authenticate when triggering a pipeline. diff --git a/doc/ci/troubleshooting.md b/doc/ci/troubleshooting.md index bbbda7815d0..d3de1e6b88a 100644 --- a/doc/ci/troubleshooting.md +++ b/doc/ci/troubleshooting.md @@ -52,8 +52,8 @@ and check if their values are what you expect. The [complete `gitlab-ci.yml` reference](yaml/README.md) contains a full list of every keyword you may need to use to configure your pipelines. -You can also look at a large number of pipeline configuration [examples](examples/README.md) -and [templates](examples/README.md#cicd-templates). +You can also look at a large number of pipeline configuration [examples](examples/index.md) +and [templates](examples/index.md#cicd-templates). ### Documentation for pipeline types diff --git a/doc/ci/variables/README.md b/doc/ci/variables/README.md index 872892dbe9b..68980e19399 100644 --- a/doc/ci/variables/README.md +++ b/doc/ci/variables/README.md @@ -546,7 +546,7 @@ which variables take precedence. The order of precedence for variables is (from highest to lowest): -1. [Trigger variables](../triggers/README.md#making-use-of-trigger-variables), +1. [Trigger variables](../triggers/index.md#making-use-of-trigger-variables), [scheduled pipeline variables](../pipelines/schedules.md#using-variables), and [manual pipeline run variables](#override-a-variable-when-running-a-pipeline-manually). 1. Project [variables](#custom-cicd-variables). @@ -580,7 +580,7 @@ You can override the value of a variable when you: 1. Create a pipeline by using [the API](../../api/pipelines.md#create-a-new-pipeline). 1. Run a job manually in the UI. 1. Use [push options](../../user/project/push_options.md#push-options-for-gitlab-cicd). -1. Trigger a pipeline by using [the API](../triggers/README.md#making-use-of-trigger-variables). +1. Trigger a pipeline by using [the API](../triggers/index.md#making-use-of-trigger-variables). 1. Pass variables to a downstream pipeline [by using the `variable` keyword](../multi_project_pipelines.md#pass-cicd-variables-to-a-downstream-pipeline-by-using-the-variables-keyword) or [by using variable inheritance](../multi_project_pipelines.md#pass-cicd-variables-to-a-downstream-pipeline-by-using-variable-inheritance). diff --git a/doc/ci/variables/predefined_variables.md b/doc/ci/variables/predefined_variables.md index 4350154634c..1ca40ace13c 100644 --- a/doc/ci/variables/predefined_variables.md +++ b/doc/ci/variables/predefined_variables.md @@ -73,8 +73,8 @@ There are also [Kubernetes-specific deployment variables](../../user/project/clu | `CI_PAGES_URL` | 11.8 | all | The URL for a GitLab Pages site. Always a subdomain of `CI_PAGES_DOMAIN`. | | `CI_PIPELINE_ID` | 8.10 | all | The instance-level ID of the current pipeline. This ID is unique across all projects on the GitLab instance. | | `CI_PIPELINE_IID` | 11.0 | all | The project-level IID (internal ID) of the current pipeline. This ID is unique only within the current project. | -| `CI_PIPELINE_SOURCE` | 10.0 | all | How the pipeline was triggered. Can be `push`, `web`, `schedule`, `api`, `external`, `chat`, `webide`, `merge_request_event`, `external_pull_request_event`, `parent_pipeline`, [`trigger`, or `pipeline`](../triggers/README.md#authentication-tokens). | -| `CI_PIPELINE_TRIGGERED` | all | all | `true` if the job was [triggered](../triggers/README.md). | +| `CI_PIPELINE_SOURCE` | 10.0 | all | How the pipeline was triggered. Can be `push`, `web`, `schedule`, `api`, `external`, `chat`, `webide`, `merge_request_event`, `external_pull_request_event`, `parent_pipeline`, [`trigger`, or `pipeline`](../triggers/index.md#authentication-tokens). | +| `CI_PIPELINE_TRIGGERED` | all | all | `true` if the job was [triggered](../triggers/index.md). | | `CI_PIPELINE_URL` | 11.1 | 0.5 | The URL for the pipeline details. | | `CI_PIPELINE_CREATED_AT` | 13.10 | all | The UTC datetime when the pipeline was created, in [ISO 8601](https://tools.ietf.org/html/rfc3339#appendix-A) format. | | `CI_PROJECT_CONFIG_PATH` | 13.8 to 13.12 | all | [Removed](https://gitlab.com/gitlab-org/gitlab/-/issues/322807) in GitLab 14.0. Use `CI_CONFIG_PATH`. | @@ -119,7 +119,7 @@ There are also [Kubernetes-specific deployment variables](../../user/project/clu | `GITLAB_USER_ID` | 8.12 | all | The ID of the user who started the job. | | `GITLAB_USER_LOGIN` | 10.0 | all | The username of the user who started the job. | | `GITLAB_USER_NAME` | 10.0 | all | The name of the user who started the job. | -| `TRIGGER_PAYLOAD` | 13.9 | all | The webhook payload. Only available when a pipeline is [triggered with a webhook](../triggers/README.md#using-webhook-payload-in-the-triggered-pipeline). | +| `TRIGGER_PAYLOAD` | 13.9 | all | The webhook payload. Only available when a pipeline is [triggered with a webhook](../triggers/index.md#using-webhook-payload-in-the-triggered-pipeline). | ## Predefined variables for merge request pipelines @@ -130,6 +130,7 @@ These variables are available when: | Variable | GitLab | Runner | Description | |----------------------------------------|--------|--------|-------------| +| `CI_MERGE_REQUEST_APPROVED` | 14.1 | all | Approval status of the merge request. `true` when [merge request approvals](../../user/project/merge_requests/approvals/index.md) is available and the merge request has been approved. | | `CI_MERGE_REQUEST_ASSIGNEES` | 11.9 | all | Comma-separated list of usernames of assignees for the merge request. | | `CI_MERGE_REQUEST_ID` | 11.6 | all | The instance-level ID of the merge request. This is a unique ID across all projects on GitLab. | | `CI_MERGE_REQUEST_IID` | 11.6 | all | The project-level IID (internal ID) of the merge request. This ID is unique for the current project. | diff --git a/doc/ci/yaml/README.md b/doc/ci/yaml/README.md index 48f735d3bcf..da376ce0daf 100644 --- a/doc/ci/yaml/README.md +++ b/doc/ci/yaml/README.md @@ -14,7 +14,7 @@ type: reference This document lists the configuration options for your GitLab `.gitlab-ci.yml` file. - For a quick introduction to GitLab CI/CD, follow the [quick start guide](../quick_start/index.md). -- For a collection of examples, see [GitLab CI/CD Examples](../examples/README.md). +- For a collection of examples, see [GitLab CI/CD Examples](../examples/index.md). - To view a large `.gitlab-ci.yml` file used in an enterprise, see the [`.gitlab-ci.yml` file for `gitlab`](https://gitlab.com/gitlab-org/gitlab/-/blob/master/.gitlab-ci.yml). When you are editing your `.gitlab-ci.yml` file, you can validate it with the @@ -369,7 +369,7 @@ workflow: - ... # Previously defined workflow rules here ``` -[Triggered pipelines](../triggers/README.md) that run on a branch have a `$CI_COMMIT_BRANCH` +[Triggered pipelines](../triggers/index.md) that run on a branch have a `$CI_COMMIT_BRANCH` set and could be blocked by a similar rule. Triggered pipelines have a pipeline source of `trigger` or `pipeline`, so `&& $CI_PIPELINE_SOURCE == "push"` ensures the rule does not block triggered pipelines. @@ -1340,7 +1340,7 @@ pipeline based on branch names or pipeline types. | `pushes` | For pipelines triggered by a `git push` event, including for branches and tags. | | `schedules` | For [scheduled pipelines](../pipelines/schedules.md). | | `tags` | When the Git reference for a pipeline is a tag. | - | `triggers` | For pipelines created by using a [trigger token](../triggers/README.md#trigger-token). | + | `triggers` | For pipelines created by using a [trigger token](../triggers/index.md#trigger-token). | | `web` | For pipelines created by using **Run pipeline** button in the GitLab UI, from the project's **CI/CD > Pipelines** section. | **Example of `only:refs` and `except:refs`**: @@ -3722,7 +3722,7 @@ with a trigger token. The trigger token is different than the [`trigger`](#trigger) keyword. -[Read more in the triggers documentation.](../triggers/README.md) +[Read more in the triggers documentation.](../triggers/index.md) ### `interruptible` diff --git a/doc/development/README.md b/doc/development/README.md index bc996fdff21..5ab8653dc35 100644 --- a/doc/development/README.md +++ b/doc/development/README.md @@ -1,308 +1,8 @@ --- -comments: false -type: index, dev -stage: none -group: Development -info: "See the Technical Writers assigned to Development Guidelines: https://about.gitlab.com/handbook/engineering/ux/technical-writing/#assignments-to-development-guidelines" -description: "Development Guidelines: learn how to contribute to GitLab." +redirect_to: 'index.md' --- -# Contributor and Development Docs +This document was moved to [another location](index.md). -Learn the processes and technical information needed for contributing to GitLab. - -This content is intended for members of the GitLab Team as well as community -contributors. Content specific to the GitLab Team should instead be included in -the [Handbook](https://about.gitlab.com/handbook/). - -For information on using GitLab to work on your own software projects, see the -[GitLab user documentation](../user/index.md). - -For information on working with the GitLab APIs, see the [API documentation](../api/README.md). - -For information about how to install, configure, update, and upgrade your own -GitLab instance, see the [administration documentation](../administration/index.md). - -## Get started - -- Set up the GitLab development environment with the - [GitLab Development Kit (GDK)](https://gitlab.com/gitlab-org/gitlab-development-kit/-/blob/main/README.md) -- [GitLab contributing guide](contributing/index.md) - - [Issues workflow](contributing/issue_workflow.md) for more information about: - - Issue tracker guidelines. - - Triaging. - - Labels. - - Feature proposals. - - Issue weight. - - Regression issues. - - Technical or UX debt. - - [Merge requests workflow](contributing/merge_request_workflow.md) for more - information about: - - Merge request guidelines. - - Contribution acceptance criteria. - - Definition of done. - - Dependencies. - - [Style guides](contributing/style_guides.md) - - [Implement design & UI elements](contributing/design.md) -- [GitLab Architecture Overview](architecture.md) -- [Rake tasks](rake_tasks.md) for development - -## Processes - -**Must-reads:** - -- [Guide on adapting existing and introducing new components](architecture.md#adapting-existing-and-introducing-new-components) -- [Code review guidelines](code_review.md) for reviewing code and having code - reviewed -- [Database review guidelines](database_review.md) for reviewing - database-related changes and complex SQL queries, and having them reviewed -- [Secure coding guidelines](secure_coding_guidelines.md) -- [Pipelines for the GitLab project](pipelines.md) - -Complementary reads: - -- [GitLab core team & GitLab Inc. contribution process](https://gitlab.com/gitlab-org/gitlab/-/blob/master/PROCESS.md) -- [Security process for developers](https://gitlab.com/gitlab-org/release/docs/blob/master/general/security/developer.md#security-releases-critical-non-critical-as-a-developer) -- [Guidelines for implementing Enterprise Edition features](ee_features.md) -- [Danger bot](dangerbot.md) -- [Guidelines for changelogs](changelog.md) -- [Requesting access to ChatOps on GitLab.com](chatops_on_gitlabcom.md#requesting-access) (for GitLab team members) -- [Patch release process for developers](https://gitlab.com/gitlab-org/release/docs/blob/master/general/patch/process.md#process-for-developers) -- [Adding a new service component to GitLab](adding_service_component.md) - -### Development guidelines review - -When you submit a change to the GitLab development guidelines, who -you ask for reviews depends on the level of change. - -#### Wording, style, or link changes - -Not all changes require extensive review. For example, MRs that don't change the -content's meaning or function can be reviewed, approved, and merged by any -maintainer or Technical Writer. These can include: - -- Typo fixes. -- Clarifying links, such as to external programming language documentation. -- Changes to comply with the [Documentation Style Guide](documentation/index.md) - that don't change the intent of the documentation page. - -#### Specific changes - -If the MR proposes changes that are limited to a particular stage, group, or team, -request a review and approval from an experienced GitLab Team Member in that -group. For example, if you're documenting a new internal API used exclusively by -a given group, request an engineering review from one of the group's members. - -After the engineering review is complete, assign the MR to the -[Technical Writer associated with the stage and group](https://about.gitlab.com/handbook/engineering/ux/technical-writing/#assignments) -in the modified documentation page's metadata. - -If you have questions or need further input, request a review from the -Technical Writer assigned to the [Development Guidelines](https://about.gitlab.com/handbook/engineering/ux/technical-writing/#assignments-to-development-guidelines). - -#### Broader changes - -Some changes affect more than one group. For example: - -- Changes to [code review guidelines](code_review.md). -- Changes to [commit message guidelines](contributing/merge_request_workflow.md#commit-messages-guidelines). -- Changes to guidelines in [feature flags in development of GitLab](feature_flags/). -- Changes to [feature flags documentation guidelines](documentation/feature_flags.md). - -In these cases, use the following workflow: - -1. Request a peer review from a member of your team. -1. Request a review and approval of an Engineering Manager (EM) - or Staff Engineer who's responsible for the area in question: - - - [Frontend](https://about.gitlab.com/handbook/engineering/frontend/) - - [Backend](https://about.gitlab.com/handbook/engineering/) - - [Database](https://about.gitlab.com/handbook/engineering/development/database/) - - [User Experience (UX)](https://about.gitlab.com/handbook/engineering/ux/) - - [Security](https://about.gitlab.com/handbook/engineering/security/) - - [Quality](https://about.gitlab.com/handbook/engineering/quality/) - - [Engineering Productivity](https://about.gitlab.com/handbook/engineering/quality/engineering-productivity-team/) - - [Infrastructure](https://about.gitlab.com/handbook/engineering/infrastructure/) - - [Technical Writing](https://about.gitlab.com/handbook/engineering/ux/technical-writing/) - - You can skip this step for MRs authored by EMs or Staff Engineers responsible - for their area. - - If there are several affected groups, you may need approvals at the - EM/Staff Engineer level from each affected area. - -1. After completing the reviews, consult with the EM/Staff Engineer - author / approver of the MR. - - If this is a significant change across multiple areas, request final review - and approval from the VP of Development, the DRI for Development Guidelines, - @clefelhocz1. - -1. After all approvals are complete, assign the merge request to the - Technical Writer for [Development Guidelines](https://about.gitlab.com/handbook/engineering/ux/technical-writing/#assignments-to-development-guidelines) - for final content review and merge. The Technical Writer may ask for - additional approvals as previously suggested before merging the MR. - -## UX and Frontend guides - -- [GitLab Design System](https://design.gitlab.com/), for building GitLab with - existing CSS styles and elements -- [Frontend guidelines](fe_guide/index.md) -- [Emoji guide](fe_guide/emojis.md) - -## Backend guides - -- [Directory structure](directory_structure.md) -- [GitLab utilities](utilities.md) -- [Issuable-like Rails models](issuable-like-models.md) -- [Logging](logging.md) -- [API style guide](api_styleguide.md) for contributing to the API -- [GraphQL API style guide](api_graphql_styleguide.md) for contributing to the - [GraphQL API](../api/graphql/index.md) -- [Sidekiq guidelines](sidekiq_style_guide.md) for working with Sidekiq workers -- [Working with Gitaly](gitaly.md) -- [Manage feature flags](feature_flags/index.md) -- [Licensed feature availability](licensed_feature_availability.md) -- [Dealing with email/mailers](emails.md) -- [Shell commands](shell_commands.md) in the GitLab codebase -- [`Gemfile` guidelines](gemfile.md) -- [Pry debugging](pry_debugging.md) -- [Sidekiq debugging](../administration/troubleshooting/sidekiq.md) -- [Accessing session data](session.md) -- [Gotchas](gotchas.md) to avoid -- [Avoid modules with instance variables](module_with_instance_variables.md), if - possible -- [How to dump production data to staging](db_dump.md) -- [Working with the GitHub importer](github_importer.md) -- [Import/Export development documentation](import_export.md) -- [Test Import Project](import_project.md) -- [Group migration](bulk_import.md) -- [Elasticsearch integration docs](elasticsearch.md) -- [Working with Merge Request diffs](diffs.md) -- [Kubernetes integration guidelines](kubernetes.md) -- [Permissions](permissions.md) -- [Guidelines for reusing abstractions](reusing_abstractions.md) -- [DeclarativePolicy framework](policies.md) -- [How Git object deduplication works in GitLab](git_object_deduplication.md) -- [Geo development](geo.md) -- [Routing](routing.md) -- [Repository mirroring](repository_mirroring.md) -- [Git LFS](lfs.md) -- [Developing against interacting components or features](interacting_components.md) -- [File uploads](uploads.md) -- [Auto DevOps development guide](auto_devops.md) -- [Mass Inserting Models](mass_insert.md) -- [Value Stream Analytics development guide](value_stream_analytics.md) -- [Issue types vs first-class types](issue_types.md) -- [Application limits](application_limits.md) -- [Redis guidelines](redis.md) -- [Rails initializers](rails_initializers.md) -- [Code comments](code_comments.md) -- [Renaming features](renaming_features.md) -- [Windows Development on GCP](windows.md) -- [Code Intelligence](code_intelligence/index.md) -- [Approval Rules](approval_rules.md) -- [Feature categorization](feature_categorization/index.md) -- [Wikis development guide](wikis.md) -- [Newlines style guide](newlines_styleguide.md) -- [Image scaling guide](image_scaling.md) -- [Export to CSV](export_csv.md) -- [Cascading Settings](cascading_settings.md) -- [FIPS compliance](fips_compliance.md) - -## Performance guides - -- [Instrumentation](instrumentation.md) for Ruby code running in production - environments. -- [Performance guidelines](performance.md) for writing code, benchmarks, and - certain patterns to avoid. -- [Merge request performance guidelines](merge_request_performance_guidelines.md) - for ensuring merge requests do not negatively impact GitLab performance -- [Profiling](profiling.md) a URL, measuring performance using Sherlock, or - tracking down N+1 queries using Bullet. -- [Cached queries guidelines](cached_queries.md), for tracking down N+1 queries - masked by query caching, memory profiling and why should we avoid cached - queries. - -## Database guides - -See [database guidelines](database/index.md). - -## Integration guides - -- [Jira Connect app](integrations/jira_connect.md) -- [Security Scanners](integrations/secure.md) -- [Secure Partner Integration](integrations/secure_partner_integration.md) -- [How to run Jenkins in development environment](integrations/jenkins.md) -- [How to run local `Codesandbox` integration for Web IDE Live Preview](integrations/codesandbox.md) - -## Testing guides - -- [Testing standards and style guidelines](testing_guide/index.md) -- [Frontend testing standards and style guidelines](testing_guide/frontend_testing.md) - -## Refactoring guides - -- [Refactoring guidelines](refactoring_guide/index.md) - -## Deprecation guides - -- [Deprecation guidelines](deprecation_guidelines/index.md) - -## Documentation guides - -- [Writing documentation](documentation/index.md) -- [Documentation style guide](documentation/styleguide/index.md) -- [Markdown](../user/markdown.md) - -## Internationalization (i18n) guides - -- [Introduction](i18n/index.md) -- [Externalization](i18n/externalization.md) -- [Translation](i18n/translation.md) - -## Product Intelligence guides - -- [Product Intelligence guide](https://about.gitlab.com/handbook/product/product-intelligence-guide/) -- [Usage Ping guide](usage_ping/index.md) -- [Snowplow guide](snowplow/index.md) - -## Experiment guide - -- [Introduction](experiment_guide/index.md) - -## Build guides - -- [Building a package for testing purposes](build_test_package.md) - -## Compliance - -- [Licensing](licensing.md) for ensuring license compliance - -## Go guides - -- [Go Guidelines](go_guide/index.md) - -## Shell Scripting guides - -- [Shell scripting standards and style guidelines](shell_scripting_guide/index.md) - -## Domain-specific guides - -- [CI/CD development documentation](cicd/index.md) -- [AppSec development documentation](appsec/index.md) - -## Other Development guides - -- [Defining relations between files using projections](projections.md) -- [Reference processing](reference_processing.md) -- [Compatibility with multiple versions of the application running at the same time](multi_version_compatibility.md) -- [Features inside `.gitlab/`](features_inside_dot_gitlab.md) -- [Dashboards for stage groups](stage_group_dashboards.md) -- [Preventing transient bugs](transient/prevention-patterns.md) - -## Other GitLab Development Kit (GDK) guides - -- [Run full Auto DevOps cycle in a GDK instance](https://gitlab.com/gitlab-org/gitlab-development-kit/blob/main/doc/howto/auto_devops.md) -- [Using GitLab Runner with the GDK](https://gitlab.com/gitlab-org/gitlab-development-kit/blob/main/doc/howto/runner.md) -- [Using the Web IDE terminal with the GDK](https://gitlab.com/gitlab-org/gitlab-development-kit/-/blob/main/doc/howto/web_ide_terminal_gdk_setup.md) +<!-- This redirect file can be deleted after 2021-09-28. --> +<!-- Before deletion, see: https://docs.gitlab.com/ee/development/documentation/#move-or-rename-a-page --> diff --git a/doc/development/changelog.md b/doc/development/changelog.md index f0c37af42ab..e05428ea826 100644 --- a/doc/development/changelog.md +++ b/doc/development/changelog.md @@ -188,4 +188,4 @@ documentation](https://git-scm.com/book/en/v2/Git-Tools-Rewriting-History). --- -[Return to Development documentation](README.md) +[Return to Development documentation](index.md) diff --git a/doc/development/cicd/index.md b/doc/development/cicd/index.md index 6e3cdb59fd8..64a3c74db62 100644 --- a/doc/development/cicd/index.md +++ b/doc/development/cicd/index.md @@ -61,7 +61,7 @@ successfully or fail. Each status transition for job within a pipeline triggers looks for the next jobs to be transitioned towards completion. While doing that, `ProcessPipelineService` updates the status of jobs, stages and the overall pipeline. -On the right side of the diagram we have a list of [runners](../../ci/runners/README.md) +On the right side of the diagram we have a list of [runners](../../ci/runners/index.md) connected to the GitLab instance. These can be shared runners, group runners, or project-specific runners. The communication between runners and the Rails server occurs through a set of API endpoints, grouped as the `Runner API Gateway`. diff --git a/doc/development/cicd/templates.md b/doc/development/cicd/templates.md index 8331985697e..eddaabe41d8 100644 --- a/doc/development/cicd/templates.md +++ b/doc/development/cicd/templates.md @@ -7,7 +7,7 @@ type: index, concepts, howto # Development guide for GitLab CI/CD templates -This document explains how to develop [GitLab CI/CD templates](../../ci/examples/README.md). +This document explains how to develop [GitLab CI/CD templates](../../ci/examples/index.md). ## Requirements for CI/CD templates diff --git a/doc/development/code_review.md b/doc/development/code_review.md index df09b27c6b4..6f1d916fa78 100644 --- a/doc/development/code_review.md +++ b/doc/development/code_review.md @@ -644,4 +644,4 @@ Largely based on the [`thoughtbot` code review guide](https://github.com/thought --- -[Return to Development documentation](README.md) +[Return to Development documentation](index.md) diff --git a/doc/development/contributing/merge_request_workflow.md b/doc/development/contributing/merge_request_workflow.md index 783cf7af6fc..9be89202d35 100644 --- a/doc/development/contributing/merge_request_workflow.md +++ b/doc/development/contributing/merge_request_workflow.md @@ -31,7 +31,7 @@ If you are new to GitLab development (or web development in general), see the some potentially easy issues. To start developing GitLab, download the [GitLab Development Kit](https://gitlab.com/gitlab-org/gitlab-development-kit) -and see the [Development section](../../README.md) for the required guidelines. +and see the [Development section](../../index.md) for the required guidelines. ## Merge request guidelines diff --git a/doc/development/dangerbot.md b/doc/development/dangerbot.md index 68268027b73..3ecabd2d72d 100644 --- a/doc/development/dangerbot.md +++ b/doc/development/dangerbot.md @@ -58,7 +58,7 @@ itself, increasing visibility. ## Development guidelines -Danger code is Ruby code, so all our [usual backend guidelines](README.md#backend-guides) +Danger code is Ruby code, so all our [usual backend guidelines](index.md#backend-guides) continue to apply. However, there are a few things that deserve special emphasis. ### When to use Danger diff --git a/doc/development/documentation/index.md b/doc/development/documentation/index.md index 0f8c5f8e51f..14798b747c8 100644 --- a/doc/development/documentation/index.md +++ b/doc/development/documentation/index.md @@ -115,11 +115,11 @@ each page should have a metadata tag called `type`. It can be one or more of the following: - `index`: It consists mostly of a list of links to other pages. - [Example page](../../README.md). + [Example page](../../index.md). - `concepts`: The background or context of a subject. [Example page](../../topics/autodevops/index.md). - `howto`: Specific use case instructions. - [Example page](../../ssh/README.md). + [Example page](../../ssh/index.md). - `tutorial`: Learn a process/concept by doing. [Example page](../../gitlab-basics/start-using-git.md). - `reference`: A collection of information used as a reference to use a feature @@ -395,7 +395,7 @@ This is preferred over static paths, as the helper also works on instances insta ### GitLab `/help` tests Several [RSpec tests](https://gitlab.com/gitlab-org/gitlab/-/blob/master/spec/features/help_pages_spec.rb) -are run to ensure GitLab documentation renders and works correctly. In particular, that [main docs landing page](../../README.md) works correctly from `/help`. +are run to ensure GitLab documentation renders and works correctly. In particular, that [main docs landing page](../../index.md) works correctly from `/help`. For example, [GitLab.com's `/help`](https://gitlab.com/help). ## Docs site architecture diff --git a/doc/development/documentation/structure.md b/doc/development/documentation/structure.md index 871fb26ce08..b2152858e8e 100644 --- a/doc/development/documentation/structure.md +++ b/doc/development/documentation/structure.md @@ -210,7 +210,7 @@ comments: false --- ``` -We're hiding comments only in main index pages, such as [the main documentation index](../../README.md), +We're hiding comments only in main index pages, such as [the main documentation index](../../index.md), since its content is too broad to comment on. Before omitting Disqus, you must check with a technical writer. diff --git a/doc/development/documentation/styleguide/index.md b/doc/development/documentation/styleguide/index.md index fffb4d21a26..4c4e3755451 100644 --- a/doc/development/documentation/styleguide/index.md +++ b/doc/development/documentation/styleguide/index.md @@ -1495,7 +1495,7 @@ elements: ## GitLab versions -GitLab product documentation pages (not including [Contributor and Development](../../README.md) +GitLab product documentation pages (not including [Contributor and Development](../../index.md) pages in the `/development` directory) can include version information to help users be aware of recent improvements or additions. diff --git a/doc/development/emails.md b/doc/development/emails.md index 3e651a6efb8..c1054077f9e 100644 --- a/doc/development/emails.md +++ b/doc/development/emails.md @@ -141,4 +141,4 @@ Please note that `path/to/project` is used in GitLab as the handler for the Serv --- -[Return to Development documentation](README.md) +[Return to Development documentation](index.md) diff --git a/doc/development/gitaly.md b/doc/development/gitaly.md index 2e814a9c41b..59ab3d41f16 100644 --- a/doc/development/gitaly.md +++ b/doc/development/gitaly.md @@ -235,7 +235,7 @@ changes to embed a new SHA in the `Gemfile.lock` file. --- -[Return to Development documentation](README.md) +[Return to Development documentation](index.md) ## Wrapping RPCs in Feature Flags diff --git a/doc/development/go_guide/index.md b/doc/development/go_guide/index.md index ad24353fde8..dab25711a59 100644 --- a/doc/development/go_guide/index.md +++ b/doc/development/go_guide/index.md @@ -507,4 +507,4 @@ If the scanner report is small, less than 35 lines, then feel free to [inline th --- -[Return to Development documentation](../README.md). +[Return to Development documentation](../index.md). diff --git a/doc/development/index.md b/doc/development/index.md new file mode 100644 index 00000000000..bc996fdff21 --- /dev/null +++ b/doc/development/index.md @@ -0,0 +1,308 @@ +--- +comments: false +type: index, dev +stage: none +group: Development +info: "See the Technical Writers assigned to Development Guidelines: https://about.gitlab.com/handbook/engineering/ux/technical-writing/#assignments-to-development-guidelines" +description: "Development Guidelines: learn how to contribute to GitLab." +--- + +# Contributor and Development Docs + +Learn the processes and technical information needed for contributing to GitLab. + +This content is intended for members of the GitLab Team as well as community +contributors. Content specific to the GitLab Team should instead be included in +the [Handbook](https://about.gitlab.com/handbook/). + +For information on using GitLab to work on your own software projects, see the +[GitLab user documentation](../user/index.md). + +For information on working with the GitLab APIs, see the [API documentation](../api/README.md). + +For information about how to install, configure, update, and upgrade your own +GitLab instance, see the [administration documentation](../administration/index.md). + +## Get started + +- Set up the GitLab development environment with the + [GitLab Development Kit (GDK)](https://gitlab.com/gitlab-org/gitlab-development-kit/-/blob/main/README.md) +- [GitLab contributing guide](contributing/index.md) + - [Issues workflow](contributing/issue_workflow.md) for more information about: + - Issue tracker guidelines. + - Triaging. + - Labels. + - Feature proposals. + - Issue weight. + - Regression issues. + - Technical or UX debt. + - [Merge requests workflow](contributing/merge_request_workflow.md) for more + information about: + - Merge request guidelines. + - Contribution acceptance criteria. + - Definition of done. + - Dependencies. + - [Style guides](contributing/style_guides.md) + - [Implement design & UI elements](contributing/design.md) +- [GitLab Architecture Overview](architecture.md) +- [Rake tasks](rake_tasks.md) for development + +## Processes + +**Must-reads:** + +- [Guide on adapting existing and introducing new components](architecture.md#adapting-existing-and-introducing-new-components) +- [Code review guidelines](code_review.md) for reviewing code and having code + reviewed +- [Database review guidelines](database_review.md) for reviewing + database-related changes and complex SQL queries, and having them reviewed +- [Secure coding guidelines](secure_coding_guidelines.md) +- [Pipelines for the GitLab project](pipelines.md) + +Complementary reads: + +- [GitLab core team & GitLab Inc. contribution process](https://gitlab.com/gitlab-org/gitlab/-/blob/master/PROCESS.md) +- [Security process for developers](https://gitlab.com/gitlab-org/release/docs/blob/master/general/security/developer.md#security-releases-critical-non-critical-as-a-developer) +- [Guidelines for implementing Enterprise Edition features](ee_features.md) +- [Danger bot](dangerbot.md) +- [Guidelines for changelogs](changelog.md) +- [Requesting access to ChatOps on GitLab.com](chatops_on_gitlabcom.md#requesting-access) (for GitLab team members) +- [Patch release process for developers](https://gitlab.com/gitlab-org/release/docs/blob/master/general/patch/process.md#process-for-developers) +- [Adding a new service component to GitLab](adding_service_component.md) + +### Development guidelines review + +When you submit a change to the GitLab development guidelines, who +you ask for reviews depends on the level of change. + +#### Wording, style, or link changes + +Not all changes require extensive review. For example, MRs that don't change the +content's meaning or function can be reviewed, approved, and merged by any +maintainer or Technical Writer. These can include: + +- Typo fixes. +- Clarifying links, such as to external programming language documentation. +- Changes to comply with the [Documentation Style Guide](documentation/index.md) + that don't change the intent of the documentation page. + +#### Specific changes + +If the MR proposes changes that are limited to a particular stage, group, or team, +request a review and approval from an experienced GitLab Team Member in that +group. For example, if you're documenting a new internal API used exclusively by +a given group, request an engineering review from one of the group's members. + +After the engineering review is complete, assign the MR to the +[Technical Writer associated with the stage and group](https://about.gitlab.com/handbook/engineering/ux/technical-writing/#assignments) +in the modified documentation page's metadata. + +If you have questions or need further input, request a review from the +Technical Writer assigned to the [Development Guidelines](https://about.gitlab.com/handbook/engineering/ux/technical-writing/#assignments-to-development-guidelines). + +#### Broader changes + +Some changes affect more than one group. For example: + +- Changes to [code review guidelines](code_review.md). +- Changes to [commit message guidelines](contributing/merge_request_workflow.md#commit-messages-guidelines). +- Changes to guidelines in [feature flags in development of GitLab](feature_flags/). +- Changes to [feature flags documentation guidelines](documentation/feature_flags.md). + +In these cases, use the following workflow: + +1. Request a peer review from a member of your team. +1. Request a review and approval of an Engineering Manager (EM) + or Staff Engineer who's responsible for the area in question: + + - [Frontend](https://about.gitlab.com/handbook/engineering/frontend/) + - [Backend](https://about.gitlab.com/handbook/engineering/) + - [Database](https://about.gitlab.com/handbook/engineering/development/database/) + - [User Experience (UX)](https://about.gitlab.com/handbook/engineering/ux/) + - [Security](https://about.gitlab.com/handbook/engineering/security/) + - [Quality](https://about.gitlab.com/handbook/engineering/quality/) + - [Engineering Productivity](https://about.gitlab.com/handbook/engineering/quality/engineering-productivity-team/) + - [Infrastructure](https://about.gitlab.com/handbook/engineering/infrastructure/) + - [Technical Writing](https://about.gitlab.com/handbook/engineering/ux/technical-writing/) + + You can skip this step for MRs authored by EMs or Staff Engineers responsible + for their area. + + If there are several affected groups, you may need approvals at the + EM/Staff Engineer level from each affected area. + +1. After completing the reviews, consult with the EM/Staff Engineer + author / approver of the MR. + + If this is a significant change across multiple areas, request final review + and approval from the VP of Development, the DRI for Development Guidelines, + @clefelhocz1. + +1. After all approvals are complete, assign the merge request to the + Technical Writer for [Development Guidelines](https://about.gitlab.com/handbook/engineering/ux/technical-writing/#assignments-to-development-guidelines) + for final content review and merge. The Technical Writer may ask for + additional approvals as previously suggested before merging the MR. + +## UX and Frontend guides + +- [GitLab Design System](https://design.gitlab.com/), for building GitLab with + existing CSS styles and elements +- [Frontend guidelines](fe_guide/index.md) +- [Emoji guide](fe_guide/emojis.md) + +## Backend guides + +- [Directory structure](directory_structure.md) +- [GitLab utilities](utilities.md) +- [Issuable-like Rails models](issuable-like-models.md) +- [Logging](logging.md) +- [API style guide](api_styleguide.md) for contributing to the API +- [GraphQL API style guide](api_graphql_styleguide.md) for contributing to the + [GraphQL API](../api/graphql/index.md) +- [Sidekiq guidelines](sidekiq_style_guide.md) for working with Sidekiq workers +- [Working with Gitaly](gitaly.md) +- [Manage feature flags](feature_flags/index.md) +- [Licensed feature availability](licensed_feature_availability.md) +- [Dealing with email/mailers](emails.md) +- [Shell commands](shell_commands.md) in the GitLab codebase +- [`Gemfile` guidelines](gemfile.md) +- [Pry debugging](pry_debugging.md) +- [Sidekiq debugging](../administration/troubleshooting/sidekiq.md) +- [Accessing session data](session.md) +- [Gotchas](gotchas.md) to avoid +- [Avoid modules with instance variables](module_with_instance_variables.md), if + possible +- [How to dump production data to staging](db_dump.md) +- [Working with the GitHub importer](github_importer.md) +- [Import/Export development documentation](import_export.md) +- [Test Import Project](import_project.md) +- [Group migration](bulk_import.md) +- [Elasticsearch integration docs](elasticsearch.md) +- [Working with Merge Request diffs](diffs.md) +- [Kubernetes integration guidelines](kubernetes.md) +- [Permissions](permissions.md) +- [Guidelines for reusing abstractions](reusing_abstractions.md) +- [DeclarativePolicy framework](policies.md) +- [How Git object deduplication works in GitLab](git_object_deduplication.md) +- [Geo development](geo.md) +- [Routing](routing.md) +- [Repository mirroring](repository_mirroring.md) +- [Git LFS](lfs.md) +- [Developing against interacting components or features](interacting_components.md) +- [File uploads](uploads.md) +- [Auto DevOps development guide](auto_devops.md) +- [Mass Inserting Models](mass_insert.md) +- [Value Stream Analytics development guide](value_stream_analytics.md) +- [Issue types vs first-class types](issue_types.md) +- [Application limits](application_limits.md) +- [Redis guidelines](redis.md) +- [Rails initializers](rails_initializers.md) +- [Code comments](code_comments.md) +- [Renaming features](renaming_features.md) +- [Windows Development on GCP](windows.md) +- [Code Intelligence](code_intelligence/index.md) +- [Approval Rules](approval_rules.md) +- [Feature categorization](feature_categorization/index.md) +- [Wikis development guide](wikis.md) +- [Newlines style guide](newlines_styleguide.md) +- [Image scaling guide](image_scaling.md) +- [Export to CSV](export_csv.md) +- [Cascading Settings](cascading_settings.md) +- [FIPS compliance](fips_compliance.md) + +## Performance guides + +- [Instrumentation](instrumentation.md) for Ruby code running in production + environments. +- [Performance guidelines](performance.md) for writing code, benchmarks, and + certain patterns to avoid. +- [Merge request performance guidelines](merge_request_performance_guidelines.md) + for ensuring merge requests do not negatively impact GitLab performance +- [Profiling](profiling.md) a URL, measuring performance using Sherlock, or + tracking down N+1 queries using Bullet. +- [Cached queries guidelines](cached_queries.md), for tracking down N+1 queries + masked by query caching, memory profiling and why should we avoid cached + queries. + +## Database guides + +See [database guidelines](database/index.md). + +## Integration guides + +- [Jira Connect app](integrations/jira_connect.md) +- [Security Scanners](integrations/secure.md) +- [Secure Partner Integration](integrations/secure_partner_integration.md) +- [How to run Jenkins in development environment](integrations/jenkins.md) +- [How to run local `Codesandbox` integration for Web IDE Live Preview](integrations/codesandbox.md) + +## Testing guides + +- [Testing standards and style guidelines](testing_guide/index.md) +- [Frontend testing standards and style guidelines](testing_guide/frontend_testing.md) + +## Refactoring guides + +- [Refactoring guidelines](refactoring_guide/index.md) + +## Deprecation guides + +- [Deprecation guidelines](deprecation_guidelines/index.md) + +## Documentation guides + +- [Writing documentation](documentation/index.md) +- [Documentation style guide](documentation/styleguide/index.md) +- [Markdown](../user/markdown.md) + +## Internationalization (i18n) guides + +- [Introduction](i18n/index.md) +- [Externalization](i18n/externalization.md) +- [Translation](i18n/translation.md) + +## Product Intelligence guides + +- [Product Intelligence guide](https://about.gitlab.com/handbook/product/product-intelligence-guide/) +- [Usage Ping guide](usage_ping/index.md) +- [Snowplow guide](snowplow/index.md) + +## Experiment guide + +- [Introduction](experiment_guide/index.md) + +## Build guides + +- [Building a package for testing purposes](build_test_package.md) + +## Compliance + +- [Licensing](licensing.md) for ensuring license compliance + +## Go guides + +- [Go Guidelines](go_guide/index.md) + +## Shell Scripting guides + +- [Shell scripting standards and style guidelines](shell_scripting_guide/index.md) + +## Domain-specific guides + +- [CI/CD development documentation](cicd/index.md) +- [AppSec development documentation](appsec/index.md) + +## Other Development guides + +- [Defining relations between files using projections](projections.md) +- [Reference processing](reference_processing.md) +- [Compatibility with multiple versions of the application running at the same time](multi_version_compatibility.md) +- [Features inside `.gitlab/`](features_inside_dot_gitlab.md) +- [Dashboards for stage groups](stage_group_dashboards.md) +- [Preventing transient bugs](transient/prevention-patterns.md) + +## Other GitLab Development Kit (GDK) guides + +- [Run full Auto DevOps cycle in a GDK instance](https://gitlab.com/gitlab-org/gitlab-development-kit/blob/main/doc/howto/auto_devops.md) +- [Using GitLab Runner with the GDK](https://gitlab.com/gitlab-org/gitlab-development-kit/blob/main/doc/howto/runner.md) +- [Using the Web IDE terminal with the GDK](https://gitlab.com/gitlab-org/gitlab-development-kit/-/blob/main/doc/howto/web_ide_terminal_gdk_setup.md) diff --git a/doc/development/pipelines.md b/doc/development/pipelines.md index 872dea060cd..5dd586105e9 100644 --- a/doc/development/pipelines.md +++ b/doc/development/pipelines.md @@ -585,7 +585,7 @@ several reasons: - It significantly reduces load on the file server, as smaller deltas mean less time spent in `git pack-objects`. The pre-clone step works by using the `CI_PRE_CLONE_SCRIPT` variable -[defined by GitLab.com shared runners](../ci/runners/README.md#pre-clone-script). +[defined by GitLab.com shared runners](../ci/runners/index.md#pre-clone-script). The `CI_PRE_CLONE_SCRIPT` is currently defined as a project CI/CD variable: @@ -813,4 +813,4 @@ and included in `rules` definitions via [YAML anchors](../ci/yaml/README.md#anch --- -[Return to Development documentation](README.md) +[Return to Development documentation](index.md) diff --git a/doc/development/shell_scripting_guide/index.md b/doc/development/shell_scripting_guide/index.md index 6071ae3a09d..d3b446d45da 100644 --- a/doc/development/shell_scripting_guide/index.md +++ b/doc/development/shell_scripting_guide/index.md @@ -127,4 +127,4 @@ for code review. --- -[Return to Development documentation](../README.md). +[Return to Development documentation](../index.md). diff --git a/doc/development/testing_guide/index.md b/doc/development/testing_guide/index.md index c22a4e0b3ad..889dc45d6e3 100644 --- a/doc/development/testing_guide/index.md +++ b/doc/development/testing_guide/index.md @@ -70,4 +70,4 @@ Everything you should know about how to run end-to-end tests using Everything you should know about how to test migrations. -[Return to Development documentation](../README.md) +[Return to Development documentation](../index.md) diff --git a/doc/gitlab-basics/create-your-ssh-keys.md b/doc/gitlab-basics/create-your-ssh-keys.md index a99307e6dbf..673fb2911fa 100644 --- a/doc/gitlab-basics/create-your-ssh-keys.md +++ b/doc/gitlab-basics/create-your-ssh-keys.md @@ -1,9 +1,9 @@ --- -redirect_to: '../ssh/README.md' +redirect_to: '../ssh/index.md' remove_date: '2021-07-04' --- -This document was moved to [another location](../ssh/README.md). +This document was moved to [another location](../ssh/index.md). <!-- This redirect file can be deleted after <2021-07-04>. --> <!-- Before deletion, see: https://docs.gitlab.com/ee/development/documentation/#move-or-rename-a-page -->
\ No newline at end of file diff --git a/doc/gitlab-basics/index.md b/doc/gitlab-basics/index.md index 774931aad72..d2d0c4fad39 100644 --- a/doc/gitlab-basics/index.md +++ b/doc/gitlab-basics/index.md @@ -21,7 +21,7 @@ This documentation is split into the following groups: The following are guides to basic GitLab functionality: -- [Create and add your SSH public key](../ssh/README.md), for enabling Git over SSH. +- [Create and add your SSH public key](../ssh/index.md), for enabling Git over SSH. - [Create a project](../user/project/working_with_projects.md#create-a-project), to start using GitLab. - [Create a group](../user/group/index.md#create-a-group), to combine and administer projects together. diff --git a/doc/gitlab-basics/start-using-git.md b/doc/gitlab-basics/start-using-git.md index f9623586e55..9b26e1f102c 100644 --- a/doc/gitlab-basics/start-using-git.md +++ b/doc/gitlab-basics/start-using-git.md @@ -182,7 +182,7 @@ This connection requires you to add credentials. You can either use SSH or HTTPS Clone with SSH when you want to authenticate only one time. -1. Authenticate with GitLab by following the instructions in the [SSH documentation](../ssh/README.md). +1. Authenticate with GitLab by following the instructions in the [SSH documentation](../ssh/index.md). 1. Go to your project's landing page and select **Clone**. Copy the URL for **Clone with SSH**. 1. Open a terminal and go to the directory where you want to clone the files. Git automatically creates a folder with the repository name and downloads the files there. 1. Run this command: diff --git a/doc/index.md b/doc/index.md new file mode 100644 index 00000000000..cdb1114ca40 --- /dev/null +++ b/doc/index.md @@ -0,0 +1,126 @@ +--- +stage: none +group: unassigned +info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/engineering/ux/technical-writing/#assignments +comments: false +description: 'Learn how to use and administer GitLab, the most scalable Git-based fully integrated platform for software development.' +--- + +<div class="d-none"> + <h3>Visit <a href="https://docs.gitlab.com/ee/">docs.gitlab.com</a> for the latest version + of this help information with enhanced navigation, discoverability, and readability.</h3> +</div> +<!-- the div above will not display on the docs site but will display on /help --> + +# GitLab Docs + +Welcome to [GitLab](https://about.gitlab.com/) documentation. + +Here you can access the complete documentation for GitLab, the single application for the +[entire DevOps lifecycle](#the-entire-devops-lifecycle). + +## Overview + +No matter how you use GitLab, we have documentation for you. + +| Essential documentation | Essential documentation | +|:------------------------|:------------------------| +| [**User documentation**](user/index.md)<br>Discover features and concepts for GitLab users. | [**Administrator documentation**](administration/index.md)<br/>Everything GitLab self-managed administrators need to know. | +| [**Contributing to GitLab**](#contributing-to-gitlab)<br/>At GitLab, everyone can contribute! | [**New to Git and GitLab?**](#new-to-git-and-gitlab)<br/>We have the resources to get you started. | +| [**Build an integration with GitLab**](#build-an-integration-with-gitlab)<br/>Consult our integration documentation. | [**Coming to GitLab from another platform?**](#coming-to-gitlab-from-another-platform)<br/>Consult our guides. | +| [**Install GitLab**](https://about.gitlab.com/install/)<br/>Installation options for different platforms. | [**Customers**](subscriptions/index.md)<br/>Information for new and existing customers. | +| [**Update GitLab**](update/index.md)<br/>Update your GitLab self-managed instance to the latest version. | [**Reference Architectures**](administration/reference_architectures/index.md)<br/>GitLab reference architectures. | +| [**GitLab releases**](https://about.gitlab.com/releases/)<br/>What's new in GitLab. | | + +## Popular topics + +Have a look at some of our most popular topics: + +| Popular topic | Description | +|:-------------------------------------------------------------------------------------------|:------------| +| [Two-factor authentication](user/profile/account/two_factor_authentication.md) | Improve the security of your GitLab account. | +| [GitLab groups](user/group/index.md) | Manage projects together. | +| [GitLab CI/CD pipeline configuration reference](ci/yaml/README.md) | Available configuration options for `.gitlab-ci.yml` files. | +| [Activate GitLab EE with a license](user/admin_area/license.md) | Activate GitLab Enterprise Edition functionality with a license. | +| [Back up and restore GitLab](raketasks/backup_restore.md) | Rake tasks for backing up and restoring GitLab self-managed instances. | +| [GitLab release and maintenance policy](policy/maintenance.md) | Policies for version naming and cadence, and also upgrade recommendations. | +| [Elasticsearch integration](integration/elasticsearch.md) | Integrate Elasticsearch with GitLab to enable advanced searching. | +| [Omnibus GitLab database settings](https://docs.gitlab.com/omnibus/settings/database.html) | Database settings for Omnibus GitLab self-managed instances. | +| [Omnibus GitLab NGINX settings](https://docs.gitlab.com/omnibus/settings/nginx.html) | NGINX settings for Omnibus GitLab self-managed instances. | +| [Omnibus GitLab SSL configuration](https://docs.gitlab.com/omnibus/settings/ssl.html) | SSL settings for Omnibus GitLab self-managed instances. | +| [GitLab.com settings](user/gitlab_com/index.md) | Settings used for GitLab.com. | + +## The entire DevOps lifecycle + +GitLab is the first single application for software development, security, +and operations that enables [Concurrent DevOps](https://about.gitlab.com/topics/concurrent-devops/). +GitLab makes the software lifecycle faster and radically improves the speed of business. + +GitLab provides solutions for [each of the stages of the DevOps lifecycle](https://about.gitlab.com/stages-devops-lifecycle/). + +## New to Git and GitLab? + +Working with new systems can be daunting. + +We have the following documentation to rapidly uplift your GitLab knowledge: + +| Topic | Description | +|:--------------------------------------------------------------------------------------------------|:------------| +| [GitLab basics guides](gitlab-basics/index.md) | Start working on the command line and with GitLab. | +| [GitLab workflow overview](https://about.gitlab.com/topics/version-control/what-is-gitlab-workflow/) | Enhance your workflow with the best of GitLab Workflow. | +| [Get started with GitLab CI/CD](ci/quick_start/index.md) | Quickly implement GitLab CI/CD. | +| [Auto DevOps](topics/autodevops/index.md) | Learn more about Auto DevOps in GitLab. | +| [GitLab Markdown](user/markdown.md) | Advanced formatting system (GitLab Flavored Markdown). | + +### User account + +Learn more about GitLab account management: + +| Topic | Description | +|:-----------------------------------------------------------|:------------| +| [User account](user/profile/index.md) | Manage your account. | +| [Authentication](topics/authentication/index.md) | Account security with two-factor authentication, set up your SSH keys, and deploy keys for secure access to your projects. | +| [User settings](user/profile/index.md#access-your-user-settings) | Manage your user settings, two factor authentication, and more. | +| [User permissions](user/permissions.md) | Learn what each role in a project can do. | + +### Git and GitLab + +Learn more about using Git, and using Git with GitLab: + +| Topic | Description | +|:-----------------------------------------------------------------------------|:------------| +| [Git](topics/git/index.md) | Getting started with Git, branching strategies, Git LFS, and advanced use. | +| [Git cheat sheet](https://about.gitlab.com/images/press/git-cheat-sheet.pdf) | Download a PDF describing the most used Git operations. | +| [GitLab Flow](topics/gitlab_flow.md) | Explore the best of Git with the GitLab Flow strategy. | + +## Coming to GitLab from another platform + +If you are coming to GitLab from another platform, the following information is useful: + +| Topic | Description | +|:----------------------------------------------------|:------------| +| [Importing to GitLab](user/project/import/index.md) | Import your projects from GitHub, Bitbucket, GitLab.com, FogBugz, and SVN into GitLab. | +| [Migrating from SVN](user/project/import/svn.md) | Convert a SVN repository to Git and GitLab. | + +## Build an integration with GitLab + +There are many ways to integrate with GitLab, including: + +| Topic | Description | +|:-------------------------------------------|:------------| +| [GitLab REST API](api/README.md) | Integrate with GitLab using our REST API. | +| [GitLab GraphQL API](api/graphql/index.md) | Integrate with GitLab using our GraphQL API. | +| [Integrations](integration/index.md) | Integrations with third-party products. | + +## Contributing to GitLab + +GitLab Community Edition is [open source](https://gitlab.com/gitlab-org/gitlab-foss/) +and GitLab Enterprise Edition is [open-core](https://gitlab.com/gitlab-org/gitlab/). + +Learn how to contribute to GitLab with the following resources: + +| Topic | Description | +|:------------------------------------------------------------|:------------| +| [Development](development/index.md) | How to contribute to GitLab development. | +| [Legal](legal/index.md) | Contributor license agreements. | +| [Writing documentation](development/documentation/index.md) | How to contribute to GitLab Docs. | diff --git a/doc/install/azure/index.md b/doc/install/azure/index.md index 1351489642e..1d31337fc39 100644 --- a/doc/install/azure/index.md +++ b/doc/install/azure/index.md @@ -71,7 +71,7 @@ The first items you need to configure are the basic settings of the underlying v the user Azure uses to connect to the VM through SSH. By default, the user has root access. 1. Determine if you want to provide your own SSH key or let Azure create one for you. - Read the [SSH documentation](../../ssh/README.md) to learn more about how to set up SSH + Read the [SSH documentation](../../ssh/index.md) to learn more about how to set up SSH public keys. Review your entered settings, and then proceed to the Disks tab. diff --git a/doc/install/docker.md b/doc/install/docker.md index 064c93776a8..a6bc18bd374 100644 --- a/doc/install/docker.md +++ b/doc/install/docker.md @@ -2,17 +2,638 @@ stage: Enablement group: Distribution info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/engineering/ux/technical-writing/#assignments -type: index --- -# Install GitLab with Docker **(FREE SELF)** +# GitLab Docker images -[Docker](https://www.docker.com) and container technology have been revolutionizing the software world for the past few years. They combine the performance and efficiency of native execution with the abstraction, security, and immutability of virtualization. +The GitLab Docker images are monolithic images of GitLab running all the +necessary services in a single container. If you instead want to install GitLab +on Kubernetes, see [GitLab Helm Charts](https://docs.gitlab.com/charts/). -GitLab provides official Docker images allowing you to easily take advantage of the benefits of containerization while operating your GitLab instance. A [complete usage guide](https://docs.gitlab.com/omnibus/docker/) for these images is available, as well as the [Dockerfile used for building the images](https://gitlab.com/gitlab-org/omnibus-gitlab/tree/master/docker). +Find the GitLab official Docker image at: -There's also a [Docker image for GitLab Runner](https://docs.gitlab.com/runner/install/docker.html). +- [GitLab Docker image in Docker Hub](https://hub.docker.com/r/gitlab/gitlab-ee/) -## Cloud native images +The Docker images don't include a mail transport agent (MTA). The recommended +solution is to add an MTA (such as Postfix or Sendmail) running in a separate +container. As another option, you can install an MTA directly in the GitLab +container, but this adds maintenance overhead as you'll likely need to reinstall +the MTA after every upgrade or restart. -GitLab is also working towards a [cloud native set of containers](https://docs.gitlab.com/charts/), with a single image for each component service. +In the following examples, if you want to use the latest RC image, use +`gitlab/gitlab-ee:rc` instead. + +WARNING: +Docker for Windows is not officially supported. There are known issues with volume +permissions, and potentially other unknown issues. If you are trying to run on Docker +for Windows, see the [getting help page](https://about.gitlab.com/get-help/) for links +to community resources (IRC, forum, etc.) to seek help from other users. + +## Prerequisites + +Docker is required. See the [official installation documentation](https://docs.docker.com/install/). + +## Set up the volumes location + +Before setting everything else, configure a new environment variable `$GITLAB_HOME` +pointing to the directory where the configuration, logs, and data files will reside. +Ensure that the directory exists and appropriate permission have been granted. + +For Linux users, set the path to `/srv/gitlab`: + +```shell +export GITLAB_HOME=/srv/gitlab +``` + +For macOS users, use the user's `$HOME/gitlab` directory: + +```shell +export GITLAB_HOME=$HOME/gitlab +``` + +The GitLab container uses host mounted volumes to store persistent data: + +| Local location | Container location | Usage | +|----------------------|--------------------|---------------------------------------------| +| `$GITLAB_HOME/data` | `/var/opt/gitlab` | For storing application data. | +| `$GITLAB_HOME/logs` | `/var/log/gitlab` | For storing logs. | +| `$GITLAB_HOME/config`| `/etc/gitlab` | For storing the GitLab configuration files. | + +## Installation + +The GitLab Docker images can be run in multiple ways: + +- [Using Docker Engine](#install-gitlab-using-docker-engine) +- [Using Docker Compose](#install-gitlab-using-docker-compose) +- [Using Docker swarm mode](#install-gitlab-using-docker-swarm-mode) + +### Install GitLab using Docker Engine + +You can fine tune these directories to meet your requirements. +Once you've set up the `GITLAB_HOME` variable, you can run the image: + +```shell +sudo docker run --detach \ + --hostname gitlab.example.com \ + --publish 443:443 --publish 80:80 --publish 22:22 \ + --name gitlab \ + --restart always \ + --volume $GITLAB_HOME/config:/etc/gitlab \ + --volume $GITLAB_HOME/logs:/var/log/gitlab \ + --volume $GITLAB_HOME/data:/var/opt/gitlab \ + gitlab/gitlab-ee:latest +``` + +This will download and start a GitLab container and publish ports needed to +access SSH, HTTP and HTTPS. All GitLab data will be stored as subdirectories of +`$GITLAB_HOME`. The container will automatically `restart` after a system reboot. + +If you are on SELinux, then run this instead: + +```shell +sudo docker run --detach \ + --hostname gitlab.example.com \ + --publish 443:443 --publish 80:80 --publish 22:22 \ + --name gitlab \ + --restart always \ + --volume $GITLAB_HOME/config:/etc/gitlab:Z \ + --volume $GITLAB_HOME/logs:/var/log/gitlab:Z \ + --volume $GITLAB_HOME/data:/var/opt/gitlab:Z \ + gitlab/gitlab-ee:latest +``` + +This will ensure that the Docker process has enough permissions to create the +config files in the mounted volumes. + +If you're using the [Kerberos integration](../integration/kerberos.md) **(PREMIUM ONLY)**, +you must also publish your Kerberos port (for example, `--publish 8443:8443`). +Failing to do so prevents Git operations with Kerberos. + +The initialization process may take a long time. You can track this +process with: + +```shell +sudo docker logs -f gitlab +``` + +After starting a container you can visit `gitlab.example.com` (or +`http://192.168.59.103` if you used boot2docker on macOS). It might take a while +before the Docker container starts to respond to queries. +The very first time you visit GitLab, you will be asked to set up the admin +password. After you change it, you can log in with username `root` and the +password you set up. + +### Install GitLab using Docker Compose + +With [Docker Compose](https://docs.docker.com/compose/) you can easily configure, +install, and upgrade your Docker-based GitLab installation: + +1. [Install Docker Compose](https://docs.docker.com/compose/install/). +1. Create a `docker-compose.yml` file (or [download an example](https://gitlab.com/gitlab-org/omnibus-gitlab/raw/master/docker/docker-compose.yml)): + + ```yaml + web: + image: 'gitlab/gitlab-ee:latest' + restart: always + hostname: 'gitlab.example.com' + environment: + GITLAB_OMNIBUS_CONFIG: | + external_url 'https://gitlab.example.com' + # Add any other gitlab.rb configuration here, each on its own line + ports: + - '80:80' + - '443:443' + - '22:22' + volumes: + - '$GITLAB_HOME/config:/etc/gitlab' + - '$GITLAB_HOME/logs:/var/log/gitlab' + - '$GITLAB_HOME/data:/var/opt/gitlab' + ``` + +1. Make sure you are in the same directory as `docker-compose.yml` and start + GitLab: + + ```shell + docker-compose up -d + ``` + +NOTE: +Read the ["Pre-configure Docker container"](#pre-configure-docker-container) section +to see how the `GITLAB_OMNIBUS_CONFIG` variable works. + +Below is another `docker-compose.yml` example with GitLab running on a custom +HTTP and SSH port. Notice how the `GITLAB_OMNIBUS_CONFIG` variables match the +`ports` section: + +```yaml +web: + image: 'gitlab/gitlab-ee:latest' + restart: always + hostname: 'gitlab.example.com' + environment: + GITLAB_OMNIBUS_CONFIG: | + external_url 'http://gitlab.example.com:8929' + gitlab_rails['gitlab_shell_ssh_port'] = 2224 + ports: + - '8929:8929' + - '2224:22' + volumes: + - '$GITLAB_HOME/config:/etc/gitlab' + - '$GITLAB_HOME/logs:/var/log/gitlab' + - '$GITLAB_HOME/data:/var/opt/gitlab' +``` + +This is the same as using `--publish 8929:8929 --publish 2224:22`. + +### Install GitLab using Docker swarm mode + +With [Docker swarm mode](https://docs.docker.com/engine/swarm/), you can easily +configure and deploy your +Docker-based GitLab installation in a swarm cluster. + +In swarm mode you can leverage [Docker secrets](https://docs.docker.com/engine/swarm/secrets/) +and [Docker configs](https://docs.docker.com/engine/swarm/configs/) to efficiently and securely deploy your GitLab instance. +Secrets can be used to securely pass your initial root password without exposing it as an environment variable. +Configs can help you to keep your GitLab image as generic as possible. + +Here's an example that deploys GitLab with four runners as a [stack](https://docs.docker.com/get-started/part5/), using secrets and configs: + +1. [Set up a Docker swarm](https://docs.docker.com/engine/swarm/swarm-tutorial/). +1. Create a `docker-compose.yml` file: + + ```yaml + version: "3.6" + services: + gitlab: + image: gitlab/gitlab-ee:latest + ports: + - "22:22" + - "80:80" + - "443:443" + volumes: + - $GITLAB_HOME/data:/var/opt/gitlab + - $GITLAB_HOME/logs:/var/log/gitlab + - $GITLAB_HOME/config:/etc/gitlab + environment: + GITLAB_OMNIBUS_CONFIG: "from_file('/omnibus_config.rb')" + configs: + - source: gitlab + target: /omnibus_config.rb + secrets: + - gitlab_root_password + gitlab-runner: + image: gitlab/gitlab-runner:alpine + deploy: + mode: replicated + replicas: 4 + configs: + gitlab: + file: ./gitlab.rb + secrets: + gitlab_root_password: + file: ./root_password.txt + ``` + + For simplicity reasons, the `network` configuration was omitted. + More information can be found in the official [Compose file reference](https://docs.docker.com/compose/compose-file/). + +1. Create a `gitlab.rb` file: + + ```ruby + external_url 'https://my.domain.com/' + gitlab_rails['initial_root_password'] = File.read('/run/secrets/gitlab_root_password') + ``` + +1. Create a `root_password.txt` file: + + ```plaintext + MySuperSecretAndSecurePass0rd! + ``` + +1. Make sure you are in the same directory as `docker-compose.yml` and run: + + ```shell + docker stack deploy --compose-file docker-compose.yml mystack + ``` + +## Configuration + +This container uses the official Omnibus GitLab package, so all configuration +is done in the unique configuration file `/etc/gitlab/gitlab.rb`. + +To access the GitLab configuration file, you can start a shell session in the +context of a running container. This will allow you to browse all directories +and use your favorite text editor: + +```shell +sudo docker exec -it gitlab /bin/bash +``` + +You can also just edit `/etc/gitlab/gitlab.rb`: + +```shell +sudo docker exec -it gitlab editor /etc/gitlab/gitlab.rb +``` + +Once you open `/etc/gitlab/gitlab.rb` make sure to set the `external_url` to +point to a valid URL. + +To receive e-mails from GitLab you have to configure the +[SMTP settings](https://docs.gitlab.com/omnibus/settings/smtp.html) because the GitLab Docker image doesn't +have an SMTP server installed. You may also be interested in +[enabling HTTPS](https://docs.gitlab.com/omnibus/settings/nginx.html#enable-https). + +After you make all the changes you want, you will need to restart the container +in order to reconfigure GitLab: + +```shell +sudo docker restart gitlab +``` + +GitLab will reconfigure itself whenever the container starts. +For more options about configuring GitLab, check the +[configuration documentation](https://docs.gitlab.com/omnibus/settings/configuration.html). + +### Pre-configure Docker container + +You can pre-configure the GitLab Docker image by adding the environment variable +`GITLAB_OMNIBUS_CONFIG` to Docker run command. This variable can contain any +`gitlab.rb` setting and is evaluated before the loading of the container's +`gitlab.rb` file. This behavior allows you to configure the external GitLab URL, +and make database configuration or any other option from the +[Omnibus GitLab template](https://gitlab.com/gitlab-org/omnibus-gitlab/blob/master/files/gitlab-config-template/gitlab.rb.template). +The settings contained in `GITLAB_OMNIBUS_CONFIG` aren't written to the +`gitlab.rb` configuration file, and are evaluated on load. + +Here's an example that sets the external URL and enables LFS while starting +the container: + +```shell +sudo docker run --detach \ + --hostname gitlab.example.com \ + --env GITLAB_OMNIBUS_CONFIG="external_url 'http://my.domain.com/'; gitlab_rails['lfs_enabled'] = true;" \ + --publish 443:443 --publish 80:80 --publish 22:22 \ + --name gitlab \ + --restart always \ + --volume $GITLAB_HOME/config:/etc/gitlab \ + --volume $GITLAB_HOME/logs:/var/log/gitlab \ + --volume $GITLAB_HOME/data:/var/opt/gitlab \ + gitlab/gitlab-ee:latest +``` + +Note that every time you execute a `docker run` command, you need to provide +the `GITLAB_OMNIBUS_CONFIG` option. The content of `GITLAB_OMNIBUS_CONFIG` is +_not_ preserved between subsequent runs. + +### Use tagged versions of GitLab + +Tagged versions of the GitLab Docker images are also provided. +To see all available tags see: + +- [GitLab CE tags](https://hub.docker.com/r/gitlab/gitlab-ce/tags/) +- [GitLab EE tags](https://hub.docker.com/r/gitlab/gitlab-ee/tags/) + +To use a specific tagged version, replace `gitlab/gitlab-ee:latest` with +the GitLab version you want to run, for example `gitlab/gitlab-ee:12.1.3-ce.0`. + +### Run GitLab on a public IP address + +You can make Docker to use your IP address and forward all traffic to the +GitLab container by modifying the `--publish` flag. + +To expose GitLab on IP `198.51.100.1`: + +```shell +sudo docker run --detach \ + --hostname gitlab.example.com \ + --publish 198.51.100.1:443:443 \ + --publish 198.51.100.1:80:80 \ + --publish 198.51.100.1:22:22 \ + --name gitlab \ + --restart always \ + --volume $GITLAB_HOME/config:/etc/gitlab \ + --volume $GITLAB_HOME/logs:/var/log/gitlab \ + --volume $GITLAB_HOME/data:/var/opt/gitlab \ + gitlab/gitlab-ee:latest +``` + +You can then access your GitLab instance at `http://198.51.100.1/` and `https://198.51.100.1/`. + +### Expose GitLab on different ports + +GitLab will occupy [some ports](https://docs.gitlab.com/omnibus/package-information/defaults.html) +inside the container. + +If you want to use a different host port than `80` (HTTP) or `443` (HTTPS), +you need to add a separate `--publish` directive to the `docker run` command. + +For example, to expose the web interface on the host's port `8929`, and the SSH service on +port `2289`: + +1. Use the following `docker run` command: + + ```shell + sudo docker run --detach \ + --hostname gitlab.example.com \ + --publish 8929:8929 --publish 2289:22 \ + --name gitlab \ + --restart always \ + --volume $GITLAB_HOME/config:/etc/gitlab \ + --volume $GITLAB_HOME/logs:/var/log/gitlab \ + --volume $GITLAB_HOME/data:/var/opt/gitlab \ + gitlab/gitlab-ee:latest + ``` + + NOTE: + The format for publishing ports is `hostPort:containerPort`. Read more in + Docker's documentation about + [exposing incoming ports](https://docs.docker.com/engine/reference/run/#/expose-incoming-ports). + +1. Enter the running container: + + ```shell + sudo docker exec -it gitlab /bin/bash + ``` + +1. Open `/etc/gitlab/gitlab.rb` with your editor and set `external_url`: + + ```ruby + # For HTTP + external_url "http://gitlab.example.com:8929" + + or + + # For HTTPS (notice the https) + external_url "https://gitlab.example.com:8929" + ``` + + The port specified in this URL must match the port published to the host by Docker. + Additionally, if the NGINX listen port is not explicitly set in + `nginx['listen_port']`, it will be pulled from the `external_url`. + For more information see the [NGINX documentation](https://docs.gitlab.com/omnibus/settings/nginx.html). + +1. Set `gitlab_shell_ssh_port`: + + ```ruby + gitlab_rails['gitlab_shell_ssh_port'] = 2289 + ``` + +1. Finally, reconfigure GitLab: + + ```shell + gitlab-ctl reconfigure + ``` + +Following the above example, you will be able to reach GitLab from your +web browser under `<hostIP>:8929` and push using SSH under the port `2289`. + +A `docker-compose.yml` example that uses different ports can be found in the +[Docker compose](#install-gitlab-using-docker-compose) section. + +## Update + +In most cases, updating GitLab is as easy as downloading the newest Docker +[image tag](#use-tagged-versions-of-gitlab). + +### Update GitLab using Docker Engine + +To update GitLab that was [installed using Docker Engine](#install-gitlab-using-docker-engine): + +1. Take a [backup](#back-up-gitlab). +1. Stop the running container: + + ```shell + sudo docker stop gitlab + ``` + +1. Remove the existing container: + + ```shell + sudo docker rm gitlab + ``` + +1. Pull the new image. For example, the latest GitLab image: + + ```shell + sudo docker pull gitlab/gitlab-ee:latest + ``` + +1. Create the container once again with the +[previously specified](#install-gitlab-using-docker-engine) options: + + ```shell + sudo docker run --detach \ + --hostname gitlab.example.com \ + --publish 443:443 --publish 80:80 --publish 22:22 \ + --name gitlab \ + --restart always \ + --volume $GITLAB_HOME/config:/etc/gitlab \ + --volume $GITLAB_HOME/logs:/var/log/gitlab \ + --volume $GITLAB_HOME/data:/var/opt/gitlab \ + gitlab/gitlab-ee:latest + ``` + +On the first run, GitLab will reconfigure and update itself. + +Refer to the GitLab [Update recommendations](../policy/maintenance.md#upgrade-recommendations) +when upgrading between major versions. + +### Update GitLab using Docker compose + +To update GitLab that was [installed using Docker Compose](#install-gitlab-using-docker-compose): + +1. Take a [backup](#back-up-gitlab). +1. Download the newest release and update your GitLab instance: + + ```shell + docker-compose pull + docker-compose up -d + ``` + + If you have used [tags](#use-tagged-versions-of-gitlab) instead, you'll need + to first edit `docker-compose.yml`. + +## Back up GitLab + +You can create a GitLab backup with: + +```shell +docker exec -t <container name> gitlab-backup create +``` + +Read more on how to [back up and restore GitLab](../raketasks/backup_restore.md). + +NOTE: +If configuration is provided entirely via the `GITLAB_OMNIBUS_CONFIG` environment variable +(per the ["Pre-configure Docker Container"](#pre-configure-docker-container) steps), +meaning no configuration is set directly in the `gitlab.rb` file, then there is no need +to back up the `gitlab.rb` file. + +## Installing GitLab Community Edition + +[GitLab CE Docker image](https://hub.docker.com/r/gitlab/gitlab-ce/) + +To install the Community Edition, replace `ee` with `ce` in the commands on this +page. + +## Troubleshooting + +The following information will help if you encounter problems using Omnibus GitLab and Docker. + +### Diagnose potential problems + +Read container logs: + +```shell +sudo docker logs gitlab +``` + +Enter running container: + +```shell +sudo docker exec -it gitlab /bin/bash +``` + +From within the container you can administer the GitLab container as you would +normally administer an +[Omnibus installation](https://gitlab.com/gitlab-org/omnibus-gitlab/blob/master/README.md) + +### 500 Internal Error + +When updating the Docker image you may encounter an issue where all paths +display a `500` page. If this occurs, restart the container to try to rectify the +issue: + +```shell +sudo docker restart gitlab +``` + +### Permission problems + +When updating from older GitLab Docker images you might encounter permission +problems. This happens when users in previous images were not +preserved correctly. There's script that fixes permissions for all files. + +To fix your container, execute `update-permissions` and restart the +container afterwards: + +```shell +sudo docker exec gitlab update-permissions +sudo docker restart gitlab +``` + +### Windows/Mac: `Error executing action run on resource ruby_block[directory resource: /data/GitLab]` + +This error occurs when using Docker Toolbox with VirtualBox on Windows or Mac, +and making use of Docker volumes. The `/c/Users` volume is mounted as a +VirtualBox Shared Folder, and does not support the all POSIX filesystem features. +The directory ownership and permissions cannot be changed without remounting, and +GitLab fails. + +Our recommendation is to switch to using the native Docker install for your +platform, instead of using Docker Toolbox. + +If you cannot use the native Docker install (Windows 10 Home Edition, or Windows 7/8), +then an alternative solution is to setup NFS mounts instead of VirtualBox shares for +Docker Toolbox's boot2docker. + +### Linux ACL issues + +If you are using file ACLs on the Docker host, the `docker` group requires full access to the volumes in order for GitLab to work: + +```shell +getfacl $GITLAB_HOME + +# file: $GITLAB_HOME +# owner: XXXX +# group: XXXX +user::rwx +group::rwx +group:docker:rwx +mask::rwx +default:user::rwx +default:group::rwx +default:group:docker:rwx +default:mask::rwx +default:other::r-x +``` + +If these are not correct, set them with: + +```shell +sudo setfacl -mR default:group:docker:rwx $GITLAB_HOME +``` + +The default group is `docker`. If you changed the group, be sure to update your +commands. + +### /dev/shm mount not having enough space in Docker container + +GitLab comes with a Prometheus metrics endpoint at `/-/metrics` to expose a +variety of statistics on the health and performance of GitLab. The files +required for this gets written to a temporary file system (like `/run` or +`/dev/shm`). + +By default, Docker allocates 64Mb to the shared memory directory (mounted at +`/dev/shm`). This is insufficient to hold all the Prometheus metrics related +files generated, and will generate error logs like the following: + +```plaintext +writing value to /dev/shm/gitlab/sidekiq/gauge_all_sidekiq_0-1.db failed with unmapped file +writing value to /dev/shm/gitlab/sidekiq/gauge_all_sidekiq_0-1.db failed with unmapped file +writing value to /dev/shm/gitlab/sidekiq/gauge_all_sidekiq_0-1.db failed with unmapped file +writing value to /dev/shm/gitlab/sidekiq/histogram_sidekiq_0-0.db failed with unmapped file +writing value to /dev/shm/gitlab/sidekiq/histogram_sidekiq_0-0.db failed with unmapped file +writing value to /dev/shm/gitlab/sidekiq/histogram_sidekiq_0-0.db failed with unmapped file +writing value to /dev/shm/gitlab/sidekiq/histogram_sidekiq_0-0.db failed with unmapped file +``` + +Other than disabling the Prometheus Metrics from the Admin page, the recommended +solution to fix this problem is to increase the size of shm to at least 256Mb. +If using `docker run`, this can be done by passing the flag `--shm-size 256m`. +If using a `docker-compose.yml` file, the `shm_size` key can be used for this +purpose. + +### Docker containers exhausts space due to the `json-file` + +Docker's [default logging driver is `json-file`](https://docs.docker.com/config/containers/logging/configure/#configure-the-default-logging-driver), which performs no log rotation by default. As a result of this lack of rotation, log files stored by the `json-file` driver can consume a significant amount of disk space for containers that generate a lot of output. This can lead to disk space exhaustion. To address this, use [journald](https://docs.docker.com/config/containers/logging/journald/) as the logging driver when available, or [another supported driver](https://docs.docker.com/config/containers/logging/configure/#supported-logging-drivers) with native rotation support. diff --git a/doc/install/next_steps.md b/doc/install/next_steps.md index 4e4f1f01a08..f271caef493 100644 --- a/doc/install/next_steps.md +++ b/doc/install/next_steps.md @@ -26,7 +26,7 @@ installation. ## Security -- [Secure GitLab](../security/README.md#securing-your-gitlab-installation): +- [Secure GitLab](../security/index.md#securing-your-gitlab-installation): Recommended practices to secure your GitLab instance. - Sign up for the GitLab [Security Newsletter](https://about.gitlab.com/company/preference-center/) to get notified for security updates upon release. diff --git a/doc/migrate_ci_to_ce/index.md b/doc/migrate_ci_to_ce/index.md new file mode 100644 index 00000000000..dbe5a2730b5 --- /dev/null +++ b/doc/migrate_ci_to_ce/index.md @@ -0,0 +1,9 @@ +--- +redirect_to: 'https://docs.gitlab.com/' +remove_date: '2021-06-14' +--- + +This document was moved to [another location](https://docs.gitlab.com/). + +<!-- This redirect file can be deleted after <2021-09-14>. --> +<!-- Before deletion, see: https://docs.gitlab.com/ee/development/documentation/#move-or-rename-a-page --> diff --git a/doc/raketasks/backup_restore.md b/doc/raketasks/backup_restore.md index 1e130c67724..dc408fc8157 100644 --- a/doc/raketasks/backup_restore.md +++ b/doc/raketasks/backup_restore.md @@ -1192,7 +1192,7 @@ decrypt those columns, preventing access to the following items: - [Kubernetes / GCP integration](../user/project/clusters/index.md) - [Custom Pages domains](../user/project/pages/custom_domains_ssl_tls_certification/index.md) - [Project error tracking](../operations/error_tracking.md) -- [Runner authentication](../ci/runners/README.md) +- [Runner authentication](../ci/runners/index.md) - [Project mirroring](../user/project/repository/repository_mirroring.md) - [Web hooks](../user/project/integrations/webhooks.md) diff --git a/doc/security/README.md b/doc/security/README.md index 6af3948fdcf..5ab8653dc35 100644 --- a/doc/security/README.md +++ b/doc/security/README.md @@ -1,32 +1,8 @@ --- -stage: none -group: unassigned -info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/engineering/ux/technical-writing/#assignments -comments: false -type: index +redirect_to: 'index.md' --- -# Security **(FREE)** +This document was moved to [another location](index.md). -- [Password storage](password_storage.md) -- [Password length limits](password_length_limits.md) -- [Generated passwords for users created through integrated authentication](passwords_for_integrated_authentication_methods.md) -- [Restrict SSH key technologies and minimum length](ssh_keys_restrictions.md) -- [Rate limits](rate_limits.md) -- [Webhooks and insecure internal web services](webhooks.md) -- [Information exclusivity](information_exclusivity.md) -- [Reset user password](reset_user_password.md) -- [Unlock a locked user](unlock_user.md) -- [User File Uploads](user_file_uploads.md) -- [How we manage the CRIME vulnerability](crime_vulnerability.md) -- [Enforce Two-factor authentication](two_factor_authentication.md) -- [Send email confirmation on sign-up](user_email_confirmation.md) -- [Security of running jobs](https://docs.gitlab.com/runner/security/) -- [Proxying images](asset_proxy.md) -- [CI/CD variables](../ci/variables/README.md#cicd-variable-security) -- [Token overview](token_overview.md) -- [Project Import decompressed archive size limits](project_import_decompressed_archive_size_limits.md) - -## Securing your GitLab installation - -Consider access control features like [Sign up restrictions](../user/admin_area/settings/sign_up_restrictions.md) and [Authentication options](../topics/authentication/) to harden your GitLab instance and minimize the risk of unwanted user account creation. +<!-- This redirect file can be deleted after 2021-09-28. --> +<!-- Before deletion, see: https://docs.gitlab.com/ee/development/documentation/#move-or-rename-a-page --> diff --git a/doc/security/index.md b/doc/security/index.md new file mode 100644 index 00000000000..6af3948fdcf --- /dev/null +++ b/doc/security/index.md @@ -0,0 +1,32 @@ +--- +stage: none +group: unassigned +info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/engineering/ux/technical-writing/#assignments +comments: false +type: index +--- + +# Security **(FREE)** + +- [Password storage](password_storage.md) +- [Password length limits](password_length_limits.md) +- [Generated passwords for users created through integrated authentication](passwords_for_integrated_authentication_methods.md) +- [Restrict SSH key technologies and minimum length](ssh_keys_restrictions.md) +- [Rate limits](rate_limits.md) +- [Webhooks and insecure internal web services](webhooks.md) +- [Information exclusivity](information_exclusivity.md) +- [Reset user password](reset_user_password.md) +- [Unlock a locked user](unlock_user.md) +- [User File Uploads](user_file_uploads.md) +- [How we manage the CRIME vulnerability](crime_vulnerability.md) +- [Enforce Two-factor authentication](two_factor_authentication.md) +- [Send email confirmation on sign-up](user_email_confirmation.md) +- [Security of running jobs](https://docs.gitlab.com/runner/security/) +- [Proxying images](asset_proxy.md) +- [CI/CD variables](../ci/variables/README.md#cicd-variable-security) +- [Token overview](token_overview.md) +- [Project Import decompressed archive size limits](project_import_decompressed_archive_size_limits.md) + +## Securing your GitLab installation + +Consider access control features like [Sign up restrictions](../user/admin_area/settings/sign_up_restrictions.md) and [Authentication options](../topics/authentication/) to harden your GitLab instance and minimize the risk of unwanted user account creation. diff --git a/doc/security/passwords_for_integrated_authentication_methods.md b/doc/security/passwords_for_integrated_authentication_methods.md index 7c4ada4435c..9931fd56e83 100644 --- a/doc/security/passwords_for_integrated_authentication_methods.md +++ b/doc/security/passwords_for_integrated_authentication_methods.md @@ -7,7 +7,7 @@ type: reference # Generated passwords for users created through integrated authentication **(FREE)** -GitLab allows users to set up accounts through integration with external [authentication and authorization providers](../administration/auth/README.md). +GitLab allows users to set up accounts through integration with external [authentication and authorization providers](../administration/auth/index.md). These authentication methods do not require the user to explicitly create a password for their accounts. However, to maintain data consistency, GitLab requires passwords for all user accounts. diff --git a/doc/ssh/README.md b/doc/ssh/README.md index 358323e4ef5..5ab8653dc35 100644 --- a/doc/ssh/README.md +++ b/doc/ssh/README.md @@ -1,385 +1,8 @@ --- -stage: Manage -group: Access -info: "To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/engineering/ux/technical-writing/#assignments" -type: howto, reference +redirect_to: 'index.md' --- -# GitLab and SSH keys +This document was moved to [another location](index.md). -Git is a distributed version control system, which means you can work locally, -then share or "push" your changes to a server. In this case, the server is GitLab. - -GitLab uses the SSH protocol to securely communicate with Git. -When you use SSH keys to authenticate to the GitLab remote server, -you don't need to supply your username and password each time. - -## Prerequisites - -To use SSH to communicate with GitLab, you need: - -- The OpenSSH client, which comes pre-installed on GNU/Linux, macOS, and Windows 10. -- SSH version 6.5 or later. Earlier versions used an MD5 signature, which is not secure. - -To view the version of SSH installed on your system, run `ssh -V`. - -## Supported SSH key types - -To communicate with GitLab, you can use the following SSH key types: - -- [ED25519](#ed25519-ssh-keys) -- [RSA](#rsa-ssh-keys) -- DSA ([Deprecated](https://about.gitlab.com/releases/2018/06/22/gitlab-11-0-released/#support-for-dsa-ssh-keys) in GitLab 11.0.) -- ECDSA (As noted in [Practical Cryptography With Go](https://leanpub.com/gocrypto/read#leanpub-auto-ecdsa), the security issues related to DSA also apply to ECDSA.) - -Administrators can [restrict which keys are permitted and their minimum lengths](../security/ssh_keys_restrictions.md). - -### ED25519 SSH keys - -The book [Practical Cryptography With Go](https://leanpub.com/gocrypto/read#leanpub-auto-chapter-5-digital-signatures) -suggests that [ED25519](https://ed25519.cr.yp.to/) keys are more secure and performant than RSA keys. - -OpenSSH 6.5 introduced ED25519 SSH keys in 2014 and they should be available on most -operating systems. - -### RSA SSH keys - -Available documentation suggests that ED25519 is more secure than RSA. - -If you use an RSA key, the US National Institute of Science and Technology in -[Publication 800-57 Part 3 (PDF)](https://nvlpubs.nist.gov/nistpubs/SpecialPublications/NIST.SP.800-57Pt3r1.pdf) -recommends a key size of at least 2048 bits. The default key size depends on your version of `ssh-keygen`. -Review the `man` page for your installed `ssh-keygen` command for details. - -## See if you have an existing SSH key pair - -Before you create a key pair, see if a key pair already exists. - -1. On Windows, Linux, or macOS, go to your home directory. -1. Go to the `.ssh/` subdirectory. If the `.ssh/` subdirectory doesn't exist, - you are either not in the home directory, or you haven't used `ssh` before. - In the latter case, you need to [generate an SSH key pair](#generate-an-ssh-key-pair). -1. See if a file with one of the following formats exists: - - | Algorithm | Public key | Private key | - | --------- | ---------- | ----------- | - | ED25519 (preferred) | `id_ed25519.pub` | `id_ed25519` | - | RSA (at least 2048-bit key size) | `id_rsa.pub` | `id_rsa` | - | DSA (deprecated) | `id_dsa.pub` | `id_dsa` | - | ECDSA | `id_ecdsa.pub` | `id_ecdsa` | - -## Generate an SSH key pair - -If you do not have an existing SSH key pair, generate a new one. - -1. Open a terminal. -1. Type `ssh-keygen -t` followed by the key type and an optional comment. - This comment is included in the `.pub` file that's created. - You may want to use an email address for the comment. - - For example, for ED25519: - - ```shell - ssh-keygen -t ed25519 -C "<comment>" - ``` - - For 2048-bit RSA: - - ```shell - ssh-keygen -t rsa -b 2048 -C "<comment>" - ``` - -1. Press Enter. Output similar to the following is displayed: - - ```plaintext - Generating public/private ed25519 key pair. - Enter file in which to save the key (/home/user/.ssh/id_ed25519): - ``` - -1. Accept the suggested filename and directory, unless you are generating a [deploy key](../user/project/deploy_keys/index.md) - or want to save in a specific directory where you store other keys. - - You can also dedicate the SSH key pair to a [specific host](#configure-ssh-to-point-to-a-different-directory). - -1. Specify a [passphrase](https://www.ssh.com/ssh/passphrase/): - - ```plaintext - Enter passphrase (empty for no passphrase): - Enter same passphrase again: - ``` - -1. A confirmation is displayed, including information about where your files are stored. - -A public and private key are generated. -[Add the public SSH key to your GitLab account](#add-an-ssh-key-to-your-gitlab-account) and keep -the private key secure. - -### Configure SSH to point to a different directory - -If you did not save your SSH key pair in the default directory, -configure your SSH client to point to the directory where the private key is stored. - -1. Open a terminal and run this command: - - ```shell - eval $(ssh-agent -s) - ssh-add <directory to private SSH key> - ``` - -1. Save these settings in the `~/.ssh/config` file. For example: - - ```conf - # GitLab.com - Host gitlab.com - PreferredAuthentications publickey - IdentityFile ~/.ssh/gitlab_com_rsa - - # Private GitLab instance - Host gitlab.company.com - PreferredAuthentications publickey - IdentityFile ~/.ssh/example_com_rsa - ``` - - For more information on these settings, see the [`man ssh_config`](https://man.openbsd.org/ssh_config) page in the SSH configuration manual. - -Public SSH keys must be unique to GitLab because they bind to your account. -Your SSH key is the only identifier you have when you push code with SSH. -It must uniquely map to a single user. - -### Update your SSH key passphrase - -You can update the passphrase for your SSH key. - -1. Open a terminal and run this command: - - ```shell - ssh-keygen -p -f /path/to/ssh_key - ``` - -1. At the prompts, type the passphrase and press Enter. - -### Upgrade your RSA key pair to a more secure format - -If your version of OpenSSH is between 6.5 and 7.8, -you can save your private RSA SSH keys in a more secure -OpenSSH format. - -1. Open a terminal and run this command: - - ```shell - ssh-keygen -o -f ~/.ssh/id_rsa - ``` - - Alternatively, you can generate a new RSA key with the more secure encryption format with - the following command: - - ```shell - ssh-keygen -o -t rsa -b 4096 -C "<comment>" - ``` - -## Add an SSH key to your GitLab account - -To use SSH with GitLab, copy your public key to your GitLab account. - -1. Copy the contents of your public key file. You can do this manually or use a script. - For example, to copy an ED25519 key to the clipboard: - - **macOS:** - - ```shell - tr -d '\n' < ~/.ssh/id_ed25519.pub | pbcopy - ``` - - **Linux** (requires the `xclip` package): - - ```shell - xclip -sel clip < ~/.ssh/id_ed25519.pub - ``` - - **Git Bash on Windows:** - - ```shell - cat ~/.ssh/id_ed25519.pub | clip - ``` - - Replace `id_ed25519.pub` with your filename. For example, use `id_rsa.pub` for RSA. - -1. Sign in to GitLab. -1. In the top right corner, select your avatar. -1. Select **Preferences**. -1. From the left sidebar, select **SSH Keys**. -1. In the **Key** box, paste the contents of your public key. - If you manually copied the key, make sure you copy the entire key, - which starts with `ssh-ed25519` or `ssh-rsa`, and may end with a comment. -1. In the **Title** text box, type a description, like _Work Laptop_ or - _Home Workstation_. -1. Optional. In the **Expires at** box, select an expiration date. (Introduced in [GitLab 12.9](https://gitlab.com/gitlab-org/gitlab/-/issues/36243).) - The expiration date is informational only, and does not prevent you from using - the key. However, administrators can view expiration dates and - use them for guidance when [deleting keys](../user/admin_area/credentials_inventory.md#delete-a-users-ssh-key). - - GitLab checks all SSH keys at 02:00 AM UTC every day. It emails an expiration notice for all SSH keys that expire on the current date. ([Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/322637) in GitLab 13.11.) - - GitLab checks all SSH keys at 01:00 AM UTC every day. It emails an expiration notice for all SSH keys that are scheduled to expire seven days from now. ([Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/322637) in GitLab 13.11.) -1. Select **Add key**. - -## Verify that you can connect - -Verify that your SSH key was added correctly. - -1. For GitLab.com, to ensure you're connecting to the correct server, confirm the - [SSH host keys fingerprints](../user/gitlab_com/index.md#ssh-host-keys-fingerprints). -1. Open a terminal and run this command, replacing `gitlab.example.com` with your GitLab instance URL: - - ```shell - ssh -T git@gitlab.example.com - ``` - -1. If this is the first time you connect, you should verify the - authenticity of the GitLab host. If you see a message like: - - ```plaintext - The authenticity of host 'gitlab.example.com (35.231.145.151)' can't be established. - ECDSA key fingerprint is SHA256:HbW3g8zUjNSksFbqTiUWPWg2Bq1x8xdGUrliXFzSnUw. - Are you sure you want to continue connecting (yes/no)? yes - Warning: Permanently added 'gitlab.example.com' (ECDSA) to the list of known hosts. - ``` - - Type `yes` and press Enter. - -1. Run the `ssh -T git@gitlab.example.com` command again. You should receive a _Welcome to GitLab, `@username`!_ message. - -If the welcome message doesn't appear, you can troubleshoot by running `ssh` -in verbose mode: - -```shell -ssh -Tvvv git@gitlab.example.com -``` - -## Use different keys for different repositories - -You can use a different key for each repository. - -Open a terminal and run this command: - -```shell -git config core.sshCommand "ssh -o IdentitiesOnly=yes -i ~/.ssh/private-key-filename-for-this-repository -F /dev/null" -``` - -This command does not use the SSH Agent and requires Git 2.10 or later. For more information -on `ssh` command options, see the `man` pages for both `ssh` and `ssh_config`. - -## Use different accounts on a single GitLab instance - -You can use multiple accounts to connect to a single instance of GitLab. -You can do this by using the command in the [previous topic](#use-different-keys-for-different-repositories). -However, even if you set `IdentitiesOnly` to `yes`, you cannot sign in if an `IdentityFile` exists -outside of a `Host` block. - -Instead, you can assign aliases to hosts in the `~.ssh/config` file. - -- For the `Host`, use an alias like `user_1.gitlab.com` and - `user_2.gitlab.com`. Advanced configurations - are more difficult to maintain, and these strings are easier to - understand when you use tools like `git remote`. -- For the `IdentityFile`, use the path the private key. - -```conf -# User1 Account Identity -Host <user_1.gitlab.com> - Hostname gitlab.com - PreferredAuthentications publickey - IdentityFile ~/.ssh/<example_ssh_key1> - -# User2 Account Identity -Host <user_2.gitlab.com> - Hostname gitlab.com - PreferredAuthentications publickey - IdentityFile ~/.ssh/<example_ssh_key2> -``` - -Now, to clone a repository for `user_1`, use `user_1.gitlab.com` in the `git clone` command: - -```shell -git clone git@<user_1.gitlab.com>:gitlab-org/gitlab.git -``` - -To update a previously-cloned repository that is aliased as `origin`: - -```shell -git remote set-url origin git@<user_1.gitlab.com>:gitlab-org/gitlab.git -``` - -NOTE: -Private and public keys contain sensitive data. Ensure the permissions -on the files make them readable to you but not accessible to others. - -## Configure two-factor authentication (2FA) - -You can set up two-factor authentication (2FA) for -[Git over SSH](../security/two_factor_authentication.md#two-factor-authentication-2fa-for-git-over-ssh-operations). - -## Use EGit on Eclipse - -If you are using [EGit](https://www.eclipse.org/egit/), you can [add your SSH key to Eclipse](https://wiki.eclipse.org/EGit/User_Guide#Eclipse_SSH_Configuration). - -## Use SSH on Microsoft Windows - -If you're running Windows 10, you can either use the [Windows Subsystem for Linux (WSL)](https://docs.microsoft.com/en-us/windows/wsl/install-win10) -with [WSL 2](https://docs.microsoft.com/en-us/windows/wsl/install-win10#update-to-wsl-2) which -has both `git` and `ssh` preinstalled, or install [Git for Windows](https://gitforwindows.org) to -use SSH through Powershell. - -The SSH key generated in WSL is not directly available for Git for Windows, and vice versa, -as both have a different home directory: - -- WSL: `/home/<user>` -- Git for Windows: `C:\Users\<user>` - -You can either copy over the `.ssh/` directory to use the same key, or generate a key in each environment. - -Alternative tools include: - -- [Cygwin](https://www.cygwin.com) -- [PuttyGen](https://www.chiark.greenend.org.uk/~sgtatham/putty/latest.html) - -## Overriding SSH settings on the GitLab server - -GitLab integrates with the system-installed SSH daemon and designates a user -(typically named `git`) through which all access requests are handled. Users -who connect to the GitLab server over SSH are identified by their SSH key instead -of their username. - -SSH *client* operations performed on the GitLab server are executed as this -user. You can modify this SSH configuration. For example, you can specify -a private SSH key for this user to use for authentication requests. However, this practice -is **not supported** and is strongly discouraged as it presents significant -security risks. - -GitLab checks for this condition, and directs you -to this section if your server is configured this way. For example: - -```shell -$ gitlab-rake gitlab:check - -Git user has default SSH configuration? ... no - Try fixing it: - mkdir ~/gitlab-check-backup-1504540051 - sudo mv /var/lib/git/.ssh/id_rsa ~/gitlab-check-backup-1504540051 - sudo mv /var/lib/git/.ssh/id_rsa.pub ~/gitlab-check-backup-1504540051 - For more information see: - [Overriding SSH settings on the GitLab server](#overriding-ssh-settings-on-the-gitlab-server) - Please fix the error above and rerun the checks. -``` - -Remove the custom configuration as soon as you can. These customizations -are **explicitly not supported** and may stop working at any time. - -## Troubleshooting SSH connections - -When you run `git clone`, you may be prompted for a password, like `git@gitlab.example.com's password:`. -This indicates that something is wrong with your SSH setup. - -- Ensure that you generated your SSH key pair correctly and added the public SSH - key to your GitLab profile. -- Try to manually register your private SSH key by using `ssh-agent`. -- Try to debug the connection by running `ssh -Tv git@example.com`. - Replace `example.com` with your GitLab URL. +<!-- This redirect file can be deleted after 2021-09-28. --> +<!-- Before deletion, see: https://docs.gitlab.com/ee/development/documentation/#move-or-rename-a-page --> diff --git a/doc/ssh/index.md b/doc/ssh/index.md new file mode 100644 index 00000000000..358323e4ef5 --- /dev/null +++ b/doc/ssh/index.md @@ -0,0 +1,385 @@ +--- +stage: Manage +group: Access +info: "To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/engineering/ux/technical-writing/#assignments" +type: howto, reference +--- + +# GitLab and SSH keys + +Git is a distributed version control system, which means you can work locally, +then share or "push" your changes to a server. In this case, the server is GitLab. + +GitLab uses the SSH protocol to securely communicate with Git. +When you use SSH keys to authenticate to the GitLab remote server, +you don't need to supply your username and password each time. + +## Prerequisites + +To use SSH to communicate with GitLab, you need: + +- The OpenSSH client, which comes pre-installed on GNU/Linux, macOS, and Windows 10. +- SSH version 6.5 or later. Earlier versions used an MD5 signature, which is not secure. + +To view the version of SSH installed on your system, run `ssh -V`. + +## Supported SSH key types + +To communicate with GitLab, you can use the following SSH key types: + +- [ED25519](#ed25519-ssh-keys) +- [RSA](#rsa-ssh-keys) +- DSA ([Deprecated](https://about.gitlab.com/releases/2018/06/22/gitlab-11-0-released/#support-for-dsa-ssh-keys) in GitLab 11.0.) +- ECDSA (As noted in [Practical Cryptography With Go](https://leanpub.com/gocrypto/read#leanpub-auto-ecdsa), the security issues related to DSA also apply to ECDSA.) + +Administrators can [restrict which keys are permitted and their minimum lengths](../security/ssh_keys_restrictions.md). + +### ED25519 SSH keys + +The book [Practical Cryptography With Go](https://leanpub.com/gocrypto/read#leanpub-auto-chapter-5-digital-signatures) +suggests that [ED25519](https://ed25519.cr.yp.to/) keys are more secure and performant than RSA keys. + +OpenSSH 6.5 introduced ED25519 SSH keys in 2014 and they should be available on most +operating systems. + +### RSA SSH keys + +Available documentation suggests that ED25519 is more secure than RSA. + +If you use an RSA key, the US National Institute of Science and Technology in +[Publication 800-57 Part 3 (PDF)](https://nvlpubs.nist.gov/nistpubs/SpecialPublications/NIST.SP.800-57Pt3r1.pdf) +recommends a key size of at least 2048 bits. The default key size depends on your version of `ssh-keygen`. +Review the `man` page for your installed `ssh-keygen` command for details. + +## See if you have an existing SSH key pair + +Before you create a key pair, see if a key pair already exists. + +1. On Windows, Linux, or macOS, go to your home directory. +1. Go to the `.ssh/` subdirectory. If the `.ssh/` subdirectory doesn't exist, + you are either not in the home directory, or you haven't used `ssh` before. + In the latter case, you need to [generate an SSH key pair](#generate-an-ssh-key-pair). +1. See if a file with one of the following formats exists: + + | Algorithm | Public key | Private key | + | --------- | ---------- | ----------- | + | ED25519 (preferred) | `id_ed25519.pub` | `id_ed25519` | + | RSA (at least 2048-bit key size) | `id_rsa.pub` | `id_rsa` | + | DSA (deprecated) | `id_dsa.pub` | `id_dsa` | + | ECDSA | `id_ecdsa.pub` | `id_ecdsa` | + +## Generate an SSH key pair + +If you do not have an existing SSH key pair, generate a new one. + +1. Open a terminal. +1. Type `ssh-keygen -t` followed by the key type and an optional comment. + This comment is included in the `.pub` file that's created. + You may want to use an email address for the comment. + + For example, for ED25519: + + ```shell + ssh-keygen -t ed25519 -C "<comment>" + ``` + + For 2048-bit RSA: + + ```shell + ssh-keygen -t rsa -b 2048 -C "<comment>" + ``` + +1. Press Enter. Output similar to the following is displayed: + + ```plaintext + Generating public/private ed25519 key pair. + Enter file in which to save the key (/home/user/.ssh/id_ed25519): + ``` + +1. Accept the suggested filename and directory, unless you are generating a [deploy key](../user/project/deploy_keys/index.md) + or want to save in a specific directory where you store other keys. + + You can also dedicate the SSH key pair to a [specific host](#configure-ssh-to-point-to-a-different-directory). + +1. Specify a [passphrase](https://www.ssh.com/ssh/passphrase/): + + ```plaintext + Enter passphrase (empty for no passphrase): + Enter same passphrase again: + ``` + +1. A confirmation is displayed, including information about where your files are stored. + +A public and private key are generated. +[Add the public SSH key to your GitLab account](#add-an-ssh-key-to-your-gitlab-account) and keep +the private key secure. + +### Configure SSH to point to a different directory + +If you did not save your SSH key pair in the default directory, +configure your SSH client to point to the directory where the private key is stored. + +1. Open a terminal and run this command: + + ```shell + eval $(ssh-agent -s) + ssh-add <directory to private SSH key> + ``` + +1. Save these settings in the `~/.ssh/config` file. For example: + + ```conf + # GitLab.com + Host gitlab.com + PreferredAuthentications publickey + IdentityFile ~/.ssh/gitlab_com_rsa + + # Private GitLab instance + Host gitlab.company.com + PreferredAuthentications publickey + IdentityFile ~/.ssh/example_com_rsa + ``` + + For more information on these settings, see the [`man ssh_config`](https://man.openbsd.org/ssh_config) page in the SSH configuration manual. + +Public SSH keys must be unique to GitLab because they bind to your account. +Your SSH key is the only identifier you have when you push code with SSH. +It must uniquely map to a single user. + +### Update your SSH key passphrase + +You can update the passphrase for your SSH key. + +1. Open a terminal and run this command: + + ```shell + ssh-keygen -p -f /path/to/ssh_key + ``` + +1. At the prompts, type the passphrase and press Enter. + +### Upgrade your RSA key pair to a more secure format + +If your version of OpenSSH is between 6.5 and 7.8, +you can save your private RSA SSH keys in a more secure +OpenSSH format. + +1. Open a terminal and run this command: + + ```shell + ssh-keygen -o -f ~/.ssh/id_rsa + ``` + + Alternatively, you can generate a new RSA key with the more secure encryption format with + the following command: + + ```shell + ssh-keygen -o -t rsa -b 4096 -C "<comment>" + ``` + +## Add an SSH key to your GitLab account + +To use SSH with GitLab, copy your public key to your GitLab account. + +1. Copy the contents of your public key file. You can do this manually or use a script. + For example, to copy an ED25519 key to the clipboard: + + **macOS:** + + ```shell + tr -d '\n' < ~/.ssh/id_ed25519.pub | pbcopy + ``` + + **Linux** (requires the `xclip` package): + + ```shell + xclip -sel clip < ~/.ssh/id_ed25519.pub + ``` + + **Git Bash on Windows:** + + ```shell + cat ~/.ssh/id_ed25519.pub | clip + ``` + + Replace `id_ed25519.pub` with your filename. For example, use `id_rsa.pub` for RSA. + +1. Sign in to GitLab. +1. In the top right corner, select your avatar. +1. Select **Preferences**. +1. From the left sidebar, select **SSH Keys**. +1. In the **Key** box, paste the contents of your public key. + If you manually copied the key, make sure you copy the entire key, + which starts with `ssh-ed25519` or `ssh-rsa`, and may end with a comment. +1. In the **Title** text box, type a description, like _Work Laptop_ or + _Home Workstation_. +1. Optional. In the **Expires at** box, select an expiration date. (Introduced in [GitLab 12.9](https://gitlab.com/gitlab-org/gitlab/-/issues/36243).) + The expiration date is informational only, and does not prevent you from using + the key. However, administrators can view expiration dates and + use them for guidance when [deleting keys](../user/admin_area/credentials_inventory.md#delete-a-users-ssh-key). + - GitLab checks all SSH keys at 02:00 AM UTC every day. It emails an expiration notice for all SSH keys that expire on the current date. ([Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/322637) in GitLab 13.11.) + - GitLab checks all SSH keys at 01:00 AM UTC every day. It emails an expiration notice for all SSH keys that are scheduled to expire seven days from now. ([Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/322637) in GitLab 13.11.) +1. Select **Add key**. + +## Verify that you can connect + +Verify that your SSH key was added correctly. + +1. For GitLab.com, to ensure you're connecting to the correct server, confirm the + [SSH host keys fingerprints](../user/gitlab_com/index.md#ssh-host-keys-fingerprints). +1. Open a terminal and run this command, replacing `gitlab.example.com` with your GitLab instance URL: + + ```shell + ssh -T git@gitlab.example.com + ``` + +1. If this is the first time you connect, you should verify the + authenticity of the GitLab host. If you see a message like: + + ```plaintext + The authenticity of host 'gitlab.example.com (35.231.145.151)' can't be established. + ECDSA key fingerprint is SHA256:HbW3g8zUjNSksFbqTiUWPWg2Bq1x8xdGUrliXFzSnUw. + Are you sure you want to continue connecting (yes/no)? yes + Warning: Permanently added 'gitlab.example.com' (ECDSA) to the list of known hosts. + ``` + + Type `yes` and press Enter. + +1. Run the `ssh -T git@gitlab.example.com` command again. You should receive a _Welcome to GitLab, `@username`!_ message. + +If the welcome message doesn't appear, you can troubleshoot by running `ssh` +in verbose mode: + +```shell +ssh -Tvvv git@gitlab.example.com +``` + +## Use different keys for different repositories + +You can use a different key for each repository. + +Open a terminal and run this command: + +```shell +git config core.sshCommand "ssh -o IdentitiesOnly=yes -i ~/.ssh/private-key-filename-for-this-repository -F /dev/null" +``` + +This command does not use the SSH Agent and requires Git 2.10 or later. For more information +on `ssh` command options, see the `man` pages for both `ssh` and `ssh_config`. + +## Use different accounts on a single GitLab instance + +You can use multiple accounts to connect to a single instance of GitLab. +You can do this by using the command in the [previous topic](#use-different-keys-for-different-repositories). +However, even if you set `IdentitiesOnly` to `yes`, you cannot sign in if an `IdentityFile` exists +outside of a `Host` block. + +Instead, you can assign aliases to hosts in the `~.ssh/config` file. + +- For the `Host`, use an alias like `user_1.gitlab.com` and + `user_2.gitlab.com`. Advanced configurations + are more difficult to maintain, and these strings are easier to + understand when you use tools like `git remote`. +- For the `IdentityFile`, use the path the private key. + +```conf +# User1 Account Identity +Host <user_1.gitlab.com> + Hostname gitlab.com + PreferredAuthentications publickey + IdentityFile ~/.ssh/<example_ssh_key1> + +# User2 Account Identity +Host <user_2.gitlab.com> + Hostname gitlab.com + PreferredAuthentications publickey + IdentityFile ~/.ssh/<example_ssh_key2> +``` + +Now, to clone a repository for `user_1`, use `user_1.gitlab.com` in the `git clone` command: + +```shell +git clone git@<user_1.gitlab.com>:gitlab-org/gitlab.git +``` + +To update a previously-cloned repository that is aliased as `origin`: + +```shell +git remote set-url origin git@<user_1.gitlab.com>:gitlab-org/gitlab.git +``` + +NOTE: +Private and public keys contain sensitive data. Ensure the permissions +on the files make them readable to you but not accessible to others. + +## Configure two-factor authentication (2FA) + +You can set up two-factor authentication (2FA) for +[Git over SSH](../security/two_factor_authentication.md#two-factor-authentication-2fa-for-git-over-ssh-operations). + +## Use EGit on Eclipse + +If you are using [EGit](https://www.eclipse.org/egit/), you can [add your SSH key to Eclipse](https://wiki.eclipse.org/EGit/User_Guide#Eclipse_SSH_Configuration). + +## Use SSH on Microsoft Windows + +If you're running Windows 10, you can either use the [Windows Subsystem for Linux (WSL)](https://docs.microsoft.com/en-us/windows/wsl/install-win10) +with [WSL 2](https://docs.microsoft.com/en-us/windows/wsl/install-win10#update-to-wsl-2) which +has both `git` and `ssh` preinstalled, or install [Git for Windows](https://gitforwindows.org) to +use SSH through Powershell. + +The SSH key generated in WSL is not directly available for Git for Windows, and vice versa, +as both have a different home directory: + +- WSL: `/home/<user>` +- Git for Windows: `C:\Users\<user>` + +You can either copy over the `.ssh/` directory to use the same key, or generate a key in each environment. + +Alternative tools include: + +- [Cygwin](https://www.cygwin.com) +- [PuttyGen](https://www.chiark.greenend.org.uk/~sgtatham/putty/latest.html) + +## Overriding SSH settings on the GitLab server + +GitLab integrates with the system-installed SSH daemon and designates a user +(typically named `git`) through which all access requests are handled. Users +who connect to the GitLab server over SSH are identified by their SSH key instead +of their username. + +SSH *client* operations performed on the GitLab server are executed as this +user. You can modify this SSH configuration. For example, you can specify +a private SSH key for this user to use for authentication requests. However, this practice +is **not supported** and is strongly discouraged as it presents significant +security risks. + +GitLab checks for this condition, and directs you +to this section if your server is configured this way. For example: + +```shell +$ gitlab-rake gitlab:check + +Git user has default SSH configuration? ... no + Try fixing it: + mkdir ~/gitlab-check-backup-1504540051 + sudo mv /var/lib/git/.ssh/id_rsa ~/gitlab-check-backup-1504540051 + sudo mv /var/lib/git/.ssh/id_rsa.pub ~/gitlab-check-backup-1504540051 + For more information see: + [Overriding SSH settings on the GitLab server](#overriding-ssh-settings-on-the-gitlab-server) + Please fix the error above and rerun the checks. +``` + +Remove the custom configuration as soon as you can. These customizations +are **explicitly not supported** and may stop working at any time. + +## Troubleshooting SSH connections + +When you run `git clone`, you may be prompted for a password, like `git@gitlab.example.com's password:`. +This indicates that something is wrong with your SSH setup. + +- Ensure that you generated your SSH key pair correctly and added the public SSH + key to your GitLab profile. +- Try to manually register your private SSH key by using `ssh-agent`. +- Try to debug the connection by running `ssh -Tv git@example.com`. + Replace `example.com` with your GitLab URL. diff --git a/doc/topics/authentication/index.md b/doc/topics/authentication/index.md index 4181e32fcf2..cb77811a5da 100644 --- a/doc/topics/authentication/index.md +++ b/doc/topics/authentication/index.md @@ -10,7 +10,7 @@ This page gathers all the resources for the topic **Authentication** within GitL ## GitLab users -- [SSH](../../ssh/README.md) +- [SSH](../../ssh/index.md) - [Two-Factor Authentication (2FA)](../../user/profile/account/two_factor_authentication.md#two-factor-authentication) - [Why do I keep getting signed out?](../../user/profile/index.md#why-do-i-keep-getting-signed-out) - **Articles:** diff --git a/doc/topics/autodevops/quick_start_guide.md b/doc/topics/autodevops/quick_start_guide.md index 71b55bfed16..ef5a110049a 100644 --- a/doc/topics/autodevops/quick_start_guide.md +++ b/doc/topics/autodevops/quick_start_guide.md @@ -14,7 +14,7 @@ to create a Kubernetes cluster manually using the Google Cloud Platform console. You are creating and deploying a simple application that you create from a GitLab template. These instructions also work for a self-managed GitLab instance; -ensure your own [runners are configured](../../ci/runners/README.md) and +ensure your own [runners are configured](../../ci/runners/index.md) and [Google OAuth is enabled](../../integration/google.md). ## Configure your Google account diff --git a/doc/topics/cron/index.md b/doc/topics/cron/index.md index 88f8bd1858f..17b7d6657bb 100644 --- a/doc/topics/cron/index.md +++ b/doc/topics/cron/index.md @@ -9,7 +9,7 @@ info: To determine the technical writer assigned to the Stage/Group associated w Cron syntax is used to schedule when jobs should run. You may need to use a cron syntax string to -[trigger nightly pipelines](../../ci/triggers/README.md#using-cron-to-trigger-nightly-pipelines), +[trigger nightly pipelines](../../ci/triggers/index.md#using-cron-to-trigger-nightly-pipelines), create a [pipeline schedule](../../api/pipeline_schedules.md#create-a-new-pipeline-schedule), or to prevent unintentional releases by setting a [deploy freeze](../../user/project/releases/index.md#prevent-unintentional-releases-by-setting-a-deploy-freeze). diff --git a/doc/topics/git/how_to_install_git/index.md b/doc/topics/git/how_to_install_git/index.md index 8b097c4c1da..fc9c0e0ec63 100644 --- a/doc/topics/git/how_to_install_git/index.md +++ b/doc/topics/git/how_to_install_git/index.md @@ -62,7 +62,7 @@ To verify that Git works on your system, run: git --version ``` -Next, read our article on [adding an SSH key to GitLab](../../../ssh/README.md). +Next, read our article on [adding an SSH key to GitLab](../../../ssh/index.md). ## Install Git on Ubuntu Linux @@ -86,13 +86,13 @@ To verify that Git works on your system, run: git --version ``` -Next, read our article on [adding an SSH key to GitLab](../../../ssh/README.md). +Next, read our article on [adding an SSH key to GitLab](../../../ssh/index.md). ## Installing Git on Windows from the Git website Open the [Git website](https://git-scm.com/) and download and install Git for Windows. -Next, read our article on [adding an SSH key to GitLab](../../../ssh/README.md). +Next, read our article on [adding an SSH key to GitLab](../../../ssh/index.md). <!-- ## Troubleshooting diff --git a/doc/topics/git/troubleshooting_git.md b/doc/topics/git/troubleshooting_git.md index 8db683f6291..cc2631c9445 100644 --- a/doc/topics/git/troubleshooting_git.md +++ b/doc/topics/git/troubleshooting_git.md @@ -45,7 +45,7 @@ set to 50MB. The default is 1MB. **If pushing over SSH**, first check your SSH configuration as 'Broken pipe' errors can sometimes be caused by underlying issues with SSH (such as authentication). Make sure that SSH is correctly configured by following the -instructions in the [SSH troubleshooting](../../ssh/README.md#troubleshooting-ssh-connections) documentation. +instructions in the [SSH troubleshooting](../../ssh/index.md#troubleshooting-ssh-connections) documentation. If you're a GitLab administrator with server access, you can also prevent session timeouts by configuring SSH `keep-alive` on the client or the server. diff --git a/doc/topics/set_up_organization.md b/doc/topics/set_up_organization.md index d8b1ab59b9e..d6e82a6ce87 100644 --- a/doc/topics/set_up_organization.md +++ b/doc/topics/set_up_organization.md @@ -12,5 +12,5 @@ and give everyone access to the projects they need. - [Members](../user/project/members/index.md) - [Groups](../user/group/index.md) - [User account options](../user/profile/index.md) -- [SSH keys](../ssh/README.md) +- [SSH keys](../ssh/index.md) - [GitLab.com settings](../user/gitlab_com/index.md) diff --git a/doc/user/admin_area/settings/continuous_integration.md b/doc/user/admin_area/settings/continuous_integration.md index 1de31acf476..8f14dd0a563 100644 --- a/doc/user/admin_area/settings/continuous_integration.md +++ b/doc/user/admin_area/settings/continuous_integration.md @@ -224,7 +224,7 @@ This feature is being re-evaluated in favor of a different We recommend that users who haven't yet implemented this feature wait for the new solution. -You can set a [CI/CD template](../../../ci/examples/README.md#cicd-templates) +You can set a [CI/CD template](../../../ci/examples/index.md#cicd-templates) as a required pipeline configuration for all projects on a GitLab instance. You can use a template from: diff --git a/doc/user/admin_area/settings/sign_in_restrictions.md b/doc/user/admin_area/settings/sign_in_restrictions.md index ecd259a345c..333e9465c31 100644 --- a/doc/user/admin_area/settings/sign_in_restrictions.md +++ b/doc/user/admin_area/settings/sign_in_restrictions.md @@ -21,7 +21,7 @@ To access sign-in restriction settings: You can restrict the password authentication for web interface and Git over HTTP(S): -- **Web interface**: When this feature is disabled, the **Standard** sign-in tab is removed and an [external authentication provider](../../../administration/auth/README.md) must be used. +- **Web interface**: When this feature is disabled, the **Standard** sign-in tab is removed and an [external authentication provider](../../../administration/auth/index.md) must be used. - **Git over HTTP(S)**: When this feature is disabled, a [Personal Access Token](../../profile/personal_access_tokens.md) must be used to authenticate. ## Admin Mode diff --git a/doc/user/application_security/dast/index.md b/doc/user/application_security/dast/index.md index 27311d4b55e..d8d2a607cab 100644 --- a/doc/user/application_security/dast/index.md +++ b/doc/user/application_security/dast/index.md @@ -50,7 +50,7 @@ results. On failure, the analyzer outputs an ## Prerequisites -- [GitLab Runner](../../../ci/runners/README.md) available, with the +- [GitLab Runner](../../../ci/runners/index.md) available, with the [`docker` executor](https://docs.gitlab.com/runner/executors/docker.html). - Target application deployed. For more details, read [Deployment options](#deployment-options). diff --git a/doc/user/clusters/applications.md b/doc/user/clusters/applications.md index 5bd28406fd7..7b283ee89fb 100644 --- a/doc/user/clusters/applications.md +++ b/doc/user/clusters/applications.md @@ -391,7 +391,7 @@ For GitLab Runner to function, you _must_ specify the following: - `gitlabUrl`: The GitLab server full URL (for example, `https://gitlab.example.com`) to register the Runner against. - `runnerRegistrationToken`: The registration token for adding new runners to GitLab. - This must be [retrieved from your GitLab instance](../../ci/runners/README.md). + This must be [retrieved from your GitLab instance](../../ci/runners/index.md). These values can be specified using [CI/CD variables](../../ci/variables/README.md): diff --git a/doc/user/gitlab_com/index.md b/doc/user/gitlab_com/index.md index 00dee5ee489..3c9ea45cbf1 100644 --- a/doc/user/gitlab_com/index.md +++ b/doc/user/gitlab_com/index.md @@ -181,7 +181,7 @@ The following limits apply for [Webhooks](../project/integrations/webhooks.md): GitLab has shared runners on GitLab.com that you can use to run your CI jobs. -For more information, see [choosing a runner](../../ci/runners/README.md). +For more information, see [choosing a runner](../../ci/runners/index.md). ## Sidekiq diff --git a/doc/user/group/saml_sso/index.md b/doc/user/group/saml_sso/index.md index 8a5cdb79186..3a5ea85ef15 100644 --- a/doc/user/group/saml_sso/index.md +++ b/doc/user/group/saml_sso/index.md @@ -10,7 +10,7 @@ info: To determine the technical writer assigned to the Stage/Group associated w > Introduced in GitLab 11.0. This page describes SAML for Groups. For instance-wide SAML on self-managed GitLab instances, see [SAML OmniAuth Provider](../../../integration/saml.md). -[View the differences between SaaS and Self-Managed Authentication and Authorization Options](../../../administration/auth/README.md#saas-vs-self-managed-comparison). +[View the differences between SaaS and Self-Managed Authentication and Authorization Options](../../../administration/auth/index.md#saas-vs-self-managed-comparison). SAML on GitLab.com allows users to sign in through their SAML identity provider. If the user is not already a member, the sign-in process automatically adds the user to the appropriate group. diff --git a/doc/user/group/settings/import_export.md b/doc/user/group/settings/import_export.md index c097790ef16..1d5db6757bc 100644 --- a/doc/user/group/settings/import_export.md +++ b/doc/user/group/settings/import_export.md @@ -84,7 +84,7 @@ As an administrator, you can modify the maximum import file size. To do so, use You can export groups from the [Community Edition to the Enterprise Edition](https://about.gitlab.com/install/ce-or-ee/) and vice versa. -The Enterprise Edition retains some group data that isn't part of the Community Edition. If you're exporting a group from the Enterprise Edition to the Community Edition, you may lose this data. For more information, see [downgrading from EE to CE](../../../README.md). +The Enterprise Edition retains some group data that isn't part of the Community Edition. If you're exporting a group from the Enterprise Edition to the Community Edition, you may lose this data. For more information, see [downgrading from EE to CE](../../../index.md). ## Importing the group diff --git a/doc/user/group/subgroups/index.md b/doc/user/group/subgroups/index.md index 4532a391eef..7d674b5deac 100644 --- a/doc/user/group/subgroups/index.md +++ b/doc/user/group/subgroups/index.md @@ -125,7 +125,7 @@ When you add a member to a group, that member is also added to all subgroups. Permission level is inherited from the group's parent. This model allows access to subgroups if you have membership in one of its parents. -Jobs for pipelines in subgroups can use [runners](../../../ci/runners/README.md) registered to the parent group(s). +Jobs for pipelines in subgroups can use [runners](../../../ci/runners/index.md) registered to the parent group(s). This means secrets configured for the parent group are available to subgroup jobs. In addition, maintainers of projects that belong to subgroups can see the details of runners registered to parent group(s). diff --git a/doc/user/markdown.md b/doc/user/markdown.md index fdfd953e52a..db2d6846b7c 100644 --- a/doc/user/markdown.md +++ b/doc/user/markdown.md @@ -1181,7 +1181,7 @@ Do not edit the following codeblock. It uses HTML to skip the Vale ReferenceLink <pre class="highlight"><code>- This is an [inline-style link](https://www.google.com) - This is a [link to a repository file in the same directory](index.md) -- This is a [relative link to a readme one directory higher](../README.md) +- This is a [relative link to a readme one directory higher](../index.md) - This is a [link that also has title text](https://www.google.com "This link takes you to Google!") Using header ID anchors: @@ -1204,7 +1204,7 @@ Some text to show that the reference links can follow later. - This is an [inline-style link](https://www.google.com) - This is a [link to a repository file in the same directory](index.md) -- This is a [relative link to a README one directory higher](../README.md) +- This is a [relative link to a README one directory higher](../index.md) - This is a [link that also has title text](https://www.google.com "This link takes you to Google!") Using header ID anchors: diff --git a/doc/user/packages/container_registry/index.md b/doc/user/packages/container_registry/index.md index ef422cdfbf9..0925787a135 100644 --- a/doc/user/packages/container_registry/index.md +++ b/doc/user/packages/container_registry/index.md @@ -154,7 +154,7 @@ To use CI/CD to authenticate, you can use: docker login -u $CI_REGISTRY_USER -p $CI_REGISTRY_PASSWORD $CI_REGISTRY ``` -- A [CI job token](../../../ci/triggers/README.md#ci-job-token). +- A [CI job token](../../../ci/triggers/index.md#ci-job-token). ```shell docker login -u $CI_JOB_USER -p $CI_JOB_TOKEN $CI_REGISTRY diff --git a/doc/user/profile/index.md b/doc/user/profile/index.md index 9c2fccfea82..70abb423712 100644 --- a/doc/user/profile/index.md +++ b/doc/user/profile/index.md @@ -280,6 +280,6 @@ Without the `config.extend_remember_period` flag, you would be forced to sign in - [Receive emails for sign-ins from unknown IP addresses or devices](unknown_sign_in_notification.md) - Manage applications that can [use GitLab as an OAuth provider](../../integration/oauth_provider.md#introduction-to-oauth) - Manage [personal access tokens](personal_access_tokens.md) to access your account via API and authorized applications -- Manage [SSH keys](../../ssh/README.md) to access your account via SSH +- Manage [SSH keys](../../ssh/index.md) to access your account via SSH - Change your [syntax highlighting theme](preferences.md#syntax-highlighting-theme) - [View your active sessions](active_sessions.md) and revoke any of them if necessary diff --git a/doc/user/project/deploy_boards.md b/doc/user/project/deploy_boards.md index 89c82d4dc6f..e7f7e75a6d3 100644 --- a/doc/user/project/deploy_boards.md +++ b/doc/user/project/deploy_boards.md @@ -88,7 +88,7 @@ To display the Deploy Boards for a specific [environment](../../ci/environments/ [OpenShift docs](https://docs.openshift.com/container-platform/3.7/dev_guide/deployments/kubernetes_deployments.html#kubernetes-deployments-vs-deployment-configurations) and [GitLab issue #4584](https://gitlab.com/gitlab-org/gitlab/-/issues/4584). -1. [Configure GitLab Runner](../../ci/runners/README.md) with the [`docker`](https://docs.gitlab.com/runner/executors/docker.html) or +1. [Configure GitLab Runner](../../ci/runners/index.md) with the [`docker`](https://docs.gitlab.com/runner/executors/docker.html) or [`kubernetes`](https://docs.gitlab.com/runner/executors/kubernetes.html) executor. 1. Configure the [Kubernetes integration](clusters/index.md) in your project for the cluster. The Kubernetes namespace is of particular note as you need it diff --git a/doc/user/project/merge_requests/load_performance_testing.md b/doc/user/project/merge_requests/load_performance_testing.md index d1b697add08..f2642c5521d 100644 --- a/doc/user/project/merge_requests/load_performance_testing.md +++ b/doc/user/project/merge_requests/load_performance_testing.md @@ -93,7 +93,7 @@ template that is included with GitLab. NOTE: For large scale k6 tests you need to ensure the GitLab Runner instance performing the actual test is able to handle running the test. Refer to [k6's guidance](https://k6.io/docs/testing-guides/running-large-tests#hardware-considerations) -for spec details. The [default shared GitLab.com runners](../../../ci/runners/README.md#linux-shared-runners) +for spec details. The [default shared GitLab.com runners](../../../ci/runners/index.md#linux-shared-runners) likely have insufficient specs to handle most large k6 tests. This template runs the diff --git a/doc/user/project/settings/import_export.md b/doc/user/project/settings/import_export.md index 890784cecf5..08d00149037 100644 --- a/doc/user/project/settings/import_export.md +++ b/doc/user/project/settings/import_export.md @@ -110,7 +110,7 @@ and the exports between them are compatible. You can export projects from the [Community Edition to the Enterprise Edition](https://about.gitlab.com/install/ce-or-ee/) and vice versa. This assumes [version history](#version-history) requirements are met. -If you're exporting a project from the Enterprise Edition to the Community Edition, you may lose data that is retained only in the Enterprise Edition. For more information, see [downgrading from EE to CE](../../../README.md). +If you're exporting a project from the Enterprise Edition to the Community Edition, you may lose data that is retained only in the Enterprise Edition. For more information, see [downgrading from EE to CE](../../../index.md). ## Exported contents |