diff options
Diffstat (limited to 'doc/integration')
| -rw-r--r-- | doc/integration/bitbucket.md | 2 | ||||
| -rw-r--r-- | doc/integration/elasticsearch.md | 190 | ||||
| -rw-r--r-- | doc/integration/jenkins.md | 24 | ||||
| -rw-r--r-- | doc/integration/jenkins_deprecated.md | 3 | ||||
| -rw-r--r-- | doc/integration/jira_development_panel.md | 146 | ||||
| -rw-r--r-- | doc/integration/kerberos.md | 2 | ||||
| -rw-r--r-- | doc/integration/omniauth.md | 62 | ||||
| -rw-r--r-- | doc/integration/recaptcha.md | 4 | ||||
| -rw-r--r-- | doc/integration/saml.md | 9 | ||||
| -rw-r--r-- | doc/integration/shibboleth.md | 2 | ||||
| -rw-r--r-- | doc/integration/sourcegraph.md | 3 |
11 files changed, 314 insertions, 133 deletions
diff --git a/doc/integration/bitbucket.md b/doc/integration/bitbucket.md index dc5c7a138f8..a151fbf50e7 100644 --- a/doc/integration/bitbucket.md +++ b/doc/integration/bitbucket.md @@ -21,7 +21,7 @@ Bitbucket.org. ## Bitbucket OmniAuth provider -> **Note:** +NOTE: **Note:** GitLab 8.15 significantly simplified the way to integrate Bitbucket.org with GitLab. You are encouraged to upgrade your GitLab instance if you haven't done so already. If you're using GitLab 8.14 or below, [use the previous integration diff --git a/doc/integration/elasticsearch.md b/doc/integration/elasticsearch.md index 8ca45547f11..67b256cc944 100644 --- a/doc/integration/elasticsearch.md +++ b/doc/integration/elasticsearch.md @@ -23,9 +23,8 @@ special searches: ## Installing Elasticsearch -Elasticsearch is _not_ included in the Omnibus packages. You will have to -[install it yourself](https://www.elastic.co/guide/en/elasticsearch/reference/6.8/install-elasticsearch.html "Elasticsearch 6.8 installation documentation") -whether you are using the Omnibus package or installed GitLab from source. +Elasticsearch is _not_ included in the Omnibus packages or when you install from source. You must +[install it separately](https://www.elastic.co/guide/en/elasticsearch/reference/7.x/install-elasticsearch.html "Elasticsearch 7.x installation documentation"). Be sure to select your version. Providing detailed information on installing Elasticsearch is out of the scope of this document. @@ -118,14 +117,21 @@ Once installed, enable it under your instance's Elasticsearch settings explained ## System Requirements Elasticsearch requires additional resources in excess of those documented in the -[GitLab system requirements](../install/requirements.md). These will vary by -installation size, but you should ensure **at least** an additional **8 GiB of RAM** -for each Elasticsearch node, per the [official guidelines](https://www.elastic.co/guide/en/elasticsearch/guide/current/hardware.html). +[GitLab system requirements](../install/requirements.md). -Keep in mind, this is the **minimum requirements** as per Elasticsearch. For -production instances, they recommend considerably more resources. +The amount of resources (memory, CPU, storage) will vary greatly, based on the amount of data being indexed into the Elasticsearch cluster. According to [Elasticsearch official guidelines](https://www.elastic.co/guide/en/elasticsearch/guide/current/hardware.html#_memory), each node should have: -The necessary storage space largely depends on the size of the repositories you want to store in GitLab but as a rule of thumb you should have at least 50% of free space as all your repositories combined take up. +- [RAM](https://www.elastic.co/guide/en/elasticsearch/guide/current/hardware.html#_disks): **8 GiB as the bare minimum** +- [CPU](https://www.elastic.co/guide/en/elasticsearch/guide/current/hardware.html#_cpus): Modern processor with multiple cores +- [Storage](https://www.elastic.co/guide/en/elasticsearch/guide/current/hardware.html#_disks): Use SSD storage. As a guide you will need enough storage for 50% of the total size of your Git repositories. + +A few notes on CPU and storage: + +- CPU requirements for Elasticsearch tend to be light. There are specific scenarios where this isn't true, but GitLab.com isn't using Elasticsearch in an exceptionally CPU-heavy way. More cores will be more performant than faster CPUs. Extra concurrency from multiple cores will far outweigh a slightly faster clock speed in Elasticsearch. + +- Storage requirements for Elasticsearch are important, especially for indexing-heavy clusters. When possible, use SSDs, Their speed is far superior to any spinning media for Elasticsearch. In testing, nodes that use SSD storage see boosts in both query and indexing performance. + +Keep in mind, these are **minimum requirements** for Elasticsearch. Heavily-utilized Elasticsearch clusters will likely require considerably more resources. ## Enabling Elasticsearch @@ -142,13 +148,14 @@ The following Elasticsearch settings are available: | `Elasticsearch pause indexing` | Enables/disables temporary indexing pause. This is useful for cluster migration/reindexing. All changes are still tracked, but they are not committed to the Elasticsearch index until unpaused. | | `Search with Elasticsearch enabled` | Enables/disables using Elasticsearch in search. | | `URL` | The URL to use for connecting to Elasticsearch. Use a comma-separated list to support clustering (e.g., `http://host1, https://host2:9200`). If your Elasticsearch instance is password protected, pass the `username:password` in the URL (e.g., `http://<username>:<password>@<elastic_host>:9200/`). | -| `Number of Elasticsearch shards` | Elasticsearch indexes are split into multiple shards for performance reasons. In general, larger indexes need to have more shards. Changes to this value do not take effect until the index is recreated. You can read more about tradeoffs in the [Elasticsearch documentation](https://www.elastic.co/guide/en/elasticsearch/reference/current/indices-create-index.html#create-index-settings) | +| `Number of Elasticsearch shards` | Elasticsearch indexes are split into multiple shards for performance reasons. In general, larger indexes need to have more shards. Changes to this value do not take effect until the index is recreated. You can read more about tradeoffs in the [Elasticsearch documentation](https://www.elastic.co/guide/en/elasticsearch/reference/current/scalability.html). | | `Number of Elasticsearch replicas` | Each Elasticsearch shard can have a number of replicas. These are a complete copy of the shard, and can provide increased query performance or resilience against hardware failure. Increasing this value will greatly increase total disk space required by the index. | | `Limit namespaces and projects that can be indexed` | Enabling this will allow you to select namespaces and projects to index. All other namespaces and projects will use database search instead. Please note that if you enable this option but do not select any namespaces or projects, none will be indexed. [Read more below](#limiting-namespaces-and-projects). -| `Using AWS hosted Elasticsearch with IAM credentials` | Sign your Elasticsearch requests using [AWS IAM authorization](https://docs.aws.amazon.com/IAM/latest/UserGuide/id_credentials_access-keys.html) or [AWS EC2 Instance Profile Credentials](https://docs.aws.amazon.com/codedeploy/latest/userguide/getting-started-create-iam-instance-profile.html#getting-started-create-iam-instance-profile-cli). The policies must be configured to allow `es:*` actions. | +| `Using AWS hosted Elasticsearch with IAM credentials` | Sign your Elasticsearch requests using [AWS IAM authorization](https://docs.aws.amazon.com/IAM/latest/UserGuide/id_credentials_access-keys.html), [AWS EC2 Instance Profile Credentials](https://docs.aws.amazon.com/codedeploy/latest/userguide/getting-started-create-iam-instance-profile.html#getting-started-create-iam-instance-profile-cli), or [AWS ECS Tasks Credentials](https://docs.aws.amazon.com/AmazonECS/latest/userguide/task-iam-roles.html). The policies must be configured to allow `es:*` actions. | | `AWS Region` | The AWS region your Elasticsearch service is located in. | | `AWS Access Key` | The AWS access key. | | `AWS Secret Access Key` | The AWS secret access key. | +| `Maximum file size indexed` | See [the explanation in instance limits.](../administration/instance_limits.md#maximum-file-size-indexed). | | `Maximum field length` | See [the explanation in instance limits.](../administration/instance_limits.md#maximum-field-length). | | `Maximum bulk request size (MiB)` | The Maximum Bulk Request size is used by GitLab's Golang-based indexer processes and indicates how much data it ought to collect (and store in memory) in a given indexing process before submitting the payload to Elasticsearch’s Bulk API. This setting should be used with the Bulk request concurrency setting (see below) and needs to accommodate the resource constraints of both the Elasticsearch host(s) and the host(s) running GitLab's Golang-based indexer either from the `gitlab-rake` command or the Sidekiq tasks. | | `Bulk request concurrency` | The Bulk request concurrency indicates how many of GitLab's Golang-based indexer processes (or threads) can run in parallel to collect data to subsequently submit to Elasticsearch’s Bulk API. This increases indexing performance, but fills the Elasticsearch bulk requests queue faster. This setting should be used together with the Maximum bulk request size setting (see above) and needs to accommodate the resource constraints of both the Elasticsearch host(s) and the host(s) running GitLab's Golang-based indexer either from the `gitlab-rake` command or the Sidekiq tasks. | @@ -284,9 +291,16 @@ or creating [extra Sidekiq processes](../administration/operations/extra_sidekiq ``` 1. [Enable **Elasticsearch indexing**](#enabling-elasticsearch). -1. Indexing large Git repositories can take a while. To speed up the process, you - can temporarily disable auto-refreshing and replicating. In our experience, you can expect a 20% - decrease in indexing time. We'll enable them when indexing is done. This step is optional! +1. Indexing large Git repositories can take a while. To speed up the process, you can [tune for indexing speed](https://www.elastic.co/guide/en/elasticsearch/reference/current/tune-for-indexing-speed.html#tune-for-indexing-speed): + + - You can temporarily disable [`refresh`](https://www.elastic.co/guide/en/elasticsearch/reference/current/indices-refresh.html), the operation responsible for making changes to an index available to search. + + - You can set the number of replicas to 0. This setting controls the number of copies each primary shard of an index will have. Thus, having 0 replicas effectively disables the replication of shards across nodes, which should increase the indexing performance. This is an important trade-off in terms of reliability and query performance. It is important to remember to set the replicas to a considered value after the initial indexing is complete. + + In our experience, you can expect a 20% decrease in indexing time. After completing indexing in a later step, you can return `refresh` and `number_of_replicas` to their desired settings. + + NOTE: **Note:** + This step is optional but may help significantly speed up large indexing operations. ```shell curl --request PUT localhost:9200/gitlab-production/_settings --header 'Content-Type: application/json' --data '{ @@ -409,6 +423,144 @@ or creating [extra Sidekiq processes](../administration/operations/extra_sidekiq For repository and snippet files, GitLab will only index up to 1 MiB of content, in order to avoid indexing timeouts. +## Zero downtime reindexing + +The idea behind this reindexing method is to leverage Elasticsearch index alias feature to atomically swap between two indices. +We will refer to each index as `primary` (online and used by GitLab for read/writes) and `secondary` (offline, for reindexing purpose). + +Instead of connecting directly to the `primary` index, we'll setup an index alias such as we can change the underlying index at will. + +NOTE: **Note:** +Any index attached to the production alias is deemed a `primary` and will end up being used by the GitLab Elasticsearch integration. + +### Pause the indexing + +Under **Admin Area > Integration > Elasticsearch**, check the **Pause Elasticsearch Indexing** setting and save. + +With this, all updates that should happen on your Elasticsearch index will be buffered and caught up once unpaused. + +### Setup + +TIP: **Tip:** +If your index has been created with GitLab v13.0+ you can skip directly to [trigger the reindex](#trigger-the-reindex-via-the-elasticsearch-administration). + +This process involves multiple shell commands and curl invocations, so a good initial setup will help down the road: + +```shell +# You can find this value under Admin Area > Integration > Elasticsearch > URL +export CLUSTER_URL="http://localhost:9200" +export PRIMARY_INDEX="gitlab-production" +export SECONDARY_INDEX="gitlab-production-$(date +%s)" +``` + +### Reclaiming the `gitlab-production` index name + +CAUTION: **Caution:** +It is highly recommended that you take a snapshot of your cluster to make sure there is a recovery path if anything goes wrong. + +NOTE: **Note:** +Due to a technical limitation, there will be a slight downtime because of the fact that we need to reclaim the current `primary` index to be used as the alias. + +To reclaim the `gitlab-production` index name, you need to first create a `secondary` index and then trigger the re-index from `primary`. + +#### Creating a secondary index + +To create a secondary index, run the following Rake task. The `SKIP_ALIAS` +environment variable will disable the automatic creation of the Elasticsearch +alias, which would conflict with the existing index under `$PRIMARY_INDEX`: + +```shell +# Omnibus installation +sudo SKIP_ALIAS=1 gitlab-rake "gitlab:elastic:create_empty_index[$SECONDARY_INDEX]" + +# Source installation +SKIP_ALIAS=1 bundle exec rake "gitlab:elastic:create_empty_index[$SECONDARY_INDEX]" +``` + +The index should be created successfully, with the latest index options and mappings. + +#### Trigger the re-index from `primary` + +To trigger the re-index from `primary` index: + +1. Use the Elasticsearch [Reindex API](https://www.elastic.co/guide/en/elasticsearch/reference/7.6/docs-reindex.html): + + ```shell + curl --request POST \ + --header 'Content-Type: application/json' \ + --data "{ \"source\": { \"index\": \"$PRIMARY_INDEX\" }, \"dest\": { \"index\": \"$SECONDARY_INDEX\" } }" \ + "$CLUSTER_URL/_reindex?slices=auto&wait_for_completion=false" + ``` + + There will be an output like: + + ```plaintext + {"task":"3qw_Tr0YQLq7PF16Xek8YA:1012"} + ``` + + Note the `task` value here as it will be useful to follow the reindex progress. + +1. Wait for the reindex process to complete, by checking the `completed` value. + Using the `task` value form the previous step: + + ```shell + export TASK_ID=3qw_Tr0YQLq7PF16Xek8YA:1012 + curl "$CLUSTER_URL/_tasks/$TASK_ID?pretty" + ``` + + The output will be like: + + ```plaintext + {"completed":false, …} + ``` + + Once the returned value is `true`, you may continue to the next step. + +1. Make sure that the secondary index has data in it. You can use the Elasticsearch + API to look for the index size and compare our two indices: + + ```shell + curl $CLUSTER_URL/$PRIMARY_INDEX/_count => 123123 + curl $CLUSTER_URL/$SECONDARY_INDEX/_count => 123123 + ``` + + TIP: **Tip:** + Comparing the document count is more accurate than using the index size, as improvements to the storage might cause the new index to be smaller than the original one. + +1. Once you are confident your `secondary` index is valid, you can process to the creation of the alias. + + ```shell + # Delete the original index + curl --request DELETE $CLUSTER_URL/$PRIMARY_INDEX + + # Create the alias and add the `secondary` index to it + curl --request POST \ + --header 'Content-Type: application/json' \ + --data "{\"actions\":[{\"add\":{\"index\":\"$SECONDARY_INDEX\",\"alias\":\"$PRIMARY_INDEX\"}}]}}" \ + $CLUSTER_URL/_aliases + ``` + + The reindexing is now completed. Your GitLab instance is now ready to use the [automated in-cluster reindexing](#trigger-the-reindex-via-the-elasticsearch-administration) feature for future reindexing. + +1. Unpause the indexing + + Under **Admin Area > Integration > Elasticsearch**, uncheck the **Pause Elasticsearch Indexing** setting and save. + +### Trigger the reindex via the Elasticsearch administration + +> - [Introduced](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/34069) in [GitLab Starter](https://about.gitlab.com/pricing/) 13.2. +> - A scheduled index deletion and the ability to cancel it was [introduced](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/38914) in GitLab Starter 13.3. + +Under **Admin Area > Integration > Elasticsearch zero-downtime reindexing**, click on **Trigger cluster reindexing**. + +NOTE: **Note:** +Reindexing can be a lengthy process depending on the size of your Elasticsearch cluster. + +CAUTION: **Caution:** +After the reindexing is completed, the original index will be scheduled to be deleted after 14 days. You can cancel this action by pressing the cancel button. + +While the reindexing is running, you will be able to follow its progress under that same section. + ## GitLab Elasticsearch Rake tasks Rake tasks are available to: @@ -572,7 +724,7 @@ Here are some common pitfalls and how to overcome them: - **I indexed all the repositories but then switched Elasticsearch servers and now I can't find anything** - You will need to re-run all the Rake tasks to re-index the database, repositories, and wikis. + You will need to re-run all the Rake tasks to reindex the database, repositories, and wikis. - **The indexing process is taking a very long time** @@ -602,7 +754,7 @@ Here are some common pitfalls and how to overcome them: ``` This is because we changed the index mapping in GitLab 8.12 and the old indexes should be removed and built from scratch again, - see details in the [8-11-to-8-12 update guide](https://gitlab.com/gitlab-org/gitlab/blob/master/doc/update/8.11-to-8.12.md#11-elasticsearch-index-update-if-you-currently-use-elasticsearch). + see details in the [update guide](../update/upgrading_from_source.md). - Exception `Elasticsearch::Transport::Transport::Errors::BadRequest` @@ -657,11 +809,11 @@ There is a [more structured, lower-level troubleshooting document](../administra ### Known Issues -- **[Elasticsearch `code_analyzer` doesn't account for all code cases](https://gitlab.com/gitlab-org/gitlab/-/issues/10693)** +- **[Elasticsearch `code_analyzer` doesn't account for all code cases](https://gitlab.com/groups/gitlab-org/-/epics/3621)** - The `code_analyzer` pattern and filter configuration is being evaluated for improvement. We have noticed [several edge cases](https://gitlab.com/gitlab-org/gitlab/-/issues/10693#note_158382332) that are not returning expected search results due to our pattern and filter configuration. + The `code_analyzer` pattern and filter configuration is being evaluated for improvement. We have fixed [most edge cases](https://gitlab.com/groups/gitlab-org/-/epics/3621#note_363429094) that were not returning expected search results due to our pattern and filter configuration. - An improved strategy for the `code_analyzer` pattern and filters are being discussed in [issue 29443](https://gitlab.com/gitlab-org/gitlab/-/issues/29443). + Improvements to the `code_analyzer` pattern and filters is being discussed in [epic 3621](https://gitlab.com/groups/gitlab-org/-/epics/3621). ### Reverting to basic search diff --git a/doc/integration/jenkins.md b/doc/integration/jenkins.md index 485ca67c9fc..8fc638db95a 100644 --- a/doc/integration/jenkins.md +++ b/doc/integration/jenkins.md @@ -3,10 +3,10 @@ NOTE: **Note:** This documentation focuses only on how to **configure** a Jenkins *integration* with GitLab. Learn how to **migrate** from Jenkins to GitLab CI/CD in our -[Migrating from Jenkins](../ci/jenkins/index.md) documentation. +[Migrating from Jenkins](../ci/migration/jenkins.md) documentation. From GitLab, you can trigger a Jenkins build when you push code to a repository, or when a merge -request is created. In return, Jenkins shows the pipeline status on merge requests widgets and +request is created. In return, the Jenkins pipeline status is shown on merge requests widgets and on the GitLab project's home page. To better understand GitLab's Jenkins integration, watch the following video: @@ -62,7 +62,7 @@ Grant a GitLab user access to the select GitLab projects. Create a personal access token to authorize Jenkins' access to GitLab. 1. Log in to GitLab as the user to be used with Jenkins. -1. Click your avatar, then **Settings. +1. Click your avatar, then **Settings**. 1. Click **Access Tokens** in the sidebar. 1. Create a personal access token with the **API** scope checkbox checked. For more details, see [Personal access tokens](../user/profile/personal_access_tokens.md). @@ -147,24 +147,6 @@ Configure the GitLab integration with Jenkins. authentication. 1. Click **Test settings and save changes**. GitLab tests the connection to Jenkins. -## Plugin functional overview - -GitLab does not contain a database table listing commits. Commits are always -read from the repository directly. Therefore, it's not possible to retain the -build status of a commit in GitLab. This is overcome by requesting build -information from the integrated CI tool. The CI tool is responsible for creating -and storing build status for Commits and Merge Requests. - -### Steps required to implement a similar integration - -**Note:** -All steps are implemented using AJAX requests on the merge request page. - -1. In order to display the build status in a merge request you must create a project service in GitLab. -1. Your project service will do a (JSON) query to a URL of the CI tool with the SHA1 of the commit. -1. The project service builds this URL and payload based on project service settings and knowledge of the CI tool. -1. The response is parsed to give a response in GitLab (success/failed/pending). - ## Troubleshooting ### Error in merge requests - "Could not connect to the CI server" diff --git a/doc/integration/jenkins_deprecated.md b/doc/integration/jenkins_deprecated.md index 31bb271782a..5fc30bf3305 100644 --- a/doc/integration/jenkins_deprecated.md +++ b/doc/integration/jenkins_deprecated.md @@ -1,6 +1,7 @@ # Jenkins CI (deprecated) service ->**Note:** In GitLab 8.3, Jenkins integration using the +NOTE: **Note:** +In GitLab 8.3, Jenkins integration using the [GitLab Hook Plugin](https://wiki.jenkins.io/display/JENKINS/GitLab+Hook+Plugin) was deprecated in favor of the [GitLab Plugin](https://wiki.jenkins.io/display/JENKINS/GitLab+Plugin). diff --git a/doc/integration/jira_development_panel.md b/doc/integration/jira_development_panel.md index 85404ff3164..dc19d42ee2e 100644 --- a/doc/integration/jira_development_panel.md +++ b/doc/integration/jira_development_panel.md @@ -1,132 +1,148 @@ -# GitLab Jira development panel integration **(PREMIUM)** +--- +stage: Create +group: Ecosystem +info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/engineering/ux/technical-writing/#designated-technical-writers +--- + +# GitLab Jira Development Panel integration **(PREMIUM)** > [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/2381) in [GitLab Premium](https://about.gitlab.com/pricing/) 10.0. -Complementary to our [existing Jira](../user/project/integrations/jira.md) project integration, you're now able to integrate -GitLab projects with [Jira Development Panel](https://confluence.atlassian.com/adminjiraserver070/). Both can be used -simultaneously. This works with self-managed GitLab or GitLab.com integrated with: +The Jira Development Panel integration allows you to reference Jira issues within GitLab, displaying activity in the [Development panel](https://support.atlassian.com/jira-software-cloud/docs/view-development-information-for-an-issue/) in the issue. It complements the [GitLab Jira integration](../user/project/integrations/jira.md). You may choose to configure both integrations to take advantage of both sets of features. (See a [feature comparison](../user/project/integrations/jira_integrations.md#feature-comparison)). + +Depending on your environment, you can enable this integration by configuring the Jira DVCS connector or by using the GitLab for Jira app in the Atlassian Marketplace. See the [Configuration](#configuration) section for details. + +## Features -- Jira hosted by you. -- Cloud Jira. +| Your mention of Jira issue ID in GitLab context | Automated effect in Jira issue | +|---------------------------------------------------|--------------------------------------------------------------------------------------------------------| +| In a merge request | Link to the MR is displayed in Development panel. | +| In a branch name | Link to the branch is displayed in Development panel. | +| In a commit message | Link to the commit is displayed in Development panel. | +| In a commit message with Jira Smart Commit format | Displays your custom comment or logged time spent and/or performs specified issue transition on merge. | -By doing this you can easily access related GitLab merge requests, branches, and commits directly from a Jira issue. +With this integration, you can access related GitLab merge requests, branches, and commits directly from a Jira issue, reflecting your work in GitLab. From the Development panel, you can open a detailed view and take actions including creating a new merge request from a branch. For more information, see [Usage](#usage). This integration connects all GitLab projects within a top-level group or a personal namespace to projects in the Jira instance. A top-level GitLab group is one that does not have any parent group itself. All the projects of that top-level group, as well as projects of the top-level group's subgroups nesting down, are connected. Alternatively, you can specify a GitLab personal namespace in the Jira configuration, which will then connect the projects in that personal namespace to Jira. -NOTE: **Note:** -Note this is different from the [existing Jira](../user/project/integrations/jira.md) project integration, where the mapping -is one GitLab project to the entire Jira instance. +This differs from the [Jira integration](../user/project/integrations/jira.md), where the mapping is between one GitLab project and the entire Jira instance. + +## Configuration + +<i class="fa fa-youtube-play youtube" aria-hidden="true"></i> +For an overview, see [Agile Management - GitLab-Jira Development Panel Integration](https://www.youtube.com/watch?v=VjVTOmMl85M&feature=youtu.be). + +- If you're using GitLab.com and Jira Cloud, the recommended method to enable this integration is to install the [GitLab for Jira app](#gitlab-for-jira-app) from the Atlassian Marketplace, which offers a real-time sync between GitLab and Jira. +- If you're using self-managed GitLab, self-managed Jira, or both, configure the integration using [Jira's DVCS Connector](#jira-dvcs-configuration), which syncs data hourly. We recommend that a GitLab group admin or instance admin (in the case of self-managed GitLab) set up the integration, in order to simplify administration. -TIP: **Tip:** -Create and use a single-purpose `jira` user in GitLab, so that removing -regular users won't impact your integration. - -## Requirements - -### Self-managed GitLab +### Jira DVCS configuration -If you are using self-managed GitLab, make sure your GitLab instance is accessible by Jira. +NOTE: **Note:** +If you're using GitLab.com and Jira Cloud, we recommend you use the [GitLab for Jira app](#gitlab-for-jira-app), unless you have a specific need for the DVCS Connector. -- If you are connecting to Jira Cloud, make sure your instance is accessible via the internet. +- If you are using self-managed GitLab, make sure your GitLab instance is accessible by Jira. +- If you're connecting to Jira Cloud, ensure your instance is accessible through the internet. - If you are using Jira Server, make sure your instance is accessible however your network is set up. -### GitLab.com - -There are no special requirements if you are using GitLab.com. +#### GitLab account configuration for DVCS -## GitLab Configuration +TIP: **Tip:** +To ensure that regular user account maintenance doesn't impact your integration, +create and use a single-purpose `jira` user in GitLab. -1. In GitLab, create a new application in order to allow Jira to connect with your GitLab account +1. In GitLab, create a new application to allow Jira to connect with your GitLab account. - While logged-in, go to `Settings -> Applications`. (Click your profile avatar at - the top right, choose `Settings`, and then navigate to `Applications` from the left - navigation menu.) Use the form to create a new application. + While signed in to the GitLab account that you want Jira to use to connect to GitLab, + click your profile avatar at the top right, and then click **Settings > Applications**. + Use the form to create a new application. - Enter a useful name for the `Name` field. + In the **Name** field, enter a descriptive name for the integration, such as `Jira`. - For the `Redirect URI` field, enter `https://<your-gitlab-instance-domain>/login/oauth/callback`, - replacing `<your-gitlab-instance-domain>` appropriately. So for example, if you are using GitLab.com, + For the **Redirect URI** field, enter `https://<gitlab.example.com>/login/oauth/callback`, + replacing `<gitlab.example.com>` with your GitLab instance domain. For example, if you are using GitLab.com, this would be `https://gitlab.com/login/oauth/callback`. NOTE: **Note:** If using a GitLab version earlier than 11.3, the `Redirect URI` must be - `https://<your-gitlab-instance-domain>/-/jira/login/oauth/callback`. If you want Jira - to have access to all projects, GitLab recommends an administrator creates the - Application. + `https://<gitlab.example.com>/-/jira/login/oauth/callback`. If you want Jira + to have access to all projects, GitLab recommends that an administrator create the + application. -  +  - - Check `api` in the Scopes section. + - Check **API** in the Scopes section and uncheck any other checkboxes. -1. Click `Save application`. You will see the generated 'Application ID' and 'Secret' values. - Copy these values that you will use on the Jira configuration side. +1. Click **Save application**. GitLab displays the generated **Application ID** + and **Secret** values. Copy these values, which you will use in Jira. -## Jira Configuration +#### Jira DVCS Connector setup -### GitLab self-managed +NOTE: **Note:** +If you're using GitLab.com and Jira Cloud, we recommend you use the [GitLab for Jira app](#gitlab-for-jira-app), unless you have a specific need for the DVCS Connector. -1. In Jira, go to **Jira Settings > Applications > DVCS accounts**, then click **Link GitHub Enterprise account** to start creating a new integration. - (We are pretending to be GitHub in this integration until there is further platform support from Jira.) +1. Ensure you have completed the [GitLab configuration](#gitlab-account-configuration-for-dvcs). +1. If you're using Jira Server, go to **Settings (gear) > Applications > DVCS accounts**. + If you're using Jira Cloud, go to **Settings (gear) > Products > DVCS accounts**. +1. Click **Link GitHub Enterprise account** to start creating a new integration. + (We're pretending to be GitHub in this integration, until there's additional platform support in Jira.)  -1. Complete the form +1. Complete the form: - Select GitHub Enterprise for the `Host` field. + Select **GitHub Enterprise** for the **Host** field. - For the `Team or User Account` field, enter the relative path of a top-level GitLab group that you have access to, + In the **Team or User Account** field, enter the relative path of a top-level GitLab group that you have access to, or the relative path of your personal namespace.  - For the `Host URL` field, enter `https://<your-gitlab-instance-domain>/`, - replacing `<your-gitlab-instance-domain>` appropriately. So for example, if you are using GitLab.com, + In the **Host URL** field, enter `https://<gitlab.example.com>/`, + replacing `<gitlab.example.com>` with your GitLab instance domain. For example, if you are using GitLab.com, this would be `https://gitlab.com/`. NOTE: **Note:** - If using a GitLab version earlier than 11.3 the `Host URL` value should be `https://<your-gitlab-instance-domain>/-/jira` + If using a GitLab version earlier than 11.3 the **Host URL** value should be `https://<gitlab.example.com>/-/jira` - For the `Client ID` field, use the `Application ID` value from the previous section. + For the **Client ID** field, use the **Application ID** value from the previous section. - For the `Client Secret` field, use the `Secret` value from the previous section. + For the **Client Secret** field, use the **Secret** value from the previous section. Ensure that the rest of the checkboxes are checked. -1. Click `Add` to complete and create the integration. +1. Click **Add** to complete and create the integration. Jira takes up to a few minutes to know about (import behind the scenes) all the commits and branches for all the projects in the GitLab group you specified in the previous step. These are refreshed every 60 minutes. - > **Note:** - > In the future, we plan on implementing real-time integration. If you need - > to refresh the data manually, you can do this from the `Applications -> DVCS - > accounts` screen where you initially set up the integration: - > - >  + In the future, we plan on implementing real-time integration. If you need + to refresh the data manually, you can do this from the `Applications -> DVCS + accounts` screen where you initially set up the integration: + +  -To connect additional GitLab projects from other GitLab top-level groups (or personal namespaces), repeat the above +To connect additional GitLab projects from other GitLab top-level groups (or personal namespaces), repeat the previous steps with additional Jira DVCS accounts. -### GitLab.com +Now that the integration is configured, read more about how to test and use it in [Usage](#usage). -You can integrate GitLab.com and Jira Cloud using the **GitLab for Jira** App in the [Atlassian Marketplace](https://marketplace.atlassian.com/apps/1221011/gitlab-for-jira). +### GitLab for Jira app -GitLab and Jira can also be integrated using the DVCS connector as described in the [GitLab self-managed section](#gitlab-self-managed). The [GitLab for Jira App](https://marketplace.atlassian.com/apps/1221011/gitlab-for-jira) is recommended when using GitLab.com and Jira Cloud because data is synchronized in real time, while the DVCS connector updates data only once per hour. +You can integrate GitLab.com and Jira Cloud using the [GitLab for Jira](https://marketplace.atlassian.com/apps/1221011/gitlab-for-jira) app in the Atlassian Marketplace. + +This method is recommended when using GitLab.com and Jira Cloud because data is synchronized in realtime, while the DVCS connector updates data only once per hour. If you are not using both of these environments, use the [Jira DVCS Connector](#jira-dvcs-configuration) method. <i class="fa fa-youtube-play youtube" aria-hidden="true"></i> For a walkthrough of the integration with GitLab for Jira, watch [Configure GitLab Jira Integration using Marketplace App](https://youtu.be/SwR-g1s1zTo) on YouTube. -NOTE: **Note:** -The **GitLab for Jira** App is only compatible with GitLab.com **and** Jira Cloud. - 1. Go to **Jira Settings > Apps > Find new apps**, then search for GitLab. 1. Click **GitLab for Jira**, then click **Get it now**. Or go the [App in the marketplace directly](https://marketplace.atlassian.com/apps/1221011/gitlab-for-jira) @@ -143,6 +159,8 @@ The GitLab user only needs access when adding a new namespace. For syncing with After a namespace is added, all future commits, branches and merge requests of all projects under that namespace will be synced to Jira. Past data cannot be synced at the moment. +For more information, see [Usage](#usage). + #### Troubleshooting GitLab for Jira The GitLab for Jira App uses an iframe to add namespaces on the settings page. Some browsers block cross-site cookies which can lead to a message saying that the user needs to log in on GitLab.com even though the user is already logged in. @@ -165,6 +183,8 @@ Click the links to see your GitLab repository data.  +For more information on using Jira Smart Commits to track time against an issue, specify an issue transition, or add a custom comment, see the Atlassian page [Using Smart Commits](https://confluence.atlassian.com/fisheye/using-smart-commits-960155400.html). + ## Limitations - This integration is currently not supported on GitLab instances under a [relative URL](https://docs.gitlab.com/omnibus/settings/configuration.html#configuring-a-relative-url-for-gitlab) (for example, `http://example.com/gitlab`). diff --git a/doc/integration/kerberos.md b/doc/integration/kerberos.md index 56f7b50496c..09bc795f7ef 100644 --- a/doc/integration/kerberos.md +++ b/doc/integration/kerberos.md @@ -42,7 +42,7 @@ sudo chmod 0600 /etc/http.keytab **Installations from source** ->**Note:** +NOTE: **Note:** For source installations, make sure the `kerberos` gem group [has been installed](../install/installation.md#install-gems). diff --git a/doc/integration/omniauth.md b/doc/integration/omniauth.md index 2ace05a8320..9dd7f2cd9e1 100644 --- a/doc/integration/omniauth.md +++ b/doc/integration/omniauth.md @@ -1,7 +1,8 @@ # OmniAuth GitLab leverages OmniAuth to allow users to sign in using Twitter, GitHub, and -other popular services. +other popular services. [OmniAuth](https://rubygems.org/gems/omniauth/) is +"a generalized Rack framework for multiple-provider authentication, built on Ruby. Configuring OmniAuth does not prevent standard GitLab authentication or LDAP (if configured) from continuing to work. Users can choose to sign in using any @@ -42,9 +43,9 @@ contains some settings that are common for all providers. Before configuring individual OmniAuth providers there are a few global settings that are in common for all providers that we need to consider. -> **NOTE:** -> Starting from GitLab 11.4, OmniAuth is enabled by default. If you're using an -> earlier version, you'll need to explicitly enable it. +NOTE: **Note:** +Starting from GitLab 11.4, OmniAuth is enabled by default. If you're using an +earlier version, you'll need to explicitly enable it. - `allow_single_sign_on` allows you to specify the providers you want to allow to automatically create an account. It defaults to `false`. If `false` users must @@ -56,16 +57,16 @@ that are in common for all providers that we need to consider. be blocked by default and will have to be unblocked by an administrator before they are able to sign in. -> **Note:** -> If you set `block_auto_created_users` to `false`, make sure to only -> define providers under `allow_single_sign_on` that you are able to control, like -> SAML, Shibboleth, Crowd or Google, or set it to `false` otherwise any user on -> the Internet will be able to successfully sign in to your GitLab without -> administrative approval. -> -> **Note:** -> `auto_link_ldap_user` requires the `uid` of the user to be the same in both LDAP -> and the OmniAuth provider. +NOTE: **Note:** +If you set `block_auto_created_users` to `false`, make sure to only +define providers under `allow_single_sign_on` that you are able to control, like +SAML, Shibboleth, Crowd or Google, or set it to `false` otherwise any user on +the Internet will be able to successfully sign in to your GitLab without +administrative approval. + +NOTE: **Note:** +`auto_link_ldap_user` requires the `uid` of the user to be the same in both LDAP +and the OmniAuth provider. To change these settings: @@ -139,10 +140,26 @@ OmniAuth provider for an existing user. The chosen OmniAuth provider is now active and can be used to sign in to GitLab from then on. +## Automatically Link Existing Users to OmniAuth Users + +You can automatically link OmniAuth users with existing GitLab users if their email addresses match by adding the following setting: + +**For Omnibus installations** + +```ruby +gitlab_rails['omniauth_auto_link_user'] = true +``` + +**For installations from source** + +```yaml +omniauth: + auto_link_user: true +``` + ## Configure OmniAuth Providers as External ->**Note:** -This setting was introduced with version 8.7 of GitLab +> Introduced in GitLab 8.7. You can define which OmniAuth providers you want to be `external` so that all users **creating accounts, or logging in via these providers** will not be able to have @@ -150,7 +167,7 @@ access to internal projects. You will need to use the full name of the provider, like `google_oauth2` for Google. Refer to the examples for the full names of the supported providers. ->**Note:** +NOTE: **Note:** If you decide to remove an OmniAuth provider from the external providers list you will need to manually update the users that use this method to login, if you want their accounts to be upgraded to full internal accounts. @@ -170,7 +187,7 @@ omniauth: ## Using Custom OmniAuth Providers ->**Note:** +NOTE: **Note:** The following information only applies for installations from source. GitLab uses [OmniAuth](https://github.com/omniauth/omniauth) for authentication and already ships @@ -223,12 +240,11 @@ we'd like to at least help those with specific needs. ## Enable or disable Sign In with an OmniAuth provider without disabling import sources ->**Note:** -This setting was introduced with version 8.8 of GitLab. +> Introduced in GitLab 8.8. Administrators are able to enable or disable Sign In via some OmniAuth providers. ->**Note:** +NOTE: **Note:** By default Sign In is enabled via all the OAuth Providers that have been configured in `config/gitlab.yml`. In order to enable/disable an OmniAuth provider, go to Admin Area -> Settings -> Sign-in Restrictions section -> Enabled OAuth Sign-In sources and select the providers you want to enable or disable. @@ -325,3 +341,7 @@ of the OmniAuth users has admin permissions. You may also bypass the auto signin feature by browsing to `https://gitlab.example.com/users/sign_in?auto_sign_in=false`. + +## Passwords for users created via OmniAuth + +The [Generated passwords for users created through integrated authentication](../security/passwords_for_integrated_authentication_methods.md) guide provides an overview of how GitLab generates and sets passwords for users created via OmniAuth. diff --git a/doc/integration/recaptcha.md b/doc/integration/recaptcha.md index 48c83f1393b..1868711ca9c 100644 --- a/doc/integration/recaptcha.md +++ b/doc/integration/recaptcha.md @@ -1,6 +1,6 @@ # reCAPTCHA -GitLab leverages [Google's reCAPTCHA](https://www.google.com/recaptcha/intro/index.html) +GitLab leverages [Google's reCAPTCHA](https://www.google.com/recaptcha/about/) to protect against spam and abuse. GitLab displays the CAPTCHA form on the sign-up page to confirm that a real user, not a bot, is attempting to create an account. @@ -36,5 +36,5 @@ proxy_set_header X-GitLab-Show-Login-Captcha 1; In Omnibus GitLab, this can be configured via `/etc/gitlab/gitlab.rb`: ```ruby -nginx['proxy_set_headers'] = { 'X-GitLab-Show-Login-Captcha' => 1 } +nginx['proxy_set_headers'] = { 'X-GitLab-Show-Login-Captcha' => '1' } ``` diff --git a/doc/integration/saml.md b/doc/integration/saml.md index 9dce7269c86..e7e94b21683 100644 --- a/doc/integration/saml.md +++ b/doc/integration/saml.md @@ -253,8 +253,7 @@ considered admin users. ### Auditor Groups **(STARTER ONLY)** ->**Note:** -This setting is only available on GitLab 11.4 EE and above. +> Introduced in [GitLab Starter](https://about.gitlab.com/pricing/) 11.4. The requirements are the same as the previous settings, your IdP needs to pass Group information to GitLab, you need to tell GitLab where to look for the groups in the SAML response, and which group(s) should be @@ -379,7 +378,7 @@ You may also bypass the auto signin feature by browsing to ### `attribute_statements` ->**Note:** +NOTE: **Note:** This setting should only be used to map attributes that are part of the OmniAuth `info` hash schema. @@ -585,6 +584,10 @@ These attributes define the SAML user. If users can change these attributes, the Refer to the documentation for your SAML Identity Provider for information on how to fix these attributes. +## Passwords for users created via SAML + +The [Generated passwords for users created through integrated authentication](../security/passwords_for_integrated_authentication_methods.md) guide provides an overview of how GitLab generates and sets passwords for users created via SAML. + ## Troubleshooting You can find the base64-encoded SAML Response in the [`production_json.log`](../administration/logs.md#production_jsonlog). diff --git a/doc/integration/shibboleth.md b/doc/integration/shibboleth.md index a4a71c655a2..1b645541cec 100644 --- a/doc/integration/shibboleth.md +++ b/doc/integration/shibboleth.md @@ -46,7 +46,7 @@ The following changes are needed to enable Shibboleth: RequestHeader set X_FORWARDED_PROTO 'https' ``` - **NOTE:** + NOTE: **Note:** Starting from GitLab 11.4, OmniAuth is enabled by default. If you're using an earlier version, you'll need to explicitly enable it in `/etc/gitlab/gitlab.rb`. diff --git a/doc/integration/sourcegraph.md b/doc/integration/sourcegraph.md index 90e9c7b9534..c366dab49b1 100644 --- a/doc/integration/sourcegraph.md +++ b/doc/integration/sourcegraph.md @@ -1,4 +1,7 @@ --- +stage: Create +group: Source Code +info: "To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/engineering/ux/technical-writing/#designated-technical-writers" type: reference, how-to --- |