diff options
author | GitLab Bot <gitlab-bot@gitlab.com> | 2020-10-21 10:08:36 +0300 |
---|---|---|
committer | GitLab Bot <gitlab-bot@gitlab.com> | 2020-10-21 10:08:36 +0300 |
commit | 48aff82709769b098321c738f3444b9bdaa694c6 (patch) | |
tree | e00c7c43e2d9b603a5a6af576b1685e400410dee /doc/integration | |
parent | 879f5329ee916a948223f8f43d77fba4da6cd028 (diff) |
Add latest changes from gitlab-org/gitlab@13-5-stable-eev13.5.0-rc42
Diffstat (limited to 'doc/integration')
-rw-r--r-- | doc/integration/README.md | 15 | ||||
-rw-r--r-- | doc/integration/cas.md | 8 | ||||
-rw-r--r-- | doc/integration/elasticsearch.md | 260 | ||||
-rw-r--r-- | doc/integration/external-issue-tracker.md | 19 | ||||
-rw-r--r-- | doc/integration/facebook.md | 2 | ||||
-rw-r--r-- | doc/integration/github.md | 26 | ||||
-rw-r--r-- | doc/integration/gitlab.md | 7 | ||||
-rw-r--r-- | doc/integration/gitpod.md | 19 | ||||
-rw-r--r-- | doc/integration/google.md | 7 | ||||
-rw-r--r-- | doc/integration/jira_development_panel.md | 25 | ||||
-rw-r--r-- | doc/integration/kerberos.md | 51 | ||||
-rw-r--r-- | doc/integration/omniauth.md | 28 | ||||
-rw-r--r-- | doc/integration/salesforce.md | 2 | ||||
-rw-r--r-- | doc/integration/saml.md | 116 | ||||
-rw-r--r-- | doc/integration/sourcegraph.md | 2 | ||||
-rw-r--r-- | doc/integration/twitter.md | 3 |
16 files changed, 353 insertions, 237 deletions
diff --git a/doc/integration/README.md b/doc/integration/README.md index c5c21644d1c..c8ce367e99f 100644 --- a/doc/integration/README.md +++ b/doc/integration/README.md @@ -8,15 +8,8 @@ GitLab can be integrated with external services for enhanced functionality. ## Issue trackers -You can use an [external issue tracker](external-issue-tracker.md) at the same time as the GitLab issue tracker, or use only the external issue tracker. - -GitLab can be integrated with the following external issue trackers: - -- Jira -- Redmine -- Bugzilla -- EWM -- YouTrack +You can use an [external issue tracker](external-issue-tracker.md) at the same time as the GitLab +issue tracker, or use only the external issue tracker. ## Authentication sources @@ -26,10 +19,10 @@ GitLab can be configured to authenticate access requests with the following auth - Enable sign in with [Bitbucket](bitbucket.md) accounts. - Configure GitLab to sign in using [CAS](cas.md). - Integrate with [Kerberos](kerberos.md). -- Enable sign in via [LDAP](ldap.md). +- Enable sign in via [LDAP](../administration/auth/ldap/index.md). - Enable [OAuth2 provider](oauth_provider.md) application creation. - Use [OmniAuth](omniauth.md) to enable sign in via Twitter, GitHub, GitLab.com, Google, -Bitbucket, Facebook, Shibboleth, SAML, Crowd, Azure or Authentiq ID. + Bitbucket, Facebook, Shibboleth, SAML, Crowd, Azure or Authentiq ID. - Use GitLab as an [OpenID Connect](openid_connect_provider.md) identity provider. - Authenticate to [Vault](vault.md) through GitLab OpenID Connect. - Configure GitLab as a [SAML](saml.md) 2.0 Service Provider. diff --git a/doc/integration/cas.md b/doc/integration/cas.md index cf4e501e772..eee801350eb 100644 --- a/doc/integration/cas.md +++ b/doc/integration/cas.md @@ -45,10 +45,10 @@ To enable the CAS OmniAuth provider you must register your application with your - { name: 'cas3', label: 'cas', args: { - url: 'CAS_SERVER', - login_url: '/CAS_PATH/login', - service_validate_url: '/CAS_PATH/p3/serviceValidate', - logout_url: '/CAS_PATH/logout'} } + url: 'CAS_SERVER', + login_url: '/CAS_PATH/login', + service_validate_url: '/CAS_PATH/p3/serviceValidate', + logout_url: '/CAS_PATH/logout' } } ``` 1. Change 'CAS_PATH' to the root of your CAS instance (ie. `cas`). diff --git a/doc/integration/elasticsearch.md b/doc/integration/elasticsearch.md index f40955ad8ff..a88f2db5c26 100644 --- a/doc/integration/elasticsearch.md +++ b/doc/integration/elasticsearch.md @@ -10,8 +10,8 @@ info: To determine the technical writer assigned to the Stage/Group associated w > - [Introduced](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/109 "Elasticsearch Merge Request") in GitLab [Starter](https://about.gitlab.com/pricing/) 8.4. > - Support for [Amazon Elasticsearch](https://docs.aws.amazon.com/elasticsearch-service/latest/developerguide/es-gsg.html) was [introduced](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/1305) in GitLab [Starter](https://about.gitlab.com/pricing/) 9.0. -This document describes how to set up Elasticsearch with GitLab. After -Elasticsearch is enabled, you'll have the benefit of fast search response times +This document describes how to enable Advanced Search. After +Advanced Search is enabled, you'll have the benefit of fast search response times and the advantage of the following special searches: - [Advanced Search](../user/search/advanced_global_search.md) @@ -66,21 +66,19 @@ source. You must [install it separately](https://www.elastic.co/guide/en/elastic Be sure to select your version. Providing detailed information on installing Elasticsearch is out of the scope of this document. -NOTE: **Note:** Elasticsearch should be installed on a separate server, whether you install it yourself or use a cloud hosted offering like Elastic's [Elasticsearch Service](https://www.elastic.co/elasticsearch/service) (available on AWS, GCP, or Azure) or the [Amazon Elasticsearch](https://docs.aws.amazon.com/elasticsearch-service/latest/developerguide/es-gsg.html) service. Running Elasticsearch on the same server as GitLab is not recommended and can cause a degradation in GitLab instance performance. -NOTE: **Note:** **For a single node Elasticsearch cluster the functional cluster health status will be yellow** (will never be green) because the primary shard is allocated but replicas can not be as there is no other node to which Elasticsearch can assign a replica. After the data is added to the database or repository and [Elasticsearch is -enabled in the Admin Area](#enabling-elasticsearch) the search index will be +enabled in the Admin Area](#enabling-advanced-search) the search index will be updated automatically. ## Elasticsearch repository indexer @@ -155,15 +153,19 @@ Example: PREFIX=/usr sudo -E make install ``` -After installation, be sure to [enable Elasticsearch](#enabling-elasticsearch). - -## Enabling Elasticsearch +After installation, be sure to [enable Elasticsearch](#enabling-advanced-search). NOTE: **Note:** +If you see an error such as `Permission denied - /home/git/gitlab-elasticsearch-indexer/` while indexing, you +may need to set the `production -> elasticsearch -> indexer_path` setting in your `gitlab.yml` file to +`/usr/local/bin/gitlab-elasticsearch-indexer`, which is where the binary is installed. + +## Enabling Advanced Search + For GitLab instances with more than 50GB repository data you can follow the instructions for [Indexing large instances](#indexing-large-instances) below. -To enable Elasticsearch, you need to have admin access to GitLab: +To enable Advanced Search, you need to have admin access to GitLab: 1. Navigate to **Admin Area** (wrench icon), then **Settings > General** and expand the **Advanced Search** section. @@ -172,23 +174,12 @@ To enable Elasticsearch, you need to have admin access to GitLab: To see the Advanced Search section, you need an active Starter [license](../user/admin_area/license.md). -1. Configure the [Elasticsearch settings](#elasticsearch-configuration) for - your Elasticsearch cluster. Do not enable **Elasticsearch indexing** or - **Search with Elasticsearch enabled** yet. -1. Click **Save changes** for the changes to take effect. -1. Before enabling **Elasticsearch indexing** you need to create an index by - running the Rake task: - - ```shell - # Omnibus installations - sudo gitlab-rake gitlab:elastic:create_empty_index - - # Installations from source - bundle exec rake gitlab:elastic:create_empty_index RAILS_ENV=production - ``` - -1. Now enable `Elasticsearch indexing` in **Admin Area > Settings > - General > Advanced Search** and click **Save changes**. +1. Configure the [Advanced Search settings](#advanced-search-configuration) for + your Elasticsearch cluster. Do not enable **Search with Elasticsearch enabled** + yet. +1. Now enable **Elasticsearch indexing** in **Admin Area > Settings > + General > Advanced Search** and click **Save changes**. This will create + an empty index if one does not already exist. 1. Click **Index all projects**. 1. Click **Check progress** in the confirmation message to see the status of the background jobs. @@ -206,13 +197,13 @@ To enable Elasticsearch, you need to have admin access to GitLab: **Admin Area > Settings > General > Advanced Search** and click **Save changes**. -### Elasticsearch configuration +### Advanced Search configuration The following Elasticsearch settings are available: | Parameter | Description | |-------------------------------------------------------|-------------| -| `Elasticsearch indexing` | Enables or disables Elasticsearch indexing. You may want to enable indexing but disable search in order to give the index time to be fully completed, for example. Also, keep in mind that this option doesn't have any impact on existing data, this only enables/disables the background indexer which tracks data changes and ensures new data is indexed. | +| `Elasticsearch indexing` | Enables or disables Elasticsearch indexing and creates an empty index if one does not already exist. You may want to enable indexing but disable search in order to give the index time to be fully completed, for example. Also, keep in mind that this option doesn't have any impact on existing data, this only enables/disables the background indexer which tracks data changes and ensures new data is indexed. | | `Pause Elasticsearch indexing` | Enables or disables temporary indexing pause. This is useful for cluster migration/reindexing. All changes are still tracked, but they are not committed to the Elasticsearch index until unpaused. | | `Search with Elasticsearch enabled` | Enables or disables using Elasticsearch in search. | | `URL` | The URL to use for connecting to Elasticsearch. Use a comma-separated list to support clustering (e.g., `http://host1, https://host2:9200`). If your Elasticsearch instance is password protected, pass the `username:password` in the URL (e.g., `http://<username>:<password>@<elastic_host>:9200/`). | @@ -238,14 +229,14 @@ If you select `Limit namespaces and projects that can be indexed`, more options You can select namespaces and projects to index exclusively. Note that if the namespace is a group it will include any sub-groups and projects belonging to those sub-groups to be indexed as well. -Elasticsearch only provides cross-group code/commit search (global) if all name-spaces are indexed. In this particular scenario where only a subset of namespaces are indexed, a global search will not provide a code or commit scope. This will be possible only in the scope of an indexed namespace. Currently there is no way to code/commit search in multiple indexed namespaces (when only a subset of namespaces has been indexed). For example if two groups are indexed, there is no way to run a single code search on both. You can only run a code search on the first group and then on the second. +Advanced Search only provides cross-group code/commit search (global) if all name-spaces are indexed. In this particular scenario where only a subset of namespaces are indexed, a global search will not provide a code or commit scope. This will be possible only in the scope of an indexed namespace. Currently there is no way to code/commit search in multiple indexed namespaces (when only a subset of namespaces has been indexed). For example if two groups are indexed, there is no way to run a single code search on both. You can only run a code search on the first group and then on the second. You can filter the selection dropdown by writing part of the namespace or project name you're interested in. ![limit namespace filter](img/limit_namespace_filter.png) NOTE: **Note:** -If no namespaces or projects are selected, no Elasticsearch indexing will take place. +If no namespaces or projects are selected, no Advanced Search indexing will take place. CAUTION: **Warning:** If you have already indexed your instance, you will have to regenerate the index in order to delete all existing data @@ -253,7 +244,7 @@ for filtering to work correctly. To do this run the Rake tasks `gitlab:elastic:r `gitlab:elastic:clear_index_status`. Afterwards, removing a namespace or a project from the list will delete the data from the Elasticsearch index as expected. -## Disabling Elasticsearch +## Disabling Advanced Search To disable the Elasticsearch integration: @@ -283,7 +274,7 @@ alias such as we can change the underlying index at will. NOTE: **Note:** Any index attached to the production alias is deemed a `primary` and will be -used by the GitLab Elasticsearch integration. +used by the GitLab Advanced Search integration. ### Pause the indexing @@ -315,7 +306,6 @@ CAUTION: **Caution:** It is highly recommended that you take a snapshot of your cluster to ensure there is a recovery path if anything goes wrong. -NOTE: **Note:** Due to a technical limitation, there will be a slight downtime because of the fact that we need to reclaim the current `primary` index to be used as the alias. @@ -413,7 +403,6 @@ To trigger the re-index from `primary` index: Under **Admin Area > Settings > General > Advanced Search > Elasticsearch zero-downtime reindexing**, click on **Trigger cluster reindexing**. -NOTE: **Note:** Reindexing can be a lengthy process depending on the size of your Elasticsearch cluster. CAUTION: **Caution:** @@ -421,12 +410,12 @@ After the reindexing is completed, the original index will be scheduled to be de While the reindexing is running, you will be able to follow its progress under that same section. -## GitLab Elasticsearch Rake tasks +## GitLab Advanced Search Rake tasks Rake tasks are available to: - [Build and install](#building-and-installing) the indexer. -- Delete indexes when [disabling Elasticsearch](#disabling-elasticsearch). +- Delete indexes when [disabling Elasticsearch](#disabling-advanced-search). - Add GitLab data to an index. The following are some available Rake tasks: @@ -436,7 +425,7 @@ The following are some available Rake tasks: | [`sudo gitlab-rake gitlab:elastic:index`](https://gitlab.com/gitlab-org/gitlab/blob/master/ee/lib/tasks/gitlab/elastic.rake) | Enables Elasticsearch indexing and run `gitlab:elastic:create_empty_index`, `gitlab:elastic:clear_index_status`, `gitlab:elastic:index_projects`, and `gitlab:elastic:index_snippets`. | | [`sudo gitlab-rake gitlab:elastic:index_projects`](https://gitlab.com/gitlab-org/gitlab/blob/master/ee/lib/tasks/gitlab/elastic.rake) | Iterates over all projects and queues Sidekiq jobs to index them in the background. | | [`sudo gitlab-rake gitlab:elastic:index_projects_status`](https://gitlab.com/gitlab-org/gitlab/blob/master/ee/lib/tasks/gitlab/elastic.rake) | Determines the overall status of the indexing. It is done by counting the total number of indexed projects, dividing by a count of the total number of projects, then multiplying by 100. | -| [`sudo gitlab-rake gitlab:elastic:clear_index_status`](https://gitlab.com/gitlab-org/gitlab/blob/master/ee/lib/tasks/gitlab/elastic.rake) | Deletes all instances of IndexStatus for all projects. | +| [`sudo gitlab-rake gitlab:elastic:clear_index_status`](https://gitlab.com/gitlab-org/gitlab/blob/master/ee/lib/tasks/gitlab/elastic.rake) | Deletes all instances of IndexStatus for all projects. Note that this command will result in a complete wipe of the index, and it should be used with caution. | | [`sudo gitlab-rake gitlab:elastic:create_empty_index[<TARGET_NAME>]`](https://gitlab.com/gitlab-org/gitlab/blob/master/ee/lib/tasks/gitlab/elastic.rake) | Generates an empty index and assigns an alias for it on the Elasticsearch side only if it doesn't already exist. | | [`sudo gitlab-rake gitlab:elastic:delete_index[<TARGET_NAME>]`](https://gitlab.com/gitlab-org/gitlab/blob/master/ee/lib/tasks/gitlab/elastic.rake) | Removes the GitLab index and alias (if exists) on the Elasticsearch instance. | | [`sudo gitlab-rake gitlab:elastic:recreate_index[<TARGET_NAME>]`](https://gitlab.com/gitlab-org/gitlab/blob/master/ee/lib/tasks/gitlab/elastic.rake) | Wrapper task for `gitlab:elastic:delete_index[<TARGET_NAME>]` and `gitlab:elastic:create_empty_index[<TARGET_NAME>]`. | @@ -467,7 +456,7 @@ Indexing project repositories...I, [2019-03-04T21:27:03.083410 #3384] INFO -- : I, [2019-03-04T21:27:05.215266 #3384] INFO -- : Indexing GitLab User / test (ID=33) is done! ``` -## Elasticsearch index scopes +## Advanced Search index scopes When performing a search, the GitLab index will use the following scopes: @@ -499,7 +488,7 @@ For basic guidance on choosing a cluster configuration you may refer to [Elastic - `refresh_interval` is a per index setting. You may want to adjust that from default `1s` to a bigger value if you don't need data in realtime. This will change how soon you will see fresh results. If that's important for you, you should leave it as close as possible to the default value. - You might want to raise [`indices.memory.index_buffer_size`](https://www.elastic.co/guide/en/elasticsearch/reference/current/indexing-buffer.html) to 30% or 40% if you have a lot of heavy indexing operations. -### Elasticsearch integration settings guidance +### Advanced Search integration settings guidance - The `Number of Elasticsearch shards` setting usually corresponds with the number of CPUs available in your cluster. For example, if you have a 3-node cluster with 4 cores each, this means you will benefit from having at least 3*4=12 shards in the cluster. Please note, it's only possible to change the shards number by using [Split index API](https://www.elastic.co/guide/en/elasticsearch/reference/current/indices-split-index.html) or by reindexing to a different index with a changed number of shards. - The `Number of Elasticsearch replicas` setting should most of the time be equal to `1` (each shard will have 1 replica). Using `0` is not recommended, because losing one node will corrupt the index. @@ -507,7 +496,7 @@ For basic guidance on choosing a cluster configuration you may refer to [Elastic ### Indexing large instances This section may be helpful in the event that the other -[basic instructions](#enabling-elasticsearch) cause problems +[basic instructions](#enabling-advanced-search) cause problems due to large volumes of data being indexed. CAUTION: **Warning:** @@ -516,7 +505,7 @@ Make sure to prepare for this task by having a [Scalable and Highly Available Setup](../administration/reference_architectures/index.md) or creating [extra Sidekiq processes](../administration/operations/extra_sidekiq_processes.md). -1. [Configure your Elasticsearch host and port](#enabling-elasticsearch). +1. [Configure your Elasticsearch host and port](#enabling-advanced-search). 1. Create empty indexes: ```shell @@ -537,7 +526,7 @@ Sidekiq processes](../administration/operations/extra_sidekiq_processes.md). bundle exec rake gitlab:elastic:clear_index_status RAILS_ENV=production ``` -1. [Enable **Elasticsearch indexing**](#enabling-elasticsearch). +1. [Enable **Elasticsearch indexing**](#enabling-advanced-search). 1. Indexing large Git repositories can take a while. To speed up the process, you can [tune for indexing speed](https://www.elastic.co/guide/en/elasticsearch/reference/current/tune-for-indexing-speed.html#tune-for-indexing-speed): - You can temporarily disable [`refresh`](https://www.elastic.co/guide/en/elasticsearch/reference/current/indices-refresh.html), the operation responsible for making changes to an index available to search. @@ -664,7 +653,7 @@ Sidekiq processes](../administration/operations/extra_sidekiq_processes.md). } }' ``` -1. After the indexing has completed, enable [**Search with Elasticsearch enabled**](#enabling-elasticsearch). +1. After the indexing has completed, enable [**Search with Elasticsearch enabled**](#enabling-advanced-search). ### Deleted documents @@ -698,96 +687,114 @@ However, some larger installations may wish to tune the merge policy settings: ## Troubleshooting -### Common issues +Here are some common pitfalls and how to overcome them. + +### How can I verify that my GitLab instance is using Elasticsearch? -Here are some common pitfalls and how to overcome them: +There are a couple of ways to achieve that: -- **How can I verify my GitLab instance is using Elasticsearch?** +- Whenever you perform a search there will be a link on the search results page + in the top right hand corner saying "Advanced search functionality is enabled". + This is always correctly identifying whether the current project/namespace + being searched is using Elasticsearch. - The easiest method is via the rails console (`sudo gitlab-rails console`) by running the following: +- From the admin area under **Settings > General > Elasticsearch** check that the + Advanced Search settings are checked. + + Those same settings there can be obtained from the Rails console if necessary: ```ruby - u = User.find_by_username('your-username') - s = SearchService.new(u, {:search => 'search_term'}) - pp s.search_objects.class.name + ::Gitlab::CurrentSettings.elasticsearch_search? # Whether or not searches will use Elasticsearch + ::Gitlab::CurrentSettings.elasticsearch_indexing? # Whether or not content will be indexed in Elasticsearch + ::Gitlab::CurrentSettings.elasticsearch_limit_indexing? # Whether or not Elasticsearch is limited only to certain projects/namespaces ``` - If you see `"ActiveRecord::Relation"`, you are **not** using Elasticsearch. - - If you see `"Kaminari::PaginatableArray"` you are using Elasticsearch. +- If Elasticsearch is limited to specific namespaces and you need to know if + Elasticsearch is being used for a specific project or namespace, you can use + the Rails console: - NOTE: **Note:** - The above instructions are used to verify that GitLab is using Elasticsearch only when indexing all namespaces. This is not to be used for scenarios that only index a [subset of namespaces](#limiting-namespaces-and-projects). + ```ruby + ::Gitlab::CurrentSettings.search_using_elasticsearch?(scope: Namespace.find_by_full_path("/my-namespace")) + ::Gitlab::CurrentSettings.search_using_elasticsearch?(scope: Project.find_by_full_path("/my-namespace/my-project")) + ``` -- **I updated GitLab and now I can't find anything** +### I updated GitLab and now I can't find anything - We continuously make updates to our indexing strategies and aim to support - newer versions of Elasticsearch. When indexing changes are made, it may - be necessary for you to [reindex](#zero-downtime-reindexing) after updating GitLab. +We continuously make updates to our indexing strategies and aim to support +newer versions of Elasticsearch. When indexing changes are made, it may +be necessary for you to [reindex](#zero-downtime-reindexing) after updating GitLab. -- **I indexed all the repositories but I can't find anything** +### I indexed all the repositories but I can't get any hits for my search term in the UI - Make sure you indexed all the database data [as stated above](#enabling-elasticsearch). +Make sure you indexed all the database data [as stated above](#enabling-advanced-search). - Beyond that, check via the [Elasticsearch Search API](https://www.elastic.co/guide/en/elasticsearch/reference/current/search-search.html) to see if the data shows up on the Elasticsearch side. +If there aren't any results (hits) in the UI search, check if you are seeing the same results via the rails console (`sudo gitlab-rails console`): - If it shows up via the [Elasticsearch Search API](https://www.elastic.co/guide/en/elasticsearch/reference/current/search-search.html), check that it shows up via the rails console (`sudo gitlab-rails console`): +```ruby +u = User.find_by_username('your-username') +s = SearchService.new(u, {:search => 'search_term', :scope => 'blobs'}) +pp s.search_objects.to_a +``` - ```ruby - u = User.find_by_username('your-username') - s = SearchService.new(u, {:search => 'search_term', :scope => 'blobs'}) - pp s.search_objects.to_a - ``` +Beyond that, check via the [Elasticsearch Search API](https://www.elastic.co/guide/en/elasticsearch/reference/current/search-search.html) to see if the data shows up on the Elasticsearch side: - NOTE: **Note:** - The above instructions are not to be used for scenarios that only index a [subset of namespaces](#limiting-namespaces-and-projects). +```shell +curl --request GET <elasticsearch_server_ip>:9200/gitlab-production/_search?q=<search_term> +``` - See [Elasticsearch Index Scopes](#elasticsearch-index-scopes) for more information on searching for specific types of data. +More [complex Elasticsearch API calls](https://www.elastic.co/guide/en/elasticsearch/reference/current/query-filter-context.html) are also possible. -- **I indexed all the repositories but then switched Elasticsearch servers and now I can't find anything** +It is important to understand at which level the problem is manifesting (UI, Rails code, Elasticsearch side) to be able to [troubleshoot further](../administration/troubleshooting/elasticsearch.md#search-results-workflow). - You will need to re-run all the Rake tasks to reindex the database, repositories, and wikis. +NOTE: **Note:** +The above instructions are not to be used for scenarios that only index a [subset of namespaces](#limiting-namespaces-and-projects). -- **The indexing process is taking a very long time** +See [Elasticsearch Index Scopes](#advanced-search-index-scopes) for more information on searching for specific types of data. - The more data present in your GitLab instance, the longer the indexing process takes. +### I indexed all the repositories but then switched Elasticsearch servers and now I can't find anything -- **There are some projects that weren't indexed, but we don't know which ones** +You will need to re-run all the Rake tasks to reindex the database, repositories, and wikis. - You can run `sudo gitlab-rake gitlab:elastic:projects_not_indexed` to display projects that aren't indexed. +### The indexing process is taking a very long time -- **No new data is added to the Elasticsearch index when I push code** +The more data present in your GitLab instance, the longer the indexing process takes. - NOTE: **Note:** - This was [fixed in GitLab 13.2](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/35936) and the Rake task is not available for versions greater than that. +### There are some projects that weren't indexed, but I don't know which ones - When performing the initial indexing of blobs, we lock all projects until the project finishes indexing. It could happen that an error during the process causes one or multiple projects to remain locked. In order to unlock them, run: +You can run `sudo gitlab-rake gitlab:elastic:projects_not_indexed` to display projects that aren't indexed. - ```shell - sudo gitlab-rake gitlab:elastic:clear_locked_projects - ``` +### No new data is added to the Elasticsearch index when I push code -- **"Can't specify parent if no parent field has been configured"** +NOTE: **Note:** +This was [fixed in GitLab 13.2](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/35936) and the Rake task is not available for versions greater than that. - If you enabled Elasticsearch before GitLab 8.12 and have not rebuilt indexes you will get - exception in lots of different cases: +When performing the initial indexing of blobs, we lock all projects until the project finishes indexing. It could happen that an error during the process causes one or multiple projects to remain locked. In order to unlock them, run: - ```plaintext - Elasticsearch::Transport::Transport::Errors::BadRequest([400] { - "error": { - "root_cause": [{ - "type": "illegal_argument_exception", - "reason": "Can't specify parent if no parent field has been configured" - }], - "type": "illegal_argument_exception", - "reason": "Can't specify parent if no parent field has been configured" - }, - "status": 400 - }): - ``` +```shell +sudo gitlab-rake gitlab:elastic:clear_locked_projects +``` - This is because we changed the index mapping in GitLab 8.12 and the old indexes should be removed and built from scratch again, - see details in the [update guide](../update/upgrading_from_source.md). +### `Can't specify parent if no parent field has been configured` error + +If you enabled Elasticsearch before GitLab 8.12 and have not rebuilt indexes you will get +exception in lots of different cases: + +```plaintext +Elasticsearch::Transport::Transport::Errors::BadRequest([400] { + "error": { + "root_cause": [{ + "type": "illegal_argument_exception", + "reason": "Can't specify parent if no parent field has been configured" + }], + "type": "illegal_argument_exception", + "reason": "Can't specify parent if no parent field has been configured" + }, + "status": 400 +}): +``` + +This is because we changed the index mapping in GitLab 8.12 and the old indexes should be removed and built from scratch again, +see details in the [update guide](../update/upgrading_from_source.md). - Exception `Elasticsearch::Transport::Transport::Errors::BadRequest` @@ -809,50 +816,51 @@ Here are some common pitfalls and how to overcome them: for this setting ("Maximum Size of HTTP Request Payloads"), based on the size of the underlying instance. -- **My single node Elasticsearch cluster status never goes from `yellow` to `green` even though everything seems to be running properly** +### My single node Elasticsearch cluster status never goes from `yellow` to `green` even though everything seems to be running properly - **For a single node Elasticsearch cluster the functional cluster health status will be yellow** (will never be green) because the primary shard is allocated but replicas can not be as there is no other node to which Elasticsearch can assign a replica. This also applies if you are using the -[Amazon Elasticsearch](https://docs.aws.amazon.com/elasticsearch-service/latest/developerguide/aes-handling-errors.html#aes-handling-errors-yellow-cluster-status) service. +**For a single node Elasticsearch cluster the functional cluster health status will be yellow** (never green) because the primary shard is allocated but replicas cannot be as there is no other node to which Elasticsearch can assign a replica. This also applies if you are using the [Amazon Elasticsearch](https://docs.aws.amazon.com/elasticsearch-service/latest/developerguide/aes-handling-errors.html#aes-handling-errors-yellow-cluster-status) service. - CAUTION: **Warning:** - Setting the number of replicas to `0` is not something that we recommend (this is not allowed in the GitLab Elasticsearch Integration menu). If you are planning to add more Elasticsearch nodes (for a total of more than 1 Elasticsearch) the number of replicas will need to be set to an integer value larger than `0`. Failure to do so will result in lack of redundancy (losing one node will corrupt the index). +CAUTION: **Warning:** +Setting the number of replicas to `0` is discouraged (this is not allowed in the GitLab Elasticsearch Integration menu). If you are planning to add more Elasticsearch nodes (for a total of more than 1 Elasticsearch) the number of replicas will need to be set to an integer value larger than `0`. Failure to do so will result in lack of redundancy (losing one node will corrupt the index). - If you have a **hard requirement to have a green status for your single node Elasticsearch cluster**, please make sure you understand the risks outlined in the previous paragraph and then simply run the following query to set the number of replicas to `0`(the cluster will no longer try to create any shard replicas): +If you have a **hard requirement to have a green status for your single node Elasticsearch cluster**, please make sure you understand the risks outlined in the previous paragraph and then run the following query to set the number of replicas to `0`(the cluster will no longer try to create any shard replicas): - ```shell - curl --request PUT localhost:9200/gitlab-production/_settings --header 'Content-Type: application/json' --data '{ - "index" : { - "number_of_replicas" : 0 - } - }' - ``` +```shell +curl --request PUT localhost:9200/gitlab-production/_settings --header 'Content-Type: application/json' --data '{ +"index" : { + "number_of_replicas" : 0 + } +}' +``` -- **I'm getting a `health check timeout: no Elasticsearch node available` error in Sidekiq during the indexing process** +### `health check timeout: no Elasticsearch node available` error in Sidekiq - ```plaintext - Gitlab::Elastic::Indexer::Error: time="2020-01-23T09:13:00Z" level=fatal msg="health check timeout: no Elasticsearch node available" - ``` +If you're getting a `health check timeout: no Elasticsearch node available` error in Sidekiq during the indexing process: + +```plaintext +Gitlab::Elastic::Indexer::Error: time="2020-01-23T09:13:00Z" level=fatal msg="health check timeout: no Elasticsearch node available" +``` - You probably have not used either `http://` or `https://` as part of your value in the **"URL"** field of the Elasticsearch Integration Menu. Please make sure you are using either `http://` or `https://` in this field as the [Elasticsearch client for Go](https://github.com/olivere/elastic) that we are using [needs the prefix for the URL to be accepted as valid](https://github.com/olivere/elastic/commit/a80af35aa41856dc2c986204e2b64eab81ccac3a). - Once you have corrected the formatting of the URL, delete the index (via the [dedicated Rake task](#gitlab-elasticsearch-rake-tasks)) and [reindex the content of your instance](#enabling-elasticsearch). +You probably have not used either `http://` or `https://` as part of your value in the **"URL"** field of the Elasticsearch Integration Menu. Please make sure you are using either `http://` or `https://` in this field as the [Elasticsearch client for Go](https://github.com/olivere/elastic) that we are using [needs the prefix for the URL to be accepted as valid](https://github.com/olivere/elastic/commit/a80af35aa41856dc2c986204e2b64eab81ccac3a). +Once you have corrected the formatting of the URL, delete the index (via the [dedicated Rake task](#gitlab-advanced-search-rake-tasks)) and [reindex the content of your instance](#enabling-advanced-search). ### Low-level troubleshooting There is a [more structured, lower-level troubleshooting document](../administration/troubleshooting/elasticsearch.md) for when you experience other issues, including poor performance. -### Known Issues +### Known issues -- **[Elasticsearch `code_analyzer` doesn't account for all code cases](https://gitlab.com/groups/gitlab-org/-/epics/3621)** +[Elasticsearch `code_analyzer` doesn't account for all code cases](https://gitlab.com/groups/gitlab-org/-/epics/3621). - The `code_analyzer` pattern and filter configuration is being evaluated for improvement. We have fixed [most edge cases](https://gitlab.com/groups/gitlab-org/-/epics/3621#note_363429094) that were not returning expected search results due to our pattern and filter configuration. +The `code_analyzer` pattern and filter configuration is being evaluated for improvement. We have fixed [most edge cases](https://gitlab.com/groups/gitlab-org/-/epics/3621#note_363429094) that were not returning expected search results due to our pattern and filter configuration. - Improvements to the `code_analyzer` pattern and filters is being discussed in [epic 3621](https://gitlab.com/groups/gitlab-org/-/epics/3621). +Improvements to the `code_analyzer` pattern and filters are being discussed in [epic 3621](https://gitlab.com/groups/gitlab-org/-/epics/3621). ### Reverting to Basic Search Sometimes there may be issues with your Elasticsearch index data and as such GitLab will allow you to revert to "basic search" when there are no search results and assuming that basic search is supported in that scope. This "basic -search" will behave as though you don't have Elasticsearch enabled at all for +search" will behave as though you don't have Advanced Search enabled at all for your instance and search using other data sources (ie. PostgreSQL data and Git data). diff --git a/doc/integration/external-issue-tracker.md b/doc/integration/external-issue-tracker.md index cde0093f53e..96c9b9d7f62 100644 --- a/doc/integration/external-issue-tracker.md +++ b/doc/integration/external-issue-tracker.md @@ -1,7 +1,7 @@ # External issue tracker -GitLab has a great [issue tracker](../user/project/issues/index.md) but you can also use an external one -such as Jira, Redmine, YouTrack, Bugzilla, or EWM. External issue trackers are configurable per GitLab project. +GitLab has a great [issue tracker](../user/project/issues/index.md) but you can also use an external +one. External issue trackers are configurable per GitLab project. Once configured, you can reference external issues using the format `CODE-123`, where: @@ -15,24 +15,23 @@ GitLab menu always opens the internal issue tracker. When disabled, the link is ## Configuration -The configuration is done via a project's **Integrations**. +The configuration is done via a project's **Settings > Integrations**. ### Integration To enable an external issue tracker you must configure the appropriate **Integration**. Visit the links below for details: -- [Redmine](../user/project/integrations/redmine.md) -- [YouTrack](../user/project/integrations/youtrack.md) -- [Jira](../user/project/integrations/jira.md) - [Bugzilla](../user/project/integrations/bugzilla.md) -- [EWM](../user/project/integrations/ewm.md) - [Custom Issue Tracker](../user/project/integrations/custom_issue_tracker.md) +- [Engineering Workflow Management](../user/project/integrations/ewm.md) +- [Jira](../user/project/integrations/jira.md) +- [Redmine](../user/project/integrations/redmine.md) +- [YouTrack](../user/project/integrations/youtrack.md) ### Service Template -To save you the hassle from configuring each project's service individually, -GitLab provides the ability to set Service Templates which can then be -overridden in each project's settings. +To avoid configuring each project's service individually, GitLab provides the ability to set +Service Templates. These can then be overridden in each project's settings. Read more on [Services Templates](../user/project/integrations/services_templates.md). diff --git a/doc/integration/facebook.md b/doc/integration/facebook.md index fc65191994d..dbefb560fe7 100644 --- a/doc/integration/facebook.md +++ b/doc/integration/facebook.md @@ -83,7 +83,7 @@ To enable the Facebook OmniAuth provider you must register your application with ```yaml - { name: 'facebook', app_id: 'YOUR_APP_ID', - app_secret: 'YOUR_APP_SECRET' } + app_secret: 'YOUR_APP_SECRET' } ``` 1. Change 'YOUR_APP_ID' to the API key from Facebook page in step 10. diff --git a/doc/integration/github.md b/doc/integration/github.md index ccce4672cbf..ce2b50acc54 100644 --- a/doc/integration/github.md +++ b/doc/integration/github.md @@ -71,17 +71,18 @@ Follow these steps to incorporate the GitHub OAuth 2 app in your GitLab server: ```yaml - { name: 'github', app_id: 'YOUR_APP_ID', - app_secret: 'YOUR_APP_SECRET', - args: { scope: 'user:email' } } + app_secret: 'YOUR_APP_SECRET', + args: { scope: 'user:email' } } ``` For GitHub Enterprise: ```yaml - - { name: 'github', app_id: 'YOUR_APP_ID', - app_secret: 'YOUR_APP_SECRET', - url: "https://github.example.com/", - args: { scope: 'user:email' } } + - { name: 'github', + app_id: 'YOUR_APP_ID', + app_secret: 'YOUR_APP_SECRET', + url: "https://github.example.com/", + args: { scope: 'user:email' } } ``` **Replace `https://github.example.com/` with your GitHub URL.** @@ -125,11 +126,12 @@ omnibus_gitconfig['system'] = { "http" => ["sslVerify = false"] } For installation from source: ```yaml -- { name: 'github', app_id: 'YOUR_APP_ID', - app_secret: 'YOUR_APP_SECRET', - url: "https://github.example.com/", - verify_ssl: false, - args: { scope: 'user:email' } } +- { name: 'github', + app_id: 'YOUR_APP_ID', + app_secret: 'YOUR_APP_SECRET', + url: "https://github.example.com/", + verify_ssl: false, + args: { scope: 'user:email' } } ``` You will also need to disable Git SSL verification on the server hosting GitLab. @@ -148,7 +150,7 @@ via Omnibus, or [restart GitLab](../administration/restart_gitlab.md#installatio Check the [`production.log`](../administration/logs.md#productionlog) on your GitLab server to obtain further details. If you are getting the error like `Faraday::ConnectionFailed (execution expired)` in the log, there may be a connectivity issue -between your GitLab instance and GitHub Enterprise. To verify it, [start the rails console](../administration/troubleshooting/debug.md#starting-a-rails-console-session) +between your GitLab instance and GitHub Enterprise. To verify it, [start the rails console](../administration/operations/rails_console.md#starting-a-rails-console-session) and run the commands below replacing `<github_url>` with the URL of your GitHub Enterprise instance: ```ruby diff --git a/doc/integration/gitlab.md b/doc/integration/gitlab.md index f423863ce8e..a200f6b6470 100644 --- a/doc/integration/gitlab.md +++ b/doc/integration/gitlab.md @@ -63,9 +63,10 @@ GitLab.com will generate an application ID and secret key for you to use. For installations from source: ```yaml - - { name: 'gitlab', app_id: 'YOUR_APP_ID', - app_secret: 'YOUR_APP_SECRET', - args: { scope: 'api' } } + - { name: 'gitlab', + app_id: 'YOUR_APP_ID', + app_secret: 'YOUR_APP_SECRET', + args: { scope: 'api' } } ``` 1. Change 'YOUR_APP_ID' to the Application ID from the GitLab.com application page. diff --git a/doc/integration/gitpod.md b/doc/integration/gitpod.md index f26483e3b5e..b4d12b90be0 100644 --- a/doc/integration/gitpod.md +++ b/doc/integration/gitpod.md @@ -8,7 +8,8 @@ info: "To determine the technical writer assigned to the Stage/Group associated # Gitpod Integration > - [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/228893) in GitLab 13.4. -> - It's [deployed behind a feature flag](#enable-or-disable-the-gitpod-integration), disabled by default. +> - It was [deployed behind a feature flag](#enable-or-disable-the-gitpod-integration), disabled by default. +> - [Became enabled by default](https://gitlab.com/gitlab-org/gitlab/-/issues/258206) in GitLab 13.5. > - It's enabled on GitLab.com. > - It's recommended for production use. > - To use it in GitLab self-managed instances, ask a GitLab administrator to [enable it](#configure-your-gitlab-instance-with-gitpod). **(CORE ONLY)** @@ -50,25 +51,25 @@ can follow the same steps once the integration has been enabled and configured b If you are new to Gitpod, head over to the [Gitpod documentation](https://www.gitpod.io/docs/self-hosted/latest/self-hosted/) and get your instance up and running. -1. In GitLab, go to **Admin Area > Settings > Integrations**. +1. In GitLab, go to **Admin Area > Settings > General**. 1. Expand the **Gitpod** configuration section. 1. Check **Enable Gitpod**. 1. Add your Gitpod instance URL (for example, `https://gitpod.example.com`). ## Enable or disable the Gitpod integration **(CORE ONLY)** -The Gitpod integration is under development and not ready for production use. It is deployed behind a -feature flag that is **disabled by default**. +The Gitpod integration is deployed behind a feature flag that is **enabled by default**. [GitLab administrators with access to the GitLab Rails console](../administration/feature_flags.md) -can enable it. +can enable or disable it. -To enable it: +To disable it: ```ruby -Feature.enable(:gitpod) +Feature.disable(:gitpod) ``` -To disable it: +To enable it: ```ruby -Feature.disable(:gitpod) +Feature.enable(:gitpod) +``` diff --git a/doc/integration/google.md b/doc/integration/google.md index 0f848bbc7aa..4cf589c1da8 100644 --- a/doc/integration/google.md +++ b/doc/integration/google.md @@ -84,9 +84,10 @@ On your GitLab server: For installations from source: ```yaml - - { name: 'google_oauth2', app_id: 'YOUR_APP_ID', - app_secret: 'YOUR_APP_SECRET', - args: { access_type: 'offline', approval_prompt: '' } } + - { name: 'google_oauth2', + app_id: 'YOUR_APP_ID', + app_secret: 'YOUR_APP_SECRET', + args: { access_type: 'offline', approval_prompt: '' } } ``` 1. Change `YOUR_APP_ID` to the client ID from the Google Developer page diff --git a/doc/integration/jira_development_panel.md b/doc/integration/jira_development_panel.md index 9b7aa5829c1..b86eb1c38b6 100644 --- a/doc/integration/jira_development_panel.md +++ b/doc/integration/jira_development_panel.md @@ -148,7 +148,7 @@ GitLab. No other error messages appear in any logs. If there was an issue with SSL/TLS, this error message will be generated. -- The [GitLab Jira integration](jira.md) requires GitLab to connect to Jira. Any +- The [GitLab Jira integration](../user/project/integrations/jira.md) requires GitLab to connect to Jira. Any TLS issues that arise from a private certificate authority or self-signed certificate [are resolved on the GitLab server](https://docs.gitlab.com/omnibus/settings/ssl.html#other-certificate-authorities), as GitLab is the TLS client. @@ -184,6 +184,16 @@ The message `Successfully connected` indicates a successful TLS handshake. If there are problems, the Java TLS library generates errors that you can look up for more detail. +##### Scope error when connecting Jira via DVCS + +```plaintext +The requested scope is invalid, unknown, or malformed. +``` + +Potential resolutions: + +- Verify the URL includes `scope=api` on the end of the URL. + ##### Jira error adding account and no repositories listed ```plaintext @@ -253,13 +263,18 @@ The GitLab for Jira App uses an iframe to add namespaces on the settings page. S > "You need to sign in or sign up before continuing." -In this case, enable cross-site cookies in your browser. +In this case, use [Firefox](https://www.mozilla.org/en-US/firefox/), [Google Chrome](https://www.google.com/chrome/index.html) or enable cross-site cookies in your browser. ## Usage -Once the integration is set up on GitLab and Jira you may refer any Jira issue by its ID in branch names, commit messages and merge request titles on GitLab's side, -and you will be able to see the linked `branches`, `commits`, and `merge requests` when entering a Jira issue -(inside the Jira issue, merge requests will be called "pull requests"). +After the integration is set up on GitLab and Jira, you can: + +- Refer to any Jira issue by its ID in GitLab branch names, commit messages, and merge request + titles. +- See the linked branches, commits, and merge requests in Jira issues (merge requests are + called "pull requests" in Jira issues). + +Jira issue IDs must be formatted in uppercase for the integration to work. ![Branch, Commit and Pull Requests links on Jira issue](img/jira_dev_panel_jira_setup_3.png) diff --git a/doc/integration/kerberos.md b/doc/integration/kerberos.md index 1b14b5a986f..1a193deca18 100644 --- a/doc/integration/kerberos.md +++ b/doc/integration/kerberos.md @@ -1,3 +1,10 @@ +--- +stage: Create +group: Source Code +info: "To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/engineering/ux/technical-writing/#designated-technical-writers" +type: reference, how-to +--- + # Kerberos integration **(STARTER ONLY)** GitLab can integrate with [Kerberos](https://web.mit.edu/kerberos/) as an authentication mechanism. @@ -114,6 +121,40 @@ Taken together, these rules mean that linking will only work if your users' Kerberos usernames are of the form `foo@AD.EXAMPLE.COM` and their LDAP Distinguished Names look like `sAMAccountName=foo,dc=ad,dc=example,dc=com`. +### Custom allowed realms + +[Introduced](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/9962) in GitLab 13.5. + +You can configure custom allowed realms when +the user's Kerberos realm doesn't match the domain from the user's LDAP DN. The +configuration value must specify all domains that users may be expected to +have. Any other domains will be ignored and an LDAP identity will not be linked. + +**For Omnibus installations** + +1. Edit `/etc/gitlab/gitlab.rb`: + + ```ruby + gitlab_rails['kerberos_simple_ldap_linking_allowed_realms'] = ['example.com','kerberos.example.com'] + ``` + +1. Save the file and [reconfigure](../administration/restart_gitlab.md#omnibus-gitlab-reconfigure) + GitLab for the changes to take effect. + +--- + +**For installations from source** + +1. Edit `config/gitlab.yml`: + + ```yaml + kerberos: + simple_ldap_linking_allowed_realms: ['example.com','kerberos.example.com'] + ``` + +1. Save the file and [restart](../administration/restart_gitlab.md#installations-from-source) + GitLab for the changes to take effect. + ## HTTP Git access A linked Kerberos account enables you to `git pull` and `git push` using your @@ -123,6 +164,13 @@ GitLab users with a linked Kerberos account can also `git pull` and `git push` using Kerberos tokens, i.e., without having to send their password with each operation. +DANGER: **Danger:** +There is a [known issue](https://github.com/curl/curl/issues/1261) with `libcurl` +older than version 7.64.1 wherein it won't reuse connections when negotiating. +This leads to authorization issues when push is larger than `http.postBuffer` +config. Ensure that Git is using at least `libcurl` 7.64.1 to avoid this. To +know the `libcurl` version installed, run `curl-config --version`. + ### HTTP Git access with Kerberos token (passwordless authentication) #### Support for Git before 2.4 @@ -207,9 +255,10 @@ remove the OmniAuth provider named `kerberos` from your `gitlab.yml` / ```yaml omniauth: + # Rest of configuration omitted # ... providers: - - { name: 'kerberos' } # <-- remove this line + - { name: 'kerberos' } # <-- remove this line ``` 1. [Restart GitLab](../administration/restart_gitlab.md#installations-from-source) for the changes to take effect. diff --git a/doc/integration/omniauth.md b/doc/integration/omniauth.md index dd183ad9eb0..cf09c2f2803 100644 --- a/doc/integration/omniauth.md +++ b/doc/integration/omniauth.md @@ -50,7 +50,7 @@ earlier version, you'll need to explicitly enable it. - `allow_single_sign_on` allows you to specify the providers you want to allow to automatically create an account. It defaults to `false`. If `false` users must be created manually or they will not be able to sign in via OmniAuth. -- `auto_link_ldap_user` can be used if you have [LDAP / ActiveDirectory](ldap.md) +- `auto_link_ldap_user` can be used if you have [LDAP / ActiveDirectory](../administration/auth/ldap/index.md) integration enabled. It defaults to `false`. When enabled, users automatically created through an OmniAuth provider will have their LDAP identity created in GitLab as well. - `block_auto_created_users` defaults to `true`. If `true` auto created users will @@ -104,21 +104,21 @@ To change these settings: ```yaml ## OmniAuth settings - omniauth: - # Allow login via Twitter, Google, etc. using OmniAuth providers - # Versions prior to 11.4 require this to be set to true - # enabled: true + omniauth: + # Allow login via Twitter, Google, etc. using OmniAuth providers + # Versions prior to 11.4 require this to be set to true + # enabled: true - # CAUTION! - # This allows users to login without having a user account first. Define the allowed providers - # using an array, e.g. ["saml", "twitter"], or as true/false to allow all providers or none. - # User accounts will be created automatically when authentication was successful. - allow_single_sign_on: ["saml", "twitter"] + # CAUTION! + # This allows users to login without having a user account first. Define the allowed providers + # using an array, e.g. ["saml", "twitter"], or as true/false to allow all providers or none. + # User accounts will be created automatically when authentication was successful. + allow_single_sign_on: ["saml", "twitter"] - auto_link_ldap_user: true + auto_link_ldap_user: true - # Locks down those users until they have been cleared by the admin (default: true). - block_auto_created_users: true + # Locks down those users until they have been cleared by the admin (default: true). + block_auto_created_users: true ``` Now we can choose one or more of the [Supported Providers](#supported-providers) @@ -142,7 +142,7 @@ The chosen OmniAuth provider is now active and can be used to sign in to GitLab ## Automatically Link Existing Users to OmniAuth Users -> [Introduced in GitLab 13.4.](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/36664) +> [Introduced](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/36664) in GitLab 13.4. You can automatically link OmniAuth users with existing GitLab users if their email addresses match. For example, the following setting is used to enable the auto link feature for both a SAML provider and the Twitter OAuth provider: diff --git a/doc/integration/salesforce.md b/doc/integration/salesforce.md index 7e0b2518e76..dbd0a03e3cf 100644 --- a/doc/integration/salesforce.md +++ b/doc/integration/salesforce.md @@ -64,7 +64,7 @@ To get the credentials (a pair of Client ID and Client Secret), you must [create - { name: 'salesforce', app_id: 'SALESFORCE_CLIENT_ID', app_secret: 'SALESFORCE_CLIENT_SECRET' - } + } ``` 1. Change `SALESFORCE_CLIENT_ID` to the Consumer Key from the Salesforce connected application page. diff --git a/doc/integration/saml.md b/doc/integration/saml.md index e7e94b21683..ee08a0026cd 100644 --- a/doc/integration/saml.md +++ b/doc/integration/saml.md @@ -113,16 +113,16 @@ in your SAML IdP: omniauth: providers: - { - name: 'saml', - args: { - assertion_consumer_service_url: 'https://gitlab.example.com/users/auth/saml/callback', - idp_cert_fingerprint: '43:51:43:a1:b5:fc:8b:b7:0a:3a:a9:b1:0f:66:73:a8', - idp_sso_target_url: 'https://login.example.com/idp', - issuer: 'https://gitlab.example.com', - name_identifier_format: 'urn:oasis:names:tc:SAML:2.0:nameid-format:persistent' - }, - label: 'Company Login' # optional label for SAML login button, defaults to "Saml" - } + name: 'saml', + args: { + assertion_consumer_service_url: 'https://gitlab.example.com/users/auth/saml/callback', + idp_cert_fingerprint: '43:51:43:a1:b5:fc:8b:b7:0a:3a:a9:b1:0f:66:73:a8', + idp_sso_target_url: 'https://login.example.com/idp', + issuer: 'https://gitlab.example.com', + name_identifier_format: 'urn:oasis:names:tc:SAML:2.0:nameid-format:persistent' + }, + label: 'Company Login' # optional label for SAML login button, defaults to "Saml" + } ``` 1. Change the value for `assertion_consumer_service_url` to match the HTTPS endpoint @@ -210,7 +210,7 @@ Example: idp_sso_target_url: 'https://login.example.com/idp', issuer: 'https://gitlab.example.com', name_identifier_format: 'urn:oasis:names:tc:SAML:2.0:nameid-format:transient' - } } + } } ``` ### External Groups **(STARTER ONLY)** @@ -228,7 +228,7 @@ SAML login supports automatic identification on whether a user should be conside idp_sso_target_url: 'https://login.example.com/idp', issuer: 'https://gitlab.example.com', name_identifier_format: 'urn:oasis:names:tc:SAML:2.0:nameid-format:persistent' - } } + } } ``` ### Admin Groups **(STARTER ONLY)** @@ -248,7 +248,7 @@ considered admin users. idp_sso_target_url: 'https://login.example.com/idp', issuer: 'https://gitlab.example.com', name_identifier_format: 'urn:oasis:names:tc:SAML:2.0:nameid-format:transient' - } } + } } ``` ### Auditor Groups **(STARTER ONLY)** @@ -270,7 +270,7 @@ considered auditor users. idp_sso_target_url: 'https://login.example.com/idp', issuer: 'https://gitlab.example.com', name_identifier_format: 'urn:oasis:names:tc:SAML:2.0:nameid-format:transient' - } } + } } ``` ## Bypass two factor authentication @@ -328,22 +328,22 @@ In addition to the changes in GitLab, make sure that your IdP is returning the omniauth: providers: - { - name: 'saml', - args: { - assertion_consumer_service_url: 'https://gitlab.example.com/users/auth/saml/callback', - idp_cert_fingerprint: '43:51:43:a1:b5:fc:8b:b7:0a:3a:a9:b1:0f:66:73:a8', - idp_sso_target_url: 'https://login.example.com/idp', - issuer: 'https://gitlab.example.com', - name_identifier_format: 'urn:oasis:names:tc:SAML:2.0:nameid-format:persistent', - upstream_two_factor_authn_contexts: - [ - 'urn:oasis:names:tc:SAML:2.0:ac:classes:CertificateProtectedTransport', - 'urn:oasis:names:tc:SAML:2.0:ac:classes:SecondFactorOTPSMS', - 'urn:oasis:names:tc:SAML:2.0:ac:classes:SecondFactorIGTOKEN' - ] - }, - label: 'Company Login' # optional label for SAML login button, defaults to "Saml" - } + name: 'saml', + args: { + assertion_consumer_service_url: 'https://gitlab.example.com/users/auth/saml/callback', + idp_cert_fingerprint: '43:51:43:a1:b5:fc:8b:b7:0a:3a:a9:b1:0f:66:73:a8', + idp_sso_target_url: 'https://login.example.com/idp', + issuer: 'https://gitlab.example.com', + name_identifier_format: 'urn:oasis:names:tc:SAML:2.0:nameid-format:persistent', + upstream_two_factor_authn_contexts: + [ + 'urn:oasis:names:tc:SAML:2.0:ac:classes:CertificateProtectedTransport', + 'urn:oasis:names:tc:SAML:2.0:ac:classes:SecondFactorOTPSMS', + 'urn:oasis:names:tc:SAML:2.0:ac:classes:SecondFactorIGTOKEN' + ] + }, + label: 'Company Login' # optional label for SAML login button, defaults to "Saml" + } ``` 1. Save the file and [restart GitLab](../administration/restart_gitlab.md#installations-from-source) for the changes to take effect @@ -436,7 +436,7 @@ args: { issuer: 'https://gitlab.example.com', name_identifier_format: 'urn:oasis:names:tc:SAML:2.0:nameid-format:persistent', attribute_statements: { email: ['EmailAddress'] }, - allowed_clock_drift: 1 # for one second clock drift + allowed_clock_drift: 1 # for one second clock drift } ``` @@ -561,10 +561,10 @@ args: { <redacted> -----END PRIVATE KEY-----', security: { - authn_requests_signed: true, # enable signature on AuthNRequest - want_assertions_signed: true, # enable the requirement of signed assertion - embed_sign: true, # embedded signature or HTTP GET parameter signature - metadata_signed: false, # enable signature on Metadata + authn_requests_signed: true, # enable signature on AuthNRequest + want_assertions_signed: true, # enable the requirement of signed assertion + embed_sign: true, # embedded signature or HTTP GET parameter signature + metadata_signed: false, # enable signature on Metadata signature_method: 'http://www.w3.org/2001/04/xmldsig-more#rsa-sha256', digest_method: 'http://www.w3.org/2001/04/xmlenc#sha256', } @@ -588,6 +588,52 @@ Refer to the documentation for your SAML Identity Provider for information on ho The [Generated passwords for users created through integrated authentication](../security/passwords_for_integrated_authentication_methods.md) guide provides an overview of how GitLab generates and sets passwords for users created via SAML. +## Configuring Group SAML on a self-managed GitLab instance **(PREMIUM ONLY)** + +For information on the GitLab.com implementation, please see the [SAML SSO for GitLab.com groups page](../user/group/saml_sso). + +Group SAML SSO helps if you need to allow access via multiple SAML identity providers, but as a multi-tenant solution is less suited to cases where you administer your own GitLab instance. + +To proceed with configuring Group SAML SSO instead, you'll need to enable the `group_saml` OmniAuth provider. This can be done from: + +- `gitlab.rb` for [Omnibus GitLab installations](#omnibus-installations). +- `gitlab/config/gitlab.yml` for [source installations](#source-installations). + +### Limitations + +Group SAML on a self-managed instance is limited when compared to the recommended +[instance-wide SAML](../user/group/saml_sso/index.md). The recommended solution allows you to take advantage of: + +- [LDAP compatibility](../administration/auth/ldap/index.md). +- [LDAP Group Sync](../user/group/index.md#manage-group-memberships-via-ldap). +- [Required groups](#required-groups). +- [Admin groups](#admin-groups). +- [Auditor groups](#auditor-groups). + +### Omnibus installations + +1. Make sure GitLab is + [configured with HTTPS](../install/installation.md#using-https). +1. Enable OmniAuth and the `group_saml` provider in `gitlab.rb`: + + ```ruby + gitlab_rails['omniauth_enabled'] = true + gitlab_rails['omniauth_providers'] = [{ name: 'group_saml' }] + ``` + +### Source installations + +1. Make sure GitLab is + [configured with HTTPS](../install/installation.md#using-https). +1. Enable OmniAuth and the `group_saml` provider in `gitlab/config/gitlab.yml`: + + ```yaml + omniauth: + enabled: true + providers: + - { name: 'group_saml' } + ``` + ## Troubleshooting You can find the base64-encoded SAML Response in the [`production_json.log`](../administration/logs.md#production_jsonlog). diff --git a/doc/integration/sourcegraph.md b/doc/integration/sourcegraph.md index c366dab49b1..47c84643a7d 100644 --- a/doc/integration/sourcegraph.md +++ b/doc/integration/sourcegraph.md @@ -74,7 +74,7 @@ You can skip this step if you already have your GitLab repositories searchable i ### Configure your GitLab instance with Sourcegraph -1. In GitLab, go to **Admin Area > Settings > Integrations**. +1. In GitLab, go to **Admin Area > Settings > General**. 1. Expand the **Sourcegraph** configuration section. 1. Check **Enable Sourcegraph**. 1. Set the Sourcegraph URL to your Sourcegraph instance, e.g., `https://sourcegraph.example.com`. diff --git a/doc/integration/twitter.md b/doc/integration/twitter.md index 95de56f24d8..e501eac0c5f 100644 --- a/doc/integration/twitter.md +++ b/doc/integration/twitter.md @@ -65,7 +65,8 @@ To enable the Twitter OmniAuth provider you must register your application with For installations from source: ```yaml - - { name: 'twitter', app_id: 'YOUR_APP_ID', + - { name: 'twitter', + app_id: 'YOUR_APP_ID', app_secret: 'YOUR_APP_SECRET' } ``` |