diff options
Diffstat (limited to 'doc/administration')
| -rw-r--r-- | doc/administration/auth/oidc.md | 225 | ||||
| -rw-r--r-- | doc/administration/geo/replication/configuration.md | 24 | ||||
| -rw-r--r-- | doc/administration/geo/replication/usage.md | 8 | ||||
| -rw-r--r-- | doc/administration/troubleshooting/log_parsing.md | 39 |
4 files changed, 246 insertions, 50 deletions
diff --git a/doc/administration/auth/oidc.md b/doc/administration/auth/oidc.md index b51f9e0eb99..fc24a9da604 100644 --- a/doc/administration/auth/oidc.md +++ b/doc/administration/auth/oidc.md @@ -134,25 +134,26 @@ See the [Google documentation](https://developers.google.com/identity/protocols/ for more details: ```ruby - gitlab_rails['omniauth_providers'] = [ - { - 'name' => 'openid_connect', - 'label' => 'Google OpenID', - 'args' => { - 'name' => 'openid_connect', - 'scope' => ['openid', 'profile', 'email'], - 'response_type' => 'code', - 'issuer' => 'https://accounts.google.com', - 'client_auth_method' => 'query', - 'discovery' => true, - 'uid_field' => 'preferred_username', - 'client_options' => { - 'identifier' => '<YOUR PROJECT CLIENT ID>', - 'secret' => '<YOUR PROJECT CLIENT SECRET>', - 'redirect_uri' => 'https://example.com/users/auth/openid_connect/callback', +gitlab_rails['omniauth_providers'] = [ + { + 'name' => 'openid_connect', + 'label' => 'Google OpenID', + 'args' => { + 'name' => 'openid_connect', + 'scope' => ['openid', 'profile', 'email'], + 'response_type' => 'code', + 'issuer' => 'https://accounts.google.com', + 'client_auth_method' => 'query', + 'discovery' => true, + 'uid_field' => 'preferred_username', + 'client_options' => { + 'identifier' => '<YOUR PROJECT CLIENT ID>', + 'secret' => '<YOUR PROJECT CLIENT SECRET>', + 'redirect_uri' => 'https://example.com/users/auth/openid_connect/callback', + } } - } - } + } +] ``` ### Microsoft Azure @@ -170,30 +171,178 @@ to obtain the tenant ID, client ID, and client secret for your app. Example Omnibus configuration block: ```ruby - gitlab_rails['omniauth_providers'] = [ - { - 'name' => 'openid_connect', - 'label' => 'Azure OIDC', - 'args' => { - 'name' => 'openid_connect', - 'scope' => ['openid', 'profile', 'email'], - 'response_type' => 'code', - 'issuer' => 'https://login.microsoftonline.com/<YOUR-TENANT-ID>/v2.0', - 'client_auth_method' => 'query', - 'discovery' => true, - 'uid_field' => 'preferred_username', - 'client_options' => { - 'identifier' => '<YOUR APP CLIENT ID>', - 'secret' => '<YOUR APP CLIENT SECRET>', - 'redirect_uri' => 'https://gitlab.example.com/users/auth/openid_connect/callback' - } - } - } +gitlab_rails['omniauth_providers'] = [ + { + 'name' => 'openid_connect', + 'label' => 'Azure OIDC', + 'args' => { + 'name' => 'openid_connect', + 'scope' => ['openid', 'profile', 'email'], + 'response_type' => 'code', + 'issuer' => 'https://login.microsoftonline.com/<YOUR-TENANT-ID>/v2.0', + 'client_auth_method' => 'query', + 'discovery' => true, + 'uid_field' => 'preferred_username', + 'client_options' => { + 'identifier' => '<YOUR APP CLIENT ID>', + 'secret' => '<YOUR APP CLIENT SECRET>', + 'redirect_uri' => 'https://gitlab.example.com/users/auth/openid_connect/callback' + } + } + } +] ``` Microsoft has documented how its platform works with [the OIDC protocol](https://docs.microsoft.com/en-us/azure/active-directory/develop/v2-protocols-oidc). -## Troubleshooting +### Microsoft Azure Active Directory B2C + +While GitLab works with [Azure Active Directory B2C](https://docs.microsoft.com/en-us/azure/active-directory-b2c/overview), it requires special +configuration to work. To get started, sign in to the [Azure Portal](https://portal.azure.com). +For your app, you'll need the following information from Azure: + +- A tenant ID. You may already have one. For more information, review the + [Microsoft Azure Tenant](https://docs.microsoft.com/en-us/azure/active-directory/develop/quickstart-create-new-tenant) documentation. +- A client ID and a client secret. Follow the instructions in the + [Microsoft tutorial](https://docs.microsoft.com/en-us/azure/active-directory-b2c/tutorial-register-applications?tabs=app-reg-ga) documentation to obtain the + client ID and client secret for your app. +- The user flow or policy name. Follow the instructions in the [Microsoft tutorial](https://docs.microsoft.com/en-us/azure/active-directory-b2c/tutorial-create-user-flows?pivots=b2c-user-flow). + +If your GitLab domain is `gitlab.example.com`, ensure the app has the following `Redirect URI`: + +`https://gitlab.example.com/users/auth/openid_connect/callback` + +In addition, ensure that [ID tokens are enabled](https://docs.microsoft.com/en-us/azure/active-directory-b2c/tutorial-register-applications?tabs=app-reg-ga#enable-id-token-implicit-grant). + +Add the following API permissions to the app: + +1. `openid` +1. `offline_access` + +#### Configure custom policies + +Azure B2C [offers two ways of defining the business logic for logging in a user](https://docs.microsoft.com/en-us/azure/active-directory-b2c/user-flow-overview): + +- [User flows](https://docs.microsoft.com/en-us/azure/active-directory-b2c/user-flow-overview#user-flows) +- [Custom policies](https://docs.microsoft.com/en-us/azure/active-directory-b2c/user-flow-overview#custom-policies) + +While cumbersome to configure, custom policies are required because +standard Azure B2C user flows [do not send the OpenID `email` claim](https://github.com/MicrosoftDocs/azure-docs/issues/16566). In +other words, they do not work with the [`allow_single_sign_on` or `auto_link_user` +parameters](../../integration/omniauth.md#initial-omniauth-configuration). +With a standard Azure B2C policy, GitLab cannot create a new account or +link to an existing one with an e-mail address. + +Carefully follow the instructions for [creating a custom policy](https://docs.microsoft.com/en-us/azure/active-directory-b2c/tutorial-create-user-flows?pivots=b2c-custom-policy). + +The Microsoft instructions use `SocialAndLocalAccounts` in the [custom policy starter pack](https://docs.microsoft.com/en-us/azure/active-directory-b2c/tutorial-create-user-flows?pivots=b2c-custom-policy#custom-policy-starter-pack), +but `LocalAccounts` works for authenticating against local, Active Directory accounts. Before you follow the instructions to [upload the polices](https://docs.microsoft.com/en-us/azure/active-directory-b2c/tutorial-create-user-flows?pivots=b2c-custom-policy#upload-the-policies), do the following: + +1. To export the `email` claim, modify the `SignUpOrSignin.xml`. Replace the following line: + + ```xml + <OutputClaim ClaimTypeReferenceId="email" /> + ``` + + with: + + ```xml + <OutputClaim ClaimTypeReferenceId="signInNames.emailAddress" PartnerClaimType="email" /> + ``` + +1. For OIDC discovery to work with B2C, the policy must be configured with an issuer compatible with the [OIDC +specification](https://openid.net/specs/openid-connect-discovery-1_0.html#rfc.section.4.3). +See the [token compatibility settings](https://docs.microsoft.com/en-us/azure/active-directory-b2c/configure-tokens?pivots=b2c-custom-policy#token-compatibility-settings). +In `TrustFrameworkBase.xml` under `JwtIssuer`, set `IssuanceClaimPattern` to `AuthorityWithTfp`: + + ```xml + <ClaimsProvider> + <DisplayName>Token Issuer</DisplayName> + <TechnicalProfiles> + <TechnicalProfile Id="JwtIssuer"> + <DisplayName>JWT Issuer</DisplayName> + <Protocol Name="None" /> + <OutputTokenFormat>JWT</OutputTokenFormat> + <Metadata> + <Item Key="IssuanceClaimPattern">AuthorityWithTfp</Item> + ... + ``` + +1. Now [upload the policy](https://docs.microsoft.com/en-us/azure/active-directory-b2c/tutorial-create-user-flows?pivots=b2c-custom-policy#upload-the-policies). Overwrite +the existing files if you are updating an existing policy. + +1. Determine the issuer URL using the sign-in policy. The issuer URL will be in the form: + + ```markdown + https://<YOUR-DOMAIN>/tfp/<YOUR-TENANT-ID>/<YOUR-SIGN-IN-POLICY-NAME>/v2.0/ + ``` + + The policy name is lowercased in the URL. For example, `B2C_1A_signup_signin` + policy appears as `b2c_1a_signup_sigin`. + + Note that the trailing forward slash is required. + +1. Verify the operation of the OIDC discovery URL and issuer URL, append `.well-known/openid-configuration` +to the issuer URL: + + ```markdown + https://<YOUR-DOMAIN>/tfp/<YOUR-TENANT-ID>/<YOUR-SIGN-IN-POLICY-NAME>/v2.0/.well-known/openid-configuration + ``` + + For example, if `domain` is `example.b2clogin.com` and tenant ID is `fc40c736-476c-4da1-b489-ee48cee84386`, you can use `curl` and `jq` to +extract the issuer: + + ```shell + $ curl --silent "https://example.b2clogin.com/tfp/fc40c736-476c-4da1-b489-ee48cee84386/b2c_1a_signup_signin/v2.0/.well-known/openid-configuration" | jq .issuer + "https://example.b2clogin.com/tfp/fc40c736-476c-4da1-b489-ee48cee84386/b2c_1a_signup_signin/v2.0/" + ``` + +1. Configure the issuer URL with the custom policy used for +`signup_signin`. For example, this is the Omnibus configuration with a +custom policy for `b2c_1a_signup_signin`: + +```ruby +gitlab_rails['omniauth_providers'] = [ +{ + 'name' => 'openid_connect', + 'label' => 'Azure B2C OIDC', + 'args' => { + 'name' => 'openid_connect', + 'scope' => ['openid'], + 'response_mode' => 'query', + 'response_type' => 'id_token', + 'issuer' => 'https://<YOUR-DOMAIN>/tfp/<YOUR-TENANT-ID>/b2c_1a_signup_signin/v2.0/', + 'client_auth_method' => 'query', + 'discovery' => true, + 'send_scope_to_token_endpoint' => true, + 'client_options' => { + 'identifier' => '<YOUR APP CLIENT ID>', + 'secret' => '<YOUR APP CLIENT SECRET>', + 'redirect_uri' => 'https://gitlab.example.com/users/auth/openid_connect/callback' + } + } +}] +``` + +#### Troubleshooting Azure B2C + +- Ensure all occurrences of `yourtenant.onmicrosoft.com`, `ProxyIdentityExperienceFrameworkAppId`, and `IdentityExperienceFrameworkAppId` match your B2C tenant hostname and +the respective client IDs in the XML policy files. + +- Add `https://jwt.ms` as a redirect URI to the app, and use the [custom policy tester](https://docs.microsoft.com/en-us/azure/active-directory-b2c/tutorial-create-user-flows?pivots=b2c-custom-policy#test-the-custom-policy). +Make sure the payload includes `email` that matches the user's e-mail access. + +- After you enable the custom policy, users might see "Invalid username or password" after they try to sign in. This might be a configuration +issue with the `IdentityExperienceFramework` app. See [this Microsoft comment](https://docs.microsoft.com/en-us/answers/questions/50355/unable-to-sign-on-using-custom-policy.html?childToView=122370#comment-122370) +that suggests checking that the app manifest contains these settings: + + - `"accessTokenAcceptedVersion": null` + - `"signInAudience": "AzureADMyOrg"` + 1. `"signInAudience": "AzureADMyOrg"` + + Note that this configuration corresponds with the `Supported account types` setting used when creating the `IdentityExperienceFramework` app. + +## General troubleshooting If you're having trouble, here are some tips: diff --git a/doc/administration/geo/replication/configuration.md b/doc/administration/geo/replication/configuration.md index 7dbb0c78166..76910e73055 100644 --- a/doc/administration/geo/replication/configuration.md +++ b/doc/administration/geo/replication/configuration.md @@ -24,7 +24,7 @@ You are encouraged to first read through all the steps before executing them in your testing/production environment. NOTE: -**Do not** set up any custom authentication for the **secondary** nodes. This will be handled by the **primary** node. +**Do not** set up any custom authentication for the **secondary** nodes. This is handled by the **primary** node. Any change that requires access to the **Admin Area** needs to be done in the **primary** node because the **secondary** node is a read-only replica. @@ -41,7 +41,7 @@ they must be manually replicated to the **secondary** node. sudo cat /etc/gitlab/gitlab-secrets.json ``` - This will display the secrets that need to be replicated, in JSON format. + This displays the secrets that need to be replicated, in JSON format. 1. SSH into the **secondary** node and login as the `root` user: @@ -85,11 +85,11 @@ GitLab integrates with the system-installed SSH daemon, designating a user (typically named `git`) through which all access requests are handled. In a [Disaster Recovery](../disaster_recovery/index.md) situation, GitLab system -administrators will promote a **secondary** node to the **primary** node. DNS records for the +administrators promote a **secondary** node to the **primary** node. DNS records for the **primary** domain should also be updated to point to the new **primary** node -(previously a **secondary** node). Doing so will avoid the need to update Git remotes and API URLs. +(previously a **secondary** node). Doing so avoids the need to update Git remotes and API URLs. -This will cause all SSH requests to the newly promoted **primary** node to +This causes all SSH requests to the newly promoted **primary** node to fail due to SSH host key mismatch. To prevent this, the primary SSH host keys must be manually replicated to the **secondary** node. @@ -183,7 +183,7 @@ keys must be manually replicated to the **secondary** node. sudo -i ``` -1. Edit `/etc/gitlab/gitlab.rb` and add a **unique** name for your node. You will need this in the next steps: +1. Edit `/etc/gitlab/gitlab.rb` and add a **unique** name for your node. You need this in the next steps: ```ruby # The unique identifier for the Geo node. @@ -229,9 +229,9 @@ keys must be manually replicated to the **secondary** node. gitlab-rake gitlab:geo:check ``` -Once added to the Geo administration page and restarted, the **secondary** node will automatically start +Once added to the Geo administration page and restarted, the **secondary** node automatically starts replicating missing data from the **primary** node in a process known as **backfill**. -Meanwhile, the **primary** node will start to notify each **secondary** node of any changes, so +Meanwhile, the **primary** node starts to notify each **secondary** node of any changes, so that the **secondary** node can act on those notifications immediately. Be sure the _secondary_ node is running and accessible. You can sign in to the @@ -241,7 +241,7 @@ _secondary_ node with the same credentials as were used with the _primary_ node. You can safely skip this step if your **primary** node uses a CA-issued HTTPS certificate. -If your **primary** node is using a self-signed certificate for *HTTPS* support, you will +If your **primary** node is using a self-signed certificate for *HTTPS* support, you need to add that certificate to the **secondary** node's trust store. Retrieve the certificate from the **primary** node and follow [these instructions](https://docs.gitlab.com/omnibus/settings/ssl.html) @@ -265,7 +265,7 @@ the _primary_ node. Visit the _secondary_ node's **Admin Area > Geo** (`/admin/geo/nodes`) in your browser to determine if it's correctly identified as a _secondary_ Geo node, and if Geo is enabled. -The initial replication, or 'backfill', will probably still be in progress. You +The initial replication, or 'backfill', is probably still in progress. You can monitor the synchronization process on each Geo node from the **primary** node's **Geo Nodes** dashboard in your browser. @@ -282,7 +282,7 @@ The two most obvious issues that can become apparent in the dashboard are: - You are using a custom certificate or custom CA (see the [troubleshooting document](troubleshooting.md)). - The instance is firewalled (check your firewall rules). -Please note that disabling a **secondary** node will stop the synchronization process. +Please note that disabling a **secondary** node stops the synchronization process. Please note that if `git_data_dirs` is customized on the **primary** node for multiple repository shards you must duplicate the same configuration on each **secondary** node. @@ -312,7 +312,7 @@ It is important to note that selective synchronization: 1. Does not hide project metadata from **secondary** nodes. - Since Geo currently relies on PostgreSQL replication, all project metadata gets replicated to **secondary** nodes, but repositories that have not been - selected will be empty. + selected are empty. 1. Does not reduce the number of events generated for the Geo event log. - The **primary** node generates events as long as any **secondary** nodes are present. Selective synchronization restrictions are implemented on the **secondary** nodes, diff --git a/doc/administration/geo/replication/usage.md b/doc/administration/geo/replication/usage.md index 2fcc0565567..1491aa3427e 100644 --- a/doc/administration/geo/replication/usage.md +++ b/doc/administration/geo/replication/usage.md @@ -25,3 +25,11 @@ remote: ssh://git@primary.geo/user/repo.git remote: Everything up-to-date ``` + +NOTE: +If you're using HTTPS instead of [SSH](../../../ssh/README.md) to push to the secondary, +you can't store credentials in the URL like `user:password@URL`. Instead, you can use a +[`.netrc` file](https://www.gnu.org/software/inetutils/manual/html_node/The-_002enetrc-file.html) +for Unix-like operating systems or `_netrc` for Windows. In that case, the credentials +will be stored as a plain text. If you're looking for a more secure way to store credentials, +you can use [Git Credential Storage](https://git-scm.com/book/en/v2/Git-Tools-Credential-Storage). diff --git a/doc/administration/troubleshooting/log_parsing.md b/doc/administration/troubleshooting/log_parsing.md index 2900ce58940..a0f71960e14 100644 --- a/doc/administration/troubleshooting/log_parsing.md +++ b/doc/administration/troubleshooting/log_parsing.md @@ -201,3 +201,42 @@ grep "fatal: " /var/log/gitlab/gitaly/current | \ jq '."grpc.request.glProjectPath"' | \ sort | uniq ``` + +### Parsing `gitlab-shell.log` + +For investigating Git calls via SSH, from [GitLab 12.10](https://gitlab.com/gitlab-org/gitlab-shell/-/merge_requests/367). + +Find the top 20 calls by project and user: + +```shell +jq --raw-output --slurp ' + map( + select( + .username != null and + .gl_project_path !=null + ) + ) + | group_by(.username+.gl_project_path) + | sort_by(-length) + | limit(20; .[]) + | "count: \(length)\tuser: \(.[0].username)\tproject: \(.[0].gl_project_path)" ' \ + /var/log/gitlab/gitlab-shell/gitlab-shell.log +``` + +Find the top 20 calls by project, user, and command: + +```shell +jq --raw-output --slurp ' + map( + select( + .command != null and + .username != null and + .gl_project_path !=null + ) + ) + | group_by(.username+.gl_project_path+.command) + | sort_by(-length) + | limit(20; .[]) + | "count: \(length)\tcommand: \(.[0].command)\tuser: \(.[0].username)\tproject: \(.[0].gl_project_path)" ' \ + /var/log/gitlab/gitlab-shell/gitlab-shell.log +``` |