diff options
Diffstat (limited to 'doc')
| -rw-r--r-- | doc/administration/operations/fast_ssh_key_lookup.md | 170 | ||||
| -rw-r--r-- | doc/administration/operations/img/write_to_authorized_keys_setting.png | bin | 0 -> 94218 bytes | |||
| -rw-r--r-- | doc/administration/operations/index.md | 3 | ||||
| -rw-r--r-- | doc/administration/operations/speed_up_ssh.md | 1 | ||||
| -rw-r--r-- | doc/api/commits.md | 1 | ||||
| -rw-r--r-- | doc/api/repositories.md | 2 | ||||
| -rw-r--r-- | doc/ci/examples/code_climate.md | 3 | ||||
| -rw-r--r-- | doc/ci/ssh_keys/README.md | 2 | ||||
| -rw-r--r-- | doc/development/architecture.md | 2 | ||||
| -rw-r--r-- | doc/development/changelog.md | 2 | ||||
| -rw-r--r-- | doc/development/fe_guide/style_guide_js.md | 37 | ||||
| -rw-r--r-- | doc/user/permissions.md | 19 | ||||
| -rw-r--r-- | doc/user/project/clusters/index.md | 3 | ||||
| -rw-r--r-- | doc/user/project/integrations/irker.md | 6 | ||||
| -rw-r--r-- | doc/user/project/integrations/webhooks.md | 6 |
15 files changed, 215 insertions, 42 deletions
diff --git a/doc/administration/operations/fast_ssh_key_lookup.md b/doc/administration/operations/fast_ssh_key_lookup.md new file mode 100644 index 00000000000..835ed8c8006 --- /dev/null +++ b/doc/administration/operations/fast_ssh_key_lookup.md @@ -0,0 +1,170 @@ +# Fast lookup of authorized SSH keys in the database + +Regular SSH operations become slow as the number of users grows because OpenSSH +searches for a key to authorize a user via a linear search. In the worst case, +such as when the user is not authorized to access GitLab, OpenSSH will scan the +entire file to search for a key. This can take significant time and disk I/O, +which will delay users attempting to push or pull to a repository. Making +matters worse, if users add or remove keys frequently, the operating system may +not be able to cache the `authorized_keys` file, which causes the disk to be +accessed repeatedly. + +GitLab Shell solves this by providing a way to authorize SSH users via a fast, +indexed lookup in the GitLab database. This page describes how to enable the fast +lookup of authorized SSH keys. + +> **Warning:** OpenSSH version 6.9+ is required because +`AuthorizedKeysCommand` must be able to accept a fingerprint. These +instructions will break installations using older versions of OpenSSH, such as +those included with CentOS 6 as of September 2017. If you want to use this +feature for CentOS 6, follow [the instructions on how to build and install a custom OpenSSH package](#compiling-a-custom-version-of-openssh-for-centos-6) before continuing. + +## Setting up fast lookup via GitLab Shell + +GitLab Shell provides a way to authorize SSH users via a fast, indexed lookup +to the GitLab database. GitLab Shell uses the fingerprint of the SSH key to +check whether the user is authorized to access GitLab. + +Add the following to your `sshd_config` file. This is usuaully located at +`/etc/ssh/sshd_config`, but it will be `/assets/sshd_config` if you're using +Omnibus Docker: + +``` +AuthorizedKeysCommand /opt/embedded/gitlab-shell/bin/gitlab-shell-authorized-keys-check git %u %k +AuthorizedKeysCommandUser git +``` + +Reload OpenSSH: + +```bash +# Debian or Ubuntu installations +sudo service ssh reload + +# CentOS installations +sudo service sshd reload +``` + +Confirm that SSH is working by removing your user's SSH key in the UI, adding a +new one, and attempting to pull a repo. + +> **Warning:** Do not disable writes until SSH is confirmed to be working +perfectly, because the file will quickly become out-of-date. + +In the case of lookup failures (which are not uncommon), the `authorized_keys` +file will still be scanned. So git SSH performance will still be slow for many +users as long as a large file exists. + +You can disable any more writes to the `authorized_keys` file by unchecking +`Write to "authorized_keys" file` in the Application Settings of your GitLab +installation. + + + +Again, confirm that SSH is working by removing your user's SSH key in the UI, +adding a new one, and attempting to pull a repo. + +Then you can backup and delete your `authorized_keys` file for best performance. + +## How to go back to using the `authorized_keys` file + +This is a brief overview. Please refer to the above instructions for more context. + +1. [Rebuild the `authorized_keys` file](../raketasks/maintenance.md#rebuild-authorized_keys-file) +1. Enable writes to the `authorized_keys` file in Application Settings +1. Remove the `AuthorizedKeysCommand` lines from `/etc/ssh/sshd_config` or from `/assets/sshd_config` if you are using Omnibus Docker. +1. Reload sshd: `sudo service sshd reload` +1. Remove the `/opt/gitlab-shell/authorized_keys` file + +## Compiling a custom version of OpenSSH for CentOS 6 + +Building a custom version of OpenSSH is not necessary for Ubuntu 16.04 users, +since Ubuntu 16.04 ships with OpenSSH 7.2. + +It is also unnecessary for CentOS 7.4 users, as that version ships with +OpenSSH 7.4. If you are using CentOS 7.0 - 7.3, we strongly recommend that you +upgrade to CentOS 7.4 instead of following this procedure. This should be as +simple as running `yum update`. + +CentOS 6 users must build their own OpenSSH package to enable SSH lookups via +the database. The following instructions can be used to build OpenSSH 7.5: + +1. First, download the package and install the required packages: + + ``` + sudo su - + cd /tmp + curl --remote-name https://mirrors.evowise.com/pub/OpenBSD/OpenSSH/portable/openssh-7.5p1.tar.gz + tar xzvf openssh-7.5p1.tar.gz + yum install rpm-build gcc make wget openssl-devel krb5-devel pam-devel libX11-devel xmkmf libXt-devel + ``` + +3. Prepare the build by copying files to the right place: + + ``` + mkdir -p /root/rpmbuild/{SOURCES,SPECS} + cp ./openssh-7.5p1/contrib/redhat/openssh.spec /root/rpmbuild/SPECS/ + cp openssh-7.5p1.tar.gz /root/rpmbuild/SOURCES/ + cd /root/rpmbuild/SPECS + ``` + +3. Next, set the spec settings properly: + + ``` + sed -i -e "s/%define no_gnome_askpass 0/%define no_gnome_askpass 1/g" openssh.spec + sed -i -e "s/%define no_x11_askpass 0/%define no_x11_askpass 1/g" openssh.spec + sed -i -e "s/BuildPreReq/BuildRequires/g" openssh.spec + ``` + +3. Build the RPMs: + + ``` + rpmbuild -bb openssh.spec + ``` + +4. Ensure the RPMs were built: + + ``` + ls -al /root/rpmbuild/RPMS/x86_64/ + ``` + + You should see something as the following: + + ``` + total 1324 + drwxr-xr-x. 2 root root 4096 Jun 20 19:37 . + drwxr-xr-x. 3 root root 19 Jun 20 19:37 .. + -rw-r--r--. 1 root root 470828 Jun 20 19:37 openssh-7.5p1-1.x86_64.rpm + -rw-r--r--. 1 root root 490716 Jun 20 19:37 openssh-clients-7.5p1-1.x86_64.rpm + -rw-r--r--. 1 root root 17020 Jun 20 19:37 openssh-debuginfo-7.5p1-1.x86_64.rpm + -rw-r--r--. 1 root root 367516 Jun 20 19:37 openssh-server-7.5p1-1.x86_64.rpm + ``` + +5. Install the packages. OpenSSH packages will replace `/etc/pam.d/sshd` + with its own version, which may prevent users from logging in, so be sure + that the file is backed up and restored after installation: + + ``` + timestamp=$(date +%s) + cp /etc/pam.d/sshd pam-ssh-conf-$timestamp + rpm -Uvh /root/rpmbuild/RPMS/x86_64/*.rpm + yes | cp pam-ssh-conf-$timestamp /etc/pam.d/sshd + ``` + +6. Verify the installed version. In another window, attempt to login to the server: + + ``` + ssh -v <your-centos-machine> + ``` + + You should see a line that reads: "debug1: Remote protocol version 2.0, remote software version OpenSSH_7.5" + + If not, you may need to restart sshd (e.g. `systemctl restart sshd.service`). + +7. *IMPORTANT!* Open a new SSH session to your server before exiting to make + sure everything is working! If you need to downgrade, simple install the + older package: + + ``` + # Only run this if you run into a problem logging in + yum downgrade openssh-server openssh openssh-clients + ``` diff --git a/doc/administration/operations/img/write_to_authorized_keys_setting.png b/doc/administration/operations/img/write_to_authorized_keys_setting.png Binary files differnew file mode 100644 index 00000000000..232765f1917 --- /dev/null +++ b/doc/administration/operations/img/write_to_authorized_keys_setting.png diff --git a/doc/administration/operations/index.md b/doc/administration/operations/index.md index 320d71a9527..5655b7efec6 100644 --- a/doc/administration/operations/index.md +++ b/doc/administration/operations/index.md @@ -13,4 +13,5 @@ by GitLab to another file system or another server. that to prioritize important jobs. - [Sidekiq MemoryKiller](sidekiq_memory_killer.md): Configure Sidekiq MemoryKiller to restart Sidekiq. -- [Unicorn](unicorn.md): Understand Unicorn and unicorn-worker-killer.
\ No newline at end of file +- [Unicorn](unicorn.md): Understand Unicorn and unicorn-worker-killer. +- [Speed up SSH operations](fast_ssh_key_lookup.md): Authorize SSH users via a fast, indexed lookup to the GitLab database. diff --git a/doc/administration/operations/speed_up_ssh.md b/doc/administration/operations/speed_up_ssh.md new file mode 100644 index 00000000000..89265b3018b --- /dev/null +++ b/doc/administration/operations/speed_up_ssh.md @@ -0,0 +1 @@ +This document was moved to [another location](fast_ssh_key_lookup.md). diff --git a/doc/api/commits.md b/doc/api/commits.md index c9b72d4a1dd..63554c63057 100644 --- a/doc/api/commits.md +++ b/doc/api/commits.md @@ -159,6 +159,7 @@ Parameters: | --------- | ---- | -------- | ----------- | | `id` | integer/string | yes | The ID or [URL-encoded path of the project](README.md#namespaced-path-encoding) owned by the authenticated user | `sha` | string | yes | The commit hash or name of a repository branch or tag | +| `stats` | boolean | no | Include commit stats. Default is true | ```bash curl --header "PRIVATE-TOKEN: 9koXpg98eAheJpvBs5tK" "https://gitlab.example.com/api/v4/projects/5/repository/commits/master diff --git a/doc/api/repositories.md b/doc/api/repositories.md index 03b32577872..5fb25e40ed7 100644 --- a/doc/api/repositories.md +++ b/doc/api/repositories.md @@ -113,7 +113,7 @@ GET /projects/:id/repository/archive Parameters: - `id` (required) - The ID or [URL-encoded path of the project](README.md#namespaced-path-encoding) owned by the authenticated user -- `sha` (optional) - The commit SHA to download defaults to the tip of the default branch +- `sha` (optional) - The commit SHA to download. A tag, branch reference or sha can be used. This defaults to the tip of the default branch if not specified ## Compare branches, tags or commits diff --git a/doc/ci/examples/code_climate.md b/doc/ci/examples/code_climate.md index 6a5821762cc..f919ed3c797 100644 --- a/doc/ci/examples/code_climate.md +++ b/doc/ci/examples/code_climate.md @@ -16,7 +16,8 @@ codequality: - docker:dind script: - docker pull codeclimate/codeclimate - - docker run --env CODECLIMATE_CODE="$PWD" --volume "$PWD":/code --volume /var/run/docker.sock:/var/run/docker.sock --volume /tmp/cc:/tmp/cc codeclimate/codeclimate analyze -f json > codeclimate.json || true + - docker run --env CODECLIMATE_CODE="$PWD" --volume "$PWD":/code --volume /var/run/docker.sock:/var/run/docker.sock --volume /tmp/cc:/tmp/cc codeclimate/codeclimate:0.69.0 init + - docker run --env CODECLIMATE_CODE="$PWD" --volume "$PWD":/code --volume /var/run/docker.sock:/var/run/docker.sock --volume /tmp/cc:/tmp/cc codeclimate/codeclimate:0.69.0 analyze -f json > codeclimate.json || true artifacts: paths: [codeclimate.json] ``` diff --git a/doc/ci/ssh_keys/README.md b/doc/ci/ssh_keys/README.md index df0e1521150..b8df0bfba20 100644 --- a/doc/ci/ssh_keys/README.md +++ b/doc/ci/ssh_keys/README.md @@ -181,7 +181,7 @@ before_script: ## Assuming you created the SSH_KNOWN_HOSTS variable, uncomment the ## following two lines. ## - - echo "$SSH_KNOWN_HOSTS" > ~/.ssh/known_hosts' + - echo "$SSH_KNOWN_HOSTS" > ~/.ssh/known_hosts - chmod 644 ~/.ssh/known_hosts ## diff --git a/doc/development/architecture.md b/doc/development/architecture.md index 54029e00507..d1ba7d3dfc3 100644 --- a/doc/development/architecture.md +++ b/doc/development/architecture.md @@ -133,8 +133,6 @@ Usage: /etc/init.d/postgresql {start|stop|restart|reload|force-reload|status} [v ### Log locations of the services -Note: `/home/git/` is shorthand for `/home/git`. - gitlabhq (includes Unicorn and Sidekiq logs) - `/home/git/gitlab/log/` contains `application.log`, `production.log`, `sidekiq.log`, `unicorn.stdout.log`, `githost.log` and `unicorn.stderr.log` normally. diff --git a/doc/development/changelog.md b/doc/development/changelog.md index 48cffc0dd18..18f4177a5e5 100644 --- a/doc/development/changelog.md +++ b/doc/development/changelog.md @@ -127,7 +127,7 @@ type: If you're working on the GitLab EE repository, the entry will be added to `changelogs/unreleased-ee/` instead. -#### Arguments +### Arguments | Argument | Shorthand | Purpose | | ----------------- | --------- | ---------------------------------------------------------------------------------------------------------- | diff --git a/doc/development/fe_guide/style_guide_js.md b/doc/development/fe_guide/style_guide_js.md index 1cd66f27492..02773162801 100644 --- a/doc/development/fe_guide/style_guide_js.md +++ b/doc/development/fe_guide/style_guide_js.md @@ -101,16 +101,16 @@ followed by any global declarations, then a blank newline prior to any imports o ``` Import statements are following usual naming guidelines, for example object literals use camel case: - + ```javascript // some_object file export default { key: 'value', }; - + // bad import ObjectLiteral from 'some_object'; - + // good import objectLiteral from 'some_object'; ``` @@ -255,6 +255,10 @@ A forEach will cause side effects, it will be mutating the array being iterated. ### Vue.js +#### `eslint-vue-plugin` +We default to [eslint-vue-plugin][eslint-plugin-vue], with the `plugin:vue/recommended`. +Please check this [rules][eslint-plugin-vue-rules] for more documentation. + #### Basic Rules 1. The service has it's own file 1. The store has it's own file @@ -360,6 +364,10 @@ A forEach will cause side effects, it will be mutating the array being iterated. <component bar="bar" /> + + // bad + <component + bar="bar" /> ``` #### Quotes @@ -509,25 +517,7 @@ On those a default key should not be provided. ``` 1. Properties in a Vue Component: - 1. `name` - 1. `props` - 1. `mixins` - 1. `directives` - 1. `data` - 1. `components` - 1. `computedProps` - 1. `methods` - 1. `beforeCreate` - 1. `created` - 1. `beforeMount` - 1. `mounted` - 1. `beforeUpdate` - 1. `updated` - 1. `activated` - 1. `deactivated` - 1. `beforeDestroy` - 1. `destroyed` - + Check [order of properties in components rule][vue-order]. #### Vue and Bootstrap @@ -582,3 +572,6 @@ The goal of this accord is to make sure we are all on the same page. [eslintrc]: https://gitlab.com/gitlab-org/gitlab-ce/blob/master/.eslintrc [eslint-this]: http://eslint.org/docs/rules/class-methods-use-this [eslint-new]: http://eslint.org/docs/rules/no-new +[eslint-plugin-vue]: https://github.com/vuejs/eslint-plugin-vue +[eslint-plugin-vue-rules]: https://github.com/vuejs/eslint-plugin-vue#bulb-rules +[vue-order]: https://github.com/vuejs/eslint-plugin-vue/blob/master/docs/rules/order-in-components.md diff --git a/doc/user/permissions.md b/doc/user/permissions.md index 4fa83388d0c..708d07fcec9 100644 --- a/doc/user/permissions.md +++ b/doc/user/permissions.md @@ -200,7 +200,7 @@ instance and project. In addition, all admins can use the admin interface under |---------------------------------------|-----------------|-------------|----------|--------| | See commits and jobs | ✓ | ✓ | ✓ | ✓ | | Retry or cancel job | | ✓ | ✓ | ✓ | -| Erase job artifacts and trace | | ✓ [^7] | ✓ | ✓ | +| Erase job artifacts and trace | | ✓ [^5] | ✓ | ✓ | | Remove project | | | ✓ | ✓ | | Create project | | | ✓ | ✓ | | Change project configuration | | | ✓ | ✓ | @@ -223,13 +223,13 @@ users: | Run CI job | | ✓ | ✓ | ✓ | | Clone source and LFS from current project | | ✓ | ✓ | ✓ | | Clone source and LFS from public projects | | ✓ | ✓ | ✓ | -| Clone source and LFS from internal projects | | ✓ [^5] | ✓ [^5] | ✓ | -| Clone source and LFS from private projects | | ✓ [^6] | ✓ [^6] | ✓ [^6] | +| Clone source and LFS from internal projects | | ✓ [^6] | ✓ [^6] | ✓ | +| Clone source and LFS from private projects | | ✓ [^7] | ✓ [^7] | ✓ [^7] | | Push source and LFS | | | | | | Pull container images from current project | | ✓ | ✓ | ✓ | | Pull container images from public projects | | ✓ | ✓ | ✓ | -| Pull container images from internal projects| | ✓ [^5] | ✓ [^5] | ✓ | -| Pull container images from private projects | | ✓ [^6] | ✓ [^6] | ✓ [^6] | +| Pull container images from internal projects| | ✓ [^6] | ✓ [^6] | ✓ | +| Pull container images from private projects | | ✓ [^7] | ✓ [^7] | ✓ [^7] | | Push container images to current project | | ✓ | ✓ | ✓ | | Push container images to other projects | | | | | @@ -259,12 +259,13 @@ with the permissions described on the documentation on [auditor users permission Auditor users are available in [GitLab Enterprise Edition Premium](https://about.gitlab.com/gitlab-ee/) only. -[^1]: On public and internal projects, all users are able to perform this action. +[^1]: On public and internal projects, all users are able to perform this action [^2]: Guest users can only view the confidential issues they created themselves [^3]: If **Public pipelines** is enabled in **Project Settings > CI/CD** [^4]: Not allowed for Guest, Reporter, Developer, Master, or Owner -[^5]: Only if user is not external one. -[^6]: Only if user is a member of the project. -[^7]: Only if the build was triggered by the user +[^5]: Only if the job was triggered by the user +[^6]: Only if user is not external one +[^7]: Only if user is a member of the project + [ce-18994]: https://gitlab.com/gitlab-org/gitlab-ce/issues/18994 [new-mod]: project/new_ci_build_permissions_model.md diff --git a/doc/user/project/clusters/index.md b/doc/user/project/clusters/index.md index d5619c7b563..5f14d232cb1 100644 --- a/doc/user/project/clusters/index.md +++ b/doc/user/project/clusters/index.md @@ -2,9 +2,6 @@ > [Introduced](https://gitlab.com/gitlab-org/gitlab-ce/issues/35954) in 10.1. -CAUTION: **Warning:** -The Cluster integration is currently in **Beta**. - With a cluster associated to your project, you can use Review Apps, deploy your applications, run your pipelines, and much more, in an easy way. diff --git a/doc/user/project/integrations/irker.md b/doc/user/project/integrations/irker.md index c63ea1316fe..ecdd83ce8f0 100644 --- a/doc/user/project/integrations/irker.md +++ b/doc/user/project/integrations/irker.md @@ -47,4 +47,8 @@ Irker accepts channel names of the form `chan` and `#chan`, both for the case, `Aorimn` is treated as a nick and no more as a channel name. Irker can also join password-protected channels. Users need to append -`?key=thesecretpassword` to the chan name. +`?key=thesecretpassword` to the chan name. When using this feature remember to +**not** put the `#` sign in front of the channel name; failing to do so will +result on irker joining a channel literally named `#chan?key=password` henceforth +leaking the channel key through the `/whois` IRC command (depending on IRC server +configuration). This is due to a long standing irker bug. diff --git a/doc/user/project/integrations/webhooks.md b/doc/user/project/integrations/webhooks.md index eafdd28071d..82175c70e49 100644 --- a/doc/user/project/integrations/webhooks.md +++ b/doc/user/project/integrations/webhooks.md @@ -54,6 +54,12 @@ Below are described the supported events. Triggered when you push to the repository except when pushing tags. +> **Note:** When more than 20 commits are pushed at once, the `commits` web hook + attribute will only contain the first 20 for performance reasons. Loading + detailed commit data is expensive. Note that despite only 20 commits being + present in the `commits` attribute, the `total_commits_count` attribute will + contain the actual total. + **Request header**: ``` |
