diff options
author | Felipe Artur <felipefac@gmail.com> | 2016-06-29 17:31:50 +0300 |
---|---|---|
committer | Felipe Artur <felipefac@gmail.com> | 2016-06-29 17:32:38 +0300 |
commit | d2971315abd0e6e94860505e50ec71082c3679da (patch) | |
tree | 0f3dcf23406ec9fb3d6362b7f746e978ace692b3 /doc | |
parent | 36d48120f66d52dcc8697893ca1ab85322ba7c8b (diff) | |
parent | bef4294c58e0bbadc01f633fc8bea514801de7ae (diff) |
Merge branch 'master' into issue_3359_3
Diffstat (limited to 'doc')
53 files changed, 792 insertions, 146 deletions
diff --git a/doc/README.md b/doc/README.md index f1283cea0ad..be0d17084c7 100644 --- a/doc/README.md +++ b/doc/README.md @@ -44,6 +44,7 @@ - [Housekeeping](administration/housekeeping.md) Keep your Git repository tidy and fast. - [GitLab Performance Monitoring](monitoring/performance/introduction.md) Configure GitLab and InfluxDB for measuring performance metrics. - [Monitoring uptime](monitoring/health_check.md) Check the server status using the health check endpoint. +- [Debugging Tips](administration/troubleshooting/debug.md) Tips to debug problems when things go wrong - [Sidekiq Troubleshooting](administration/troubleshooting/sidekiq.md) Debug when Sidekiq appears hung and is not processing jobs. - [High Availability](administration/high_availability/README.md) Configure multiple servers for scaling or high availability. - [Container Registry](administration/container_registry.md) Configure Docker Registry with GitLab. diff --git a/doc/administration/troubleshooting/debug.md b/doc/administration/troubleshooting/debug.md new file mode 100644 index 00000000000..e5701b86cf3 --- /dev/null +++ b/doc/administration/troubleshooting/debug.md @@ -0,0 +1,120 @@ +# Debugging Tips + +Sometimes things don't work the way they should. Here are some tips on debugging issues out +in production. + +## The GNU Project Debugger (gdb) + +`gdb` is a must-have tool for debugging issues. To install on Ubuntu/Debian: + +``` +sudo apt-get install gdb +``` + +On CentOS: + +``` +sudo yum install gdb +``` + +## Common Problems + +Many of the tips to diagnose issues below apply to many different situations. We'll use one +concrete example to illustrate what you can do to learn what is going wrong. + +### 502 Gateway Timeout after unicorn spins at 100% CPU + +This error occurs when the Web server times out (default: 60 s) after not +hearing back from the unicorn worker. If the CPU spins to 100% while this in +progress, there may be something taking longer than it should. + +To fix this issue, we first need to figure out what is happening. The +following tips are only recommended if you do NOT mind users being affected by +downtime. Otherwise skip to the next section. + +1. Load the problematic URL +1. Run `sudo gdb -p <PID>` to attach to the unicorn process. +1. In the gdb window, type: + + ``` + call (void) rb_backtrace() + ``` + +1. This forces the process to generate a Ruby backtrace. Check + `/var/log/gitlab/unicorn/unicorn_stderr.log` for the backtace. For example, you may see: + + ```ruby + from /opt/gitlab/embedded/service/gitlab-rails/lib/gitlab/metrics/sampler.rb:33:in `block in start' + from /opt/gitlab/embedded/service/gitlab-rails/lib/gitlab/metrics/sampler.rb:33:in `loop' + from /opt/gitlab/embedded/service/gitlab-rails/lib/gitlab/metrics/sampler.rb:36:in `block (2 levels) in start' + from /opt/gitlab/embedded/service/gitlab-rails/lib/gitlab/metrics/sampler.rb:44:in `sample' + from /opt/gitlab/embedded/service/gitlab-rails/lib/gitlab/metrics/sampler.rb:68:in `sample_objects' + from /opt/gitlab/embedded/service/gitlab-rails/lib/gitlab/metrics/sampler.rb:68:in `each_with_object' + from /opt/gitlab/embedded/service/gitlab-rails/lib/gitlab/metrics/sampler.rb:68:in `each' + from /opt/gitlab/embedded/service/gitlab-rails/lib/gitlab/metrics/sampler.rb:69:in `block in sample_objects' + from /opt/gitlab/embedded/service/gitlab-rails/lib/gitlab/metrics/sampler.rb:69:in `name' + ``` + +1. To see the current threads, run: + + ``` + apply all thread bt + ``` + +1. Once you're done debugging with `gdb`, be sure to detach from the process and exit: + + ``` + detach + exit + ``` + +Note that if the unicorn process terminates before you are able to run these +commands, gdb will report an error. To buy more time, you can always raise the +Unicorn timeout. For omnibus users, you can edit `/etc/gitlab/gitlab.rb` and +increase it from 60 seconds to 300: + +```ruby +unicorn['worker_timeout'] = 300 +``` + +For source installations, edit `config/unicorn.rb`. + +[Reconfigure] GitLab for the changes to take effect. + +[Reconfigure]: ../restart_gitlab.md#omnibus-gitlab-reconfigure + +#### Troubleshooting without affecting other users + +The previous section attached to a running unicorn process, and this may have +undesirable effects for users trying to access GitLab during this time. If you +are concerned about affecting others during a production system, you can run a +separate Rails process to debug the issue: + +1. Log in to your GitLab account. +1. Copy the URL that is causing problems (e.g. https://gitlab.com/ABC). +1. Obtain the private token for your user (Profile Settings -> Account). +1. Bring up the GitLab Rails console. For omnibus users, run: + + ```` + sudo gitlab-rails console + ``` + +1. At the Rails console, run: + + ```ruby + [1] pry(main)> app.get '<URL FROM STEP 1>/private_token?<TOKEN FROM STEP 2>' + ``` + + For example: + + ```ruby + [1] pry(main)> app.get 'https://gitlab.com/gitlab-org/gitlab-ce/issues/1?private_token=123456' + ``` + +1. In a new window, run `top`. It should show this ruby process using 100% CPU. Write down the PID. +1. Follow step 2 from the previous section on using gdb. + +# More information + +* [Debugging Stuck Ruby Processes](https://blog.newrelic.com/2013/04/29/debugging-stuck-ruby-processes-what-to-do-before-you-kill-9/) +* [Cheatsheet of using gdb and ruby processes](gdb-stuck-ruby.txt) diff --git a/doc/administration/troubleshooting/gdb-stuck-ruby.txt b/doc/administration/troubleshooting/gdb-stuck-ruby.txt new file mode 100644 index 00000000000..13d5dfcffa4 --- /dev/null +++ b/doc/administration/troubleshooting/gdb-stuck-ruby.txt @@ -0,0 +1,142 @@ +# Here's the script I'll use to demonstrate - it just loops forever: + +$ cat test.rb +#!/usr/bin/env ruby + +loop do + sleep 1 +end + +# Now, I'll start the script in the background, and redirect stdout and stderr +# to /dev/null: + +$ ruby ./test.rb >/dev/null 2>/dev/null & +[1] 1343 + +# Next, I'll grab the PID of the script (1343): + +$ ps aux | grep test.rb +vagrant 1343 0.0 0.4 3884 1652 pts/0 S 14:42 0:00 ruby ./test.rb +vagrant 1345 0.0 0.2 4624 852 pts/0 S+ 14:42 0:00 grep --color=auto test.rb + +# Now I start gdb. Note that I'm using sudo here. This may or may not be +# necessary in your setup. I'd try without sudo first, and fall back to adding +# it if the next step fails: + +$ sudo gdb +GNU gdb (Ubuntu/Linaro 7.4-2012.04-0ubuntu2.1) 7.4-2012.04 +Copyright (C) 2012 Free Software Foundation, Inc. +License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html> +This is free software: you are free to change and redistribute it. +There is NO WARRANTY, to the extent permitted by law. Type "show copying" +and "show warranty" for details. +This GDB was configured as "i686-linux-gnu". +For bug reporting instructions, please see: +<http://bugs.launchpad.net/gdb-linaro/>. + +# OK, now I'm in gdb, and I want to instruct it to attach to our Ruby process. +# I can do that using the 'attach' command, which takes a PID (the one we +# gathered above): + +(gdb) attach 1343 +Attaching to process 1343 +Reading symbols from /opt/vagrant_ruby/bin/ruby...done. +Reading symbols from /lib/i386-linux-gnu/librt.so.1...(no debugging symbols found)...done. +Loaded symbols for /lib/i386-linux-gnu/librt.so.1 +Reading symbols from /lib/i386-linux-gnu/libdl.so.2...(no debugging symbols found)...done. +Loaded symbols for /lib/i386-linux-gnu/libdl.so.2 +Reading symbols from /lib/i386-linux-gnu/libcrypt.so.1...(no debugging symbols found)...done. +Loaded symbols for /lib/i386-linux-gnu/libcrypt.so.1 +Reading symbols from /lib/i386-linux-gnu/libm.so.6...(no debugging symbols found)...done. +Loaded symbols for /lib/i386-linux-gnu/libm.so.6 +Reading symbols from /lib/i386-linux-gnu/libc.so.6...(no debugging symbols found)...done. +Loaded symbols for /lib/i386-linux-gnu/libc.so.6 +Reading symbols from /lib/i386-linux-gnu/libpthread.so.0...(no debugging symbols found)...done. +[Thread debugging using libthread_db enabled] +Using host libthread_db library "/lib/i386-linux-gnu/libthread_db.so.1". +Loaded symbols for /lib/i386-linux-gnu/libpthread.so.0 +Reading symbols from /lib/ld-linux.so.2...(no debugging symbols found)...done. +Loaded symbols for /lib/ld-linux.so.2 +0xb770c424 in __kernel_vsyscall () + +# Great, now gdb is attached to the target process. If the step above fails, try +# going back and running gdb under sudo. The next thing I want to do is gather +# C-level backtraces from all threads in the process. The following command +# stands for 'thread apply all backtrace': + +(gdb) t a a bt + +Thread 1 (Thread 0xb74d76c0 (LWP 1343)): +#0 0xb770c424 in __kernel_vsyscall () +#1 0xb75d7abd in select () from /lib/i386-linux-gnu/libc.so.6 +#2 0x08069c56 in rb_thread_wait_for (time=...) at eval.c:11376 +#3 0x080a20fd in rb_f_sleep (argc=1, argv=0xbf85f490) at process.c:1633 +#4 0x0805e0e2 in call_cfunc (argv=0xbf85f490, argc=1, len=-1, recv=3075299660, func=0x80a20b0 <rb_f_sleep>) + at eval.c:5778 +#5 rb_call0 (klass=3075304600, recv=3075299660, id=9393, oid=9393, argc=1, argv=0xbf85f490, body=0xb74c85a8, flags=2) + at eval.c:5928 +#6 0x0805e35d in rb_call (klass=3075304600, recv=3075299660, mid=9393, argc=1, argv=0xbf85f490, scope=1, + self=<optimized out>) at eval.c:6176 +#7 0x080651ec in rb_eval (self=3075299660, n=0xb74c4e1c) at eval.c:3521 +#8 0x0805c31c in rb_yield_0 (val=6, self=3075299660, klass=<optimized out>, flags=0, avalue=0) at eval.c:5095 +#9 0x0806a1e5 in loop_i () at eval.c:5227 +#10 0x08058dbd in rb_rescue2 (b_proc=0x806a1c0 <loop_i>, data1=0, r_proc=0, data2=0) at eval.c:5491 +#11 0x08058f28 in rb_f_loop () at eval.c:5252 +#12 0x0805e0c1 in call_cfunc (argv=0x0, argc=0, len=0, recv=3075299660, func=0x8058ef0 <rb_f_loop>) at eval.c:5781 +#13 rb_call0 (klass=3075304600, recv=3075299660, id=4121, oid=4121, argc=0, argv=0x0, body=0xb74d4dbc, flags=2) + at eval.c:5928 +#14 0x0805e35d in rb_call (klass=3075304600, recv=3075299660, mid=4121, argc=0, argv=0x0, scope=1, self=<optimized out>) + at eval.c:6176 +#15 0x080651ec in rb_eval (self=3075299660, n=0xb74c4dcc) at eval.c:3521 +#16 0x080662c6 in rb_eval (self=3075299660, n=0xb74c4de0) at eval.c:3236 +#17 0x08068ee4 in ruby_exec_internal () at eval.c:1654 +#18 0x08068f24 in ruby_exec () at eval.c:1674 +#19 0x0806b2cd in ruby_run () at eval.c:1684 +#20 0x08053771 in main (argc=2, argv=0xbf860204, envp=0xbf860210) at main.c:48 + +# C backtraces are sometimes sufficient, but often Ruby backtraces are necessary +# for debugging as well. Ruby has a built-in function called rb_backtrace() that +# we can use to dump out a Ruby backtrace, but it prints to stdout or stderr +# (depending on your Ruby version), which might have been redirected to a file +# or to /dev/null (as in our example) when the process started up. +# +# To get aroundt this, we'll do a little trick and redirect the target process's +# stdout and stderr to the current TTY, so that any output from the process +# will appear directly on our screen. + +# First, let's close the existing file descriptors for stdout and stderr +# (FD 1 and 2, respectively): +(gdb) call (void) close(1) +(gdb) call (void) close(2) + +# Next, we need to figure out the device name for the current TTY: +(gdb) shell tty +/dev/pts/0 + +# OK, now we can pass the device name obtained above to open() and attach +# file descriptors 1 and 2 back to the current TTY with these calls: + +(gdb) call (int) open("/dev/pts/0", 2, 0) +$1 = 1 +(gdb) call (int) open("/dev/pts/0", 2, 0) +$2 = 2 + +# Finally, we call rb_backtrace() in order to dump the Ruby backtrace: + +(gdb) call (void) rb_backtrace() + from ./test.rb:4:in `sleep' + from ./test.rb:4 + from ./test.rb:3:in `loop' + from ./test.rb:3 + +# And here's how we get out of gdb. Once you've quit, you'll probably want to +# clean up the stuck process by killing it. + +(gdb) quit +A debugging session is active. + + Inferior 1 [process 1343] will be detached. + +Quit anyway? (y or n) y +Detaching from program: /opt/vagrant_ruby/bin/ruby, process 1343 +$ diff --git a/doc/api/builds.md b/doc/api/builds.md index de998944352..2adea11247e 100644 --- a/doc/api/builds.md +++ b/doc/api/builds.md @@ -107,6 +107,11 @@ Example of response Get a list of builds for specific commit in a project. +This endpoint will return all builds, from all pipelines for a given commit. +If the commit SHA is not found, it will respond with 404, otherwise it will +return an array of builds (an empty array if there are no builds for this +particular commit). + ``` GET /projects/:id/repository/commits/:sha/builds ``` diff --git a/doc/api/issues.md b/doc/api/issues.md index 0bc82ef9edb..708fc691f67 100644 --- a/doc/api/issues.md +++ b/doc/api/issues.md @@ -28,7 +28,7 @@ GET /issues?labels=foo,bar&state=opened | Attribute | Type | Required | Description | | --------- | ---- | -------- | ----------- | | `state` | string | no | Return all issues or just those that are `opened` or `closed`| -| `labels` | string | no | Comma-separated list of label names | +| `labels` | string | no | Comma-separated list of label names, issues with any of the labels will be returned | | `order_by`| string | no | Return requests ordered by `created_at` or `updated_at` fields. Default is `created_at` | | `sort` | string | no | Return requests sorted in `asc` or `desc` order. Default is `desc` | @@ -83,6 +83,82 @@ Example response: ] ``` +## List group issues + +Get a list of a group's issues. + +``` +GET /groups/:id/issues +GET /groups/:id/issues?state=opened +GET /groups/:id/issues?state=closed +GET /groups/:id/issues?labels=foo +GET /groups/:id/issues?labels=foo,bar +GET /groups/:id/issues?labels=foo,bar&state=opened +GET /groups/:id/issues?milestone=1.0.0 +GET /groups/:id/issues?milestone=1.0.0&state=opened +``` + +| Attribute | Type | Required | Description | +| --------- | ---- | -------- | ----------- | +| `id` | integer | yes | The ID of a group | +| `state` | string | no | Return all issues or just those that are `opened` or `closed`| +| `labels` | string | no | Comma-separated list of label names, issues must have all labels to be returned | +| `milestone` | string| no | The milestone title | +| `order_by`| string | no | Return requests ordered by `created_at` or `updated_at` fields. Default is `created_at` | +| `sort` | string | no | Return requests sorted in `asc` or `desc` order. Default is `desc` | + + +```bash +curl -H "PRIVATE-TOKEN: 9koXpg98eAheJpvBs5tK" https://gitlab.example.com/api/v3/groups/4/issues +``` + +Example response: + +```json +[ + { + "project_id" : 4, + "milestone" : { + "due_date" : null, + "project_id" : 4, + "state" : "closed", + "description" : "Rerum est voluptatem provident consequuntur molestias similique ipsum dolor.", + "iid" : 3, + "id" : 11, + "title" : "v3.0", + "created_at" : "2016-01-04T15:31:39.788Z", + "updated_at" : "2016-01-04T15:31:39.788Z" + }, + "author" : { + "state" : "active", + "web_url" : "https://gitlab.example.com/u/root", + "avatar_url" : null, + "username" : "root", + "id" : 1, + "name" : "Administrator" + }, + "description" : "Omnis vero earum sunt corporis dolor et placeat.", + "state" : "closed", + "iid" : 1, + "assignee" : { + "avatar_url" : null, + "web_url" : "https://gitlab.example.com/u/lennie", + "state" : "active", + "username" : "lennie", + "id" : 9, + "name" : "Dr. Luella Kovacek" + }, + "labels" : [], + "id" : 41, + "title" : "Ut commodi ullam eos dolores perferendis nihil sunt.", + "updated_at" : "2016-01-04T15:31:46.176Z", + "created_at" : "2016-01-04T15:31:46.176Z", + "subscribed" : false, + "user_notes_count": 1 + } +] +``` + ## List project issues Get a list of a project's issues. @@ -104,7 +180,7 @@ GET /projects/:id/issues?iid=42 | `id` | integer | yes | The ID of a project | | `iid` | integer | no | Return the issue having the given `iid` | | `state` | string | no | Return all issues or just those that are `opened` or `closed`| -| `labels` | string | no | Comma-separated list of label names | +| `labels` | string | no | Comma-separated list of label names, issues with any of the labels will be returned | | `milestone` | string| no | The milestone title | | `order_by`| string | no | Return requests ordered by `created_at` or `updated_at` fields. Default is `created_at` | | `sort` | string | no | Return requests sorted in `asc` or `desc` order. Default is `desc` | diff --git a/doc/api/oauth2.md b/doc/api/oauth2.md index d416a826f79..31902e145f6 100644 --- a/doc/api/oauth2.md +++ b/doc/api/oauth2.md @@ -65,6 +65,13 @@ curl -H "Authorization: Bearer OAUTH-TOKEN" https://localhost:3000/api/v3/user ## Resource Owner Password Credentials +## Deprecation Notice + +1. Starting in GitLab 9.0, the Resource Owner Password Credentials will be *disabled* for users with two-factor authentication turned on. +2. These users can access the API using [personal access tokens] instead. + +--- + In this flow, a token is requested in exchange for the resource owner credentials (username and password). The credentials should only be used when there is a high degree of trust between the resource owner and the client (e.g. the client is part of the device operating system or a highly privileged application), and when other authorization grant types are not @@ -100,3 +107,5 @@ client = OAuth2::Client.new('the_client_id', 'the_client_secret', :site => "http access_token = client.password.get_token('user@example.com', 'sekret') puts access_token.token ``` + +[personal access tokens]: ./README.md#personal-access-tokens diff --git a/doc/api/services.md b/doc/api/services.md index ccfc0fccb7f..32d6e2dea78 100644 --- a/doc/api/services.md +++ b/doc/api/services.md @@ -374,40 +374,6 @@ Get Gemnasium service settings for a project. GET /projects/:id/services/gemnasium ``` -## GitLab CI - -Continuous integration server from GitLab - -### Create/Edit GitLab CI service - -Set GitLab CI service for a project. - -``` -PUT /projects/:id/services/gitlab-ci -``` - -Parameters: - -- `token` (**required**) - GitLab CI project specific token -- `project_url` (**required**) - http://ci.gitlabhq.com/projects/3 -- `enable_ssl_verification` (optional) - Enable SSL verification - -### Delete GitLab CI service - -Delete GitLab CI service for a project. - -``` -DELETE /projects/:id/services/gitlab-ci -``` - -### Get GitLab CI service settings - -Get GitLab CI service settings for a project. - -``` -GET /projects/:id/services/gitlab-ci -``` - ## HipChat Private group chat and IM diff --git a/doc/api/session.md b/doc/api/session.md index 71e93d0bb0a..066a055702d 100644 --- a/doc/api/session.md +++ b/doc/api/session.md @@ -1,5 +1,12 @@ # Session +## Deprecation Notice + +1. Starting in GitLab 9.0, this feature will be *disabled* for users with two-factor authentication turned on. +2. These users can access the API using [personal access tokens] instead. + +--- + You can login with both GitLab and LDAP credentials in order to obtain the private token. @@ -45,3 +52,5 @@ Example response: "private_token": "9koXpg98eAheJpvBs5tK" } ``` + +[personal access tokens]: ./README.md#personal-access-tokens diff --git a/doc/api/settings.md b/doc/api/settings.md index 43a0fe35e42..b5152311f28 100644 --- a/doc/api/settings.md +++ b/doc/api/settings.md @@ -56,7 +56,7 @@ PUT /application/settings | `gravatar_enabled` | boolean | no | Enable Gravatar | | `sign_in_text` | string | no | Text on login page | | `home_page_url` | string | no | Redirect to this URL when not logged in | -| `default_branch_protection` | integer | no | Determine if developers can push to master. Can take `0` _(not protected, both developers and masters can push new commits, force push or delete the branch)_, `1` _(partially protected, developers can push new commits, but cannot force push or delete the branch, masters can do anything)_ or `2` _(fully protected, developers cannot push new commits, force push or delete the branch, masters can do anything)_ as a parameter. Default is `1`. | +| `default_branch_protection` | integer | no | Determine if developers can push to master. Can take `0` _(not protected, both developers and masters can push new commits, force push or delete the branch)_, `1` _(partially protected, developers can push new commits, but cannot force push or delete the branch, masters can do anything)_ or `2` _(fully protected, developers cannot push new commits, force push or delete the branch, masters can do anything)_ as a parameter. Default is `2`. | | `restricted_visibility_levels` | array of integers | no | Selected levels cannot be used by non-admin users for projects or snippets. Can take `0` _(Private)_, `1` _(Internal)_ and `2` _(Public)_ as a parameter. Default is null which means there is no restriction. | | `max_attachment_size` | integer | no | Limit attachment size in MB | | `session_expire_delay` | integer | no | Session duration in minutes. GitLab restart is required to apply changes | diff --git a/doc/ci/yaml/README.md b/doc/ci/yaml/README.md index 1892acda29b..d2d1b04f893 100644 --- a/doc/ci/yaml/README.md +++ b/doc/ci/yaml/README.md @@ -1034,8 +1034,8 @@ You can find the link under `/ci/lint` of your gitlab instance. ## Skipping builds -If your commit message contains `[ci skip]`, the commit will be created but the -builds will be skipped. +If your commit message contains `[ci skip]` or `[skip ci]`, using any +capitalization, the commit will be created but the builds will be skipped. ## Examples diff --git a/doc/customization/issue_closing.md b/doc/customization/issue_closing.md index 194b8e00299..4620bb2dcde 100644 --- a/doc/customization/issue_closing.md +++ b/doc/customization/issue_closing.md @@ -8,7 +8,7 @@ the matched text will be closed. This happens when the commit is pushed to a pro When not specified, the default `issue_closing_pattern` as shown below will be used: ```bash -((?:[Cc]los(?:e[sd]?|ing)|[Ff]ix(?:e[sd]|ing)?) +(?:(?:issues? +)?%{issue_ref}(?:(?:, *| +and +)?))+) +((?:[Cc]los(?:e[sd]?|ing)|[Ff]ix(?:e[sd]|ing)?|[Rr]esolv(?:e[sd]?|ing))(:?) +(?:(?:issues? +)?%{issue_ref}(?:(?:, *| +and +)?)|([A-Z][A-Z0-9_]+-\d+))+) ``` Here, `%{issue_ref}` is a complex regular expression defined inside GitLab, that matches a reference to a local issue (`#123`), cross-project issue (`group/project#123`) or a link to an issue (`https://gitlab.example.com/group/project/issues/123`). diff --git a/doc/development/architecture.md b/doc/development/architecture.md index 12e33406cb6..33fd50f4c11 100644 --- a/doc/development/architecture.md +++ b/doc/development/architecture.md @@ -52,7 +52,9 @@ To serve repositories over SSH there's an add-on application called gitlab-shell ### Components -![GitLab Diagram Overview](gitlab_diagram_overview.png) +![GitLab Diagram Overview](gitlab_architecture_diagram.png) + +_[edit diagram (for GitLab team members only)](https://docs.google.com/drawings/d/1fBzAyklyveF-i-2q-OHUIqDkYfjjxC4mq5shwKSZHLs/edit)_ A typical install of GitLab will be on GNU/Linux. It uses Nginx or Apache as a web front end to proxypass the Unicorn web server. By default, communication between Unicorn and the front end is via a Unix domain socket but forwarding requests via TCP is also supported. The web front end accesses `/home/git/gitlab/public` bypassing the Unicorn server to serve static pages, uploads (e.g. avatar images or attachments), and precompiled assets. GitLab serves web pages and a [GitLab API](https://gitlab.com/gitlab-org/gitlab-ce/tree/master/doc/api) using the Unicorn web server. It uses Sidekiq as a job queue which, in turn, uses redis as a non-persistent database backend for job information, meta data, and incoming jobs. diff --git a/doc/development/gitlab_architecture_diagram.png b/doc/development/gitlab_architecture_diagram.png Binary files differnew file mode 100644 index 00000000000..9ab7ffd3c7b --- /dev/null +++ b/doc/development/gitlab_architecture_diagram.png diff --git a/doc/development/gitlab_diagram_overview.png b/doc/development/gitlab_diagram_overview.png Binary files differdeleted file mode 100644 index d9b9eed3d8f..00000000000 --- a/doc/development/gitlab_diagram_overview.png +++ /dev/null diff --git a/doc/development/gotchas.md b/doc/development/gotchas.md index 21078c8d6f9..9d7fe7440d2 100644 --- a/doc/development/gotchas.md +++ b/doc/development/gotchas.md @@ -46,7 +46,7 @@ Rubocop](https://gitlab.com/gitlab-org/gitlab-ce/blob/8-4-stable/.rubocop.yml#L9 Using the inline `:coffee` or `:coffeescript` Haml filters comes with a performance overhead. -_**Note:** We've [removed these two filters](https://gitlab.com/gitlab-org/gitlab-ce/blob/8-5-stable/config/initializers/haml.rb) +_**Note:** We've [removed these two filters](https://gitlab.com/gitlab-org/gitlab-ce/blob/master/config/initializers/hamlit.rb) in an initializer._ ### Further reading diff --git a/doc/development/migration_style_guide.md b/doc/development/migration_style_guide.md index 8a7547e5322..e2ca46504e7 100644 --- a/doc/development/migration_style_guide.md +++ b/doc/development/migration_style_guide.md @@ -37,7 +37,6 @@ First, you need to provide information on whether the migration can be applied: For example: ``` -# rubocop:disable all # Migration type: online without errors (works on previous version and new one) class MyMigration < ActiveRecord::Migration ... diff --git a/doc/development/rake_tasks.md b/doc/development/rake_tasks.md index 6d04b9590e6..41685c7ee41 100644 --- a/doc/development/rake_tasks.md +++ b/doc/development/rake_tasks.md @@ -33,3 +33,23 @@ bundle exec rake gitlab:generate_docs ``` bundle exec rake services:doc ``` + +## Updating Emoji Digests + +To update the Emoji digests file (used for Emoji autocomplete) you must run the +following: + +``` +bundle exec rake gemojione:digests +``` + +This will update the file `fixtures/emojis/digests.json` based on the currently +available Emoji. + +## Emoji Sprites + +Generating a sprite file containing all the Emoji can be done by running: + +``` +bundle exec rake gemojione:sprite +``` diff --git a/doc/downgrade_ee_to_ce/README.md b/doc/downgrade_ee_to_ce/README.md index 3625c4191b8..a6d22e5a04a 100644 --- a/doc/downgrade_ee_to_ce/README.md +++ b/doc/downgrade_ee_to_ce/README.md @@ -44,13 +44,13 @@ to avoid getting this error, you need to remove all instances of the **Omnibus Installation** ``` -$ sudo gitlab-rails runner "Service.where(type: 'JenkinsService').delete_all" +$ sudo gitlab-rails runner "Service.where(type: ['JenkinsService', 'JenkinsDeprecatedService']).delete_all" ``` **Source Installation** ``` -$ bundle exec rails runner "Service.where(type: 'JenkinsService').delete_all" production +$ bundle exec rails runner "Service.where(type: ['JenkinsService', 'JenkinsDeprecatedService']).delete_all" production ``` ## Downgrade to CE diff --git a/doc/install/installation.md b/doc/install/installation.md index d9290b1fa76..dc8d9c65535 100644 --- a/doc/install/installation.md +++ b/doc/install/installation.md @@ -391,6 +391,10 @@ GitLab Shell is an SSH access and repository management software developed speci ### Install gitlab-workhorse +GitLab-Workhorse uses [GNU Make](https://www.gnu.org/software/make/). +If you are not using Linux you may have to run `gmake` instead of +`make` below. + cd /home/git sudo -u git -H git clone https://gitlab.com/gitlab-org/gitlab-workhorse.git cd gitlab-workhorse diff --git a/doc/install/requirements.md b/doc/install/requirements.md index 09c6211b3ab..a65ac8a5f79 100644 --- a/doc/install/requirements.md +++ b/doc/install/requirements.md @@ -52,7 +52,7 @@ If you have enough RAM memory and a recent CPU the speed of GitLab is mainly lim ### CPU -- 1 core works supports up to 100 users but the application can be a bit slower due to having all workers and background jobs running on the same core +- 1 core supports up to 100 users but the application can be a bit slower due to having all workers and background jobs running on the same core - **2 cores** is the **recommended** number of cores and supports up to 500 users - 4 cores supports up to 2,000 users - 8 cores supports up to 5,000 users diff --git a/doc/integration/external-issue-tracker.md b/doc/integration/external-issue-tracker.md index a2d7e922aad..8d2c6351fb8 100644 --- a/doc/integration/external-issue-tracker.md +++ b/doc/integration/external-issue-tracker.md @@ -1,7 +1,7 @@ # External issue tracker GitLab has a great issue tracker but you can also use an external one such as -Jira or Redmine. Issue trackers are configurable per GitLab project and allow +Jira, Redmine, or Bugzilla. Issue trackers are configurable per GitLab project and allow you to do the following: - the **Issues** link on the GitLab project pages takes you to the appropriate @@ -20,6 +20,7 @@ Visit the links below for details: - [Redmine](../project_services/redmine.md) - [Jira](../project_services/jira.md) +- [Bugzilla](../project_services/bugzilla.md) ### Service Template diff --git a/doc/monitoring/performance/grafana_configuration.md b/doc/monitoring/performance/grafana_configuration.md index 168bd85c26a..7947b0fedc4 100644 --- a/doc/monitoring/performance/grafana_configuration.md +++ b/doc/monitoring/performance/grafana_configuration.md @@ -44,70 +44,32 @@ on a separate server) ## Apply retention policies and create continuous queries -If you intend to import the GitLab provided Grafana dashboards, you will need -to copy and run a set of queries against InfluxDB to create the needed data -sets. +If you intend to import the GitLab provided Grafana dashboards, you will need to +set up the right retention policies and continuous queries. The easiest way of +doing this is by using the [influxdb-management](https://gitlab.com/gitlab-org/influxdb-management) +repository. -On the InfluxDB server, run the following command, substituting your InfluxDB -user and password: +To use this repository you must first clone it: -```bash -influxdb --username admin -password super_secret +``` +git clone https://gitlab.com/gitlab-org/influxdb-management.git +cd influxdb-management ``` -This will drop you in to an InfluxDB interactive session. Copy the entire -contents below and paste it in to the interactive session: +Next you must install the required dependencies: ``` -CREATE RETENTION POLICY default ON gitlab DURATION 1h REPLICATION 1 DEFAULT -CREATE RETENTION POLICY downsampled ON gitlab DURATION 7d REPLICATION 1 -CREATE CONTINUOUS QUERY grape_git_timings_per_action ON gitlab BEGIN SELECT mean("duration") AS duration_mean, percentile("duration", 95) AS duration_95th, percentile("duration", 99) AS duration_99th INTO gitlab.downsampled.grape_git_timings_per_action FROM gitlab."default".rails_method_calls WHERE (action !~ /.+/ OR action =~ /^Grape#/) AND method =~ /^(Rugged|Gitlab::Git)/ GROUP BY time(1m), action END; -CREATE CONTINUOUS QUERY grape_markdown_render_timings_overall ON gitlab BEGIN SELECT mean(banzai_cached_render_real_time) AS cached_real_mean, percentile(banzai_cached_render_real_time, 95) AS cached_real_95th, percentile(banzai_cached_render_real_time, 99) AS cached_real_99th, mean(banzai_cached_render_cpu_time) AS cached_cpu_mean, percentile(banzai_cached_render_cpu_time, 95) AS cached_cpu_95th, percentile(banzai_cached_render_cpu_time, 99) AS cached_cpu_99th, sum(banzai_cached_render_call_count) AS cached_call_count, mean(banzai_cacheless_render_real_time) AS cacheless_real_mean, percentile(banzai_cacheless_render_real_time, 95) AS cacheless_real_95th, percentile(banzai_cacheless_render_real_time, 99) AS cacheless_real_99th, mean(banzai_cacheless_render_cpu_time) AS cacheless_cpu_mean, percentile(banzai_cacheless_render_cpu_time, 95) AS cacheless_cpu_95th, percentile(banzai_cacheless_render_cpu_time, 99) AS cacheless_cpu_99th, sum(banzai_cacheless_render_call_count) AS cacheless_call_count INTO gitlab.downsampled.grape_markdown_render_timings_overall FROM gitlab."default".rails_transactions WHERE (action !~ /.+/ OR action =~ /^Grape#/) AND (banzai_cached_render_call_count > 0 OR banzai_cacheless_render_call_count > 0) GROUP BY time(1m) END; -CREATE CONTINUOUS QUERY grape_markdown_render_timings_per_action ON gitlab BEGIN SELECT mean(banzai_cached_render_real_time) AS cached_real_mean, percentile(banzai_cached_render_real_time, 95) AS cached_real_95th, percentile(banzai_cached_render_real_time, 99) AS cached_real_99th, mean(banzai_cached_render_cpu_time) AS cached_cpu_mean, percentile(banzai_cached_render_cpu_time, 95) AS cached_cpu_95th, percentile(banzai_cached_render_cpu_time, 99) AS cached_cpu_99th, sum(banzai_cached_render_call_count) AS cached_call_count, mean(banzai_cacheless_render_real_time) AS cacheless_real_mean, percentile(banzai_cacheless_render_real_time, 95) AS cacheless_real_95th, percentile(banzai_cacheless_render_real_time, 99) AS cacheless_real_99th, mean(banzai_cacheless_render_cpu_time) AS cacheless_cpu_mean, percentile(banzai_cacheless_render_cpu_time, 95) AS cacheless_cpu_95th, percentile(banzai_cacheless_render_cpu_time, 99) AS cacheless_cpu_99th, sum(banzai_cacheless_render_call_count) AS cacheless_call_count INTO gitlab.downsampled.grape_markdown_render_timings_per_action FROM gitlab."default".rails_transactions WHERE (action !~ /.+/ OR action =~ /^Grape#/) AND (banzai_cached_render_call_count > 0 OR banzai_cacheless_render_call_count > 0) GROUP BY time(1m), action END; -CREATE CONTINUOUS QUERY grape_markdown_timings_overall ON gitlab BEGIN SELECT mean("duration") AS duration_mean, percentile("duration", 95) AS duration_95th, percentile("duration", 99) AS duration_99th INTO gitlab.downsampled.grape_markdown_timings_overall FROM gitlab."default".rails_method_calls WHERE (action !~ /.+/ OR action =~ /^Grape#/) AND method =~ /^Banzai/ GROUP BY time(1m) END; -CREATE CONTINUOUS QUERY grape_method_call_timings_per_action_and_method ON gitlab BEGIN SELECT mean("duration") AS duration_mean, percentile("duration", 95) AS duration_95th, percentile("duration", 99) AS duration_99th INTO gitlab.downsampled.grape_method_call_timings_per_action_and_method FROM gitlab."default".rails_method_calls WHERE action !~ /.+/ OR action =~ /^Grape#/ GROUP BY time(1m), method, action END; -CREATE CONTINUOUS QUERY grape_method_call_timings_per_method ON gitlab BEGIN SELECT mean("duration") AS duration_mean, percentile("duration", 95) AS duration_95th, percentile("duration", 99) AS duration_99th INTO gitlab.downsampled.grape_method_call_timings_per_method FROM gitlab."default".rails_method_calls WHERE action !~ /.+/ OR action =~ /^Grape#/ GROUP BY time(1m), method END; -CREATE CONTINUOUS QUERY grape_transaction_counts_overall ON gitlab BEGIN SELECT count("duration") AS count INTO gitlab.downsampled.grape_transaction_counts_overall FROM gitlab."default".rails_transactions WHERE action !~ /.+/ OR action =~ /^Grape#/ GROUP BY time(1m) END; -CREATE CONTINUOUS QUERY grape_transaction_counts_per_action ON gitlab BEGIN SELECT count("duration") AS count INTO gitlab.downsampled.grape_transaction_counts_per_action FROM gitlab."default".rails_transactions WHERE action !~ /.+/ OR action =~ /^Grape#/ GROUP BY time(1m), action END; -CREATE CONTINUOUS QUERY grape_transaction_timings_overall ON gitlab BEGIN SELECT mean("duration") AS duration_mean, percentile("duration", 95) AS duration_95th, percentile("duration", 99) AS duration_99th, mean(sql_duration) AS sql_duration_mean, percentile(sql_duration, 95) AS sql_duration_95th, percentile(sql_duration, 99) AS sql_duration_99th, mean(view_duration) AS view_duration_mean, percentile(view_duration, 95) AS view_duration_95th, percentile(view_duration, 99) AS view_duration_99th, mean(cache_read_duration) AS cache_read_duration_mean, percentile(cache_read_duration, 99) AS cache_read_duration_99th, percentile(cache_read_duration, 95) AS cache_read_duration_95th, mean(cache_write_duration) AS cache_write_duration_mean, percentile(cache_write_duration, 99) AS cache_write_duration_99th, percentile(cache_write_duration, 95) AS cache_write_duration_95th, mean(cache_delete_duration) AS cache_delete_duration_mean, percentile(cache_delete_duration, 99) AS cache_delete_duration_99th, percentile(cache_delete_duration, 95) AS cache_delete_duration_95th, mean(cache_exists_duration) AS cache_exists_duration_mean, percentile(cache_exists_duration, 99) AS cache_exists_duration_99th, percentile(cache_exists_duration, 95) AS cache_exists_duration_95th, mean(cache_duration) AS cache_duration_mean, percentile(cache_duration, 99) AS cache_duration_99th, percentile(cache_duration, 95) AS cache_duration_95th, mean(method_duration) AS method_duration_mean, percentile(method_duration, 99) AS method_duration_99th, percentile(method_duration, 95) AS method_duration_95th INTO gitlab.downsampled.grape_transaction_timings_overall FROM gitlab."default".rails_transactions WHERE action !~ /.+/ OR action =~ /^Grape#/ GROUP BY time(1m) END; -CREATE CONTINUOUS QUERY grape_transaction_timings_per_action ON gitlab BEGIN SELECT mean("duration") AS duration_mean, percentile("duration", 95) AS duration_95th, percentile("duration", 99) AS duration_99th, mean(sql_duration) AS sql_duration_mean, percentile(sql_duration, 95) AS sql_duration_95th, percentile(sql_duration, 99) AS sql_duration_99th, mean(view_duration) AS view_duration_mean, percentile(view_duration, 95) AS view_duration_95th, percentile(view_duration, 99) AS view_duration_99th, mean(cache_read_duration) AS cache_read_duration_mean, percentile(cache_read_duration, 99) AS cache_read_duration_99th, percentile(cache_read_duration, 95) AS cache_read_duration_95th, mean(cache_write_duration) AS cache_write_duration_mean, percentile(cache_write_duration, 99) AS cache_write_duration_99th, percentile(cache_write_duration, 95) AS cache_write_duration_95th, mean(cache_delete_duration) AS cache_delete_duration_mean, percentile(cache_delete_duration, 99) AS cache_delete_duration_99th, percentile(cache_delete_duration, 95) AS cache_delete_duration_95th, mean(cache_exists_duration) AS cache_exists_duration_mean, percentile(cache_exists_duration, 99) AS cache_exists_duration_99th, percentile(cache_exists_duration, 95) AS cache_exists_duration_95th, mean(cache_duration) AS cache_duration_mean, percentile(cache_duration, 99) AS cache_duration_99th, percentile(cache_duration, 95) AS cache_duration_95th, mean(method_duration) AS method_duration_mean, percentile(method_duration, 99) AS method_duration_99th, percentile(method_duration, 95) AS method_duration_95th INTO gitlab.downsampled.grape_transaction_timings_per_action FROM gitlab."default".rails_transactions WHERE action !~ /.+/ OR action =~ /^Grape#/ GROUP BY time(1m), action END; -CREATE CONTINUOUS QUERY rails_file_descriptor_counts ON gitlab BEGIN SELECT sum(value) AS count INTO gitlab.downsampled.rails_file_descriptor_counts FROM gitlab."default".rails_file_descriptors GROUP BY time(1m) END; -CREATE CONTINUOUS QUERY rails_gc_counts ON gitlab BEGIN SELECT sum(count) AS total, sum(minor_gc_count) AS minor, sum(major_gc_count) AS major INTO gitlab.downsampled.rails_gc_counts FROM gitlab."default".rails_gc_statistics GROUP BY time(1m) END; -CREATE CONTINUOUS QUERY rails_gc_timings ON gitlab BEGIN SELECT mean(total_time) AS duration_mean, percentile(total_time, 95) AS duration_95th, percentile(total_time, 99) AS duration_99th INTO gitlab.downsampled.rails_gc_timings FROM gitlab."default".rails_gc_statistics GROUP BY time(1m) END; -CREATE CONTINUOUS QUERY rails_git_timings_per_action ON gitlab BEGIN SELECT mean("duration") AS duration_mean, percentile("duration", 95) AS duration_95th, percentile("duration", 99) AS duration_99th INTO gitlab.downsampled.rails_git_timings_per_action FROM gitlab."default".rails_method_calls WHERE (action =~ /.+/ AND action !~ /^Grape#/) AND method =~ /^(Rugged|Gitlab::Git)/ GROUP BY time(1m), action END; -CREATE CONTINUOUS QUERY rails_markdown_render_timings_overall ON gitlab BEGIN SELECT mean(banzai_cached_render_real_time) AS cached_real_mean, percentile(banzai_cached_render_real_time, 95) AS cached_real_95th, percentile(banzai_cached_render_real_time, 99) AS cached_real_99th, mean(banzai_cached_render_cpu_time) AS cached_cpu_mean, percentile(banzai_cached_render_cpu_time, 95) AS cached_cpu_95th, percentile(banzai_cached_render_cpu_time, 99) AS cached_cpu_99th, sum(banzai_cached_render_call_count) AS cached_call_count, mean(banzai_cacheless_render_real_time) AS cacheless_real_mean, percentile(banzai_cacheless_render_real_time, 95) AS cacheless_real_95th, percentile(banzai_cacheless_render_real_time, 99) AS cacheless_real_99th, mean(banzai_cacheless_render_cpu_time) AS cacheless_cpu_mean, percentile(banzai_cacheless_render_cpu_time, 95) AS cacheless_cpu_95th, percentile(banzai_cacheless_render_cpu_time, 99) AS cacheless_cpu_99th, sum(banzai_cacheless_render_call_count) AS cacheless_call_count INTO gitlab.downsampled.rails_markdown_render_timings_overall FROM gitlab."default".rails_transactions WHERE (action =~ /.+/ AND action !~ /^Grape#/) AND (banzai_cached_render_call_count > 0 OR banzai_cacheless_render_call_count > 0) GROUP BY time(1m) END; -CREATE CONTINUOUS QUERY rails_markdown_render_timings_per_action ON gitlab BEGIN SELECT mean(banzai_cached_render_real_time) AS cached_real_mean, percentile(banzai_cached_render_real_time, 95) AS cached_real_95th, percentile(banzai_cached_render_real_time, 99) AS cached_real_99th, mean(banzai_cached_render_cpu_time) AS cached_cpu_mean, percentile(banzai_cached_render_cpu_time, 95) AS cached_cpu_95th, percentile(banzai_cached_render_cpu_time, 99) AS cached_cpu_99th, sum(banzai_cached_render_call_count) AS cached_call_count, mean(banzai_cacheless_render_real_time) AS cacheless_real_mean, percentile(banzai_cacheless_render_real_time, 95) AS cacheless_real_95th, percentile(banzai_cacheless_render_real_time, 99) AS cacheless_real_99th, mean(banzai_cacheless_render_cpu_time) AS cacheless_cpu_mean, percentile(banzai_cacheless_render_cpu_time, 95) AS cacheless_cpu_95th, percentile(banzai_cacheless_render_cpu_time, 99) AS cacheless_cpu_99th, sum(banzai_cacheless_render_call_count) AS cacheless_call_count INTO gitlab.downsampled.rails_markdown_render_timings_per_action FROM gitlab."default".rails_transactions WHERE (action =~ /.+/ AND action !~ /^Grape#/) AND (banzai_cached_render_call_count > 0 OR banzai_cacheless_render_call_count > 0) GROUP BY time(1m), action END; -CREATE CONTINUOUS QUERY rails_markdown_timings_overall ON gitlab BEGIN SELECT mean("duration") AS duration_mean, percentile("duration", 95) AS duration_95th, percentile("duration", 99) AS duration_99th INTO gitlab.downsampled.rails_markdown_timings_overall FROM gitlab."default".rails_method_calls WHERE (action =~ /.+/ AND action !~ /^Grape#/) AND method =~ /^Banzai/ GROUP BY time(1m) END; -CREATE CONTINUOUS QUERY rails_memory_usage_overall ON gitlab BEGIN SELECT mean(value) AS memory_mean, percentile(value, 95) AS memory_95th, percentile(value, 99) AS memory_99th INTO gitlab.downsampled.rails_memory_usage_overall FROM gitlab."default".rails_memory_usage GROUP BY time(1m) END; -CREATE CONTINUOUS QUERY rails_method_call_timings_per_action_and_method ON gitlab BEGIN SELECT mean("duration") AS duration_mean, percentile("duration", 95) AS duration_95th, percentile("duration", 99) AS duration_99th INTO gitlab.downsampled.rails_method_call_timings_per_action_and_method FROM gitlab."default".rails_method_calls WHERE action =~ /.+/ AND action !~ /^Grape#/ GROUP BY time(1m), method, action END; -CREATE CONTINUOUS QUERY rails_method_call_timings_per_method ON gitlab BEGIN SELECT mean("duration") AS duration_mean, percentile("duration", 95) AS duration_95th, percentile("duration", 99) AS duration_99th INTO gitlab.downsampled.rails_method_call_timings_per_method FROM gitlab."default".rails_method_calls WHERE action =~ /.+/ AND action !~ /^Grape#/ GROUP BY time(1m), method END; -CREATE CONTINUOUS QUERY rails_object_counts_overall ON gitlab BEGIN SELECT sum(count) AS count INTO gitlab.downsampled.rails_object_counts_overall FROM gitlab."default".rails_object_counts GROUP BY time(1m) END; -CREATE CONTINUOUS QUERY rails_object_counts_per_type ON gitlab BEGIN SELECT sum(count) AS count INTO gitlab.downsampled.rails_object_counts_per_type FROM gitlab."default".rails_object_counts GROUP BY time(1m), type END; -CREATE CONTINUOUS QUERY rails_transaction_counts_overall ON gitlab BEGIN SELECT count("duration") AS count INTO gitlab.downsampled.rails_transaction_counts_overall FROM gitlab."default".rails_transactions WHERE action =~ /.+/ AND action !~ /^Grape#/ GROUP BY time(1m) END; -CREATE CONTINUOUS QUERY rails_transaction_counts_per_action ON gitlab BEGIN SELECT count("duration") AS count INTO gitlab.downsampled.rails_transaction_counts_per_action FROM gitlab."default".rails_transactions WHERE action =~ /.+/ AND action !~ /^Grape#/ GROUP BY time(1m), action END; -CREATE CONTINUOUS QUERY rails_transaction_timings_overall ON gitlab BEGIN SELECT mean("duration") AS duration_mean, percentile("duration", 95) AS duration_95th, percentile("duration", 99) AS duration_99th, mean(sql_duration) AS sql_duration_mean, percentile(sql_duration, 95) AS sql_duration_95th, percentile(sql_duration, 99) AS sql_duration_99th, mean(view_duration) AS view_duration_mean, percentile(view_duration, 95) AS view_duration_95th, percentile(view_duration, 99) AS view_duration_99th, mean(cache_read_duration) AS cache_read_duration_mean, percentile(cache_read_duration, 99) AS cache_read_duration_99th, percentile(cache_read_duration, 95) AS cache_read_duration_95th, mean(cache_write_duration) AS cache_write_duration_mean, percentile(cache_write_duration, 99) AS cache_write_duration_99th, percentile(cache_write_duration, 95) AS cache_write_duration_95th, mean(cache_delete_duration) AS cache_delete_duration_mean, percentile(cache_delete_duration, 99) AS cache_delete_duration_99th, percentile(cache_delete_duration, 95) AS cache_delete_duration_95th, mean(cache_exists_duration) AS cache_exists_duration_mean, percentile(cache_exists_duration, 99) AS cache_exists_duration_99th, percentile(cache_exists_duration, 95) AS cache_exists_duration_95th, mean(cache_duration) AS cache_duration_mean, percentile(cache_duration, 99) AS cache_duration_99th, percentile(cache_duration, 95) AS cache_duration_95th, mean(method_duration) AS method_duration_mean, percentile(method_duration, 99) AS method_duration_99th, percentile(method_duration, 95) AS method_duration_95th INTO gitlab.downsampled.rails_transaction_timings_overall FROM gitlab."default".rails_transactions WHERE action =~ /.+/ AND action !~ /^Grape#/ GROUP BY time(1m) END; -CREATE CONTINUOUS QUERY rails_transaction_timings_per_action ON gitlab BEGIN SELECT mean("duration") AS duration_mean, percentile("duration", 95) AS duration_95th, percentile("duration", 99) AS duration_99th, mean(sql_duration) AS sql_duration_mean, percentile(sql_duration, 95) AS sql_duration_95th, percentile(sql_duration, 99) AS sql_duration_99th, mean(view_duration) AS view_duration_mean, percentile(view_duration, 95) AS view_duration_95th, percentile(view_duration, 99) AS view_duration_99th, mean(cache_read_duration) AS cache_read_duration_mean, percentile(cache_read_duration, 99) AS cache_read_duration_99th, percentile(cache_read_duration, 95) AS cache_read_duration_95th, mean(cache_write_duration) AS cache_write_duration_mean, percentile(cache_write_duration, 99) AS cache_write_duration_99th, percentile(cache_write_duration, 95) AS cache_write_duration_95th, mean(cache_delete_duration) AS cache_delete_duration_mean, percentile(cache_delete_duration, 99) AS cache_delete_duration_99th, percentile(cache_delete_duration, 95) AS cache_delete_duration_95th, mean(cache_exists_duration) AS cache_exists_duration_mean, percentile(cache_exists_duration, 99) AS cache_exists_duration_99th, percentile(cache_exists_duration, 95) AS cache_exists_duration_95th, mean(cache_duration) AS cache_duration_mean, percentile(cache_duration, 99) AS cache_duration_99th, percentile(cache_duration, 95) AS cache_duration_95th, mean(method_duration) AS method_duration_mean, percentile(method_duration, 99) AS method_duration_99th, percentile(method_duration, 95) AS method_duration_95th INTO gitlab.downsampled.rails_transaction_timings_per_action FROM gitlab."default".rails_transactions WHERE action =~ /.+/ AND action !~ /^Grape#/ GROUP BY time(1m), action END; -CREATE CONTINUOUS QUERY rails_view_timings_per_action_and_view ON gitlab BEGIN SELECT mean("duration") AS duration_mean, percentile("duration", 95) AS duration_95th, percentile("duration", 99) AS duration_99th INTO gitlab.downsampled.rails_view_timings_per_action_and_view FROM gitlab."default".rails_views WHERE action =~ /.+/ AND action !~ /^Grape#/ GROUP BY time(1m), action, view END; -CREATE CONTINUOUS QUERY sidekiq_file_descriptor_counts ON gitlab BEGIN SELECT sum(value) AS count INTO gitlab.downsampled.sidekiq_file_descriptor_counts FROM gitlab."default".sidekiq_file_descriptors GROUP BY time(1m) END; -CREATE CONTINUOUS QUERY sidekiq_gc_counts ON gitlab BEGIN SELECT sum(count) AS total, sum(minor_gc_count) AS minor, sum(major_gc_count) AS major INTO gitlab.downsampled.sidekiq_gc_counts FROM gitlab."default".sidekiq_gc_statistics GROUP BY time(1m) END; -CREATE CONTINUOUS QUERY sidekiq_gc_timings ON gitlab BEGIN SELECT mean(total_time) AS duration_mean, percentile(total_time, 95) AS duration_95th, percentile(total_time, 99) AS duration_99th INTO gitlab.downsampled.sidekiq_gc_timings FROM gitlab."default".sidekiq_gc_statistics GROUP BY time(1m) END; -CREATE CONTINUOUS QUERY sidekiq_git_timings_per_action ON gitlab BEGIN SELECT mean("duration") AS duration_mean, percentile("duration", 95) AS duration_95th, percentile("duration", 99) AS duration_99th INTO gitlab.downsampled.sidekiq_git_timings_per_action FROM gitlab."default".sidekiq_method_calls WHERE method =~ /^(Rugged|Gitlab::Git)/ GROUP BY time(1m), action END; -CREATE CONTINUOUS QUERY sidekiq_markdown_render_timings_overall ON gitlab BEGIN SELECT mean(banzai_cached_render_real_time) AS cached_real_mean, percentile(banzai_cached_render_real_time, 95) AS cached_real_95th, percentile(banzai_cached_render_real_time, 99) AS cached_real_99th, mean(banzai_cached_render_cpu_time) AS cached_cpu_mean, percentile(banzai_cached_render_cpu_time, 95) AS cached_cpu_95th, percentile(banzai_cached_render_cpu_time, 99) AS cached_cpu_99th, sum(banzai_cached_render_call_count) AS cached_call_count, mean(banzai_cacheless_render_real_time) AS cacheless_real_mean, percentile(banzai_cacheless_render_real_time, 95) AS cacheless_real_95th, percentile(banzai_cacheless_render_real_time, 99) AS cacheless_real_99th, mean(banzai_cacheless_render_cpu_time) AS cacheless_cpu_mean, percentile(banzai_cacheless_render_cpu_time, 95) AS cacheless_cpu_95th, percentile(banzai_cacheless_render_cpu_time, 99) AS cacheless_cpu_99th, sum(banzai_cacheless_render_call_count) AS cacheless_call_count INTO gitlab.downsampled.sidekiq_markdown_render_timings_overall FROM gitlab."default".sidekiq_transactions WHERE (banzai_cached_render_call_count > 0 OR banzai_cacheless_render_call_count > 0) GROUP BY time(1m) END; -CREATE CONTINUOUS QUERY sidekiq_markdown_render_timings_per_action ON gitlab BEGIN SELECT mean(banzai_cached_render_real_time) AS cached_real_mean, percentile(banzai_cached_render_real_time, 95) AS cached_real_95th, percentile(banzai_cached_render_real_time, 99) AS cached_real_99th, mean(banzai_cached_render_cpu_time) AS cached_cpu_mean, percentile(banzai_cached_render_cpu_time, 95) AS cached_cpu_95th, percentile(banzai_cached_render_cpu_time, 99) AS cached_cpu_99th, sum(banzai_cached_render_call_count) AS cached_call_count, mean(banzai_cacheless_render_real_time) AS cacheless_real_mean, percentile(banzai_cacheless_render_real_time, 95) AS cacheless_real_95th, percentile(banzai_cacheless_render_real_time, 99) AS cacheless_real_99th, mean(banzai_cacheless_render_cpu_time) AS cacheless_cpu_mean, percentile(banzai_cacheless_render_cpu_time, 95) AS cacheless_cpu_95th, percentile(banzai_cacheless_render_cpu_time, 99) AS cacheless_cpu_99th, sum(banzai_cacheless_render_call_count) AS cacheless_call_count INTO gitlab.downsampled.sidekiq_markdown_render_timings_per_action FROM gitlab."default".sidekiq_transactions WHERE (banzai_cached_render_call_count > 0 OR banzai_cacheless_render_call_count > 0) GROUP BY time(1m), action END; -CREATE CONTINUOUS QUERY sidekiq_markdown_timings_overall ON gitlab BEGIN SELECT mean("duration") AS duration_mean, percentile("duration", 95) AS duration_95th, percentile("duration", 99) AS duration_99th INTO gitlab.downsampled.sidekiq_markdown_timings_overall FROM gitlab."default".sidekiq_method_calls WHERE method =~ /^Banzai/ GROUP BY time(1m) END; -CREATE CONTINUOUS QUERY sidekiq_memory_usage_overall ON gitlab BEGIN SELECT mean(value) AS memory_mean, percentile(value, 95) AS memory_95th, percentile(value, 99) AS memory_99th INTO gitlab.downsampled.sidekiq_memory_usage_overall FROM gitlab."default".sidekiq_memory_usage GROUP BY time(1m) END; -CREATE CONTINUOUS QUERY sidekiq_method_call_timings_per_action_and_method ON gitlab BEGIN SELECT mean("duration") AS duration_mean, percentile("duration", 95) AS duration_95th, percentile("duration", 99) AS duration_99th INTO gitlab.downsampled.sidekiq_method_call_timings_per_action_and_method FROM gitlab."default".sidekiq_method_calls GROUP BY time(1m), method, action END; -CREATE CONTINUOUS QUERY sidekiq_method_call_timings_per_method ON gitlab BEGIN SELECT mean("duration") AS duration_mean, percentile("duration", 95) AS duration_95th, percentile("duration", 99) AS duration_99th INTO gitlab.downsampled.sidekiq_method_call_timings_per_method FROM gitlab."default".sidekiq_method_calls GROUP BY time(1m), method END; -CREATE CONTINUOUS QUERY sidekiq_object_counts_overall ON gitlab BEGIN SELECT sum(count) AS count INTO gitlab.downsampled.sidekiq_object_counts_overall FROM gitlab."default".sidekiq_object_counts GROUP BY time(1m) END; -CREATE CONTINUOUS QUERY sidekiq_object_counts_per_type ON gitlab BEGIN SELECT sum(count) AS count INTO gitlab.downsampled.sidekiq_object_counts_per_type FROM gitlab."default".sidekiq_object_counts GROUP BY time(1m), type END; -CREATE CONTINUOUS QUERY sidekiq_transaction_counts_overall ON gitlab BEGIN SELECT count("duration") AS count INTO gitlab.downsampled.sidekiq_transaction_counts_overall FROM gitlab."default".sidekiq_transactions GROUP BY time(1m) END; -CREATE CONTINUOUS QUERY sidekiq_transaction_counts_per_action ON gitlab BEGIN SELECT count("duration") AS count INTO gitlab.downsampled.sidekiq_transaction_counts_per_action FROM gitlab."default".sidekiq_transactions GROUP BY time(1m), action END; -CREATE CONTINUOUS QUERY sidekiq_transaction_timings_overall ON gitlab BEGIN SELECT mean("duration") AS duration_mean, percentile("duration", 95) AS duration_95th, percentile("duration", 99) AS duration_99th, mean(sql_duration) AS sql_duration_mean, percentile(sql_duration, 95) AS sql_duration_95th, percentile(sql_duration, 99) AS sql_duration_99th, mean(view_duration) AS view_duration_mean, percentile(view_duration, 95) AS view_duration_95th, percentile(view_duration, 99) AS view_duration_99th, mean(cache_read_duration) AS cache_read_duration_mean, percentile(cache_read_duration, 99) AS cache_read_duration_99th, percentile(cache_read_duration, 95) AS cache_read_duration_95th, mean(cache_write_duration) AS cache_write_duration_mean, percentile(cache_write_duration, 99) AS cache_write_duration_99th, percentile(cache_write_duration, 95) AS cache_write_duration_95th, mean(cache_delete_duration) AS cache_delete_duration_mean, percentile(cache_delete_duration, 99) AS cache_delete_duration_99th, percentile(cache_delete_duration, 95) AS cache_delete_duration_95th, mean(cache_exists_duration) AS cache_exists_duration_mean, percentile(cache_exists_duration, 99) AS cache_exists_duration_99th, percentile(cache_exists_duration, 95) AS cache_exists_duration_95th, mean(cache_duration) AS cache_duration_mean, percentile(cache_duration, 99) AS cache_duration_99th, percentile(cache_duration, 95) AS cache_duration_95th, mean(method_duration) AS method_duration_mean, percentile(method_duration, 99) AS method_duration_99th, percentile(method_duration, 95) AS method_duration_95th INTO gitlab.downsampled.sidekiq_transaction_timings_overall FROM gitlab."default".sidekiq_transactions GROUP BY time(1m) END; -CREATE CONTINUOUS QUERY sidekiq_transaction_timings_per_action ON gitlab BEGIN SELECT mean("duration") AS duration_mean, percentile("duration", 95) AS duration_95th, percentile("duration", 99) AS duration_99th, mean(sql_duration) AS sql_duration_mean, percentile(sql_duration, 95) AS sql_duration_95th, percentile(sql_duration, 99) AS sql_duration_99th, mean(view_duration) AS view_duration_mean, percentile(view_duration, 95) AS view_duration_95th, percentile(view_duration, 99) AS view_duration_99th, mean(cache_read_duration) AS cache_read_duration_mean, percentile(cache_read_duration, 99) AS cache_read_duration_99th, percentile(cache_read_duration, 95) AS cache_read_duration_95th, mean(cache_write_duration) AS cache_write_duration_mean, percentile(cache_write_duration, 99) AS cache_write_duration_99th, percentile(cache_write_duration, 95) AS cache_write_duration_95th, mean(cache_delete_duration) AS cache_delete_duration_mean, percentile(cache_delete_duration, 99) AS cache_delete_duration_99th, percentile(cache_delete_duration, 95) AS cache_delete_duration_95th, mean(cache_exists_duration) AS cache_exists_duration_mean, percentile(cache_exists_duration, 99) AS cache_exists_duration_99th, percentile(cache_exists_duration, 95) AS cache_exists_duration_95th, mean(cache_duration) AS cache_duration_mean, percentile(cache_duration, 99) AS cache_duration_99th, percentile(cache_duration, 95) AS cache_duration_95th, mean(method_duration) AS method_duration_mean, percentile(method_duration, 99) AS method_duration_99th, percentile(method_duration, 95) AS method_duration_95th INTO gitlab.downsampled.sidekiq_transaction_timings_per_action FROM gitlab."default".sidekiq_transactions GROUP BY time(1m), action END; -CREATE CONTINUOUS QUERY sidekiq_view_timings_per_action_and_view ON gitlab BEGIN SELECT mean("duration") AS duration_mean, percentile("duration", 95) AS duration_95th, percentile("duration", 99) AS duration_99th INTO gitlab.downsampled.sidekiq_view_timings_per_action_and_view FROM gitlab."default".sidekiq_views GROUP BY time(1m), action, view END; -CREATE CONTINUOUS QUERY web_transaction_counts_overall ON gitlab BEGIN SELECT count("duration") AS count INTO gitlab.downsampled.web_transaction_counts_overall FROM gitlab."default".rails_transactions GROUP BY time(1m) END; +gem install bundler +bundle install ``` +Now you must configure the repository by first copying `.env.example` to `.env` +and then editing the `.env` file to contain the correct InfluxDB settings. Once +configured you can simply run `bundle exec rake` and the InfluxDB database will +be configured for you. + +For more information see the [influxdb-management README](https://gitlab.com/gitlab-org/influxdb-management/blob/master/README.md). + ## Import Dashboards You can now import a set of default dashboards that will give you a good diff --git a/doc/project_services/bugzilla.md b/doc/project_services/bugzilla.md new file mode 100644 index 00000000000..215ed6fe9cc --- /dev/null +++ b/doc/project_services/bugzilla.md @@ -0,0 +1,17 @@ +# Bugzilla Service + +Go to your project's **Settings > Services > Bugzilla** and fill in the required +details as described in the table below. + +| Field | Description | +| ----- | ----------- | +| `description` | A name for the issue tracker (to differentiate between instances, for example) | +| `project_url` | The URL to the project in Bugzilla which is being linked to this GitLab project. Note that the `project_url` requires PRODUCT_NAME to be updated with the product/project name in Bugzilla. | +| `issues_url` | The URL to the issue in Bugzilla project that is linked to this GitLab project. Note that the `issues_url` requires `:id` in the URL. This ID is used by GitLab as a placeholder to replace the issue number. | +| `new_issue_url` | This is the URL to create a new issue in Bugzilla for the project linked to this GitLab project. Note that the `new_issue_url` requires PRODUCT_NAME to be updated with the product/project name in Bugzilla. | + +Once you have configured and enabled Bugzilla: + +- the **Issues** link on the GitLab project pages takes you to the appropriate + Bugzilla product page +- clicking **New issue** on the project dashboard takes you to Bugzilla for entering a new issue diff --git a/doc/project_services/project_services.md b/doc/project_services/project_services.md index f81a035f70b..e15d5db3253 100644 --- a/doc/project_services/project_services.md +++ b/doc/project_services/project_services.md @@ -30,6 +30,7 @@ further configuration instructions and details. Contributions are welcome. | [Atlassian Bamboo CI](bamboo.md) | A continuous integration and build server | | Buildkite | Continuous integration and deployments | | [Builds emails](builds_emails.md) | Email the builds status to a list of recipients | +| [Bugzilla](bugzilla.md) | Bugzilla issue tracker | | Campfire | Simple web-based real-time group chat | | Custom Issue Tracker | Custom issue tracker | | Drone CI | Continuous Integration platform built on Docker, written in Go | diff --git a/doc/update/2.6-to-3.0.md b/doc/update/2.6-to-3.0.md index 4827ef9501a..fb70eaacbc9 100644 --- a/doc/update/2.6-to-3.0.md +++ b/doc/update/2.6-to-3.0.md @@ -13,6 +13,10 @@ git fetch origin git checkout v3.0.3 +# The Modernizr gem was yanked from RubyGems. It is required for GitLab >= 2.8.0 +# Edit `Gemfile` and change `gem "modernizr", "2.5.3"` to +# `gem "modernizr-rails", "2.7.1"`` +sudo -u gitlab -H vim Gemfile # Install libs sudo -u gitlab bundle install --without development test postgres diff --git a/doc/update/2.9-to-3.0.md b/doc/update/2.9-to-3.0.md index f4a997a8c5e..ce46b57c09a 100644 --- a/doc/update/2.9-to-3.0.md +++ b/doc/update/2.9-to-3.0.md @@ -13,6 +13,11 @@ sudo -u gitlab -H git fetch origin sudo -u gitlab -H git checkout v3.0.3 +# The Modernizr gem was yanked from RubyGems. It is required for GitLab >= 2.8.0 +# Edit `Gemfile` and change `gem "modernizr", "2.5.3"` to +# `gem "modernizr-rails", "2.7.1"`` +sudo -u gitlab -H vim Gemfile + # Install gems sudo -u gitlab -H bundle install --without development test postgres diff --git a/doc/update/3.0-to-3.1.md b/doc/update/3.0-to-3.1.md index a30485c42f7..6ac83f3b60d 100644 --- a/doc/update/3.0-to-3.1.md +++ b/doc/update/3.0-to-3.1.md @@ -25,6 +25,11 @@ sudo -u gitlab -H git checkout v3.1.0 # Install new charlock_holmes sudo gem install charlock_holmes --version '0.6.9' +# The Modernizr gem was yanked from RubyGems. It is required for GitLab >= 2.8.0 +# Edit `Gemfile` and change `gem "modernizr", "2.5.3"` to +# `gem "modernizr-rails", "2.7.1"`` +sudo -u gitlab -H vim Gemfile + # Install gems for MySQL sudo -u gitlab -H bundle install --without development test postgres sqlite diff --git a/doc/update/3.1-to-4.0.md b/doc/update/3.1-to-4.0.md index f1ef4df4744..df53ed6de83 100644 --- a/doc/update/3.1-to-4.0.md +++ b/doc/update/3.1-to-4.0.md @@ -26,6 +26,11 @@ I wrote a bash script which will do it automatically for you. Just make sure all sudo -u gitlab -H git fetch sudo -u gitlab -H git checkout 4-0-stable +# The Modernizr gem was yanked from RubyGems. It is required for GitLab >= 2.8.0 +# Edit `Gemfile` and change `gem "modernizr", "2.5.3"` to +# `gem "modernizr-rails", "2.7.1"`` +sudo -u gitlab -H vim Gemfile + # Install gems for MySQL sudo -u gitlab -H bundle install --without development test postgres diff --git a/doc/update/4.0-to-4.1.md b/doc/update/4.0-to-4.1.md index d89d5235917..c163bfd348d 100644 --- a/doc/update/4.0-to-4.1.md +++ b/doc/update/4.0-to-4.1.md @@ -22,6 +22,11 @@ cd /home/gitlab/gitlab/ sudo -u gitlab -H git fetch sudo -u gitlab -H git checkout 4-1-stable +# The Modernizr gem was yanked from RubyGems. It is required for GitLab >= 2.8.0 +# Edit `Gemfile` and change `gem "modernizr", "2.5.3"` to +# `gem "modernizr-rails", "2.7.1"`` +sudo -u gitlab -H vim Gemfile + # Install gems for MySQL sudo -u gitlab -H bundle install --without development test postgres diff --git a/doc/update/4.1-to-4.2.md b/doc/update/4.1-to-4.2.md index 6fe4412ff90..97367c5f347 100644 --- a/doc/update/4.1-to-4.2.md +++ b/doc/update/4.1-to-4.2.md @@ -17,7 +17,15 @@ sudo -u gitlab -H git fetch sudo -u gitlab -H git checkout 4-2-stable -# Install libs +# The Modernizr gem was yanked from RubyGems. It is required for GitLab >= 2.8.0 +# Edit `Gemfile` and change `gem "modernizr", "2.5.3"` to +# `gem "modernizr-rails", "2.7.1"`` +sudo -u gitlab -H vim Gemfile + +# Run a bundle install without deployment to generate the new Gemfile +sudo -u gitlab -H bundle install --without development test postgres --no-deployment + +# Install libs (with deployment this time) sudo -u gitlab -H bundle install --without development test postgres --deployment # update db diff --git a/doc/update/4.2-to-5.0.md b/doc/update/4.2-to-5.0.md index f9faf65f952..ee6de51c923 100644 --- a/doc/update/4.2-to-5.0.md +++ b/doc/update/4.2-to-5.0.md @@ -85,8 +85,17 @@ sudo -u git -H cp config/gitlab.yml.example config/gitlab.yml # edit it sudo -u git -H vim config/gitlab.yml +# The Modernizr gem was yanked from RubyGems. It is required for GitLab >= 2.8.0 +# Edit `Gemfile` and change `gem "modernizr", "2.5.3"` to +# `gem "modernizr-rails", "2.7.1"`` +sudo -u git -H vim Gemfile +# Run a bundle install without deployment to generate the new Gemfile +sudo -u git -H bundle install --without development test postgres --no-deployment + +# Install libs (with deployment this time) sudo -u git -H bundle install --without development test postgres --deployment + sudo -u git -H bundle exec rake db:migrate RAILS_ENV=production sudo -u git -H bundle exec rake gitlab:shell:setup RAILS_ENV=production sudo -u git -H bundle exec rake gitlab:shell:build_missing_projects RAILS_ENV=production diff --git a/doc/update/5.0-to-5.1.md b/doc/update/5.0-to-5.1.md index 9fbd1f88515..f0fddcf83af 100644 --- a/doc/update/5.0-to-5.1.md +++ b/doc/update/5.0-to-5.1.md @@ -42,7 +42,17 @@ cd /home/git/gitlab sudo rm tmp/sockets/gitlab.socket sudo -u git -H cp config/puma.rb.example config/puma.rb +# The Modernizr gem was yanked from RubyGems. It is required for GitLab >= 2.8.0 +# Edit `Gemfile` and change `gem "modernizr", "2.5.3"` to +# `gem "modernizr-rails", "2.7.1"`` +sudo -u git -H vim Gemfile + +# Run a bundle install without deployment to generate the new Gemfile +sudo -u git -H bundle install --without development test postgres --no-deployment + +# Install libs (with deployment this time) sudo -u git -H bundle install --without development test postgres --deployment + sudo -u git -H bundle exec rake db:migrate RAILS_ENV=production sudo -u git -H bundle exec rake migrate_merge_requests RAILS_ENV=production sudo -u git -H bundle exec rake assets:precompile RAILS_ENV=production diff --git a/doc/update/5.1-to-5.2.md b/doc/update/5.1-to-5.2.md index cf9c4e4f770..625fcc33852 100644 --- a/doc/update/5.1-to-5.2.md +++ b/doc/update/5.1-to-5.2.md @@ -40,12 +40,28 @@ sudo -u git -H git checkout v1.4.0 ```bash cd /home/git/gitlab +# The Modernizr gem was yanked from RubyGems. It is required for GitLab >= 2.8.0 +# Edit `Gemfile` and change `gem "modernizr", "2.5.3"` to +# `gem "modernizr-rails", "2.7.1"`` +sudo -u git -H vim Gemfile + # MySQL + +# Run a bundle install without deployment to generate the new Gemfile +sudo -u git -H bundle install --without development test postgres --no-deployment + +# Install libs (with deployment this time) sudo -u git -H bundle install --without development test postgres --deployment -#PostgreSQL +# PostgreSQL + +# Run a bundle install without deployment to generate the new Gemfile +sudo -u git -H bundle install --without development test mysql --no-deployment + +# Install libs (with deployment this time) sudo -u git -H bundle install --without development test mysql --deployment +# Both MySQL and PostgreSQL sudo -u git -H bundle exec rake db:migrate RAILS_ENV=production sudo -u git -H bundle exec rake assets:precompile RAILS_ENV=production diff --git a/doc/update/5.1-to-5.4.md b/doc/update/5.1-to-5.4.md index 97a98ede070..547d453914c 100644 --- a/doc/update/5.1-to-5.4.md +++ b/doc/update/5.1-to-5.4.md @@ -37,12 +37,28 @@ sudo -u git -H git checkout v1.7.9 # Addresses multiple critical security vulner ```bash cd /home/git/gitlab +# The Modernizr gem was yanked from RubyGems. It is required for GitLab >= 2.8.0 +# Edit `Gemfile` and change `gem "modernizr", "2.5.3"` to +# `gem "modernizr-rails", "2.7.1"`` +sudo -u git -H vim Gemfile + # MySQL + +# Run a bundle install without deployment to generate the new Gemfile +sudo -u git -H bundle install --without development test postgres --no-deployment + +# Install libs (with deployment this time) sudo -u git -H bundle install --without development test postgres --deployment -#PostgreSQL +# PostgreSQL + +# Run a bundle install without deployment to generate the new Gemfile +sudo -u git -H bundle install --without development test mysql --no-deployment + +# Install libs (with deployment this time) sudo -u git -H bundle install --without development test mysql --deployment +# Both MySQL and PostgreSQL sudo -u git -H bundle exec rake db:migrate RAILS_ENV=production sudo -u git -H bundle exec rake assets:precompile RAILS_ENV=production diff --git a/doc/update/5.1-to-6.0.md b/doc/update/5.1-to-6.0.md index a3fdd92bd2f..c992c69678e 100644 --- a/doc/update/5.1-to-6.0.md +++ b/doc/update/5.1-to-6.0.md @@ -137,12 +137,28 @@ sudo apt-get install python-docutils ```bash cd /home/git/gitlab +# The Modernizr gem was yanked from RubyGems. It is required for GitLab >= 2.8.0 +# Edit `Gemfile` and change `gem "modernizr", "2.5.3"` to +# `gem "modernizr-rails", "2.7.1"`` +sudo -u git -H vim Gemfile + # MySQL + +# Run a bundle install without deployment to generate the new Gemfile +sudo -u git -H bundle install --without development test postgres --no-deployment + +# Install libs (with deployment this time) sudo -u git -H bundle install --without development test postgres --deployment -#PostgreSQL +# PostgreSQL + +# Run a bundle install without deployment to generate the new Gemfile +sudo -u git -H bundle install --without development test mysql --no-deployment + +# Install libs (with deployment this time) sudo -u git -H bundle install --without development test mysql --deployment +# Both MySQL and PostgreSQL sudo -u git -H bundle exec rake db:migrate RAILS_ENV=production sudo -u git -H bundle exec rake migrate_groups RAILS_ENV=production sudo -u git -H bundle exec rake migrate_global_projects RAILS_ENV=production diff --git a/doc/update/5.2-to-5.3.md b/doc/update/5.2-to-5.3.md index 27613aeda07..c5254f6fb0c 100644 --- a/doc/update/5.2-to-5.3.md +++ b/doc/update/5.2-to-5.3.md @@ -31,12 +31,28 @@ sudo -u git -H git checkout 5-3-stable ```bash cd /home/git/gitlab +# The Modernizr gem was yanked from RubyGems. It is required for GitLab >= 2.8.0 +# Edit `Gemfile` and change `gem "modernizr", "2.5.3"` to +# `gem "modernizr-rails", "2.7.1"`` +sudo -u git -H vim Gemfile + # MySQL + +# Run a bundle install without deployment to generate the new Gemfile +sudo -u git -H bundle install --without development test postgres --no-deployment + +# Install libs (with deployment this time) sudo -u git -H bundle install --without development test postgres --deployment -#PostgreSQL +# PostgreSQL + +# Run a bundle install without deployment to generate the new Gemfile +sudo -u git -H bundle install --without development test mysql --no-deployment + +# Install libs (with deployment this time) sudo -u git -H bundle install --without development test mysql --deployment +# Both MySQL and PostgreSQL sudo -u git -H bundle exec rake db:migrate RAILS_ENV=production sudo -u git -H bundle exec rake assets:precompile RAILS_ENV=production diff --git a/doc/update/5.3-to-5.4.md b/doc/update/5.3-to-5.4.md index 577b9a585ff..c4a6146dcda 100644 --- a/doc/update/5.3-to-5.4.md +++ b/doc/update/5.3-to-5.4.md @@ -35,12 +35,28 @@ sudo -u git -H git checkout v1.7.9 # Addresses multiple critical security vulner ```bash cd /home/git/gitlab +# The Modernizr gem was yanked from RubyGems. It is required for GitLab >= 2.8.0 +# Edit `Gemfile` and change `gem "modernizr", "2.5.3"` to +# `gem "modernizr-rails", "2.7.1"`` +sudo -u git -H vim Gemfile + # MySQL + +# Run a bundle install without deployment to generate the new Gemfile +sudo -u git -H bundle install --without development test postgres --no-deployment + +# Install libs (with deployment this time) sudo -u git -H bundle install --without development test postgres --deployment -#PostgreSQL +# PostgreSQL + +# Run a bundle install without deployment to generate the new Gemfile +sudo -u git -H bundle install --without development test mysql --no-deployment + +# Install libs (with deployment this time) sudo -u git -H bundle install --without development test mysql --deployment +# Both MySQL and PostgreSQL sudo -u git -H bundle exec rake db:migrate RAILS_ENV=production sudo -u git -H bundle exec rake assets:precompile RAILS_ENV=production diff --git a/doc/update/5.4-to-6.0.md b/doc/update/5.4-to-6.0.md index d9c6d9bfb91..f0fee634322 100644 --- a/doc/update/5.4-to-6.0.md +++ b/doc/update/5.4-to-6.0.md @@ -73,12 +73,28 @@ sudo apt-get install python-docutils ```bash cd /home/git/gitlab +# The Modernizr gem was yanked from RubyGems. It is required for GitLab >= 2.8.0 +# Edit `Gemfile` and change `gem "modernizr", "2.5.3"` to +# `gem "modernizr-rails", "2.7.1"`` +sudo -u git -H vim Gemfile + # MySQL + +# Run a bundle install without deployment to generate the new Gemfile +sudo -u git -H bundle install --without development test postgres --no-deployment + +# Install libs (with deployment this time) sudo -u git -H bundle install --without development test postgres --deployment # PostgreSQL + +# Run a bundle install without deployment to generate the new Gemfile +sudo -u git -H bundle install --without development test mysql --no-deployment + +# Install libs (with deployment this time) sudo -u git -H bundle install --without development test mysql --deployment +# Both MySQL and PostgreSQL sudo -u git -H bundle exec rake db:migrate RAILS_ENV=production sudo -u git -H bundle exec rake migrate_groups RAILS_ENV=production sudo -u git -H bundle exec rake migrate_global_projects RAILS_ENV=production diff --git a/doc/update/6.0-to-6.1.md b/doc/update/6.0-to-6.1.md index c5eba1c01c4..409faf30902 100644 --- a/doc/update/6.0-to-6.1.md +++ b/doc/update/6.0-to-6.1.md @@ -50,13 +50,28 @@ sudo -u git -H git checkout v1.7.9 ```bash cd /home/git/gitlab +# The Modernizr gem was yanked from RubyGems. It is required for GitLab >= 2.8.0 +# Edit `Gemfile` and change `gem "modernizr", "2.5.3"` to +# `gem "modernizr-rails", "2.7.1"`` +sudo -u git -H vim Gemfile + # MySQL + +# Run a bundle install without deployment to generate the new Gemfile +sudo -u git -H bundle install --without development test postgres --no-deployment + +# Install libs (with deployment this time) sudo -u git -H bundle install --without development test postgres --deployment -#PostgreSQL -sudo -u git -H bundle install --without development test mysql --deployment +# PostgreSQL + +# Run a bundle install without deployment to generate the new Gemfile +sudo -u git -H bundle install --without development test mysql --no-deployment +# Install libs (with deployment this time) +sudo -u git -H bundle install --without development test mysql --deployment +# Both MySQL and PostgreSQL sudo -u git -H bundle exec rake db:migrate RAILS_ENV=production sudo -u git -H bundle exec rake migrate_iids RAILS_ENV=production sudo -u git -H bundle exec rake assets:clean RAILS_ENV=production diff --git a/doc/update/6.1-to-6.2.md b/doc/update/6.1-to-6.2.md index a534528108a..150c7ae1c83 100644 --- a/doc/update/6.1-to-6.2.md +++ b/doc/update/6.1-to-6.2.md @@ -45,13 +45,28 @@ sudo apt-get install logrotate ```bash cd /home/git/gitlab +# The Modernizr gem was yanked from RubyGems. It is required for GitLab >= 2.8.0 +# Edit `Gemfile` and change `gem "modernizr", "2.5.3"` to +# `gem "modernizr-rails", "2.7.1"`` +sudo -u git -H vim Gemfile + # MySQL + +# Run a bundle install without deployment to generate the new Gemfile +sudo -u git -H bundle install --without development test postgres --no-deployment + +# Install libs (with deployment this time) sudo -u git -H bundle install --without development test postgres --deployment -#PostgreSQL -sudo -u git -H bundle install --without development test mysql --deployment +# PostgreSQL + +# Run a bundle install without deployment to generate the new Gemfile +sudo -u git -H bundle install --without development test mysql --no-deployment +# Install libs (with deployment this time) +sudo -u git -H bundle install --without development test mysql --deployment +# Both MySQL and PostgreSQL sudo -u git -H bundle exec rake db:migrate RAILS_ENV=production sudo -u git -H bundle exec rake assets:clean RAILS_ENV=production sudo -u git -H bundle exec rake assets:precompile RAILS_ENV=production diff --git a/doc/update/6.2-to-6.3.md b/doc/update/6.2-to-6.3.md index b08ebde0808..b96dfb8add7 100644 --- a/doc/update/6.2-to-6.3.md +++ b/doc/update/6.2-to-6.3.md @@ -40,13 +40,28 @@ The gitlab-shell config changed recently, so check for config file changes and m ```bash cd /home/git/gitlab +# The Modernizr gem was yanked from RubyGems. It is required for GitLab >= 2.8.0 +# Edit `Gemfile` and change `gem "modernizr", "2.5.3"` to +# `gem "modernizr-rails", "2.7.1"`` +sudo -u git -H vim Gemfile + # MySQL + +# Run a bundle install without deployment to generate the new Gemfile +sudo -u git -H bundle install --without development test postgres --no-deployment + +# Install libs (with deployment this time) sudo -u git -H bundle install --without development test postgres --deployment # PostgreSQL -sudo -u git -H bundle install --without development test mysql --deployment +# Run a bundle install without deployment to generate the new Gemfile +sudo -u git -H bundle install --without development test mysql --no-deployment + +# Install libs (with deployment this time) +sudo -u git -H bundle install --without development test mysql --deployment +# Both MySQL and PostgreSQL # Run database migrations sudo -u git -H bundle exec rake db:migrate RAILS_ENV=production diff --git a/doc/update/6.3-to-6.4.md b/doc/update/6.3-to-6.4.md index 951d92dfeb5..37028be055f 100644 --- a/doc/update/6.3-to-6.4.md +++ b/doc/update/6.3-to-6.4.md @@ -36,13 +36,28 @@ sudo -u git -H git checkout v1.8.0 ```bash cd /home/git/gitlab +# The Modernizr gem was yanked from RubyGems. It is required for GitLab >= 2.8.0 +# Edit `Gemfile` and change `gem "modernizr", "2.5.3"` to +# `gem "modernizr-rails", "2.7.1"`` +sudo -u git -H vim Gemfile + # MySQL + +# Run a bundle install without deployment to generate the new Gemfile +sudo -u git -H bundle install --without development test postgres --no-deployment + +# Install libs (with deployment this time) sudo -u git -H bundle install --without development test postgres --deployment # PostgreSQL -sudo -u git -H bundle install --without development test mysql --deployment +# Run a bundle install without deployment to generate the new Gemfile +sudo -u git -H bundle install --without development test mysql --no-deployment + +# Install libs (with deployment this time) +sudo -u git -H bundle install --without development test mysql --deployment +# Both MySQL and PostgreSQL # Run database migrations sudo -u git -H bundle exec rake db:migrate RAILS_ENV=production diff --git a/doc/update/6.4-to-6.5.md b/doc/update/6.4-to-6.5.md index 0dae9a9fe59..982381a4db0 100644 --- a/doc/update/6.4-to-6.5.md +++ b/doc/update/6.4-to-6.5.md @@ -46,13 +46,28 @@ sudo -u git -H git checkout v1.8.0 ```bash cd /home/git/gitlab -# MySQL installations (note: the line below states '--without ... postgres') +# The Modernizr gem was yanked from RubyGems. It is required for GitLab >= 2.8.0 +# Edit `Gemfile` and change `gem "modernizr", "2.5.3"` to +# `gem "modernizr-rails", "2.7.1"`` +sudo -u git -H vim Gemfile + +# MySQL + +# Run a bundle install without deployment to generate the new Gemfile +sudo -u git -H bundle install --without development test postgres --no-deployment + +# Install libs (with deployment this time) sudo -u git -H bundle install --without development test postgres --deployment -# PostgreSQL installations (note: the line below states '--without ... mysql') -sudo -u git -H bundle install --without development test mysql --deployment +# PostgreSQL + +# Run a bundle install without deployment to generate the new Gemfile +sudo -u git -H bundle install --without development test mysql --no-deployment +# Install libs (with deployment this time) +sudo -u git -H bundle install --without development test mysql --deployment +# Both MySQL and PostgreSQL # Run database migrations sudo -u git -H bundle exec rake db:migrate RAILS_ENV=production diff --git a/doc/update/6.5-to-6.6.md b/doc/update/6.5-to-6.6.md index c24e83eb006..bbed2b30215 100644 --- a/doc/update/6.5-to-6.6.md +++ b/doc/update/6.5-to-6.6.md @@ -46,12 +46,28 @@ sudo -u git -H git checkout v1.8.0 ```bash cd /home/git/gitlab -# MySQL installations (note: the line below states '--without ... postgres') +# The Modernizr gem was yanked from RubyGems. It is required for GitLab >= 2.8.0 +# Edit `Gemfile` and change `gem "modernizr", "2.5.3"` to +# `gem "modernizr-rails", "2.7.1"`` +sudo -u git -H vim Gemfile + +# MySQL + +# Run a bundle install without deployment to generate the new Gemfile +sudo -u git -H bundle install --without development test postgres --no-deployment + +# Install libs (with deployment this time) sudo -u git -H bundle install --without development test postgres --deployment -# PostgreSQL installations (note: the line below states '--without ... mysql') +# PostgreSQL + +# Run a bundle install without deployment to generate the new Gemfile +sudo -u git -H bundle install --without development test mysql --no-deployment + +# Install libs (with deployment this time) sudo -u git -H bundle install --without development test mysql --deployment +# Both MySQL and PostgreSQL # Run database migrations sudo -u git -H bundle exec rake db:migrate RAILS_ENV=production diff --git a/doc/update/6.6-to-6.7.md b/doc/update/6.6-to-6.7.md index b4298c93429..8e82942a1a0 100644 --- a/doc/update/6.6-to-6.7.md +++ b/doc/update/6.6-to-6.7.md @@ -46,13 +46,23 @@ sudo -u git -H git checkout v1.9.1 ```bash cd /home/git/gitlab -# MySQL installations (note: the line below states '--without ... postgres') +# MySQL + +# Run a bundle install without deployment to generate the new Gemfile +sudo -u git -H bundle install --without development test postgres --no-deployment + +# Install libs (with deployment this time) sudo -u git -H bundle install --without development test postgres --deployment -# PostgreSQL installations (note: the line below states '--without ... mysql') -sudo -u git -H bundle install --without development test mysql --deployment +# PostgreSQL +# Run a bundle install without deployment to generate the new Gemfile +sudo -u git -H bundle install --without development test mysql --no-deployment + +# Install libs (with deployment this time) +sudo -u git -H bundle install --without development test mysql --deployment +# Both MySQL and PostgreSQL # Run database migrations sudo -u git -H bundle exec rake db:migrate RAILS_ENV=production diff --git a/doc/update/6.x-or-7.x-to-7.14.md b/doc/update/6.x-or-7.x-to-7.14.md index c45fc9340ea..f170a0021b7 100644 --- a/doc/update/6.x-or-7.x-to-7.14.md +++ b/doc/update/6.x-or-7.x-to-7.14.md @@ -147,12 +147,15 @@ sudo -u git -H bundle install --without development test postgres --deployment # PostgreSQL installations (note: the line below states '--without ... mysql') sudo -u git -H bundle install --without development test mysql --deployment -# Run database migrations -sudo -u git -H bundle exec rake db:migrate RAILS_ENV=production +# Run database migrations from 6.0 to 6.1 +sudo -u git -H bundle exec rake db:migrate RAILS_ENV=production VERSION=20130909132950 # Enable internal issue IDs (introduced in GitLab 6.1) sudo -u git -H bundle exec rake migrate_iids RAILS_ENV=production +# Run left database migrations +sudo -u git -H bundle exec rake db:migrate RAILS_ENV=production + # Clean up assets and cache sudo -u git -H bundle exec rake assets:clean assets:precompile cache:clear RAILS_ENV=production diff --git a/doc/update/8.8-to-8.9.md b/doc/update/8.8-to-8.9.md index f14046bb4be..423140a92c7 100644 --- a/doc/update/8.8-to-8.9.md +++ b/doc/update/8.8-to-8.9.md @@ -122,6 +122,19 @@ via [/etc/default/gitlab]. [Apache templates]: https://gitlab.com/gitlab-org/gitlab-recipes/tree/master/web-server/apache [/etc/default/gitlab]: https://gitlab.com/gitlab-org/gitlab-ce/blob/8-9-stable/lib/support/init.d/gitlab.default.example#L37 +#### SMTP configuration + +If you're installing from source and use SMTP to deliver mail, you will need to add the following line +to config/initializers/smtp_settings.rb: + +```ruby +ActionMailer::Base.delivery_method = :smtp +``` + +See [smtp_settings.rb.sample] as an example. + +[smtp_settings.rb.sample]: https://gitlab.com/gitlab-org/gitlab-ce/blob/v8.9.0/config/initializers/smtp_settings.rb.sample#L13 + #### Init script Ensure you're still up-to-date with the latest init script changes: diff --git a/doc/user/project/highlighting.md b/doc/user/project/highlighting.md new file mode 100644 index 00000000000..73a2d176b54 --- /dev/null +++ b/doc/user/project/highlighting.md @@ -0,0 +1,31 @@ +[Rouge]: https://rubygems.org/gems/rouge + +# Syntax Highlighting + +GitLab provides syntax highlighting on all files and snippets through the [Rouge][] rubygem. It will try to guess what language to use based on the file extension, which most of the time is sufficient. + +If GitLab is guessing wrong, you can override its choice of language using the `gitlab-language` attribute in `.gitattributes`. For example, if you are working in a Prolog project and using the `.pl` file extension (which would normally be highlighted as Perl), you can add the following to your `.gitattributes` file: + +``` conf +*.pl gitlab-language=prolog +``` + +When you check in and push that change, all `*.pl` files in your project will be highlighted as Prolog. + +The paths here are simply git's builtin [`.gitattributes` interface](https://git-scm.com/docs/gitattributes). So, if you were to invent a file format called a `Nicefile` at the root of your project that used ruby syntax, all you need is: + +``` conf +/Nicefile gitlab-language=ruby +``` + +To disable highlighting entirely, use `gitlab-language=text`. Lots more fun shenanigans are available through CGI options, such as: + +``` conf +# json with erb in it +/my-cool-file gitlab-language=erb?parent=json + +# an entire file of highlighting errors! +/other-file gitlab-language=text?token=Error +``` + +Please note that these configurations will only take effect when the `.gitattributes` file is in your default branch (usually `master`). diff --git a/doc/workflow/img/todo_list_item.png b/doc/workflow/img/todo_list_item.png Binary files differnew file mode 100644 index 00000000000..884ba1d22a3 --- /dev/null +++ b/doc/workflow/img/todo_list_item.png diff --git a/doc/workflow/img/todos_add_todo_sidebar.png b/doc/workflow/img/todos_add_todo_sidebar.png Binary files differnew file mode 100644 index 00000000000..126ecc2c82f --- /dev/null +++ b/doc/workflow/img/todos_add_todo_sidebar.png diff --git a/doc/workflow/img/todos_icon.png b/doc/workflow/img/todos_icon.png Binary files differindex 879b3b51c21..a63bad0c258 100644 --- a/doc/workflow/img/todos_icon.png +++ b/doc/workflow/img/todos_icon.png diff --git a/doc/workflow/img/todos_mark_done_sidebar.png b/doc/workflow/img/todos_mark_done_sidebar.png Binary files differnew file mode 100644 index 00000000000..f449f977dd6 --- /dev/null +++ b/doc/workflow/img/todos_mark_done_sidebar.png diff --git a/doc/workflow/todos.md b/doc/workflow/todos.md index 5f440fdafdd..9524ffd5420 100644 --- a/doc/workflow/todos.md +++ b/doc/workflow/todos.md @@ -1,4 +1,4 @@ -# GitLab ToDos +# GitLab Todos >**Note:** This feature was [introduced][ce-2817] in GitLab 8.5. @@ -14,8 +14,9 @@ in a simple dashboard. --- -You can access quickly your Todos dashboard by clicking the round gray icon -next to the search bar in the upper right corner. +You can quickly access the Todos dashboard using the bell icon next to the +search bar in the upper right corner. The number in blue is the number of Todos +you still have open. ![Todos icon](img/todos_icon.png) @@ -29,45 +30,61 @@ A Todo appears in your Todos dashboard when: >**Note:** Commenting on a commit will _not_ trigger a Todo. -## How a Todo is marked as Done +### Manually creating a Todo + +You can also add an issue or merge request to your Todos dashboard by clicking +the "Add Todo" button in the issue or merge request sidebar. + +![Adding a Todo from the issuable sidebar](img/todos_add_todo_sidebar.png) + +## Marking a Todo as done Any action to the corresponding issue or merge request will mark your Todo as -**Done**. This action can include: +**Done**. Actions that dismiss Todos include: - changing the assignee - changing the milestone - adding/removing a label - commenting on the issue -In case where you think no action is needed, you can manually mark the todo as -done by clicking the corresponding **Done** button, and it will disappear from -your Todos list. If you want to mark all your Todos as done, just click on the -**Mark all as done** button. - --- -In order for a Todo to be marked as done, the action must be coming from you. -So, if you close the related issue or merge the merge request yourself, and you -had a Todo for that, it will automatically get marked as done. On the other -hand, if someone else closes, merges or takes action on the issue or merge -request, your Todo will remain pending. This makes sense because you may need -to give attention to an issue even if it has been resolved. +Todos are personal, and they're only marked as done if the action is coming from +you. If you close the issue or merge request, your Todo will automatically +be marked as done. + +If someone else closes, merges, or takes action on the issue or merge +request, your Todo will remain pending. This prevents other users from closing issues without you being notified. There is just one Todo per issue or merge request, so mentioning a user a hundred times in an issue will only trigger one Todo. +--- + +If no action is needed, you can manually mark the Todo as done by clicking the +corresponding **Done** button, and it will disappear from your Todo list. + +![A Todo in the Todos dashboard](img/todo_list_item.png) + +A Todo can also be marked as done from the issue or merge request sidebar using +the "Mark Done" button. + +![Mark Done from the issuable sidebar](img/todos_mark_done_sidebar.png) + +You can mark all your Todos as done at once by clicking on the **Mark all as +done** button. + ## Filtering your Todos -In general, there are four kinds of filters you can use on your Todos -dashboard: +There are four kinds of filters you can use on your Todos dashboard. -| Filter | Description | -| ------ | ----------- | +| Filter | Description | +| ------- | ----------- | | Project | Filter by project | | Author | Filter by the author that triggered the Todo | | Type | Filter by issue or merge request | | Action | Filter by the action that triggered the Todo (Assigned or Mentioned)| -You can choose more than one filters at the same time. +You can also filter by more than one of these at the same time. [ce-2817]: https://gitlab.com/gitlab-org/gitlab-ce/merge_requests/2817 |