Welcome to mirror list, hosted at ThFree Co, Russian Federation.

gitlab.com/gitlab-org/gitlab-foss.git - Unnamed repository; edit this file 'description' to name the repository.
summaryrefslogtreecommitdiff
path: root/doc
diff options
context:
space:
mode:
authorAchilleas Pipinellis <axilleas@axilleas.me>2016-06-18 10:23:31 +0300
committerAchilleas Pipinellis <axilleas@axilleas.me>2016-06-18 10:23:31 +0300
commit56777e89562fe9c81f6e9237bff9cee6420fc093 (patch)
treea18d1dfad849311b38812db98d3f9cad520a49dc /doc
parent9169c7e81fb906cf9f419d195d73a585b19dafbc (diff)
parent00906b5bb6cde8cb60281109060a519a54000c61 (diff)
Merge branch 'master' into ci-scala-example
Diffstat (limited to 'doc')
-rw-r--r--doc/README.md15
-rw-r--r--doc/administration/auth/README.md11
-rw-r--r--doc/administration/auth/ldap.md277
-rw-r--r--doc/administration/container_registry.md375
-rw-r--r--doc/administration/environment_variables.md2
-rw-r--r--doc/administration/high_availability/README.md39
-rw-r--r--doc/administration/high_availability/database.md116
-rw-r--r--doc/administration/high_availability/gitlab.md131
-rw-r--r--doc/administration/high_availability/load_balancer.md63
-rw-r--r--doc/administration/high_availability/nfs.md116
-rw-r--r--doc/administration/high_availability/redis.md62
-rw-r--r--doc/administration/img/high_availability/active-active-diagram.pngbin0 -> 29607 bytes
-rw-r--r--doc/administration/img/high_availability/active-passive-diagram.pngbin0 -> 24246 bytes
-rw-r--r--doc/administration/logs.md137
-rw-r--r--doc/administration/repository_checks.md44
-rw-r--r--doc/administration/troubleshooting/sidekiq.md171
-rw-r--r--doc/api/README.md109
-rw-r--r--doc/api/award_emoji.md367
-rw-r--r--doc/api/build_triggers.md12
-rw-r--r--doc/api/builds.md552
-rw-r--r--doc/api/ci/README.md24
-rw-r--r--doc/api/ci/builds.md138
-rw-r--r--doc/api/ci/runners.md57
-rw-r--r--doc/api/commits.md2
-rw-r--r--doc/api/groups.md84
-rw-r--r--doc/api/issues.md209
-rw-r--r--doc/api/labels.md159
-rw-r--r--doc/api/licenses.md147
-rw-r--r--doc/api/merge_requests.md220
-rw-r--r--doc/api/milestones.md20
-rw-r--r--doc/api/notes.md145
-rw-r--r--doc/api/projects.md301
-rw-r--r--doc/api/runners.md4
-rw-r--r--doc/api/services.md8
-rw-r--r--doc/api/settings.md10
-rw-r--r--doc/api/sidekiq_metrics.md152
-rw-r--r--doc/api/tags.md46
-rw-r--r--doc/api/users.md86
-rw-r--r--doc/ci/README.md2
-rw-r--r--doc/ci/api/README.md21
-rw-r--r--doc/ci/api/builds.md72
-rw-r--r--doc/ci/api/runners.md45
-rw-r--r--doc/ci/build_artifacts/README.md13
-rw-r--r--doc/ci/docker/using_docker_build.md285
-rw-r--r--doc/ci/docker/using_docker_images.md14
-rw-r--r--doc/ci/examples/README.md2
-rw-r--r--doc/ci/examples/deployment/README.md (renamed from doc/ci/deployment/README.md)0
-rw-r--r--doc/ci/examples/php.md12
-rw-r--r--doc/ci/examples/test-and-deploy-python-application-to-heroku.md10
-rw-r--r--doc/ci/examples/test-and-deploy-ruby-application-to-heroku.md6
-rw-r--r--doc/ci/examples/test-scala-application.md29
-rw-r--r--doc/ci/quick_start/README.md6
-rw-r--r--doc/ci/runners/README.md22
-rw-r--r--doc/ci/services/mysql.md4
-rw-r--r--doc/ci/services/postgres.md2
-rw-r--r--doc/ci/services/redis.md2
-rw-r--r--doc/ci/ssh_keys/README.md10
-rw-r--r--doc/ci/triggers/README.md8
-rw-r--r--doc/ci/variables/README.md24
-rw-r--r--doc/ci/yaml/README.md230
-rw-r--r--doc/container_registry/README.md94
-rw-r--r--doc/container_registry/img/container_registry.pngbin0 -> 354050 bytes
-rw-r--r--doc/container_registry/img/project_feature.pngbin0 -> 392842 bytes
-rw-r--r--doc/customization/libravatar.md13
-rw-r--r--doc/development/README.md6
-rw-r--r--doc/development/code_review.md78
-rw-r--r--doc/development/doc_styleguide.md56
-rw-r--r--doc/development/instrumentation.md139
-rw-r--r--doc/development/licensing.md93
-rw-r--r--doc/development/migration_style_guide.md62
-rw-r--r--doc/development/performance.md258
-rw-r--r--doc/development/rake_tasks.md2
-rw-r--r--doc/development/scss_styleguide.md27
-rw-r--r--doc/development/testing.md137
-rw-r--r--doc/development/ui_guide.md52
-rw-r--r--doc/downgrade_ee_to_ce/README.md82
-rw-r--r--doc/gitlab-basics/README.md34
-rw-r--r--doc/gitlab-basics/basic-git-commands.md58
-rw-r--r--doc/gitlab-basics/create-issue.md2
-rw-r--r--doc/gitlab-basics/create-project.md2
-rw-r--r--doc/gitlab-basics/start-using-git.md63
-rw-r--r--doc/hooks/custom_hooks.md2
-rw-r--r--doc/incoming_email/README.md108
-rw-r--r--doc/install/installation.md80
-rw-r--r--doc/install/relative_url.md2
-rw-r--r--doc/install/requirements.md32
-rw-r--r--doc/integration/README.md17
-rw-r--r--doc/integration/cas.md19
-rw-r--r--doc/integration/github.md20
-rw-r--r--doc/integration/google.md4
-rw-r--r--doc/integration/img/enabled-oauth-sign-in-sources.pngbin0 -> 49081 bytes
-rw-r--r--doc/integration/ldap.md227
-rw-r--r--doc/integration/omniauth.md38
-rw-r--r--doc/integration/saml.md71
-rw-r--r--doc/integration/shibboleth.md47
-rw-r--r--doc/intro/README.md42
-rw-r--r--doc/logs/logs.md93
-rw-r--r--doc/markdown/markdown.md57
-rw-r--r--doc/migrate_ci_to_ce/README.md2
-rw-r--r--doc/monitoring/health_check.md66
-rw-r--r--doc/monitoring/img/health_check_token.pngbin0 -> 10884 bytes
-rw-r--r--doc/monitoring/performance/gitlab_configuration.md1
-rw-r--r--doc/monitoring/performance/grafana_configuration.md149
-rw-r--r--doc/monitoring/performance/img/grafana_dashboard_dropdown.pngbin0 -> 29419 bytes
-rw-r--r--doc/monitoring/performance/img/grafana_dashboard_import.pngbin0 -> 40974 bytes
-rw-r--r--doc/monitoring/performance/img/grafana_data_source_configuration.pngbin0 -> 53402 bytes
-rw-r--r--doc/monitoring/performance/img/grafana_data_source_empty.pngbin0 -> 44058 bytes
-rw-r--r--doc/monitoring/performance/img/grafana_save_icon.pngbin0 -> 16024 bytes
-rw-r--r--doc/monitoring/performance/influxdb_configuration.md1
-rw-r--r--doc/monitoring/performance/influxdb_schema.md1
-rw-r--r--doc/monitoring/performance/introduction.md3
-rw-r--r--doc/operations/moving_repositories.md8
-rw-r--r--doc/operations/sidekiq_memory_killer.md2
-rw-r--r--doc/permissions/permissions.md11
-rw-r--r--doc/profile/2fa_u2f_authenticate.pngbin0 -> 54413 bytes
-rw-r--r--doc/profile/2fa_u2f_register.pngbin0 -> 112414 bytes
-rw-r--r--doc/profile/two_factor_authentication.md63
-rw-r--r--doc/project_services/img/jira_service_page.pngbin35496 -> 49122 bytes
-rw-r--r--doc/project_services/jira.md22
-rw-r--r--doc/project_services/project_services.md19
-rw-r--r--doc/public_access/public_access.md22
-rw-r--r--doc/raketasks/README.md2
-rw-r--r--doc/raketasks/backup_restore.md38
-rw-r--r--doc/release/README.md10
-rw-r--r--doc/release/howto_rc1.md55
-rw-r--r--doc/release/howto_update_guides.md55
-rw-r--r--doc/release/master.md62
-rw-r--r--doc/release/monthly.md245
-rw-r--r--doc/release/patch.md81
-rw-r--r--doc/release/security.md76
-rw-r--r--doc/security/README.md1
-rw-r--r--doc/security/user_email_confirmation.md7
-rw-r--r--doc/system_hooks/system_hooks.md113
-rw-r--r--doc/update/8.2-to-8.3.md9
-rw-r--r--doc/update/8.3-to-8.4.md9
-rw-r--r--doc/update/8.4-to-8.5.md9
-rw-r--r--doc/update/8.5-to-8.6.md55
-rw-r--r--doc/update/8.6-to-8.7.md162
-rw-r--r--doc/update/8.7-to-8.8.md162
-rw-r--r--doc/update/8.8-to-8.9.md162
-rw-r--r--doc/update/README.md94
-rw-r--r--doc/update/patch_versions.md2
-rw-r--r--doc/update/restore_after_failure.md83
-rw-r--r--doc/web_hooks/web_hooks.md81
-rw-r--r--doc/workflow/README.md1
-rw-r--r--doc/workflow/award_emoji.md48
-rw-r--r--doc/workflow/cherry_pick_changes.md53
-rw-r--r--doc/workflow/gitlab_flow.md4
-rw-r--r--doc/workflow/groups.md2
-rw-r--r--doc/workflow/img/award_emoji_select.pngbin0 -> 65985 bytes
-rw-r--r--doc/workflow/img/award_emoji_votes_least_popular.pngbin0 -> 144501 bytes
-rw-r--r--doc/workflow/img/award_emoji_votes_most_popular.pngbin0 -> 136577 bytes
-rw-r--r--doc/workflow/img/award_emoji_votes_sort_options.pngbin0 -> 162251 bytes
-rw-r--r--doc/workflow/img/cherry_pick_changes_commit.pngbin0 -> 353067 bytes
-rw-r--r--doc/workflow/img/cherry_pick_changes_commit_modal.pngbin0 -> 312659 bytes
-rw-r--r--doc/workflow/img/cherry_pick_changes_mr.pngbin0 -> 252085 bytes
-rw-r--r--doc/workflow/img/cherry_pick_changes_mr_modal.pngbin0 -> 225569 bytes
-rw-r--r--doc/workflow/img/new_branch_from_issue.pngbin0 -> 120622 bytes
-rw-r--r--doc/workflow/importing/import_projects_from_github.md16
-rw-r--r--doc/workflow/importing/import_projects_from_gitlab_com.md2
-rw-r--r--doc/workflow/lfs/lfs_administration.md4
-rw-r--r--doc/workflow/lfs/manage_large_binaries_with_git_lfs.md8
-rw-r--r--doc/workflow/merge_requests.md17
-rw-r--r--doc/workflow/merge_requests/commit_compare.pngbin89631 -> 110376 bytes
-rw-r--r--doc/workflow/merge_requests/merge_request_diff.pngbin120422 -> 166226 bytes
-rw-r--r--doc/workflow/merge_requests/merge_request_diff_without_whitespace.pngbin98887 -> 121476 bytes
-rw-r--r--doc/workflow/merge_requests/only_allow_merge_if_build_succeeds.pngbin0 -> 17552 bytes
-rw-r--r--doc/workflow/notifications.md8
-rw-r--r--doc/workflow/notifications/settings.pngbin114727 -> 90986 bytes
-rw-r--r--doc/workflow/shortcuts.pngbin25005 -> 90936 bytes
-rw-r--r--doc/workflow/web_editor.md32
171 files changed, 7653 insertions, 1832 deletions
diff --git a/doc/README.md b/doc/README.md
index 08d0a6a5bfb..5d89d0c9821 100644
--- a/doc/README.md
+++ b/doc/README.md
@@ -3,7 +3,7 @@
## User documentation
- [API](api/README.md) Automate GitLab via a simple and powerful API.
-- [CI](ci/README.md)
+- [CI](ci/README.md) GitLab Continuous Integration (CI) getting started, `.gitlab-ci.yml` options, and examples.
- [GitLab as OAuth2 authentication service provider](integration/oauth_provider.md). It allows you to login to other applications from GitLab.
- [GitLab Basics](gitlab-basics/README.md) Find step by step how to start working on your commandline and on GitLab.
- [Importing to GitLab](workflow/importing/README.md).
@@ -13,22 +13,26 @@
- [Profile Settings](profile/README.md)
- [Project Services](project_services/project_services.md) Integrate a project with external services, such as CI and chat.
- [Public access](public_access/public_access.md) Learn how you can allow public and internal access to projects.
+- [Container Registry](container_registry/README.md) Learn how to use GitLab Container Registry.
- [SSH](ssh/README.md) Setup your ssh keys and deploy keys for secure access to your projects.
- [Webhooks](web_hooks/web_hooks.md) Let GitLab notify you when new code has been pushed to your project.
- [Workflow](workflow/README.md) Using GitLab functionality and importing projects from GitHub and SVN.
## Administrator documentation
+- [Authentication/Authorization](administration/auth/README.md) Configure
+ external authentication with LDAP, SAML, CAS and additional Omniauth providers.
- [Custom git hooks](hooks/custom_hooks.md) Custom git hooks (on the filesystem) for when webhooks aren't enough.
- [Install](install/README.md) Requirements, directory structures and installation from source.
- [Restart GitLab](administration/restart_gitlab.md) Learn how to restart GitLab and its components
-- [Integration](integration/README.md) How to integrate with systems such as JIRA, Redmine, LDAP and Twitter.
+- [Integration](integration/README.md) How to integrate with systems such as JIRA, Redmine, Twitter.
- [Issue closing](customization/issue_closing.md) Customize how to close an issue from commit messages.
- [Libravatar](customization/libravatar.md) Use Libravatar for user avatars.
-- [Log system](logs/logs.md) Log system.
+- [Log system](administration/logs.md) Log system.
- [Environment Variables](administration/environment_variables.md) to configure GitLab.
- [Operations](operations/README.md) Keeping GitLab up and running
- [Raketasks](raketasks/README.md) Backups, maintenance, automatic webhook setup and the importing of projects.
+- [Repository checks](administration/repository_checks.md) Periodic Git repository checks
- [Security](security/README.md) Learn what you can do to further secure your GitLab instance.
- [System hooks](system_hooks/system_hooks.md) Notifications when users, projects and keys are changed.
- [Update](update/README.md) Update guides to upgrade your installation.
@@ -38,6 +42,10 @@
- [Git LFS configuration](workflow/lfs/lfs_administration.md)
- [Housekeeping](administration/housekeeping.md) Keep your Git repository tidy and fast.
- [GitLab Performance Monitoring](monitoring/performance/introduction.md) Configure GitLab and InfluxDB for measuring performance metrics
+- [Monitoring uptime](monitoring/health_check.md) Check the server status using the health check endpoint
+- [Sidekiq Troubleshooting](administration/troubleshooting/sidekiq.md) Debug when Sidekiq appears hung and is not processing jobs
+- [High Availability](administration/high_availability/README.md) Configure multiple servers for scaling or high availability
+- [Container Registry](administration/container_registry.md) Configure Docker Registry with GitLab
## Contributor documentation
@@ -45,4 +53,3 @@
contributing to documentation.
- [Development](development/README.md) Explains the architecture and the guidelines for shell commands.
- [Legal](legal/README.md) Contributor license agreements.
-- [Release](release/README.md) How to make the monthly and security releases.
diff --git a/doc/administration/auth/README.md b/doc/administration/auth/README.md
new file mode 100644
index 00000000000..07e548aaabe
--- /dev/null
+++ b/doc/administration/auth/README.md
@@ -0,0 +1,11 @@
+# Authentication and Authorization
+
+GitLab integrates with the following external authentication and authorization
+providers.
+
+- [LDAP](ldap.md) Includes Active Directory, Apple Open Directory, Open LDAP,
+ and 389 Server
+- [OmniAuth](../../integration/omniauth.md) Sign in via Twitter, GitHub, GitLab.com, Google,
+ Bitbucket, Facebook, Shibboleth, Crowd and Azure
+- [SAML](../../integration/saml.md) Configure GitLab as a SAML 2.0 Service Provider
+- [CAS](../../integration/cas.md) Configure GitLab to sign in using CAS
diff --git a/doc/administration/auth/ldap.md b/doc/administration/auth/ldap.md
new file mode 100644
index 00000000000..10096779844
--- /dev/null
+++ b/doc/administration/auth/ldap.md
@@ -0,0 +1,277 @@
+# LDAP
+
+GitLab integrates with LDAP to support user authentication.
+This integration works with most LDAP-compliant directory
+servers, including Microsoft Active Directory, Apple Open Directory, Open LDAP,
+and 389 Server. GitLab EE includes enhanced integration, including group
+membership syncing.
+
+## Security
+
+GitLab assumes that LDAP users are not able to change their LDAP 'mail', 'email'
+or 'userPrincipalName' attribute. An LDAP user who is allowed to change their
+email on the LDAP server can potentially
+[take over any account](#enabling-ldap-sign-in-for-existing-gitlab-users)
+on your GitLab server.
+
+We recommend against using LDAP integration if your LDAP users are
+allowed to change their 'mail', 'email' or 'userPrincipalName' attribute on
+the LDAP server.
+
+### User deletion
+
+If a user is deleted from the LDAP server, they will be blocked in GitLab, as
+well. Users will be immediately blocked from logging in. However, there is an
+LDAP check cache time (sync time) of one hour (see note). This means users that
+are already logged in or are using Git over SSH will still be able to access
+GitLab for up to one hour. Manually block the user in the GitLab Admin area to
+immediately block all access.
+
+>**Note**: GitLab EE supports a configurable sync time, with a default
+of one hour.
+
+## Configuration
+
+To enable LDAP integration you need to add your LDAP server settings in
+`/etc/gitlab/gitlab.rb` or `/home/git/gitlab/config/gitlab.yml`.
+
+>**Note**: In GitLab EE, you can configure multiple LDAP servers to connect to
+one GitLab server.
+
+Prior to version 7.4, GitLab used a different syntax for configuring
+LDAP integration. The old LDAP integration syntax still works but may be
+removed in a future version. If your `gitlab.rb` or `gitlab.yml` file contains
+LDAP settings in both the old syntax and the new syntax, only the __old__
+syntax will be used by GitLab.
+
+The configuration inside `gitlab_rails['ldap_servers']` below is sensitive to
+incorrect indentation. Be sure to retain the indentation given in the example.
+Copy/paste can sometimes cause problems.
+
+**Omnibus configuration**
+
+```ruby
+gitlab_rails['ldap_enabled'] = true
+gitlab_rails['ldap_servers'] = YAML.load <<-EOS # remember to close this block with 'EOS' below
+main: # 'main' is the GitLab 'provider ID' of this LDAP server
+ ## label
+ #
+ # A human-friendly name for your LDAP server. It is OK to change the label later,
+ # for instance if you find out it is too large to fit on the web page.
+ #
+ # Example: 'Paris' or 'Acme, Ltd.'
+ label: 'LDAP'
+
+ host: '_your_ldap_server'
+ port: 389
+ uid: 'sAMAccountName'
+ method: 'plain' # "tls" or "ssl" or "plain"
+ bind_dn: '_the_full_dn_of_the_user_you_will_bind_with'
+ password: '_the_password_of_the_bind_user'
+
+ # Set a timeout, in seconds, for LDAP queries. This helps avoid blocking
+ # a request if the LDAP server becomes unresponsive.
+ # A value of 0 means there is no timeout.
+ timeout: 10
+
+ # This setting specifies if LDAP server is Active Directory LDAP server.
+ # For non AD servers it skips the AD specific queries.
+ # If your LDAP server is not AD, set this to false.
+ active_directory: true
+
+ # If allow_username_or_email_login is enabled, GitLab will ignore everything
+ # after the first '@' in the LDAP username submitted by the user on login.
+ #
+ # Example:
+ # - the user enters 'jane.doe@example.com' and 'p@ssw0rd' as LDAP credentials;
+ # - GitLab queries the LDAP server with 'jane.doe' and 'p@ssw0rd'.
+ #
+ # If you are using "uid: 'userPrincipalName'" on ActiveDirectory you need to
+ # disable this setting, because the userPrincipalName contains an '@'.
+ allow_username_or_email_login: false
+
+ # To maintain tight control over the number of active users on your GitLab installation,
+ # enable this setting to keep new users blocked until they have been cleared by the admin
+ # (default: false).
+ block_auto_created_users: false
+
+ # Base where we can search for users
+ #
+ # Ex. ou=People,dc=gitlab,dc=example
+ #
+ base: ''
+
+ # Filter LDAP users
+ #
+ # Format: RFC 4515 https://tools.ietf.org/search/rfc4515
+ # Ex. (employeeType=developer)
+ #
+ # Note: GitLab does not support omniauth-ldap's custom filter syntax.
+ #
+ user_filter: ''
+
+ # LDAP attributes that GitLab will use to create an account for the LDAP user.
+ # The specified attribute can either be the attribute name as a string (e.g. 'mail'),
+ # or an array of attribute names to try in order (e.g. ['mail', 'email']).
+ # Note that the user's LDAP login will always be the attribute specified as `uid` above.
+ attributes:
+ # The username will be used in paths for the user's own projects
+ # (like `gitlab.example.com/username/project`) and when mentioning
+ # them in issues, merge request and comments (like `@username`).
+ # If the attribute specified for `username` contains an email address,
+ # the GitLab username will be the part of the email address before the '@'.
+ username: ['uid', 'userid', 'sAMAccountName']
+ email: ['mail', 'email', 'userPrincipalName']
+
+ # If no full name could be found at the attribute specified for `name`,
+ # the full name is determined using the attributes specified for
+ # `first_name` and `last_name`.
+ name: 'cn'
+ first_name: 'givenName'
+ last_name: 'sn'
+
+ ## EE only
+
+ # Base where we can search for groups
+ #
+ # Ex. ou=groups,dc=gitlab,dc=example
+ #
+ group_base: ''
+
+ # The CN of a group containing GitLab administrators
+ #
+ # Ex. administrators
+ #
+ # Note: Not `cn=administrators` or the full DN
+ #
+ admin_group: ''
+
+ # The LDAP attribute containing a user's public SSH key
+ #
+ # Ex. ssh_public_key
+ #
+ sync_ssh_keys: false
+
+# GitLab EE only: add more LDAP servers
+# Choose an ID made of a-z and 0-9 . This ID will be stored in the database
+# so that GitLab can remember which LDAP server a user belongs to.
+# uswest2:
+# label:
+# host:
+# ....
+EOS
+```
+
+**Source configuration**
+
+Use the same format as `gitlab_rails['ldap_servers']` for the contents under
+`servers:` in the example below:
+
+```
+production:
+ # snip...
+ ldap:
+ enabled: false
+ servers:
+ main: # 'main' is the GitLab 'provider ID' of this LDAP server
+ ## label
+ #
+ # A human-friendly name for your LDAP server. It is OK to change the label later,
+ # for instance if you find out it is too large to fit on the web page.
+ #
+ # Example: 'Paris' or 'Acme, Ltd.'
+ label: 'LDAP'
+ # snip...
+```
+
+## Using an LDAP filter to limit access to your GitLab server
+
+If you want to limit all GitLab access to a subset of the LDAP users on your
+LDAP server, the first step should be to narrow the configured `base`. However,
+it is sometimes necessary to filter users further. In this case, you can set up
+an LDAP user filter. The filter must comply with
+[RFC 4515](https://tools.ietf.org/search/rfc4515).
+
+**Omnibus configuration**
+
+```ruby
+gitlab_rails['ldap_servers'] = YAML.load <<-EOS
+main:
+ # snip...
+ user_filter: '(employeeType=developer)'
+EOS
+```
+
+**Source configuration**
+
+```yaml
+production:
+ ldap:
+ servers:
+ main:
+ # snip...
+ user_filter: '(employeeType=developer)'
+```
+
+Tip: If you want to limit access to the nested members of an Active Directory
+group you can use the following syntax:
+
+```
+(memberOf:1.2.840.113556.1.4.1941:=CN=My Group,DC=Example,DC=com)
+```
+
+Please note that GitLab does not support the custom filter syntax used by
+omniauth-ldap.
+
+## Enabling LDAP sign-in for existing GitLab users
+
+When a user signs in to GitLab with LDAP for the first time, and their LDAP
+email address is the primary email address of an existing GitLab user, then
+the LDAP DN will be associated with the existing user. If the LDAP email
+attribute is not found in GitLab's database, a new user is created.
+
+In other words, if an existing GitLab user wants to enable LDAP sign-in for
+themselves, they should check that their GitLab email address matches their
+LDAP email address, and then sign into GitLab via their LDAP credentials.
+
+## Limitations
+
+### TLS Client Authentication
+
+Not implemented by `Net::LDAP`.
+You should disable anonymous LDAP authentication and enable simple or SASL
+authentication. The TLS client authentication setting in your LDAP server cannot
+be mandatory and clients cannot be authenticated with the TLS protocol.
+
+### TLS Server Authentication
+
+Not supported by GitLab's configuration options.
+When setting `method: ssl`, the underlying authentication method used by
+`omniauth-ldap` is `simple_tls`. This method establishes TLS encryption with
+the LDAP server before any LDAP-protocol data is exchanged but no validation of
+the LDAP server's SSL certificate is performed.
+
+## Troubleshooting
+
+### Invalid credentials when logging in
+
+- Make sure the user you are binding with has enough permissions to read the user's
+tree and traverse it.
+- Check that the `user_filter` is not blocking otherwise valid users.
+- Run the following check command to make sure that the LDAP settings are
+ correct and GitLab can see your users:
+
+ ```bash
+ # For Omnibus installations
+ sudo gitlab-rake gitlab:ldap:check
+
+ # For installations from source
+ sudo -u git -H bundle exec rake gitlab:ldap:check RAILS_ENV=production
+ ```
+
+### Connection Refused
+
+If you are getting 'Connection Refused' errors when trying to connect to the
+LDAP server please double-check the LDAP `port` and `method` settings used by
+GitLab. Common combinations are `method: 'plain'` and `port: 389`, OR
+`method: 'ssl'` and `port: 636`.
diff --git a/doc/administration/container_registry.md b/doc/administration/container_registry.md
new file mode 100644
index 00000000000..7870669fa77
--- /dev/null
+++ b/doc/administration/container_registry.md
@@ -0,0 +1,375 @@
+# GitLab Container Registry Administration
+
+> **Note:**
+This feature was [introduced][ce-4040] in GitLab 8.8.
+
+With the Docker Container Registry integrated into GitLab, every project can
+have its own space to store its Docker images.
+
+You can read more about Docker Registry at https://docs.docker.com/registry/introduction/.
+
+---
+
+<!-- START doctoc generated TOC please keep comment here to allow auto update -->
+<!-- DON'T EDIT THIS SECTION, INSTEAD RE-RUN doctoc TO UPDATE -->
+**Table of Contents** *generated with [DocToc](https://github.com/thlorenz/doctoc)*
+
+- [Enable the Container Registry](#enable-the-container-registry)
+- [Container Registry domain configuration](#container-registry-domain-configuration)
+ - [Configure Container Registry under an existing GitLab domain](#configure-container-registry-under-an-existing-gitlab-domain)
+ - [Configure Container Registry under its own domain](#configure-container-registry-under-its-own-domain)
+- [Disable Container Registry site-wide](#disable-container-registry-site-wide)
+- [Disable Container Registry per project](#disable-container-registry-per-project)
+- [Disable Container Registry for new projects site-wide](#disable-container-registry-for-new-projects-site-wide)
+- [Container Registry storage path](#container-registry-storage-path)
+- [Storage limitations](#storage-limitations)
+- [Changelog](#changelog)
+
+<!-- END doctoc generated TOC please keep comment here to allow auto update -->
+
+## Enable the Container Registry
+
+**Omnibus GitLab installations**
+
+All you have to do is configure the domain name under which the Container
+Registry will listen to. Read [#container-registry-domain-configuration](#container-registry-domain-configuration)
+and pick one of the two options that fits your case.
+
+>**Note:**
+The container Registry works under HTTPS by default. Using HTTP is possible
+but not recommended and out of the scope of this document.
+Read the [insecure Registry documentation][docker-insecure] if you want to
+implement this.
+
+---
+
+**Installations from source**
+
+If you have installed GitLab from source:
+
+1. You will have to [install Docker Registry][registry-deploy] by yourself.
+1. After the installation is complete, you will have to configure the Registry's
+ settings in `gitlab.yml` in order to enable it.
+1. Use the sample NGINX configuration file that is found under
+ [`lib/support/nginx/registry-ssl`][registry-ssl] and edit it to match the
+ `host`, `port` and TLS certs paths.
+
+The contents of `gitlab.yml` are:
+
+```
+registry:
+ enabled: true
+ host: registry.gitlab.example.com
+ port: 5005
+ api_url: http://localhost:5000/
+ key: config/registry.key
+ path: shared/registry
+ issuer: gitlab-issuer
+```
+
+where:
+
+| Parameter | Description |
+| --------- | ----------- |
+| `enabled` | `true` or `false`. Enables the Registry in GitLab. By default this is `false`. |
+| `host` | The host URL under which the Registry will run and the users will be able to use. |
+| `port` | The port under which the external Registry domain will listen on. |
+| `api_url` | The internal API URL under which the Registry is exposed to. It defaults to `http://localhost:5000`. |
+| `key` | The private key location that is a pair of Registry's `rootcertbundle`. Read the [token auth configuration documentation][token-config]. |
+| `path` | This should be the same directory like specified in Registry's `rootdirectory`. Read the [storage configuration documentation][storage-config]. This path needs to be readable by the GitLab user, the web-server user and the Registry user. Read more in [#container-registry-storage-path](#container-registry-storage-path). |
+| `issuer` | This should be the same value as configured in Registry's `issuer`. Read the [token auth configuration documentation][token-config]. |
+
+>**Note:**
+GitLab does not ship with a Registry init file. Hence, [restarting GitLab][restart gitlab]
+will not restart the Registry should you modify its settings. Read the upstream
+documentation on how to achieve that.
+
+## Container Registry domain configuration
+
+There are two ways you can configure the Registry's external domain.
+
+- Either [use the existing GitLab domain][existing-domain] where in that case
+ the Registry will have to listen on a port and reuse GitLab's TLS certificate,
+- or [use a completely separate domain][new-domain] with a new TLS certificate
+ for that domain.
+
+Since the container Registry requires a TLS certificate, in the end it all boils
+down to how easy or pricey is to get a new one.
+
+Please take this into consideration before configuring the Container Registry
+for the first time.
+
+### Configure Container Registry under an existing GitLab domain
+
+If the Registry is configured to use the existing GitLab domain, you can
+expose the Registry on a port so that you can reuse the existing GitLab TLS
+certificate.
+
+Assuming that the GitLab domain is `https://gitlab.example.com` and the port the
+Registry is exposed to the outside world is `4567`, here is what you need to set
+in `gitlab.rb` or `gitlab.yml` if you are using Omnibus GitLab or installed
+GitLab from source respectively.
+
+---
+
+**Omnibus GitLab installations**
+
+1. Your `/etc/gitlab/gitlab.rb` should contain the Registry URL as well as the
+ path to the existing TLS certificate and key used by GitLab:
+
+ ```ruby
+ registry_external_url 'https://gitlab.example.com:4567'
+ ```
+
+ Note how the `registry_external_url` is listening on HTTPS under the
+ existing GitLab URL, but on a different port.
+
+ If your TLS certificate is not in `/etc/gitlab/ssl/gitlab.example.com.crt`
+ and key not in `/etc/gitlab/ssl/gitlab.example.com.key` uncomment the lines
+ below:
+
+ ```ruby
+ registry_nginx['ssl_certificate'] = "/path/to/certificate.pem"
+ registry_nginx['ssl_certificate_key'] = "/path/to/certificate.key"
+ ```
+
+1. Save the file and [reconfigure GitLab][] for the changes to take effect.
+
+---
+
+**Installations from source**
+
+1. Open `/home/git/gitlab/config/gitlab.yml`, find the `registry` entry and
+ configure it with the following settings:
+
+ ```
+ registry:
+ enabled: true
+ host: gitlab.example.com
+ port: 4567
+ ```
+
+1. Save the file and [restart GitLab][] for the changes to take effect.
+1. Make the relevant changes in NGINX as well (domain, port, TLS certificates path).
+
+---
+
+Users should now be able to login to the Container Registry with their GitLab
+credentials using:
+
+```bash
+docker login gitlab.example.com:4567
+```
+
+### Configure Container Registry under its own domain
+
+If the Registry is configured to use its own domain, you will need a TLS
+certificate for that specific domain (e.g., `registry.example.com`) or maybe
+a wildcard certificate if hosted under a subdomain of your existing GitLab
+domain (e.g., `registry.gitlab.example.com`).
+
+Let's assume that you want the container Registry to be accessible at
+`https://registry.gitlab.example.com`.
+
+---
+
+**Omnibus GitLab installations**
+
+1. Place your TLS certificate and key in
+ `/etc/gitlab/ssl/registry.gitlab.example.com.crt` and
+ `/etc/gitlab/ssl/registry.gitlab.example.com.key` and make sure they have
+ correct permissions:
+
+ ```bash
+ chmod 600 /etc/gitlab/ssl/registry.gitlab.example.com.*
+ ```
+
+1. Once the TLS certificate is in place, edit `/etc/gitlab/gitlab.rb` with:
+
+ ```ruby
+ registry_external_url 'https://registry.gitlab.example.com'
+ ```
+
+ Note how the `registry_external_url` is listening on HTTPS.
+
+1. Save the file and [reconfigure GitLab][] for the changes to take effect.
+
+> **Note:**
+If you have a [wildcard certificate][], you need to specify the path to the
+certificate in addition to the URL, in this case `/etc/gitlab/gitlab.rb` will
+look like:
+>
+```ruby
+registry_nginx['ssl_certificate'] = "/etc/gitlab/ssl/certificate.pem"
+registry_nginx['ssl_certificate_key'] = "/etc/gitlab/ssl/certificate.key"
+```
+
+---
+
+**Installations from source**
+
+1. Open `/home/git/gitlab/config/gitlab.yml`, find the `registry` entry and
+ configure it with the following settings:
+
+ ```
+ registry:
+ enabled: true
+ host: registry.gitlab.example.com
+ ```
+
+1. Save the file and [restart GitLab][] for the changes to take effect.
+1. Make the relevant changes in NGINX as well (domain, port, TLS certificates path).
+
+---
+
+Users should now be able to login to the Container Registry using their GitLab
+credentials:
+
+```bash
+docker login registry.gitlab.example.com
+```
+
+## Disable Container Registry site-wide
+
+>**Note:**
+Disabling the Registry in the Rails GitLab application as set by the following
+steps, will not remove any existing Docker images. This is handled by the
+Registry application itself.
+
+**Omnibus GitLab**
+
+1. Open `/etc/gitlab/gitlab.rb` and set `registry['enable']` to `false`:
+
+ ```ruby
+ registry['enable'] = false
+ ```
+
+1. Save the file and [reconfigure GitLab][] for the changes to take effect.
+
+---
+
+**Installations from source**
+
+1. Open `/home/git/gitlab/config/gitlab.yml`, find the `registry` entry and
+ set `enabled` to `false`:
+
+ ```
+ registry:
+ enabled: false
+ ```
+
+1. Save the file and [restart GitLab][] for the changes to take effect.
+
+## Disable Container Registry per project
+
+If Registry is enabled in your GitLab instance, but you don't need it for your
+project, you can disable it from your project's settings. Read the user guide
+on how to achieve that.
+
+## Disable Container Registry for new projects site-wide
+
+If the Container Registry is enabled, then it will be available on all new
+projects. To disable this function and let the owners of a project to enable
+the Container Registry by themselves, follow the steps below.
+
+---
+
+**Omnibus GitLab installations**
+
+1. Edit `/etc/gitlab/gitlab.rb` and add the following line:
+
+ ```ruby
+ gitlab_rails['gitlab_default_projects_features_container_registry'] = false
+ ```
+
+1. Save the file and [reconfigure GitLab][] for the changes to take effect.
+
+---
+
+**Installations from source**
+
+1. Open `/home/git/gitlab/config/gitlab.yml`, find the `default_projects_features`
+ entry and configure it so that `container_registry` is set to `false`:
+
+ ```
+ ## Default project features settings
+ default_projects_features:
+ issues: true
+ merge_requests: true
+ wiki: true
+ snippets: false
+ builds: true
+ container_registry: false
+ ```
+
+1. Save the file and [restart GitLab][] for the changes to take effect.
+
+## Container Registry storage path
+
+To change the storage path where Docker images will be stored, follow the
+steps below.
+
+This path is accessible to:
+
+- the user running the Container Registry daemon,
+- the user running GitLab
+
+> **Warning** You should confirm that all GitLab, Registry and web server users
+have access to this directory.
+
+---
+
+**Omnibus GitLab installations**
+
+The default location where images are stored in Omnibus, is
+`/var/opt/gitlab/gitlab-rails/shared/registry`. To change it:
+
+1. Edit `/etc/gitlab/gitlab.rb`:
+
+ ```ruby
+ gitlab_rails['registry_path'] = "/path/to/registry/storage"
+ ```
+
+1. Save the file and [reconfigure GitLab][] for the changes to take effect.
+
+---
+
+**Installations from source**
+
+The default location where images are stored in source installations, is
+`/home/git/gitlab/shared/registry`. To change it:
+
+1. Open `/home/git/gitlab/config/gitlab.yml`, find the `registry` entry and
+ change the `path` setting:
+
+ ```
+ registry:
+ path: shared/registry
+ ```
+
+1. Save the file and [restart GitLab][] for the changes to take effect.
+
+## Storage limitations
+
+Currently, there is no storage limitation, which means a user can upload an
+infinite amount of Docker images with arbitrary sizes. This setting will be
+configurable in future releases.
+
+## Changelog
+
+**GitLab 8.8 ([source docs][8-8-docs])**
+
+- GitLab Container Registry feature was introduced.
+
+[reconfigure gitlab]: restart_gitlab.md#omnibus-gitlab-reconfigure
+[restart gitlab]: restart_gitlab.md#installations-from-source
+[wildcard certificate]: https://en.wikipedia.org/wiki/Wildcard_certificate
+[ce-4040]: https://gitlab.com/gitlab-org/gitlab-ce/merge_requests/4040
+[docker-insecure]: https://docs.docker.com/registry/insecure/
+[registry-deploy]: https://docs.docker.com/registry/deploying/
+[storage-config]: https://docs.docker.com/registry/configuration/#storage
+[token-config]: https://docs.docker.com/registry/configuration/#token
+[8-8-docs]: https://gitlab.com/gitlab-org/gitlab-ce/blob/8-8-stable/doc/administration/container_registry.md
+[registry-ssl]: https://gitlab.com/gitlab-org/gitlab-ce/blob/master/lib/support/nginx/registry-ssl
+[existing-domain]: #configure-container-registry-under-an-existing-gitlab-domain
+[new-domain]: #configure-container-registry-under-its-own-domain
diff --git a/doc/administration/environment_variables.md b/doc/administration/environment_variables.md
index 43ab153d76d..7f53915a4d7 100644
--- a/doc/administration/environment_variables.md
+++ b/doc/administration/environment_variables.md
@@ -58,4 +58,4 @@ to the naming scheme `GITLAB_#{name in 1_settings.rb in upper case}`.
It's possible to preconfigure the GitLab docker image by adding the environment
variable `GITLAB_OMNIBUS_CONFIG` to the `docker run` command.
-For more information see the ['preconfigure-docker-container' section in the Omnibus documentation](http://doc.gitlab.com/omnibus/docker/#preconfigure-docker-container).
+For more information see the ['preconfigure-docker-container' section in the Omnibus documentation](http://docs.gitlab.com/omnibus/docker/#preconfigure-docker-container).
diff --git a/doc/administration/high_availability/README.md b/doc/administration/high_availability/README.md
new file mode 100644
index 00000000000..d74a786ac24
--- /dev/null
+++ b/doc/administration/high_availability/README.md
@@ -0,0 +1,39 @@
+# High Availability
+
+GitLab supports several different types of clustering and high-availability.
+The solution you choose will be based on the level of scalability and
+availability you require. The easiest solutions are scalable, but not necessarily
+highly available.
+
+## Architecture
+
+### Active/Passive
+
+For pure high-availability/failover with no scaling you can use an
+active/passive configuration. This utilizes DRBD (Distributed Replicated
+Block Device) to keep all data in sync. DRBD requires a low latency link to
+remain in sync. It is not advisable to attempt to run DRBD between data centers
+or in different cloud availability zones.
+
+Components/Servers Required:
+
+- 2 servers/virtual machines (one active/one passive)
+
+![Active/Passive HA Diagram](../img/high_availability/active-passive-diagram.png)
+
+### Active/Active
+
+This architecture scales easily because all application servers handle
+user requests simultaneously. The database, Redis, and GitLab application are
+all deployed on separate servers. The configuration is **only** highly-available
+if the database, Redis and storage are also configured as such.
+
+![Active/Active HA Diagram](../img/high_availability/active-active-diagram.png)
+
+**Steps to configure active/active:**
+
+1. [Configure the database](database.md)
+1. [Configure Redis](redis.md)
+1. [Configure NFS](nfs.md)
+1. [Configure the GitLab application servers](gitlab.md)
+1. [Configure the load balancers](load_balancer.md)
diff --git a/doc/administration/high_availability/database.md b/doc/administration/high_availability/database.md
new file mode 100644
index 00000000000..538dada1bae
--- /dev/null
+++ b/doc/administration/high_availability/database.md
@@ -0,0 +1,116 @@
+# Configuring a Database for GitLab HA
+
+You can choose to install and manage a database server (PostgreSQL/MySQL)
+yourself, or you can use GitLab Omnibus packages to help. GitLab recommends
+PostgreSQL. This is the database that will be installed if you use the
+Omnibus package to manage your database.
+
+## Configure your own database server
+
+If you're hosting GitLab on a cloud provider, you can optionally use a
+managed service for PostgreSQL. For example, AWS offers a managed Relational
+Database Service (RDS) that runs PostgreSQL.
+
+If you use a cloud-managed service, or provide your own PostgreSQL:
+
+1. Set up a `gitlab` username with a password of your choice. The `gitlab` user
+ needs privileges to create the `gitlabhq_production` database.
+1. Configure the GitLab application servers with the appropriate details.
+ This step is covered in [Configuring GitLab for HA](gitlab.md)
+
+## Configure using Omnibus
+
+1. Download/install GitLab Omnibus using **steps 1 and 2** from
+ [GitLab downloads](https://about.gitlab.com/downloads). Do not complete other
+ steps on the download page.
+1. Create/edit `/etc/gitlab/gitlab.rb` and use the following configuration.
+ Be sure to change the `external_url` to match your eventual GitLab front-end
+ URL.
+
+ ```ruby
+ external_url 'https://gitlab.example.com'
+
+ # Disable all components except PostgreSQL
+ postgresql['enable'] = true
+ bootstrap['enable'] = false
+ nginx['enable'] = false
+ unicorn['enable'] = false
+ sidekiq['enable'] = false
+ redis['enable'] = false
+ gitlab_workhorse['enable'] = false
+ mailroom['enable'] = false
+
+ # PostgreSQL configuration
+ postgresql['sql_password'] = 'DB password'
+ postgresql['md5_auth_cidr_addresses'] = ['0.0.0.0/0']
+ postgresql['listen_address'] = '0.0.0.0'
+ ```
+
+1. Run `sudo gitlab-ctl reconfigure` to install and configure PostgreSQL.
+
+ > **Note**: This `reconfigure` step will result in some errors.
+ That's OK - don't be alarmed.
+
+1. Open a database prompt:
+
+ ```
+ su - gitlab-psql
+ /bin/bash
+ psql -h /var/opt/gitlab/postgresql -d template1
+
+ # Output:
+
+ psql (9.2.15)
+ Type "help" for help.
+
+ template1=#
+ ```
+
+1. Run the following command at the database prompt and you will be asked to
+ enter the new password for the PostgreSQL superuser.
+
+ ```
+ \password
+
+ # Output:
+
+ Enter new password:
+ Enter it again:
+ ```
+
+1. Similarly, set the password for the `gitlab` database user. Use the same
+ password that you specified in the `/etc/gitlab/gitlab.rb` file for
+ `postgresql['sql_password']`.
+
+ ```
+ \password gitlab
+
+ # Output:
+
+ Enter new password:
+ Enter it again:
+ ```
+
+1. Enable the `pg_trgm` extension:
+ ```
+ CREATE EXTENSION pg_trgm;
+
+ # Output:
+
+ CREATE EXTENSION
+ ```
+1. Exit the database prompt by typing `\q` and Enter.
+1. Exit the `gitlab-psql` user by running `exit` twice.
+1. Run `sudo gitlab-ctl reconfigure` a final time.
+1. Run `touch /etc/gitlab/skip-auto-migrations` to prevent database migrations
+ from running on upgrade. Only the primary GitLab application server should
+ handle migrations.
+
+---
+
+Read more on high-availability configuration:
+
+1. [Configure Redis](redis.md)
+1. [Configure NFS](nfs.md)
+1. [Configure the GitLab application servers](gitlab.md)
+1. [Configure the load balancers](load_balancer.md)
diff --git a/doc/administration/high_availability/gitlab.md b/doc/administration/high_availability/gitlab.md
new file mode 100644
index 00000000000..8a881ce8863
--- /dev/null
+++ b/doc/administration/high_availability/gitlab.md
@@ -0,0 +1,131 @@
+# Configuring GitLab for HA
+
+Assuming you have already configured a database, Redis, and NFS, you can
+configure the GitLab application server(s) now. Complete the steps below
+for each GitLab application server in your environment.
+
+> **Note:** There is some additional configuration near the bottom for
+ secondary GitLab application servers. It's important to read and understand
+ these additional steps before proceeding with GitLab installation.
+
+1. If necessary, install the NFS client utility packages using the following
+ commands:
+
+ ```
+ # Ubuntu/Debian
+ apt-get install nfs-common
+
+ # CentOS/Red Hat
+ yum install nfs-utils nfs-utils-lib
+ ```
+
+1. Specify the necessary NFS shares. Mounts are specified in
+ `/etc/fstab`. The exact contents of `/etc/fstab` will depend on how you chose
+ to configure your NFS server. See [NFS documentation](nfs.md) for the various
+ options. Here is an example snippet to add to `/etc/fstab`:
+
+ ```
+ 10.1.0.1:/var/opt/gitlab/.ssh /var/opt/gitlab/.ssh nfs defaults,soft,rsize=1048576,wsize=1048576,noatime,nobootwait,lookupcache=positive 0 2
+ 10.1.0.1:/var/opt/gitlab/gitlab-rails/uploads /var/opt/gitlab/gitlab-rails/uploads nfs defaults,soft,rsize=1048576,wsize=1048576,noatime,nobootwait,lookupcache=positive 0 2
+ 10.1.0.1:/var/opt/gitlab/gitlab-rails/shared /var/opt/gitlab/gitlab-rails/shared nfs defaults,soft,rsize=1048576,wsize=1048576,noatime,nobootwait,lookupcache=positive 0 2
+ 10.1.0.1:/var/opt/gitlab/gitlab-ci/builds /var/opt/gitlab/gitlab-ci/builds nfs defaults,soft,rsize=1048576,wsize=1048576,noatime,nobootwait,lookupcache=positive 0 2
+ 10.1.1.1:/var/opt/gitlab/git-data /var/opt/gitlab/git-data nfs defaults,soft,rsize=1048576,wsize=1048576,noatime,nobootwait,lookupcache=positive 0 2
+ ```
+
+1. Create the shared directories. These may be different depending on your NFS
+ mount locations.
+
+ ```
+ mkdir -p /var/opt/gitlab/.ssh /var/opt/gitlab/gitlab-rails/uploads /var/opt/gitlab/gitlab-rails/shared /var/opt/gitlab/gitlab-ci/builds /var/opt/gitlab/git-data
+ ```
+
+1. Download/install GitLab Omnibus using **steps 1 and 2** from
+ [GitLab downloads](https://about.gitlab.com/downloads). Do not complete other
+ steps on the download page.
+1. Create/edit `/etc/gitlab/gitlab.rb` and use the following configuration.
+ Be sure to change the `external_url` to match your eventual GitLab front-end
+ URL. Depending your the NFS configuration, you may need to change some GitLab
+ data locations. See [NFS documentation](nfs.md) for `/etc/gitlab/gitlab.rb`
+ configuration values for various scenarios. The example below assumes you've
+ added NFS mounts in the default data locations.
+
+ ```ruby
+ external_url 'https://gitlab.example.com'
+
+ # Prevent GitLab from starting if NFS data mounts are not available
+ high_availability['mountpoint'] = '/var/opt/gitlab/git-data'
+
+ # Disable components that will not be on the GitLab application server
+ postgresql['enable'] = false
+ redis['enable'] = false
+
+ # PostgreSQL connection details
+ gitlab_rails['db_adapter'] = 'postgresql'
+ gitlab_rails['db_encoding'] = 'unicode'
+ gitlab_rails['db_host'] = '10.1.0.5' # IP/hostname of database server
+ gitlab_rails['db_password'] = 'DB password'
+
+ # Redis connection details
+ gitlab_rails['redis_port'] = '6379'
+ gitlab_rails['redis_host'] = '10.1.0.6' # IP/hostname of Redis server
+ gitlab_rails['redis_password'] = 'Redis Password'
+ ```
+
+1. Run `sudo gitlab-ctl reconfigure` to compile the configuration.
+
+## Primary GitLab application server
+
+As a final step, run the setup rake task on the first GitLab application server.
+It is not necessary to run this on additional application servers.
+
+1. Initialize the database by running `sudo gitlab-rake gitlab:setup`.
+
+> **WARNING:** Only run this setup task on **NEW** GitLab instances because it
+ will wipe any existing data.
+
+> **Note:** When you specify `https` in the `external_url`, as in the example
+ above, GitLab assumes you have SSL certificates in `/etc/gitlab/ssl/`. If
+ certificates are not present, Nginx will fail to start. See
+ [Nginx documentation](http://docs.gitlab.com/omnibus/settings/nginx.html#enable-https)
+ for more information.
+
+## Additional configuration for secondary GitLab application servers
+
+Secondary GitLab servers (servers configured **after** the first GitLab server)
+need some additional configuration.
+
+1. Configure shared secrets. These values can be obtained from the primary
+ GitLab server in `/etc/gitlab/gitlab-secrets.json`. Add these to
+ `/etc/gitlab/gitlab.rb` **prior to** running the first `reconfigure` in
+ the steps above.
+
+ ```ruby
+ gitlab_shell['secret_token'] = 'fbfb19c355066a9afb030992231c4a363357f77345edd0f2e772359e5be59b02538e1fa6cae8f93f7d23355341cea2b93600dab6d6c3edcdced558fc6d739860'
+ gitlab_rails['secret_token'] = 'b719fe119132c7810908bba18315259ed12888d4f5ee5430c42a776d840a396799b0a5ef0a801348c8a357f07aa72bbd58e25a84b8f247a25c72f539c7a6c5fa'
+ gitlab_ci['secret_key_base'] = '6e657410d57c71b4fc3ed0d694e7842b1895a8b401d812c17fe61caf95b48a6d703cb53c112bc01ebd197a85da81b18e29682040e99b4f26594772a4a2c98c6d'
+ gitlab_ci['db_key_base'] = 'bf2e47b68d6cafaef1d767e628b619365becf27571e10f196f98dc85e7771042b9203199d39aff91fcb6837c8ed83f2a912b278da50999bb11a2fbc0fba52964'
+ ```
+
+1. Run `touch /etc/gitlab/skip-auto-migrations` to prevent database migrations
+ from running on upgrade. Only the primary GitLab application server should
+ handle migrations.
+
+## Troubleshooting
+
+- `mount: wrong fs type, bad option, bad superblock on`
+
+You have not installed the necessary NFS client utilities. See step 1 above.
+
+- `mount: mount point /var/opt/gitlab/... does not exist`
+
+This particular directory does not exist on the NFS server. Ensure
+the share is exported and exists on the NFS server and try to remount.
+
+---
+
+Read more on high-availability configuration:
+
+1. [Configure the database](database.md)
+1. [Configure Redis](redis.md)
+1. [Configure NFS](nfs.md)
+1. [Configure the load balancers](load_balancer.md)
diff --git a/doc/administration/high_availability/load_balancer.md b/doc/administration/high_availability/load_balancer.md
new file mode 100644
index 00000000000..136f570ac27
--- /dev/null
+++ b/doc/administration/high_availability/load_balancer.md
@@ -0,0 +1,63 @@
+# Load Balancer for GitLab HA
+
+In an active/active GitLab configuration, you will need a load balancer to route
+traffic to the application servers. The specifics on which load balancer to use
+or the exact configuration is beyond the scope of GitLab documentation. We hope
+that if you're managing HA systems like GitLab you have a load balancer of
+choice already. Some examples including HAProxy (open-source), F5 Big-IP LTM,
+and Citrix Net Scaler. This documentation will outline what ports and protocols
+you need to use with GitLab.
+
+## Basic ports
+
+| LB Port | Backend Port | Protocol |
+| ------- | ------------ | -------- |
+| 80 | 80 | HTTP |
+| 443 | 443 | HTTPS [^1] |
+| 22 | 22 | TCP |
+
+## GitLab Pages Ports
+
+If you're using GitLab Pages you will need some additional port configurations.
+GitLab Pages requires a separate VIP. Configure DNS to point the
+`pages_external_url` from `/etc/gitlab/gitlab.rb` at the new VIP. See the
+[GitLab Pages documentation][gitlab-pages] for more information.
+
+| LB Port | Backend Port | Protocol |
+| ------- | ------------ | -------- |
+| 80 | Varies [^2] | HTTP |
+| 443 | Varies [^2] | TCP [^3] |
+
+## Alternate SSH Port
+
+Some organizations have policies against opening SSH port 22. In this case,
+it may be helpful to configure an alternate SSH hostname that allows users
+to use SSH on port 443. An alternate SSH hostname will require a new VIP
+compared to the other GitLab HTTP configuration above.
+
+Configure DNS for an alternate SSH hostname such as altssh.gitlab.example.com.
+
+| LB Port | Backend Port | Protocol |
+| ------- | ------------ | -------- |
+| 443 | 22 | TCP |
+
+---
+
+Read more on high-availability configuration:
+
+1. [Configure the database](database.md)
+1. [Configure Redis](redis.md)
+1. [Configure NFS](nfs.md)
+1. [Configure the GitLab application servers](gitlab.md)
+
+[^1]: When using HTTPS protocol for port 443, you will need to add an SSL
+ certificate to the load balancers. If you wish to terminate SSL at the
+ GitLab application server instead, use TCP protocol.
+[^2]: The backend port for GitLab Pages depends on the
+ `gitlab_pages['external_http']` and `gitlab_pages['external_https']`
+ setting. See [GitLab Pages documentation][gitlab-pages] for more details.
+[^3]: Port 443 for GitLab Pages should always use the TCP protocol. Users can
+ configure custom domains with custom SSL, which would not be possible
+ if SSL was terminated at the load balancer.
+
+[gitlab-pages]: http://docs.gitlab.com/ee/pages/administration.html
diff --git a/doc/administration/high_availability/nfs.md b/doc/administration/high_availability/nfs.md
new file mode 100644
index 00000000000..537f4f3501d
--- /dev/null
+++ b/doc/administration/high_availability/nfs.md
@@ -0,0 +1,116 @@
+# NFS
+
+## Required NFS Server features
+
+**File locking**: GitLab **requires** advisory file locking, which is only
+supported natively in NFS version 4. NFSv3 also supports locking as long as
+Linux Kernel 2.6.5+ is used. We recommend using version 4 and do not
+specifically test NFSv3.
+
+**no_root_squash**: NFS normally changes the `root` user to `nobody`. This is
+a good security measure when NFS shares will be accessed by many different
+users. However, in this case only GitLab will use the NFS share so it
+is safe. GitLab requires the `no_root_squash` setting because we need to
+manage file permissions automatically. Without the setting you will receive
+errors when the Omnibus package tries to alter permissions. Note that GitLab
+and other bundled components do **not** run as `root` but as non-privileged
+users. The requirement for `no_root_squash` is to allow the Omnibus package to
+set ownership and permissions on files, as needed.
+
+### Recommended options
+
+When you define your NFS exports, we recommend you also add the following
+options:
+
+- `sync` - Force synchronous behavior. Default is asynchronous and under certain
+ circumstances it could lead to data loss if a failure occurs before data has
+ synced.
+
+## Client mount options
+
+Below is an example of an NFS mount point we use on GitLab.com:
+
+```
+10.1.1.1:/var/opt/gitlab/git-data /var/opt/gitlab/git-data nfs4 defaults,soft,rsize=1048576,wsize=1048576,noatime,nobootwait,lookupcache=positive 0 2
+```
+
+Notice several options that you should consider using:
+
+| Setting | Description |
+| ------- | ----------- |
+| `nobootwait` | Don't halt boot process waiting for this mount to become available
+| `lookupcache=positive` | Tells the NFS client to honor `positive` cache results but invalidates any `negative` cache results. Negative cache results cause problems with Git. Specifically, a `git push` can fail to register uniformly across all NFS clients. The negative cache causes the clients to 'remember' that the files did not exist previously.
+
+## Mount locations
+
+When using default Omnibus configuration you will need to share 5 data locations
+between all GitLab cluster nodes. No other locations should be shared. The
+following are the 5 locations you need to mount:
+
+| Location | Description |
+| -------- | ----------- |
+| `/var/opt/gitlab/git-data` | Git repository data. This will account for a large portion of your data
+| `/var/opt/gitlab/.ssh` | SSH `authorized_keys` file and keys used to import repositories from some other Git services
+| `/var/opt/gitlab/gitlab-rails/uploads` | User uploaded attachments
+| `/var/opt/gitlab/gitlab-rails/shared` | Build artifacts, GitLab Pages, LFS objects, temp files, etc. If you're using LFS this may also account for a large portion of your data
+| `/var/opt/gitlab/gitlab-ci/builds` | GitLab CI build traces
+
+Other GitLab directories should not be shared between nodes. They contain
+node-specific files and GitLab code that does not need to be shared. To ship
+logs to a central location consider using remote syslog. GitLab Omnibus packages
+provide configuration for [UDP log shipping][udp-log-shipping].
+
+### Consolidating mount points
+
+If you don't want to configure 5-6 different NFS mount points, you have a few
+alternative options.
+
+#### Change default file locations
+
+Omnibus allows you to configure the file locations. With custom configuration
+you can specify just one main mountpoint and have all of these locations
+as subdirectories. Mount `/gitlab-data` then use the following Omnibus
+configuration to move each data location to a subdirectory:
+
+```ruby
+user['home'] = '/gitlab-data/home'
+git_data_dir '/gitlab-data/git-data'
+gitlab_rails['shared_path'] = '/gitlab-data/shared'
+gitlab_rails['uploads_directory'] = "/gitlab-data/uploads"
+gitlab_ci['builds_directory'] = '/gitlab-data/builds'
+```
+
+To move the `git` home directory, all GitLab services must be stopped. Run
+`gitlab-ctl stop && initctl stop gitlab-runsvdir`. Then continue with the
+reconfigure.
+
+Run `sudo gitlab-ctl reconfigure` to start using the central location. Please
+be aware that if you had existing data you will need to manually copy/rsync it
+to these new locations and then restart GitLab.
+
+#### Bind mounts
+
+Bind mounts provide a way to specify just one NFS mount and then
+bind the default GitLab data locations to the NFS mount. Start by defining your
+single NFS mount point as you normally would in `/etc/fstab`. Let's assume your
+NFS mount point is `/gitlab-data`. Then, add the following bind mounts in
+`/etc/fstab`:
+
+```bash
+/gitlab-data/git-data /var/opt/gitlab/git-data none bind 0 0
+/gitlab-data/.ssh /var/opt/gitlab/.ssh none bind 0 0
+/gitlab-data/uploads /var/opt/gitlab/gitlab-rails/uploads none bind 0 0
+/gitlab-data/shared /var/opt/gitlab/gitlab-rails/shared none bind 0 0
+/gitlab-data/builds /var/opt/gitlab/gitlab-ci/builds none bind 0 0
+```
+
+---
+
+Read more on high-availability configuration:
+
+1. [Configure the database](database.md)
+1. [Configure Redis](redis.md)
+1. [Configure the GitLab application servers](gitlab.md)
+1. [Configure the load balancers](load_balancer.md)
+
+[udp-log-shipping]: http://docs.gitlab.com/omnibus/settings/logs.html#udp-log-shipping-gitlab-enterprise-edition-only "UDP log shipping"
diff --git a/doc/administration/high_availability/redis.md b/doc/administration/high_availability/redis.md
new file mode 100644
index 00000000000..f6153216f33
--- /dev/null
+++ b/doc/administration/high_availability/redis.md
@@ -0,0 +1,62 @@
+# Configuring Redis for GitLab HA
+
+You can choose to install and manage Redis yourself, or you can use GitLab
+Omnibus packages to help.
+
+## Configure your own Redis server
+
+If you're hosting GitLab on a cloud provider, you can optionally use a
+managed service for Redis. For example, AWS offers a managed ElastiCache service
+that runs Redis.
+
+> **Note:** Redis does not require authentication by default. See
+ [Redis Security](http://redis.io/topics/security) documentation for more
+ information. We recommend using a combination of a Redis password and tight
+ firewall rules to secure your Redis service.
+
+## Configure using Omnibus
+
+1. Download/install GitLab Omnibus using **steps 1 and 2** from
+ [GitLab downloads](https://about.gitlab.com/downloads). Do not complete other
+ steps on the download page.
+1. Create/edit `/etc/gitlab/gitlab.rb` and use the following configuration.
+ Be sure to change the `external_url` to match your eventual GitLab front-end
+ URL.
+
+ ```ruby
+ external_url 'https://gitlab.example.com'
+
+ # Disable all components except Redis
+ redis['enable'] = true
+ bootstrap['enable'] = false
+ nginx['enable'] = false
+ unicorn['enable'] = false
+ sidekiq['enable'] = false
+ postgresql['enable'] = false
+ gitlab_workhorse['enable'] = false
+ mailroom['enable'] = false
+
+ # Redis configuration
+ redis['port'] = 6379
+ redis['bind'] = '0.0.0.0'
+
+ # If you wish to use Redis authentication (recommended)
+ redis['password'] = 'Redis Password'
+ ```
+
+1. Run `sudo gitlab-ctl reconfigure` to install and configure PostgreSQL.
+
+ > **Note**: This `reconfigure` step will result in some errors.
+ That's OK - don't be alarmed.
+1. Run `touch /etc/gitlab/skip-auto-migrations` to prevent database migrations
+ from running on upgrade. Only the primary GitLab application server should
+ handle migrations.
+
+---
+
+Read more on high-availability configuration:
+
+1. [Configure the database](database.md)
+1. [Configure NFS](nfs.md)
+1. [Configure the GitLab application servers](gitlab.md)
+1. [Configure the load balancers](load_balancer.md)
diff --git a/doc/administration/img/high_availability/active-active-diagram.png b/doc/administration/img/high_availability/active-active-diagram.png
new file mode 100644
index 00000000000..81259e0ae93
--- /dev/null
+++ b/doc/administration/img/high_availability/active-active-diagram.png
Binary files differ
diff --git a/doc/administration/img/high_availability/active-passive-diagram.png b/doc/administration/img/high_availability/active-passive-diagram.png
new file mode 100644
index 00000000000..f69ff1d0357
--- /dev/null
+++ b/doc/administration/img/high_availability/active-passive-diagram.png
Binary files differ
diff --git a/doc/administration/logs.md b/doc/administration/logs.md
new file mode 100644
index 00000000000..737b39db16c
--- /dev/null
+++ b/doc/administration/logs.md
@@ -0,0 +1,137 @@
+## Log system
+
+GitLab has an advanced log system where everything is logged so that you
+can analyze your instance using various system log files. In addition to
+system log files, GitLab Enterprise Edition comes with Audit Events.
+Find more about them [in Audit Events
+documentation](http://docs.gitlab.com/ee/administration/audit_events.html)
+
+System log files are typically plain text in a standard log file format.
+This guide talks about how to read and use these system log files.
+
+### production.log
+
+This file lives in `/var/log/gitlab/gitlab-rails/production.log` for
+omnibus package or in `/home/git/gitlab/log/production.log` for
+installations from source.
+
+It contains information about all performed requests. You can see the
+URL and type of request, IP address and what exactly parts of code were
+involved to service this particular request. Also you can see all SQL
+request that have been performed and how much time it took. This task is
+more useful for GitLab contributors and developers. Use part of this log
+file when you are going to report bug. For example:
+
+```
+Started GET "/gitlabhq/yaml_db/tree/master" for 168.111.56.1 at 2015-02-12 19:34:53 +0200
+Processing by Projects::TreeController#show as HTML
+ Parameters: {"project_id"=>"gitlabhq/yaml_db", "id"=>"master"}
+
+ ... [CUT OUT]
+
+ Namespaces"."created_at" DESC, "namespaces"."id" DESC LIMIT 1 [["id", 26]]
+ CACHE (0.0ms) SELECT "members".* FROM "members" WHERE "members"."source_type" = 'Project' AND "members"."type" IN ('ProjectMember') AND "members"."source_id" = $1 AND "members"."source_type" = $2 AND "members"."user_id" = 1 ORDER BY "members"."created_at" DESC, "members"."id" DESC LIMIT 1 [["source_id", 18], ["source_type", "Project"]]
+ CACHE (0.0ms) SELECT "members".* FROM "members" WHERE "members"."source_type" = 'Project' AND "members".
+ (1.4ms) SELECT COUNT(*) FROM "merge_requests" WHERE "merge_requests"."target_project_id" = $1 AND ("merge_requests"."state" IN ('opened','reopened')) [["target_project_id", 18]]
+ Rendered layouts/nav/_project.html.haml (28.0ms)
+ Rendered layouts/_collapse_button.html.haml (0.2ms)
+ Rendered layouts/_flash.html.haml (0.1ms)
+ Rendered layouts/_page.html.haml (32.9ms)
+Completed 200 OK in 166ms (Views: 117.4ms | ActiveRecord: 27.2ms)
+```
+
+In this example we can see that server processed an HTTP request with URL
+`/gitlabhq/yaml_db/tree/master` from IP 168.111.56.1 at 2015-02-12
+19:34:53 +0200. Also we can see that request was processed by
+`Projects::TreeController`.
+
+### application.log
+
+This file lives in `/var/log/gitlab/gitlab-rails/application.log` for
+omnibus package or in `/home/git/gitlab/log/application.log` for
+installations from source.
+
+It helps you discover events happening in your instance such as user creation,
+project removing and so on. For example:
+
+```
+October 06, 2014 11:56: User "Administrator" (admin@example.com) was created
+October 06, 2014 11:56: Documentcloud created a new project "Documentcloud / Underscore"
+October 06, 2014 11:56: Gitlab Org created a new project "Gitlab Org / Gitlab Ce"
+October 07, 2014 11:25: User "Claudie Hodkiewicz" (nasir_stehr@olson.co.uk) was removed
+October 07, 2014 11:25: Project "project133" was removed
+```
+
+### githost.log
+
+This file lives in `/var/log/gitlab/gitlab-rails/githost.log` for
+omnibus package or in `/home/git/gitlab/log/githost.log` for
+installations from source.
+
+GitLab has to interact with Git repositories but in some rare cases
+something can go wrong and in this case you will know what exactly
+happened. This log file contains all failed requests from GitLab to Git
+repositories. In the majority of cases this file will be useful for developers
+only. For example:
+
+```
+December 03, 2014 13:20 -> ERROR -> Command failed [1]: /usr/bin/git --git-dir=/Users/vsizov/gitlab-development-kit/gitlab/tmp/tests/gitlab-satellites/group184/gitlabhq/.git --work-tree=/Users/vsizov/gitlab-development-kit/gitlab/tmp/tests/gitlab-satellites/group184/gitlabhq merge --no-ff -mMerge branch 'feature_conflict' into 'feature' source/feature_conflict
+
+error: failed to push some refs to '/Users/vsizov/gitlab-development-kit/repositories/gitlabhq/gitlab_git.git'
+```
+
+### sidekiq.log
+
+This file lives in `/var/log/gitlab/gitlab-rails/sidekiq.log` for
+omnibus package or in `/home/git/gitlab/log/sidekiq.log` for
+installations from source.
+
+GitLab uses background jobs for processing tasks which can take a long
+time. All information about processing these jobs are written down to
+this file. For example:
+
+```
+2014-06-10T07:55:20Z 2037 TID-tm504 ERROR: /opt/bitnami/apps/discourse/htdocs/vendor/bundle/ruby/1.9.1/gems/redis-3.0.7/lib/redis/client.rb:228:in `read'
+2014-06-10T18:18:26Z 14299 TID-55uqo INFO: Booting Sidekiq 3.0.0 with redis options {:url=>"redis://localhost:6379/0", :namespace=>"sidekiq"}
+```
+
+### gitlab-shell.log
+
+This file lives in `/var/log/gitlab/gitlab-shell/gitlab-shell.log` for
+omnibus package or in `/home/git/gitlab-shell/gitlab-shell.log` for
+installations from source.
+
+GitLab shell is used by Gitlab for executing Git commands and provide
+SSH access to Git repositories. For example:
+
+```
+I, [2015-02-13T06:17:00.671315 #9291] INFO -- : Adding project root/example.git at </var/opt/gitlab/git-data/repositories/root/dcdcdcdcd.git>.
+I, [2015-02-13T06:17:00.679433 #9291] INFO -- : Moving existing hooks directory and symlinking global hooks directory for /var/opt/gitlab/git-data/repositories/root/example.git.
+```
+
+### unicorn\_stderr.log
+
+This file lives in `/var/log/gitlab/unicorn/unicorn_stderr.log` for
+omnibus package or in `/home/git/gitlab/log/unicorn_stderr.log` for
+installations from source.
+
+Unicorn is a high-performance forking Web server which is used for
+serving the GitLab application. You can look at this log if, for
+example, your application does not respond. This log contains all
+information about the state of unicorn processes at any given time.
+
+```
+I, [2015-02-13T06:14:46.680381 #9047] INFO -- : Refreshing Gem list
+I, [2015-02-13T06:14:56.931002 #9047] INFO -- : listening on addr=127.0.0.1:8080 fd=12
+I, [2015-02-13T06:14:56.931381 #9047] INFO -- : listening on addr=/var/opt/gitlab/gitlab-rails/sockets/gitlab.socket fd=13
+I, [2015-02-13T06:14:56.936638 #9047] INFO -- : master process ready
+I, [2015-02-13T06:14:56.946504 #9092] INFO -- : worker=0 spawned pid=9092
+I, [2015-02-13T06:14:56.946943 #9092] INFO -- : worker=0 ready
+I, [2015-02-13T06:14:56.947892 #9094] INFO -- : worker=1 spawned pid=9094
+I, [2015-02-13T06:14:56.948181 #9094] INFO -- : worker=1 ready
+W, [2015-02-13T07:16:01.312916 #9094] WARN -- : #<Unicorn::HttpServer:0x0000000208f618>: worker (pid: 9094) exceeds memory limit (320626688 bytes > 247066940 bytes)
+W, [2015-02-13T07:16:01.313000 #9094] WARN -- : Unicorn::WorkerKiller send SIGQUIT (pid: 9094) alive: 3621 sec (trial 1)
+I, [2015-02-13T07:16:01.530733 #9047] INFO -- : reaped #<Process::Status: pid 9094 exit 0> worker=1
+I, [2015-02-13T07:16:01.534501 #13379] INFO -- : worker=1 spawned pid=13379
+I, [2015-02-13T07:16:01.534848 #13379] INFO -- : worker=1 ready
+```
diff --git a/doc/administration/repository_checks.md b/doc/administration/repository_checks.md
new file mode 100644
index 00000000000..4172b604cec
--- /dev/null
+++ b/doc/administration/repository_checks.md
@@ -0,0 +1,44 @@
+# Repository checks
+
+>**Note:**
+This feature was [introduced][ce-3232] in GitLab 8.7. It is OFF by
+default because it still causes too many false alarms.
+
+Git has a built-in mechanism, [git fsck][git-fsck], to verify the
+integrity of all data committed to a repository. GitLab administrators
+can trigger such a check for a project via the project page under the
+admin panel. The checks run asynchronously so it may take a few minutes
+before the check result is visible on the project admin page. If the
+checks failed you can see their output on the admin log page under
+'repocheck.log'.
+
+## Periodic checks
+
+GitLab periodically runs a repository check on all project repositories and
+wiki repositories in order to detect data corruption problems. A
+project will be checked no more than once per week. If any projects
+fail their repository checks all GitLab administrators will receive an email
+notification of the situation. This notification is sent out no more
+than once a day.
+
+## Disabling periodic checks
+
+You can disable the periodic checks on the 'Settings' page of the admin
+panel.
+
+## What to do if a check failed
+
+If the repository check fails for some repository you should look up the error
+in repocheck.log (in the admin panel or on disk; see
+`/var/log/gitlab/gitlab-rails` for Omnibus installations or
+`/home/git/gitlab/log` for installations from source). Once you have
+resolved the issue use the admin panel to trigger a new repository check on
+the project. This will clear the 'check failed' state.
+
+If for some reason the periodic repository check caused a lot of false
+alarms you can choose to clear ALL repository check states from the
+'Settings' page of the admin panel.
+
+---
+[ce-3232]: https://gitlab.com/gitlab-org/gitlab-ce/merge_requests/3232 "Auto git fsck"
+[git-fsck]: https://www.kernel.org/pub/software/scm/git/docs/git-fsck.html "git fsck documentation"
diff --git a/doc/administration/troubleshooting/sidekiq.md b/doc/administration/troubleshooting/sidekiq.md
new file mode 100644
index 00000000000..b71f8fabbc8
--- /dev/null
+++ b/doc/administration/troubleshooting/sidekiq.md
@@ -0,0 +1,171 @@
+# Troubleshooting Sidekiq
+
+Sidekiq is the background job processor GitLab uses to asynchronously run
+tasks. When things go wrong it can be difficult to troubleshoot. These
+situations also tend to be high-pressure because a production system job queue
+may be filling up. Users will notice when this happens because new branches
+may not show up and merge requests may not be updated. The following are some
+troubleshooting steps that will help you diagnose the bottleneck.
+
+> **Note:** GitLab administrators/users should consider working through these
+debug steps with GitLab Support so the backtraces can be analyzed by our team.
+It may reveal a bug or necessary improvement in GitLab.
+
+> **Note:** In any of the backtraces, be weary of suspecting cases where every
+ thread appears to be waiting in the database, Redis, or waiting to acquire
+ a mutex. This **may** mean there's contention in the database, for example,
+ but look for one thread that is different than the rest. This other thread
+ may be using all available CPU, or have a Ruby Global Interpreter Lock,
+ preventing other threads from continuing.
+
+## Thread dump
+
+Send the Sidekiq process ID the `TTIN` signal and it will output thread
+backtraces in the log file.
+
+```
+kill -TTIN <sidekiq_pid>
+```
+
+Check in `/var/log/gitlab/sidekiq/current` or `$GITLAB_HOME/log/sidekiq.log` for
+the backtrace output. The backtraces will be lengthy and generally start with
+several `WARN` level messages. Here's an example of a single thread's backtrace:
+
+```
+2016-04-13T06:21:20.022Z 31517 TID-orn4urby0 WARN: ActiveRecord::RecordNotFound: Couldn't find Note with 'id'=3375386
+2016-04-13T06:21:20.022Z 31517 TID-orn4urby0 WARN: /opt/gitlab/embedded/service/gem/ruby/2.1.0/gems/activerecord-4.2.5.2/lib/active_record/core.rb:155:in `find'
+/opt/gitlab/embedded/service/gitlab-rails/app/workers/new_note_worker.rb:7:in `perform'
+/opt/gitlab/embedded/service/gem/ruby/2.1.0/gems/sidekiq-4.0.1/lib/sidekiq/processor.rb:150:in `execute_job'
+/opt/gitlab/embedded/service/gem/ruby/2.1.0/gems/sidekiq-4.0.1/lib/sidekiq/processor.rb:132:in `block (2 levels) in process'
+/opt/gitlab/embedded/service/gem/ruby/2.1.0/gems/sidekiq-4.0.1/lib/sidekiq/middleware/chain.rb:127:in `block in invoke'
+/opt/gitlab/embedded/service/gitlab-rails/lib/gitlab/sidekiq_middleware/memory_killer.rb:17:in `call'
+/opt/gitlab/embedded/service/gem/ruby/2.1.0/gems/sidekiq-4.0.1/lib/sidekiq/middleware/chain.rb:129:in `block in invoke'
+/opt/gitlab/embedded/service/gitlab-rails/lib/gitlab/sidekiq_middleware/arguments_logger.rb:6:in `call'
+...
+```
+
+In some cases Sidekiq may be hung and unable to respond to the `TTIN` signal.
+Move on to other troubleshooting methods if this happens.
+
+## Process profiling with `perf`
+
+Linux has a process profiling tool called `perf` that is helpful when a certain
+process is eating up a lot of CPU. If you see high CPU usage and Sidekiq won't
+respond to the `TTIN` signal, this is a good next step.
+
+If `perf` is not installed on your system, install it with `apt-get` or `yum`:
+
+```
+# Debian
+sudo apt-get install linux-tools
+
+# Ubuntu (may require these additional Kernel packages)
+sudo apt-get install linux-tools-common linux-tools-generic linux-tools-`uname -r`
+
+# Red Hat/CentOS
+sudo yum install perf
+```
+
+Run perf against the Sidekiq PID:
+
+```
+sudo perf record -p <sidekiq_pid>
+```
+
+Let this run for 30-60 seconds and then press Ctrl-C. Then view the perf report:
+
+```
+sudo perf report
+
+# Sample output
+Samples: 348K of event 'cycles', Event count (approx.): 280908431073
+ 97.69% ruby nokogiri.so [.] xmlXPathNodeSetMergeAndClear
+ 0.18% ruby libruby.so.2.1.0 [.] objspace_malloc_increase
+ 0.12% ruby libc-2.12.so [.] _int_malloc
+ 0.10% ruby libc-2.12.so [.] _int_free
+```
+
+Above you see sample output from a perf report. It shows that 97% of the CPU is
+being spent inside Nokogiri and `xmlXPathNodeSetMergeAndClear`. For something
+this obvious you should then go investigate what job in GitLab would use
+Nokogiri and XPath. Combine with `TTIN` or `gdb` output to show the
+corresponding Ruby code where this is happening.
+
+## The GNU Project Debugger (gdb)
+
+`gdb` can be another effective tool for debugging Sidekiq. It gives you a little
+more interactive way to look at each thread and see what's causing problems.
+
+> **Note:** Attaching to a process with `gdb` will suspends the normal operation
+ of the process (Sidekiq will not process jobs while `gdb` is attached).
+
+Start by attaching to the Sidekiq PID:
+
+```
+gdb -p <sidekiq_pid>
+```
+
+Then gather information on all the threads:
+
+```
+info threads
+
+# Example output
+30 Thread 0x7fe5fbd63700 (LWP 26060) 0x0000003f7cadf113 in poll () from /lib64/libc.so.6
+29 Thread 0x7fe5f2b3b700 (LWP 26533) 0x0000003f7ce0b68c in pthread_cond_wait@@GLIBC_2.3.2 () from /lib64/libpthread.so.0
+28 Thread 0x7fe5f2a3a700 (LWP 26534) 0x0000003f7ce0ba5e in pthread_cond_timedwait@@GLIBC_2.3.2 () from /lib64/libpthread.so.0
+27 Thread 0x7fe5f2939700 (LWP 26535) 0x0000003f7ce0b68c in pthread_cond_wait@@GLIBC_2.3.2 () from /lib64/libpthread.so.0
+26 Thread 0x7fe5f2838700 (LWP 26537) 0x0000003f7ce0b68c in pthread_cond_wait@@GLIBC_2.3.2 () from /lib64/libpthread.so.0
+25 Thread 0x7fe5f2737700 (LWP 26538) 0x0000003f7ce0b68c in pthread_cond_wait@@GLIBC_2.3.2 () from /lib64/libpthread.so.0
+24 Thread 0x7fe5f2535700 (LWP 26540) 0x0000003f7ce0b68c in pthread_cond_wait@@GLIBC_2.3.2 () from /lib64/libpthread.so.0
+23 Thread 0x7fe5f2434700 (LWP 26541) 0x0000003f7ce0b68c in pthread_cond_wait@@GLIBC_2.3.2 () from /lib64/libpthread.so.0
+22 Thread 0x7fe5f2232700 (LWP 26543) 0x0000003f7ce0b68c in pthread_cond_wait@@GLIBC_2.3.2 () from /lib64/libpthread.so.0
+21 Thread 0x7fe5f2131700 (LWP 26544) 0x00007fe5f7b570f0 in xmlXPathNodeSetMergeAndClear ()
+from /opt/gitlab/embedded/service/gem/ruby/2.1.0/gems/nokogiri-1.6.7.2/lib/nokogiri/nokogiri.so
+...
+```
+
+If you see a suspicious thread, like the Nokogiri one above, you may want
+to get more information:
+
+```
+thread 21
+bt
+
+# Example output
+#0 0x00007ff0d6afe111 in xmlXPathNodeSetMergeAndClear () from /opt/gitlab/embedded/service/gem/ruby/2.1.0/gems/nokogiri-1.6.7.2/lib/nokogiri/nokogiri.so
+#1 0x00007ff0d6b0b836 in xmlXPathNodeCollectAndTest () from /opt/gitlab/embedded/service/gem/ruby/2.1.0/gems/nokogiri-1.6.7.2/lib/nokogiri/nokogiri.so
+#2 0x00007ff0d6b09037 in xmlXPathCompOpEval () from /opt/gitlab/embedded/service/gem/ruby/2.1.0/gems/nokogiri-1.6.7.2/lib/nokogiri/nokogiri.so
+#3 0x00007ff0d6b09017 in xmlXPathCompOpEval () from /opt/gitlab/embedded/service/gem/ruby/2.1.0/gems/nokogiri-1.6.7.2/lib/nokogiri/nokogiri.so
+#4 0x00007ff0d6b092e0 in xmlXPathCompOpEval () from /opt/gitlab/embedded/service/gem/ruby/2.1.0/gems/nokogiri-1.6.7.2/lib/nokogiri/nokogiri.so
+#5 0x00007ff0d6b0bc37 in xmlXPathRunEval () from /opt/gitlab/embedded/service/gem/ruby/2.1.0/gems/nokogiri-1.6.7.2/lib/nokogiri/nokogiri.so
+#6 0x00007ff0d6b0be5f in xmlXPathEvalExpression () from /opt/gitlab/embedded/service/gem/ruby/2.1.0/gems/nokogiri-1.6.7.2/lib/nokogiri/nokogiri.so
+#7 0x00007ff0d6a97dc3 in evaluate (argc=2, argv=0x1022d058, self=<value optimized out>) at xml_xpath_context.c:221
+#8 0x00007ff0daeab0ea in vm_call_cfunc_with_frame (th=0x1022a4f0, reg_cfp=0x1032b810, ci=<value optimized out>) at vm_insnhelper.c:1510
+```
+
+To output a backtrace from all threads at once:
+
+```
+set pagination off
+thread apply all bt
+```
+
+Once you're done debugging with `gdb`, be sure to detach from the process and
+exit:
+
+```
+detach
+exit
+```
+
+## Check for blocking queries
+
+Sometimes the speed at which Sidekiq processes jobs can be so fast that it can
+cause database contention. Check for blocking queries when backtraces above
+show that many threads are stuck in the database adapter.
+
+The PostgreSQL wiki has details on the query you can run to see blocking
+queries. The query is different based on PostgreSQL version. See
+[Lock Monitoring](https://wiki.postgresql.org/wiki/Lock_Monitoring) for
+the query details.
diff --git a/doc/api/README.md b/doc/api/README.md
index 7629ef294ac..73f44603688 100644
--- a/doc/api/README.md
+++ b/doc/api/README.md
@@ -8,41 +8,48 @@ under [`/lib/api`](https://gitlab.com/gitlab-org/gitlab-ce/tree/master/lib/api).
Documentation for various API resources can be found separately in the
following locations:
-- [Users](users.md)
-- [Session](session.md)
-- [Projects](projects.md) including setting Webhooks
-- [Project Snippets](project_snippets.md)
-- [Services](services.md)
-- [Repositories](repositories.md)
-- [Repository Files](repository_files.md)
-- [Commits](commits.md)
-- [Tags](tags.md)
+- [Award Emoji](award_emoji.md)
- [Branches](branches.md)
-- [Merge Requests](merge_requests.md)
+- [Builds](builds.md)
+- [Build triggers](build_triggers.md)
+- [Build Variables](build_variables.md)
+- [Commits](commits.md)
+- [Deploy Keys](deploy_keys.md)
+- [Groups](groups.md)
- [Issues](issues.md)
+- [Keys](keys.md)
- [Labels](labels.md)
+- [Merge Requests](merge_requests.md)
- [Milestones](milestones.md)
-- [Notes](notes.md) (comments)
-- [Deploy Keys](deploy_keys.md)
-- [System Hooks](system_hooks.md)
-- [Groups](groups.md)
+- [Open source license templates](licenses.md)
- [Namespaces](namespaces.md)
-- [Settings](settings.md)
-- [Keys](keys.md)
-- [Builds](builds.md)
-- [Build triggers](build_triggers.md)
-- [Build Variables](build_variables.md)
+- [Notes](notes.md) (comments)
+- [Projects](projects.md) including setting Webhooks
+- [Project Snippets](project_snippets.md)
+- [Repositories](repositories.md)
+- [Repository Files](repository_files.md)
- [Runners](runners.md)
+- [Services](services.md)
+- [Session](session.md)
+- [Settings](settings.md)
+- [System Hooks](system_hooks.md)
+- [Tags](tags.md)
+- [Users](users.md)
+
+### Internal CI API
+
+The following documentation is for the [internal CI API](ci/README.md):
+
+- [Builds](ci/builds.md)
+- [Runners](ci/runners.md)
## Authentication
-All API requests require authentication. You need to pass a `private_token`
-parameter via query string or header. If passed as a header, the header name
-must be `PRIVATE-TOKEN` (uppercase and with a dash instead of an underscore).
-You can find or reset your private token in your account page (`/profile/account`).
+All API requests require authentication via a token. There are three types of tokens
+available: private tokens, OAuth 2 tokens, and personal access tokens.
-If `private_token` is invalid or omitted, then an error message will be
-returned with status code `401`:
+If a token is invalid or omitted, an error message will be returned with
+status code `401`:
```json
{
@@ -50,42 +57,56 @@ returned with status code `401`:
}
```
-API requests should be prefixed with `api` and the API version. The API version
-is defined in [`lib/api.rb`][lib-api-url].
+### Private Tokens
-Example of a valid API request:
+You need to pass a `private_token` parameter via query string or header. If passed as a
+header, the header name must be `PRIVATE-TOKEN` (uppercase and with a dash instead of
+an underscore). You can find or reset your private token in your account page
+(`/profile/account`).
-```shell
-GET https://gitlab.example.com/api/v3/projects?private_token=9koXpg98eAheJpvBs5tK
-```
+### OAuth 2 Tokens
-Example of a valid API request using cURL and authentication via header:
+You can use an OAuth 2 token to authenticate with the API by passing it either in the
+`access_token` parameter or in the `Authorization` header.
+
+Example of using the OAuth2 token in the header:
```shell
-curl --header "PRIVATE-TOKEN: 9koXpg98eAheJpvBs5tK" "https://gitlab.example.com/api/v3/projects"
+curl -H "Authorization: Bearer OAUTH-TOKEN" https://gitlab.example.com/api/v3/projects
```
-The API uses JSON to serialize data. You don't need to specify `.json` at the
-end of an API URL.
+Read more about [GitLab as an OAuth2 client](oauth2.md).
+
+### Personal Access Tokens
-## Authentication with OAuth2 token
+> **Note:** This feature was [introduced][ce-3749] in GitLab 8.8
-Instead of the `private_token` you can transmit the OAuth2 access token as a
-header or as a parameter.
+You can create as many personal access tokens as you like from your GitLab
+profile (`/profile/personal_access_tokens`); perhaps one for each application
+that needs access to the GitLab API.
+
+Once you have your token, pass it to the API using either the `private_token`
+parameter or the `PRIVATE-TOKEN` header.
+
+## Basic Usage
+
+API requests should be prefixed with `api` and the API version. The API version
+is defined in [`lib/api.rb`][lib-api-url].
-Example of OAuth2 token as a parameter:
+Example of a valid API request:
```shell
-curl https://gitlab.example.com/api/v3/user?access_token=OAUTH-TOKEN
+GET https://gitlab.example.com/api/v3/projects?private_token=9koXpg98eAheJpvBs5tK
```
-Example of OAuth2 token as a header:
+Example of a valid API request using cURL and authentication via header:
```shell
-curl -H "Authorization: Bearer OAUTH-TOKEN" https://example.com/api/v3/user
+curl --header "PRIVATE-TOKEN: 9koXpg98eAheJpvBs5tK" "https://gitlab.example.com/api/v3/projects"
```
-Read more about [GitLab as an OAuth2 client](oauth2.md).
+The API uses JSON to serialize data. You don't need to specify `.json` at the
+end of an API URL.
## Status codes
@@ -108,6 +129,7 @@ The following table shows the possible return codes for API requests.
| ------------- | ----------- |
| `200 OK` | The `GET`, `PUT` or `DELETE` request was successful, the resource(s) itself is returned as JSON. |
| `201 Created` | The `POST` request was successful and the resource is returned as JSON. |
+| `304 Not Modified` | Indicates that the resource has not been modified since the last request. |
| `400 Bad Request` | A required attribute of the API request is missing, e.g., the title of an issue is not given. |
| `401 Unauthorized` | The user is not authenticated, a valid [user token](#authentication) is necessary. |
| `403 Forbidden` | The request is not allowed, e.g., the user is not allowed to delete a project. |
@@ -321,3 +343,4 @@ programming languages. Visit the [GitLab website] for a complete list.
[GitLab website]: https://about.gitlab.com/applications/#api-clients "Clients using the GitLab API"
[lib-api-url]: https://gitlab.com/gitlab-org/gitlab-ce/tree/master/lib/api/api.rb
+[ce-3749]: https://gitlab.com/gitlab-org/gitlab-ce/merge_requests/3749
diff --git a/doc/api/award_emoji.md b/doc/api/award_emoji.md
new file mode 100644
index 00000000000..b44f8cfd628
--- /dev/null
+++ b/doc/api/award_emoji.md
@@ -0,0 +1,367 @@
+# Award Emoji
+
+ >**Note:** This feature was introduced in GitLab 8.9
+
+An awarded emoji tells a thousand words, and can be awarded on issues, merge
+requests and notes/comments. Issues, merge requests and notes are further called
+`awardables`.
+
+## Issues and merge requests
+
+### List an awardable's award emoji
+
+Gets a list of all award emoji
+
+```
+GET /projects/:id/issues/:issue_id/award_emoji
+GET /projects/:id/merge_requests/:merge_request_id/award_emoji
+```
+
+Parameters:
+
+| Attribute | Type | Required | Description |
+| --------- | ---- | -------- | ----------- |
+| `id` | integer | yes | The ID of a project |
+| `awardable_id` | integer | yes | The ID of an awardable |
+
+```bash
+curl -H "PRIVATE-TOKEN: 9koXpg98eAheJpvBs5tK" http://gitlab.example.com/api/v3/projects/1/issues/80/award_emoji
+```
+
+Example Response:
+
+```json
+[
+ {
+ "id": 4,
+ "name": "1234",
+ "user": {
+ "name": "Administrator",
+ "username": "root",
+ "id": 1,
+ "state": "active",
+ "avatar_url": "http://www.gravatar.com/avatar/e64c7d89f26bd1972efa854d13d7dd61?s=80&d=identicon",
+ "web_url": "http://gitlab.example.com/u/root"
+ },
+ "created_at": "2016-06-15T10:09:34.206Z",
+ "updated_at": "2016-06-15T10:09:34.206Z",
+ "awardable_id": 80,
+ "awardable_type": "Issue"
+ },
+ {
+ "id": 1,
+ "name": "microphone",
+ "user": {
+ "name": "User 4",
+ "username": "user4",
+ "id": 26,
+ "state": "active",
+ "avatar_url": "http://www.gravatar.com/avatar/7e65550957227bd38fe2d7fbc6fd2f7b?s=80&d=identicon",
+ "web_url": "http://gitlab.example.com/u/user4"
+ },
+ "created_at": "2016-06-15T10:09:34.177Z",
+ "updated_at": "2016-06-15T10:09:34.177Z",
+ "awardable_id": 80,
+ "awardable_type": "Issue"
+ }
+]
+```
+
+### Get single issue note
+
+Gets a single award emoji
+
+```
+GET /projects/:id/issues/:issue_id/award_emoji/:award_id
+GET /projects/:id/merge_requests/:merge_request_id/award_emoji/:award_id
+```
+
+Parameters:
+
+| Attribute | Type | Required | Description |
+| --------- | ---- | -------- | ----------- |
+| `id` | integer | yes | The ID of a project |
+| `awardable_id` | integer | yes | The ID of an awardable |
+| `award_id` | integer | yes | The ID of the award emoji |
+
+```bash
+curl -H "PRIVATE-TOKEN: 9koXpg98eAheJpvBs5tK" http://gitlab.example.com/api/v3/projects/1/issues/80/award_emoji/1
+```
+
+Example Response:
+
+```json
+{
+ "id": 1,
+ "name": "microphone",
+ "user": {
+ "name": "User 4",
+ "username": "user4",
+ "id": 26,
+ "state": "active",
+ "avatar_url": "http://www.gravatar.com/avatar/7e65550957227bd38fe2d7fbc6fd2f7b?s=80&d=identicon",
+ "web_url": "http://gitlab.example.com/u/user4"
+ },
+ "created_at": "2016-06-15T10:09:34.177Z",
+ "updated_at": "2016-06-15T10:09:34.177Z",
+ "awardable_id": 80,
+ "awardable_type": "Issue"
+}
+```
+
+### Award a new emoji
+
+This end point creates an award emoji on the specified resource
+
+```
+POST /projects/:id/issues/:issue_id/award_emoji
+POST /projects/:id/merge_requests/:merge_request_id/award_emoji
+```
+
+Parameters:
+
+| Attribute | Type | Required | Description |
+| --------- | ---- | -------- | ----------- |
+| `id` | integer | yes | The ID of a project |
+| `awardable_id` | integer | yes | The ID of an awardable |
+| `name` | string | yes | The name of the emoji, without colons |
+
+```bash
+curl -X POST -H "PRIVATE-TOKEN: 9koXpg98eAheJpvBs5tK" http://gitlab.example.com/api/v3/projects/1/issues/80/award_emoji?name=blowfish
+```
+
+Example Response:
+
+```json
+{
+ "id": 344,
+ "name": "blowfish",
+ "user": {
+ "name": "Administrator",
+ "username": "root",
+ "id": 1,
+ "state": "active",
+ "avatar_url": "http://www.gravatar.com/avatar/e64c7d89f26bd1972efa854d13d7dd61?s=80&d=identicon",
+ "web_url": "http://gitlab.example.com/u/root"
+ },
+ "created_at": "2016-06-17T17:47:29.266Z",
+ "updated_at": "2016-06-17T17:47:29.266Z",
+ "awardable_id": 80,
+ "awardable_type": "Issue"
+}
+```
+
+### Delete an award emoji
+
+Sometimes its just not meant to be, and you'll have to remove your award. Only available to
+admins or the author of the award. Status code 200 on success, 401 if unauthorized.
+
+```
+DELETE /projects/:id/issues/:issue_id/award_emoji/:award_id
+DELETE /projects/:id/merge_requests/:merge_request_id/award_emoji/:award_id
+```
+
+Parameters:
+
+| Attribute | Type | Required | Description |
+| --------- | ---- | -------- | ----------- |
+| `id` | integer | yes | The ID of a project |
+| `issue_id` | integer | yes | The ID of an issue |
+| `award_id` | integer | yes | The ID of a award_emoji |
+
+```bash
+curl -X DELETE -H "PRIVATE-TOKEN: 9koXpg98eAheJpvBs5tK" http://gitlab.example.com/api/v3/projects/1/issues/80/award_emoji/344
+```
+
+Example Response:
+
+```json
+{
+ "id": 344,
+ "name": "blowfish",
+ "user": {
+ "name": "Administrator",
+ "username": "root",
+ "id": 1,
+ "state": "active",
+ "avatar_url": "http://www.gravatar.com/avatar/e64c7d89f26bd1972efa854d13d7dd61?s=80&d=identicon",
+ "web_url": "http://gitlab.example.com/u/root"
+ },
+ "created_at": "2016-06-17T17:47:29.266Z",
+ "updated_at": "2016-06-17T17:47:29.266Z",
+ "awardable_id": 80,
+ "awardable_type": "Issue"
+}
+```
+
+## Award Emoji on Notes
+
+The endpoints documented above are available for Notes as well. Notes
+are a sub-resource of Issues and Merge Requests. The examples below
+describe working with Award Emoji on notes for an Issue, but can be
+easily adapted for notes on a Merge Request.
+
+### List a note's award emoji
+
+```
+GET /projects/:id/issues/:issue_id/notes/:note_id/award_emoji
+```
+
+Parameters:
+
+| Attribute | Type | Required | Description |
+| --------- | ---- | -------- | ----------- |
+| `id` | integer | yes | The ID of a project |
+| `issue_id` | integer | yes | The ID of an issue |
+| `note_id` | integer | yes | The ID of an note |
+
+
+```bash
+curl -H "PRIVATE-TOKEN: 9koXpg98eAheJpvBs5tK" http://gitlab.example.com/api/v3/projects/1/issues/80/notes/1/award_emoji
+```
+
+Example Response:
+
+```json
+[
+ {
+ "id": 2,
+ "name": "mood_bubble_lightning",
+ "user": {
+ "name": "User 4",
+ "username": "user4",
+ "id": 26,
+ "state": "active",
+ "avatar_url": "http://www.gravatar.com/avatar/7e65550957227bd38fe2d7fbc6fd2f7b?s=80&d=identicon",
+ "web_url": "http://gitlab.example.com/u/user4"
+ },
+ "created_at": "2016-06-15T10:09:34.197Z",
+ "updated_at": "2016-06-15T10:09:34.197Z",
+ "awardable_id": 1,
+ "awardable_type": "Note"
+ }
+]
+```
+
+### Get single note's award emoji
+
+```
+GET /projects/:id/issues/:issue_id/notes/:note_id/award_emoji/:award_id
+```
+
+Parameters:
+
+| Attribute | Type | Required | Description |
+| --------- | ---- | -------- | ----------- |
+| `id` | integer | yes | The ID of a project |
+| `issue_id` | integer | yes | The ID of an issue |
+| `note_id` | integer | yes | The ID of a note |
+| `award_id` | integer | yes | The ID of the award emoji |
+
+```bash
+curl -H "PRIVATE-TOKEN: 9koXpg98eAheJpvBs5tK" http://gitlab.example.com/api/v3/projects/1/issues/80/notes/1/award_emoji/2
+```
+
+Example Response:
+
+```json
+{
+ "id": 2,
+ "name": "mood_bubble_lightning",
+ "user": {
+ "name": "User 4",
+ "username": "user4",
+ "id": 26,
+ "state": "active",
+ "avatar_url": "http://www.gravatar.com/avatar/7e65550957227bd38fe2d7fbc6fd2f7b?s=80&d=identicon",
+ "web_url": "http://gitlab.example.com/u/user4"
+ },
+ "created_at": "2016-06-15T10:09:34.197Z",
+ "updated_at": "2016-06-15T10:09:34.197Z",
+ "awardable_id": 1,
+ "awardable_type": "Note"
+}
+```
+
+### Award a new emoji on a note
+
+```
+POST /projects/:id/issues/:issue_id/notes/:note_id/award_emoji
+```
+
+Parameters:
+
+| Attribute | Type | Required | Description |
+| --------- | ---- | -------- | ----------- |
+| `id` | integer | yes | The ID of a project |
+| `issue_id` | integer | yes | The ID of an issue |
+| `note_id` | integer | yes | The ID of a note |
+| `name` | string | yes | The name of the emoji, without colons |
+
+```bash
+curl -X POST -H "PRIVATE-TOKEN: 9koXpg98eAheJpvBs5tK" http://gitlab.example.com/api/v3/projects/1/issues/80/notes/1/award_emoji?name=rocket
+```
+
+Example Response:
+
+```json
+{
+ "id": 345,
+ "name": "rocket",
+ "user": {
+ "name": "Administrator",
+ "username": "root",
+ "id": 1,
+ "state": "active",
+ "avatar_url": "http://www.gravatar.com/avatar/e64c7d89f26bd1972efa854d13d7dd61?s=80&d=identicon",
+ "web_url": "http://gitlab.example.com/u/root"
+ },
+ "created_at": "2016-06-17T19:59:55.888Z",
+ "updated_at": "2016-06-17T19:59:55.888Z",
+ "awardable_id": 1,
+ "awardable_type": "Note"
+}
+```
+
+### Delete an award emoji
+
+Sometimes its just not meant to be, and you'll have to remove your award. Only available to
+admins or the author of the award. Status code 200 on success, 401 if unauthorized.
+
+```
+DELETE /projects/:id/issues/:issue_id/notes/:note_id/award_emoji/:award_id
+```
+
+Parameters:
+
+| Attribute | Type | Required | Description |
+| --------- | ---- | -------- | ----------- |
+| `id` | integer | yes | The ID of a project |
+| `issue_id` | integer | yes | The ID of an issue |
+| `note_id` | integer | yes | The ID of a note |
+| `award_id` | integer | yes | The ID of a award_emoji |
+
+```bash
+curl -X DELETE -H "PRIVATE-TOKEN: 9koXpg98eAheJpvBs5tK" http://gitlab.example.com/api/v3/projects/1/issues/80/award_emoji/345
+```
+
+Example Response:
+
+```json
+{
+ "id": 345,
+ "name": "rocket",
+ "user": {
+ "name": "Administrator",
+ "username": "root",
+ "id": 1,
+ "state": "active",
+ "avatar_url": "http://www.gravatar.com/avatar/e64c7d89f26bd1972efa854d13d7dd61?s=80&d=identicon",
+ "web_url": "http://gitlab.example.com/u/root"
+ },
+ "created_at": "2016-06-17T19:59:55.888Z",
+ "updated_at": "2016-06-17T19:59:55.888Z",
+ "awardable_id": 1,
+ "awardable_type": "Note"
+}
+```
diff --git a/doc/api/build_triggers.md b/doc/api/build_triggers.md
index 4a12e962b62..0881a7d7a90 100644
--- a/doc/api/build_triggers.md
+++ b/doc/api/build_triggers.md
@@ -101,8 +101,18 @@ DELETE /projects/:id/triggers/:token
| Attribute | Type | required | Description |
|-----------|---------|----------|--------------------------|
| `id` | integer | yes | The ID of a project |
-| `token` | string | yes | The `token` of a project |
+| `token` | string | yes | The `token` of a trigger |
```
curl -X DELETE -H "PRIVATE_TOKEN: 9koXpg98eAheJpvBs5tK" "https://gitlab.example.com/api/v3/projects/1/triggers/7b9148c158980bbd9bcea92c17522d"
```
+
+```json
+{
+ "created_at": "2015-12-23T16:25:56.760Z",
+ "deleted_at": "2015-12-24T12:32:20.100Z",
+ "last_used": null,
+ "token": "7b9148c158980bbd9bcea92c17522d",
+ "updated_at": "2015-12-24T12:32:20.100Z"
+}
+```
diff --git a/doc/api/builds.md b/doc/api/builds.md
index 4c0a47d1ea0..de998944352 100644
--- a/doc/api/builds.md
+++ b/doc/api/builds.md
@@ -21,85 +21,85 @@ Example of response
```json
[
- {
- "commit": {
- "author_email": "admin@example.com",
- "author_name": "Administrator",
- "created_at": "2015-12-24T16:51:14.000+01:00",
- "id": "0ff3ae198f8601a285adcf5c0fff204ee6fba5fd",
- "message": "Test the CI integration.",
- "short_id": "0ff3ae19",
- "title": "Test the CI integration."
- },
- "coverage": null,
- "created_at": "2015-12-24T15:51:21.802Z",
- "artifacts_file": {
- "filename": "artifacts.zip",
- "size": 1000
- },
- "finished_at": "2015-12-24T17:54:27.895Z",
- "id": 7,
- "name": "teaspoon",
- "ref": "master",
- "runner": null,
- "stage": "test",
- "started_at": "2015-12-24T17:54:27.722Z",
- "status": "failed",
- "tag": false,
- "user": {
- "avatar_url": "http://www.gravatar.com/avatar/e64c7d89f26bd1972efa854d13d7dd61?s=80&d=identicon",
- "bio": null,
- "created_at": "2015-12-21T13:14:24.077Z",
- "id": 1,
- "is_admin": true,
- "linkedin": "",
- "name": "Administrator",
- "skype": "",
- "state": "active",
- "twitter": "",
- "username": "root",
- "web_url": "http://gitlab.dev/u/root",
- "website_url": ""
- }
+ {
+ "commit": {
+ "author_email": "admin@example.com",
+ "author_name": "Administrator",
+ "created_at": "2015-12-24T16:51:14.000+01:00",
+ "id": "0ff3ae198f8601a285adcf5c0fff204ee6fba5fd",
+ "message": "Test the CI integration.",
+ "short_id": "0ff3ae19",
+ "title": "Test the CI integration."
+ },
+ "coverage": null,
+ "created_at": "2015-12-24T15:51:21.802Z",
+ "artifacts_file": {
+ "filename": "artifacts.zip",
+ "size": 1000
},
- {
- "commit": {
- "author_email": "admin@example.com",
- "author_name": "Administrator",
- "created_at": "2015-12-24T16:51:14.000+01:00",
- "id": "0ff3ae198f8601a285adcf5c0fff204ee6fba5fd",
- "message": "Test the CI integration.",
- "short_id": "0ff3ae19",
- "title": "Test the CI integration."
- },
- "coverage": null,
- "created_at": "2015-12-24T15:51:21.727Z",
- "artifacts_file": null,
- "finished_at": "2015-12-24T17:54:24.921Z",
- "id": 6,
- "name": "spinach:other",
- "ref": "master",
- "runner": null,
- "stage": "test",
- "started_at": "2015-12-24T17:54:24.729Z",
- "status": "failed",
- "tag": false,
- "user": {
- "avatar_url": "http://www.gravatar.com/avatar/e64c7d89f26bd1972efa854d13d7dd61?s=80&d=identicon",
- "bio": null,
- "created_at": "2015-12-21T13:14:24.077Z",
- "id": 1,
- "is_admin": true,
- "linkedin": "",
- "name": "Administrator",
- "skype": "",
- "state": "active",
- "twitter": "",
- "username": "root",
- "web_url": "http://gitlab.dev/u/root",
- "website_url": ""
- }
+ "finished_at": "2015-12-24T17:54:27.895Z",
+ "id": 7,
+ "name": "teaspoon",
+ "ref": "master",
+ "runner": null,
+ "stage": "test",
+ "started_at": "2015-12-24T17:54:27.722Z",
+ "status": "failed",
+ "tag": false,
+ "user": {
+ "avatar_url": "http://www.gravatar.com/avatar/e64c7d89f26bd1972efa854d13d7dd61?s=80&d=identicon",
+ "bio": null,
+ "created_at": "2015-12-21T13:14:24.077Z",
+ "id": 1,
+ "is_admin": true,
+ "linkedin": "",
+ "name": "Administrator",
+ "skype": "",
+ "state": "active",
+ "twitter": "",
+ "username": "root",
+ "web_url": "http://gitlab.dev/u/root",
+ "website_url": ""
}
+ },
+ {
+ "commit": {
+ "author_email": "admin@example.com",
+ "author_name": "Administrator",
+ "created_at": "2015-12-24T16:51:14.000+01:00",
+ "id": "0ff3ae198f8601a285adcf5c0fff204ee6fba5fd",
+ "message": "Test the CI integration.",
+ "short_id": "0ff3ae19",
+ "title": "Test the CI integration."
+ },
+ "coverage": null,
+ "created_at": "2015-12-24T15:51:21.727Z",
+ "artifacts_file": null,
+ "finished_at": "2015-12-24T17:54:24.921Z",
+ "id": 6,
+ "name": "spinach:other",
+ "ref": "master",
+ "runner": null,
+ "stage": "test",
+ "started_at": "2015-12-24T17:54:24.729Z",
+ "status": "failed",
+ "tag": false,
+ "user": {
+ "avatar_url": "http://www.gravatar.com/avatar/e64c7d89f26bd1972efa854d13d7dd61?s=80&d=identicon",
+ "bio": null,
+ "created_at": "2015-12-21T13:14:24.077Z",
+ "id": 1,
+ "is_admin": true,
+ "linkedin": "",
+ "name": "Administrator",
+ "skype": "",
+ "state": "active",
+ "twitter": "",
+ "username": "root",
+ "web_url": "http://gitlab.dev/u/root",
+ "website_url": ""
+ }
+ }
]
```
@@ -125,68 +125,68 @@ Example of response
```json
[
- {
- "commit": {
- "author_email": "admin@example.com",
- "author_name": "Administrator",
- "created_at": "2015-12-24T16:51:14.000+01:00",
- "id": "0ff3ae198f8601a285adcf5c0fff204ee6fba5fd",
- "message": "Test the CI integration.",
- "short_id": "0ff3ae19",
- "title": "Test the CI integration."
- },
- "coverage": null,
- "created_at": "2016-01-11T10:13:33.506Z",
- "artifacts_file": null,
- "finished_at": "2016-01-11T10:14:09.526Z",
- "id": 69,
- "name": "rubocop",
- "ref": "master",
- "runner": null,
- "stage": "test",
- "started_at": null,
- "status": "canceled",
- "tag": false,
- "user": null
+ {
+ "commit": {
+ "author_email": "admin@example.com",
+ "author_name": "Administrator",
+ "created_at": "2015-12-24T16:51:14.000+01:00",
+ "id": "0ff3ae198f8601a285adcf5c0fff204ee6fba5fd",
+ "message": "Test the CI integration.",
+ "short_id": "0ff3ae19",
+ "title": "Test the CI integration."
+ },
+ "coverage": null,
+ "created_at": "2016-01-11T10:13:33.506Z",
+ "artifacts_file": null,
+ "finished_at": "2016-01-11T10:14:09.526Z",
+ "id": 69,
+ "name": "rubocop",
+ "ref": "master",
+ "runner": null,
+ "stage": "test",
+ "started_at": null,
+ "status": "canceled",
+ "tag": false,
+ "user": null
+ },
+ {
+ "commit": {
+ "author_email": "admin@example.com",
+ "author_name": "Administrator",
+ "created_at": "2015-12-24T16:51:14.000+01:00",
+ "id": "0ff3ae198f8601a285adcf5c0fff204ee6fba5fd",
+ "message": "Test the CI integration.",
+ "short_id": "0ff3ae19",
+ "title": "Test the CI integration."
},
- {
- "commit": {
- "author_email": "admin@example.com",
- "author_name": "Administrator",
- "created_at": "2015-12-24T16:51:14.000+01:00",
- "id": "0ff3ae198f8601a285adcf5c0fff204ee6fba5fd",
- "message": "Test the CI integration.",
- "short_id": "0ff3ae19",
- "title": "Test the CI integration."
- },
- "coverage": null,
- "created_at": "2015-12-24T15:51:21.957Z",
- "artifacts_file": null,
- "finished_at": "2015-12-24T17:54:33.913Z",
- "id": 9,
- "name": "brakeman",
- "ref": "master",
- "runner": null,
- "stage": "test",
- "started_at": "2015-12-24T17:54:33.727Z",
- "status": "failed",
- "tag": false,
- "user": {
- "avatar_url": "http://www.gravatar.com/avatar/e64c7d89f26bd1972efa854d13d7dd61?s=80&d=identicon",
- "bio": null,
- "created_at": "2015-12-21T13:14:24.077Z",
- "id": 1,
- "is_admin": true,
- "linkedin": "",
- "name": "Administrator",
- "skype": "",
- "state": "active",
- "twitter": "",
- "username": "root",
- "web_url": "http://gitlab.dev/u/root",
- "website_url": ""
- }
+ "coverage": null,
+ "created_at": "2015-12-24T15:51:21.957Z",
+ "artifacts_file": null,
+ "finished_at": "2015-12-24T17:54:33.913Z",
+ "id": 9,
+ "name": "brakeman",
+ "ref": "master",
+ "runner": null,
+ "stage": "test",
+ "started_at": "2015-12-24T17:54:33.727Z",
+ "status": "failed",
+ "tag": false,
+ "user": {
+ "avatar_url": "http://www.gravatar.com/avatar/e64c7d89f26bd1972efa854d13d7dd61?s=80&d=identicon",
+ "bio": null,
+ "created_at": "2015-12-21T13:14:24.077Z",
+ "id": 1,
+ "is_admin": true,
+ "linkedin": "",
+ "name": "Administrator",
+ "skype": "",
+ "state": "active",
+ "twitter": "",
+ "username": "root",
+ "web_url": "http://gitlab.dev/u/root",
+ "website_url": ""
}
+ }
]
```
@@ -211,42 +211,42 @@ Example of response
```json
{
- "commit": {
- "author_email": "admin@example.com",
- "author_name": "Administrator",
- "created_at": "2015-12-24T16:51:14.000+01:00",
- "id": "0ff3ae198f8601a285adcf5c0fff204ee6fba5fd",
- "message": "Test the CI integration.",
- "short_id": "0ff3ae19",
- "title": "Test the CI integration."
- },
- "coverage": null,
- "created_at": "2015-12-24T15:51:21.880Z",
- "artifacts_file": null,
- "finished_at": "2015-12-24T17:54:31.198Z",
- "id": 8,
- "name": "rubocop",
- "ref": "master",
- "runner": null,
- "stage": "test",
- "started_at": "2015-12-24T17:54:30.733Z",
- "status": "failed",
- "tag": false,
- "user": {
- "avatar_url": "http://www.gravatar.com/avatar/e64c7d89f26bd1972efa854d13d7dd61?s=80&d=identicon",
- "bio": null,
- "created_at": "2015-12-21T13:14:24.077Z",
- "id": 1,
- "is_admin": true,
- "linkedin": "",
- "name": "Administrator",
- "skype": "",
- "state": "active",
- "twitter": "",
- "username": "root",
- "web_url": "http://gitlab.dev/u/root",
- "website_url": ""
- }
+ "commit": {
+ "author_email": "admin@example.com",
+ "author_name": "Administrator",
+ "created_at": "2015-12-24T16:51:14.000+01:00",
+ "id": "0ff3ae198f8601a285adcf5c0fff204ee6fba5fd",
+ "message": "Test the CI integration.",
+ "short_id": "0ff3ae19",
+ "title": "Test the CI integration."
+ },
+ "coverage": null,
+ "created_at": "2015-12-24T15:51:21.880Z",
+ "artifacts_file": null,
+ "finished_at": "2015-12-24T17:54:31.198Z",
+ "id": 8,
+ "name": "rubocop",
+ "ref": "master",
+ "runner": null,
+ "stage": "test",
+ "started_at": "2015-12-24T17:54:30.733Z",
+ "status": "failed",
+ "tag": false,
+ "user": {
+ "avatar_url": "http://www.gravatar.com/avatar/e64c7d89f26bd1972efa854d13d7dd61?s=80&d=identicon",
+ "bio": null,
+ "created_at": "2015-12-21T13:14:24.077Z",
+ "id": 1,
+ "is_admin": true,
+ "linkedin": "",
+ "name": "Administrator",
+ "skype": "",
+ "state": "active",
+ "twitter": "",
+ "username": "root",
+ "web_url": "http://gitlab.dev/u/root",
+ "website_url": ""
+ }
}
```
@@ -278,6 +278,30 @@ Response:
[ce-2893]: https://gitlab.com/gitlab-org/gitlab-ce/merge_requests/2893
+## Get a trace file
+
+Get a trace of a specific build of a project
+
+```
+GET /projects/:id/builds/:build_id/trace
+```
+
+| Attribute | Type | Required | Description |
+|------------|---------|----------|---------------------|
+| id | integer | yes | The ID of a project |
+| build_id | integer | yes | The ID of a build |
+
+```
+curl -H "PRIVATE-TOKEN: 9koXpg98eAheJpvBs5tK" "https://gitlab.example.com/api/v3/projects/1/builds/8/trace"
+```
+
+Response:
+
+| Status | Description |
+|-----------|-----------------------------------|
+| 200 | Serves the trace file |
+| 404 | Build not found or no trace file |
+
## Cancel a build
Cancel a single build of a project
@@ -299,28 +323,28 @@ Example of response
```json
{
- "commit": {
- "author_email": "admin@example.com",
- "author_name": "Administrator",
- "created_at": "2015-12-24T16:51:14.000+01:00",
- "id": "0ff3ae198f8601a285adcf5c0fff204ee6fba5fd",
- "message": "Test the CI integration.",
- "short_id": "0ff3ae19",
- "title": "Test the CI integration."
- },
- "coverage": null,
- "created_at": "2016-01-11T10:13:33.506Z",
- "artifacts_file": null,
- "finished_at": "2016-01-11T10:14:09.526Z",
- "id": 69,
- "name": "rubocop",
- "ref": "master",
- "runner": null,
- "stage": "test",
- "started_at": null,
- "status": "canceled",
- "tag": false,
- "user": null
+ "commit": {
+ "author_email": "admin@example.com",
+ "author_name": "Administrator",
+ "created_at": "2015-12-24T16:51:14.000+01:00",
+ "id": "0ff3ae198f8601a285adcf5c0fff204ee6fba5fd",
+ "message": "Test the CI integration.",
+ "short_id": "0ff3ae19",
+ "title": "Test the CI integration."
+ },
+ "coverage": null,
+ "created_at": "2016-01-11T10:13:33.506Z",
+ "artifacts_file": null,
+ "finished_at": "2016-01-11T10:14:09.526Z",
+ "id": 69,
+ "name": "rubocop",
+ "ref": "master",
+ "runner": null,
+ "stage": "test",
+ "started_at": null,
+ "status": "canceled",
+ "tag": false,
+ "user": null
}
```
@@ -345,28 +369,28 @@ Example of response
```json
{
- "commit": {
- "author_email": "admin@example.com",
- "author_name": "Administrator",
- "created_at": "2015-12-24T16:51:14.000+01:00",
- "id": "0ff3ae198f8601a285adcf5c0fff204ee6fba5fd",
- "message": "Test the CI integration.",
- "short_id": "0ff3ae19",
- "title": "Test the CI integration."
- },
- "coverage": null,
- "created_at": "2016-01-11T10:13:33.506Z",
- "artifacts_file": null,
- "finished_at": null,
- "id": 69,
- "name": "rubocop",
- "ref": "master",
- "runner": null,
- "stage": "test",
- "started_at": null,
- "status": "pending",
- "tag": false,
- "user": null
+ "commit": {
+ "author_email": "admin@example.com",
+ "author_name": "Administrator",
+ "created_at": "2015-12-24T16:51:14.000+01:00",
+ "id": "0ff3ae198f8601a285adcf5c0fff204ee6fba5fd",
+ "message": "Test the CI integration.",
+ "short_id": "0ff3ae19",
+ "title": "Test the CI integration."
+ },
+ "coverage": null,
+ "created_at": "2016-01-11T10:13:33.506Z",
+ "artifacts_file": null,
+ "finished_at": null,
+ "id": 69,
+ "name": "rubocop",
+ "ref": "master",
+ "runner": null,
+ "stage": "test",
+ "started_at": null,
+ "status": "pending",
+ "tag": false,
+ "user": null
}
```
@@ -395,27 +419,77 @@ Example of response
```json
{
- "commit": {
- "author_email": "admin@example.com",
- "author_name": "Administrator",
- "created_at": "2015-12-24T16:51:14.000+01:00",
- "id": "0ff3ae198f8601a285adcf5c0fff204ee6fba5fd",
- "message": "Test the CI integration.",
- "short_id": "0ff3ae19",
- "title": "Test the CI integration."
- },
- "coverage": null,
- "download_url": null,
- "id": 69,
- "name": "rubocop",
- "ref": "master",
- "runner": null,
- "stage": "test",
- "created_at": "2016-01-11T10:13:33.506Z",
- "started_at": "2016-01-11T10:13:33.506Z",
- "finished_at": "2016-01-11T10:15:10.506Z",
- "status": "failed",
- "tag": false,
- "user": null
+ "commit": {
+ "author_email": "admin@example.com",
+ "author_name": "Administrator",
+ "created_at": "2015-12-24T16:51:14.000+01:00",
+ "id": "0ff3ae198f8601a285adcf5c0fff204ee6fba5fd",
+ "message": "Test the CI integration.",
+ "short_id": "0ff3ae19",
+ "title": "Test the CI integration."
+ },
+ "coverage": null,
+ "download_url": null,
+ "id": 69,
+ "name": "rubocop",
+ "ref": "master",
+ "runner": null,
+ "stage": "test",
+ "created_at": "2016-01-11T10:13:33.506Z",
+ "started_at": "2016-01-11T10:13:33.506Z",
+ "finished_at": "2016-01-11T10:15:10.506Z",
+ "status": "failed",
+ "tag": false,
+ "user": null
+}
+```
+
+## Keep artifacts
+
+Prevents artifacts from being deleted when expiration is set.
+
+```
+POST /projects/:id/builds/:build_id/artifacts/keep
+```
+
+Parameters
+
+| Attribute | Type | required | Description |
+|-------------|---------|----------|---------------------|
+| `id` | integer | yes | The ID of a project |
+| `build_id` | integer | yes | The ID of a build |
+
+Example request:
+
+```
+curl -X POST -H "PRIVATE-TOKEN: 9koXpg98eAheJpvBs5tK" "https://gitlab.example.com/api/v3/projects/1/builds/1/artifacts/keep"
+```
+
+Example response:
+
+```json
+{
+ "commit": {
+ "author_email": "admin@example.com",
+ "author_name": "Administrator",
+ "created_at": "2015-12-24T16:51:14.000+01:00",
+ "id": "0ff3ae198f8601a285adcf5c0fff204ee6fba5fd",
+ "message": "Test the CI integration.",
+ "short_id": "0ff3ae19",
+ "title": "Test the CI integration."
+ },
+ "coverage": null,
+ "download_url": null,
+ "id": 69,
+ "name": "rubocop",
+ "ref": "master",
+ "runner": null,
+ "stage": "test",
+ "created_at": "2016-01-11T10:13:33.506Z",
+ "started_at": "2016-01-11T10:13:33.506Z",
+ "finished_at": "2016-01-11T10:15:10.506Z",
+ "status": "failed",
+ "tag": false,
+ "user": null
}
```
diff --git a/doc/api/ci/README.md b/doc/api/ci/README.md
new file mode 100644
index 00000000000..96a281e27c8
--- /dev/null
+++ b/doc/api/ci/README.md
@@ -0,0 +1,24 @@
+# GitLab CI API
+
+## Purpose
+
+The main purpose of GitLab CI API is to provide the necessary data and context
+for GitLab CI Runners.
+
+All relevant information about the consumer API can be found in a
+[separate document](../../api/README.md).
+
+## API Prefix
+
+The current CI API prefix is `/ci/api/v1`.
+
+You need to prepend this prefix to all examples in this documentation, like:
+
+```bash
+GET /ci/api/v1/builds/:id/artifacts
+```
+
+## Resources
+
+- [Builds](builds.md)
+- [Runners](runners.md)
diff --git a/doc/api/ci/builds.md b/doc/api/ci/builds.md
new file mode 100644
index 00000000000..d779463fd8c
--- /dev/null
+++ b/doc/api/ci/builds.md
@@ -0,0 +1,138 @@
+# Builds API
+
+API used by runners to receive and update builds.
+
+>**Note:**
+This API is intended to be used only by Runners as their own
+communication channel. For the consumer API see the
+[Builds API](../builds.md).
+
+## Authentication
+
+This API uses two types of authentication:
+
+1. Unique Runner's token which is the token assigned to the Runner after it
+ has been registered.
+
+2. Using the build authorization token.
+ This is project's CI token that can be found under the **Builds** section of
+ a project's settings. The build authorization token can be passed as a
+ parameter or a value of `BUILD-TOKEN` header.
+
+These two methods of authentication are interchangeable.
+
+## Builds
+
+### Runs oldest pending build by runner
+
+```
+POST /ci/api/v1/builds/register
+```
+
+| Attribute | Type | Required | Description |
+|-----------|---------|----------|---------------------|
+| `token` | string | yes | Unique runner token |
+
+
+```
+curl -X POST "https://gitlab.example.com/ci/api/v1/builds/register" -F "token=t0k3n"
+```
+
+### Update details of an existing build
+
+```
+PUT /ci/api/v1/builds/:id
+```
+
+| Attribute | Type | Required | Description |
+|-----------|---------|----------|----------------------|
+| `id` | integer | yes | The ID of a project |
+| `token` | string | yes | Unique runner token |
+| `state` | string | no | The state of a build |
+| `trace` | string | no | The trace of a build |
+
+```
+curl -X PUT "https://gitlab.example.com/ci/api/v1/builds/1234" -F "token=t0k3n" -F "state=running" -F "trace=Running git clone...\n"
+```
+
+### Incremental build trace update
+
+Using this method you need to send trace content as a request body. You also need to provide the `Content-Range` header
+with a range of sent trace part. Note that you need to send parts in the proper order, so the begining of the part
+must start just after the end of the previous part. If you provide the wrong part, then GitLab CI API will return `416
+Range Not Satisfiable` response with a header `Range: 0-X`, where `X` is the current trace length.
+
+For example, if you receive `Range: 0-11` in the response, then your next part must contain a `Content-Range: 11-...`
+header and a trace part covered by this range.
+
+For a valid update API will return `202` response with:
+* `Build-Status: {status}` header containing current status of the build,
+* `Range: 0-{length}` header with the current trace length.
+
+```
+PATCH /ci/api/v1/builds/:id/trace.txt
+```
+
+Parameters:
+
+| Attribute | Type | Required | Description |
+|-----------|---------|----------|----------------------|
+| `id` | integer | yes | The ID of a build |
+
+Headers:
+
+| Attribute | Type | Required | Description |
+|-----------------|---------|----------|-----------------------------------|
+| `BUILD-TOKEN` | string | yes | The build authorization token |
+| `Content-Range` | string | yes | Bytes range of trace that is sent |
+
+```
+curl -X PATCH "https://gitlab.example.com/ci/api/v1/builds/1234/trace.txt" -H "BUILD-TOKEN=build_t0k3n" -H "Content-Range=0-21" -d "Running git clone...\n"
+```
+
+
+### Upload artifacts to build
+
+```
+POST /ci/api/v1/builds/:id/artifacts
+```
+
+| Attribute | Type | Required | Description |
+|-----------|---------|----------|-------------------------------|
+| `id` | integer | yes | The ID of a build |
+| `token` | string | yes | The build authorization token |
+| `file` | mixed | yes | Artifacts file |
+
+```
+curl -X POST "https://gitlab.example.com/ci/api/v1/builds/1234/artifacts" -F "token=build_t0k3n" -F "file=@/path/to/file"
+```
+
+### Download the artifacts file from build
+
+```
+GET /ci/api/v1/builds/:id/artifacts
+```
+
+| Attribute | Type | Required | Description |
+|-----------|---------|----------|-------------------------------|
+| `id` | integer | yes | The ID of a build |
+| `token` | string | yes | The build authorization token |
+
+```
+curl "https://gitlab.example.com/ci/api/v1/builds/1234/artifacts" -F "token=build_t0k3n"
+```
+
+### Remove the artifacts file from build
+
+```
+DELETE /ci/api/v1/builds/:id/artifacts
+```
+
+| Attribute | Type | Required | Description |
+|-----------|---------|----------|-------------------------------|
+| ` id` | integer | yes | The ID of a build |
+| `token` | string | yes | The build authorization token |
+
+```
+curl -X DELETE "https://gitlab.example.com/ci/api/v1/builds/1234/artifacts" -F "token=build_t0k3n"
+```
diff --git a/doc/api/ci/runners.md b/doc/api/ci/runners.md
new file mode 100644
index 00000000000..96b3c42f773
--- /dev/null
+++ b/doc/api/ci/runners.md
@@ -0,0 +1,57 @@
+# Runners API
+
+API used by Runners to register and delete themselves.
+
+>**Note:**
+This API is intended to be used only by Runners as their own
+communication channel. For the consumer API see the
+[new Runners API](../runners.md).
+
+## Authentication
+
+This API uses two types of authentication:
+
+1. Unique Runner's token, which is the token assigned to the Runner after it
+ has been registered.
+
+2. Using Runners' registration token.
+ This is a token that can be found in project's settings.
+ It can also be found in the **Admin > Runners** settings area.
+ There are two types of tokens you can pass: shared Runner registration
+ token or project specific registration token.
+
+## Register a new runner
+
+Used to make GitLab CI aware of available runners.
+
+```sh
+POST /ci/api/v1/runners/register
+```
+
+| Attribute | Type | Required | Description |
+| --------- | ------- | --------- | ----------- |
+| `token` | string | yes | Runner's registration token |
+
+Example request:
+
+```sh
+curl -X POST "https://gitlab.example.com/ci/api/v1/runners/register" -F "token=t0k3n"
+```
+
+## Delete a Runner
+
+Used to remove a Runner.
+
+```sh
+DELETE /ci/api/v1/runners/delete
+```
+
+| Attribute | Type | Required | Description |
+| --------- | ------- | --------- | ----------- |
+| `token` | string | yes | Runner's registration token |
+
+Example request:
+
+```sh
+curl -X DELETE "https://gitlab.example.com/ci/api/v1/runners/delete" -F "token=t0k3n"
+```
diff --git a/doc/api/commits.md b/doc/api/commits.md
index 6341440c58b..57c2e1d9b87 100644
--- a/doc/api/commits.md
+++ b/doc/api/commits.md
@@ -12,6 +12,8 @@ GET /projects/:id/repository/commits
| --------- | ---- | -------- | ----------- |
| `id` | integer | yes | The ID of a project |
| `ref_name` | string | no | The name of a repository branch or tag or if not given the default branch |
+| `since` | string | no | Only commits after or in this date will be returned in ISO 8601 format YYYY-MM-DDTHH:MM:SSZ |
+| `until` | string | no | Only commits before or in this date will be returned in ISO 8601 format YYYY-MM-DDTHH:MM:SSZ |
```bash
curl -H "PRIVATE-TOKEN: 9koXpg98eAheJpvBs5tK" "https://gitlab.example.com/api/v3/projects/5/repository/commits"
diff --git a/doc/api/groups.md b/doc/api/groups.md
index d47e79ba47f..1ccb9715e96 100644
--- a/doc/api/groups.md
+++ b/doc/api/groups.md
@@ -111,6 +111,7 @@ Parameters:
- `name` (required) - The name of the group
- `path` (required) - The path of the group
- `description` (optional) - The group's description
+- `visibility_level` (optional) - The group's visibility. 0 for private, 10 for internal, 20 for public.
## Transfer project to group
@@ -125,6 +126,87 @@ Parameters:
- `id` (required) - The ID or path of a group
- `project_id` (required) - The ID of a project
+## Update group
+
+Updates the project group. Only available to group owners and administrators.
+
+```
+PUT /groups/:id
+```
+
+| Attribute | Type | Required | Description |
+| --------- | ---- | -------- | ----------- |
+| `id` | integer | yes | The ID of the group |
+| `name` | string | no | The name of the group |
+| `path` | string | no | The path of the group |
+| `description` | string | no | The description of the group |
+| `visibility_level` | integer | no | The visibility level of the group. 0 for private, 10 for internal, 20 for public. |
+
+```bash
+curl -X PUT -H "PRIVATE-TOKEN: 9koXpg98eAheJpvBs5tK" "https://gitlab.example.com/api/v3/groups/5?name=Experimental"
+
+```
+
+Example response:
+
+```json
+{
+ "id": 5,
+ "name": "Experimental",
+ "path": "h5bp",
+ "description": "foo",
+ "visibility_level": 10,
+ "avatar_url": null,
+ "web_url": "http://gitlab.example.com/groups/h5bp",
+ "projects": [
+ {
+ "id": 9,
+ "description": "foo",
+ "default_branch": "master",
+ "tag_list": [],
+ "public": false,
+ "archived": false,
+ "visibility_level": 10,
+ "ssh_url_to_repo": "git@gitlab.example.com/html5-boilerplate.git",
+ "http_url_to_repo": "http://gitlab.example.com/h5bp/html5-boilerplate.git",
+ "web_url": "http://gitlab.example.com/h5bp/html5-boilerplate",
+ "name": "Html5 Boilerplate",
+ "name_with_namespace": "Experimental / Html5 Boilerplate",
+ "path": "html5-boilerplate",
+ "path_with_namespace": "h5bp/html5-boilerplate",
+ "issues_enabled": true,
+ "merge_requests_enabled": true,
+ "wiki_enabled": true,
+ "builds_enabled": true,
+ "snippets_enabled": true,
+ "created_at": "2016-04-05T21:40:50.169Z",
+ "last_activity_at": "2016-04-06T16:52:08.432Z",
+ "shared_runners_enabled": true,
+ "creator_id": 1,
+ "namespace": {
+ "id": 5,
+ "name": "Experimental",
+ "path": "h5bp",
+ "owner_id": null,
+ "created_at": "2016-04-05T21:40:49.152Z",
+ "updated_at": "2016-04-07T08:07:48.466Z",
+ "description": "foo",
+ "avatar": {
+ "url": null
+ },
+ "share_with_group_lock": false,
+ "visibility_level": 10
+ },
+ "avatar_url": null,
+ "star_count": 1,
+ "forks_count": 0,
+ "open_issues_count": 3,
+ "public_builds": true
+ }
+ ]
+}
+```
+
## Remove group
Removes group with all projects inside.
@@ -183,7 +265,6 @@ GET /groups/:id/members
{
"id": 1,
"username": "raymond_smith",
- "email": "ray@smith.org",
"name": "Raymond Smith",
"state": "active",
"created_at": "2012-10-22T14:13:35Z",
@@ -192,7 +273,6 @@ GET /groups/:id/members
{
"id": 2,
"username": "john_doe",
- "email": "joh@doe.org",
"name": "John Doe",
"state": "active",
"created_at": "2012-10-22T14:13:35Z",
diff --git a/doc/api/issues.md b/doc/api/issues.md
index 9e704648b25..0bc82ef9edb 100644
--- a/doc/api/issues.md
+++ b/doc/api/issues.md
@@ -76,8 +76,10 @@ Example response:
"title" : "Consequatur vero maxime deserunt laboriosam est voluptas dolorem.",
"created_at" : "2016-01-04T15:31:51.081Z",
"iid" : 6,
- "labels" : []
- },
+ "labels" : [],
+ "subscribed" : false,
+ "user_notes_count": 1
+ }
]
```
@@ -152,7 +154,9 @@ Example response:
"id" : 41,
"title" : "Ut commodi ullam eos dolores perferendis nihil sunt.",
"updated_at" : "2016-01-04T15:31:46.176Z",
- "created_at" : "2016-01-04T15:31:46.176Z"
+ "created_at" : "2016-01-04T15:31:46.176Z",
+ "subscribed" : false,
+ "user_notes_count": 1
}
]
```
@@ -213,7 +217,9 @@ Example response:
"id" : 41,
"title" : "Ut commodi ullam eos dolores perferendis nihil sunt.",
"updated_at" : "2016-01-04T15:31:46.176Z",
- "created_at" : "2016-01-04T15:31:46.176Z"
+ "created_at" : "2016-01-04T15:31:46.176Z",
+ "subscribed": false,
+ "user_notes_count": 1
}
```
@@ -237,6 +243,7 @@ POST /projects/:id/issues
| `assignee_id` | integer | no | The ID of a user to assign issue |
| `milestone_id` | integer | no | The ID of a milestone to assign issue |
| `labels` | string | no | Comma-separated label names for an issue |
+| `created_at` | string | no | Date time string, ISO 8601 formatted, e.g. `2016-03-11T03:45:40Z` |
```bash
curl -X POST -H "PRIVATE-TOKEN: 9koXpg98eAheJpvBs5tK" https://gitlab.example.com/api/v3/projects/4/issues?title=Issues%20with%20auth&labels=bug
@@ -266,7 +273,9 @@ Example response:
},
"description" : null,
"updated_at" : "2016-01-07T12:44:33.959Z",
- "milestone" : null
+ "milestone" : null,
+ "subscribed" : true,
+ "user_notes_count": 0
}
```
@@ -293,6 +302,7 @@ PUT /projects/:id/issues/:issue_id
| `milestone_id` | integer | no | The ID of a milestone to assign the issue to |
| `labels` | string | no | Comma-separated label names for an issue |
| `state_event` | string | no | The state event of an issue. Set `close` to close the issue and `reopen` to reopen it |
+| `updated_at` | string | no | Date time string, ISO 8601 formatted, e.g. `2016-03-11T03:45:40Z` |
```bash
curl -X PUT -H "PRIVATE-TOKEN: 9koXpg98eAheJpvBs5tK" https://gitlab.example.com/api/v3/projects/4/issues/85?state_event=close
@@ -322,21 +332,198 @@ Example response:
],
"id" : 85,
"assignee" : null,
- "milestone" : null
+ "milestone" : null,
+ "subscribed" : true,
+ "user_notes_count": 0
}
```
-## Delete existing issue (**Deprecated**)
+## Delete an issue
-This call is deprecated and returns a `405 Method Not Allowed` error if called.
-An issue gets now closed and is done by calling
-`PUT /projects/:id/issues/:issue_id` with the parameter `state_event` set to
-`close`. See [edit issue](#edit-issue) for more details.
+Only for admins and project owners. Soft deletes the issue in question.
+If the operation is successful, a status code `200` is returned. In case you cannot
+destroy this issue, or it is not present, code `404` is given.
```
DELETE /projects/:id/issues/:issue_id
```
+| Attribute | Type | Required | Description |
+| --------- | ---- | -------- | ----------- |
+| `id` | integer | yes | The ID of a project |
+| `issue_id` | integer | yes | The ID of a project's issue |
+
+```bash
+curl -X DELETE -H "PRIVATE-TOKEN: 9koXpg98eAheJpvBs5tK" https://gitlab.example.com/api/v3/projects/4/issues/85
+```
+
+## Move an issue
+
+Moves an issue to a different project. If the operation is successful, a status
+code `201` together with moved issue is returned. If the project, issue, or
+target project is not found, error `404` is returned. If the target project
+equals the source project or the user has insufficient permissions to move an
+issue, error `400` together with an explaining error message is returned.
+
+If a given label and/or milestone with the same name also exists in the target
+project, it will then be assigned to the issue that is being moved.
+
+```
+POST /projects/:id/issues/:issue_id/move
+```
+
+| Attribute | Type | Required | Description |
+| --------- | ---- | -------- | ----------- |
+| `id` | integer | yes | The ID of a project |
+| `issue_id` | integer | yes | The ID of a project's issue |
+| `to_project_id` | integer | yes | The ID of the new project |
+
+```bash
+curl -X POST -H "PRIVATE-TOKEN: 9koXpg98eAheJpvBs5tK" https://gitlab.example.com/api/v3/projects/4/issues/85/move
+```
+
+Example response:
+
+```json
+{
+ "id": 92,
+ "iid": 11,
+ "project_id": 5,
+ "title": "Sit voluptas tempora quisquam aut doloribus et.",
+ "description": "Repellat voluptas quibusdam voluptatem exercitationem.",
+ "state": "opened",
+ "created_at": "2016-04-05T21:41:45.652Z",
+ "updated_at": "2016-04-07T12:20:17.596Z",
+ "labels": [],
+ "milestone": null,
+ "assignee": {
+ "name": "Miss Monserrate Beier",
+ "username": "axel.block",
+ "id": 12,
+ "state": "active",
+ "avatar_url": "http://www.gravatar.com/avatar/46f6f7dc858ada7be1853f7fb96e81da?s=80&d=identicon",
+ "web_url": "https://gitlab.example.com/u/axel.block"
+ },
+ "author": {
+ "name": "Kris Steuber",
+ "username": "solon.cremin",
+ "id": 10,
+ "state": "active",
+ "avatar_url": "http://www.gravatar.com/avatar/7a190fecbaa68212a4b68aeb6e3acd10?s=80&d=identicon",
+ "web_url": "https://gitlab.example.com/u/solon.cremin"
+ }
+}
+```
+
+## Subscribe to an issue
+
+Subscribes the authenticated user to an issue to receive notifications. If the
+operation is successful, status code `201` together with the updated issue is
+returned. If the user is already subscribed to the issue, the status code `304`
+is returned. If the project or issue is not found, status code `404` is
+returned.
+
+```
+POST /projects/:id/issues/:issue_id/subscription
+```
+
+| Attribute | Type | Required | Description |
+| --------- | ---- | -------- | ----------- |
+| `id` | integer | yes | The ID of a project |
+| `issue_id` | integer | yes | The ID of a project's issue |
+
+```bash
+curl -X POST -H "PRIVATE-TOKEN: 9koXpg98eAheJpvBs5tK" https://gitlab.example.com/api/v3/projects/5/issues/93/subscription
+```
+
+Example response:
+
+```json
+{
+ "id": 92,
+ "iid": 11,
+ "project_id": 5,
+ "title": "Sit voluptas tempora quisquam aut doloribus et.",
+ "description": "Repellat voluptas quibusdam voluptatem exercitationem.",
+ "state": "opened",
+ "created_at": "2016-04-05T21:41:45.652Z",
+ "updated_at": "2016-04-07T12:20:17.596Z",
+ "labels": [],
+ "milestone": null,
+ "assignee": {
+ "name": "Miss Monserrate Beier",
+ "username": "axel.block",
+ "id": 12,
+ "state": "active",
+ "avatar_url": "http://www.gravatar.com/avatar/46f6f7dc858ada7be1853f7fb96e81da?s=80&d=identicon",
+ "web_url": "https://gitlab.example.com/u/axel.block"
+ },
+ "author": {
+ "name": "Kris Steuber",
+ "username": "solon.cremin",
+ "id": 10,
+ "state": "active",
+ "avatar_url": "http://www.gravatar.com/avatar/7a190fecbaa68212a4b68aeb6e3acd10?s=80&d=identicon",
+ "web_url": "https://gitlab.example.com/u/solon.cremin"
+ }
+}
+```
+
+## Unsubscribe from an issue
+
+Unsubscribes the authenticated user from the issue to not receive notifications
+from it. If the operation is successful, status code `200` together with the
+updated issue is returned. If the user is not subscribed to the issue, the
+status code `304` is returned. If the project or issue is not found, status code
+`404` is returned.
+
+```
+DELETE /projects/:id/issues/:issue_id/subscription
+```
+
+| Attribute | Type | Required | Description |
+| --------- | ---- | -------- | ----------- |
+| `id` | integer | yes | The ID of a project |
+| `issue_id` | integer | yes | The ID of a project's issue |
+
+```bash
+curl -X DELETE -H "PRIVATE-TOKEN: 9koXpg98eAheJpvBs5tK" https://gitlab.example.com/api/v3/projects/5/issues/93/subscription
+```
+
+Example response:
+
+```json
+{
+ "id": 93,
+ "iid": 12,
+ "project_id": 5,
+ "title": "Incidunt et rerum ea expedita iure quibusdam.",
+ "description": "Et cumque architecto sed aut ipsam.",
+ "state": "opened",
+ "created_at": "2016-04-05T21:41:45.217Z",
+ "updated_at": "2016-04-07T13:02:37.905Z",
+ "labels": [],
+ "milestone": null,
+ "assignee": {
+ "name": "Edwardo Grady",
+ "username": "keyon",
+ "id": 21,
+ "state": "active",
+ "avatar_url": "http://www.gravatar.com/avatar/3e6f06a86cf27fa8b56f3f74f7615987?s=80&d=identicon",
+ "web_url": "https://gitlab.example.com/u/keyon"
+ },
+ "author": {
+ "name": "Vivian Hermann",
+ "username": "orville",
+ "id": 11,
+ "state": "active",
+ "avatar_url": "http://www.gravatar.com/avatar/5224fd70153710e92fb8bcf79ac29d67?s=80&d=identicon",
+ "web_url": "http://lgitlab.example.com/u/orville"
+ },
+ "subscribed": false
+}
+```
+
## Comments on issues
Comments are done via the [notes](notes.md) resource.
diff --git a/doc/api/labels.md b/doc/api/labels.md
index 6496ffe9fd1..a181c0f57a2 100644
--- a/doc/api/labels.md
+++ b/doc/api/labels.md
@@ -8,9 +8,9 @@ Get all labels for a given project.
GET /projects/:id/labels
```
-| Attribute | Type | Required | Description |
-| --------- | ---- | -------- | ----------- |
-| `id` | integer | yes | The ID of the project |
+| Attribute | Type | Required | Description |
+| --------- | ------- | -------- | --------------------- |
+| `id` | integer | yes | The ID of the project |
```bash
curl -H "PRIVATE-TOKEN: 9koXpg98eAheJpvBs5tK" https://gitlab.example.com/api/v3/projects/1/labels
@@ -22,35 +22,43 @@ Example response:
[
{
"name" : "bug",
- "color" : "#d9534f"
+ "color" : "#d9534f",
+ "description": "Bug reported by user",
+ "open_issues_count": 1,
+ "closed_issues_count": 0,
+ "open_merge_requests_count": 1
},
{
"color" : "#d9534f",
- "name" : "confirmed"
+ "name" : "confirmed",
+ "description": "Confirmed issue",
+ "open_issues_count": 2,
+ "closed_issues_count": 5,
+ "open_merge_requests_count": 0
},
{
"name" : "critical",
- "color" : "#d9534f"
- },
- {
- "color" : "#428bca",
- "name" : "discussion"
+ "color" : "#d9534f",
+ "description": "Critical issue. Need fix ASAP",
+ "open_issues_count": 1,
+ "closed_issues_count": 3,
+ "open_merge_requests_count": 1
},
{
"name" : "documentation",
- "color" : "#f0ad4e"
+ "color" : "#f0ad4e",
+ "description": "Issue about documentation",
+ "open_issues_count": 1,
+ "closed_issues_count": 0,
+ "open_merge_requests_count": 2
},
{
"color" : "#5cb85c",
- "name" : "enhancement"
- },
- {
- "color" : "#428bca",
- "name" : "suggestion"
- },
- {
- "color" : "#f0ad4e",
- "name" : "support"
+ "name" : "enhancement",
+ "description": "Enhancement proposal",
+ "open_issues_count": 1,
+ "closed_issues_count": 0,
+ "open_merge_requests_count": 1
}
]
```
@@ -66,11 +74,12 @@ and 409 if the label already exists.
POST /projects/:id/labels
```
-| Attribute | Type | Required | Description |
-| --------- | ---- | -------- | ----------- |
-| `id` | integer | yes | The ID of the project |
-| `name` | string | yes | The name of the label |
-| `color` | string | yes | The color of the label in 6-digit hex notation with leading `#` sign |
+| Attribute | Type | Required | Description |
+| ------------- | ------- | -------- | ---------------------------- |
+| `id` | integer | yes | The ID of the project |
+| `name` | string | yes | The name of the label |
+| `color` | string | yes | The color of the label in 6-digit hex notation with leading `#` sign |
+| `description` | string | no | The description of the label |
```bash
curl --data "name=feature&color=#5843AD" -H "PRIVATE-TOKEN: 9koXpg98eAheJpvBs5tK" "https://gitlab.example.com/api/v3/projects/1/labels"
@@ -81,7 +90,8 @@ Example response:
```json
{
"name" : "feature",
- "color" : "#5843AD"
+ "color" : "#5843AD",
+ "description":null
}
```
@@ -97,10 +107,10 @@ In case of an error, an additional error message is returned.
DELETE /projects/:id/labels
```
-| Attribute | Type | Required | Description |
-| --------- | ---- | -------- | ----------- |
-| `id` | integer | yes | The ID of the project |
-| `name` | string | yes | The name of the label |
+| Attribute | Type | Required | Description |
+| --------- | ------- | -------- | --------------------- |
+| `id` | integer | yes | The ID of the project |
+| `name` | string | yes | The name of the label |
```bash
curl -X DELETE -H "PRIVATE-TOKEN: 9koXpg98eAheJpvBs5tK" "https://gitlab.example.com/api/v3/projects/1/labels?name=bug"
@@ -112,6 +122,7 @@ Example response:
{
"title" : "feature",
"color" : "#5843AD",
+ "description": "New feature proposal",
"updated_at" : "2015-11-03T21:22:30.737Z",
"template" : false,
"project_id" : 1,
@@ -133,15 +144,16 @@ In case of an error, an additional error message is returned.
PUT /projects/:id/labels
```
-| Attribute | Type | Required | Description |
-| --------- | ---- | -------- | ----------- |
-| `id` | integer | yes | The ID of the project |
-| `name` | string | yes | The name of the existing label |
-| `new_name` | string | yes if `color` if not provided | The new name of the label |
-| `color` | string | yes if `new_name` is not provided | The new color of the label in 6-digit hex notation with leading `#` sign |
+| Attribute | Type | Required | Description |
+| --------------- | ------- | --------------------------------- | ------------------------------- |
+| `id` | integer | yes | The ID of the project |
+| `name` | string | yes | The name of the existing label |
+| `new_name` | string | yes if `color` if not provided | The new name of the label |
+| `color` | string | yes if `new_name` is not provided | The new color of the label in 6-digit hex notation with leading `#` sign |
+| `description` | string | no | The new description of the label |
```bash
-curl -X PUT --data "name=documentation&new_name=docs&color=#8E44AD" -H "PRIVATE-TOKEN: 9koXpg98eAheJpvBs5tK" "https://gitlab.example.com/api/v3/projects/1/labels"
+curl -X PUT --data "name=documentation&new_name=docs&color=#8E44AD&description=Documentation" -H "PRIVATE-TOKEN: 9koXpg98eAheJpvBs5tK" "https://gitlab.example.com/api/v3/projects/1/labels"
```
Example response:
@@ -149,6 +161,77 @@ Example response:
```json
{
"color" : "#8E44AD",
- "name" : "docs"
+ "name" : "docs",
+ "description": "Documentation"
+}
+```
+
+## Subscribe to a label
+
+Subscribes the authenticated user to a label to receive notifications. If the
+operation is successful, status code `201` together with the updated label is
+returned. If the user is already subscribed to the label, the status code `304`
+is returned. If the project or label is not found, status code `404` is
+returned.
+
+```
+POST /projects/:id/labels/:label_id/subscription
+```
+
+| Attribute | Type | Required | Description |
+| ---------- | ----------------- | -------- | ------------------------------------ |
+| `id` | integer | yes | The ID of a project |
+| `label_id` | integer or string | yes | The ID or title of a project's label |
+
+```bash
+curl -X POST -H "PRIVATE-TOKEN: 9koXpg98eAheJpvBs5tK" https://gitlab.example.com/api/v3/projects/5/labels/1/subscription
+```
+
+Example response:
+
+```json
+{
+ "name": "Docs",
+ "color": "#cc0033",
+ "description": "",
+ "open_issues_count": 0,
+ "closed_issues_count": 0,
+ "open_merge_requests_count": 0,
+ "subscribed": true
+}
+```
+
+## Unsubscribe from a label
+
+Unsubscribes the authenticated user from a label to not receive notifications
+from it. If the operation is successful, status code `200` together with the
+updated label is returned. If the user is not subscribed to the label, the
+status code `304` is returned. If the project or label is not found, status code
+`404` is returned.
+
+```
+DELETE /projects/:id/labels/:label_id/subscription
+```
+
+| Attribute | Type | Required | Description |
+| ---------- | ----------------- | -------- | ------------------------------------ |
+| `id` | integer | yes | The ID of a project |
+| `label_id` | integer or string | yes | The ID or title of a project's label |
+
+```bash
+curl -X DELETE -H "PRIVATE-TOKEN: 9koXpg98eAheJpvBs5tK" https://gitlab.example.com/api/v3/projects/5/labels/1/subscription
+```
+
+Example response:
+
+```json
+{
+ "name": "Docs",
+ "color": "#cc0033",
+ "description": "",
+ "open_issues_count": 0,
+ "closed_issues_count": 0,
+ "open_merge_requests_count": 0,
+ "subscribed": false
}
```
diff --git a/doc/api/licenses.md b/doc/api/licenses.md
new file mode 100644
index 00000000000..855b0eab56f
--- /dev/null
+++ b/doc/api/licenses.md
@@ -0,0 +1,147 @@
+# Licenses
+
+## List license templates
+
+Get all license templates.
+
+```
+GET /licenses
+```
+
+| Attribute | Type | Required | Description |
+| --------- | ------- | -------- | --------------------- |
+| `popular` | boolean | no | If passed, returns only popular licenses |
+
+```bash
+curl https://gitlab.example.com/api/v3/licenses?popular=1
+```
+
+Example response:
+
+```json
+[
+ {
+ "key": "apache-2.0",
+ "name": "Apache License 2.0",
+ "nickname": null,
+ "featured": true,
+ "html_url": "http://choosealicense.com/licenses/apache-2.0/",
+ "source_url": "http://www.apache.org/licenses/LICENSE-2.0.html",
+ "description": "A permissive license that also provides an express grant of patent rights from contributors to users.",
+ "conditions": [
+ "include-copyright",
+ "document-changes"
+ ],
+ "permissions": [
+ "commercial-use",
+ "modifications",
+ "distribution",
+ "patent-use",
+ "private-use"
+ ],
+ "limitations": [
+ "trademark-use",
+ "no-liability"
+ ],
+ "content": " Apache License\n Version 2.0, January 2004\n [...]"
+ },
+ {
+ "key": "gpl-3.0",
+ "name": "GNU General Public License v3.0",
+ "nickname": "GNU GPLv3",
+ "featured": true,
+ "html_url": "http://choosealicense.com/licenses/gpl-3.0/",
+ "source_url": "http://www.gnu.org/licenses/gpl-3.0.txt",
+ "description": "The GNU GPL is the most widely used free software license and has a strong copyleft requirement. When distributing derived works, the source code of the work must be made available under the same license.",
+ "conditions": [
+ "include-copyright",
+ "document-changes",
+ "disclose-source",
+ "same-license"
+ ],
+ "permissions": [
+ "commercial-use",
+ "modifications",
+ "distribution",
+ "patent-use",
+ "private-use"
+ ],
+ "limitations": [
+ "no-liability"
+ ],
+ "content": " GNU GENERAL PUBLIC LICENSE\n Version 3, 29 June 2007\n [...]"
+ },
+ {
+ "key": "mit",
+ "name": "MIT License",
+ "nickname": null,
+ "featured": true,
+ "html_url": "http://choosealicense.com/licenses/mit/",
+ "source_url": "http://opensource.org/licenses/MIT",
+ "description": "A permissive license that is short and to the point. It lets people do anything with your code with proper attribution and without warranty.",
+ "conditions": [
+ "include-copyright"
+ ],
+ "permissions": [
+ "commercial-use",
+ "modifications",
+ "distribution",
+ "private-use"
+ ],
+ "limitations": [
+ "no-liability"
+ ],
+ "content": "The MIT License (MIT)\n\nCopyright (c) [year] [fullname]\n [...]"
+ }
+]
+```
+
+## Single license template
+
+Get a single license template. You can pass parameters to replace the license
+placeholder.
+
+```
+GET /licenses/:key
+```
+
+| Attribute | Type | Required | Description |
+| ---------- | ------ | -------- | ----------- |
+| `key` | string | yes | The key of the license template |
+| `project` | string | no | The copyrighted project name |
+| `fullname` | string | no | The full-name of the copyright holder |
+
+>**Note:**
+If you omit the `fullname` parameter but authenticate your request, the name of
+the authenticated user will be used to replace the copyright holder placeholder.
+
+```bash
+curl -H "PRIVATE-TOKEN: 9koXpg98eAheJpvBs5tK" https://gitlab.example.com/api/v3/licenses/mit?project=My+Cool+Project
+```
+
+Example response:
+
+```json
+{
+ "key": "mit",
+ "name": "MIT License",
+ "nickname": null,
+ "featured": true,
+ "html_url": "http://choosealicense.com/licenses/mit/",
+ "source_url": "http://opensource.org/licenses/MIT",
+ "description": "A permissive license that is short and to the point. It lets people do anything with your code with proper attribution and without warranty.",
+ "conditions": [
+ "include-copyright"
+ ],
+ "permissions": [
+ "commercial-use",
+ "modifications",
+ "distribution",
+ "private-use"
+ ],
+ "limitations": [
+ "no-liability"
+ ],
+ "content": "The MIT License (MIT)\n\nCopyright (c) 2016 John Doe\n [...]"
+}
+```
diff --git a/doc/api/merge_requests.md b/doc/api/merge_requests.md
index 5c527d55481..2930f615fc1 100644
--- a/doc/api/merge_requests.md
+++ b/doc/api/merge_requests.md
@@ -66,7 +66,9 @@ Parameters:
"due_date": null
},
"merge_when_build_succeeds": true,
- "merge_status": "can_be_merged"
+ "merge_status": "can_be_merged",
+ "subscribed" : false,
+ "user_notes_count": 1
}
]
```
@@ -128,7 +130,9 @@ Parameters:
"due_date": null
},
"merge_when_build_succeeds": true,
- "merge_status": "can_be_merged"
+ "merge_status": "can_be_merged",
+ "subscribed" : true,
+ "user_notes_count": 1
}
```
@@ -227,6 +231,8 @@ Parameters:
},
"merge_when_build_succeeds": true,
"merge_status": "can_be_merged",
+ "subscribed" : true,
+ "user_notes_count": 1,
"changes": [
{
"old_path": "VERSION",
@@ -304,7 +310,9 @@ Parameters:
"due_date": null
},
"merge_when_build_succeeds": true,
- "merge_status": "can_be_merged"
+ "merge_status": "can_be_merged",
+ "subscribed" : true,
+ "user_notes_count": 0
}
```
@@ -373,22 +381,45 @@ Parameters:
"due_date": null
},
"merge_when_build_succeeds": true,
- "merge_status": "can_be_merged"
+ "merge_status": "can_be_merged",
+ "subscribed" : true,
+ "user_notes_count": 1
}
```
If the operation is successful, 200 and the updated merge request is returned.
If an error occurs, an error number and a message explaining the reason is returned.
+## Delete a merge request
+
+Only for admins and project owners. Soft deletes the merge request in question.
+If the operation is successful, a status code `200` is returned. In case you cannot
+destroy this merge request, or it is not present, code `404` is given.
+
+```
+DELETE /projects/:id/merge_requests/:merge_request_id
+```
+
+| Attribute | Type | Required | Description |
+| --------- | ---- | -------- | ----------- |
+| `id` | integer | yes | The ID of a project |
+| `merge_request_id` | integer | yes | The ID of a project's merge request |
+
+```bash
+curl -X DELETE -H "PRIVATE-TOKEN: 9koXpg98eAheJpvBs5tK" https://gitlab.example.com/api/v3/projects/4/merge_request/85
+```
+
## Accept MR
Merge changes submitted with MR using this API.
-If merge success you get `200 OK`.
+If the merge succeeds you'll get a `200 OK`.
+
+If it has some conflicts and can not be merged - you'll get a 405 and the error message 'Branch cannot be merged'
-If it has some conflicts and can not be merged - you get 405 and error message 'Branch cannot be merged'
+If merge request is already merged or closed - you'll get a 406 and the error message 'Method Not Allowed'
-If merge request is already merged or closed - you get 405 and error message 'Method Not Allowed'
+If the `sha` parameter is passed and does not match the HEAD of the source - you'll get a 409 and the error message 'SHA does not match HEAD of source branch'
If you don't have permissions to accept this merge request - you'll get a 401
@@ -402,7 +433,8 @@ Parameters:
- `merge_request_id` (required) - ID of MR
- `merge_commit_message` (optional) - Custom merge commit message
- `should_remove_source_branch` (optional) - if `true` removes the source branch
-- `merged_when_build_succeeds` (optional) - if `true` the MR is merge when the build succeeds
+- `merged_when_build_succeeds` (optional) - if `true` the MR is merged when the build succeeds
+- `sha` (optional) - if present, then this SHA must match the HEAD of the source branch, otherwise the merge will fail
```json
{
@@ -447,7 +479,9 @@ Parameters:
"due_date": null
},
"merge_when_build_succeeds": true,
- "merge_status": "can_be_merged"
+ "merge_status": "can_be_merged",
+ "subscribed" : true,
+ "user_notes_count": 1
}
```
@@ -511,7 +545,9 @@ Parameters:
"due_date": null
},
"merge_when_build_succeeds": true,
- "merge_status": "can_be_merged"
+ "merge_status": "can_be_merged",
+ "subscribed" : true,
+ "user_notes_count": 1
}
```
@@ -536,7 +572,7 @@ GET /projects/:id/merge_requests/:merge_request_id/closes_issues
curl -H "PRIVATE-TOKEN: 9koXpg98eAheJpvBs5tK" https://gitlab.example.com/api/v3/projects/76/merge_requests/1/closes_issues
```
-Example response:
+Example response when the GitLab issue tracker is used:
```json
[
@@ -576,7 +612,167 @@ Example response:
"title" : "Consequatur vero maxime deserunt laboriosam est voluptas dolorem.",
"created_at" : "2016-01-04T15:31:51.081Z",
"iid" : 6,
- "labels" : []
+ "labels" : [],
+ "user_notes_count": 1
},
]
```
+
+Example response when an external issue tracker (e.g. JIRA) is used:
+
+```json
+[
+ {
+ "id" : "PROJECT-123",
+ "title" : "Title of this issue"
+ }
+]
+```
+
+## Subscribe to a merge request
+
+Subscribes the authenticated user to a merge request to receive notification. If
+the operation is successful, status code `201` together with the updated merge
+request is returned. If the user is already subscribed to the merge request, the
+status code `304` is returned. If the project or merge request is not found,
+status code `404` is returned.
+
+```
+POST /projects/:id/merge_requests/:merge_request_id/subscription
+```
+
+| Attribute | Type | Required | Description |
+| --------- | ---- | -------- | ----------- |
+| `id` | integer | yes | The ID of a project |
+| `merge_request_id` | integer | yes | The ID of the merge request |
+
+```bash
+curl -X POST -H "PRIVATE-TOKEN: 9koXpg98eAheJpvBs5tK" https://gitlab.example.com/api/v3/projects/5/merge_requests/17/subscription
+```
+
+Example response:
+
+```json
+{
+ "id": 17,
+ "iid": 1,
+ "project_id": 5,
+ "title": "Et et sequi est impedit nulla ut rem et voluptatem.",
+ "description": "Consequatur velit eos rerum optio autem. Quia id officia quaerat dolorum optio. Illo laudantium aut ipsum dolorem.",
+ "state": "opened",
+ "created_at": "2016-04-05T21:42:23.233Z",
+ "updated_at": "2016-04-05T22:11:52.900Z",
+ "target_branch": "ui-dev-kit",
+ "source_branch": "version-1-9",
+ "upvotes": 0,
+ "downvotes": 0,
+ "author": {
+ "name": "Eileen Skiles",
+ "username": "leila",
+ "id": 19,
+ "state": "active",
+ "avatar_url": "http://www.gravatar.com/avatar/39ce4a2822cc896933ffbd68c1470e55?s=80&d=identicon",
+ "web_url": "https://gitlab.example.com/u/leila"
+ },
+ "assignee": {
+ "name": "Celine Wehner",
+ "username": "carli",
+ "id": 16,
+ "state": "active",
+ "avatar_url": "http://www.gravatar.com/avatar/f4cd5605b769dd2ce405a27c6e6f2684?s=80&d=identicon",
+ "web_url": "https://gitlab.example.com/u/carli"
+ },
+ "source_project_id": 5,
+ "target_project_id": 5,
+ "labels": [],
+ "work_in_progress": false,
+ "milestone": {
+ "id": 7,
+ "iid": 1,
+ "project_id": 5,
+ "title": "v2.0",
+ "description": "Corrupti eveniet et velit occaecati dolorem est rerum aut.",
+ "state": "closed",
+ "created_at": "2016-04-05T21:41:40.905Z",
+ "updated_at": "2016-04-05T21:41:40.905Z",
+ "due_date": null
+ },
+ "merge_when_build_succeeds": false,
+ "merge_status": "cannot_be_merged",
+ "subscribed": true
+}
+```
+
+## Unsubscribe from a merge request
+
+Unsubscribes the authenticated user from a merge request to not receive
+notifications from that merge request. If the operation is successful, status
+code `200` together with the updated merge request is returned. If the user is
+not subscribed to the merge request, the status code `304` is returned. If the
+project or merge request is not found, status code `404` is returned.
+
+```
+DELETE /projects/:id/merge_requests/:merge_request_id/subscription
+```
+
+| Attribute | Type | Required | Description |
+| --------- | ---- | -------- | ----------- |
+| `id` | integer | yes | The ID of a project |
+| `merge_request_id` | integer | yes | The ID of the merge request |
+
+```bash
+curl -X DELETE -H "PRIVATE-TOKEN: 9koXpg98eAheJpvBs5tK" https://gitlab.example.com/api/v3/projects/5/merge_requests/17/subscription
+```
+
+Example response:
+
+```json
+{
+ "id": 17,
+ "iid": 1,
+ "project_id": 5,
+ "title": "Et et sequi est impedit nulla ut rem et voluptatem.",
+ "description": "Consequatur velit eos rerum optio autem. Quia id officia quaerat dolorum optio. Illo laudantium aut ipsum dolorem.",
+ "state": "opened",
+ "created_at": "2016-04-05T21:42:23.233Z",
+ "updated_at": "2016-04-05T22:11:52.900Z",
+ "target_branch": "ui-dev-kit",
+ "source_branch": "version-1-9",
+ "upvotes": 0,
+ "downvotes": 0,
+ "author": {
+ "name": "Eileen Skiles",
+ "username": "leila",
+ "id": 19,
+ "state": "active",
+ "avatar_url": "http://www.gravatar.com/avatar/39ce4a2822cc896933ffbd68c1470e55?s=80&d=identicon",
+ "web_url": "https://gitlab.example.com/u/leila"
+ },
+ "assignee": {
+ "name": "Celine Wehner",
+ "username": "carli",
+ "id": 16,
+ "state": "active",
+ "avatar_url": "http://www.gravatar.com/avatar/f4cd5605b769dd2ce405a27c6e6f2684?s=80&d=identicon",
+ "web_url": "https://gitlab.example.com/u/carli"
+ },
+ "source_project_id": 5,
+ "target_project_id": 5,
+ "labels": [],
+ "work_in_progress": false,
+ "milestone": {
+ "id": 7,
+ "iid": 1,
+ "project_id": 5,
+ "title": "v2.0",
+ "description": "Corrupti eveniet et velit occaecati dolorem est rerum aut.",
+ "state": "closed",
+ "created_at": "2016-04-05T21:41:40.905Z",
+ "updated_at": "2016-04-05T21:41:40.905Z",
+ "due_date": null
+ },
+ "merge_when_build_succeeds": false,
+ "merge_status": "cannot_be_merged",
+ "subscribed": false
+}
+```
diff --git a/doc/api/milestones.md b/doc/api/milestones.md
index a6828728264..e4202025f80 100644
--- a/doc/api/milestones.md
+++ b/doc/api/milestones.md
@@ -7,8 +7,24 @@ Returns a list of project milestones.
```
GET /projects/:id/milestones
GET /projects/:id/milestones?iid=42
+GET /projects/:id/milestones?state=active
+GET /projects/:id/milestones?state=closed
```
+Parameters:
+
+| Attribute | Type | Required | Description |
+| --------- | ---- | -------- | ----------- |
+| `id` | integer | yes | The ID of a project |
+| `iid` | integer | optional | Return only the milestone having the given `iid` |
+| `state` | string | optional | Return only `active` or `closed` milestones` |
+
+```bash
+curl -H "PRIVATE-TOKEN: 9koXpg98eAheJpvBs5tK" https://gitlab.example.com/api/v3/projects/5/milestones
+```
+
+Example Response:
+
```json
[
{
@@ -25,10 +41,6 @@ GET /projects/:id/milestones?iid=42
]
```
-Parameters:
-
-- `id` (required) - The ID of a project
-- `iid` (optional) - Return the milestone having the given `iid`
## Get single milestone
diff --git a/doc/api/notes.md b/doc/api/notes.md
index d4d63e825ab..7aa1c2155bf 100644
--- a/doc/api/notes.md
+++ b/doc/api/notes.md
@@ -32,6 +32,7 @@ Parameters:
"created_at": "2013-09-30T13:46:01Z"
},
"created_at": "2013-10-02T09:22:45Z",
+ "updated_at": "2013-10-02T10:22:45Z",
"system": true,
"upvote": false,
"downvote": false,
@@ -51,6 +52,7 @@ Parameters:
"created_at": "2013-09-30T13:46:01Z"
},
"created_at": "2013-10-02T09:56:03Z",
+ "updated_at": "2013-10-02T09:56:03Z",
"system": true,
"upvote": false,
"downvote": false,
@@ -87,6 +89,7 @@ Parameters:
- `id` (required) - The ID of a project
- `issue_id` (required) - The ID of an issue
- `body` (required) - The content of a note
+- `created_at` (optional) - Date time string, ISO 8601 formatted, e.g. 2016-03-11T03:45:40Z
### Modify existing issue note
@@ -103,6 +106,53 @@ Parameters:
- `note_id` (required) - The ID of a note
- `body` (required) - The content of a note
+### Delete an issue note
+
+Deletes an existing note of an issue. On success, this API method returns 200
+and the deleted note. If the note does not exist, the API returns 404.
+
+```
+DELETE /projects/:id/issues/:issue_id/notes/:note_id
+```
+
+Parameters:
+
+| Attribute | Type | Required | Description |
+| --------- | ---- | -------- | ----------- |
+| `id` | integer | yes | The ID of a project |
+| `issue_id` | integer | yes | The ID of an issue |
+| `note_id` | integer | yes | The ID of a note |
+
+```bash
+curl -X DELETE -H "PRIVATE-TOKEN: 9koXpg98eAheJpvBs5tK" https://gitlab.example.com/api/v3/projects/5/issues/11/notes/636
+```
+
+Example Response:
+
+```json
+{
+ "id": 636,
+ "body": "This is a good idea.",
+ "attachment": null,
+ "author": {
+ "id": 1,
+ "username": "pipin",
+ "email": "admin@example.com",
+ "name": "Pip",
+ "state": "active",
+ "created_at": "2013-09-30T13:46:01Z",
+ "avatar_url": "http://www.gravatar.com/avatar/5224fd70153710e92fb8bcf79ac29d67?s=80&d=identicon",
+ "web_url": "https://gitlab.example.com/u/pipin"
+ },
+ "created_at": "2016-04-05T22:10:44.164Z",
+ "system": false,
+ "noteable_id": 11,
+ "noteable_type": "Issue",
+ "upvote": false,
+ "downvote": false
+}
+```
+
## Snippets
### List all snippet notes
@@ -180,6 +230,53 @@ Parameters:
- `note_id` (required) - The ID of a note
- `body` (required) - The content of a note
+### Delete a snippet note
+
+Deletes an existing note of a snippet. On success, this API method returns 200
+and the deleted note. If the note does not exist, the API returns 404.
+
+```
+DELETE /projects/:id/snippets/:snippet_id/notes/:note_id
+```
+
+Parameters:
+
+| Attribute | Type | Required | Description |
+| --------- | ---- | -------- | ----------- |
+| `id` | integer | yes | The ID of a project |
+| `snippet_id` | integer | yes | The ID of a snippet |
+| `note_id` | integer | yes | The ID of a note |
+
+```bash
+curl -X DELETE -H "PRIVATE-TOKEN: 9koXpg98eAheJpvBs5tK" https://gitlab.example.com/api/v3/projects/5/snippets/52/notes/1659
+```
+
+Example Response:
+
+```json
+{
+ "id": 1659,
+ "body": "This is a good idea.",
+ "attachment": null,
+ "author": {
+ "id": 1,
+ "username": "pipin",
+ "email": "admin@example.com",
+ "name": "Pip",
+ "state": "active",
+ "created_at": "2013-09-30T13:46:01Z",
+ "avatar_url": "http://www.gravatar.com/avatar/5224fd70153710e92fb8bcf79ac29d67?s=80&d=identicon",
+ "web_url": "https://gitlab.example.com/u/pipin"
+ },
+ "created_at": "2016-04-06T16:51:53.239Z",
+ "system": false,
+ "noteable_id": 52,
+ "noteable_type": "Snippet",
+ "upvote": false,
+ "downvote": false
+}
+```
+
## Merge Requests
### List all merge request notes
@@ -223,6 +320,7 @@ Parameters:
"created_at": "2013-09-30T13:46:01Z"
},
"created_at": "2013-10-02T08:57:14Z",
+ "updated_at": "2013-10-02T08:57:14Z",
"system": false,
"upvote": false,
"downvote": false,
@@ -259,3 +357,50 @@ Parameters:
- `merge_request_id` (required) - The ID of a merge request
- `note_id` (required) - The ID of a note
- `body` (required) - The content of a note
+
+### Delete a merge request note
+
+Deletes an existing note of a merge request. On success, this API method returns
+200 and the deleted note. If the note does not exist, the API returns 404.
+
+```
+DELETE /projects/:id/merge_requests/:merge_request_id/notes/:note_id
+```
+
+Parameters:
+
+| Attribute | Type | Required | Description |
+| --------- | ---- | -------- | ----------- |
+| `id` | integer | yes | The ID of a project |
+| `merge_request_id` | integer | yes | The ID of a merge request |
+| `note_id` | integer | yes | The ID of a note |
+
+```bash
+curl -X DELETE -H "PRIVATE-TOKEN: 9koXpg98eAheJpvBs5tK" https://gitlab.example.com/api/v3/projects/5/merge_requests/7/notes/1602
+```
+
+Example Response:
+
+```json
+{
+ "id": 1602,
+ "body": "This is a good idea.",
+ "attachment": null,
+ "author": {
+ "id": 1,
+ "username": "pipin",
+ "email": "admin@example.com",
+ "name": "Pip",
+ "state": "active",
+ "created_at": "2013-09-30T13:46:01Z",
+ "avatar_url": "http://www.gravatar.com/avatar/5224fd70153710e92fb8bcf79ac29d67?s=80&d=identicon",
+ "web_url": "https://gitlab.example.com/u/pipin"
+ },
+ "created_at": "2016-04-05T22:11:59.923Z",
+ "system": false,
+ "noteable_id": 7,
+ "noteable_type": "MergeRequest",
+ "upvote": false,
+ "downvote": false
+}
+```
diff --git a/doc/api/projects.md b/doc/api/projects.md
index 3703f4b327a..f5f195b97df 100644
--- a/doc/api/projects.md
+++ b/doc/api/projects.md
@@ -424,6 +424,7 @@ Parameters:
- `builds_enabled` (optional)
- `wiki_enabled` (optional)
- `snippets_enabled` (optional)
+- `container_registry_enabled` (optional)
- `public` (optional) - if `true` same as setting visibility_level = 20
- `visibility_level` (optional)
- `import_url` (optional)
@@ -447,6 +448,7 @@ Parameters:
- `builds_enabled` (optional)
- `wiki_enabled` (optional)
- `snippets_enabled` (optional)
+- `container_registry_enabled` (optional)
- `public` (optional) - if `true` same as setting visibility_level = 20
- `visibility_level` (optional)
- `import_url` (optional)
@@ -472,6 +474,7 @@ Parameters:
- `builds_enabled` (optional)
- `wiki_enabled` (optional)
- `snippets_enabled` (optional)
+- `container_registry_enabled` (optional)
- `public` (optional) - if `true` same as setting visibility_level = 20
- `visibility_level` (optional)
- `public_builds` (optional)
@@ -491,6 +494,298 @@ Parameters:
- `id` (required) - The ID of the project to be forked
+### Star a project
+
+Stars a given project. Returns status code `201` and the project on success and
+`304` if the project is already starred.
+
+```
+POST /projects/:id/star
+```
+
+| Attribute | Type | Required | Description |
+| --------- | ---- | -------- | ----------- |
+| `id` | integer | yes | The ID of the project |
+
+```bash
+curl -X POST -H "PRIVATE-TOKEN: 9koXpg98eAheJpvBs5tK" "https://gitlab.example.com/api/v3/projects/5/star"
+```
+
+Example response:
+
+```json
+{
+ "id": 3,
+ "description": null,
+ "default_branch": "master",
+ "public": false,
+ "visibility_level": 10,
+ "ssh_url_to_repo": "git@example.com:diaspora/diaspora-project-site.git",
+ "http_url_to_repo": "http://example.com/diaspora/diaspora-project-site.git",
+ "web_url": "http://example.com/diaspora/diaspora-project-site",
+ "tag_list": [
+ "example",
+ "disapora project"
+ ],
+ "name": "Diaspora Project Site",
+ "name_with_namespace": "Diaspora / Diaspora Project Site",
+ "path": "diaspora-project-site",
+ "path_with_namespace": "diaspora/diaspora-project-site",
+ "issues_enabled": true,
+ "open_issues_count": 1,
+ "merge_requests_enabled": true,
+ "builds_enabled": true,
+ "wiki_enabled": true,
+ "snippets_enabled": false,
+ "created_at": "2013-09-30T13: 46: 02Z",
+ "last_activity_at": "2013-09-30T13: 46: 02Z",
+ "creator_id": 3,
+ "namespace": {
+ "created_at": "2013-09-30T13: 46: 02Z",
+ "description": "",
+ "id": 3,
+ "name": "Diaspora",
+ "owner_id": 1,
+ "path": "diaspora",
+ "updated_at": "2013-09-30T13: 46: 02Z"
+ },
+ "archived": true,
+ "avatar_url": "http://example.com/uploads/project/avatar/3/uploads/avatar.png",
+ "shared_runners_enabled": true,
+ "forks_count": 0,
+ "star_count": 1
+}
+```
+
+### Unstar a project
+
+Unstars a given project. Returns status code `200` and the project on success
+and `304` if the project is not starred.
+
+```
+DELETE /projects/:id/star
+```
+
+| Attribute | Type | Required | Description |
+| --------- | ---- | -------- | ----------- |
+| `id` | integer | yes | The ID of the project |
+
+```bash
+curl -X DELETE -H "PRIVATE-TOKEN: 9koXpg98eAheJpvBs5tK" "https://gitlab.example.com/api/v3/projects/5/star"
+```
+
+Example response:
+
+```json
+{
+ "id": 3,
+ "description": null,
+ "default_branch": "master",
+ "public": false,
+ "visibility_level": 10,
+ "ssh_url_to_repo": "git@example.com:diaspora/diaspora-project-site.git",
+ "http_url_to_repo": "http://example.com/diaspora/diaspora-project-site.git",
+ "web_url": "http://example.com/diaspora/diaspora-project-site",
+ "tag_list": [
+ "example",
+ "disapora project"
+ ],
+ "name": "Diaspora Project Site",
+ "name_with_namespace": "Diaspora / Diaspora Project Site",
+ "path": "diaspora-project-site",
+ "path_with_namespace": "diaspora/diaspora-project-site",
+ "issues_enabled": true,
+ "open_issues_count": 1,
+ "merge_requests_enabled": true,
+ "builds_enabled": true,
+ "wiki_enabled": true,
+ "snippets_enabled": false,
+ "created_at": "2013-09-30T13: 46: 02Z",
+ "last_activity_at": "2013-09-30T13: 46: 02Z",
+ "creator_id": 3,
+ "namespace": {
+ "created_at": "2013-09-30T13: 46: 02Z",
+ "description": "",
+ "id": 3,
+ "name": "Diaspora",
+ "owner_id": 1,
+ "path": "diaspora",
+ "updated_at": "2013-09-30T13: 46: 02Z"
+ },
+ "archived": true,
+ "avatar_url": "http://example.com/uploads/project/avatar/3/uploads/avatar.png",
+ "shared_runners_enabled": true,
+ "forks_count": 0,
+ "star_count": 0
+}
+```
+
+### Archive a project
+
+Archives the project if the user is either admin or the project owner of this project. This action is
+idempotent, thus archiving an already archived project will not change the project.
+
+Status code 201 with the project as body is given when successful, in case the user doesn't
+have the proper access rights, code 403 is returned. Status 404 is returned if the project
+doesn't exist, or is hidden to the user.
+
+```
+POST /projects/:id/archive
+```
+
+| Attribute | Type | Required | Description |
+| --------- | ---- | -------- | ----------- |
+| `id` | integer | yes | The ID of the project |
+
+```bash
+curl -X POST -H "PRIVATE-TOKEN: 9koXpg98eAheJpvBs5tK" "https://gitlab.example.com/api/v3/projects/archive"
+```
+
+Example response:
+
+```json
+{
+ "id": 3,
+ "description": null,
+ "default_branch": "master",
+ "public": false,
+ "visibility_level": 0,
+ "ssh_url_to_repo": "git@example.com:diaspora/diaspora-project-site.git",
+ "http_url_to_repo": "http://example.com/diaspora/diaspora-project-site.git",
+ "web_url": "http://example.com/diaspora/diaspora-project-site",
+ "tag_list": [
+ "example",
+ "disapora project"
+ ],
+ "owner": {
+ "id": 3,
+ "name": "Diaspora",
+ "created_at": "2013-09-30T13: 46: 02Z"
+ },
+ "name": "Diaspora Project Site",
+ "name_with_namespace": "Diaspora / Diaspora Project Site",
+ "path": "diaspora-project-site",
+ "path_with_namespace": "diaspora/diaspora-project-site",
+ "issues_enabled": true,
+ "open_issues_count": 1,
+ "merge_requests_enabled": true,
+ "builds_enabled": true,
+ "wiki_enabled": true,
+ "snippets_enabled": false,
+ "created_at": "2013-09-30T13: 46: 02Z",
+ "last_activity_at": "2013-09-30T13: 46: 02Z",
+ "creator_id": 3,
+ "namespace": {
+ "created_at": "2013-09-30T13: 46: 02Z",
+ "description": "",
+ "id": 3,
+ "name": "Diaspora",
+ "owner_id": 1,
+ "path": "diaspora",
+ "updated_at": "2013-09-30T13: 46: 02Z"
+ },
+ "permissions": {
+ "project_access": {
+ "access_level": 10,
+ "notification_level": 3
+ },
+ "group_access": {
+ "access_level": 50,
+ "notification_level": 3
+ }
+ },
+ "archived": true,
+ "avatar_url": "http://example.com/uploads/project/avatar/3/uploads/avatar.png",
+ "shared_runners_enabled": true,
+ "forks_count": 0,
+ "star_count": 0,
+ "runners_token": "b8bc4a7a29eb76ea83cf79e4908c2b"
+}
+```
+
+### Unarchive a project
+
+Unarchives the project if the user is either admin or the project owner of this project. This action is
+idempotent, thus unarchiving an non-archived project will not change the project.
+
+Status code 201 with the project as body is given when successful, in case the user doesn't
+have the proper access rights, code 403 is returned. Status 404 is returned if the project
+doesn't exist, or is hidden to the user.
+
+```
+POST /projects/:id/archive
+```
+
+| Attribute | Type | Required | Description |
+| --------- | ---- | -------- | ----------- |
+| `id` | integer | yes | The ID of the project |
+
+```bash
+curl -X POST -H "PRIVATE-TOKEN: 9koXpg98eAheJpvBs5tK" "https://gitlab.example.com/api/v3/projects/unarchive"
+```
+
+Example response:
+
+```json
+{
+ "id": 3,
+ "description": null,
+ "default_branch": "master",
+ "public": false,
+ "visibility_level": 0,
+ "ssh_url_to_repo": "git@example.com:diaspora/diaspora-project-site.git",
+ "http_url_to_repo": "http://example.com/diaspora/diaspora-project-site.git",
+ "web_url": "http://example.com/diaspora/diaspora-project-site",
+ "tag_list": [
+ "example",
+ "disapora project"
+ ],
+ "owner": {
+ "id": 3,
+ "name": "Diaspora",
+ "created_at": "2013-09-30T13: 46: 02Z"
+ },
+ "name": "Diaspora Project Site",
+ "name_with_namespace": "Diaspora / Diaspora Project Site",
+ "path": "diaspora-project-site",
+ "path_with_namespace": "diaspora/diaspora-project-site",
+ "issues_enabled": true,
+ "open_issues_count": 1,
+ "merge_requests_enabled": true,
+ "builds_enabled": true,
+ "wiki_enabled": true,
+ "snippets_enabled": false,
+ "created_at": "2013-09-30T13: 46: 02Z",
+ "last_activity_at": "2013-09-30T13: 46: 02Z",
+ "creator_id": 3,
+ "namespace": {
+ "created_at": "2013-09-30T13: 46: 02Z",
+ "description": "",
+ "id": 3,
+ "name": "Diaspora",
+ "owner_id": 1,
+ "path": "diaspora",
+ "updated_at": "2013-09-30T13: 46: 02Z"
+ },
+ "permissions": {
+ "project_access": {
+ "access_level": 10,
+ "notification_level": 3
+ },
+ "group_access": {
+ "access_level": 50,
+ "notification_level": 3
+ }
+ },
+ "archived": false,
+ "avatar_url": "http://example.com/uploads/project/avatar/3/uploads/avatar.png",
+ "shared_runners_enabled": true,
+ "forks_count": 0,
+ "star_count": 0,
+ "runners_token": "b8bc4a7a29eb76ea83cf79e4908c2b"
+}
+```
+
### Remove project
Removes a project including all associated resources (issues, merge requests etc.)
@@ -614,8 +909,10 @@ Parameters:
- `id` (required) - The ID or NAMESPACE/PROJECT_NAME of a project
- `user_id` (required) - The ID of a team member
-This method is idempotent and can be called multiple times with the same parameters.
-Revoking team membership for a user who is not currently a team member is considered success.
+This method removes the project member if the user has the proper access rights to do so.
+It returns a status code 403 if the member does not have the proper rights to perform this action.
+In all other cases this method is idempotent and revoking team membership for a user who is not
+currently a team member is considered success.
Please note that the returned JSON currently differs slightly. Thus you should not
rely on the returned JSON structure.
diff --git a/doc/api/runners.md b/doc/api/runners.md
index cc6c6b7cb2f..ddfa298f79d 100644
--- a/doc/api/runners.md
+++ b/doc/api/runners.md
@@ -275,7 +275,7 @@ POST /projects/:id/runners
| `runner_id` | integer | yes | The ID of a runner |
```
-curl -X POST -H "PRIVATE-TOKEN: 9koXpg98eAheJpvBs5tK" "https://gitlab.example.com/api/v3/project/9/runners" -F "runner_id=9"
+curl -X POST -H "PRIVATE-TOKEN: 9koXpg98eAheJpvBs5tK" "https://gitlab.example.com/api/v3/projects/9/runners" -F "runner_id=9"
```
Example response:
@@ -306,7 +306,7 @@ DELETE /projects/:id/runners/:runner_id
| `runner_id` | integer | yes | The ID of a runner |
```
-curl -X DELETE -H "PRIVATE-TOKEN: 9koXpg98eAheJpvBs5tK" "https://gitlab.example.com/api/v3/project/9/runners/9"
+curl -X DELETE -H "PRIVATE-TOKEN: 9koXpg98eAheJpvBs5tK" "https://gitlab.example.com/api/v3/projects/9/runners/9"
```
Example response:
diff --git a/doc/api/services.md b/doc/api/services.md
index 7d45b2cf463..ccfc0fccb7f 100644
--- a/doc/api/services.md
+++ b/doc/api/services.md
@@ -16,8 +16,8 @@ PUT /projects/:id/services/asana
Parameters:
-- `api_key` (**required**) - User API token. User must have access to task,all comments will be attributed to this user.
-- `restrict_to_branch` (optional) - Comma-separated list of branches which will beautomatically inspected. Leave blank to include all branches.
+- `api_key` (**required**) - User API token. User must have access to task, all comments will be attributed to this user.
+- `restrict_to_branch` (optional) - Comma-separated list of branches which will be automatically inspected. Leave blank to include all branches.
### Delete Asana service
@@ -491,7 +491,7 @@ Jira issue tracker
Set JIRA service for a project.
-> Setting `project_url`, `issues_url` and `new_issue_url` will allow a user to easily navigate to the Jira issue tracker. See the [integration doc](http://doc.gitlab.com/ce/integration/external-issue-tracker.html) for details. Support for referencing commits and automatic closing of Jira issues directly from GitLab is [available in GitLab EE.](http://doc.gitlab.com/ee/integration/jira.html)
+> Setting `project_url`, `issues_url` and `new_issue_url` will allow a user to easily navigate to the Jira issue tracker. See the [integration doc](http://docs.gitlab.com/ce/integration/external-issue-tracker.html) for details. Support for referencing commits and automatic closing of Jira issues directly from GitLab is [available in GitLab EE.](http://docs.gitlab.com/ee/integration/jira.html)
```
PUT /projects/:id/services/jira
@@ -503,6 +503,8 @@ Parameters:
- `project_url` (**required**) - Project url
- `issues_url` (**required**) - Issue url
- `description` (optional) - Jira issue tracker
+- `username` (optional) - Jira username
+- `password` (optional) - Jira password
### Delete JIRA service
diff --git a/doc/api/settings.md b/doc/api/settings.md
index 001de76c7af..43a0fe35e42 100644
--- a/doc/api/settings.md
+++ b/doc/api/settings.md
@@ -26,7 +26,6 @@ Example response:
"default_branch_protection" : 2,
"restricted_visibility_levels" : [],
"signin_enabled" : true,
- "twitter_sharing_enabled" : true,
"after_sign_out_path" : null,
"max_attachment_size" : 10,
"user_oauth_applications" : true,
@@ -38,7 +37,8 @@ Example response:
"created_at" : "2016-01-04T15:44:55.176Z",
"default_project_visibility" : 0,
"gravatar_enabled" : true,
- "sign_in_text" : null
+ "sign_in_text" : null,
+ "container_registry_token_expire_delay": 5
}
```
@@ -57,7 +57,6 @@ PUT /application/settings
| `sign_in_text` | string | no | Text on login page |
| `home_page_url` | string | no | Redirect to this URL when not logged in |
| `default_branch_protection` | integer | no | Determine if developers can push to master. Can take `0` _(not protected, both developers and masters can push new commits, force push or delete the branch)_, `1` _(partially protected, developers can push new commits, but cannot force push or delete the branch, masters can do anything)_ or `2` _(fully protected, developers cannot push new commits, force push or delete the branch, masters can do anything)_ as a parameter. Default is `1`. |
-| `twitter_sharing_enabled` | boolean | no | Allow users to share project creation on Twitter |
| `restricted_visibility_levels` | array of integers | no | Selected levels cannot be used by non-admin users for projects or snippets. Can take `0` _(Private)_, `1` _(Internal)_ and `2` _(Public)_ as a parameter. Default is null which means there is no restriction. |
| `max_attachment_size` | integer | no | Limit attachment size in MB |
| `session_expire_delay` | integer | no | Session duration in minutes. GitLab restart is required to apply changes |
@@ -66,6 +65,7 @@ PUT /application/settings
| `restricted_signup_domains` | array of strings | no | Force people to use only corporate emails for sign-up. Default is null, meaning there is no restriction. |
| `user_oauth_applications` | boolean | no | Allow users to register any application to use GitLab as an OAuth provider |
| `after_sign_out_path` | string | no | Where to redirect users after logout |
+| `container_registry_token_expire_delay` | integer | no | Container Registry token duration in minutes |
```bash
curl -X PUT -H "PRIVATE-TOKEN: 9koXpg98eAheJpvBs5tK" https://gitlab.example.com/api/v3/application/settings?signup_enabled=false&default_project_visibility=1
@@ -85,7 +85,6 @@ Example response:
"updated_at": "2015-06-30T13:22:42.210Z",
"home_page_url": "",
"default_branch_protection": 2,
- "twitter_sharing_enabled": true,
"restricted_visibility_levels": [],
"max_attachment_size": 10,
"session_expire_delay": 10080,
@@ -93,6 +92,7 @@ Example response:
"default_snippet_visibility": 0,
"restricted_signup_domains": [],
"user_oauth_applications": true,
- "after_sign_out_path": ""
+ "after_sign_out_path": "",
+ "container_registry_token_expire_delay": 5
}
```
diff --git a/doc/api/sidekiq_metrics.md b/doc/api/sidekiq_metrics.md
new file mode 100644
index 00000000000..ebd131c94ca
--- /dev/null
+++ b/doc/api/sidekiq_metrics.md
@@ -0,0 +1,152 @@
+# Sidekiq Metrics
+
+>**Note:** This endpoint is only available on GitLab 8.9 and above.
+
+This API endpoint allows you to retrieve some information about the current state
+of Sidekiq, its jobs, queues, and processes.
+
+## Get the current Queue Metrics
+
+List information about all the registered queues, their backlog and their
+latency.
+
+```
+GET /sidekiq/queue_metrics
+```
+
+```bash
+curl -H "PRIVATE-TOKEN: 9koXpg98eAheJpvBs5tK" https://gitlab.example.com/api/v3/sidekiq/queue_metrics
+```
+
+Example response:
+
+```json
+{
+ "queues": {
+ "default": {
+ "backlog": 0,
+ "latency": 0
+ }
+ }
+}
+```
+
+## Get the current Process Metrics
+
+List information about all the Sidekiq workers registered to process your queues.
+
+```
+GET /sidekiq/process_metrics
+```
+
+```bash
+curl -H "PRIVATE-TOKEN: 9koXpg98eAheJpvBs5tK" https://gitlab.example.com/api/v3/sidekiq/process_metrics
+```
+
+Example response:
+
+```json
+{
+ "processes": [
+ {
+ "hostname": "gitlab.example.com",
+ "pid": 5649,
+ "tag": "gitlab",
+ "started_at": "2016-06-14T10:45:07.159-05:00",
+ "queues": [
+ "post_receive",
+ "mailers",
+ "archive_repo",
+ "system_hook",
+ "project_web_hook",
+ "gitlab_shell",
+ "incoming_email",
+ "runner",
+ "common",
+ "default"
+ ],
+ "labels": [],
+ "concurrency": 25,
+ "busy": 0
+ }
+ ]
+}
+```
+
+## Get the current Job Statistics
+
+List information about the jobs that Sidekiq has performed.
+
+```
+GET /sidekiq/job_stats
+```
+
+```bash
+curl -H "PRIVATE-TOKEN: 9koXpg98eAheJpvBs5tK" https://gitlab.example.com/api/v3/sidekiq/job_stats
+```
+
+Example response:
+
+```json
+{
+ "jobs": {
+ "processed": 2,
+ "failed": 0,
+ "enqueued": 0
+ }
+}
+```
+
+## Get a compound response of all the previously mentioned metrics
+
+List all the currently available information about Sidekiq.
+
+```
+GET /sidekiq/compound_metrics
+```
+
+```bash
+curl -H "PRIVATE-TOKEN: 9koXpg98eAheJpvBs5tK" https://gitlab.example.com/api/v3/sidekiq/compound_metrics
+```
+
+Example response:
+
+```json
+{
+ "queues": {
+ "default": {
+ "backlog": 0,
+ "latency": 0
+ }
+ },
+ "processes": [
+ {
+ "hostname": "gitlab.example.com",
+ "pid": 5649,
+ "tag": "gitlab",
+ "started_at": "2016-06-14T10:45:07.159-05:00",
+ "queues": [
+ "post_receive",
+ "mailers",
+ "archive_repo",
+ "system_hook",
+ "project_web_hook",
+ "gitlab_shell",
+ "incoming_email",
+ "runner",
+ "common",
+ "default"
+ ],
+ "labels": [],
+ "concurrency": 25,
+ "busy": 0
+ }
+ ],
+ "jobs": {
+ "processed": 2,
+ "failed": 0,
+ "enqueued": 0
+ }
+}
+```
+
diff --git a/doc/api/tags.md b/doc/api/tags.md
index 17d12e9cc62..ac9fac92f4c 100644
--- a/doc/api/tags.md
+++ b/doc/api/tags.md
@@ -38,6 +38,50 @@ Parameters:
]
```
+## Get a single repository tag
+
+Get a specific repository tag determined by its name. It returns `200` together
+with the tag information if the tag exists. It returns `404` if the tag does not
+exist.
+
+```
+GET /projects/:id/repository/tags/:tag_name
+```
+
+Parameters:
+
+| Attribute | Type | Required | Description |
+| --------- | ---- | -------- | ----------- |
+| `id` | integer | yes | The ID of a project |
+| `tag_name` | string | yes | The name of the tag |
+
+```bash
+curl -H "PRIVATE-TOKEN: 9koXpg98eAheJpvBs5tK" https://gitlab.example.com/api/v3/projects/5/repository/tags/v1.0.0
+```
+
+Example Response:
+
+```json
+{
+ "name": "v5.0.0",
+ "message": null,
+ "commit": {
+ "id": "60a8ff033665e1207714d6670fcd7b65304ec02f",
+ "message": "v5.0.0\n",
+ "parent_ids": [
+ "f61c062ff8bcbdb00e0a1b3317a91aed6ceee06b"
+ ],
+ "authored_date": "2015-02-01T21:56:31.000+01:00",
+ "author_name": "Arthur Verschaeve",
+ "author_email": "contact@arthurverschaeve.be",
+ "committed_date": "2015-02-01T21:56:31.000+01:00",
+ "committer_name": "Arthur Verschaeve",
+ "committer_email": "contact@arthurverschaeve.be"
+ },
+ "release": null
+}
+```
+
## Create a new tag
Creates a new tag in the repository that points to the supplied ref.
@@ -148,4 +192,4 @@ Parameters:
"tag_name": "1.0.0",
"description": "Amazing release. Wow"
}
-``` \ No newline at end of file
+```
diff --git a/doc/api/users.md b/doc/api/users.md
index 383e7c76ab0..7e848586dbd 100644
--- a/doc/api/users.md
+++ b/doc/api/users.md
@@ -20,6 +20,7 @@ GET /users
"name": "John Smith",
"state": "active",
"avatar_url": "http://localhost:3000/uploads/user/avatar/1/cd8.jpeg",
+ "web_url": "http://localhost:3000/u/john_smith"
},
{
"id": 2,
@@ -27,6 +28,7 @@ GET /users
"name": "Jack Smith",
"state": "blocked",
"avatar_url": "http://gravatar.com/../e32131cd8.jpeg",
+ "web_url": "http://localhost:3000/u/jack_smith"
}
]
```
@@ -45,21 +47,31 @@ GET /users
"email": "john@example.com",
"name": "John Smith",
"state": "active",
+ "avatar_url": "http://localhost:3000/uploads/user/avatar/1/index.jpg",
+ "web_url": "http://localhost:3000/u/john_smith",
"created_at": "2012-05-23T08:00:58Z",
+ "is_admin": false,
"bio": null,
+ "location": null,
"skype": "",
"linkedin": "",
"twitter": "",
"website_url": "",
- "extern_uid": "john.smith",
- "provider": "provider_name",
+ "last_sign_in_at": "2012-06-01T11:41:01Z",
+ "confirmed_at": "2012-05-23T09:05:22Z",
"theme_id": 1,
"color_scheme_id": 2,
- "is_admin": false,
- "avatar_url": "http://localhost:3000/uploads/user/avatar/1/cd8.jpeg",
+ "projects_limit": 100,
+ "current_sign_in_at": "2012-06-02T06:36:55Z",
+ "identities": [
+ {"provider": "github", "extern_uid": "2435223452345"},
+ {"provider": "bitbucket", "extern_uid": "john.smith"},
+ {"provider": "google_oauth2", "extern_uid": "8776128412476123468721346"}
+ ],
"can_create_group": true,
- "current_sign_in_at": "2014-03-19T13:12:15Z",
- "two_factor_enabled": true
+ "can_create_project": true,
+ "two_factor_enabled": true,
+ "external": false
},
{
"id": 2,
@@ -67,23 +79,27 @@ GET /users
"email": "jack@example.com",
"name": "Jack Smith",
"state": "blocked",
+ "avatar_url": "http://localhost:3000/uploads/user/avatar/2/index.jpg",
+ "web_url": "http://localhost:3000/u/jack_smith",
"created_at": "2012-05-23T08:01:01Z",
+ "is_admin": false,
"bio": null,
+ "location": null,
"skype": "",
"linkedin": "",
"twitter": "",
"website_url": "",
- "extern_uid": "jack.smith",
- "provider": "provider_name",
+ "last_sign_in_at": null,
+ "confirmed_at": "2012-05-30T16:53:06.148Z",
"theme_id": 1,
"color_scheme_id": 3,
- "is_admin": false,
- "avatar_url": "http://localhost:3000/uploads/user/avatar/1/cd8.jpeg",
- "can_create_group": true,
- "can_create_project": true,
"projects_limit": 100,
"current_sign_in_at": "2014-03-19T17:54:13Z",
- "two_factor_enabled": false
+ "identities": [],
+ "can_create_group": true,
+ "can_create_project": true,
+ "two_factor_enabled": true,
+ "external": false
}
]
```
@@ -123,9 +139,11 @@ Parameters:
"name": "John Smith",
"state": "active",
"avatar_url": "http://localhost:3000/uploads/user/avatar/1/cd8.jpeg",
+ "web_url": "http://localhost:3000/u/john_smith",
"created_at": "2012-05-23T08:00:58Z",
"is_admin": false,
"bio": null,
+ "location": null,
"skype": "",
"linkedin": "",
"twitter": "",
@@ -150,22 +168,31 @@ Parameters:
"email": "john@example.com",
"name": "John Smith",
"state": "active",
+ "avatar_url": "http://localhost:3000/uploads/user/avatar/1/index.jpg",
+ "web_url": "http://localhost:3000/u/john_smith",
"created_at": "2012-05-23T08:00:58Z",
- "confirmed_at": "2012-05-23T08:00:58Z",
- "last_sign_in_at": "2015-03-23T08:00:58Z",
+ "is_admin": false,
"bio": null,
+ "location": null,
"skype": "",
"linkedin": "",
"twitter": "",
"website_url": "",
- "extern_uid": "john.smith",
- "provider": "provider_name",
+ "last_sign_in_at": "2012-06-01T11:41:01Z",
+ "confirmed_at": "2012-05-23T09:05:22Z",
"theme_id": 1,
"color_scheme_id": 2,
- "is_admin": false,
+ "projects_limit": 100,
+ "current_sign_in_at": "2012-06-02T06:36:55Z",
+ "identities": [
+ {"provider": "github", "extern_uid": "2435223452345"},
+ {"provider": "bitbucket", "extern_uid": "john.smith"},
+ {"provider": "google_oauth2", "extern_uid": "8776128412476123468721346"}
+ ],
"can_create_group": true,
"can_create_project": true,
- "projects_limit": 100
+ "two_factor_enabled": true,
+ "external": false
}
```
@@ -191,6 +218,7 @@ Parameters:
- `extern_uid` (optional) - External UID
- `provider` (optional) - External provider name
- `bio` (optional) - User's biography
+- `location` (optional) - User's location
- `admin` (optional) - User is admin - true or false (default)
- `can_create_group` (optional) - User can create groups - true or false
- `confirm` (optional) - Require confirmation - true (default) or false
@@ -218,6 +246,7 @@ Parameters:
- `extern_uid` - External UID
- `provider` - External provider name
- `bio` - User's biography
+- `location` (optional) - User's location
- `admin` (optional) - User is admin - true or false (default)
- `can_create_group` (optional) - User can create groups - true or false
- `external` (optional) - Flags the user as external - true or false(default)
@@ -256,20 +285,33 @@ GET /user
"username": "john_smith",
"email": "john@example.com",
"name": "John Smith",
- "private_token": "dd34asd13as",
"state": "active",
+ "avatar_url": "http://localhost:3000/uploads/user/avatar/1/index.jpg",
+ "web_url": "http://localhost:3000/u/john_smith",
"created_at": "2012-05-23T08:00:58Z",
+ "is_admin": false,
"bio": null,
+ "location": null,
"skype": "",
"linkedin": "",
"twitter": "",
"website_url": "",
+ "last_sign_in_at": "2012-06-01T11:41:01Z",
+ "confirmed_at": "2012-05-23T09:05:22Z",
"theme_id": 1,
"color_scheme_id": 2,
- "is_admin": false,
+ "projects_limit": 100,
+ "current_sign_in_at": "2012-06-02T06:36:55Z",
+ "identities": [
+ {"provider": "github", "extern_uid": "2435223452345"},
+ {"provider": "bitbucket", "extern_uid": "john_smith"},
+ {"provider": "google_oauth2", "extern_uid": "8776128412476123468721346"}
+ ],
"can_create_group": true,
"can_create_project": true,
- "projects_limit": 100
+ "two_factor_enabled": true,
+ "external": false,
+ "private_token": "dd34asd13as"
}
```
diff --git a/doc/ci/README.md b/doc/ci/README.md
index 4abc45bf9bb..ef72df97ce6 100644
--- a/doc/ci/README.md
+++ b/doc/ci/README.md
@@ -14,5 +14,5 @@
- [Trigger builds through the API](triggers/README.md)
- [Build artifacts](build_artifacts/README.md)
- [User permissions](permissions/README.md)
-- [API](api/README.md)
+- [API](../../api/ci/README.md)
- [CI services (linked docker containers)](services/README.md)
diff --git a/doc/ci/api/README.md b/doc/ci/api/README.md
index aea808007fc..4ca8d92d7cc 100644
--- a/doc/ci/api/README.md
+++ b/doc/ci/api/README.md
@@ -1,22 +1,3 @@
# GitLab CI API
-## Purpose
-
-Main purpose of GitLab CI API is to provide necessary data and context for
-GitLab CI Runners.
-
-For consumer API take a look at this [documentation](../../api/README.md) where
-you will find all relevant information.
-
-## API Prefix
-
-Current CI API prefix is `/ci/api/v1`.
-
-You need to prepend this prefix to all examples in this documentation, like:
-
- GET /ci/api/v1/builds/:id/artifacts
-
-## Resources
-
-- [Builds](builds.md)
-- [Runners](runners.md)
+This document was moved to a [new location](../../api/ci/README.md).
diff --git a/doc/ci/api/builds.md b/doc/ci/api/builds.md
index d100e261178..f5bd3181c02 100644
--- a/doc/ci/api/builds.md
+++ b/doc/ci/api/builds.md
@@ -1,73 +1,3 @@
# Builds API
-API used by runners to receive and update builds.
-
-_**Note:** This API is intended to be used only by Runners as their own
-communication channel. For the consumer API see the
-[Builds API](../../api/builds.md)._
-
-## Authentication
-
-This API uses two types of authentication:
-
-1. Unique runner's token
-
- Token assigned to runner after it has been registered.
-
-2. Using build authorization token
-
- This is project's CI token that can be found in Continuous Integration
- project settings.
-
- Build authorization token can be passed as a parameter or a value of
- `BUILD-TOKEN` header. This method are interchangeable.
-
-## Builds
-
-### Runs oldest pending build by runner
-
- POST /ci/api/v1/builds/register
-
-Parameters:
-
- * `token` (required) - Unique runner token
-
-
-### Update details of an existing build
-
- PUT /ci/api/v1/builds/:id
-
-Parameters:
-
- * `id` (required) - The ID of a project
- * `token` (required) - Unique runner token
- * `state` (optional) - The state of a build
- * `trace` (optional) - The trace of a build
-
-### Upload artifacts to build
-
- POST /ci/api/v1/builds/:id/artifacts
-
-Parameters:
-
- * `id` (required) - The ID of a build
- * `token` (required) - The build authorization token
- * `file` (required) - Artifacts file
-
-### Download the artifacts file from build
-
- GET /ci/api/v1/builds/:id/artifacts
-
-Parameters:
-
- * `id` (required) - The ID of a build
- * `token` (required) - The build authorization token
-
-### Remove the artifacts file from build
-
- DELETE /ci/api/v1/builds/:id/artifacts
-
-Parameters:
-
- * ` id` (required) - The ID of a build
- * `token` (required) - The build authorization token
+This document was moved to a [new location](../../api/ci/builds.md).
diff --git a/doc/ci/api/runners.md b/doc/ci/api/runners.md
index 2f01da4bd76..b14ea99db76 100644
--- a/doc/ci/api/runners.md
+++ b/doc/ci/api/runners.md
@@ -1,46 +1,3 @@
# Runners API
-API used by runners to register and delete themselves.
-
-_**Note:** This API is intended to be used only by Runners as their own
-communication channel. For the consumer API see the
-[new Runners API](../../api/runners.md)._
-
-## Authentication
-
-This API uses two types of authentication:
-
-1. Unique runner's token
-
- Token assigned to runner after it has been registered.
-
-2. Using runners' registration token
-
- This is a token that can be found in project's settings.
- It can be also found in Admin area &raquo; Runners settings.
-
- There are two types of tokens you can pass - shared runner registration
- token or project specific registration token.
-
-## Runners
-
-### Register a new runner
-
-Used to make GitLab CI aware of available runners.
-
- POST /ci/api/v1/runners/register
-
-Parameters:
-
- * `token` (required) - Registration token
-
-
-### Delete a runner
-
-Used to remove runner.
-
- DELETE /ci/api/v1/runners/delete
-
-Parameters:
-
- * `token` (required) - Unique runner token
+This document was moved to a [new location](../../api/ci/runners.md).
diff --git a/doc/ci/build_artifacts/README.md b/doc/ci/build_artifacts/README.md
index 71db5aa5dc8..9553bb11e9d 100644
--- a/doc/ci/build_artifacts/README.md
+++ b/doc/ci/build_artifacts/README.md
@@ -1,7 +1,10 @@
# Introduction to build artifacts
Artifacts is a list of files and directories which are attached to a build
-after it completes successfully.
+after it completes successfully. This feature is enabled by default in all GitLab installations.
+
+_If you are searching for ways to use artifacts, jump to
+[Defining artifacts in `.gitlab-ci.yml`](#defining-artifacts-in-gitlab-ciyml)._
Since GitLab 8.2 and [GitLab Runner] 0.7.0, build artifacts that are created by
GitLab Runner are uploaded to GitLab and are downloadable as a single archive
@@ -16,13 +19,9 @@ The artifacts browser will be available only for new artifacts that are sent
to GitLab using GitLab Runner version 1.0 and up. It will not be possible to
browse old artifacts already uploaded to GitLab.
-## Enabling build artifacts
-
-_If you are searching for ways to use artifacts, jump to
-[Defining artifacts in `.gitlab-ci.yml`](#defining-artifacts-in-gitlab-ciyml)._
+## Disabling build artifacts
-The artifacts feature is enabled by default in all GitLab installations.
-To disable it site-wide, follow the steps below.
+To disable artifacts site-wide, follow the steps below.
---
diff --git a/doc/ci/docker/using_docker_build.md b/doc/ci/docker/using_docker_build.md
index 4b1788a9af0..7f83f846454 100644
--- a/doc/ci/docker/using_docker_build.md
+++ b/doc/ci/docker/using_docker_build.md
@@ -4,14 +4,14 @@ GitLab CI allows you to use Docker Engine to build and test docker-based project
**This also allows to you to use `docker-compose` and other docker-enabled tools.**
-This is one of new trends in Continuous Integration/Deployment to:
+One of the new trends in Continuous Integration/Deployment is to:
-1. create application image,
-1. run test against created image,
-1. push image to remote registry,
-1. deploy server from pushed image
+1. create an application image,
+1. run tests against the created image,
+1. push image to a remote registry, and
+1. deploy to a server from the pushed image.
-It's also useful in case when your application already has the `Dockerfile` that can be used to create and test image:
+It's also useful when your application already has the `Dockerfile` that can be used to create and test an image:
```bash
$ docker build -t my-image dockerfiles/
$ docker run my-docker-image /script/to/run/tests
@@ -19,49 +19,50 @@ $ docker tag my-image my-registry:5000/my-image
$ docker push my-registry:5000/my-image
```
-However, this requires special configuration of GitLab Runner to enable `docker` support during build.
-**This requires running GitLab Runner in privileged mode which can be harmful when untrusted code is run.**
+This requires special configuration of GitLab Runner to enable `docker` support during builds.
-There are two methods to enable the use of `docker build` and `docker run` during build.
+## Runner Configuration
-## 1. Use shell executor
+There are three methods to enable the use of `docker build` and `docker run` during builds; each with their own tradeoffs.
+
+### Use shell executor
The simplest approach is to install GitLab Runner in `shell` execution mode.
-GitLab Runner then executes build scripts as `gitlab-runner` user.
+GitLab Runner then executes build scripts as the `gitlab-runner` user.
1. Install [GitLab Runner](https://gitlab.com/gitlab-org/gitlab-ci-multi-runner/#installation).
1. During GitLab Runner installation select `shell` as method of executing build scripts or use command:
```bash
- $ sudo gitlab-runner register -n \
+ $ sudo gitlab-ci-multi-runner register -n \
--url https://gitlab.com/ci \
- --token RUNNER_TOKEN \
+ --registration-token REGISTRATION_TOKEN \
--executor shell
--description "My Runner"
```
-2. Install Docker on server.
+2. Install Docker Engine on server.
- For more information how to install Docker on different systems checkout the [Supported installations](https://docs.docker.com/installation/).
+ For more information how to install Docker Engine on different systems checkout the [Supported installations](https://docs.docker.com/engine/installation/).
3. Add `gitlab-runner` user to `docker` group:
-
+
```bash
$ sudo usermod -aG docker gitlab-runner
```
4. Verify that `gitlab-runner` has access to Docker:
-
+
```bash
$ sudo -u gitlab-runner -H docker info
```
-
+
You can now verify that everything works by adding `docker info` to `.gitlab-ci.yml`:
```yaml
before_script:
- docker info
-
+
build_image:
script:
- docker build -t my-docker-image .
@@ -70,42 +71,246 @@ GitLab Runner then executes build scripts as `gitlab-runner` user.
5. You can now use `docker` command and install `docker-compose` if needed.
-6. However, by adding `gitlab-runner` to `docker` group you are effectively granting `gitlab-runner` full root permissions.
-For more information please checkout [On Docker security: `docker` group considered harmful](https://www.andreas-jung.com/contents/on-docker-security-docker-group-considered-harmful).
+> **Note:**
+* By adding `gitlab-runner` to the `docker` group you are effectively granting `gitlab-runner` full root permissions.
+For more information please read [On Docker security: `docker` group considered harmful](https://www.andreas-jung.com/contents/on-docker-security-docker-group-considered-harmful).
-## 2. Use docker-in-docker executor
+### Use docker-in-docker executor
-Second approach is to use special Docker image with all tools installed (`docker` and `docker-compose`) and run build script in context of that image in privileged mode.
-In order to do that follow the steps:
+The second approach is to use the special docker-in-docker (dind)
+[Docker image](https://hub.docker.com/_/docker/) with all tools installed
+(`docker` and `docker-compose`) and run the build script in context of that
+image in privileged mode.
+
+In order to do that, follow the steps:
1. Install [GitLab Runner](https://gitlab.com/gitlab-org/gitlab-ci-multi-runner/#installation).
-1. Register GitLab Runner from command line to use `docker` and `privileged` mode:
+1. Register GitLab Runner from the command line to use `docker` and `privileged`
+ mode:
```bash
- $ sudo gitlab-runner register -n \
+ sudo gitlab-ci-multi-runner register -n \
--url https://gitlab.com/ci \
- --token RUNNER_TOKEN \
+ --registration-token REGISTRATION_TOKEN \
--executor docker \
--description "My Docker Runner" \
- --docker-image "gitlab/dind:latest" \
+ --docker-image "docker:latest" \
--docker-privileged
```
-
- The above command will register new Runner to use special [gitlab/dind](https://registry.hub.docker.com/u/gitlab/dind/) image which is provided by GitLab Inc.
- The image at the start runs Docker daemon in [docker-in-docker](https://blog.docker.com/2013/09/docker-can-now-run-within-docker/) mode.
-1. You can now use `docker` from build script:
-
+ The above command will register a new Runner to use the special
+ `docker:latest` image which is provided by Docker. **Notice that it's using
+ the `privileged` mode to start the build and service containers.** If you
+ want to use [docker-in-docker] mode, you always have to use `privileged = true`
+ in your Docker containers.
+
+ The above command will create a `config.toml` entry similar to this:
+
+ ```
+ [[runners]]
+ url = "https://gitlab.com/ci"
+ token = TOKEN
+ executor = "docker"
+ [runners.docker]
+ tls_verify = false
+ image = "docker:latest"
+ privileged = true
+ disable_cache = false
+ volumes = ["/cache"]
+ [runners.cache]
+ Insecure = false
+ ```
+
+1. You can now use `docker` in the build script (note the inclusion of the `docker:dind` service):
+
```yaml
+ image: docker:latest
+
+ services:
+ - docker:dind
+
before_script:
- - docker info
-
- build_image:
+ - docker info
+
+ build:
+ stage: build
script:
- - docker build -t my-docker-image .
- - docker run my-docker-image /script/to/run/tests
+ - docker build -t my-docker-image .
+ - docker run my-docker-image /script/to/run/tests
+ ```
+
+Docker-in-Docker works well, and is the recommended configuration, but it is not without its own challenges:
+* By enabling `--docker-privileged`, you are effectively disabling all of
+the security mechanisms of containers and exposing your host to privilege
+escalation which can lead to container breakout. For more information, check out the official Docker documentation on
+[Runtime privilege and Linux capabilities][docker-cap].
+* Using docker-in-docker, each build is in a clean environment without the past
+history. Concurrent builds work fine because every build gets it's own instance of docker engine so they won't conflict with each other. But this also means builds can be slower because there's no caching of layers.
+* By default, `docker:dind` uses `--storage-driver vfs` which is the slowest form
+offered.
+
+An example project using this approach can be found here: https://gitlab.com/gitlab-examples/docker.
+
+### Use Docker socket binding
+
+The third approach is to bind-mount `/var/run/docker.sock` into the container so that docker is available in the context of that image.
+
+In order to do that, follow the steps:
+
+1. Install [GitLab Runner](https://gitlab.com/gitlab-org/gitlab-ci-multi-runner/#installation).
+
+1. Register GitLab Runner from the command line to use `docker` and share `/var/run/docker.sock`:
+
+ ```bash
+ sudo gitlab-ci-multi-runner register -n \
+ --url https://gitlab.com/ci \
+ --registration-token REGISTRATION_TOKEN \
+ --executor docker \
+ --description "My Docker Runner" \
+ --docker-image "docker:latest" \
+ --docker-volumes /var/run/docker.sock:/var/run/docker.sock
```
-1. However, by enabling `--docker-privileged` you are effectively disables all security mechanisms of containers and exposing your host to privilege escalation which can lead to container breakout.
-For more information, check out [Runtime privilege](https://docs.docker.com/reference/run/#runtime-privilege-linux-capabilities-and-lxc-configuration). \ No newline at end of file
+ The above command will register a new Runner to use the special
+ `docker:latest` image which is provided by Docker. **Notice that it's using
+ the Docker daemon of the Runner itself, and any containers spawned by docker commands will be siblings of the Runner rather than children of the runner.** This may have complications and limitations that are unsuitable for your workflow.
+
+ The above command will create a `config.toml` entry similar to this:
+
+ ```
+ [[runners]]
+ url = "https://gitlab.com/ci"
+ token = REGISTRATION_TOKEN
+ executor = "docker"
+ [runners.docker]
+ tls_verify = false
+ image = "docker:latest"
+ privileged = false
+ disable_cache = false
+ volumes = ["/var/run/docker.sock", "/cache"]
+ [runners.cache]
+ Insecure = false
+ ```
+
+1. You can now use `docker` in the build script (note that you don't need to include the `docker:dind` service as when using the Docker in Docker executor):
+
+ ```yaml
+ image: docker:latest
+
+ before_script:
+ - docker info
+
+ build:
+ stage: build
+ script:
+ - docker build -t my-docker-image .
+ - docker run my-docker-image /script/to/run/tests
+ ```
+
+While the above method avoids using Docker in privileged mode, you should be aware of the following implications:
+* By sharing the docker daemon, you are effectively disabling all
+the security mechanisms of containers and exposing your host to privilege
+escalation which can lead to container breakout. For example, if a project
+ran `docker rm -f $(docker ps -a -q)` it would remove the GitLab Runner
+containers.
+* Concurrent builds may not work; if your tests
+create containers with specific names, they may conflict with each other.
+* Sharing files and directories from the source repo into containers may not
+work as expected since volume mounting is done in the context of the host
+machine, not the build container.
+e.g. `docker run --rm -t -i -v $(pwd)/src:/home/app/src test-image:latest run_app_tests`
+
+## Using the GitLab Container Registry
+
+> **Note:**
+This feature requires GitLab 8.8 and GitLab Runner 1.2.
+
+Once you've built a Docker image, you can push it up to the built-in [GitLab Container Registry](../../container_registry/README.md). For example, if you're using
+docker-in-docker on your runners, this is how your `.gitlab-ci.yml` could look:
+
+
+```yaml
+ build:
+ image: docker:latest
+ services:
+ - docker:dind
+ stage: build
+ script:
+ - docker login -u gitlab-ci-token -p $CI_BUILD_TOKEN registry.example.com
+ - docker build -t registry.example.com/group/project:latest .
+ - docker push registry.example.com/group/project:latest
+```
+
+You have to use the credentials of the special `gitlab-ci-token` user with its
+password stored in `$CI_BUILD_TOKEN` in order to push to the Registry connected
+to your project. This allows you to automate building and deployment of your
+Docker images.
+
+Here's a more elaborate example that splits up the tasks into 4 pipeline stages,
+including two tests that run in parallel. The build is stored in the container
+registry and used by subsequent stages, downloading the image
+when needed. Changes to `master` also get tagged as `latest` and deployed using
+an application-specific deploy script:
+
+```yaml
+image: docker:latest
+services:
+- docker:dind
+
+stages:
+- build
+- test
+- release
+- deploy
+
+variables:
+ CONTAINER_TEST_IMAGE: registry.example.com/my-group/my-project:$CI_BUILD_REF_NAME
+ CONTAINER_RELEASE_IMAGE: registry.example.com/my-group/my-project:latest
+
+before_script:
+ - docker login -u gitlab-ci-token -p $CI_BUILD_TOKEN registry.example.com
+
+build:
+ stage: build
+ script:
+ - docker build --pull -t $CONTAINER_TEST_IMAGE .
+ - docker push $CONTAINER_TEST_IMAGE
+
+test1:
+ stage: test
+ script:
+ - docker pull $CONTAINER_TEST_IMAGE
+ - docker run $CONTAINER_TEST_IMAGE /script/to/run/tests
+
+test2:
+ stage: test
+ script:
+ - docker pull $CONTAINER_TEST_IMAGE
+ - docker run $CONTAINER_TEST_IMAGE /script/to/run/another/test
+
+release-image:
+ stage: release
+ script:
+ - docker pull $CONTAINER_TEST_IMAGE
+ - docker tag $CONTAINER_TEST_IMAGE $CONTAINER_RELEASE_IMAGE
+ - docker push $CONTAINER_RELEASE_IMAGE
+ only:
+ - master
+
+deploy:
+ stage: deploy
+ script:
+ - ./deploy.sh
+ only:
+ - master
+```
+
+Some things you should be aware of when using the Container Registry:
+* You must log in to the container registry before running commands. Putting this in `before_script` will run it before each build job.
+* Using `docker build --pull` makes sure that Docker fetches any changes to base images before building just in case your cache is stale. It takes slightly longer, but means you don’t get stuck without security patches to base images.
+* Doing an explicit `docker pull` before each `docker run` makes sure to fetch the latest image that was just built. This is especially important if you are using multiple runners that cache images locally. Using the git SHA in your image tag makes this less necessary since each build will be unique and you shouldn't ever have a stale image, but it's still possible if you re-build a given commit after a dependency has changed.
+* You don't want to build directly to `latest` in case there are multiple builds happening simultaneously.
+
+[docker-in-docker]: https://blog.docker.com/2013/09/docker-can-now-run-within-docker/
+[docker-cap]: https://docs.docker.com/engine/reference/run/#runtime-privilege-and-linux-capabilities
diff --git a/doc/ci/docker/using_docker_images.md b/doc/ci/docker/using_docker_images.md
index bd748f1b986..a849905ac6b 100644
--- a/doc/ci/docker/using_docker_images.md
+++ b/doc/ci/docker/using_docker_images.md
@@ -23,7 +23,7 @@ To use GitLab Runner with docker you need to register a new runner to use the
`docker` executor:
```bash
-gitlab-runner register \
+gitlab-ci-multi-runner register \
--url "https://gitlab.com/" \
--registration-token "PROJECT_REGISTRATION_TOKEN" \
--description "docker-ruby-2.1" \
@@ -64,7 +64,7 @@ You can see some widely used services examples in the relevant documentation of
### How is service linked to the build
To better understand how the container linking works, read
-[Linking containers together](https://docs.docker.com/userguide/dockerlinks/).
+[Linking containers together][linking-containers].
To summarize, if you add `mysql` as service to your application, the image will
then be used to create a container that is linked to the build container.
@@ -239,8 +239,8 @@ is specific to your project.
Then create some service containers:
```
-docker run -d -n service-mysql mysql:latest
-docker run -d -n service-postgres postgres:latest
+docker run -d --name service-mysql mysql:latest
+docker run -d --name service-postgres postgres:latest
```
This will create two service containers, named `service-mysql` and
@@ -273,7 +273,7 @@ creation.
[Docker Fundamentals]: https://docs.docker.com/engine/understanding-docker/
[hub]: https://hub.docker.com/
[linking-containers]: https://docs.docker.com/engine/userguide/networking/default_network/dockerlinks/
-[tutum/wordpress]: https://registry.hub.docker.com/u/tutum/wordpress/
-[postgres-hub]: https://registry.hub.docker.com/u/library/postgres/
-[mysql-hub]: https://registry.hub.docker.com/u/library/mysql/
+[tutum/wordpress]: https://hub.docker.com/r/tutum/wordpress/
+[postgres-hub]: https://hub.docker.com/r/_/postgres/
+[mysql-hub]: https://hub.docker.com/r/_/mysql/
[runner-priv-reg]: https://gitlab.com/gitlab-org/gitlab-ci-multi-runner/blob/master/docs/configuration/advanced-configuration.md#using-a-private-docker-registry
diff --git a/doc/ci/examples/README.md b/doc/ci/examples/README.md
index 20eeaad3fa3..27bc21c2922 100644
--- a/doc/ci/examples/README.md
+++ b/doc/ci/examples/README.md
@@ -11,6 +11,6 @@
## Outside the documentation
-- [Blost post about using GitLab CI for iOS projects](https://about.gitlab.com/2016/03/10/setting-up-gitlab-ci-for-ios-projects/)
+- [Blog post about using GitLab CI for iOS projects](https://about.gitlab.com/2016/03/10/setting-up-gitlab-ci-for-ios-projects/)
- [Repo's with examples for various languages](https://gitlab.com/groups/gitlab-examples)
- [The .gitlab-ci.yml file for GitLab itself](https://gitlab.com/gitlab-org/gitlab-ce/blob/master/.gitlab-ci.yml)
diff --git a/doc/ci/deployment/README.md b/doc/ci/examples/deployment/README.md
index 7d91ce6710f..7d91ce6710f 100644
--- a/doc/ci/deployment/README.md
+++ b/doc/ci/examples/deployment/README.md
diff --git a/doc/ci/examples/php.md b/doc/ci/examples/php.md
index aeadd6a448e..17e1c64bb8a 100644
--- a/doc/ci/examples/php.md
+++ b/doc/ci/examples/php.md
@@ -40,7 +40,7 @@ repository with the following content:
#!/bin/bash
# We need to install dependencies only for Docker
-[[ ! -e /.dockerinit ]] && exit 0
+[[ ! -e /.dockerenv ]] && [[ ! -e /.dockerinit ]] && exit 0
set -xe
@@ -60,7 +60,7 @@ docker-php-ext-install pdo_mysql
You might wonder what `docker-php-ext-install` is. In short, it is a script
provided by the official php docker image that you can use to easilly install
extensions. For more information read the the documentation at
-<https://hub.docker.com/_/php/>.
+<https://hub.docker.com/r/_/php/>.
Now that we created the script that contains all prerequisites for our build
environment, let's add it in `.gitlab-ci.yml`:
@@ -92,7 +92,7 @@ Finally, commit your files and push them to GitLab to see your build succeeding
The final `.gitlab-ci.yml` should look similar to this:
```yaml
-# Select image from https://hub.docker.com/_/php/
+# Select image from https://hub.docker.com/r/_/php/
image: php:5.6
before_script:
@@ -263,10 +263,10 @@ terminal execute:
```bash
# Check using docker executor
-gitlab-runner exec docker test:app
+gitlab-ci-multi-runner exec docker test:app
# Check using shell executor
-gitlab-runner exec shell test:app
+gitlab-ci-multi-runner exec shell test:app
```
## Example project
@@ -278,7 +278,7 @@ that runs on [GitLab.com](https://gitlab.com) using our publicly available
Want to hack on it? Simply fork it, commit and push your changes. Within a few
moments the changes will be picked by a public runner and the build will begin.
-[php-hub]: https://hub.docker.com/_/php/
+[php-hub]: https://hub.docker.com/r/_/php/
[phpenv]: https://github.com/phpenv/phpenv
[phpenv-installation]: https://github.com/phpenv/phpenv#installation
[php-example-repo]: https://gitlab.com/gitlab-examples/php
diff --git a/doc/ci/examples/test-and-deploy-python-application-to-heroku.md b/doc/ci/examples/test-and-deploy-python-application-to-heroku.md
index a236da53fe9..e4d3970deac 100644
--- a/doc/ci/examples/test-and-deploy-python-application-to-heroku.md
+++ b/doc/ci/examples/test-and-deploy-python-application-to-heroku.md
@@ -8,7 +8,7 @@ This is what the `.gitlab-ci.yml` file looks like for this project:
```yaml
test:
script:
- # this configures django application to use attached postgres database that is run on `postgres` host
+ # this configures Django application to use attached postgres database that is run on `postgres` host
- export DATABASE_URL=postgres://postgres:@postgres:5432/python-test-app
- apt-get update -qy
- apt-get install -y python-dev python-pip
@@ -37,7 +37,7 @@ production:
```
This project has three jobs:
-1. `test` - used to test rails application,
+1. `test` - used to test Django application,
2. `staging` - used to automatically deploy staging environment every push to `master` branch
3. `production` - used to automatically deploy production environmnet for every created tag
@@ -61,12 +61,12 @@ gitlab-ci-multi-runner register \
--non-interactive \
--url "https://gitlab.com/ci/" \
--registration-token "PROJECT_REGISTRATION_TOKEN" \
- --description "python-3.2" \
+ --description "python-3.5" \
--executor "docker" \
- --docker-image python:3.2 \
+ --docker-image python:3.5 \
--docker-postgres latest
```
-With the command above, you create a runner that uses [python:3.2](https://registry.hub.docker.com/u/library/python/) image and uses [postgres](https://registry.hub.docker.com/u/library/postgres/) database.
+With the command above, you create a runner that uses [python:3.5](https://hub.docker.com/r/_/python/) image and uses [postgres](https://hub.docker.com/r/_/postgres/) database.
To access PostgreSQL database you need to connect to `host: postgres` as user `postgres` without password.
diff --git a/doc/ci/examples/test-and-deploy-ruby-application-to-heroku.md b/doc/ci/examples/test-and-deploy-ruby-application-to-heroku.md
index f5645d586ae..08c10d391ea 100644
--- a/doc/ci/examples/test-and-deploy-ruby-application-to-heroku.md
+++ b/doc/ci/examples/test-and-deploy-ruby-application-to-heroku.md
@@ -1,5 +1,5 @@
## Test and Deploy a ruby application
-This example will guide you how to run tests in your Ruby application and deploy it automatically as Heroku application.
+This example will guide you how to run tests in your Ruby on Rails application and deploy it automatically as Heroku application.
You can checkout the example [source](https://gitlab.com/ayufan/ruby-getting-started) and check [CI status](https://gitlab.com/ayufan/ruby-getting-started/builds?scope=all).
@@ -32,7 +32,7 @@ production:
```
This project has three jobs:
-1. `test` - used to test rails application,
+1. `test` - used to test Rails application,
2. `staging` - used to automatically deploy staging environment every push to `master` branch
3. `production` - used to automatically deploy production environmnet for every created tag
@@ -62,6 +62,6 @@ gitlab-ci-multi-runner register \
--docker-postgres latest
```
-With the command above, you create a runner that uses [ruby:2.2](https://registry.hub.docker.com/u/library/ruby/) image and uses [postgres](https://registry.hub.docker.com/u/library/postgres/) database.
+With the command above, you create a runner that uses [ruby:2.2](https://hub.docker.com/r/_/ruby/) image and uses [postgres](https://hub.docker.com/r/_/postgres/) database.
To access PostgreSQL database you need to connect to `host: postgres` as user `postgres` without password.
diff --git a/doc/ci/examples/test-scala-application.md b/doc/ci/examples/test-scala-application.md
index 58947f0f9f4..7412fdbbc78 100644
--- a/doc/ci/examples/test-scala-application.md
+++ b/doc/ci/examples/test-scala-application.md
@@ -1,15 +1,21 @@
## Test a Scala application
-This example demonstrates the integration of Gitlab CI with Scala applications using SBT. Checkout the example [project](https://gitlab.com/gitlab-examples/scala-sbt) and [build status](https://gitlab.com/gitlab-examples/scala-sbt/builds).
+This example demonstrates the integration of Gitlab CI with Scala
+applications using SBT. Checkout the example
+[project](https://gitlab.com/gitlab-examples/scala-sbt) and
+[build status](https://gitlab.com/gitlab-examples/scala-sbt/builds).
### Add `.gitlab-ci.yml` file to project
-The following `.gitlab-ci.yml` should be added in the root of your repository to trigger CI:
+The following `.gitlab-ci.yml` should be added in the root of your
+repository to trigger CI:
-```yaml
+``` yaml
image: java:8
before_script:
+ - apt-get update -y
+ - apt-get install apt-transport-https -y
# Install SBT
- echo "deb http://dl.bintray.com/sbt/debian /" | tee -a /etc/apt/sources.list.d/sbt.list
- apt-key adv --keyserver hkp://keyserver.ubuntu.com:80 --recv 642AC823
@@ -22,9 +28,20 @@ test:
- sbt clean coverage test coverageReport
```
-The `before_script` installs [SBT](http://www.scala-sbt.org/) and displays the version that is being used. The `test` stage executes SBT to compile and test the project. [scoverage](https://github.com/scoverage/sbt-scoverage) is used as a SBT plugin to measure test coverage.
+The `before_script` installs [SBT](http://www.scala-sbt.org/) and
+displays the version that is being used. The `test` stage executes SBT
+to compile and test the project.
+[scoverage](https://github.com/scoverage/sbt-scoverage) is used as an SBT
+plugin to measure test coverage.
-You can use other versions of Scala and SBT by defining them in `build.sbt`.
+You can use other versions of Scala and SBT by defining them in
+`build.sbt`.
### Display test coverage in build
-Add the `Coverage was \[\d+.\d+\%\]` regular expression in the `Continuous Integration > Test coverage parsing` project setting to retrieve the test coverage rate from the build trace and have it displayed with your builds.
+
+Add the `Coverage was \[\d+.\d+\%\]` regular expression in the
+**Settings > Edit Project > Test coverage parsing** project setting to
+retrieve the test coverage rate from the build trace and have it
+displayed with your builds.
+
+**Builds** must be enabled for this option to appear.
diff --git a/doc/ci/quick_start/README.md b/doc/ci/quick_start/README.md
index 9aba4326e11..386b8e29fcf 100644
--- a/doc/ci/quick_start/README.md
+++ b/doc/ci/quick_start/README.md
@@ -13,7 +13,7 @@ GitLab offers a [continuous integration][ci] service. If you
and configure your GitLab project to use a [Runner], then each merge request or
push triggers a build.
-The `.gitlab-ci.yml` file tells the GitLab runner what do to. By default it
+The `.gitlab-ci.yml` file tells the GitLab runner what to do. By default it
runs three [stages]: `build`, `test`, and `deploy`.
If everything runs OK (no non-zero return values), you'll get a nice green
@@ -212,8 +212,8 @@ If you want to receive e-mail notifications about the result status of the
builds, you should explicitly enable the **Builds Emails** service under your
project's settings.
-For more information read the [Builds emails service documentation]
-(../../project_services/builds_emails.md).
+For more information read the
+[Builds emails service documentation](../../project_services/builds_emails.md).
## Builds badge
diff --git a/doc/ci/runners/README.md b/doc/ci/runners/README.md
index 295d953db11..400784da617 100644
--- a/doc/ci/runners/README.md
+++ b/doc/ci/runners/README.md
@@ -7,6 +7,10 @@ through the coordinator API of GitLab CI.
A runner can be specific to a certain project or serve any project
in GitLab CI. A runner that serves all projects is called a shared runner.
+Ideally, GitLab Runner should not be installed on the same machine as GitLab.
+Read the [requirements documentation](../../install/requirements.md#gitlab-runner)
+for more information.
+
## Shared vs. Specific Runners
A runner that is specific only runs for the specified project. A shared runner
@@ -19,7 +23,7 @@ many projects, you can have a single or a small number of runners that handle
multiple projects. This makes it easier to maintain and update runners.
**Specific runners** are useful for jobs that have special requirements or for
-projects with a very demand. If a job has certain requirements, you can set
+projects with a specific demand. If a job has certain requirements, you can set
up the specific runner with this in mind, while not having to do this for all
runners. For example, if you want to deploy a certain project, you can setup
a specific runner to have the right credentials for this.
@@ -59,10 +63,10 @@ instance.
Now simply register the runner as any runner:
```
-sudo gitlab-runner register
+sudo gitlab-ci-multi-runner register
```
-Shared runners are enabled by default as of GitLab 8.2, but can be disabled with the
+Shared runners are enabled by default as of GitLab 8.2, but can be disabled with the
`DISABLE SHARED RUNNERS` button. Previous versions of GitLab defaulted shared runners to
disabled.
@@ -89,7 +93,7 @@ setup a specific runner for this project.
To register the runner, run the command below and follow instructions:
```
-sudo gitlab-runner register
+sudo gitlab-ci-multi-runner register
```
### Making an existing Shared Runner Specific
@@ -121,7 +125,13 @@ shared runners will only run the jobs they are equipped to run.
For instance, at GitLab we have runners tagged with "rails" if they contain
the appropriate dependencies to run Rails test suites.
-### Be Careful with Sensitive Information
+### Prevent runner with tags from picking jobs without tags
+
+You can configure a runner to prevent it from picking jobs with tags when
+the runnner does not have tags assigned. This setting is available on each
+runner in *Project Settings* > *Runners*.
+
+### Be careful with sensitive information
If you can run a build on a runner, you can get access to any code it runs
and get the token of the runner. With shared runners, this means that anyone
@@ -140,7 +150,7 @@ to it. This means that if you have shared runners setup for a project and
someone forks that project, the shared runners will also serve jobs of this
project.
-# Attack vectors in runners
+## Attack vectors in Runners
Mentioned briefly earlier, but the following things of runners can be exploited.
We're always looking for contributions that can mitigate these [Security Considerations](https://gitlab.com/gitlab-org/gitlab-ci-multi-runner/blob/master/docs/security/index.md).
diff --git a/doc/ci/services/mysql.md b/doc/ci/services/mysql.md
index c66d77122b2..aaf3aa77837 100644
--- a/doc/ci/services/mysql.md
+++ b/doc/ci/services/mysql.md
@@ -16,7 +16,7 @@ services:
- mysql:latest
variables:
- # Configure mysql environment variables (https://hub.docker.com/_/mysql/)
+ # Configure mysql environment variables (https://hub.docker.com/r/_/mysql/)
MYSQL_DATABASE: el_duderino
MYSQL_ROOT_PASSWORD: mysql_strong_password
```
@@ -114,5 +114,5 @@ available [shared runners](../runners/README.md).
Want to hack on it? Simply fork it, commit and push your changes. Within a few
moments the changes will be picked by a public runner and the build will begin.
-[hub-mysql]: https://hub.docker.com/_/mysql/
+[hub-mysql]: https://hub.docker.com/r/_/mysql/
[mysql-example-repo]: https://gitlab.com/gitlab-examples/mysql
diff --git a/doc/ci/services/postgres.md b/doc/ci/services/postgres.md
index 17d21dbda1c..f787cc0a124 100644
--- a/doc/ci/services/postgres.md
+++ b/doc/ci/services/postgres.md
@@ -110,5 +110,5 @@ available [shared runners](../runners/README.md).
Want to hack on it? Simply fork it, commit and push your changes. Within a few
moments the changes will be picked by a public runner and the build will begin.
-[hub-pg]: https://hub.docker.com/_/postgres/
+[hub-pg]: https://hub.docker.com/r/_/postgres/
[postgres-example-repo]: https://gitlab.com/gitlab-examples/postgres
diff --git a/doc/ci/services/redis.md b/doc/ci/services/redis.md
index b281e8f9f60..80705024d2f 100644
--- a/doc/ci/services/redis.md
+++ b/doc/ci/services/redis.md
@@ -65,5 +65,5 @@ that runs on [GitLab.com](https://gitlab.com) using our publicly available
Want to hack on it? Simply fork it, commit and push your changes. Within a few
moments the changes will be picked by a public runner and the build will begin.
-[hub-redis]: https://hub.docker.com/_/redis/
+[hub-redis]: https://hub.docker.com/r/_/redis/
[redis-example-repo]: https://gitlab.com/gitlab-examples/redis
diff --git a/doc/ci/ssh_keys/README.md b/doc/ci/ssh_keys/README.md
index 210f9c3e849..7c0fb225dac 100644
--- a/doc/ci/ssh_keys/README.md
+++ b/doc/ci/ssh_keys/README.md
@@ -30,7 +30,7 @@ This is the universal solution which works with any type of executor
## SSH keys when using the Docker executor
You will first need to create an SSH key pair. For more information, follow the
-instructions to [generate an SSH key](../ssh/README.md).
+instructions to [generate an SSH key](../../ssh/README.md).
Then, create a new **Secret Variable** in your project settings on GitLab
following **Settings > Variables**. As **Key** add the name `SSH_PRIVATE_KEY`
@@ -57,13 +57,13 @@ before_script:
# WARNING: Use this only with the Docker executor, if you use it with shell
# you will overwrite your user's SSH config.
- mkdir -p ~/.ssh
- - '[[ -f /.dockerinit ]] && echo -e "Host *\n\tStrictHostKeyChecking no\n\n" > ~/.ssh/config`
+ - '[[ -f /.dockerenv ]] && echo -e "Host *\n\tStrictHostKeyChecking no\n\n" > ~/.ssh/config'
```
As a final step, add the _public_ key from the one you created earlier to the
services that you want to have an access to from within the build environment.
If you are accessing a private GitLab repository you need to add it as a
-[deploy key](../ssh/README.md#deploy-keys).
+[deploy key](../../ssh/README.md#deploy-keys).
That's it! You can now have access to private servers or repositories in your
build environment.
@@ -79,12 +79,12 @@ on, and use that key for all projects that are run on this machine.
First, you need to login to the server that runs your builds.
Then from the terminal login as the `gitlab-runner` user and generate the SSH
-key pair as described in the [SSH keys documentation](../ssh/README.md).
+key pair as described in the [SSH keys documentation](../../ssh/README.md).
As a final step, add the _public_ key from the one you created earlier to the
services that you want to have an access to from within the build environment.
If you are accessing a private GitLab repository you need to add it as a
-[deploy key](../ssh/README.md#deploy-keys).
+[deploy key](../../ssh/README.md#deploy-keys).
Once done, try to login to the remote server in order to accept the fingerprint:
diff --git a/doc/ci/triggers/README.md b/doc/ci/triggers/README.md
index 9f7c1bfe6a0..5c316510d0e 100644
--- a/doc/ci/triggers/README.md
+++ b/doc/ci/triggers/README.md
@@ -33,7 +33,7 @@ POST /projects/:id/trigger/builds
The required parameters are the trigger's `token` and the Git `ref` on which
the trigger will be performed. Valid refs are the branch, the tag or the commit
-SHA. The `:id` of a project can be found by [querying the API](../api/projects.md)
+SHA. The `:id` of a project can be found by [querying the API](../../api/projects.md)
or by visiting the **Triggers** page which provides self-explanatory examples.
When a rebuild is triggered, the information is exposed in GitLab's UI under
@@ -85,6 +85,12 @@ curl -X POST \
In this case, the project with ID `9` will get rebuilt on `master` branch.
+Alternatively, you can pass the `token` and `ref` arguments in the query string:
+
+```bash
+curl -X POST \
+ "https://gitlab.example.com/api/v3/projects/9/trigger/builds?token=TOKEN&ref=master"
+```
### Triggering a build within `.gitlab-ci.yml`
diff --git a/doc/ci/variables/README.md b/doc/ci/variables/README.md
index b0e53cbc261..137b080a8f7 100644
--- a/doc/ci/variables/README.md
+++ b/doc/ci/variables/README.md
@@ -1,17 +1,20 @@
## Variables
+
When receiving a build from GitLab CI, the runner prepares the build environment.
It starts by setting a list of **predefined variables** (Environment Variables) and a list of **user-defined variables**
The variables can be overwritten. They take precedence over each other in this order:
+1. Trigger variables
1. Secure variables
-1. YAML-defined variables
+1. YAML-defined job-level variables
+1. YAML-defined global variables
1. Predefined variables
For example, if you define:
-1. API_TOKEN=SECURE as Secure Variable
-1. API_TOKEN=YAML as YAML-defined variable
+1. `API_TOKEN=SECURE` as Secure Variable
+1. `API_TOKEN=YAML` as YAML-defined variable
-The API_TOKEN will take the Secure Variable value: `SECURE`.
+The `API_TOKEN` will take the Secure Variable value: `SECURE`.
### Predefined variables (Environment Variables)
@@ -31,6 +34,7 @@ The API_TOKEN will take the Secure Variable value: `SECURE`.
| **CI_BUILD_ID** | all | The unique id of the current build that GitLab CI uses internally |
| **CI_BUILD_REPO** | all | The URL to clone the Git repository |
| **CI_BUILD_TRIGGERED** | 0.5 | The flag to indicate that build was [triggered] |
+| **CI_BUILD_TOKEN** | 1.2 | Token used for authenticating with the GitLab Container Registry |
| **CI_PROJECT_ID** | all | The unique id of the current project that GitLab CI uses internally |
| **CI_PROJECT_DIR** | all | The full path where the repository is cloned and where the build is ran |
@@ -47,6 +51,7 @@ export CI_BUILD_TAG="1.0.0"
export CI_BUILD_NAME="spec:other"
export CI_BUILD_STAGE="test"
export CI_BUILD_TRIGGERED="true"
+export CI_BUILD_TOKEN="abcde-1234ABCD5678ef"
export CI_PROJECT_DIR="/builds/gitlab-org/gitlab-ce"
export CI_PROJECT_ID="34"
export CI_SERVER="yes"
@@ -70,15 +75,20 @@ These variables can be later used in all executed commands and scripts.
The YAML-defined variables are also set to all created service containers, thus allowing to fine tune them.
+Variables can be defined at a global level, but also at a job level.
+
More information about Docker integration can be found in [Using Docker Images](../docker/using_docker_images.md).
### User-defined variables (Secure Variables)
**This feature requires GitLab Runner 0.4.0 or higher**
-GitLab CI allows you to define per-project **Secure Variables** that are set in build environment.
+GitLab CI allows you to define per-project **Secure Variables** that are set in
+the build environment.
The secure variables are stored out of the repository (the `.gitlab-ci.yml`).
-The variables are securely passed to GitLab Runner and are available in build environment.
-It's desired method to use them for storing passwords, secret keys or whatever you want.
+The variables are securely passed to GitLab Runner and are available in the
+build environment.
+It's desired method to use them for storing passwords, secret keys or whatever
+you want.
**The value of the variable can be shown in build log if explicitly asked to do so.**
If your project is public or internal you can make the builds private.
diff --git a/doc/ci/yaml/README.md b/doc/ci/yaml/README.md
index a9b79bbdb1b..9c98f9c98c6 100644
--- a/doc/ci/yaml/README.md
+++ b/doc/ci/yaml/README.md
@@ -1,5 +1,53 @@
# Configuration of your builds with .gitlab-ci.yml
+This document describes the usage of `.gitlab-ci.yml`, the file that is used by
+GitLab Runner to manage your project's builds.
+
+If you want a quick introduction to GitLab CI, follow our
+[quick start guide](../quick_start/README.md).
+
+---
+
+<!-- START doctoc generated TOC please keep comment here to allow auto update -->
+<!-- DON'T EDIT THIS SECTION, INSTEAD RE-RUN doctoc TO UPDATE -->
+**Table of Contents** *generated with [DocToc](https://github.com/thlorenz/doctoc)*
+
+- [.gitlab-ci.yml](#gitlab-ci-yml)
+ - [image and services](#image-and-services)
+ - [before_script](#before_script)
+ - [after_script](#after_script)
+ - [stages](#stages)
+ - [types](#types)
+ - [variables](#variables)
+ - [cache](#cache)
+ - [cache:key](#cache-key)
+- [Jobs](#jobs)
+ - [script](#script)
+ - [stage](#stage)
+ - [job variables](#job-variables)
+ - [only and except](#only-and-except)
+ - [tags](#tags)
+ - [when](#when)
+ - [environment](#environment)
+ - [artifacts](#artifacts)
+ - [artifacts:name](#artifacts-name)
+ - [artifacts:when](#artifacts-when)
+ - [artifacts:expire_in](#artifacts-expire_in)
+ - [dependencies](#dependencies)
+ - [before_script and after_script](#before_script-and-after_script)
+- [Hidden jobs](#hidden-jobs)
+- [Special YAML features](#special-yaml-features)
+ - [Anchors](#anchors)
+- [Validate the .gitlab-ci.yml](#validate-the-gitlab-ci-yml)
+- [Skipping builds](#skipping-builds)
+- [Examples](#examples)
+
+<!-- END doctoc generated TOC please keep comment here to allow auto update -->
+
+---
+
+## .gitlab-ci.yml
+
From version 7.12, GitLab CI uses a [YAML](https://en.wikipedia.org/wiki/YAML)
file (`.gitlab-ci.yml`) for the project configuration. It is placed in the root
of your repository and contains definitions of how your project should be built.
@@ -23,12 +71,10 @@ Of course a command can execute code directly (`./configure;make;make install`)
or run a script (`test.sh`) in the repository.
Jobs are used to create builds, which are then picked up by
-[runners](../runners/README.md) and executed within the environment of the
-runner. What is important, is that each job is run independently from each
+[Runners](../runners/README.md) and executed within the environment of the
+Runner. What is important, is that each job is run independently from each
other.
-## .gitlab-ci.yml
-
The YAML syntax allows for using more complex job specifications than in the
above example:
@@ -38,7 +84,10 @@ services:
- postgres
before_script:
- - bundle_install
+ - bundle install
+
+after_script:
+ - rm secrets
stages:
- build
@@ -64,6 +113,7 @@ There are a few reserved `keywords` that **cannot** be used as job names:
| stages | no | Define build stages |
| types | no | Alias for `stages` |
| before_script | no | Define commands that run before each job's script |
+| after_script | no | Define commands that run after each job's script |
| variables | no | Define build variables |
| cache | no | Define list of files that should be cached between subsequent runs |
@@ -71,13 +121,21 @@ There are a few reserved `keywords` that **cannot** be used as job names:
This allows to specify a custom Docker image and a list of services that can be
used for time of the build. The configuration of this feature is covered in
-separate document: [Use Docker](../docker/README.md).
+[a separate document](../docker/README.md).
### before_script
`before_script` is used to define the command that should be run before all
builds, including deploy builds. This can be an array or a multi-line string.
+### after_script
+
+>**Note:**
+Introduced in GitLab 8.7 and requires Gitlab Runner v1.2 (not yet released)
+
+`after_script` is used to define the command that will be run after for all
+builds. This has to be an array or a multi-line string.
+
### stages
`stages` is used to define build stages that can be used by jobs.
@@ -86,7 +144,8 @@ The specification of `stages` allows for having flexible multi stage pipelines.
The ordering of elements in `stages` defines the ordering of builds' execution:
1. Builds of the same stage are run in parallel.
-1. Builds of next stage are run after success.
+1. Builds of the next stage are run after the jobs from the previous stage
+ complete successfully.
Let's consider the following example, which defines 3 stages:
@@ -98,9 +157,9 @@ stages:
```
1. First all jobs of `build` are executed in parallel.
-1. If all jobs of `build` succeeds, the `test` jobs are executed in parallel.
-1. If all jobs of `test` succeeds, the `deploy` jobs are executed in parallel.
-1. If all jobs of `deploy` succeeds, the commit is marked as `success`.
+1. If all jobs of `build` succeed, the `test` jobs are executed in parallel.
+1. If all jobs of `test` succeed, the `deploy` jobs are executed in parallel.
+1. If all jobs of `deploy` succeed, the commit is marked as `success`.
1. If any of the previous jobs fails, the commit is marked as `failed` and no
jobs of further stage are executed.
@@ -133,6 +192,8 @@ These variables can be later used in all executed commands and scripts.
The YAML-defined variables are also set to all created service containers,
thus allowing to fine tune them.
+Variables can be also defined on [job level](#job-variables).
+
### cache
>**Note:**
@@ -278,21 +339,27 @@ job_name:
| Keyword | Required | Description |
|---------------|----------|-------------|
-| script | yes | Defines a shell script which is executed by runner |
+| script | yes | Defines a shell script which is executed by Runner |
+| image | no | Use docker image, covered in [Using Docker Images](../docker/using_docker_images.md#define-image-and-services-from-gitlab-ciyml) |
+| services | no | Use docker services, covered in [Using Docker Images](../docker/using_docker_images.md#define-image-and-services-from-gitlab-ciyml) |
| stage | no | Defines a build stage (default: `test`) |
| type | no | Alias for `stage` |
+| variables | no | Define build variables on a job level |
| only | no | Defines a list of git refs for which build is created |
| except | no | Defines a list of git refs for which build is not created |
-| tags | no | Defines a list of tags which are used to select runner |
+| tags | no | Defines a list of tags which are used to select Runner |
| allow_failure | no | Allow build to fail. Failed build doesn't contribute to commit status |
| when | no | Define when to run build. Can be `on_success`, `on_failure` or `always` |
| dependencies | no | Define other builds that a build depends on so that you can pass artifacts between them|
-| artifacts | no | Define list build artifacts |
+| artifacts | no | Define list of build artifacts |
| cache | no | Define list of files that should be cached between subsequent runs |
+| before_script | no | Override a set of commands that are executed before build |
+| after_script | no | Override a set of commands that are executed after build |
+| environment | no | Defines a name of environment to which deployment is done by this build |
### script
-`script` is a shell script which is executed by the runner. For example:
+`script` is a shell script which is executed by the Runner. For example:
```yaml
job:
@@ -329,7 +396,7 @@ There are a few rules that apply to the usage of refs policy:
* `only` and `except` are inclusive. If both `only` and `except` are defined
in a job specification, the ref is filtered by `only` and `except`.
* `only` and `except` allow the use of regular expressions.
-* `only` and `except` allow the use of special keywords: `branches` and `tags`.
+* `only` and `except` allow the use of special keywords: `branches`, `tags`, and `triggers`.
* `only` and `except` allow to specify a repository path to filter jobs for
forks.
@@ -346,6 +413,17 @@ job:
- branches
```
+In this example, `job` will run only for refs that are tagged, or if a build is explicitly requested
+via an API trigger.
+
+```yaml
+job:
+ # use special keywords
+ only:
+ - tags
+ - triggers
+```
+
The repository path can be used to have jobs executed only for the parent
repository and not forks:
@@ -360,15 +438,27 @@ job:
The above example will run `job` for all branches on `gitlab-org/gitlab-ce`,
except master.
+### job variables
+
+It is possible to define build variables using a `variables` keyword on a job
+level. It works basically the same way as its global-level equivalent but
+allows you to define job-specific build variables.
+
+When the `variables` keyword is used on a job level, it overrides global YAML
+build variables and predefined variables.
+
+Build variables priority is defined in
+[variables documentation](../variables/README.md).
+
### tags
-`tags` is used to select specific runners from the list of all runners that are
+`tags` is used to select specific Runners from the list of all Runners that are
allowed to run this project.
-During the registration of a runner, you can specify the runner's tags, for
+During the registration of a Runner, you can specify the Runner's tags, for
example `ruby`, `postgres`, `development`.
-`tags` allow you to run builds with runners that have the specified tags
+`tags` allow you to run builds with Runners that have the specified tags
assigned to them:
```yaml
@@ -378,7 +468,7 @@ job:
- postgres
```
-The specification above, will make sure that `job` is built by a runner that
+The specification above, will make sure that `job` is built by a Runner that
has both `ruby` AND `postgres` tags defined.
### when
@@ -437,6 +527,31 @@ The above script will:
1. Execute `cleanup_build_job` only when `build_job` fails
2. Always execute `cleanup_job` as the last step in pipeline.
+### environment
+
+>**Note:**
+Introduced in GitLab v8.9.0.
+
+`environment` is used to define that job does deployment to specific environment.
+This allows to easily track all deployments to your environments straight from GitLab.
+
+If `environment` is specified and no environment under that name does exist a new one will be created automatically.
+
+The `environment` name must contain only letters, digits, '-' and '_'.
+
+---
+
+**Example configurations**
+
+```
+deploy to production:
+ stage: deploy
+ script: git push production HEAD:master
+ environment: production
+```
+
+The `deploy to production` job will be marked as doing deployment to `production` environment.
+
### artifacts
>**Notes:**
@@ -565,6 +680,66 @@ job:
untracked: true
```
+#### artifacts:when
+
+>**Note:**
+Introduced in GitLab 8.9 and GitLab Runner v1.3.0.
+
+`artifacts:when` is used to upload artifacts on build failure or despite the
+failure.
+
+`artifacts:when` can be set to one of the following values:
+
+1. `on_success` - upload artifacts only when build succeeds. This is the default
+1. `on_failure` - upload artifacts only when build fails
+1. `always` - upload artifacts despite the build status
+
+---
+
+**Example configurations**
+
+To upload artifacts only when build fails.
+
+```yaml
+job:
+ artifacts:
+ when: on_failure
+```
+
+#### artifacts:expire_in
+
+>**Note:**
+Introduced in GitLab 8.9 and GitLab Runner v1.3.0.
+
+`artifacts:expire_in` is used to remove uploaded artifacts after specified time.
+By default artifacts are stored on GitLab forver.
+`expire_in` allows to specify after what time the artifacts should be removed.
+The artifacts will expire counting from the moment when they are uploaded and stored on GitLab.
+
+After artifacts uploading you can use the **Keep** button on build page to keep the artifacts forever.
+
+Artifacts are removed every hour, but they are not accessible after expire date.
+
+The value of `expire_in` is a elapsed time. The example of parsable values:
+- '3 mins 4 sec'
+- '2 hrs 20 min'
+- '2h20min'
+- '6 mos 1 day'
+- '47 yrs 6 mos and 4d'
+- '3 weeks and 2 days'
+
+---
+
+**Example configurations**
+
+To expire artifacts after 1 week from the moment that they are uploaded:
+
+```yaml
+job:
+ artifacts:
+ expire_in: 1 week
+```
+
### dependencies
>**Note:**
@@ -622,6 +797,23 @@ deploy:
script: make deploy
```
+### before_script and after_script
+
+It's possible to overwrite globally defined `before_script` and `after_script`:
+
+```yaml
+before_script
+- global before script
+
+job:
+ before_script:
+ - execute this instead of global before script
+ script:
+ - my command
+ after_script:
+ - execute this after my script
+```
+
## Hidden jobs
>**Note:**
diff --git a/doc/container_registry/README.md b/doc/container_registry/README.md
new file mode 100644
index 00000000000..1b465434498
--- /dev/null
+++ b/doc/container_registry/README.md
@@ -0,0 +1,94 @@
+# GitLab Container Registry
+
+> **Note:**
+This feature was [introduced][ce-4040] in GitLab 8.8.
+
+> **Note:**
+This document is about the user guide. To learn how to enable GitLab Container
+Registry across your GitLab instance, visit the
+[administrator documentation](../administration/container_registry.md).
+
+With the Docker Container Registry integrated into GitLab, every project can
+have its own space to store its Docker images.
+
+You can read more about Docker Registry at https://docs.docker.com/registry/introduction/.
+
+---
+
+## Enable the Container Registry for your project
+
+1. First, ask your system administrator to enable GitLab Container Registry
+ following the [administration documentation](../administration/container_registry.md).
+ If you are using GitLab.com, this is enabled by default so you can start using
+ the Registry immediately.
+
+1. Go to your project's settings and enable the **Container Registry** feature
+ on your project. For new projects this might be enabled by default. For
+ existing projects you will have to explicitly enable it.
+
+ ![Enable Container Registry](img/project_feature.png)
+
+## Build and push images
+
+After you save your project's settings, you should see a new link in the
+sidebar called **Container Registry**. Following this link will get you to
+your project's Registry panel where you can see how to login to the Container
+Registry using your GitLab credentials.
+
+For example if the Registry's URL is `registry.example.com`, the you should be
+able to login with:
+
+```
+docker login registry.example.com
+```
+
+Building and publishing images should be a straightforward process. Just make
+sure that you are using the Registry URL with the namespace and project name
+that is hosted on GitLab:
+
+```
+docker build -t registry.example.com/group/project .
+docker push registry.example.com/group/project
+```
+
+## Use images from GitLab Container Registry
+
+To download and run a container from images hosted in GitLab Container Registry,
+use `docker run`:
+
+```
+docker run [options] registry.example.com/group/project [arguments]
+```
+
+For more information on running Docker containers, visit the
+[Docker documentation][docker-docs].
+
+## Control Container Registry from within GitLab
+
+GitLab offers a simple Container Registry management panel. Go to your project
+and click **Container Registry** in the left sidebar.
+
+This view will show you all tags in your project and will easily allow you to
+delete them.
+
+![Container Registry panel](img/container_registry.png)
+
+## Build and push images using GitLab CI
+
+> **Note:**
+This feature requires GitLab 8.8 and GitLab Runner 1.2.
+
+Make sure that your GitLab Runner is configured to allow building docker images.
+You have to check the [Using Docker Build documentation](../ci/docker/using_docker_build.md).
+Then see the CI documentation on [Using the GitLab Container Registry](../ci/docker/using_docker_build.md#using-the-gitlab-container-registry).
+
+## Limitations
+
+In order to use a container image from your private project as an `image:` in
+your `.gitlab-ci.yml`, you have to follow the
+[Using a private Docker Registry][private-docker]
+documentation. This workflow will be simplified in the future.
+
+[ce-4040]: https://gitlab.com/gitlab-org/gitlab-ce/merge_requests/4040
+[docker-docs]: https://docs.docker.com/engine/userguide/intro/
+[private-docker]: https://gitlab.com/gitlab-org/gitlab-ci-multi-runner/blob/master/docs/configuration/advanced-configuration.md#using-a-private-docker-registry
diff --git a/doc/container_registry/img/container_registry.png b/doc/container_registry/img/container_registry.png
new file mode 100644
index 00000000000..e9505a73b40
--- /dev/null
+++ b/doc/container_registry/img/container_registry.png
Binary files differ
diff --git a/doc/container_registry/img/project_feature.png b/doc/container_registry/img/project_feature.png
new file mode 100644
index 00000000000..57a73d253c0
--- /dev/null
+++ b/doc/container_registry/img/project_feature.png
Binary files differ
diff --git a/doc/customization/libravatar.md b/doc/customization/libravatar.md
index bd2c242afc2..c46ce2ee203 100644
--- a/doc/customization/libravatar.md
+++ b/doc/customization/libravatar.md
@@ -67,3 +67,16 @@ Run `sudo gitlab-ctl reconfigure` for changes to take effect.
In order to use a different set other than `identicon`, replace `&d=identicon` portion of the URL with another supported set.
For example, you can use `retro` set in which case the URL would look like: `plain_url: "http://cdn.libravatar.org/avatar/%{hash}?s=%{size}&d=retro"`
+
+
+## Usage examples
+
+#### For Microsoft Office 365
+
+If your users are Office 365-users, the "GetPersonaPhoto" service can be used. Note that this service requires login, so this use case is
+most useful in a corporate installation, where all users have access to Office 365.
+
+```ruby
+gitlab_rails['gravatar_plain_url'] = 'http://outlook.office365.com/owa/service.svc/s/GetPersonaPhoto?email=%{email}&size=HR120x120'
+gitlab_rails['gravatar_ssl_url'] = 'https://outlook.office365.com/owa/service.svc/s/GetPersonaPhoto?email=%{email}&size=HR120x120'
+```
diff --git a/doc/development/README.md b/doc/development/README.md
index 1b281809afc..c5d5af43864 100644
--- a/doc/development/README.md
+++ b/doc/development/README.md
@@ -2,11 +2,17 @@
- [Architecture](architecture.md) of GitLab
- [CI setup](ci_setup.md) for testing GitLab
+- [Code review guidelines](code_review.md) for reviewing code and having code
+ reviewed.
- [Gotchas](gotchas.md) to avoid
- [How to dump production data to staging](db_dump.md)
+- [Instrumentation](instrumentation.md)
+- [Licensing](licensing.md) for ensuring license compliance
- [Migration Style Guide](migration_style_guide.md) for creating safe migrations
+- [Performance guidelines](performance.md)
- [Rake tasks](rake_tasks.md) for development
- [Shell commands](shell_commands.md) in the GitLab codebase
- [Sidekiq debugging](sidekiq_debugging.md)
- [SQL guidelines](sql.md) for SQL guidelines
+- [Testing standards and style guidelines](testing.md)
- [UI guide](ui_guide.md) for building GitLab with existing css styles and elements
diff --git a/doc/development/code_review.md b/doc/development/code_review.md
new file mode 100644
index 00000000000..40ae55ab905
--- /dev/null
+++ b/doc/development/code_review.md
@@ -0,0 +1,78 @@
+# Code Review Guidelines
+
+This guide contains advice and best practices for performing code review, and
+having your code reviewed.
+
+All merge requests for GitLab CE and EE, whether written by a GitLab team member
+or a volunteer contributor, must go through a code review process to ensure the
+code is effective, understandable, and maintainable.
+
+Any developer can, and is encouraged to, perform code review on merge requests
+of colleagues and contributors. However, the final decision to accept a merge
+request is up to one of our merge request "endbosses", denoted on the
+[team page](https://about.gitlab.com/team).
+
+## Everyone
+
+- Accept that many programming decisions are opinions. Discuss tradeoffs, which
+ you prefer, and reach a resolution quickly.
+- Ask questions; don't make demands. ("What do you think about naming this
+ `:user_id`?")
+- Ask for clarification. ("I didn't understand. Can you clarify?")
+- Avoid selective ownership of code. ("mine", "not mine", "yours")
+- Avoid using terms that could be seen as referring to personal traits. ("dumb",
+ "stupid"). Assume everyone is attractive, intelligent, and well-meaning.
+- Be explicit. Remember people don't always understand your intentions online.
+- Be humble. ("I'm not sure - let's look it up.")
+- Don't use hyperbole. ("always", "never", "endlessly", "nothing")
+- Be careful about the use of sarcasm. Everything we do is public; what seems
+ like good-natured ribbing to you and a long-time colleague might come off as
+ mean and unwelcoming to a person new to the project.
+- Consider one-on-one chats or video calls if there are too many "I didn't
+ understand" or "Alternative solution:" comments. Post a follow-up comment
+ summarizing one-on-one discussion.
+
+## Having your code reviewed
+
+- The first reviewer of your code is _you_. Before you perform that first push
+ of your shiny new branch, read through the entire diff. Does it make sense?
+ Did you include something unrelated to the overall purpose of the changes? Did
+ you forget to remove any debugging code?
+- Be grateful for the reviewer's suggestions. ("Good call. I'll make that
+ change.")
+- Don't take it personally. The review is of the code, not of you.
+- Explain why the code exists. ("It's like that because of these reasons. Would
+ it be more clear if I rename this class/file/method/variable?")
+- Extract unrelated changes and refactorings into future merge requests/issues.
+- Seek to understand the reviewer's perspective.
+- Try to respond to every comment.
+- Push commits based on earlier rounds of feedback as isolated commits to the
+ branch. Do not squash until the branch is ready to merge. Reviewers should be
+ able to read individual updates based on their earlier feedback.
+
+## Reviewing code
+
+Understand why the change is necessary (fixes a bug, improves the user
+experience, refactors the existing code). Then:
+
+- Communicate which ideas you feel strongly about and those you don't.
+- Identify ways to simplify the code while still solving the problem.
+- Offer alternative implementations, but assume the author already considered
+ them. ("What do you think about using a custom validator here?")
+- Seek to understand the author's perspective.
+- If you don't understand a piece of code, _say so_. There's a good chance
+ someone else would be confused by it as well.
+- After a round of line notes, it can be helpful to post a summary note such as
+ "LGTM :thumbsup:", or "Just a couple things to address."
+- Avoid accepting a merge request before the build succeeds ("Merge when build
+ succeeds" is fine).
+
+## Credits
+
+Largely based on the [thoughtbot code review guide].
+
+[thoughtbot code review guide]: https://github.com/thoughtbot/guides/tree/master/code-review
+
+---
+
+[Return to Development documentation](README.md)
diff --git a/doc/development/doc_styleguide.md b/doc/development/doc_styleguide.md
index 187ec9e7b75..f5d97179f8a 100644
--- a/doc/development/doc_styleguide.md
+++ b/doc/development/doc_styleguide.md
@@ -103,14 +103,14 @@ Inside the document:
- Every piece of documentation that comes with a new feature should declare the
GitLab version that feature got introduced. Right below the heading add a
- note: `_**Note:** This feature was introduced in GitLab 8.3_`
+ note: `>**Note:** This feature was introduced in GitLab 8.3`
- If possible every feature should have a link to the MR that introduced it.
The above note would be then transformed to:
- `_**Note:** This feature was [introduced][ce-1242] in GitLab 8.3_`, where
+ `>**Note:** This feature was [introduced][ce-1242] in GitLab 8.3`, where
the [link identifier](#links) is named after the repository (CE) and the MR
number
- If the feature is only in GitLab EE, don't forget to mention it, like:
- `_**Note:** This feature was introduced in GitLab EE 8.3_`. Otherwise, leave
+ `>**Note:** This feature was introduced in GitLab EE 8.3`. Otherwise, leave
this mention out
## References
@@ -127,7 +127,7 @@ Inside the document:
```
If the document you are editing resides in a place other than the GitLab CE/EE
`doc/` directory, instead of the relative link, use the full path:
- `http://doc.gitlab.com/ce/administration/restart_gitlab.html`.
+ `http://docs.gitlab.com/ce/administration/restart_gitlab.html`.
Replace `reconfigure` with `restart` where appropriate.
## Installation guide
@@ -141,6 +141,48 @@ Inside the document:
[ruby-dl]: https://www.ruby-lang.org/en/downloads/ "Ruby download website"
+## Changing document location
+
+Changing a document's location is not to be taken lightly. Remember that the
+documentation is available to all installations under `help/` and not only to
+GitLab.com or http://docs.gitlab.com. Make sure this is discussed with the
+Documentation team beforehand.
+
+If you indeed need to change a document's location, do NOT remove the old
+document, but rather put a text in it that points to the new location, like:
+
+```
+This document was moved to [path/to/new_doc.md](path/to/new_doc.md).
+```
+
+where `path/to/new_doc.md` is the relative path to the root directory `doc/`.
+
+---
+
+For example, if you were to move `doc/workflow/lfs/lfs_administration.md` to
+`doc/administration/lfs.md`, then the steps would be:
+
+1. Copy `doc/workflow/lfs/lfs_administration.md` to `doc/administration/lfs.md`
+1. Replace the contents of `doc/workflow/lfs/lfs_administration.md` with:
+
+ ```
+ This document was moved to [administration/lfs.md](../../administration/lfs.md).
+ ```
+
+1. Find and replace any occurrences of the old location with the new one.
+ A quick way to find them is to use `grep`:
+
+ ```
+ grep -nR "lfs_administration.md" doc/
+ ```
+
+ The above command will search in the `doc/` directory for
+ `lfs_administration.md` recursively and will print the file and the line
+ where this file is mentioned. Note that we used just the filename
+ (`lfs_administration.md`) and not the whole the relative path
+ (`workflow/lfs/lfs_administration.md`).
+
+
## API
Here is a list of must-have items. Use them in the exact order that appears
@@ -222,8 +264,8 @@ curl --data "name=foo" -H "PRIVATE-TOKEN: 9koXpg98eAheJpvBs5tK" "https://gitlab.
#### Post data using JSON content
-_**Note:** In this example we create a new group. Watch carefully the single
-and double quotes._
+> **Note:** In this example we create a new group. Watch carefully the single
+and double quotes.
```bash
curl -X POST -H "PRIVATE-TOKEN: 9koXpg98eAheJpvBs5tK" -H "Content-Type: application/json" --data '{"path": "my-group", "name": "My group"}' https://gitlab.example.com/api/v3/groups
@@ -266,5 +308,5 @@ curl -X PUT -H "PRIVATE-TOKEN: 9koXpg98eAheJpvBs5tK" -d "restricted_signup_domai
[cURL]: http://curl.haxx.se/ "cURL website"
[single spaces]: http://www.slate.com/articles/technology/technology/2011/01/space_invaders.html
-[gfm]: http://doc.gitlab.com/ce/markdown/markdown.html#newlines "GitLab flavored markdown documentation"
+[gfm]: http://docs.gitlab.com/ce/markdown/markdown.html#newlines "GitLab flavored markdown documentation"
[doc-restart]: ../administration/restart_gitlab.md "GitLab restart documentation"
diff --git a/doc/development/instrumentation.md b/doc/development/instrumentation.md
new file mode 100644
index 00000000000..c2272ab0a2b
--- /dev/null
+++ b/doc/development/instrumentation.md
@@ -0,0 +1,139 @@
+# Instrumenting Ruby Code
+
+GitLab Performance Monitoring allows instrumenting of both methods and custom
+blocks of Ruby code. Method instrumentation is the primary form of
+instrumentation with block-based instrumentation only being used when we want to
+drill down to specific regions of code within a method.
+
+## Instrumenting Methods
+
+Instrumenting methods is done by using the `Gitlab::Metrics::Instrumentation`
+module. This module offers a few different methods that can be used to
+instrument code:
+
+* `instrument_method`: instruments a single class method.
+* `instrument_instance_method`: instruments a single instance method.
+* `instrument_class_hierarchy`: given a Class this method will recursively
+ instrument all sub-classes (both class and instance methods).
+* `instrument_methods`: instruments all public and private class methods of a Module.
+* `instrument_instance_methods`: instruments all public and private instance methods of a
+ Module.
+
+To remove the need for typing the full `Gitlab::Metrics::Instrumentation`
+namespace you can use the `configure` class method. This method simply yields
+the supplied block while passing `Gitlab::Metrics::Instrumentation` as its
+argument. An example:
+
+```
+Gitlab::Metrics::Instrumentation.configure do |conf|
+ conf.instrument_method(Foo, :bar)
+ conf.instrument_method(Foo, :baz)
+end
+```
+
+Using this method is in general preferred over directly calling the various
+instrumentation methods.
+
+Method instrumentation should be added in the initializer
+`config/initializers/metrics.rb`.
+
+### Examples
+
+Instrumenting a single method:
+
+```
+Gitlab::Metrics::Instrumentation.configure do |conf|
+ conf.instrument_method(User, :find_by)
+end
+```
+
+Instrumenting an entire class hierarchy:
+
+```
+Gitlab::Metrics::Instrumentation.configure do |conf|
+ conf.instrument_class_hierarchy(ActiveRecord::Base)
+end
+```
+
+Instrumenting all public class methods:
+
+```
+Gitlab::Metrics::Instrumentation.configure do |conf|
+ conf.instrument_methods(User)
+end
+```
+
+### Checking Instrumented Methods
+
+The easiest way to check if a method has been instrumented is to check its
+source location. For example:
+
+```
+method = Rugged::TagCollection.instance_method(:[])
+
+method.source_location
+```
+
+If the source location points to `lib/gitlab/metrics/instrumentation.rb` you
+know the method has been instrumented.
+
+If you're using Pry you can use the `$` command to display the source code of a
+method (along with its source location), this is easier than running the above
+Ruby code. In case of the above snippet you'd run the following:
+
+```
+$ Rugged::TagCollection#[]
+```
+
+This will print out something along the lines of:
+
+```
+From: /path/to/your/gitlab/lib/gitlab/metrics/instrumentation.rb @ line 148:
+Owner: #<Module:0x0055f0865c6d50>
+Visibility: public
+Number of lines: 21
+
+def #{name}(#{args_signature})
+ if trans = Gitlab::Metrics::Instrumentation.transaction
+ trans.measure_method(#{label.inspect}) { super }
+ else
+ super
+ end
+end
+```
+
+## Instrumenting Ruby Blocks
+
+Measuring blocks of Ruby code is done by calling `Gitlab::Metrics.measure` and
+passing it a block. For example:
+
+```ruby
+Gitlab::Metrics.measure(:foo) do
+ ...
+end
+```
+
+The block is executed and the execution time is stored as a set of fields in the
+currently running transaction. If no transaction is present the block is yielded
+without measuring anything.
+
+3 values are measured for a block:
+
+1. The real time elapsed, stored in NAME_real_time.
+2. The CPU time elapsed, stored in NAME_cpu_time.
+3. The call count, stored in NAME_call_count.
+
+Both the real and CPU timings are measured in milliseconds.
+
+Multiple calls to the same block will result in the final values being the sum
+of all individual values. Take this code for example:
+
+```ruby
+3.times do
+ Gitlab::Metrics.measure(:sleep) do
+ sleep 1
+ end
+end
+```
+
+Here the final value of `sleep_real_time` will be `3`, _not_ `1`.
diff --git a/doc/development/licensing.md b/doc/development/licensing.md
new file mode 100644
index 00000000000..8c8c7486fff
--- /dev/null
+++ b/doc/development/licensing.md
@@ -0,0 +1,93 @@
+# GitLab Licensing and Compatibility
+
+GitLab CE is licensed under the terms of the MIT License. GitLab EE is licensed under "The GitLab Enterprise Edition (EE) license" wherein there are more restrictions. See their respective LICENSE files ([CE][CE], [EE][EE]) for more information.
+
+## Automated Testing
+
+In order to comply with the terms the libraries we use are licensed under, we have to make sure to check new gems for compatible licenses whenever they're added. To automate this process, we use the [license_finder][license_finder] gem by Pivotal. It runs every time a new commit is pushed and verifies that all gems in the bundle use a license that doesn't conflict with the licensing of either GitLab Community Edition or GitLab Enterprise Edition.
+
+There are some limitations with the automated testing, however. CSS and JavaScript libraries, as well as any Ruby libraries not included by way of Bundler, must be verified manually and independently. Take care whenever one such library is used, as automated tests won't catch problematic licenses from them.
+
+Some gems may not include their license information in their `gemspec` file. These won't be detected by License Finder, and will have to be verified manually.
+
+### License Finder commands
+
+There are a few basic commands License Finder provides that you'll need in order to manage license detection.
+
+To verify that the checks are passing, and/or to see what dependencies are causing the checks to fail:
+
+```
+bundle exec license_finder
+```
+
+To whitelist a new license:
+
+```
+license_finder whitelist add MIT
+```
+
+To blacklist a new license:
+
+```
+license_finder blacklist add GPLv2
+```
+
+To tell License Finder about a dependency's license if it isn't auto-detected:
+
+```
+license_finder licenses add my_unknown_dependency MIT
+```
+
+For all of the above, please include `--why "Reason"` and `--who "My Name"` so the `decisions.yml` file can keep track of when, why, and who approved of a dependency.
+
+More detailed information on how the gem and its commands work is available in the [License Finder README][license_finder].
+
+## Acceptable Licenses
+
+Libraries with the following licenses are acceptable for use:
+
+- [The MIT License][MIT] (the MIT Expat License specifically): The MIT License requires that the license itself is included with all copies of the source. It is a permissive (non-copyleft) license as defined by the Open Source Initiative.
+- [LGPL][LGPL] (version 2, version 3): GPL constraints regarding modification and redistribution under the same license are not required of projects using an LGPL library, only upon modification of the LGPL-licensed library itself.
+- [Apache 2.0 License][apache-2]: A permissive license that also provides an express grant of patent rights from contributors to users.
+- [Ruby 1.8 License][ruby-1.8]: Dual-licensed under either itself or the GPLv2, defer to the Ruby License itself. Acceptable because of point 3b: "You may distribute the software in object code or binary form, provided that you do at least ONE of the following: b) accompany the distribution with the machine-readable source of the software."
+- [Ruby 1.9 License][ruby-1.9]: Dual-licensed under either itself or the BSD 2-Clause License, defer to BSD 2-Clause.
+- [BSD 2-Clause License][BSD-2-Clause]: A permissive (non-copyleft) license as defined by the Open Source Initiative.
+- [BSD 3-Clause License][BSD-3-Clause] (also known as New BSD or Modified BSD): A permissive (non-copyleft) license as defined by the Open Source Initiative
+- [ISC License][ISC] (also known as the OpenBSD License): A permissive (non-copyleft) license as defined by the Open Source Initiative.
+
+## Unacceptable Licenses
+
+Libraries with the following licenses are unacceptable for use:
+
+- [GNU GPL][GPL] (version 1, [version 2][GPLv2], [version 3][GPLv3], or any future versions): GPL-licensed libraries cannot be linked to from non-GPL projects.
+- [GNU AGPLv3][AGPLv3]: AGPL-licensed libraries cannot be linked to from non-GPL projects.
+
+## Notes
+
+Decisions regarding the GNU GPL licenses are based on information provided by [The GNU Project][GNU-GPL-FAQ], as well as [the Open Source Initiative][OSI-GPL], which both state that linking GPL libraries makes the program itself GPL.
+
+If a gem uses a license which is not listed above, open an issue and ask. If a license is not included in the "acceptable" list, operate under the assumption that it is not acceptable.
+
+Keep in mind that each license has its own restrictions (typically defined in their body text). Please make sure to comply with those restrictions at all times whenever an external library is used.
+
+Gems which are included only in the "development" or "test" groups by Bundler are exempt from license requirements, as they're not distributed for use in production.
+
+**NOTE:** This document is **not** legal advice, nor is it comprehensive. It should not be taken as such.
+
+[CE]: https://gitlab.com/gitlab-org/gitlab-ce/blob/master/LICENSE
+[EE]: https://gitlab.com/gitlab-org/gitlab-ee/blob/master/LICENSE
+[license_finder]: https://github.com/pivotal/LicenseFinder
+[MIT]: http://choosealicense.com/licenses/mit/
+[LGPL]: http://choosealicense.com/licenses/lgpl-3.0/
+[apache-2]: http://choosealicense.com/licenses/apache-2.0/
+[ruby-1.8]: https://github.com/ruby/ruby/blob/ruby_1_8_6/COPYING
+[ruby-1.9]: https://www.ruby-lang.org/en/about/license.txt
+[BSD-2-Clause]: https://opensource.org/licenses/BSD-2-Clause
+[BSD-3-Clause]: https://opensource.org/licenses/BSD-3-Clause
+[ISC]: https://opensource.org/licenses/ISC
+[GPL]: http://choosealicense.com/licenses/gpl-3.0/
+[GPLv2]: http://www.gnu.org/licenses/gpl-2.0.txt
+[GPLv3]: http://www.gnu.org/licenses/gpl-3.0.txt
+[AGPLv3]: http://choosealicense.com/licenses/agpl-3.0/
+[GNU-GPL-FAQ]: http://www.gnu.org/licenses/gpl-faq.html#IfLibraryIsGPL
+[OSI-GPL]: https://opensource.org/faq#linking-proprietary-code
diff --git a/doc/development/migration_style_guide.md b/doc/development/migration_style_guide.md
index 28dedf3978c..8a7547e5322 100644
--- a/doc/development/migration_style_guide.md
+++ b/doc/development/migration_style_guide.md
@@ -8,7 +8,10 @@ In addition, having to take a server offline for a an upgrade small or big is
a big burden for most organizations. For this reason it is important that your
migrations are written carefully, can be applied online and adhere to the style guide below.
-It's advised to have offline migrations only in major GitLab releases.
+Migrations should not require GitLab installations to be taken offline unless
+_absolutely_ necessary. If a migration requires downtime this should be
+clearly mentioned during the review process as well as being documented in the
+monthly release post.
When writing your migrations, also consider that databases might have stale data
or inconsistencies and guard for that. Try to make as little assumptions as possible
@@ -31,6 +34,15 @@ First, you need to provide information on whether the migration can be applied:
3. online with errors on new instances while migrating
4. offline (needs to happen without app servers to prevent db corruption)
+For example:
+
+```
+# rubocop:disable all
+# Migration type: online without errors (works on previous version and new one)
+class MyMigration < ActiveRecord::Migration
+...
+```
+
It is always preferable to have a migration run online. If you expect the migration
to take particularly long (for instance, if it loops through all notes),
this is valuable information to add.
@@ -45,7 +57,6 @@ be possible to downgrade in case of a vulnerability or bugs.
In your migration, add a comment describing how the reversibility of the
migration was tested.
-
## Removing indices
If you need to remove index, please add a condition like in following example:
@@ -58,6 +69,49 @@ remove_index :namespaces, column: :name if index_exists?(:namespaces, :name)
If you need to add an unique index please keep in mind there is possibility of existing duplicates. If it is possible write a separate migration for handling this situation. It can be just removing or removing with overwriting all references to these duplicates depend on situation.
+When adding an index make sure to use the method `add_concurrent_index` instead
+of the regular `add_index` method. The `add_concurrent_index` method
+automatically creates concurrent indexes when using PostgreSQL, removing the
+need for downtime. To use this method you must disable transactions by calling
+the method `disable_ddl_transaction!` in the body of your migration class like
+so:
+
+```
+class MyMigration < ActiveRecord::Migration
+ include Gitlab::Database::MigrationHelpers
+ disable_ddl_transaction!
+
+ def change
+
+ end
+end
+```
+
+## Adding Columns With Default Values
+
+When adding columns with default values you should use the method
+`add_column_with_default`. This method ensures the table is updated without
+requiring downtime. This method is not reversible so you must manually define
+the `up` and `down` methods in your migration class.
+
+For example, to add the column `foo` to the `projects` table with a default
+value of `10` you'd write the following:
+
+```
+class MyMigration < ActiveRecord::Migration
+ include Gitlab::Database::MigrationHelpers
+ disable_ddl_transaction!
+
+ def up
+ add_column_with_default(:projects, :foo, :integer, default: 10)
+ end
+
+ def down
+ remove_column(:projects, :foo)
+ end
+end
+```
+
## Testing
Make sure that your migration works with MySQL and PostgreSQL with data. An empty database does not guarantee that your migration is correct.
@@ -74,7 +128,7 @@ Example with Arel:
users = Arel::Table.new(:users)
users.group(users[:user_id]).having(users[:id].count.gt(5))
-#updtae other tables with this results
+#update other tables with these results
```
Example with plain SQL and `quote_string` helper:
@@ -89,4 +143,4 @@ select_all("SELECT name, COUNT(id) as cnt FROM tags GROUP BY name HAVING COUNT(i
execute("UPDATE taggings SET tag_id = #{origin_tag_id} WHERE tag_id IN(#{duplicate_ids.join(",")})")
execute("DELETE FROM tags WHERE id IN(#{duplicate_ids.join(",")})")
end
-``` \ No newline at end of file
+```
diff --git a/doc/development/performance.md b/doc/development/performance.md
new file mode 100644
index 00000000000..fb37b3a889c
--- /dev/null
+++ b/doc/development/performance.md
@@ -0,0 +1,258 @@
+# Performance Guidelines
+
+This document describes various guidelines to follow to ensure good and
+consistent performance of GitLab.
+
+## Workflow
+
+The process of solving performance problems is roughly as follows:
+
+1. Make sure there's an issue open somewhere (e.g., on the GitLab CE issue
+ tracker), create one if there isn't. See [#15607][#15607] for an example.
+2. Measure the performance of the code in a production environment such as
+ GitLab.com (see the [Tooling](#tooling) section below). Performance should be
+ measured over a period of _at least_ 24 hours.
+3. Add your findings based on the measurement period (screenshots of graphs,
+ timings, etc) to the issue mentioned in step 1.
+4. Solve the problem.
+5. Create a merge request, assign the "performance" label and ping the right
+ people (e.g. [@yorickpeterse][yorickpeterse] and [@joshfng][joshfng]).
+6. Once a change has been deployed make sure to _again_ measure for at least 24
+ hours to see if your changes have any impact on the production environment.
+7. Repeat until you're done.
+
+When providing timings make sure to provide:
+
+* The 95th percentile
+* The 99th percentile
+* The mean
+
+When providing screenshots of graphs, make sure that both the X and Y axes and
+the legend are clearly visible. If you happen to have access to GitLab.com's own
+monitoring tools you should also provide a link to any relevant
+graphs/dashboards.
+
+## Tooling
+
+GitLab provides two built-in tools to aid the process of improving performance:
+
+* [Sherlock](doc/development/profiling.md#sherlock)
+* [GitLab Performance Monitoring](doc/monitoring/performance/monitoring.md)
+
+GitLab employees can use GitLab.com's performance monitoring systems located at
+<http://performance.gitlab.net>, this requires you to log in using your
+`@gitlab.com` Email address. Non-GitLab employees are advised to set up their
+own InfluxDB + Grafana stack.
+
+## Benchmarks
+
+Benchmarks are almost always useless. Benchmarks usually only test small bits of
+code in isolation and often only measure the best case scenario. On top of that,
+benchmarks for libraries (e.g., a Gem) tend to be biased in favour of the
+library. After all there's little benefit to an author publishing a benchmark
+that shows they perform worse than their competitors.
+
+Benchmarks are only really useful when you need a rough (emphasis on "rough")
+understanding of the impact of your changes. For example, if a certain method is
+slow a benchmark can be used to see if the changes you're making have any impact
+on the method's performance. However, even when a benchmark shows your changes
+improve performance there's no guarantee the performance also improves in a
+production environment.
+
+When writing benchmarks you should almost always use
+[benchmark-ips](https://github.com/evanphx/benchmark-ips). Ruby's `Benchmark`
+module that comes with the standard library is rarely useful as it runs either a
+single iteration (when using `Benchmark.bm`) or two iterations (when using
+`Benchmark.bmbm`). Running this few iterations means external factors (e.g. a
+video streaming in the background) can very easily skew the benchmark
+statistics.
+
+Another problem with the `Benchmark` module is that it displays timings, not
+iterations. This means that if a piece of code completes in a very short period
+of time it can be very difficult to compare the timings before and after a
+certain change. This in turn leads to patterns such as the following:
+
+```ruby
+Benchmark.bmbm(10) do |bench|
+ bench.report 'do something' do
+ 100.times do
+ ... work here ...
+ end
+ end
+end
+```
+
+This however leads to the question: how many iterations should we run to get
+meaningful statistics?
+
+The benchmark-ips Gem basically takes care of all this and much more, and as a
+result of this should be used instead of the `Benchmark` module.
+
+In short:
+
+1. Don't trust benchmarks you find on the internet.
+2. Never make claims based on just benchmarks, always measure in production to
+ confirm your findings.
+3. X being N times faster than Y is meaningless if you don't know what impact it
+ will actually have on your production environment.
+4. A production environment is the _only_ benchmark that always tells the truth
+ (unless your performance monitoring systems are not set up correctly).
+5. If you must write a benchmark use the benchmark-ips Gem instead of Ruby's
+ `Benchmark` module.
+
+## Importance of Changes
+
+When working on performance improvements, it's important to always ask yourself
+the question "How important is it to improve the performance of this piece of
+code?". Not every piece of code is equally important and it would be a waste to
+spend a week trying to improve something that only impacts a tiny fraction of
+our users. For example, spending a week trying to squeeze 10 milliseconds out of
+a method is a waste of time when you could have spent a week squeezing out 10
+seconds elsewhere.
+
+There is no clear set of steps that you can follow to determine if a certain
+piece of code is worth optimizing. The only two things you can do are:
+
+1. Think about what the code does, how it's used, how many times it's called and
+ how much time is spent in it relative to the total execution time (e.g., the
+ total time spent in a web request).
+2. Ask others (preferably in the form of an issue).
+
+Some examples of changes that aren't really important/worth the effort:
+
+* Replacing double quotes with single quotes.
+* Replacing usage of Array with Set when the list of values is very small.
+* Replacing library A with library B when both only take up 0.1% of the total
+ execution time.
+* Calling `freeze` on every string (see [String Freezing](#string-freezing)).
+
+## Slow Operations & Sidekiq
+
+Slow operations (e.g. merging branches) or operations that are prone to errors
+(using external APIs) should be performed in a Sidekiq worker instead of
+directly in a web request as much as possible. This has numerous benefits such
+as:
+
+1. An error won't prevent the request from completing.
+2. The process being slow won't affect the loading time of a page.
+3. In case of a failure it's easy to re-try the process (Sidekiq takes care of
+ this automatically).
+4. By isolating the code from a web request it will hopefully be easier to test
+ and maintain.
+
+It's especially important to use Sidekiq as much as possible when dealing with
+Git operations as these operations can take quite some time to complete
+depending on the performance of the underlying storage system.
+
+## Git Operations
+
+Care should be taken to not run unnecessary Git operations. For example,
+retrieving the list of branch names using `Repository#branch_names` can be done
+without an explicit check if a repository exists or not. In other words, instead
+of this:
+
+```ruby
+if repository.exists?
+ repository.branch_names.each do |name|
+ ...
+ end
+end
+```
+
+You can just write:
+
+```ruby
+repository.branch_names.each do |name|
+ ...
+end
+```
+
+## Caching
+
+Operations that will often return the same result should be cached using Redis,
+in particular Git operations. When caching data in Redis, make sure the cache is
+flushed whenever needed. For example, a cache for the list of tags should be
+flushed whenever a new tag is pushed or a tag is removed.
+
+When adding cache expiration code for repositories, this code should be placed
+in one of the before/after hooks residing in the Repository class. For example,
+if a cache should be flushed after importing a repository this code should be
+added to `Repository#after_import`. This ensures the cache logic stays within
+the Repository class instead of leaking into other classes.
+
+When caching data, make sure to also memoize the result in an instance variable.
+While retrieving data from Redis is much faster than raw Git operations, it still
+has overhead. By caching the result in an instance variable, repeated calls to
+the same method won't end up retrieving data from Redis upon every call. When
+memoizing cached data in an instance variable, make sure to also reset the
+instance variable when flushing the cache. An example:
+
+
+```ruby
+def first_branch
+ @first_branch ||= cache.fetch(:first_branch) { branches.first }
+end
+
+def expire_first_branch_cache
+ cache.expire(:first_branch)
+ @first_branch = nil
+end
+```
+
+## Anti-Patterns
+
+This is a collection of [anti-patterns][anti-pattern] that should be avoided
+unless these changes have a measurable, significant and positive impact on
+production environments.
+
+### String Freezing
+
+In recent Ruby versions calling `freeze` on a String leads to it being allocated
+only once and re-used. For example, on Ruby 2.3 this will only allocate the
+"foo" String once:
+
+```ruby
+10.times do
+ 'foo'.freeze
+end
+```
+
+Blindly adding a `.freeze` call to every String is an anti-pattern that should
+be avoided unless one can prove (using production data) the call actually has a
+positive impact on performance.
+
+This feature of Ruby wasn't really meant to make things faster directly, instead
+it was meant to reduce the number of allocations. Depending on the size of the
+String and how frequently it would be allocated (before the `.freeze` call was
+added), this _may_ make things faster, but there's no guarantee it will.
+
+Another common flavour of this is to not only freeze a String, but also assign
+it to a constant, for example:
+
+```ruby
+SOME_CONSTANT = 'foo'.freeze
+
+9000.times do
+ SOME_CONSTANT
+end
+```
+
+The only reason you should be doing this is to prevent somebody from mutating
+the global String. However, since you can just re-assign constants in Ruby
+there's nothing stopping somebody from doing this elsewhere in the code:
+
+```ruby
+SOME_CONSTANT = 'bar'
+```
+
+### Moving Allocations to Constants
+
+Storing an object as a constant so you only allocate it once _may_ improve
+performance, but there's no guarantee this will. Looking up constants has an
+impact on runtime performance, and as such, using a constant instead of
+referencing an object directly may even slow code down.
+
+[#15607]: https://gitlab.com/gitlab-org/gitlab-ce/issues/15607
+[yorickpeterse]: https://gitlab.com/u/yorickpeterse
+[joshfng]: https://gitlab.com/u/joshfng
+[anti-pattern]: https://en.wikipedia.org/wiki/Anti-pattern
diff --git a/doc/development/rake_tasks.md b/doc/development/rake_tasks.md
index 9f3fd69fc4e..6d04b9590e6 100644
--- a/doc/development/rake_tasks.md
+++ b/doc/development/rake_tasks.md
@@ -9,7 +9,7 @@ bundle exec rake setup
```
The `setup` task is a alias for `gitlab:setup`.
-This tasks calls `db:setup` to create the database, calls `add_limits_mysql` that adds limits to the database schema in case of a MySQL database and finally it calls `db:seed_fu` to seed the database.
+This tasks calls `db:reset` to create the database, calls `add_limits_mysql` that adds limits to the database schema in case of a MySQL database and finally it calls `db:seed_fu` to seed the database.
Note: `db:setup` calls `db:seed` but this does nothing.
## Run tests
diff --git a/doc/development/scss_styleguide.md b/doc/development/scss_styleguide.md
index 6c48c25448b..a79f4073cde 100644
--- a/doc/development/scss_styleguide.md
+++ b/doc/development/scss_styleguide.md
@@ -72,9 +72,9 @@ p { margin: 0; padding: 0; }
### Colors
-HEX (hexadecimal) colors short-form should use shortform where possible, and
-should use lower case letters to differenciate between letters and numbers, e.
-g. `#E3E3E3` vs. `#e3e3e3`.
+HEX (hexadecimal) colors should use shorthand where possible, and should use
+lower case letters to differentiate between letters and numbers, e.g. `#E3E3E3`
+vs. `#e3e3e3`.
```scss
// Bad
@@ -160,6 +160,7 @@ is slightly more performant.
```
### Selectors with a `js-` Prefix
+
Do not use any selector prefixed with `js-` for styling purposes. These
selectors are intended for use only with JavaScript to allow for removal or
renaming without breaking styling.
@@ -187,8 +188,28 @@ CSSComb globally (system-wide). Run it in the GitLab directory with
Note that this won't fix every problem, but it should fix a majority.
+### Ignoring issues
+
+If you want a line or set of lines to be ignored by the linter, you can use
+`// scss-lint:disable RuleName` ([more info][disabling-linters]):
+
+```scss
+// This lint rule is disabled because the class name comes from a gem.
+// scss-lint:disable SelectorFormat
+.ui_charcoal {
+ background-color: #333;
+}
+// scss-lint:enable SelectorFormat
+```
+
+Make sure a comment is added on the line above the `disable` rule, otherwise the
+linter will throw a warning. `DisableLinterReason` is enabled to make sure the
+style guide isn't being ignored, and to communicate to others why the style
+guide is ignored in this instance.
+
[csscomb]: https://github.com/csscomb/csscomb.js
[node]: https://github.com/nodejs/node
[npm]: https://www.npmjs.com/
[scss-lint]: https://github.com/brigade/scss-lint
[scss-lint-documentation]: https://github.com/brigade/scss-lint/blob/master/lib/scss_lint/linter/README.md
+[disabling-linters]: https://github.com/brigade/scss-lint#disabling-linters-via-source
diff --git a/doc/development/testing.md b/doc/development/testing.md
new file mode 100644
index 00000000000..513457d203a
--- /dev/null
+++ b/doc/development/testing.md
@@ -0,0 +1,137 @@
+# Testing Standards and Style Guidelines
+
+This guide outlines standards and best practices for automated testing of GitLab
+CE and EE.
+
+It is meant to be an _extension_ of the [thoughtbot testing
+styleguide](https://github.com/thoughtbot/guides/tree/master/style/testing). If
+this guide defines a rule that contradicts the thoughtbot guide, this guide
+takes precedence. Some guidelines may be repeated verbatim to stress their
+importance.
+
+## Factories
+
+GitLab uses [factory_girl] as a test fixture replacement.
+
+- Factory definitions live in `spec/factories/`, named using the pluralization
+ of their corresponding model (`User` factories are defined in `users.rb`).
+- There should be only one top-level factory definition per file.
+- FactoryGirl methods are mixed in to all RSpec groups. This means you can (and
+ should) call `create(...)` instead of `FactoryGirl.create(...)`.
+- Make use of [traits] to clean up definitions and usages.
+- When defining a factory, don't define attributes that are not required for the
+ resulting record to pass validation.
+- When instantiating from a factory, don't supply attributes that aren't
+ required by the test.
+- Factories don't have to be limited to `ActiveRecord` objects.
+ [See example](https://gitlab.com/gitlab-org/gitlab-ce/commit/0b8cefd3b2385a21cfed779bd659978c0402766d).
+
+[factory_girl]: https://github.com/thoughtbot/factory_girl
+[traits]: http://www.rubydoc.info/gems/factory_girl/file/GETTING_STARTED.md#Traits
+
+## JavaScript
+
+GitLab uses [Teaspoon] to run its [Jasmine] JavaScript specs. They can be run on
+the command line via `bundle exec teaspoon`, or via a web browser at
+`http://localhost:3000/teaspoon` when the Rails server is running.
+
+- JavaScript tests live in `spec/javascripts/`, matching the folder structure of
+ `app/assets/javascripts/`: `app/assets/javascripts/behaviors/autosize.js.coffee` has a corresponding
+ `spec/javascripts/behaviors/autosize_spec.js.coffee` file.
+- Haml fixtures required for JavaScript tests live in
+ `spec/javascripts/fixtures`. They should contain the bare minimum amount of
+ markup necessary for the test.
+
+ > **Warning:** Keep in mind that a Rails view may change and
+ invalidate your test, but everything will still pass because your fixture
+ doesn't reflect the latest view.
+
+- Keep in mind that in a CI environment, these tests are run in a headless
+ browser and you will not have access to certain APIs, such as
+ [`Notification`](https://developer.mozilla.org/en-US/docs/Web/API/notification),
+ which will have to be stubbed.
+
+[Teaspoon]: https://github.com/modeset/teaspoon
+[Jasmine]: https://github.com/jasmine/jasmine
+
+## RSpec
+
+### General Guidelines
+
+- Use a single, top-level `describe ClassName` block.
+- Use `described_class` instead of repeating the class name being described.
+- Use `.method` to describe class methods and `#method` to describe instance
+ methods.
+- Use `context` to test branching logic.
+- Don't `describe` symbols (see [Gotchas](gotchas.md#dont-describe-symbols)).
+- Don't supply the `:each` argument to hooks since it's the default.
+- Prefer `not_to` to `to_not` (_this is enforced by Rubocop_).
+- Try to match the ordering of tests to the ordering within the class.
+- Try to follow the [Four-Phase Test][four-phase-test] pattern, using newlines
+ to separate phases.
+
+[four-phase-test]: https://robots.thoughtbot.com/four-phase-test
+
+### `let` variables
+
+GitLab's RSpec suite has made extensive use of `let` variables to reduce
+duplication. However, this sometimes [comes at the cost of clarity][lets-not],
+so we need to set some guidelines for their use going forward:
+
+- `let` variables are preferable to instance variables. Local variables are
+ preferable to `let` variables.
+- Use `let` to reduce duplication throughout an entire spec file.
+- Don't use `let` to define variables used by a single test; define them as
+ local variables inside the test's `it` block.
+- Don't define a `let` variable inside the top-level `describe` block that's
+ only used in a more deeply-nested `context` or `describe` block. Keep the
+ definition as close as possible to where it's used.
+- Try to avoid overriding the definition of one `let` variable with another.
+- Don't define a `let` variable that's only used by the definition of another.
+ Use a helper method instead.
+
+[lets-not]: https://robots.thoughtbot.com/lets-not
+
+### Test speed
+
+GitLab has a massive test suite that, without parallelization, can take more
+than an hour to run. It's important that we make an effort to write tests that
+are accurate and effective _as well as_ fast.
+
+Here are some things to keep in mind regarding test performance:
+
+- `double` and `spy` are faster than `FactoryGirl.build(...)`
+- `FactoryGirl.build(...)` and `.build_stubbed` are faster than `.create`.
+- Don't `create` an object when `build`, `build_stubbed`, `attributes_for`,
+ `spy`, or `double` will do. Database persistence is slow!
+- Use `create(:empty_project)` instead of `create(:project)` when you don't need
+ the underlying Git repository. Filesystem operations are slow!
+- Don't mark a feature as requiring JavaScript (through `@javascript` in
+ Spinach or `js: true` in RSpec) unless it's _actually_ required for the test
+ to be valid. Headless browser testing is slow!
+
+### Features / Integration
+
+- Feature specs live in `spec/features/` and should be named
+ `ROLE_ACTION_spec.rb`, such as `user_changes_password_spec.rb`.
+- Use only one `feature` block per feature spec file.
+- Use scenario titles that describe the success and failure paths.
+- Avoid scenario titles that add no information, such as "successfully."
+- Avoid scenario titles that repeat the feature title.
+
+## Spinach (feature) tests
+
+GitLab [moved from Cucumber to Spinach](https://github.com/gitlabhq/gitlabhq/pull/1426)
+for its feature/integration tests in September 2012.
+
+As of March 2016, we are [trying to avoid adding new Spinach
+tests](https://gitlab.com/gitlab-org/gitlab-ce/issues/14121) going forward,
+opting for [RSpec feature](#features-integration) specs.
+
+Adding new Spinach scenarios is acceptable _only if_ the new scenario requires
+no more than one new `step` definition. If more than that is required, the
+test should be re-implemented using RSpec instead.
+
+---
+
+[Return to Development documentation](README.md)
diff --git a/doc/development/ui_guide.md b/doc/development/ui_guide.md
index 2f01defc11d..5893b7c219e 100644
--- a/doc/development/ui_guide.md
+++ b/doc/development/ui_guide.md
@@ -1,12 +1,56 @@
# UI Guide for building GitLab
-## Best practices for creating new pages in GitLab
-
-TODO: write some best practices when develop GitLab features.
-
## GitLab UI development kit
We created a page inside GitLab where you can check commonly used html and css elements.
When you run GitLab instance locally - just visit http://localhost:3000/help/ui page to see UI examples
you can use during GitLab development.
+
+## Design repository
+
+All design files are stored in the [gitlab-design](https://gitlab.com/gitlab-org/gitlab-design)
+repository and maintained by GitLab UX designers.
+
+## Navigation
+
+GitLab's layout contains 2 sections: the left sidebar and the content. The left sidebar contains a static navigation menu.
+This menu will be visible regardless of what page you visit. The left sidebar also contains the GitLab logo
+and the current user's profile picture. The content section contains a header and the content itself.
+The header describes the current GitLab page and what navigation is
+available to user in this area. Depending on the area (project, group, profile setting) the header name and navigation may change. For example when user visits one of the
+project pages the header will contain a project name and navigation for that project. When the user visits a group page it will contain a group name and navigation related to this group.
+
+### Adding new tab to header navigation
+
+We try to keep the amount of tabs in the header navigation between 5 and 10 so that it fits on a typical laptop screen. We also try not to confuse the user with too many options. Ideally each
+tab should represent separate functionality. Everything related to the issue
+tracker should be under the 'Issues' tab while everything related to the wiki should
+be under 'Wiki' tab and so on and so forth.
+
+## Mobile screen size
+
+We want GitLab to work well on small mobile screens as well. Size limitations make it is impossible to fit everything on a mobile screen. In this case it is OK to hide
+part of the UI for smaller resolutions in favor of a better user experience.
+However core functionality like browsing files, creating issues, writing comments, should
+be available on all resolutions.
+
+## Icons
+
+* `trash` icon for button or link that does destructive action like removing
+information from database or file system
+* `x` icon for closing/hiding UI element. For example close modal window
+* `pencil` icon for edit button or link
+* `eye` icon for subscribe action
+* `rss` for rss/atom feed
+* `plus` for link or dropdown that lead to page where you create new object (For example new issue page)
+
+
+## Buttons
+
+* Button should contain icon or text. Exceptions should be approved by UX designer.
+* Use red button for destructive actions (not revertable). For example removing issue.
+* Use green or blue button for primary action. Primary button should be only one.
+Do not use both green and blue button in one form.
+* For all other cases use default white button
+
diff --git a/doc/downgrade_ee_to_ce/README.md b/doc/downgrade_ee_to_ce/README.md
new file mode 100644
index 00000000000..3625c4191b8
--- /dev/null
+++ b/doc/downgrade_ee_to_ce/README.md
@@ -0,0 +1,82 @@
+# Downgrading from EE to CE
+
+If you ever decide to downgrade your Enterprise Edition back to the Community
+Edition, there are a few steps you need take before installing the CE package
+on top of the current EE package, or, if you are in an installation from source,
+before you change remotes and fetch the latest CE code.
+
+## Disable Enterprise-only features
+
+First thing to do is to disable the following features.
+
+### Authentication mechanisms
+
+Kerberos and Atlassian Crowd are only available on the Enterprise Edition, so
+you should disable these mechanisms before downgrading and you should provide
+alternative authentication methods to your users.
+
+### Git Annex
+
+Git Annex is also only available on the Enterprise Edition. This means that if
+you have repositories that use Git Annex to store large files, these files will
+no longer be easily available via Git. You should consider migrating these
+repositories to use Git LFS before downgrading to the Community Edition.
+
+### Remove Jenkins CI Service entries from the database
+
+The `JenkinsService` class is only available on the Enterprise Edition codebase,
+so if you downgrade to the Community Edition, you'll come across the following
+error:
+
+```
+Completed 500 Internal Server Error in 497ms (ActiveRecord: 32.2ms)
+
+ActionView::Template::Error (The single-table inheritance mechanism failed to locate the subclass: 'JenkinsService'. This
+error is raised because the column 'type' is reserved for storing the class in case of inheritance. Please rename this
+column if you didn't intend it to be used for storing the inheritance class or overwrite Service.inheritance_column to
+use another column for that information.)
+```
+
+All services are created automatically for every project you have, so in order
+to avoid getting this error, you need to remove all instances of the
+`JenkinsService` from your database:
+
+**Omnibus Installation**
+
+```
+$ sudo gitlab-rails runner "Service.where(type: 'JenkinsService').delete_all"
+```
+
+**Source Installation**
+
+```
+$ bundle exec rails runner "Service.where(type: 'JenkinsService').delete_all" production
+```
+
+## Downgrade to CE
+
+After performing the above mentioned steps, you are now ready to downgrade your
+GitLab installation to the Community Edition.
+
+**Omnibus Installation**
+
+To downgrade an Omnibus installation, it is sufficient to install the Community
+Edition package on top of the currently installed one. You can do this manually,
+by directly [downloading the package](https://packages.gitlab.com/gitlab/gitlab-ce)
+you need, or by adding our CE package repository and following the
+[CE installation instructions](https://about.gitlab.com/downloads/).
+
+**Source Installation**
+
+To downgrade a source installation, you need to replace the current remote of
+your GitLab installation with the Community Edition's remote, fetch the latest
+changes, and checkout the latest stable branch:
+
+```
+$ git remote set-url origin git@gitlab.com:gitlab-org/gitlab-ce.git
+$ git fetch --all
+$ git checkout 8-x-stable
+```
+
+Remember to follow the correct [update guides](../update/README.md) to make
+sure all dependencies are up to date.
diff --git a/doc/gitlab-basics/README.md b/doc/gitlab-basics/README.md
index 493e1d1b09c..3aa83975ace 100644
--- a/doc/gitlab-basics/README.md
+++ b/doc/gitlab-basics/README.md
@@ -2,26 +2,14 @@
Step-by-step guides on the basics of working with Git and GitLab.
-* [Start using Git on the command line](start-using-git.md)
-
-* [Create and add your SSH Keys](create-your-ssh-keys.md)
-
-* [Command Line basic commands](command-line-commands.md)
-
-* [Basic Git commands](basic-git-commands.md)
-
-* [Create a project](create-project.md)
-
-* [Create a group](create-group.md)
-
-* [Create a branch](create-branch.md)
-
-* [Fork a project](fork-project.md)
-
-* [Add a file](add-file.md)
-
-* [Add an image](add-image.md)
-
-* [Create a Merge Request](add-merge-request.md)
-
-* [Create an Issue](create-issue.md)
+- [Start using Git on the command line](start-using-git.md)
+- [Create and add your SSH Keys](create-your-ssh-keys.md)
+- [Command Line basics](command-line-commands.md)
+- [Create a project](create-project.md)
+- [Create a group](create-group.md)
+- [Create a branch](create-branch.md)
+- [Fork a project](fork-project.md)
+- [Add a file](add-file.md)
+- [Add an image](add-image.md)
+- [Create a Merge Request](add-merge-request.md)
+- [Create an Issue](create-issue.md)
diff --git a/doc/gitlab-basics/basic-git-commands.md b/doc/gitlab-basics/basic-git-commands.md
index 2b5767dd2d3..c2a3415cbc4 100644
--- a/doc/gitlab-basics/basic-git-commands.md
+++ b/doc/gitlab-basics/basic-git-commands.md
@@ -1,59 +1,3 @@
# Basic Git commands
-### Go to the master branch to pull the latest changes from there
-```
-git checkout master
-```
-
-### Download the latest changes in the project
-This is for you to work on an up-to-date copy (it is important to do every time you work on a project), while you setup tracking branches.
-```
-git pull REMOTE NAME-OF-BRANCH -u
-```
-(REMOTE: origin) (NAME-OF-BRANCH: could be "master" or an existing branch)
-
-### Create a branch
-Spaces won't be recognized, so you need to use a hyphen or underscore.
-```
-git checkout -b NAME-OF-BRANCH
-```
-
-### Work on a branch that has already been created
-```
-git checkout NAME-OF-BRANCH
-```
-
-### View the changes you've made
-It's important to be aware of what's happening and what's the status of your changes.
-```
-git status
-```
-
-### Add changes to commit
-You'll see your changes in red when you type "git status".
-```
-git add CHANGES IN RED
-git commit -m "DESCRIBE THE INTENTION OF THE COMMIT"
-```
-
-### Send changes to gitlab.com
-```
-git push REMOTE NAME-OF-BRANCH
-```
-
-### Delete all changes in the Git repository, but leave unstaged things
-```
-git checkout .
-```
-
-### Delete all changes in the Git repository, including untracked files
-```
-git clean -f
-```
-
-### Merge created branch with master branch
-You need to be in the created branch.
-```
-git checkout NAME-OF-BRANCH
-git merge master
-```
+This section is now merged into [Start using Git](start-using-git.md).
diff --git a/doc/gitlab-basics/create-issue.md b/doc/gitlab-basics/create-issue.md
index 87f078def04..5221d85b661 100644
--- a/doc/gitlab-basics/create-issue.md
+++ b/doc/gitlab-basics/create-issue.md
@@ -24,4 +24,4 @@ You may assign the Issue to a user, add a milestone and add labels (they are all
![Submit new issue](basicsimages/submit_new_issue.png)
-Your Issue will now be added to the Issue Tracker and will be ready to be reviewed. You can comment on it and mention the people involved. You can also link Issues to the Merge Requests where the Issues are solved. To do this, you can use an [Issue closing pattern](http://doc.gitlab.com/ce/customization/issue_closing.html).
+Your Issue will now be added to the Issue Tracker and will be ready to be reviewed. You can comment on it and mention the people involved. You can also link Issues to the Merge Requests where the Issues are solved. To do this, you can use an [Issue closing pattern](http://docs.gitlab.com/ce/customization/issue_closing.html).
diff --git a/doc/gitlab-basics/create-project.md b/doc/gitlab-basics/create-project.md
index b545d62549d..f737dffc024 100644
--- a/doc/gitlab-basics/create-project.md
+++ b/doc/gitlab-basics/create-project.md
@@ -14,7 +14,7 @@ Fill out the required information:
1. Select a [visibility level](https://gitlab.com/help/public_access/public_access)
-1. You can also [import your existing projects](http://doc.gitlab.com/ce/workflow/importing/README.html)
+1. You can also [import your existing projects](http://docs.gitlab.com/ce/workflow/importing/README.html)
1. Click on "create project"
diff --git a/doc/gitlab-basics/start-using-git.md b/doc/gitlab-basics/start-using-git.md
index b2ceda025c0..89ce8bcc3e8 100644
--- a/doc/gitlab-basics/start-using-git.md
+++ b/doc/gitlab-basics/start-using-git.md
@@ -1,6 +1,7 @@
# Start using Git on the command line
-If you want to start using a Git and GitLab, make sure that you have created an account on GitLab.
+If you want to start using a Git and GitLab, make sure that you have created an
+account on GitLab.
## Open a shell
@@ -59,3 +60,63 @@ To view the information that you entered, type:
```
git config --global --list
```
+## Basic Git commands
+
+### Go to the master branch to pull the latest changes from there
+
+```
+git checkout master
+```
+
+### Download the latest changes in the project
+This is for you to work on an up-to-date copy (it is important to do every time you work on a project), while you setup tracking branches.
+```
+git pull REMOTE NAME-OF-BRANCH -u
+```
+(REMOTE: origin) (NAME-OF-BRANCH: could be "master" or an existing branch)
+
+### Create a branch
+Spaces won't be recognized, so you need to use a hyphen or underscore.
+```
+git checkout -b NAME-OF-BRANCH
+```
+
+### Work on a branch that has already been created
+```
+git checkout NAME-OF-BRANCH
+```
+
+### View the changes you've made
+It's important to be aware of what's happening and what's the status of your changes.
+```
+git status
+```
+
+### Add changes to commit
+You'll see your changes in red when you type "git status".
+```
+git add CHANGES IN RED
+git commit -m "DESCRIBE THE INTENTION OF THE COMMIT"
+```
+
+### Send changes to gitlab.com
+```
+git push REMOTE NAME-OF-BRANCH
+```
+
+### Delete all changes in the Git repository, but leave unstaged things
+```
+git checkout .
+```
+
+### Delete all changes in the Git repository, including untracked files
+```
+git clean -f
+```
+
+### Merge created branch with master branch
+You need to be in the created branch.
+```
+git checkout NAME-OF-BRANCH
+git merge master
+```
diff --git a/doc/hooks/custom_hooks.md b/doc/hooks/custom_hooks.md
index dcdf49d3379..820934f97f1 100644
--- a/doc/hooks/custom_hooks.md
+++ b/doc/hooks/custom_hooks.md
@@ -2,7 +2,7 @@
**Note: Custom git hooks must be configured on the filesystem of the GitLab
server. Only GitLab server administrators will be able to complete these tasks.
-Please explore [webhooks](../web_hooks/web_hooks.md) as an option if you do not have filesystem access. For a user configurable Git Hooks interface, please see [GitLab Enterprise Edition Git Hooks](http://doc.gitlab.com/ee/git_hooks/git_hooks.html).**
+Please explore [webhooks](../web_hooks/web_hooks.md) as an option if you do not have filesystem access. For a user configurable Git Hooks interface, please see [GitLab Enterprise Edition Git Hooks](http://docs.gitlab.com/ee/git_hooks/git_hooks.html).**
Git natively supports hooks that are executed on different actions.
Examples of server-side git hooks include pre-receive, post-receive, and update.
diff --git a/doc/incoming_email/README.md b/doc/incoming_email/README.md
index 4cfb8402943..5a9a1582877 100644
--- a/doc/incoming_email/README.md
+++ b/doc/incoming_email/README.md
@@ -1,36 +1,99 @@
# Reply by email
-GitLab can be set up to allow users to comment on issues and merge requests by replying to notification emails.
+GitLab can be set up to allow users to comment on issues and merge requests by
+replying to notification emails.
-## Get a mailbox
+## Requirement
-Reply by email requires an IMAP-enabled email account, with a provider or server that supports [email sub-addressing](https://en.wikipedia.org/wiki/Email_address#Sub-addressing). Sub-addressing is a feature where any email to `user+some_arbitrary_tag@example.com` will end up in the mailbox for `user@example.com`, and is supported by providers such as Gmail, Google Apps, Yahoo! Mail, Outlook.com and iCloud, as well as the Postfix mail server which you can run on-premises.
+Reply by email requires an IMAP-enabled email account. GitLab allows you to use
+three strategies for this feature:
+- using email sub-addressing
+- using a dedicated email address
+- using a catch-all mailbox
-If you want to use Gmail / Google Apps with Reply by email, make sure you have [IMAP access enabled](https://support.google.com/mail/troubleshooter/1668960?hl=en#ts=1665018) and [allow less secure apps to access the account](https://support.google.com/accounts/answer/6010255).
+### Email sub-addressing
-To set up a basic Postfix mail server with IMAP access on Ubuntu, follow [these instructions](./postfix.md).
+**If your provider or server supports email sub-addressing, we recommend using it.**
+
+[Sub-addressing](https://en.wikipedia.org/wiki/Email_address#Sub-addressing) is
+a feature where any email to `user+some_arbitrary_tag@example.com` will end up
+in the mailbox for `user@example.com`, and is supported by providers such as
+Gmail, Google Apps, Yahoo! Mail, Outlook.com and iCloud, as well as the Postfix
+mail server which you can run on-premises.
+
+### Dedicated email address
+
+This solution is really simple to set up: you just have to create an email
+address dedicated to receive your users' replies to GitLab notifications.
+
+### Catch-all mailbox
+
+A [catch-all mailbox](https://en.wikipedia.org/wiki/Catch-all) for a domain will
+"catch all" the emails addressed to the domain that do not exist in the mail
+server.
+
+## How it works?
+
+### 1. GitLab sends a notification email
+
+When GitLab sends a notification and Reply by email is enabled, the `Reply-To`
+header is set to the address defined in your GitLab configuration, with the
+`%{key}` placeholder (if present) replaced by a specific "reply key". In
+addition, this "reply key" is also added to the `References` header.
+
+### 2. You reply to the notification email
+
+When you reply to the notification email, your email client will:
+
+- send the email to the `Reply-To` address it got from the notification email
+- set the `In-Reply-To` header to the value of the `Message-ID` header from the
+ notification email
+- set the `References` header to the value of the `Message-ID` plus the value of
+ the notification email's `References` header.
+
+### 3. GitLab receives your reply to the notification email
+
+When GitLab receives your reply, it will look for the "reply key" in the
+following headers, in this order:
+
+1. the `To` header
+1. the `References` header
+
+If it finds a reply key, it will be able to leave your reply as a comment on
+the entity the notification was about (issue, merge request, commit...).
+
+For more details about the `Message-ID`, `In-Reply-To`, and `References headers`,
+please consult [RFC 5322](https://tools.ietf.org/html/rfc5322#section-3.6.4).
## Set it up
+If you want to use Gmail / Google Apps with Reply by email, make sure you have
+[IMAP access enabled](https://support.google.com/mail/troubleshooter/1668960?hl=en#ts=1665018)
+and [allowed less secure apps to access the account](https://support.google.com/accounts/answer/6010255).
+
+To set up a basic Postfix mail server with IMAP access on Ubuntu, follow
+[these instructions](./postfix.md).
+
### Omnibus package installations
-1. Find the `incoming_email` section in `/etc/gitlab/gitlab.rb`, enable the feature and fill in the details for your specific IMAP server and email account:
+1. Find the `incoming_email` section in `/etc/gitlab/gitlab.rb`, enable the
+ feature and fill in the details for your specific IMAP server and email account:
```ruby
# Configuration for Postfix mail server, assumes mailbox incoming@gitlab.example.com
gitlab_rails['incoming_email_enabled'] = true
-
- # The email address including a placeholder for the key that references the item being replied to.
- # The `%{key}` placeholder is added after the user part, before the `@`.
+
+ # The email address including the `%{key}` placeholder that will be replaced to reference the item being replied to.
+ # The placeholder can be omitted but if present, it must appear in the "user" part of the address (before the `@`).
gitlab_rails['incoming_email_address'] = "incoming+%{key}@gitlab.example.com"
-
+
# Email account username
# With third party providers, this is usually the full email address.
# With self-hosted email servers, this is usually the user part of the email address.
gitlab_rails['incoming_email_email'] = "incoming"
# Email account password
gitlab_rails['incoming_email_password'] = "[REDACTED]"
-
+
# IMAP server host
gitlab_rails['incoming_email_host'] = "gitlab.example.com"
# IMAP server port
@@ -47,18 +110,18 @@ To set up a basic Postfix mail server with IMAP access on Ubuntu, follow [these
```ruby
# Configuration for Gmail / Google Apps, assumes mailbox gitlab-incoming@gmail.com
gitlab_rails['incoming_email_enabled'] = true
-
+
# The email address including the `%{key}` placeholder that will be replaced to reference the item being replied to.
- # The `%{key}` placeholder is added after the user part, after a `+` character, before the `@`.
+ # The placeholder can be omitted but if present, it must appear in the "user" part of the address (before the `@`).
gitlab_rails['incoming_email_address'] = "gitlab-incoming+%{key}@gmail.com"
-
+
# Email account username
# With third party providers, this is usually the full email address.
# With self-hosted email servers, this is usually the user part of the email address.
gitlab_rails['incoming_email_email'] = "gitlab-incoming@gmail.com"
# Email account password
gitlab_rails['incoming_email_password'] = "[REDACTED]"
-
+
# IMAP server host
gitlab_rails['incoming_email_host'] = "imap.gmail.com"
# IMAP server port
@@ -72,8 +135,6 @@ To set up a basic Postfix mail server with IMAP access on Ubuntu, follow [these
gitlab_rails['incoming_email_mailbox_name'] = "inbox"
```
- As mentioned, the part after `+` in the address is ignored, and any email sent here will end up in the mailbox for `incoming@gitlab.example.com`/`gitlab-incoming@gmail.com`.
-
1. Reconfigure GitLab and restart mailroom for the changes to take effect:
```sh
@@ -97,7 +158,8 @@ To set up a basic Postfix mail server with IMAP access on Ubuntu, follow [these
cd /home/git/gitlab
```
-1. Find the `incoming_email` section in `config/gitlab.yml`, enable the feature and fill in the details for your specific IMAP server and email account:
+1. Find the `incoming_email` section in `config/gitlab.yml`, enable the feature
+ and fill in the details for your specific IMAP server and email account:
```sh
sudo editor config/gitlab.yml
@@ -109,7 +171,7 @@ To set up a basic Postfix mail server with IMAP access on Ubuntu, follow [these
enabled: true
# The email address including the `%{key}` placeholder that will be replaced to reference the item being replied to.
- # The `%{key}` placeholder is added after the user part, after a `+` character, before the `@`.
+ # The placeholder can be omitted but if present, it must appear in the "user" part of the address (before the `@`).
address: "incoming+%{key}@gitlab.example.com"
# Email account username
@@ -138,7 +200,7 @@ To set up a basic Postfix mail server with IMAP access on Ubuntu, follow [these
enabled: true
# The email address including the `%{key}` placeholder that will be replaced to reference the item being replied to.
- # The `%{key}` placeholder is added after the user part, after a `+` character, before the `@`.
+ # The placeholder can be omitted but if present, it must appear in the "user" part of the address (before the `@`).
address: "gitlab-incoming+%{key}@gmail.com"
# Email account username
@@ -161,8 +223,6 @@ To set up a basic Postfix mail server with IMAP access on Ubuntu, follow [these
mailbox: "inbox"
```
- As mentioned, the part after `+` in the address is ignored, and any email sent here will end up in the mailbox for `incoming@gitlab.example.com`/`gitlab-incoming@gmail.com`.
-
1. Enable `mail_room` in the init script at `/etc/default/gitlab`:
```sh
@@ -195,8 +255,8 @@ To set up a basic Postfix mail server with IMAP access on Ubuntu, follow [these
incoming_email:
enabled: true
- # The email address including a placeholder for the key that references the item being replied to.
- # The `%{key}` placeholder is added after the user part, before the `@`.
+ # The email address including the `%{key}` placeholder that will be replaced to reference the item being replied to.
+ # The placeholder can be omitted but if present, it must appear in the "user" part of the address (before the `@`).
address: "gitlab-incoming+%{key}@gmail.com"
# Email account username
diff --git a/doc/install/installation.md b/doc/install/installation.md
index c567846f624..d9290b1fa76 100644
--- a/doc/install/installation.md
+++ b/doc/install/installation.md
@@ -6,7 +6,7 @@ Since an installation from source is a lot of work and error prone we strongly r
One reason the Omnibus package is more reliable is its use of Runit to restart any of the GitLab processes in case one crashes.
On heavily used GitLab instances the memory usage of the Sidekiq background worker will grow over time.
-Omnibus packages solve this by [letting the Sidekiq terminate gracefully](http://doc.gitlab.com/ce/operations/sidekiq_memory_killer.html) if it uses too much memory.
+Omnibus packages solve this by [letting the Sidekiq terminate gracefully](http://docs.gitlab.com/ce/operations/sidekiq_memory_killer.html) if it uses too much memory.
After this termination Runit will detect Sidekiq is not running and will start it.
Since installations from source don't have Runit, Sidekiq can't be terminated and its memory usage will grow over time.
@@ -157,22 +157,64 @@ Create a `git` user for GitLab:
## 5. Database
-We recommend using a PostgreSQL database. For MySQL check [MySQL setup guide](database_mysql.md). *Note*: because we need to make use of extensions you need at least pgsql 9.1.
+We recommend using a PostgreSQL database. For MySQL check the
+[MySQL setup guide](database_mysql.md).
- # Install the database packages
- sudo apt-get install -y postgresql postgresql-client libpq-dev
+> **Note**: because we need to make use of extensions you need at least pgsql 9.1.
- # Create a user for GitLab
+1. Install the database packages:
+
+ ```bash
+ sudo apt-get install -y postgresql postgresql-client libpq-dev postgresql-contrib
+ ```
+
+1. Create a database user for GitLab:
+
+ ```bash
sudo -u postgres psql -d template1 -c "CREATE USER git CREATEDB;"
+ ```
+
+1. Create the GitLab production database and grant all privileges on database:
- # Create the GitLab production database & grant all privileges on database
+ ```bash
sudo -u postgres psql -d template1 -c "CREATE DATABASE gitlabhq_production OWNER git;"
+ ```
- # Try connecting to the new database with the new user
+1. Create the `pg_trgm` extension (required for GitLab 8.6+):
+
+ ```bash
+ sudo -u postgres psql -d template1 -c "CREATE EXTENSION IF NOT EXISTS pg_trgm;"
+ ```
+
+1. Try connecting to the new database with the new user:
+
+ ```bash
sudo -u git -H psql -d gitlabhq_production
+ ```
+
+1. Check if the `pg_trgm` extension is enabled:
- # Quit the database session
+ ```bash
+ SELECT true AS enabled
+ FROM pg_available_extensions
+ WHERE name = 'pg_trgm'
+ AND installed_version IS NOT NULL;
+ ```
+
+ If the extension is enabled this will produce the following output:
+
+ ```
+ enabled
+ ---------
+ t
+ (1 row)
+ ```
+
+1. Quit the database session:
+
+ ```bash
gitlabhq_production> \q
+ ```
## 6. Redis
@@ -227,9 +269,9 @@ sudo usermod -aG redis git
### Clone the Source
# Clone GitLab repository
- sudo -u git -H git clone https://gitlab.com/gitlab-org/gitlab-ce.git -b 8-6-stable gitlab
+ sudo -u git -H git clone https://gitlab.com/gitlab-org/gitlab-ce.git -b 8-9-stable gitlab
-**Note:** You can change `8-6-stable` to `master` if you want the *bleeding edge* version, but never install master on a production server!
+**Note:** You can change `8-9-stable` to `master` if you want the *bleeding edge* version, but never install master on a production server!
### Configure It
@@ -283,9 +325,13 @@ sudo usermod -aG redis git
# Copy the example Rack attack config
sudo -u git -H cp config/initializers/rack_attack.rb.example config/initializers/rack_attack.rb
- # Configure Git global settings for git user, used when editing via web editor
+ # Configure Git global settings for git user
+ # 'autocrlf' is needed for the web editor
sudo -u git -H git config --global core.autocrlf input
+ # Disable 'git gc --auto' because GitLab already runs 'git gc' when needed
+ sudo -u git -H git config --global gc.auto 0
+
# Configure Redis connection settings
sudo -u git -H cp config/resque.yml.example config/resque.yml
@@ -348,7 +394,7 @@ GitLab Shell is an SSH access and repository management software developed speci
cd /home/git
sudo -u git -H git clone https://gitlab.com/gitlab-org/gitlab-workhorse.git
cd gitlab-workhorse
- sudo -u git -H git checkout 0.6.5
+ sudo -u git -H git checkout v0.7.5
sudo -u git -H make
### Initialize Database and Activate Advanced Features
@@ -526,6 +572,16 @@ See the [omniauth integration document](../integration/omniauth.md)
GitLab can build your projects. To enable that feature you need GitLab Runners to do that for you.
Checkout the [GitLab Runner section](https://about.gitlab.com/gitlab-ci/#gitlab-runner) to install it
+### Adding your Trusted Proxies
+
+If you are using a reverse proxy on an separate machine, you may want to add the
+proxy to the trusted proxies list. Otherwise users will appear signed in from the
+proxy's IP address.
+
+You can add trusted proxies in `config/gitlab.yml` by customizing the `trusted_proxies`
+option in section 1. Save the file and [reconfigure GitLab](../administration/restart_gitlab.md)
+for the changes to take effect.
+
### Custom Redis Connection
If you'd like Resque to connect to a Redis server on a non-standard port or on a different host, you can configure its connection string via the `config/resque.yml` file.
diff --git a/doc/install/relative_url.md b/doc/install/relative_url.md
index 0245febfcd8..44d2a14f366 100644
--- a/doc/install/relative_url.md
+++ b/doc/install/relative_url.md
@@ -132,5 +132,5 @@ To disable the relative URL:
1. Follow the same as above starting from 2. and set up the
GitLab URL to one that doesn't contain a relative path.
-[omnibus-rel]: http://doc.gitlab.com/omnibus/settings/configuration.html#configuring-a-relative-url-for-gitlab "How to setup relative URL in Omnibus GitLab"
+[omnibus-rel]: http://docs.gitlab.com/omnibus/settings/configuration.html#configuring-a-relative-url-for-gitlab "How to setup relative URL in Omnibus GitLab"
[restart gitlab]: ../administration/restart_gitlab.md#installations-from-source "How to restart GitLab"
diff --git a/doc/install/requirements.md b/doc/install/requirements.md
index 03cb08dd1f1..09c6211b3ab 100644
--- a/doc/install/requirements.md
+++ b/doc/install/requirements.md
@@ -64,7 +64,10 @@ If you have enough RAM memory and a recent CPU the speed of GitLab is mainly lim
### Memory
You need at least 2GB of addressable memory (RAM + swap) to install and use GitLab!
-With less memory GitLab will give strange errors during the reconfigure run and 500 errors during usage.
+The operating system and any other running applications will also be using memory
+so keep in mind that you need at least 2GB available before running GitLab. With
+less memory GitLab will give strange errors during the reconfigure run and 500
+errors during usage.
- 512MB RAM + 1.5GB of swap is the absolute minimum but we strongly **advise against** this amount of memory. See the unicorn worker section below for more advice.
- 1GB RAM + 1GB swap supports up to 100 users but it will be very slow
@@ -77,8 +80,32 @@ With less memory GitLab will give strange errors during the reconfigure run and
- 128GB RAM supports up to 32,000 users
- More users? Run it on [multiple application servers](https://about.gitlab.com/high-availability/)
+We recommend having at least 1GB of swap on your server, even if you currently have
+enough available RAM. Having swap will help reduce the chance of errors occurring
+if your available memory changes.
+
Notice: The 25 workers of Sidekiq will show up as separate processes in your process overview (such as top or htop) but they share the same RAM allocation since Sidekiq is a multithreaded application. Please see the section below about Unicorn workers for information about many you need of those.
+## Gitlab Runner
+
+We strongly advise against installing GitLab Runner on the same machine you plan
+to install GitLab on. Depending on how you decide to configure GitLab Runner and
+what tools you use to exercise your application in the CI environment, GitLab
+Runner can consume significant amount of available memory.
+
+Memory consumption calculations, that are available above, will not be valid if
+you decide to run GitLab Runner and the GitLab Rails application on the same
+machine.
+
+It is also not safe to install everything on a single machine, because of the
+[security reasons] - especially when you plan to use shell executor with GitLab
+Runner.
+
+We recommend using a separate machine for each GitLab Runner, if you plan to
+use the CI features.
+
+[security reasons]: https://gitlab.com/gitlab-org/gitlab-ci-multi-runner/blob/master/docs/security/index.md
+
## Unicorn Workers
It's possible to increase the amount of unicorn workers and this will usually help for to reduce the response time of the applications and increase the ability to handle parallel requests.
@@ -122,4 +149,5 @@ On a very active server (10,000 active users) the Sidekiq process can use 1GB+ o
- Firefox (Latest released version and [latest ESR version](https://www.mozilla.org/en-US/firefox/organizations/))
- Safari 7+ (known problem: required fields in html5 do not work)
- Opera (Latest released version)
-- Internet Explorer (IE) 10+ but please make sure that you have the `Compatibility View` mode disabled.
+- Internet Explorer (IE) 11+ but please make sure that you have the `Compatibility View` mode disabled.
+- Edge (Latest stable version)
diff --git a/doc/integration/README.md b/doc/integration/README.md
index 7c8f785a61f..fd330dd7a7d 100644
--- a/doc/integration/README.md
+++ b/doc/integration/README.md
@@ -19,26 +19,15 @@ See the documentation below for details on how to configure these services.
GitLab Enterprise Edition contains [advanced Jenkins support][jenkins].
+[jenkins]: http://docs.gitlab.com/ee/integration/jenkins.html
+
+
## Project services
Integration with services such as Campfire, Flowdock, Gemnasium, HipChat,
Pivotal Tracker, and Slack are available in the form of a [Project Service][].
-You can find these within GitLab in the Services page under Project Settings if
-you are at least a master on the project.
-Project Services are a bit like plugins in that they allow a lot of freedom in
-adding functionality to GitLab. For example there is also a service that can
-send an email every time someone pushes new commits.
-
-Because GitLab is open source we can ship with the code and tests for all
-plugins. This allows the community to keep the plugins up to date so that they
-always work in newer GitLab versions.
-
-For an overview of what projects services are available without logging in,
-please see the [project_services directory][projects-code].
-[jenkins]: http://doc.gitlab.com/ee/integration/jenkins.html
[Project Service]: ../project_services/project_services.md
-[projects-code]: https://gitlab.com/gitlab-org/gitlab-ce/tree/master/app/models/project_services
## SSL certificate errors
diff --git a/doc/integration/cas.md b/doc/integration/cas.md
index e6b2071f193..e34e306f9ac 100644
--- a/doc/integration/cas.md
+++ b/doc/integration/cas.md
@@ -27,17 +27,18 @@ To enable the CAS OmniAuth provider you must register your application with your
```ruby
gitlab_rails['omniauth_providers'] = [
{
- name: "cas3",
- label: "cas",
- args: {
- url: 'CAS_SERVER',
- login_url: '/CAS_PATH/login',
- service_validate_url: '/CAS_PATH/p3/serviceValidate',
- logout_url: '/CAS_PATH/logout'} }
- }
+ "name"=> "cas3",
+ "label"=> "cas",
+ "args"=> {
+ "url"=> 'CAS_SERVER',
+ "login_url"=> '/CAS_PATH/login',
+ "service_validate_url"=> '/CAS_PATH/p3/serviceValidate',
+ "logout_url"=> '/CAS_PATH/logout'
+ }
}
]
```
+
For installations from source:
@@ -57,6 +58,8 @@ To enable the CAS OmniAuth provider you must register your application with your
1. Save the configuration file.
+1. Run `gitlab-ctl reconfigure` for the omnibus package.
+
1. Restart GitLab for the changes to take effect.
On the sign in page there should now be a CAS tab in the sign in form.
diff --git a/doc/integration/github.md b/doc/integration/github.md
index 886784a27c9..e7497e475c9 100644
--- a/doc/integration/github.md
+++ b/doc/integration/github.md
@@ -9,7 +9,9 @@ GitHub will generate an application ID and secret key for you to use.
1. Navigate to your individual user settings or an organization's settings, depending on how you want the application registered. It does not matter if the application is registered as an individual or an organization - that is entirely up to you.
-1. Select "Applications" in the left menu.
+1. Select "OAuth applications" in the left menu.
+
+1. If you already have applications listed, switch to the "Developer applications" tab.
1. Select "Register new application".
@@ -17,7 +19,7 @@ GitHub will generate an application ID and secret key for you to use.
- Application name: This can be anything. Consider something like "\<Organization\>'s GitLab" or "\<Your Name\>'s GitLab" or something else descriptive.
- Homepage URL: The URL to your GitLab installation. 'https://gitlab.company.com'
- Application description: Fill this in if you wish.
- - Authorization callback URL: 'https://gitlab.company.com/'
+ - Default authorization callback URL is '${YOUR_DOMAIN}/import/github/callback'
1. Select "Register application".
1. You should now see a Client ID and Client Secret near the top right of the page (see screenshot).
@@ -60,12 +62,26 @@ GitHub will generate an application ID and secret key for you to use.
For installation from source:
+ For GitHub.com:
+
```
- { name: 'github', app_id: 'YOUR_APP_ID',
app_secret: 'YOUR_APP_SECRET',
args: { scope: 'user:email' } }
```
+
+ For GitHub Enterprise:
+
+ ```
+ - { name: 'github', app_id: 'YOUR_APP_ID',
+ app_secret: 'YOUR_APP_SECRET',
+ url: "https://github.example.com/",
+ args: { scope: 'user:email' } }
+ ```
+
+ __Replace `https://github.example.com/` with your GitHub URL.__
+
1. Change 'YOUR_APP_ID' to the client ID from the GitHub application page from step 7.
1. Change 'YOUR_APP_SECRET' to the client secret from the GitHub application page from step 7.
diff --git a/doc/integration/google.md b/doc/integration/google.md
index f9a20dd840d..82978b68a34 100644
--- a/doc/integration/google.md
+++ b/doc/integration/google.md
@@ -11,9 +11,9 @@ To enable the Google OAuth2 OmniAuth provider you must register your application
- Project ID: Must be unique to all Google Developer registered applications. Google provides a randomly generated Project ID by default. You can use the randomly generated ID or choose a new one.
1. Refresh the page. You should now see your new project in the list. Click on the project.
-1. Select "APIs & auth" in the left menu.
+1. Select the "Google APIs" tab in the Overview.
-1. Select "APIs" in the submenu.
+1. Select and enable the following Google APIs - listed under "Popular APIs"
- Enable `Contacts API`
- Enable `Google+ API`
diff --git a/doc/integration/img/enabled-oauth-sign-in-sources.png b/doc/integration/img/enabled-oauth-sign-in-sources.png
new file mode 100644
index 00000000000..95f8bbdcd24
--- /dev/null
+++ b/doc/integration/img/enabled-oauth-sign-in-sources.png
Binary files differ
diff --git a/doc/integration/ldap.md b/doc/integration/ldap.md
index cf1f98492ea..30f0c15dacc 100644
--- a/doc/integration/ldap.md
+++ b/doc/integration/ldap.md
@@ -1,228 +1,3 @@
# GitLab LDAP integration
-GitLab can be configured to allow your users to sign with their LDAP credentials to integrate with e.g. Active Directory.
-
-The first time a user signs in with LDAP credentials, GitLab will create a new GitLab user associated with the LDAP Distinguished Name (DN) of the LDAP user.
-
-GitLab user attributes such as nickname and email will be copied from the LDAP user entry.
-
-## Security
-
-GitLab assumes that LDAP users are not able to change their LDAP 'mail', 'email' or 'userPrincipalName' attribute.
-An LDAP user who is allowed to change their email on the LDAP server can [take over any account](#enabling-ldap-sign-in-for-existing-gitlab-users) on your GitLab server.
-
-We recommend against using GitLab LDAP integration if your LDAP users are allowed to change their 'mail', 'email' or 'userPrincipalName' attribute on the LDAP server.
-
-If a user is deleted from the LDAP server, they will be blocked in GitLab as well.
-Users will be immediately blocked from logging in. However, there is an LDAP check
-cache time of one hour. The means users that are already logged in or are using Git
-over SSH will still be able to access GitLab for up to one hour. Manually block
-the user in the GitLab Admin area to immediately block all access.
-
-## Configuring GitLab for LDAP integration
-
-To enable GitLab LDAP integration you need to add your LDAP server settings in `/etc/gitlab/gitlab.rb` or `/home/git/gitlab/config/gitlab.yml`.
-In GitLab Enterprise Edition you can have multiple LDAP servers connected to one GitLab server.
-
-Please note that before version 7.4, GitLab used a different syntax for configuring LDAP integration.
-The old LDAP integration syntax still works in GitLab 7.4.
-If your `gitlab.rb` or `gitlab.yml` file contains LDAP settings in both the old syntax and the new syntax, only the __old__ syntax will be used by GitLab.
-
-```ruby
-# For omnibus packages
-gitlab_rails['ldap_enabled'] = true
-gitlab_rails['ldap_servers'] = YAML.load <<-EOS # remember to close this block with 'EOS' below
-main: # 'main' is the GitLab 'provider ID' of this LDAP server
- ## label
- #
- # A human-friendly name for your LDAP server. It is OK to change the label later,
- # for instance if you find out it is too large to fit on the web page.
- #
- # Example: 'Paris' or 'Acme, Ltd.'
- label: 'LDAP'
-
- host: '_your_ldap_server'
- port: 389
- uid: 'sAMAccountName'
- method: 'plain' # "tls" or "ssl" or "plain"
- bind_dn: '_the_full_dn_of_the_user_you_will_bind_with'
- password: '_the_password_of_the_bind_user'
-
- # Set a timeout, in seconds, for LDAP queries. This helps avoid blocking
- # a request if the LDAP server becomes unresponsive.
- # A value of 0 means there is no timeout.
- timeout: 10
-
- # This setting specifies if LDAP server is Active Directory LDAP server.
- # For non AD servers it skips the AD specific queries.
- # If your LDAP server is not AD, set this to false.
- active_directory: true
-
- # If allow_username_or_email_login is enabled, GitLab will ignore everything
- # after the first '@' in the LDAP username submitted by the user on login.
- #
- # Example:
- # - the user enters 'jane.doe@example.com' and 'p@ssw0rd' as LDAP credentials;
- # - GitLab queries the LDAP server with 'jane.doe' and 'p@ssw0rd'.
- #
- # If you are using "uid: 'userPrincipalName'" on ActiveDirectory you need to
- # disable this setting, because the userPrincipalName contains an '@'.
- allow_username_or_email_login: false
-
- # To maintain tight control over the number of active users on your GitLab installation,
- # enable this setting to keep new users blocked until they have been cleared by the admin
- # (default: false).
- block_auto_created_users: false
-
- # Base where we can search for users
- #
- # Ex. ou=People,dc=gitlab,dc=example
- #
- base: ''
-
- # Filter LDAP users
- #
- # Format: RFC 4515 https://tools.ietf.org/search/rfc4515
- # Ex. (employeeType=developer)
- #
- # Note: GitLab does not support omniauth-ldap's custom filter syntax.
- #
- user_filter: ''
-
- # LDAP attributes that GitLab will use to create an account for the LDAP user.
- # The specified attribute can either be the attribute name as a string (e.g. 'mail'),
- # or an array of attribute names to try in order (e.g. ['mail', 'email']).
- # Note that the user's LDAP login will always be the attribute specified as `uid` above.
- attributes:
- # The username will be used in paths for the user's own projects
- # (like `gitlab.example.com/username/project`) and when mentioning
- # them in issues, merge request and comments (like `@username`).
- # If the attribute specified for `username` contains an email address,
- # the GitLab username will be the part of the email address before the '@'.
- username: ['uid', 'userid', 'sAMAccountName']
- email: ['mail', 'email', 'userPrincipalName']
-
- # If no full name could be found at the attribute specified for `name`,
- # the full name is determined using the attributes specified for
- # `first_name` and `last_name`.
- name: 'cn'
- first_name: 'givenName'
- last_name: 'sn'
-
-# GitLab EE only: add more LDAP servers
-# Choose an ID made of a-z and 0-9 . This ID will be stored in the database
-# so that GitLab can remember which LDAP server a user belongs to.
-# uswest2:
-# label:
-# host:
-# ....
-EOS
-```
-
-If you are getting 'Connection Refused' errors when trying to connect to the LDAP server please double-check the LDAP `port` and `method` settings used by GitLab.
-Common combinations are `method: 'plain'` and `port: 389`, OR `method: 'ssl'` and `port: 636`.
-
-If you are using a GitLab installation from source you can find the LDAP settings in `/home/git/gitlab/config/gitlab.yml`:
-
-```
-production:
- # snip...
- ldap:
- enabled: false
- servers:
- main: # 'main' is the GitLab 'provider ID' of this LDAP server
- ## label
- #
- # A human-friendly name for your LDAP server. It is OK to change the label later,
- # for instance if you find out it is too large to fit on the web page.
- #
- # Example: 'Paris' or 'Acme, Ltd.'
- label: 'LDAP'
- # snip...
-```
-
-## Enabling LDAP sign-in for existing GitLab users
-
-When a user signs in to GitLab with LDAP for the first time, and their LDAP email address is the primary email address of an existing GitLab user, then the LDAP DN will be associated with the existing user.
-
-If the LDAP email attribute is not found in GitLab's database, a new user is created.
-
-In other words, if an existing GitLab user wants to enable LDAP sign-in for themselves, they should check that their GitLab email address matches their LDAP email address, and then sign into GitLab via their LDAP credentials.
-
-GitLab recognizes the following LDAP attributes as email addresses: `mail`, `email` and `userPrincipalName`.
-
-If multiple LDAP email attributes are present, e.g. `mail: foo@bar.com` and `email: foo@example.com`, then the first attribute found wins -- in this case `foo@bar.com`.
-
-## Using an LDAP filter to limit access to your GitLab server
-
-If you want to limit all GitLab access to a subset of the LDAP users on your LDAP server you can set up an LDAP user filter.
-The filter must comply with [RFC 4515](https://tools.ietf.org/search/rfc4515).
-
-```ruby
-# For omnibus packages; new LDAP server syntax
-gitlab_rails['ldap_servers'] = YAML.load <<-EOS
-main:
- # snip...
- user_filter: '(employeeType=developer)'
-EOS
-```
-
-```yaml
-# For installations from source; new LDAP server syntax
-production:
- ldap:
- servers:
- main:
- # snip...
- user_filter: '(employeeType=developer)'
-```
-
-Tip: if you want to limit access to the nested members of an Active Directory group you can use the following syntax:
-
-```
-(memberOf:1.2.840.113556.1.4.1941:=CN=My Group,DC=Example,DC=com)
-```
-
-Please note that GitLab does not support the custom filter syntax used by omniauth-ldap.
-
-## Limitations
-
-GitLab's LDAP client is based on [omniauth-ldap](https://gitlab.com/gitlab-org/omniauth-ldap)
-which encapsulates Ruby's `Net::LDAP` class. It provides a pure-Ruby implementation
-of the LDAP client protocol. As a result, GitLab is limited by `omniauth-ldap` and may impact your LDAP
-server settings.
-
-### TLS Client Authentication
-Not implemented by `Net::LDAP`.
-So you should disable anonymous LDAP authentication and enable simple or SASL
-authentication. TLS client authentication setting in your LDAP server cannot be
-mandatory and clients cannot be authenticated with the TLS protocol.
-
-### TLS Server Authentication
-Not supported by GitLab's configuration options.
-When setting `method: ssl`, the underlying authentication method used by
-`omniauth-ldap` is `simple_tls`. This method establishes TLS encryption with
-the LDAP server before any LDAP-protocol data is exchanged but no validation of
-the LDAP server's SSL certificate is performed.
-
-## Troubleshooting
-
-### Invalid credentials when logging in
-
-Make sure the user you are binding with has enough permissions to read the user's
-tree and traverse it.
-
-Also make sure that the `user_filter` is not blocking otherwise valid users.
-
-To make sure that the LDAP settings are correct and GitLab can see your users,
-execute the following command:
-
-
-```bash
-# For Omnibus installations
-sudo gitlab-rake gitlab:ldap:check
-
-# For installations from source
-sudo -u git -H bundle exec rake gitlab:ldap:check RAILS_ENV=production
-```
-
+This document was moved under [`administration/auth/ldap`](../administration/auth/ldap.md).
diff --git a/doc/integration/omniauth.md b/doc/integration/omniauth.md
index 25f35988305..820f40f81a9 100644
--- a/doc/integration/omniauth.md
+++ b/doc/integration/omniauth.md
@@ -11,6 +11,7 @@ of the configured mechanisms.
- [Supported Providers](#supported-providers)
- [Enable OmniAuth for an Existing User](#enable-omniauth-for-an-existing-user)
- [OmniAuth configuration sample when using Omnibus GitLab](https://gitlab.com/gitlab-org/omnibus-gitlab/tree/master#omniauth-google-twitter-github-login)
+- [Enable or disable Sign In with an OmniAuth provider without disabling import sources](#enable-or-disable-sign-in-with-an-omniauth-provider-without-disabling-import-sources)
## Supported Providers
@@ -120,6 +121,29 @@ OmniAuth provider for an existing user.
The chosen OmniAuth provider is now active and can be used to sign in to GitLab from then on.
+## Configure OmniAuth Providers as External
+
+>**Note:**
+This setting was introduced with version 8.7 of GitLab
+
+You can define which OmniAuth providers you want to be `external` so that all users
+creating accounts via these providers will not be able to have access to internal
+projects. You will need to use the full name of the provider, like `google_oauth2`
+for Google. Refer to the examples for the full names of the supported providers.
+
+**For Omnibus installations**
+
+```ruby
+ gitlab_rails['omniauth_external_providers'] = ['twitter', 'google_oauth2']
+```
+
+**For installations from source**
+
+```yaml
+ omniauth:
+ external_providers: ['twitter', 'google_oauth2']
+```
+
## Using Custom Omniauth Providers
>**Note:**
@@ -168,3 +192,17 @@ experience [in the public Wiki](https://github.com/gitlabhq/gitlab-public-wiki/w
While we can't officially support every possible authentication mechanism out there,
we'd like to at least help those with specific needs.
+
+## Enable or disable Sign In with an OmniAuth provider without disabling import sources
+
+>**Note:**
+This setting was introduced with version 8.8 of GitLab.
+
+Administrators are able to enable or disable Sign In via some OmniAuth providers.
+
+>**Note:**
+By default Sign In is enabled via all the OAuth Providers that have been configured in `config/gitlab.yml`.
+
+In order to enable/disable an OmniAuth provider, go to Admin Area -> Settings -> Sign-in Restrictions section -> Enabled OAuth Sign-In sources and select the providers you want to enable or disable.
+
+![Enabled OAuth Sign-In sources](img/enabled-oauth-sign-in-sources.png)
diff --git a/doc/integration/saml.md b/doc/integration/saml.md
index 1c3dc707f6d..8a7205caaa4 100644
--- a/doc/integration/saml.md
+++ b/doc/integration/saml.md
@@ -131,8 +131,75 @@ On the sign in page there should now be a SAML button below the regular sign in
Click the icon to begin the authentication process. If everything goes well the user
will be returned to GitLab and will be signed in.
+## External Groups
+
+>**Note:**
+This setting is only available on GitLab 8.7 and above.
+
+SAML login includes support for external groups. You can define in the SAML
+settings which groups, to which your users belong in your IdP, you wish to be
+marked as [external](../permissions/permissions.md).
+
+### Requirements
+
+First you need to tell GitLab where to look for group information. For this you
+need to make sure that your IdP server sends a specific `AttributeStament` along
+with the regular SAML response. Here is an example:
+
+```xml
+<saml:AttributeStatement>
+ <saml:Attribute Name="Groups">
+ <saml:AttributeValue xsi:type="xs:string">SecurityGroup</saml:AttributeValue>
+ <saml:AttributeValue xsi:type="xs:string">Developers</saml:AttributeValue>
+ <saml:AttributeValue xsi:type="xs:string">Designers</saml:AttributeValue>
+ </saml:Attribute>
+</saml:AttributeStatement>
+```
+
+The name of the attribute can be anything you like, but it must contain the groups
+to which a user belongs. In order to tell GitLab where to find these groups, you need
+to add a `groups_attribute:` element to your SAML settings. You will also need to
+tell GitLab which groups are external via the `external_groups:` element:
+
+```yaml
+{ name: 'saml',
+ label: 'Our SAML Provider',
+ groups_attribute: 'Groups',
+ external_groups: ['Freelancers', 'Interns'],
+ args: {
+ assertion_consumer_service_url: 'https://gitlab.example.com/users/auth/saml/callback',
+ idp_cert_fingerprint: '43:51:43:a1:b5:fc:8b:b7:0a:3a:a9:b1:0f:66:73:a8',
+ idp_sso_target_url: 'https://login.example.com/idp',
+ issuer: 'https://gitlab.example.com',
+ name_identifier_format: 'urn:oasis:names:tc:SAML:2.0:nameid-format:transient'
+ } }
+```
+
## Customization
+### `auto_sign_in_with_provider`
+
+You can add this setting to your GitLab configuration to automatically redirect you
+to your SAML server for authentication, thus removing the need to click a button
+before actually signing in.
+
+For omnibus package:
+
+```ruby
+gitlab_rails['omniauth_auto_sign_in_with_provider'] = 'saml'
+```
+
+For installations from source:
+
+```yaml
+omniauth:
+ auto_sign_in_with_provider: saml
+```
+
+Please keep in mind that every sign in attempt will be redirected to the SAML server,
+so you will not be able to sign in using local credentials. Make sure that at least one
+of the SAML users has admin permissions.
+
### `attribute_statements`
>**Note:**
@@ -205,6 +272,10 @@ To bypass this you can add `skip_before_action :verify_authenticity_token` to th
where it can then be seen in the usual logs, or as a flash message in the login
screen.
+That file is located at `/opt/gitlab/embedded/service/gitlab-rails/app/controllers`
+for Omnibus installations and by default on `/home/git/gitlab/app/controllers` for
+installations from source.
+
### Invalid audience
This error means that the IdP doesn't recognize GitLab as a valid sender and
diff --git a/doc/integration/shibboleth.md b/doc/integration/shibboleth.md
index a0be3dd4e5c..b6b2d4e5e88 100644
--- a/doc/integration/shibboleth.md
+++ b/doc/integration/shibboleth.md
@@ -76,3 +76,50 @@ sudo gitlab-ctl reconfigure
```
On the sign in page there should now be a "Sign in with: Shibboleth" icon below the regular sign in form. Click the icon to begin the authentication process. You will be redirected to IdP server (Depends on your Shibboleth module configuration). If everything goes well the user will be returned to GitLab and will be signed in.
+
+## Apache 2.4 / GitLab 8.6 update
+The order of the first 2 Location directives is important. If they are reversed,
+you will not get a shibboleth session!
+
+```
+ <Location />
+ Require all granted
+ ProxyPassReverse http://127.0.0.1:8181
+ ProxyPassReverse http://YOUR_SERVER_FQDN/
+ </Location>
+
+ <Location /users/auth/shibboleth/callback>
+ AuthType shibboleth
+ ShibRequestSetting requireSession 1
+ ShibUseHeaders On
+ Require shib-session
+ </Location>
+
+ Alias /shibboleth-sp /usr/share/shibboleth
+
+ <Location /shibboleth-sp>
+ Require all granted
+ </Location>
+
+ <Location /Shibboleth.sso>
+ SetHandler shib
+ </Location>
+
+ RewriteEngine on
+
+ #Don't escape encoded characters in api requests
+ RewriteCond %{REQUEST_URI} ^/api/v3/.*
+ RewriteCond %{REQUEST_URI} !/Shibboleth.sso
+ RewriteCond %{REQUEST_URI} !/shibboleth-sp
+ RewriteRule .* http://127.0.0.1:8181%{REQUEST_URI} [P,QSA,NE]
+
+ #Forward all requests to gitlab-workhorse except existing files
+ RewriteCond %{DOCUMENT_ROOT}/%{REQUEST_FILENAME} !-f [OR]
+ RewriteCond %{REQUEST_URI} ^/uploads/.*
+ RewriteCond %{REQUEST_URI} !/Shibboleth.sso
+ RewriteCond %{REQUEST_URI} !/shibboleth-sp
+ RewriteRule .* http://127.0.0.1:8181%{REQUEST_URI} [P,QSA]
+
+ RequestHeader set X_FORWARDED_PROTO 'https'
+ RequestHeader set X-Forwarded-Ssl on
+``` \ No newline at end of file
diff --git a/doc/intro/README.md b/doc/intro/README.md
new file mode 100644
index 00000000000..382d10aaf40
--- /dev/null
+++ b/doc/intro/README.md
@@ -0,0 +1,42 @@
+# Get started with GitLab
+
+## Organize
+
+Create projects and groups.
+
+- [Create a new project](../gitlab-basics/create-project.md)
+- [Create a new group](../gitlab-basics/create-group.md)
+
+## Prioritize
+
+Create issues, labels, milestones, cast your vote, and review issues.
+
+- [Create a new issue](../gitlab-basics/create-issue.md)
+- [Assign labels to issues](../workflow/labels.md)
+- [Use milestones as an overview of your project's tracker](../workflow/milestones.md)
+- [Use voting to express your like/dislike to issues and merge requests](../workflow/award_emoji.md)
+
+## Collaborate
+
+Create merge requests and review code.
+
+- [Fork a project and contribute to it](../workflow/forking_workflow.md)
+- [Create a new merge request](../gitlab-basics/add-merge-request.md)
+- [Automatically close issues from merge requests](../customization/issue_closing.md)
+- [Automatically merge when your builds succeed](../workflow/merge_when_build_succeeds.md)
+- [Revert any commit](../workflow/revert_changes.md)
+- [Cherry-pick any commit](../workflow/cherry_pick_changes.md)
+
+## Test and Deploy
+
+Use the built-in continuous integration in GitLab.
+
+- [Get started with GitLab CI](../ci/quick_start/README.md)
+
+## Install and Update
+
+Install and update your GitLab installation.
+
+- [Install GitLab](https://about.gitlab.com/installation/)
+- [Update GitLab](https://about.gitlab.com/update/)
+- [Explore Omnibus GitLab configuration options](http://docs.gitlab.com/omnibus/settings/configuration.html)
diff --git a/doc/logs/logs.md b/doc/logs/logs.md
index 27937e51764..a2eca62d691 100644
--- a/doc/logs/logs.md
+++ b/doc/logs/logs.md
@@ -1,92 +1 @@
-## Log system
-GitLab has advanced log system so everything is logging and you can analize your instance using various system log files.
-In addition to system log files, GitLab Enterprise Edition comes with Audit Events. Find more about them [in Audit Events documentation](http://doc.gitlab.com/ee/administration/audit_events.html)
-
-System log files are typically plain text in a standard log file format. This guide talks about how to read and use these system log files.
-
-#### production.log
-This file lives in `/var/log/gitlab/gitlab-rails/production.log` for omnibus package or in `/home/git/gitlab/log/production.log` for installations from the source.
-
-This file contains information about all performed requests. You can see url and type of request, IP address and what exactly parts of code were involved to service this particular request. Also you can see all SQL request that have been performed and how much time it took.
-This task is more useful for GitLab contributors and developers. Use part of this log file when you are going to report bug.
-
-```
-Started GET "/gitlabhq/yaml_db/tree/master" for 168.111.56.1 at 2015-02-12 19:34:53 +0200
-Processing by Projects::TreeController#show as HTML
- Parameters: {"project_id"=>"gitlabhq/yaml_db", "id"=>"master"}
-
- ... [CUT OUT]
-
- amespaces"."created_at" DESC, "namespaces"."id" DESC LIMIT 1 [["id", 26]]
- CACHE (0.0ms) SELECT "members".* FROM "members" WHERE "members"."source_type" = 'Project' AND "members"."type" IN ('ProjectMember') AND "members"."source_id" = $1 AND "members"."source_type" = $2 AND "members"."user_id" = 1 ORDER BY "members"."created_at" DESC, "members"."id" DESC LIMIT 1 [["source_id", 18], ["source_type", "Project"]]
- CACHE (0.0ms) SELECT "members".* FROM "members" WHERE "members"."source_type" = 'Project' AND "members".
-  (1.4ms) SELECT COUNT(*) FROM "merge_requests" WHERE "merge_requests"."target_project_id" = $1 AND ("merge_requests"."state" IN ('opened','reopened')) [["target_project_id", 18]]
- Rendered layouts/nav/_project.html.haml (28.0ms)
- Rendered layouts/_collapse_button.html.haml (0.2ms)
- Rendered layouts/_flash.html.haml (0.1ms)
- Rendered layouts/_page.html.haml (32.9ms)
-Completed 200 OK in 166ms (Views: 117.4ms | ActiveRecord: 27.2ms)
-```
-In this example we can see that server processed HTTP request with url `/gitlabhq/yaml_db/tree/master` from IP 168.111.56.1 at 2015-02-12 19:34:53 +0200. Also we can see that request was processed by Projects::TreeController.
-
-#### application.log
-This file lives in `/var/log/gitlab/gitlab-rails/application.log` for omnibus package or in `/home/git/gitlab/log/application.log` for installations from the source.
-
-This log file helps you discover events happening in your instance such as user creation, project removing and so on.
-
-```
-October 06, 2014 11:56: User "Administrator" (admin@example.com) was created
-October 06, 2014 11:56: Documentcloud created a new project "Documentcloud / Underscore"
-October 06, 2014 11:56: Gitlab Org created a new project "Gitlab Org / Gitlab Ce"
-October 07, 2014 11:25: User "Claudie Hodkiewicz" (nasir_stehr@olson.co.uk) was removed
-October 07, 2014 11:25: Project "project133" was removed
-```
-#### githost.log
-This file lives in `/var/log/gitlab/gitlab-rails/githost.log` for omnibus package or in `/home/git/gitlab/log/githost.log` for installations from the source.
-
-The GitLab has to interact with git repositories but in some rare cases something can go wrong and in this case you will know what exactly happened. This log file contains all failed requests from GitLab to git repository. In majority of cases this file will be useful for developers only.
-```
-December 03, 2014 13:20 -> ERROR -> Command failed [1]: /usr/bin/git --git-dir=/Users/vsizov/gitlab-development-kit/gitlab/tmp/tests/gitlab-satellites/group184/gitlabhq/.git --work-tree=/Users/vsizov/gitlab-development-kit/gitlab/tmp/tests/gitlab-satellites/group184/gitlabhq merge --no-ff -mMerge branch 'feature_conflict' into 'feature' source/feature_conflict
-
-error: failed to push some refs to '/Users/vsizov/gitlab-development-kit/repositories/gitlabhq/gitlab_git.git'
-```
-
-#### sidekiq.log
-This file lives in `/var/log/gitlab/gitlab-rails/sidekiq.log` for omnibus package or in `/home/git/gitlab/log/sidekiq.log` for installations from the source.
-
-GitLab uses background jobs for processing tasks which can take a long time. All information about processing these jobs are writing down to this file.
-```
-2014-06-10T07:55:20Z 2037 TID-tm504 ERROR: /opt/bitnami/apps/discourse/htdocs/vendor/bundle/ruby/1.9.1/gems/redis-3.0.7/lib/redis/client.rb:228:in `read'
-2014-06-10T18:18:26Z 14299 TID-55uqo INFO: Booting Sidekiq 3.0.0 with redis options {:url=>"redis://localhost:6379/0", :namespace=>"sidekiq"}
-```
-
-#### gitlab-shell.log
-This file lives in `/var/log/gitlab/gitlab-shell/gitlab-shell.log` for omnibus package or in `/home/git/gitlab-shell/gitlab-shell.log` for installations from the source.
-
-gitlab-shell is using by Gitlab for executing git commands and provide ssh access to git repositories.
-
-```
-I, [2015-02-13T06:17:00.671315 #9291] INFO -- : Adding project root/example.git at </var/opt/gitlab/git-data/repositories/root/dcdcdcdcd.git>.
-I, [2015-02-13T06:17:00.679433 #9291] INFO -- : Moving existing hooks directory and simlinking global hooks directory for /var/opt/gitlab/git-data/repositories/root/example.git.
-```
-
-#### unicorn_stderr.log
-This file lives in `/var/log/gitlab/unicorn/unicorn_stderr.log` for omnibus package or in `/home/git/gitlab/log/unicorn_stderr.log` for installations from the source.
-
-Unicorn is a high-performance forking Web server which is used for serving GitLab application. You can look at this log, for example, if your application does not respond. This log cantains all information about state of unicorn processes at any given time.
-
-```
-I, [2015-02-13T06:14:46.680381 #9047] INFO -- : Refreshing Gem list
-I, [2015-02-13T06:14:56.931002 #9047] INFO -- : listening on addr=127.0.0.1:8080 fd=12
-I, [2015-02-13T06:14:56.931381 #9047] INFO -- : listening on addr=/var/opt/gitlab/gitlab-rails/sockets/gitlab.socket fd=13
-I, [2015-02-13T06:14:56.936638 #9047] INFO -- : master process ready
-I, [2015-02-13T06:14:56.946504 #9092] INFO -- : worker=0 spawned pid=9092
-I, [2015-02-13T06:14:56.946943 #9092] INFO -- : worker=0 ready
-I, [2015-02-13T06:14:56.947892 #9094] INFO -- : worker=1 spawned pid=9094
-I, [2015-02-13T06:14:56.948181 #9094] INFO -- : worker=1 ready
-W, [2015-02-13T07:16:01.312916 #9094] WARN -- : #<Unicorn::HttpServer:0x0000000208f618>: worker (pid: 9094) exceeds memory limit (320626688 bytes > 247066940 bytes)
-W, [2015-02-13T07:16:01.313000 #9094] WARN -- : Unicorn::WorkerKiller send SIGQUIT (pid: 9094) alive: 3621 sec (trial 1)
-I, [2015-02-13T07:16:01.530733 #9047] INFO -- : reaped #<Process::Status: pid 9094 exit 0> worker=1
-I, [2015-02-13T07:16:01.534501 #13379] INFO -- : worker=1 spawned pid=13379
-I, [2015-02-13T07:16:01.534848 #13379] INFO -- : worker=1 ready
-```
+This document was moved to [administration/logs.md](../administration/logs.md).
diff --git a/doc/markdown/markdown.md b/doc/markdown/markdown.md
index e6eb1cf3819..236eb7b12c4 100644
--- a/doc/markdown/markdown.md
+++ b/doc/markdown/markdown.md
@@ -8,6 +8,7 @@
* [Multiple underscores in words](#multiple-underscores-in-words)
* [URL auto-linking](#url-auto-linking)
* [Code and Syntax Highlighting](#code-and-syntax-highlighting)
+* [Inline Diff](#inline-diff)
* [Emoji](#emoji)
* [Special GitLab references](#special-gitlab-references)
* [Task lists](#task-lists)
@@ -31,7 +32,7 @@
_GitLab uses the [Redcarpet Ruby library][redcarpet] for Markdown processing._
-For GitLab we developed something we call "GitLab Flavored Markdown" (GFM). It extends the standard Markdown in a few significant ways to add some useful functionality.
+GitLab uses "GitLab Flavored Markdown" (GFM). It extends the standard Markdown in a few significant ways to add some useful functionality. It was inspired by [GitHub Flavored Markdown](https://help.github.com/articles/basic-writing-and-formatting-syntax/).
You can use GFM in
@@ -47,10 +48,10 @@ You can also use other rich text files in GitLab. You might have to install a de
GFM honors the markdown specification in how [paragraphs and line breaks are handled](https://daringfireball.net/projects/markdown/syntax#p).
-A paragraph is simply one or more consecutive lines of text, separated by one or more blank lines.
+A paragraph is simply one or more consecutive lines of text, separated by one or more blank lines.
Line-breaks, or softreturns, are rendered if you end a line with two or more spaces
- Roses are red [followed by two or more spaces]
+ Roses are red [followed by two or more spaces]
Violets are blue
Sugar is sweet
@@ -67,7 +68,7 @@ It is not reasonable to italicize just _part_ of a word, especially when you're
perform_complicated_task
do_this_and_do_that_and_another_thing
-perform_complicated_task
+perform_complicated_task
do_this_and_do_that_and_another_thing
## URL auto-linking
@@ -153,6 +154,19 @@ s = "There is no highlighting for this."
But let's throw in a <b>tag</b>.
```
+## Inline Diff
+
+With inline diffs tags you can display {+ additions +} or [- deletions -].
+
+The wrapping tags can be either curly braces or square brackets [+ additions +] or {- deletions -}.
+
+However the wrapping tags cannot be mixed as such:
+
+- {+ additions +]
+- [+ additions +}
+- {- deletions -]
+- [- deletions -}
+
## Emoji
Sometimes you want to :monkey: around a bit and add some :star2: to your :speech_balloon:. Well we have a gift for you:
@@ -185,20 +199,23 @@ GFM will turn that reference into a link so you can navigate between them easily
GFM will recognize the following:
-| input | references |
-|:-----------------------|:---------------------------|
-| `@user_name` | specific user |
-| `@group_name` | specific group |
-| `@all` | entire team |
-| `#123` | issue |
-| `!123` | merge request |
-| `$123` | snippet |
-| `~123` | label by ID |
-| `~bug` | one-word label by name |
-| `~"feature request"` | multi-word label by name |
-| `9ba12248` | specific commit |
-| `9ba12248...b19a04f5` | commit range comparison |
-| `[README](doc/README)` | repository file references |
+| input | references |
+|:-----------------------|:--------------------------- |
+| `@user_name` | specific user |
+| `@group_name` | specific group |
+| `@all` | entire team |
+| `#123` | issue |
+| `!123` | merge request |
+| `$123` | snippet |
+| `~123` | label by ID |
+| `~bug` | one-word label by name |
+| `~"feature request"` | multi-word label by name |
+| `%123` | milestone by ID |
+| `%v1.23` | one-word milestone by name |
+| `%"release candidate"` | multi-word milestone by name |
+| `9ba12248` | specific commit |
+| `9ba12248...b19a04f5` | commit range comparison |
+| `[README](doc/README)` | repository file references |
GFM also recognizes certain cross-project references:
@@ -206,6 +223,7 @@ GFM also recognizes certain cross-project references:
|:----------------------------------------|:------------------------|
| `namespace/project#123` | issue |
| `namespace/project!123` | merge request |
+| `namespace/project%123` | milestone |
| `namespace/project$123` | snippet |
| `namespace/project@9ba12248` | specific commit |
| `namespace/project@9ba12248...b19a04f5` | commit range comparison |
@@ -402,7 +420,7 @@ There are two ways to create links, inline-style and reference-style.
[I'm a reference-style link][Arbitrary case-insensitive reference text]
-[I'm a relative reference to a repository file](LICENSE)
+[I'm a relative reference to a repository file](LICENSE)[^1]
[You can use numbers for reference-style link definitions][1]
@@ -594,3 +612,4 @@ By including colons in the header row, you can align the text within that column
[rouge]: http://rouge.jneen.net/ "Rouge website"
[redcarpet]: https://github.com/vmg/redcarpet "Redcarpet website"
+[^1]: This link will be broken if you see this document from the Help page or docs.gitlab.com
diff --git a/doc/migrate_ci_to_ce/README.md b/doc/migrate_ci_to_ce/README.md
index 5ec0a2069b5..8f9ef054949 100644
--- a/doc/migrate_ci_to_ce/README.md
+++ b/doc/migrate_ci_to_ce/README.md
@@ -355,7 +355,7 @@ sudo chown git:git /var/opt/gitlab/gitlab-ci/builds
```
#### Problems when importing CI database to GitLab
-If you were migrating CI database from MySQL to PostgreSQL manually you can see errros during import about missing sequences:
+If you were migrating CI database from MySQL to PostgreSQL manually you can see errors during import about missing sequences:
```
ALTER SEQUENCE
ERROR: relation "ci_builds_id_seq" does not exist
diff --git a/doc/monitoring/health_check.md b/doc/monitoring/health_check.md
new file mode 100644
index 00000000000..0d17799372f
--- /dev/null
+++ b/doc/monitoring/health_check.md
@@ -0,0 +1,66 @@
+# Health Check
+
+>**Note:** This feature was [introduced][ce-3888] in GitLab 8.8.
+
+GitLab provides a health check endpoint for uptime monitoring on the `health_check` web
+endpoint. The health check reports on the overall system status based on the status of
+the database connection, the state of the database migrations, and the ability to write
+and access the cache. This endpoint can be provided to uptime monitoring services like
+[Pingdom][pingdom], [Nagios][nagios-health], and [NewRelic][newrelic-health].
+
+## Access Token
+
+An access token needs to be provided while accessing the health check endpoint. The current
+accepted token can be found on the `admin/health_check` page of your GitLab instance.
+
+![access token](img/health_check_token.png)
+
+The access token can be passed as a URL parameter:
+
+```
+https://gitlab.example.com/health_check.json?token=ACCESS_TOKEN
+```
+
+or as an HTTP header:
+
+```bash
+curl -H "TOKEN: ACCESS_TOKEN" https://gitlab.example.com/health_check.json
+```
+
+## Using the Endpoint
+
+Once you have the access token, health information can be retrieved as plain text, JSON,
+or XML using the `health_check` endpoint:
+
+- `https://gitlab.example.com/health_check?token=ACCESS_TOKEN`
+- `https://gitlab.example.com/health_check.json?token=ACCESS_TOKEN`
+- `https://gitlab.example.com/health_check.xml?token=ACCESS_TOKEN`
+
+You can also ask for the status of specific services:
+
+- `https://gitlab.example.com/health_check/cache.json?token=ACCESS_TOKEN`
+- `https://gitlab.example.com/health_check/database.json?token=ACCESS_TOKEN`
+- `https://gitlab.example.com/health_check/migrations.json?token=ACCESS_TOKEN`
+
+For example, the JSON output of the following health check:
+
+```bash
+curl -H "TOKEN: ACCESS_TOKEN" https://gitlab.example.com/health_check.json
+```
+
+would be like:
+
+```
+{"healthy":true,"message":"success"}
+```
+
+## Status
+
+On failure, the endpoint will return a `500` HTTP status code. On success, the endpoint
+will return a valid successful HTTP status code, and a `success` message. Ideally your
+uptime monitoring should look for the success message.
+
+[ce-3888]: https://gitlab.com/gitlab-org/gitlab-ce/merge_requests/3888
+[pingdom]: https://www.pingdom.com
+[nagios-health]: https://nagios-plugins.org/doc/man/check_http.html
+[newrelic-health]: https://docs.newrelic.com/docs/alerts/alert-policies/downtime-alerts/availability-monitoring
diff --git a/doc/monitoring/img/health_check_token.png b/doc/monitoring/img/health_check_token.png
new file mode 100644
index 00000000000..2daf8606b00
--- /dev/null
+++ b/doc/monitoring/img/health_check_token.png
Binary files differ
diff --git a/doc/monitoring/performance/gitlab_configuration.md b/doc/monitoring/performance/gitlab_configuration.md
index b856e7935a3..771584268d9 100644
--- a/doc/monitoring/performance/gitlab_configuration.md
+++ b/doc/monitoring/performance/gitlab_configuration.md
@@ -37,3 +37,4 @@ Read more on:
- [Introduction to GitLab Performance Monitoring](introduction.md)
- [InfluxDB Configuration](influxdb_configuration.md)
- [InfluxDB Schema](influxdb_schema.md)
+- [Grafana Install/Configuration](grafana_configuration.md)
diff --git a/doc/monitoring/performance/grafana_configuration.md b/doc/monitoring/performance/grafana_configuration.md
new file mode 100644
index 00000000000..168bd85c26a
--- /dev/null
+++ b/doc/monitoring/performance/grafana_configuration.md
@@ -0,0 +1,149 @@
+# Grafana Configuration
+
+[Grafana](http://grafana.org/) is a tool that allows you to visualize time
+series metrics through graphs and dashboards. It supports several backend
+data stores, including InfluxDB. GitLab writes performance data to InfluxDB
+and Grafana will allow you to query InfluxDB to display useful graphs.
+
+For the easiest installation and configuration, install Grafana on the same
+server as InfluxDB. For larger installations, you may want to split out these
+services.
+
+## Installation
+
+Grafana supplies package repositories (Yum/Apt) for easy installation.
+See [Grafana installation documentation](http://docs.grafana.org/installation/)
+for detailed steps.
+
+> **Note**: Before starting Grafana for the first time, set the admin user
+and password in `/etc/grafana/grafana.ini`. Otherwise, the default password
+will be `admin`.
+
+## Configuration
+
+Login as the admin user. Expand the menu by clicking the Grafana logo in the
+top left corner. Choose 'Data Sources' from the menu. Then, click 'Add new'
+in the top bar.
+
+![Grafana empty data source page](img/grafana_data_source_empty.png)
+
+Fill in the configuration details for the InfluxDB data source. Save and
+Test Connection to ensure the configuration is correct.
+
+- **Name**: InfluxDB
+- **Default**: Checked
+- **Type**: InfluxDB 0.9.x (Even if you're using InfluxDB 0.10.x)
+- **Url**: https://localhost:8086 (Or the remote URL if you've installed InfluxDB
+on a separate server)
+- **Access**: proxy
+- **Database**: gitlab
+- **User**: admin (Or the username configured when setting up InfluxDB)
+- **Password**: The password configured when you set up InfluxDB
+
+![Grafana data source configurations](img/grafana_data_source_configuration.png)
+
+## Apply retention policies and create continuous queries
+
+If you intend to import the GitLab provided Grafana dashboards, you will need
+to copy and run a set of queries against InfluxDB to create the needed data
+sets.
+
+On the InfluxDB server, run the following command, substituting your InfluxDB
+user and password:
+
+```bash
+influxdb --username admin -password super_secret
+```
+
+This will drop you in to an InfluxDB interactive session. Copy the entire
+contents below and paste it in to the interactive session:
+
+```
+CREATE RETENTION POLICY default ON gitlab DURATION 1h REPLICATION 1 DEFAULT
+CREATE RETENTION POLICY downsampled ON gitlab DURATION 7d REPLICATION 1
+CREATE CONTINUOUS QUERY grape_git_timings_per_action ON gitlab BEGIN SELECT mean("duration") AS duration_mean, percentile("duration", 95) AS duration_95th, percentile("duration", 99) AS duration_99th INTO gitlab.downsampled.grape_git_timings_per_action FROM gitlab."default".rails_method_calls WHERE (action !~ /.+/ OR action =~ /^Grape#/) AND method =~ /^(Rugged|Gitlab::Git)/ GROUP BY time(1m), action END;
+CREATE CONTINUOUS QUERY grape_markdown_render_timings_overall ON gitlab BEGIN SELECT mean(banzai_cached_render_real_time) AS cached_real_mean, percentile(banzai_cached_render_real_time, 95) AS cached_real_95th, percentile(banzai_cached_render_real_time, 99) AS cached_real_99th, mean(banzai_cached_render_cpu_time) AS cached_cpu_mean, percentile(banzai_cached_render_cpu_time, 95) AS cached_cpu_95th, percentile(banzai_cached_render_cpu_time, 99) AS cached_cpu_99th, sum(banzai_cached_render_call_count) AS cached_call_count, mean(banzai_cacheless_render_real_time) AS cacheless_real_mean, percentile(banzai_cacheless_render_real_time, 95) AS cacheless_real_95th, percentile(banzai_cacheless_render_real_time, 99) AS cacheless_real_99th, mean(banzai_cacheless_render_cpu_time) AS cacheless_cpu_mean, percentile(banzai_cacheless_render_cpu_time, 95) AS cacheless_cpu_95th, percentile(banzai_cacheless_render_cpu_time, 99) AS cacheless_cpu_99th, sum(banzai_cacheless_render_call_count) AS cacheless_call_count INTO gitlab.downsampled.grape_markdown_render_timings_overall FROM gitlab."default".rails_transactions WHERE (action !~ /.+/ OR action =~ /^Grape#/) AND (banzai_cached_render_call_count > 0 OR banzai_cacheless_render_call_count > 0) GROUP BY time(1m) END;
+CREATE CONTINUOUS QUERY grape_markdown_render_timings_per_action ON gitlab BEGIN SELECT mean(banzai_cached_render_real_time) AS cached_real_mean, percentile(banzai_cached_render_real_time, 95) AS cached_real_95th, percentile(banzai_cached_render_real_time, 99) AS cached_real_99th, mean(banzai_cached_render_cpu_time) AS cached_cpu_mean, percentile(banzai_cached_render_cpu_time, 95) AS cached_cpu_95th, percentile(banzai_cached_render_cpu_time, 99) AS cached_cpu_99th, sum(banzai_cached_render_call_count) AS cached_call_count, mean(banzai_cacheless_render_real_time) AS cacheless_real_mean, percentile(banzai_cacheless_render_real_time, 95) AS cacheless_real_95th, percentile(banzai_cacheless_render_real_time, 99) AS cacheless_real_99th, mean(banzai_cacheless_render_cpu_time) AS cacheless_cpu_mean, percentile(banzai_cacheless_render_cpu_time, 95) AS cacheless_cpu_95th, percentile(banzai_cacheless_render_cpu_time, 99) AS cacheless_cpu_99th, sum(banzai_cacheless_render_call_count) AS cacheless_call_count INTO gitlab.downsampled.grape_markdown_render_timings_per_action FROM gitlab."default".rails_transactions WHERE (action !~ /.+/ OR action =~ /^Grape#/) AND (banzai_cached_render_call_count > 0 OR banzai_cacheless_render_call_count > 0) GROUP BY time(1m), action END;
+CREATE CONTINUOUS QUERY grape_markdown_timings_overall ON gitlab BEGIN SELECT mean("duration") AS duration_mean, percentile("duration", 95) AS duration_95th, percentile("duration", 99) AS duration_99th INTO gitlab.downsampled.grape_markdown_timings_overall FROM gitlab."default".rails_method_calls WHERE (action !~ /.+/ OR action =~ /^Grape#/) AND method =~ /^Banzai/ GROUP BY time(1m) END;
+CREATE CONTINUOUS QUERY grape_method_call_timings_per_action_and_method ON gitlab BEGIN SELECT mean("duration") AS duration_mean, percentile("duration", 95) AS duration_95th, percentile("duration", 99) AS duration_99th INTO gitlab.downsampled.grape_method_call_timings_per_action_and_method FROM gitlab."default".rails_method_calls WHERE action !~ /.+/ OR action =~ /^Grape#/ GROUP BY time(1m), method, action END;
+CREATE CONTINUOUS QUERY grape_method_call_timings_per_method ON gitlab BEGIN SELECT mean("duration") AS duration_mean, percentile("duration", 95) AS duration_95th, percentile("duration", 99) AS duration_99th INTO gitlab.downsampled.grape_method_call_timings_per_method FROM gitlab."default".rails_method_calls WHERE action !~ /.+/ OR action =~ /^Grape#/ GROUP BY time(1m), method END;
+CREATE CONTINUOUS QUERY grape_transaction_counts_overall ON gitlab BEGIN SELECT count("duration") AS count INTO gitlab.downsampled.grape_transaction_counts_overall FROM gitlab."default".rails_transactions WHERE action !~ /.+/ OR action =~ /^Grape#/ GROUP BY time(1m) END;
+CREATE CONTINUOUS QUERY grape_transaction_counts_per_action ON gitlab BEGIN SELECT count("duration") AS count INTO gitlab.downsampled.grape_transaction_counts_per_action FROM gitlab."default".rails_transactions WHERE action !~ /.+/ OR action =~ /^Grape#/ GROUP BY time(1m), action END;
+CREATE CONTINUOUS QUERY grape_transaction_timings_overall ON gitlab BEGIN SELECT mean("duration") AS duration_mean, percentile("duration", 95) AS duration_95th, percentile("duration", 99) AS duration_99th, mean(sql_duration) AS sql_duration_mean, percentile(sql_duration, 95) AS sql_duration_95th, percentile(sql_duration, 99) AS sql_duration_99th, mean(view_duration) AS view_duration_mean, percentile(view_duration, 95) AS view_duration_95th, percentile(view_duration, 99) AS view_duration_99th, mean(cache_read_duration) AS cache_read_duration_mean, percentile(cache_read_duration, 99) AS cache_read_duration_99th, percentile(cache_read_duration, 95) AS cache_read_duration_95th, mean(cache_write_duration) AS cache_write_duration_mean, percentile(cache_write_duration, 99) AS cache_write_duration_99th, percentile(cache_write_duration, 95) AS cache_write_duration_95th, mean(cache_delete_duration) AS cache_delete_duration_mean, percentile(cache_delete_duration, 99) AS cache_delete_duration_99th, percentile(cache_delete_duration, 95) AS cache_delete_duration_95th, mean(cache_exists_duration) AS cache_exists_duration_mean, percentile(cache_exists_duration, 99) AS cache_exists_duration_99th, percentile(cache_exists_duration, 95) AS cache_exists_duration_95th, mean(cache_duration) AS cache_duration_mean, percentile(cache_duration, 99) AS cache_duration_99th, percentile(cache_duration, 95) AS cache_duration_95th, mean(method_duration) AS method_duration_mean, percentile(method_duration, 99) AS method_duration_99th, percentile(method_duration, 95) AS method_duration_95th INTO gitlab.downsampled.grape_transaction_timings_overall FROM gitlab."default".rails_transactions WHERE action !~ /.+/ OR action =~ /^Grape#/ GROUP BY time(1m) END;
+CREATE CONTINUOUS QUERY grape_transaction_timings_per_action ON gitlab BEGIN SELECT mean("duration") AS duration_mean, percentile("duration", 95) AS duration_95th, percentile("duration", 99) AS duration_99th, mean(sql_duration) AS sql_duration_mean, percentile(sql_duration, 95) AS sql_duration_95th, percentile(sql_duration, 99) AS sql_duration_99th, mean(view_duration) AS view_duration_mean, percentile(view_duration, 95) AS view_duration_95th, percentile(view_duration, 99) AS view_duration_99th, mean(cache_read_duration) AS cache_read_duration_mean, percentile(cache_read_duration, 99) AS cache_read_duration_99th, percentile(cache_read_duration, 95) AS cache_read_duration_95th, mean(cache_write_duration) AS cache_write_duration_mean, percentile(cache_write_duration, 99) AS cache_write_duration_99th, percentile(cache_write_duration, 95) AS cache_write_duration_95th, mean(cache_delete_duration) AS cache_delete_duration_mean, percentile(cache_delete_duration, 99) AS cache_delete_duration_99th, percentile(cache_delete_duration, 95) AS cache_delete_duration_95th, mean(cache_exists_duration) AS cache_exists_duration_mean, percentile(cache_exists_duration, 99) AS cache_exists_duration_99th, percentile(cache_exists_duration, 95) AS cache_exists_duration_95th, mean(cache_duration) AS cache_duration_mean, percentile(cache_duration, 99) AS cache_duration_99th, percentile(cache_duration, 95) AS cache_duration_95th, mean(method_duration) AS method_duration_mean, percentile(method_duration, 99) AS method_duration_99th, percentile(method_duration, 95) AS method_duration_95th INTO gitlab.downsampled.grape_transaction_timings_per_action FROM gitlab."default".rails_transactions WHERE action !~ /.+/ OR action =~ /^Grape#/ GROUP BY time(1m), action END;
+CREATE CONTINUOUS QUERY rails_file_descriptor_counts ON gitlab BEGIN SELECT sum(value) AS count INTO gitlab.downsampled.rails_file_descriptor_counts FROM gitlab."default".rails_file_descriptors GROUP BY time(1m) END;
+CREATE CONTINUOUS QUERY rails_gc_counts ON gitlab BEGIN SELECT sum(count) AS total, sum(minor_gc_count) AS minor, sum(major_gc_count) AS major INTO gitlab.downsampled.rails_gc_counts FROM gitlab."default".rails_gc_statistics GROUP BY time(1m) END;
+CREATE CONTINUOUS QUERY rails_gc_timings ON gitlab BEGIN SELECT mean(total_time) AS duration_mean, percentile(total_time, 95) AS duration_95th, percentile(total_time, 99) AS duration_99th INTO gitlab.downsampled.rails_gc_timings FROM gitlab."default".rails_gc_statistics GROUP BY time(1m) END;
+CREATE CONTINUOUS QUERY rails_git_timings_per_action ON gitlab BEGIN SELECT mean("duration") AS duration_mean, percentile("duration", 95) AS duration_95th, percentile("duration", 99) AS duration_99th INTO gitlab.downsampled.rails_git_timings_per_action FROM gitlab."default".rails_method_calls WHERE (action =~ /.+/ AND action !~ /^Grape#/) AND method =~ /^(Rugged|Gitlab::Git)/ GROUP BY time(1m), action END;
+CREATE CONTINUOUS QUERY rails_markdown_render_timings_overall ON gitlab BEGIN SELECT mean(banzai_cached_render_real_time) AS cached_real_mean, percentile(banzai_cached_render_real_time, 95) AS cached_real_95th, percentile(banzai_cached_render_real_time, 99) AS cached_real_99th, mean(banzai_cached_render_cpu_time) AS cached_cpu_mean, percentile(banzai_cached_render_cpu_time, 95) AS cached_cpu_95th, percentile(banzai_cached_render_cpu_time, 99) AS cached_cpu_99th, sum(banzai_cached_render_call_count) AS cached_call_count, mean(banzai_cacheless_render_real_time) AS cacheless_real_mean, percentile(banzai_cacheless_render_real_time, 95) AS cacheless_real_95th, percentile(banzai_cacheless_render_real_time, 99) AS cacheless_real_99th, mean(banzai_cacheless_render_cpu_time) AS cacheless_cpu_mean, percentile(banzai_cacheless_render_cpu_time, 95) AS cacheless_cpu_95th, percentile(banzai_cacheless_render_cpu_time, 99) AS cacheless_cpu_99th, sum(banzai_cacheless_render_call_count) AS cacheless_call_count INTO gitlab.downsampled.rails_markdown_render_timings_overall FROM gitlab."default".rails_transactions WHERE (action =~ /.+/ AND action !~ /^Grape#/) AND (banzai_cached_render_call_count > 0 OR banzai_cacheless_render_call_count > 0) GROUP BY time(1m) END;
+CREATE CONTINUOUS QUERY rails_markdown_render_timings_per_action ON gitlab BEGIN SELECT mean(banzai_cached_render_real_time) AS cached_real_mean, percentile(banzai_cached_render_real_time, 95) AS cached_real_95th, percentile(banzai_cached_render_real_time, 99) AS cached_real_99th, mean(banzai_cached_render_cpu_time) AS cached_cpu_mean, percentile(banzai_cached_render_cpu_time, 95) AS cached_cpu_95th, percentile(banzai_cached_render_cpu_time, 99) AS cached_cpu_99th, sum(banzai_cached_render_call_count) AS cached_call_count, mean(banzai_cacheless_render_real_time) AS cacheless_real_mean, percentile(banzai_cacheless_render_real_time, 95) AS cacheless_real_95th, percentile(banzai_cacheless_render_real_time, 99) AS cacheless_real_99th, mean(banzai_cacheless_render_cpu_time) AS cacheless_cpu_mean, percentile(banzai_cacheless_render_cpu_time, 95) AS cacheless_cpu_95th, percentile(banzai_cacheless_render_cpu_time, 99) AS cacheless_cpu_99th, sum(banzai_cacheless_render_call_count) AS cacheless_call_count INTO gitlab.downsampled.rails_markdown_render_timings_per_action FROM gitlab."default".rails_transactions WHERE (action =~ /.+/ AND action !~ /^Grape#/) AND (banzai_cached_render_call_count > 0 OR banzai_cacheless_render_call_count > 0) GROUP BY time(1m), action END;
+CREATE CONTINUOUS QUERY rails_markdown_timings_overall ON gitlab BEGIN SELECT mean("duration") AS duration_mean, percentile("duration", 95) AS duration_95th, percentile("duration", 99) AS duration_99th INTO gitlab.downsampled.rails_markdown_timings_overall FROM gitlab."default".rails_method_calls WHERE (action =~ /.+/ AND action !~ /^Grape#/) AND method =~ /^Banzai/ GROUP BY time(1m) END;
+CREATE CONTINUOUS QUERY rails_memory_usage_overall ON gitlab BEGIN SELECT mean(value) AS memory_mean, percentile(value, 95) AS memory_95th, percentile(value, 99) AS memory_99th INTO gitlab.downsampled.rails_memory_usage_overall FROM gitlab."default".rails_memory_usage GROUP BY time(1m) END;
+CREATE CONTINUOUS QUERY rails_method_call_timings_per_action_and_method ON gitlab BEGIN SELECT mean("duration") AS duration_mean, percentile("duration", 95) AS duration_95th, percentile("duration", 99) AS duration_99th INTO gitlab.downsampled.rails_method_call_timings_per_action_and_method FROM gitlab."default".rails_method_calls WHERE action =~ /.+/ AND action !~ /^Grape#/ GROUP BY time(1m), method, action END;
+CREATE CONTINUOUS QUERY rails_method_call_timings_per_method ON gitlab BEGIN SELECT mean("duration") AS duration_mean, percentile("duration", 95) AS duration_95th, percentile("duration", 99) AS duration_99th INTO gitlab.downsampled.rails_method_call_timings_per_method FROM gitlab."default".rails_method_calls WHERE action =~ /.+/ AND action !~ /^Grape#/ GROUP BY time(1m), method END;
+CREATE CONTINUOUS QUERY rails_object_counts_overall ON gitlab BEGIN SELECT sum(count) AS count INTO gitlab.downsampled.rails_object_counts_overall FROM gitlab."default".rails_object_counts GROUP BY time(1m) END;
+CREATE CONTINUOUS QUERY rails_object_counts_per_type ON gitlab BEGIN SELECT sum(count) AS count INTO gitlab.downsampled.rails_object_counts_per_type FROM gitlab."default".rails_object_counts GROUP BY time(1m), type END;
+CREATE CONTINUOUS QUERY rails_transaction_counts_overall ON gitlab BEGIN SELECT count("duration") AS count INTO gitlab.downsampled.rails_transaction_counts_overall FROM gitlab."default".rails_transactions WHERE action =~ /.+/ AND action !~ /^Grape#/ GROUP BY time(1m) END;
+CREATE CONTINUOUS QUERY rails_transaction_counts_per_action ON gitlab BEGIN SELECT count("duration") AS count INTO gitlab.downsampled.rails_transaction_counts_per_action FROM gitlab."default".rails_transactions WHERE action =~ /.+/ AND action !~ /^Grape#/ GROUP BY time(1m), action END;
+CREATE CONTINUOUS QUERY rails_transaction_timings_overall ON gitlab BEGIN SELECT mean("duration") AS duration_mean, percentile("duration", 95) AS duration_95th, percentile("duration", 99) AS duration_99th, mean(sql_duration) AS sql_duration_mean, percentile(sql_duration, 95) AS sql_duration_95th, percentile(sql_duration, 99) AS sql_duration_99th, mean(view_duration) AS view_duration_mean, percentile(view_duration, 95) AS view_duration_95th, percentile(view_duration, 99) AS view_duration_99th, mean(cache_read_duration) AS cache_read_duration_mean, percentile(cache_read_duration, 99) AS cache_read_duration_99th, percentile(cache_read_duration, 95) AS cache_read_duration_95th, mean(cache_write_duration) AS cache_write_duration_mean, percentile(cache_write_duration, 99) AS cache_write_duration_99th, percentile(cache_write_duration, 95) AS cache_write_duration_95th, mean(cache_delete_duration) AS cache_delete_duration_mean, percentile(cache_delete_duration, 99) AS cache_delete_duration_99th, percentile(cache_delete_duration, 95) AS cache_delete_duration_95th, mean(cache_exists_duration) AS cache_exists_duration_mean, percentile(cache_exists_duration, 99) AS cache_exists_duration_99th, percentile(cache_exists_duration, 95) AS cache_exists_duration_95th, mean(cache_duration) AS cache_duration_mean, percentile(cache_duration, 99) AS cache_duration_99th, percentile(cache_duration, 95) AS cache_duration_95th, mean(method_duration) AS method_duration_mean, percentile(method_duration, 99) AS method_duration_99th, percentile(method_duration, 95) AS method_duration_95th INTO gitlab.downsampled.rails_transaction_timings_overall FROM gitlab."default".rails_transactions WHERE action =~ /.+/ AND action !~ /^Grape#/ GROUP BY time(1m) END;
+CREATE CONTINUOUS QUERY rails_transaction_timings_per_action ON gitlab BEGIN SELECT mean("duration") AS duration_mean, percentile("duration", 95) AS duration_95th, percentile("duration", 99) AS duration_99th, mean(sql_duration) AS sql_duration_mean, percentile(sql_duration, 95) AS sql_duration_95th, percentile(sql_duration, 99) AS sql_duration_99th, mean(view_duration) AS view_duration_mean, percentile(view_duration, 95) AS view_duration_95th, percentile(view_duration, 99) AS view_duration_99th, mean(cache_read_duration) AS cache_read_duration_mean, percentile(cache_read_duration, 99) AS cache_read_duration_99th, percentile(cache_read_duration, 95) AS cache_read_duration_95th, mean(cache_write_duration) AS cache_write_duration_mean, percentile(cache_write_duration, 99) AS cache_write_duration_99th, percentile(cache_write_duration, 95) AS cache_write_duration_95th, mean(cache_delete_duration) AS cache_delete_duration_mean, percentile(cache_delete_duration, 99) AS cache_delete_duration_99th, percentile(cache_delete_duration, 95) AS cache_delete_duration_95th, mean(cache_exists_duration) AS cache_exists_duration_mean, percentile(cache_exists_duration, 99) AS cache_exists_duration_99th, percentile(cache_exists_duration, 95) AS cache_exists_duration_95th, mean(cache_duration) AS cache_duration_mean, percentile(cache_duration, 99) AS cache_duration_99th, percentile(cache_duration, 95) AS cache_duration_95th, mean(method_duration) AS method_duration_mean, percentile(method_duration, 99) AS method_duration_99th, percentile(method_duration, 95) AS method_duration_95th INTO gitlab.downsampled.rails_transaction_timings_per_action FROM gitlab."default".rails_transactions WHERE action =~ /.+/ AND action !~ /^Grape#/ GROUP BY time(1m), action END;
+CREATE CONTINUOUS QUERY rails_view_timings_per_action_and_view ON gitlab BEGIN SELECT mean("duration") AS duration_mean, percentile("duration", 95) AS duration_95th, percentile("duration", 99) AS duration_99th INTO gitlab.downsampled.rails_view_timings_per_action_and_view FROM gitlab."default".rails_views WHERE action =~ /.+/ AND action !~ /^Grape#/ GROUP BY time(1m), action, view END;
+CREATE CONTINUOUS QUERY sidekiq_file_descriptor_counts ON gitlab BEGIN SELECT sum(value) AS count INTO gitlab.downsampled.sidekiq_file_descriptor_counts FROM gitlab."default".sidekiq_file_descriptors GROUP BY time(1m) END;
+CREATE CONTINUOUS QUERY sidekiq_gc_counts ON gitlab BEGIN SELECT sum(count) AS total, sum(minor_gc_count) AS minor, sum(major_gc_count) AS major INTO gitlab.downsampled.sidekiq_gc_counts FROM gitlab."default".sidekiq_gc_statistics GROUP BY time(1m) END;
+CREATE CONTINUOUS QUERY sidekiq_gc_timings ON gitlab BEGIN SELECT mean(total_time) AS duration_mean, percentile(total_time, 95) AS duration_95th, percentile(total_time, 99) AS duration_99th INTO gitlab.downsampled.sidekiq_gc_timings FROM gitlab."default".sidekiq_gc_statistics GROUP BY time(1m) END;
+CREATE CONTINUOUS QUERY sidekiq_git_timings_per_action ON gitlab BEGIN SELECT mean("duration") AS duration_mean, percentile("duration", 95) AS duration_95th, percentile("duration", 99) AS duration_99th INTO gitlab.downsampled.sidekiq_git_timings_per_action FROM gitlab."default".sidekiq_method_calls WHERE method =~ /^(Rugged|Gitlab::Git)/ GROUP BY time(1m), action END;
+CREATE CONTINUOUS QUERY sidekiq_markdown_render_timings_overall ON gitlab BEGIN SELECT mean(banzai_cached_render_real_time) AS cached_real_mean, percentile(banzai_cached_render_real_time, 95) AS cached_real_95th, percentile(banzai_cached_render_real_time, 99) AS cached_real_99th, mean(banzai_cached_render_cpu_time) AS cached_cpu_mean, percentile(banzai_cached_render_cpu_time, 95) AS cached_cpu_95th, percentile(banzai_cached_render_cpu_time, 99) AS cached_cpu_99th, sum(banzai_cached_render_call_count) AS cached_call_count, mean(banzai_cacheless_render_real_time) AS cacheless_real_mean, percentile(banzai_cacheless_render_real_time, 95) AS cacheless_real_95th, percentile(banzai_cacheless_render_real_time, 99) AS cacheless_real_99th, mean(banzai_cacheless_render_cpu_time) AS cacheless_cpu_mean, percentile(banzai_cacheless_render_cpu_time, 95) AS cacheless_cpu_95th, percentile(banzai_cacheless_render_cpu_time, 99) AS cacheless_cpu_99th, sum(banzai_cacheless_render_call_count) AS cacheless_call_count INTO gitlab.downsampled.sidekiq_markdown_render_timings_overall FROM gitlab."default".sidekiq_transactions WHERE (banzai_cached_render_call_count > 0 OR banzai_cacheless_render_call_count > 0) GROUP BY time(1m) END;
+CREATE CONTINUOUS QUERY sidekiq_markdown_render_timings_per_action ON gitlab BEGIN SELECT mean(banzai_cached_render_real_time) AS cached_real_mean, percentile(banzai_cached_render_real_time, 95) AS cached_real_95th, percentile(banzai_cached_render_real_time, 99) AS cached_real_99th, mean(banzai_cached_render_cpu_time) AS cached_cpu_mean, percentile(banzai_cached_render_cpu_time, 95) AS cached_cpu_95th, percentile(banzai_cached_render_cpu_time, 99) AS cached_cpu_99th, sum(banzai_cached_render_call_count) AS cached_call_count, mean(banzai_cacheless_render_real_time) AS cacheless_real_mean, percentile(banzai_cacheless_render_real_time, 95) AS cacheless_real_95th, percentile(banzai_cacheless_render_real_time, 99) AS cacheless_real_99th, mean(banzai_cacheless_render_cpu_time) AS cacheless_cpu_mean, percentile(banzai_cacheless_render_cpu_time, 95) AS cacheless_cpu_95th, percentile(banzai_cacheless_render_cpu_time, 99) AS cacheless_cpu_99th, sum(banzai_cacheless_render_call_count) AS cacheless_call_count INTO gitlab.downsampled.sidekiq_markdown_render_timings_per_action FROM gitlab."default".sidekiq_transactions WHERE (banzai_cached_render_call_count > 0 OR banzai_cacheless_render_call_count > 0) GROUP BY time(1m), action END;
+CREATE CONTINUOUS QUERY sidekiq_markdown_timings_overall ON gitlab BEGIN SELECT mean("duration") AS duration_mean, percentile("duration", 95) AS duration_95th, percentile("duration", 99) AS duration_99th INTO gitlab.downsampled.sidekiq_markdown_timings_overall FROM gitlab."default".sidekiq_method_calls WHERE method =~ /^Banzai/ GROUP BY time(1m) END;
+CREATE CONTINUOUS QUERY sidekiq_memory_usage_overall ON gitlab BEGIN SELECT mean(value) AS memory_mean, percentile(value, 95) AS memory_95th, percentile(value, 99) AS memory_99th INTO gitlab.downsampled.sidekiq_memory_usage_overall FROM gitlab."default".sidekiq_memory_usage GROUP BY time(1m) END;
+CREATE CONTINUOUS QUERY sidekiq_method_call_timings_per_action_and_method ON gitlab BEGIN SELECT mean("duration") AS duration_mean, percentile("duration", 95) AS duration_95th, percentile("duration", 99) AS duration_99th INTO gitlab.downsampled.sidekiq_method_call_timings_per_action_and_method FROM gitlab."default".sidekiq_method_calls GROUP BY time(1m), method, action END;
+CREATE CONTINUOUS QUERY sidekiq_method_call_timings_per_method ON gitlab BEGIN SELECT mean("duration") AS duration_mean, percentile("duration", 95) AS duration_95th, percentile("duration", 99) AS duration_99th INTO gitlab.downsampled.sidekiq_method_call_timings_per_method FROM gitlab."default".sidekiq_method_calls GROUP BY time(1m), method END;
+CREATE CONTINUOUS QUERY sidekiq_object_counts_overall ON gitlab BEGIN SELECT sum(count) AS count INTO gitlab.downsampled.sidekiq_object_counts_overall FROM gitlab."default".sidekiq_object_counts GROUP BY time(1m) END;
+CREATE CONTINUOUS QUERY sidekiq_object_counts_per_type ON gitlab BEGIN SELECT sum(count) AS count INTO gitlab.downsampled.sidekiq_object_counts_per_type FROM gitlab."default".sidekiq_object_counts GROUP BY time(1m), type END;
+CREATE CONTINUOUS QUERY sidekiq_transaction_counts_overall ON gitlab BEGIN SELECT count("duration") AS count INTO gitlab.downsampled.sidekiq_transaction_counts_overall FROM gitlab."default".sidekiq_transactions GROUP BY time(1m) END;
+CREATE CONTINUOUS QUERY sidekiq_transaction_counts_per_action ON gitlab BEGIN SELECT count("duration") AS count INTO gitlab.downsampled.sidekiq_transaction_counts_per_action FROM gitlab."default".sidekiq_transactions GROUP BY time(1m), action END;
+CREATE CONTINUOUS QUERY sidekiq_transaction_timings_overall ON gitlab BEGIN SELECT mean("duration") AS duration_mean, percentile("duration", 95) AS duration_95th, percentile("duration", 99) AS duration_99th, mean(sql_duration) AS sql_duration_mean, percentile(sql_duration, 95) AS sql_duration_95th, percentile(sql_duration, 99) AS sql_duration_99th, mean(view_duration) AS view_duration_mean, percentile(view_duration, 95) AS view_duration_95th, percentile(view_duration, 99) AS view_duration_99th, mean(cache_read_duration) AS cache_read_duration_mean, percentile(cache_read_duration, 99) AS cache_read_duration_99th, percentile(cache_read_duration, 95) AS cache_read_duration_95th, mean(cache_write_duration) AS cache_write_duration_mean, percentile(cache_write_duration, 99) AS cache_write_duration_99th, percentile(cache_write_duration, 95) AS cache_write_duration_95th, mean(cache_delete_duration) AS cache_delete_duration_mean, percentile(cache_delete_duration, 99) AS cache_delete_duration_99th, percentile(cache_delete_duration, 95) AS cache_delete_duration_95th, mean(cache_exists_duration) AS cache_exists_duration_mean, percentile(cache_exists_duration, 99) AS cache_exists_duration_99th, percentile(cache_exists_duration, 95) AS cache_exists_duration_95th, mean(cache_duration) AS cache_duration_mean, percentile(cache_duration, 99) AS cache_duration_99th, percentile(cache_duration, 95) AS cache_duration_95th, mean(method_duration) AS method_duration_mean, percentile(method_duration, 99) AS method_duration_99th, percentile(method_duration, 95) AS method_duration_95th INTO gitlab.downsampled.sidekiq_transaction_timings_overall FROM gitlab."default".sidekiq_transactions GROUP BY time(1m) END;
+CREATE CONTINUOUS QUERY sidekiq_transaction_timings_per_action ON gitlab BEGIN SELECT mean("duration") AS duration_mean, percentile("duration", 95) AS duration_95th, percentile("duration", 99) AS duration_99th, mean(sql_duration) AS sql_duration_mean, percentile(sql_duration, 95) AS sql_duration_95th, percentile(sql_duration, 99) AS sql_duration_99th, mean(view_duration) AS view_duration_mean, percentile(view_duration, 95) AS view_duration_95th, percentile(view_duration, 99) AS view_duration_99th, mean(cache_read_duration) AS cache_read_duration_mean, percentile(cache_read_duration, 99) AS cache_read_duration_99th, percentile(cache_read_duration, 95) AS cache_read_duration_95th, mean(cache_write_duration) AS cache_write_duration_mean, percentile(cache_write_duration, 99) AS cache_write_duration_99th, percentile(cache_write_duration, 95) AS cache_write_duration_95th, mean(cache_delete_duration) AS cache_delete_duration_mean, percentile(cache_delete_duration, 99) AS cache_delete_duration_99th, percentile(cache_delete_duration, 95) AS cache_delete_duration_95th, mean(cache_exists_duration) AS cache_exists_duration_mean, percentile(cache_exists_duration, 99) AS cache_exists_duration_99th, percentile(cache_exists_duration, 95) AS cache_exists_duration_95th, mean(cache_duration) AS cache_duration_mean, percentile(cache_duration, 99) AS cache_duration_99th, percentile(cache_duration, 95) AS cache_duration_95th, mean(method_duration) AS method_duration_mean, percentile(method_duration, 99) AS method_duration_99th, percentile(method_duration, 95) AS method_duration_95th INTO gitlab.downsampled.sidekiq_transaction_timings_per_action FROM gitlab."default".sidekiq_transactions GROUP BY time(1m), action END;
+CREATE CONTINUOUS QUERY sidekiq_view_timings_per_action_and_view ON gitlab BEGIN SELECT mean("duration") AS duration_mean, percentile("duration", 95) AS duration_95th, percentile("duration", 99) AS duration_99th INTO gitlab.downsampled.sidekiq_view_timings_per_action_and_view FROM gitlab."default".sidekiq_views GROUP BY time(1m), action, view END;
+CREATE CONTINUOUS QUERY web_transaction_counts_overall ON gitlab BEGIN SELECT count("duration") AS count INTO gitlab.downsampled.web_transaction_counts_overall FROM gitlab."default".rails_transactions GROUP BY time(1m) END;
+```
+
+## Import Dashboards
+
+You can now import a set of default dashboards that will give you a good
+start on displaying useful information. GitLab has published a set of default
+[Grafana dashboards][grafana-dashboards] to get you started. Clone the
+repository or download a zip/tarball, then follow these steps to import each
+JSON file.
+
+Open the dashboard dropdown menu and click 'Import'
+
+![Grafana dashboard dropdown](img/grafana_dashboard_dropdown.png)
+
+Click 'Choose file' and browse to the location where you downloaded or cloned
+the dashboard repository. Pick one of the JSON files to import.
+
+![Grafana dashboard import](img/grafana_dashboard_import.png)
+
+Once the dashboard is imported, be sure to click save icon in the top bar. If
+you do not save the dashboard after importing it will be removed when you
+navigate away.
+
+![Grafana save icon](img/grafana_save_icon.png)
+
+Repeat this process for each dashboard you wish to import.
+
+Alternatively you can automatically import all the dashboards into your Grafana
+instance. See the README of the [Grafana dashboards][grafana-dashboards]
+repository for more information on this process.
+
+[grafana-dashboards]: https://gitlab.com/gitlab-org/grafana-dashboards
+
+---
+
+Read more on:
+
+- [Introduction to GitLab Performance Monitoring](introduction.md)
+- [GitLab Configuration](gitlab_configuration.md)
+- [InfluxDB Installation/Configuration](influxdb_configuration.md)
+- [InfluxDB Schema](influxdb_schema.md)
diff --git a/doc/monitoring/performance/img/grafana_dashboard_dropdown.png b/doc/monitoring/performance/img/grafana_dashboard_dropdown.png
new file mode 100644
index 00000000000..b4448c7a09f
--- /dev/null
+++ b/doc/monitoring/performance/img/grafana_dashboard_dropdown.png
Binary files differ
diff --git a/doc/monitoring/performance/img/grafana_dashboard_import.png b/doc/monitoring/performance/img/grafana_dashboard_import.png
new file mode 100644
index 00000000000..5a2d3c0937a
--- /dev/null
+++ b/doc/monitoring/performance/img/grafana_dashboard_import.png
Binary files differ
diff --git a/doc/monitoring/performance/img/grafana_data_source_configuration.png b/doc/monitoring/performance/img/grafana_data_source_configuration.png
new file mode 100644
index 00000000000..7e2e111f570
--- /dev/null
+++ b/doc/monitoring/performance/img/grafana_data_source_configuration.png
Binary files differ
diff --git a/doc/monitoring/performance/img/grafana_data_source_empty.png b/doc/monitoring/performance/img/grafana_data_source_empty.png
new file mode 100644
index 00000000000..11e27571e64
--- /dev/null
+++ b/doc/monitoring/performance/img/grafana_data_source_empty.png
Binary files differ
diff --git a/doc/monitoring/performance/img/grafana_save_icon.png b/doc/monitoring/performance/img/grafana_save_icon.png
new file mode 100644
index 00000000000..3d4265bee8e
--- /dev/null
+++ b/doc/monitoring/performance/img/grafana_save_icon.png
Binary files differ
diff --git a/doc/monitoring/performance/influxdb_configuration.md b/doc/monitoring/performance/influxdb_configuration.md
index 3a2b598b78f..c30cd2950d8 100644
--- a/doc/monitoring/performance/influxdb_configuration.md
+++ b/doc/monitoring/performance/influxdb_configuration.md
@@ -181,6 +181,7 @@ Read more on:
- [Introduction to GitLab Performance Monitoring](introduction.md)
- [GitLab Configuration](gitlab_configuration.md)
- [InfluxDB Schema](influxdb_schema.md)
+- [Grafana Install/Configuration](grafana_configuration.md)
[influxdb-retention]: https://docs.influxdata.com/influxdb/v0.9/query_language/database_management/#retention-policy-management
[influxdb documentation]: https://docs.influxdata.com/influxdb/v0.9/
diff --git a/doc/monitoring/performance/influxdb_schema.md b/doc/monitoring/performance/influxdb_schema.md
index a5a8aebd2d1..41861860b6d 100644
--- a/doc/monitoring/performance/influxdb_schema.md
+++ b/doc/monitoring/performance/influxdb_schema.md
@@ -85,3 +85,4 @@ Read more on:
- [Introduction to GitLab Performance Monitoring](introduction.md)
- [GitLab Configuration](gitlab_configuration.md)
- [InfluxDB Configuration](influxdb_configuration.md)
+- [Grafana Install/Configuration](grafana_configuration.md)
diff --git a/doc/monitoring/performance/introduction.md b/doc/monitoring/performance/introduction.md
index f2460d31302..79904916b7e 100644
--- a/doc/monitoring/performance/introduction.md
+++ b/doc/monitoring/performance/introduction.md
@@ -8,8 +8,9 @@ Apart from this introduction, you are advised to read through the following
documents in order to understand and properly configure GitLab Performance Monitoring:
- [GitLab Configuration](gitlab_configuration.md)
-- [InfluxDB Configuration](influxdb_configuration.md)
+- [InfluxDB Install/Configuration](influxdb_configuration.md)
- [InfluxDB Schema](influxdb_schema.md)
+- [Grafana Install/Configuration](grafana_configuration.md)
## Introduction to GitLab Performance Monitoring
diff --git a/doc/operations/moving_repositories.md b/doc/operations/moving_repositories.md
index 39086b7a251..54adb99386a 100644
--- a/doc/operations/moving_repositories.md
+++ b/doc/operations/moving_repositories.md
@@ -134,7 +134,7 @@ sudo -u git sh -c '
cat /var/opt/gitlab/transfer-logs/* | sort | uniq -u |\
/usr/bin/env JOBS=10 \
/opt/gitlab/embedded/service/gitlab-rails/bin/parallel-rsync-repos \
- /var/opt/gitlab/transfer-logs/succes-$(date +%s).log \
+ /var/opt/gitlab/transfer-logs/success-$(date +%s).log \
/var/opt/gitlab/git-data/repositories \
/mnt/gitlab/repositories
'
@@ -145,7 +145,7 @@ sudo -u git -H sh -c '
cat /home/git/transfer-logs/* | sort | uniq -u |\
/usr/bin/env JOBS=10 \
bin/parallel-rsync-repos \
- /home/git/transfer-logs/succes-$(date +%s).log \
+ /home/git/transfer-logs/success-$(date +%s).log \
/home/git/repositories \
/mnt/gitlab/repositories
`
@@ -164,7 +164,7 @@ sudo gitlab-rake gitlab:list_repos SINCE='2015-10-1 12:00 UTC' |\
sudo -u git \
/usr/bin/env JOBS=10 \
/opt/gitlab/embedded/service/gitlab-rails/bin/parallel-rsync-repos \
- succes-$(date +%s).log \
+ success-$(date +%s).log \
/var/opt/gitlab/git-data/repositories \
/mnt/gitlab/repositories
@@ -174,7 +174,7 @@ sudo -u git -H bundle exec rake gitlab:list_repos SINCE='2015-10-1 12:00 UTC' |\
sudo -u git -H \
/usr/bin/env JOBS=10 \
bin/parallel-rsync-repos \
- succes-$(date +%s).log \
+ success-$(date +%s).log \
/home/git/repositories \
/mnt/gitlab/repositories
```
diff --git a/doc/operations/sidekiq_memory_killer.md b/doc/operations/sidekiq_memory_killer.md
index 811c2192a19..b5e78348989 100644
--- a/doc/operations/sidekiq_memory_killer.md
+++ b/doc/operations/sidekiq_memory_killer.md
@@ -36,5 +36,5 @@ The MemoryKiller is controlled using environment variables.
Existing jobs get 30 seconds to finish. After that, the MemoryKiller tells
Sidekiq to shut down, and an external supervision mechanism (e.g. Runit) must
restart Sidekiq.
-- `SIDEKIQ_MEMORY_KILLER_SHUTDOWN_SIGNAL`: defaults to 'SIGTERM'. The name of
+- `SIDEKIQ_MEMORY_KILLER_SHUTDOWN_SIGNAL`: defaults to `SIGKILL`. The name of
the final signal sent to the Sidekiq process when we want it to shut down.
diff --git a/doc/permissions/permissions.md b/doc/permissions/permissions.md
index 3d375e47c8e..963b35de3a0 100644
--- a/doc/permissions/permissions.md
+++ b/doc/permissions/permissions.md
@@ -27,6 +27,8 @@ documentation](../workflow/add-user/add-user.md).
| Manage issue tracker | | ✓ | ✓ | ✓ | ✓ |
| Manage labels | | ✓ | ✓ | ✓ | ✓ |
| See a commit status | | ✓ | ✓ | ✓ | ✓ |
+| See a container registry | | ✓ | ✓ | ✓ | ✓ |
+| See environments | | ✓ | ✓ | ✓ | ✓ |
| Manage merge requests | | | ✓ | ✓ | ✓ |
| Create new merge request | | | ✓ | ✓ | ✓ |
| Create new branches | | | ✓ | ✓ | ✓ |
@@ -37,6 +39,9 @@ documentation](../workflow/add-user/add-user.md).
| Write a wiki | | | ✓ | ✓ | ✓ |
| Cancel and retry builds | | | ✓ | ✓ | ✓ |
| Create or update commit status | | | ✓ | ✓ | ✓ |
+| Update a container registry | | | ✓ | ✓ | ✓ |
+| Remove a container registry image | | | ✓ | ✓ | ✓ |
+| Create new environments | | | ✓ | ✓ | ✓ |
| Create new milestones | | | | ✓ | ✓ |
| Add new team members | | | | ✓ | ✓ |
| Push to protected branches | | | | ✓ | ✓ |
@@ -49,13 +54,15 @@ documentation](../workflow/add-user/add-user.md).
| Manage runners | | | | ✓ | ✓ |
| Manage build triggers | | | | ✓ | ✓ |
| Manage variables | | | | ✓ | ✓ |
+| Delete environments | | | | ✓ | ✓ |
| Switch visibility level | | | | | ✓ |
| Transfer project to another namespace | | | | | ✓ |
| Remove project | | | | | ✓ |
-| Force push to protected branches | | | | | |
-| Remove protected branches | | | | | |
+| Force push to protected branches [^2] | | | | | |
+| Remove protected branches [^2] | | | | | |
[^1]: If **Allow guest to access builds** is enabled in CI settings
+[^2]: Not allowed for Guest, Reporter, Developer, Master, or Owner
## Group
diff --git a/doc/profile/2fa_u2f_authenticate.png b/doc/profile/2fa_u2f_authenticate.png
new file mode 100644
index 00000000000..b9138ff60db
--- /dev/null
+++ b/doc/profile/2fa_u2f_authenticate.png
Binary files differ
diff --git a/doc/profile/2fa_u2f_register.png b/doc/profile/2fa_u2f_register.png
new file mode 100644
index 00000000000..15b3683ef73
--- /dev/null
+++ b/doc/profile/2fa_u2f_register.png
Binary files differ
diff --git a/doc/profile/two_factor_authentication.md b/doc/profile/two_factor_authentication.md
index a0e23c1586c..82505b13401 100644
--- a/doc/profile/two_factor_authentication.md
+++ b/doc/profile/two_factor_authentication.md
@@ -8,12 +8,27 @@ your phone.
By enabling 2FA, the only way someone other than you can log into your account
is to know your username and password *and* have access to your phone.
-#### Note
+> **Note:**
When you enable 2FA, don't forget to back up your recovery codes. For your safety, if you
lose your codes for GitLab.com, we can't disable or recover them.
+In addition to a phone application, GitLab supports U2F (universal 2nd factor) devices as
+the second factor of authentication. Once enabled, in addition to supplying your username and
+password to login, you'll be prompted to activate your U2F device (usually by pressing
+a button on it), and it will perform secure authentication on your behalf.
+
+> **Note:** Support for U2F devices was added in version 8.8
+
+The U2F workflow is only supported by Google Chrome at this point, so we _strongly_ recommend
+that you set up both methods of two-factor authentication, so you can still access your account
+from other browsers.
+
+> **Note:** GitLab officially only supports [Yubikey] U2F devices.
+
## Enabling 2FA
+### Enable 2FA via mobile application
+
**In GitLab:**
1. Log in to your GitLab account.
@@ -38,9 +53,26 @@ lose your codes for GitLab.com, we can't disable or recover them.
1. Click **Submit**.
If the pin you entered was correct, you'll see a message indicating that
-Two-factor Authentication has been enabled, and you'll be presented with a list
+Two-Factor Authentication has been enabled, and you'll be presented with a list
of recovery codes.
+### Enable 2FA via U2F device
+
+**In GitLab:**
+
+1. Log in to your GitLab account.
+1. Go to your **Profile Settings**.
+1. Go to **Account**.
+1. Click **Enable Two-Factor Authentication**.
+1. Plug in your U2F device.
+1. Click on **Setup New U2F Device**.
+1. A light will start blinking on your device. Activate it by pressing its button.
+
+You will see a message indicating that your device was successfully set up.
+Click on **Register U2F Device** to complete the process.
+
+![Two-Factor U2F Setup](2fa_u2f_register.png)
+
## Recovery Codes
Should you ever lose access to your phone, you can use one of the ten provided
@@ -51,21 +83,39 @@ account.
If you lose the recovery codes or just want to generate new ones, you can do so
from the **Profile Settings** > **Account** page where you first enabled 2FA.
+> **Note:** Recovery codes are not generated for U2F devices.
+
## Logging in with 2FA Enabled
Logging in with 2FA enabled is only slightly different than a normal login.
Enter your username and password credentials as you normally would, and you'll
-be presented with a second prompt for an authentication code. Enter the pin from
-your phone's application or a recovery code to log in.
+be presented with a second prompt, depending on which type of 2FA you've enabled.
+
+### Log in via mobile application
+
+Enter the pin from your phone's application or a recovery code to log in.
-![Two-factor authentication on sign in](2fa_auth.png)
+![Two-Factor Authentication on sign in via OTP](2fa_auth.png)
+
+### Log in via U2F device
+
+1. Click **Login via U2F Device**
+1. A light will start blinking on your device. Activate it by pressing its button.
+
+You will see a message indicating that your device responded to the authentication request.
+Click on **Authenticate via U2F Device** to complete the process.
+
+![Two-Factor Authentication on sign in via U2F device](2fa_u2f_authenticate.png)
## Disabling 2FA
1. Log in to your GitLab account.
1. Go to your **Profile Settings**.
1. Go to **Account**.
-1. Click **Disable Two-factor Authentication**.
+1. Click **Disable**, under **Two-Factor Authentication**.
+
+This will clear all your two-factor authentication registrations, including mobile
+applications and U2F devices.
## Note to GitLab administrators
@@ -74,3 +124,4 @@ You need to take special care to that 2FA keeps working after
[Google Authenticator]: https://support.google.com/accounts/answer/1066447?hl=en
[FreeOTP]: https://fedorahosted.org/freeotp/
+[YubiKey]: https://www.yubico.com/products/yubikey-hardware/
diff --git a/doc/project_services/img/jira_service_page.png b/doc/project_services/img/jira_service_page.png
index 2b37eda3520..c225daa81e1 100644
--- a/doc/project_services/img/jira_service_page.png
+++ b/doc/project_services/img/jira_service_page.png
Binary files differ
diff --git a/doc/project_services/jira.md b/doc/project_services/jira.md
index 27170c1eb19..b626c746c79 100644
--- a/doc/project_services/jira.md
+++ b/doc/project_services/jira.md
@@ -1,9 +1,9 @@
# GitLab JIRA integration
-_**Note:**
+>**Note:**
Full JIRA integration was previously exclusive to GitLab Enterprise Edition.
With [GitLab 8.3 forward][8_3_post], this feature in now [backported][jira-ce]
-to GitLab Community Edition as well._
+to GitLab Community Edition as well.
---
@@ -88,8 +88,9 @@ password as they will be needed when configuring GitLab in the next section.
### Configuring GitLab
-_**Note:** The currently supported JIRA versions are v6.x and v7.x. and GitLab
-7.8 or higher is required._
+>**Note:**
+The currently supported JIRA versions are v6.x and v7.x. and GitLab
+7.8 or higher is required.
---
@@ -113,13 +114,24 @@ Fill in the required details on the page, as described in the table below.
| `Api url` | The base URL of the JIRA API. It may be omitted, in which case GitLab will automatically use API version `2` based on the `project url`. It is of the form: `https://<jira_host_url>/rest/api/2`. |
| `Username` | The username of the user created in [configuring JIRA step](#configuring-jira). |
| `Password` |The password of the user created in [configuring JIRA step](#configuring-jira). |
-| `JIRA issue transition` | This setting is very important to set up correctly. It is the ID of a transition that moves issues to a closed state. You can find this number under the JIRA workflow administration (**Administration > Issues > Workflows**) by selecting **View** under **Operations** of the desired workflow of your project. The ID of each state can be found inside the parenthesis of each transition name under the **Transitions (id)** column ([see screenshot](img/jira_issues_workflow.png)). By default, this ID is set to `2` |
+| `JIRA issue transition` | This setting is very important to set up correctly. It is the ID of a transition that moves issues to a closed state. You can find this number under the JIRA workflow administration (**Administration > Issues > Workflows**) by selecting **View** under **Operations** of the desired workflow of your project. The ID of each state can be found inside the parenthesis of each transition name under the **Transitions (id)** column ([see screenshot][trans]). By default, this ID is set to `2`. |
After saving the configuration, your GitLab project will be able to interact
with the linked JIRA project.
+For example, given the settings below:
+
+- the JIRA URL is `https://jira.example.com`
+- the project is named `GITLAB`
+- the user is named `gitlab`
+- the JIRA issue transition is 151 (based on the [JIRA issue transition][trans])
+
+the following screenshot shows how the JIRA service settings should look like.
+
![JIRA service page](img/jira_service_page.png)
+[trans]: img/jira_issues_workflow.png
+
---
## JIRA issues
diff --git a/doc/project_services/project_services.md b/doc/project_services/project_services.md
index 3fea2cff0b9..a5af620d9be 100644
--- a/doc/project_services/project_services.md
+++ b/doc/project_services/project_services.md
@@ -1,7 +1,24 @@
# Project Services
Project services allow you to integrate GitLab with other applications. Below
-is list of the currently supported ones. Click on the service links to see
+is list of the currently supported ones.
+
+You can find these within GitLab in the Services page under Project Settings if
+you are at least a master on the project.
+Project Services are a bit like plugins in that they allow a lot of freedom in
+adding functionality to GitLab. For example there is also a service that can
+send an email every time someone pushes new commits.
+
+Because GitLab is open source we can ship with the code and tests for all
+plugins. This allows the community to keep the plugins up to date so that they
+always work in newer GitLab versions.
+
+For an overview of what projects services are available without logging in,
+please see the [project_services directory][projects-code].
+
+[projects-code]: https://gitlab.com/gitlab-org/gitlab-ce/tree/master/app/models/project_services
+
+Click on the service links to see
further configuration instructions and details. Contributions are welcome.
## Services
diff --git a/doc/public_access/public_access.md b/doc/public_access/public_access.md
index 6e22ea7b72a..17bb75ececd 100644
--- a/doc/public_access/public_access.md
+++ b/doc/public_access/public_access.md
@@ -35,6 +35,21 @@ the repository.
1. Go to your project's **Settings**
1. Change "Visibility Level" to either Public, Internal or Private
+## Visibility of groups
+
+>**Note:**
+[Starting with][3323] GitLab 8.6, the group visibility has changed and can be
+configured the same way as projects. In previous versions, a group's page was
+always visible to all users.
+
+Like with projects, the visibility of a group can be set to dictate whether
+anonymous users, all signed in users, or only explicit group members can view
+it. The restriction for visibility levels on the application setting level also
+applies to groups, so if that's set to internal, the explore page will be empty
+for anonymous users. The group page now has a visibility level icon.
+
+[3323]: https://gitlab.com/gitlab-org/gitlab-ce/merge_requests/3323
+
## Visibility of users
The public page of a user, located at `/u/username`, is always visible whether
@@ -43,13 +58,8 @@ you are logged in or not.
When visiting the public page of a user, you can only see the projects which
you are privileged to.
-## Visibility of groups
-
-The public page of a group, located at `/groups/groupname`, is always visible
-to everyone.
+If the public level is restricted, user profiles are only visible to logged in users.
-Logged out users will be able to see the description and the avatar of the
-group as well as all public projects belonging to that group.
## Restricting the use of public or internal projects
diff --git a/doc/raketasks/README.md b/doc/raketasks/README.md
index 6be954ad68b..a49c43b8ef2 100644
--- a/doc/raketasks/README.md
+++ b/doc/raketasks/README.md
@@ -8,4 +8,4 @@
- [User management](user_management.md)
- [Webhooks](web_hooks.md)
- [Import](import.md) of git repositories in bulk
-- [Rebuild authorized_keys file](http://doc.gitlab.com/ce/raketasks/maintenance.html#rebuild-authorized_keys-file) task for administrators
+- [Rebuild authorized_keys file](http://docs.gitlab.com/ce/raketasks/maintenance.html#rebuild-authorized_keys-file) task for administrators
diff --git a/doc/raketasks/backup_restore.md b/doc/raketasks/backup_restore.md
index f6d1234ac4a..fa976134341 100644
--- a/doc/raketasks/backup_restore.md
+++ b/doc/raketasks/backup_restore.md
@@ -249,6 +249,9 @@ reconfigure` after changing `gitlab-secrets.json`.
### Installation from source
```
+# Stop processes that are connected to the database
+sudo service gitlab stop
+
bundle exec rake gitlab:backup:restore RAILS_ENV=production
```
@@ -292,36 +295,49 @@ Deleting tmp directories...[DONE]
### Omnibus installations
-We will assume that you have installed GitLab from an omnibus package and run
-`sudo gitlab-ctl reconfigure` at least once.
+This procedure assumes that:
+
+- You have installed the exact same version of GitLab Omnibus with which the
+ backup was created
+- You have run `sudo gitlab-ctl reconfigure` at least once
+- GitLab is running. If not, start it using `sudo gitlab-ctl start`.
-First make sure your backup tar file is in `/var/opt/gitlab/backups` (or wherever `gitlab_rails['backup_path']` points to).
+First make sure your backup tar file is in the backup directory described in the
+`gitlab.rb` configuration `gitlab_rails['backup_path']`. The default is
+`/var/opt/gitlab/backups`.
```shell
sudo cp 1393513186_gitlab_backup.tar /var/opt/gitlab/backups/
```
-Next, restore the backup by running the restore command. You need to specify the
-timestamp of the backup you are restoring.
+Stop the processes that are connected to the database. Leave the rest of GitLab
+running:
```shell
-# Stop processes that are connected to the database
sudo gitlab-ctl stop unicorn
sudo gitlab-ctl stop sidekiq
+# Verify
+sudo gitlab-ctl status
+```
+
+Next, restore the backup, specifying the timestamp of the backup you wish to
+restore:
+```shell
# This command will overwrite the contents of your GitLab database!
sudo gitlab-rake gitlab:backup:restore BACKUP=1393513186
+```
-# Start GitLab
-sudo gitlab-ctl start
+Restart and check GitLab:
-# Check GitLab
+```shell
+sudo gitlab-ctl start
sudo gitlab-rake gitlab:check SANITIZE=true
```
If there is a GitLab version mismatch between your backup tar file and the installed
-version of GitLab, the restore command will abort with an error. Install a package for
-the [required version](https://www.gitlab.com/downloads/archives/) and try again.
+version of GitLab, the restore command will abort with an error. Install the
+[correct GitLab version](https://www.gitlab.com/downloads/archives/) and try again.
## Configure cron to make daily backups
diff --git a/doc/release/README.md b/doc/release/README.md
deleted file mode 100644
index 52eca7c02a6..00000000000
--- a/doc/release/README.md
+++ /dev/null
@@ -1,10 +0,0 @@
-## Release cycle
-
-Since 2011 a minor or major version of GitLab is released on the 22nd of every month. Patch and security releases are published when needed. New features are detailed on the [blog](https://about.gitlab.com/blog/) and in the [changelog](CHANGELOG). Features that will likely be in the next releases can be found on the [direction page](https://about.gitlab.com/direction/).
-
-## Release process documentation
-
-- [Monthly release](monthly.md), every month on the 22nd.
-- [Patch release](patch.md), if there are serious regressions.
-- [Security](security.md), for security problems.
-- [Master](master.md), update process for the master branch.
diff --git a/doc/release/howto_rc1.md b/doc/release/howto_rc1.md
deleted file mode 100644
index 07c703142d4..00000000000
--- a/doc/release/howto_rc1.md
+++ /dev/null
@@ -1,55 +0,0 @@
-# How to create RC1
-
-The RC1 release comes with the task to update the installation and upgrade docs. Be mindful that there might already be merge requests for this on GitLab or GitHub.
-
-### 1. Update the installation guide
-
-1. Check if it references the correct branch `x-x-stable` (doesn't exist yet, but that is okay)
-1. Check the [GitLab Shell version](/lib/tasks/gitlab/check.rake#L782)
-1. Check the [Git version](/lib/tasks/gitlab/check.rake#L794)
-1. There might be other changes. Ask around.
-
-### 2. Create update guides
-
-[Follow this guide](howto_update_guides.md) to create update guides.
-
-### 3. Code quality indicators
-
-Make sure the code quality indicators are green / good.
-
-- [![Build status](http://ci.gitlab.org/projects/1/status.png?ref=master)](http://ci.gitlab.org/projects/1?ref=master) on ci.gitlab.org (master branch)
-
-- [![Build Status](https://semaphoreapp.com/api/v1/projects/2f1a5809-418b-4cc2-a1f4-819607579fe7/243338/badge.png)](https://semaphoreapp.com/gitlabhq/gitlabhq) (master branch)
-
-- [![Code Climate](https://codeclimate.com/github/gitlabhq/gitlabhq.png)](https://codeclimate.com/github/gitlabhq/gitlabhq)
-
-- [![Dependency Status](https://gemnasium.com/gitlabhq/gitlabhq.png)](https://gemnasium.com/gitlabhq/gitlabhq) this button can be yellow (small updates are available) but must not be red (a security fix or an important update is available)
-
-- [![Coverage Status](https://coveralls.io/repos/gitlabhq/gitlabhq/badge.png?branch=master)](https://coveralls.io/r/gitlabhq/gitlabhq)
-
-### 4. Run release tool
-
-**Make sure EE `master` has latest changes from CE `master`**
-
-Get release tools
-
-```
-git clone git@dev.gitlab.org:gitlab/release-tools.git
-cd release-tools
-```
-
-Release candidate creates stable branch from master.
-So we need to sync master branch between all CE, EE and CI remotes.
-
-```
-bundle exec rake sync
-```
-
-Create release candidate and stable branch:
-
-```
-bundle exec rake release["x.x.0.rc1"]
-```
-
-Now developers can use master for merging new features.
-So you should use stable branch for future code changes related to release.
diff --git a/doc/release/howto_update_guides.md b/doc/release/howto_update_guides.md
deleted file mode 100644
index 23d0959c33d..00000000000
--- a/doc/release/howto_update_guides.md
+++ /dev/null
@@ -1,55 +0,0 @@
-# Create update guides
-
-1. Create: CE update guide from previous version. Like `7.3-to-7.4.md`
-1. Create: CE to EE update guide in EE repository for latest version.
-1. Update: `6.x-or-7.x-to-7.x.md` to latest version.
-1. Create: CI update guide from previous version
-
-It's best to copy paste the previous guide and make changes where necessary.
-The typical steps are listed below with any points you should specifically look at.
-
-#### 0. Any major changes?
-
-List any major changes here, so the user is aware of them before starting to upgrade. For instance:
-
-- Database updates
-- Web server changes
-- File structure changes
-
-#### 1. Stop server
-
-#### 2. Make backup
-
-#### 3. Do users need to update dependencies like `git`?
-
-- Check if the [GitLab Shell version](/lib/tasks/gitlab/check.rake#L782) changed since the last release.
-
-- Check if the [Git version](/lib/tasks/gitlab/check.rake#L794) changed since the last release.
-
-#### 4. Get latest code
-
-#### 5. Does GitLab shell need to be updated?
-
-#### 6. Install libs, migrations, etc.
-
-#### 7. Any config files updated since last release?
-
-Check if any of these changed since last release:
-
-- [lib/support/nginx/gitlab](/lib/support/nginx/gitlab)
-- [lib/support/nginx/gitlab-ssl](/lib/support/nginx/gitlab-ssl)
-- <https://gitlab.com/gitlab-org/gitlab-shell/commits/master/config.yml.example>
-- [config/gitlab.yml.example](/config/gitlab.yml.example)
-- [config/unicorn.rb.example](/config/unicorn.rb.example)
-- [config/database.yml.mysql](/config/database.yml.mysql)
-- [config/database.yml.postgresql](/config/database.yml.postgresql)
-- [config/initializers/rack_attack.rb.example](/config/initializers/rack_attack.rb.example)
-- [config/resque.yml.example](/config/resque.yml.example)
-
-#### 8. Need to update init script?
-
-Check if the `init.d/gitlab` script changed since last release: [lib/support/init.d/gitlab](/lib/support/init.d/gitlab)
-
-#### 9. Start application
-
-#### 10. Check application status
diff --git a/doc/release/master.md b/doc/release/master.md
deleted file mode 100644
index 9163e652003..00000000000
--- a/doc/release/master.md
+++ /dev/null
@@ -1,62 +0,0 @@
-# How to push GitLab CE master branch to all remotes.
-
-The source code of GitLab is available on multiple servers (with GitLab.com as the canonical source).
-Synchronization between the repo's is done by the lead developer if there is no rush.
-This happens a few times per workday on average.
-If somebody else with access to all repo's wants to do it the instructions are below.
-This is just to distribute changes, not to make them.
-
-## Add this to `.bashrc` or [your dotfiles](https://github.com/dosire/dotfiles/commit/52803ce3ac60d57632164b7713ff0041e86fa26c)
-
-```bash
-gpa ()
-{
- git push origin ${1:-master} && git push gh ${1:-master} && git push gl ${1:-master}
-}
-```
-
-## Then add remotes to your local repo
-
-```bash
-cd my-gitlab-ce-repo
-
-git remote add origin git@dev.gitlab.org:gitlab/gitlabhq.git
-git remote add gh git@github.com:gitlabhq/gitlabhq.git
-git remote add gl git@gitlab.com:gitlab-org/gitlab-ce.git
-```
-
-## Push to all remotes
-
-```bash
-gpa
-```
-
-# Yanking packages from packages.gitlab.com
-
-In case something went wrong with the release and there is a need to remove the packages you can yank the packages by following the
-procedure described in [package cloud documentation](https://packagecloud.io/docs#yank_pkg).
-
-You need to have:
-
-1. `package_cloud` gem installed (sudo gem install package_cloud)
-1. Email and password for packages.gitlab.com
-1. Make sure that you are supplying the url to packages.gitlab.com (default is packagecloud.io)
-
-Example of yanking a package:
-
-```bash
-package_cloud yank --url https://packages.gitlab.com gitlab/gitlab-ce/el/6 gitlab-ce-7.10.2~omnibus-1.x86_64.rpm
-```
-
-If you are attempting this for the first time the output will look something like:
-
-```bash
-Looking for repository at gitlab/gitlab-ce... No config file exists at /Users/marin/.packagecloud. Login to create one.
-Email:
-marin@gitlab.com
-Password:
-
-Got your token. Writing a config file to /Users/marin/.packagecloud... success!
-success!
-Attempting to yank package at gitlab/gitlab-ce/el/6/gitlab-ce-7.10.2~omnibus-1.x86_64.rpm...done!
-```
diff --git a/doc/release/monthly.md b/doc/release/monthly.md
deleted file mode 100644
index 907c19e65a0..00000000000
--- a/doc/release/monthly.md
+++ /dev/null
@@ -1,245 +0,0 @@
-# Monthly Release
-
-NOTE: This is a guide used by the GitLab the company to release GitLab.
-As an end user you do not need to use this guide.
-
-The process starts 7 working days before the release.
-The release manager doesn't have to perform all the work but must ensure someone is assigned.
-The current release manager must schedule the appointment of the next release manager.
-The new release manager should create overall issue to track the progress.
-The release manager should be the only person pushing/merging commits to the x-y-stable branches.
-
-## Release Manager
-
-A release manager is selected that coordinates all releases the coming month,
-including the patch releases for previous releases.
-The release manager has to make sure all the steps below are done and delegated where necessary.
-This person should also make sure this document is kept up to date and issues are created and updated.
-
-## Take vacations into account
-
-The time is measured in weekdays to compensate for weekends.
-Do everything on time to prevent problems due to rush jobs or too little testing time.
-Make sure that you take into account any vacations of maintainers.
-If the release is falling behind immediately warn the team.
-
-## Create an overall issue and follow it
-
-Create an issue in the GitLab CE project. Name it "Release x.x" and tag it with
-the `release` label for easier searching. Replace the dates with actual dates
-based on the number of workdays before the release. All steps from issue
-template are explained below:
-
-```
-### Xth: (7 working days before the 22nd)
-
-- [ ] Triage the [Omnibus milestone]
-
-### Xth: (6 working days before the 22nd)
-
-- [ ] Determine QA person and notify this person
-- [ ] Check the tasks in [how to rc1 guide](https://dev.gitlab.org/gitlab/gitlabhq/blob/master/doc/release/howto_rc1.md) and delegate tasks if necessary
-- [ ] Merge CE `master` into EE `master` via merge request (#LINK)
-- [ ] Create CE and EE RC1 versions (#LINK)
-- [ ] Build RC1 packages
-
-### Xth: (5 working days before the 22nd)
-
-- [ ] Do QA and fix anything coming out of it (#LINK)
-- [ ] Close the [Omnibus milestone]
-- [ ] Prepare the [blog post]
-
-### Xth: (4 working days before the 22nd)
-
-- [ ] Update GitLab.com with RC1
-- [ ] Create the regression issue in the CE issue tracker:
-
- ```
- This is a meta issue to index possible regressions in this monthly release
- and any patch versions.
-
- Please do not raise or discuss issues directly in this issue but link to
- issues that might warrant a patch release. If there is a Merge Request
- that fixes the issue, please link to that as well.
-
- Please only post one regression issue and/or merge request per comment.
- Comments will be updated by the release manager as they are addressed.
- ```
-
-- [ ] Tweet about RC1 release:
-
- ```
- GitLab x.y.0.rc1 is available: https://packages.gitlab.com/gitlab/unstable
- Use at your own risk. Please link regressions issues from
- LINK_TO_REGRESSION_ISSUE
- ```
-
-### Xth: (3 working days before the 22nd)
-
-- [ ] Merge `x-y-stable` into `x-y-stable-ee`
-- [ ] Check that everyone is mentioned on the [blog post] using `@all`
-
-### Xth: (2 working days before the 22nd)
-
-- [ ] Check that MVP is added to the [MVP page]
-
-### Xth: (1 working day before the 22nd)
-
-- [ ] Merge `x-y-stable` into `x-y-stable-ee`
-- [ ] Create CE and EE release candidates
-- [ ] Create Omnibus tags and build packages for the latest release candidates
-- [ ] Update GitLab.com with the latest RC
-
-### 22nd before 1200 CET:
-
-Release before 1200 CET / 2AM PST, to make sure the majority of our users
-get the new version on the 22nd and there is sufficient time in the European
-workday to quickly fix any issues.
-
-- [ ] Merge `x-y-stable` into `x-y-stable-ee`
-- [ ] Create the 'x.y.0' tag with the [release tools](https://dev.gitlab.org/gitlab/release-tools)
-- [ ] Create the 'x.y.0' version on version.gitlab.com
-- [ ] Try to do before 1100 CET: Create and push Omnibus tags for x.y.0 (will auto-release the packages)
-- [ ] Try to do before 1200 CET: Publish the release [blog post]
-- [ ] Tweet about the release
-- [ ] Schedule a second Tweet of the release announcement with the same text at 1800 CET / 8AM PST
-
-[Omnibus milestone]: LINK_TO_OMNIBUS_MILESTONE
-[blog post]: LINK_TO_WIP_BLOG_POST
-[MVP page]: https://gitlab.com/gitlab-com/www-gitlab-com/blob/master/source/mvp/index.html
-```
-
-- - -
-
-## Update changelog
-
-Any changes not yet added to the changelog are added by lead developer and in that merge request the complete team is
-asked if there is anything missing.
-
-There are three changelogs that need to be updated: CE, EE and CI.
-
-## Create RC1 (CE, EE, CI)
-
-[Follow this How-to guide](howto_rc1.md) to create RC1.
-
-## Prepare CHANGELOG for next release
-
-Once the stable branches have been created, update the CHANGELOG in `master` with the upcoming version, usually X.X.X.pre.
-
-On creating the stable branches, notify the core team and developers.
-
-## QA
-
-Create issue on dev.gitlab.org `gitlab` repository, named "GitLab X.X QA" in order to keep track of the progress.
-
-Use the omnibus packages created for RC1 of Enterprise Edition using [this guide](https://dev.gitlab.org/gitlab/gitlab-ee/blob/master/doc/release/manual_testing.md).
-
-**NOTE** Upgrader can only be tested when tags are pushed to all repositories. Do not forget to confirm it is working before releasing. Note that in the issue.
-
-#### Fix anything coming out of the QA
-
-Create an issue with description of a problem, if it is quick fix fix it yourself otherwise contact the team for advice.
-
-**NOTE** If there is a problem that cannot be fixed in a timely manner, reverting the feature is an option! If the feature is reverted,
-create an issue about it in order to discuss the next steps after the release.
-
-## Update GitLab.com with RC1
-
-Use the omnibus EE packages created for RC1.
-If there are big database migrations consider testing them with the production db on a VM.
-Try to deploy in the morning.
-It is important to do this as soon as possible, so we can catch any errors before we release the full version.
-
-## Create a regressions issue
-
-On [the GitLab CE issue tracker on GitLab.com](https://gitlab.com/gitlab-org/gitlab-ce/issues/) create an issue titled "GitLab X.X regressions" add the following text:
-
-This is a meta issue to discuss possible regressions in this monthly release and any patch versions.
-Please do not raise issues directly in this issue but link to issues that might warrant a patch release.
-The decision to create a patch release or not is with the release manager who is assigned to this issue.
-The release manager will comment here about the plans for patch releases.
-
-Assign the issue to the release manager and at mention all members of GitLab core team. If there are any known bugs in the release add them immediately.
-
-## Tweet about RC1
-
-Tweet about the RC release:
-
-> GitLab x.x.0.rc1 is out. This release candidate is only suitable for testing. Please link regressions issues from LINK_TO_REGRESSION_ISSUE
-
-## Prepare the blog post
-
-1. The blog post template for this release should already exist and might have comments that were added during the month.
-1. Fill out as much of the blog post template as you can.
-1. Make sure the blog post contains information about the GitLab CI release.
-1. Check the changelog of CE and EE for important changes.
-1. Also check the CI changelog
-1. Add a proposed tweet text to the blog post WIP MR description.
-1. Create a WIP MR for the blog post
-1. Make sure merge request title starts with `WIP` so it can not be accidentally merged until ready.
-1. Ask Dmitriy (or a team member with OS X) to add screenshots to the WIP MR.
-1. Decide with core team who will be the MVP user.
-1. Create WIP MR for adding MVP to MVP page on website
-1. Add a note if there are security fixes: This release fixes an important security issue and we advise everyone to upgrade as soon as possible.
-1. Create a merge request on [GitLab.com](https://gitlab.com/gitlab-com/www-gitlab-com/tree/master)
-1. Assign to one reviewer who will fix spelling issues by editing the branch (either with a git client or by using the online editor)
-1. Comment to the reviewer: '@person Please mention the whole team as soon as you are done (3 workdays before release at the latest)'
-1. Create a new merge request with complete copy of the [release blog template](https://gitlab.com/gitlab-com/www-gitlab-com/blob/master/doc/release_blog_template.md) for the next release using the branch name `release-x-x-x`.
-
-## Create CE, EE, CI stable versions
-
-Get release tools
-
-```
-git clone git@dev.gitlab.org:gitlab/release-tools.git
-cd release-tools
-```
-
-Bump version, create release tag and push to remotes:
-
-```
-bundle exec rake release["x.x.0"]
-```
-
-This will create correct version and tag and push to all CE, EE and CI remotes.
-
-Update [installation.md](/doc/install/installation.md) to the newest version in master.
-
-
-## Create Omnibus tags and build packages
-
-Follow the [release doc in the Omnibus repository](https://gitlab.com/gitlab-org/omnibus-gitlab/blob/master/doc/release.md).
-This can happen before tagging because Omnibus uses tags in its own repo and SHA1's to refer to the GitLab codebase.
-
-## Update GitLab.com with the stable version
-
-- Deploy the package (should not need downtime because of the small difference with RC1)
-- Deploy the package for gitlab.com/ci
-
-## Release CE, EE and CI
-
-__1. Publish packages for new release__
-
-Update `downloads/index.html` and `downloads/archive/index.html` in `www-gitlab-com` repository.
-
-__2. Publish blog for new release__
-
-Doublecheck the everyone has been mentioned in the blog post.
-Merge the [blog merge request](#1-prepare-the-blog-post) in `www-gitlab-com` repository.
-
-__3. Tweet to blog__
-
-Send out a tweet to share the good news with the world.
-List the most important features and link to the blog post.
-
-Proposed tweet "Release of GitLab X.X & CI Y.Y! FEATURE, FEATURE and FEATURE &lt;link-to-blog-post&gt; #gitlab"
-
-Consider creating a post on Hacker News.
-
-## Release new AMIs
-
-[Follow this guide](https://dev.gitlab.org/gitlab/AMI/blob/master/README.md)
-
-## Create a WIP blogpost for the next release
-
-Create a WIP blogpost using [release blog template](https://gitlab.com/gitlab-com/www-gitlab-com/blob/master/doc/release_blog_template.md).
diff --git a/doc/release/patch.md b/doc/release/patch.md
deleted file mode 100644
index 1c921439156..00000000000
--- a/doc/release/patch.md
+++ /dev/null
@@ -1,81 +0,0 @@
-# Things to do when doing a patch release
-
-NOTE: This is a guide for GitLab developers. If you are trying to install GitLab
-see the latest stable [installation guide](install/installation.md) and if you
-are trying to upgrade, see the [upgrade guides](update).
-
-## When to do a patch release
-
-Patch releases are done as-needed in order to fix regressions in the current
-major release that cannot or should not wait until the next major release.
-What's included and when to release is at the discretion of the release manager.
-
-## Release Procedure
-
-### Create a patch issue
-
-Create an issue in the GitLab CE project. Name it "Release x.y.z", tag it with
-the `release` label, and assign it to the milestone of the corresponding major
-release.
-
-Use the following template:
-
-```
-- Picked into respective `stable` branches:
-- [ ] Merge `x-y-stable` into `x-y-stable-ee`
-- [ ] release-tools: `x.y.z`
-- omnibus-gitlab
- - [ ] `x.y.z+ee.0`
- - [ ] `x.y.z+ce.0`
-- [ ] Deploy
-- [ ] Add patch notice to [x.y regressions]()
-- [ ] [Blog post]()
-- [ ] [Tweet]()
-- [ ] Add entry to version.gitlab.com
-```
-
-Update the issue with links to merge requests that need to be/have been picked
-into the `stable` branches.
-
-### Preparation
-
-1. Verify that the issue can be reproduced
-1. Note in the 'GitLab X.X regressions' that you will create a patch
-1. Fix the issue on a feature branch, do this on the private GitLab development server
-1. If it is a security issue, then assign it to the release manager and apply a 'security' label
-1. Consider creating and testing workarounds
-1. After the branch is merged into master, cherry pick the commit(s) into the current stable branch
-1. Make sure that the build has passed and all tests are passing
-1. In a separate commit in the master branch update the CHANGELOG
-1. For EE, update the CHANGELOG-EE if it is EE specific fix. Otherwise, merge the stable CE branch and add to CHANGELOG-EE "Merge community edition changes for version X.X.X"
-1. Merge CE stable branch into EE stable branch
-
-### Bump version
-
-Get release tools
-
-```
-git clone git@dev.gitlab.org:gitlab/release-tools.git
-cd release-tools
-```
-
-Bump all versions in stable branch, even if the changes affect only EE, CE, or CI. Since all the versions are synced now,
-it doesn't make sense to say upgrade CE to 7.2, EE to 7.3 and CI to 7.1.
-
-Create release tag and push to remotes:
-
-```
-bundle exec rake release["x.x.x"]
-```
-
-## Release
-
-1. [Build new packages with the latest version](https://gitlab.com/gitlab-org/omnibus-gitlab/blob/master/doc/release.md)
-1. Apply the patch to GitLab.com and the private GitLab development server
-1. Apply the patch to ci.gitLab.com and the private GitLab CI development server
-1. Create and publish a blog post, see [patch release blog template](https://gitlab.com/gitlab-com/www-gitlab-com/blob/master/doc/patch_release_blog_template.md)
-1. Send tweets about the release from `@gitlab`, tweet should include the most important feature that the release is addressing and link to the blog post
-1. Note in the 'GitLab X.X regressions' issue that the patch was published (CE only)
-1. Create the 'x.y.0' version on version.gitlab.com
-1. [Create new AMIs](https://dev.gitlab.org/gitlab/AMI/blob/master/README.md)
-1. Create a new patch release issue for the next potential release
diff --git a/doc/release/security.md b/doc/release/security.md
deleted file mode 100644
index 118c016ba4f..00000000000
--- a/doc/release/security.md
+++ /dev/null
@@ -1,76 +0,0 @@
-# Things to do when doing an out-of-bound security release
-
-NOTE: This is a guide for GitLab developers. If you are trying to install GitLab see the latest stable [installation guide](install/installation.md) and if you are trying to upgrade, see the [upgrade guides](update).
-
-## When to do a security release
-
-Do a security release when there is a critical issue that needs to be addresses before the next monthly release. Otherwise include it in the monthly release and note there was a security fix in the release announcement.
-
-## Security vulnerability disclosure
-
-Please report suspected security vulnerabilities in private to <support@gitlab.com>, also see the [disclosure section on the GitLab.com website](https://about.gitlab.com/disclosure/). Please do NOT create publicly viewable issues for suspected security vulnerabilities.
-
-## Release Procedure
-
-1. Verify that the issue can be reproduced
-1. Acknowledge the issue to the researcher that disclosed it
-1. Inform the release manager that there needs to be a security release
-1. Do the steps from [patch release document](../release/patch.md), starting with "Create an issue on private GitLab development server"
-1. The MR with the security fix should get a 'security' label and be assigned to the release manager
-1. Build the package for GitLab.com and do a deploy
-1. Build the package for ci.gitLab.com and do a deploy
-1. [Create new AMIs](https://dev.gitlab.org/gitlab/AMI/blob/master/README.md)
-1. Create feature branches for the blog post on GitLab.com and link them from the code branch
-1. Merge and publish the blog posts
-1. Send tweets about the release from `@gitlabhq`
-1. Send out an email to [the community google mailing list](https://groups.google.com/forum/#!forum/gitlabhq)
-1. Post a signed copy of our complete announcement to [oss-security](http://www.openwall.com/lists/oss-security/) and request a CVE number. CVE is only needed for bugs that allow someone to own the server (Remote Code Execution) or access to code of projects they are not a member of.
-1. Add the security researcher to the [Security Researcher Acknowledgments list](https://about.gitlab.com/vulnerability-acknowledgements/)
-1. Thank the security researcher in an email for their cooperation
-1. Update the blog post and the CHANGELOG when we receive the CVE number
-
-The timing of the code merge into master should be coordinated in advance.
-
-After the merge we strive to publish the announcements within 60 minutes.
-
-## Blog post template
-
-XXX Security Advisory for GitLab
-
-A recently discovered critical vulnerability in GitLab allows [unauthenticated API access|remote code execution|unauthorized access to repositories|XXX|PICKSOMETHING]. All users should update GitLab and gitlab-shell immediately. We [have|haven't|XXX|PICKSOMETHING|] heard of this vulnerability being actively exploited.
-
-### Version affected
-
-GitLab Community Edition XXX and lower
-
-GitLab Enterprise Edition XXX and lower
-
-### Fixed versions
-
-GitLab Community Edition XXX and up
-
-GitLab Enterprise Edition XXX and up
-
-### Impact
-
-On GitLab installations which use MySQL as their database backend it is possible for an attacker to assume the identity of any existing GitLab user in certain API calls. This attack can be performed by [unauthenticated|authenticated|XXX|PICKSOMETHING] users.
-
-### Workarounds
-
-If you are unable to upgrade you should apply the following patch and restart GitLab.
-
-XXX
-
-### Credit
-
-We want to thank XXX of XXX for the responsible disclosure of this vulnerability.
-
-## Email template
-
-We just announced a security advisory for GitLab at XXX
-
-Please contact us at support@gitlab.com if you have any questions.
-
-## Tweet template
-
-We just announced a security advisory for GitLab at XXX
diff --git a/doc/security/README.md b/doc/security/README.md
index 4cd0fdd4094..38706e48ec5 100644
--- a/doc/security/README.md
+++ b/doc/security/README.md
@@ -8,3 +8,4 @@
- [User File Uploads](user_file_uploads.md)
- [How we manage the CRIME vulnerability](crime_vulnerability.md)
- [Enforce Two-factor authentication](two_factor_authentication.md)
+- [Send email confirmation on sign-up](user_email_confirmation.md)
diff --git a/doc/security/user_email_confirmation.md b/doc/security/user_email_confirmation.md
new file mode 100644
index 00000000000..4293944ae8b
--- /dev/null
+++ b/doc/security/user_email_confirmation.md
@@ -0,0 +1,7 @@
+# User email confirmation at sign-up
+
+Gitlab admin can enable email confirmation on sign-up, if you want to confirm all
+user emails before they are able to sign-in.
+
+In the Admin area under **Settings** (`/admin/application_settings`), go to section
+**Sign-in Restrictions** and look for **Send confirmation email on sign-up** option.
diff --git a/doc/system_hooks/system_hooks.md b/doc/system_hooks/system_hooks.md
index 612376e3a49..c44930a4ceb 100644
--- a/doc/system_hooks/system_hooks.md
+++ b/doc/system_hooks/system_hooks.md
@@ -4,6 +4,12 @@ Your GitLab instance can perform HTTP POST requests on the following events: `pr
System hooks can be used, e.g. for logging or changing information in a LDAP server.
+> **Note:**
+>
+> We follow the same structure from Webhooks for Push and Tag events, but we never display commits.
+>
+> Same deprecations from Webhooks are valid here.
+
## Hooks request example
**Request header**:
@@ -240,3 +246,110 @@ X-Gitlab-Event: System Hook
"user_id": 41
}
```
+
+## Push events
+
+Triggered when you push to the repository except when pushing tags.
+
+**Request header**:
+
+```
+X-Gitlab-Event: System Hook
+```
+
+**Request body:**
+
+```json
+{
+ "event_name": "push",
+ "before": "95790bf891e76fee5e1747ab589903a6a1f80f22",
+ "after": "da1560886d4f094c3e6c9ef40349f7d38b5d27d7",
+ "ref": "refs/heads/master",
+ "checkout_sha": "da1560886d4f094c3e6c9ef40349f7d38b5d27d7",
+ "user_id": 4,
+ "user_name": "John Smith",
+ "user_email": "john@example.com",
+ "user_avatar": "https://s.gravatar.com/avatar/d4c74594d841139328695756648b6bd6?s=8://s.gravatar.com/avatar/d4c74594d841139328695756648b6bd6?s=80",
+ "project_id": 15,
+ "project":{
+ "name":"Diaspora",
+ "description":"",
+ "web_url":"http://example.com/mike/diaspora",
+ "avatar_url":null,
+ "git_ssh_url":"git@example.com:mike/diaspora.git",
+ "git_http_url":"http://example.com/mike/diaspora.git",
+ "namespace":"Mike",
+ "visibility_level":0,
+ "path_with_namespace":"mike/diaspora",
+ "default_branch":"master",
+ "homepage":"http://example.com/mike/diaspora",
+ "url":"git@example.com:mike/diaspora.git",
+ "ssh_url":"git@example.com:mike/diaspora.git",
+ "http_url":"http://example.com/mike/diaspora.git"
+ },
+ "repository":{
+ "name": "Diaspora",
+ "url": "git@example.com:mike/diaspora.git",
+ "description": "",
+ "homepage": "http://example.com/mike/diaspora",
+ "git_http_url":"http://example.com/mike/diaspora.git",
+ "git_ssh_url":"git@example.com:mike/diaspora.git",
+ "visibility_level":0
+ },
+ "commits": [],
+ "total_commits_count": 0
+}
+```
+
+## Tag events
+
+Triggered when you create (or delete) tags to the repository.
+
+**Request header**:
+
+```
+X-Gitlab-Event: System Hook
+```
+
+**Request body:**
+
+```json
+{
+ "event_name": "tag_push",
+ "before": "0000000000000000000000000000000000000000",
+ "after": "82b3d5ae55f7080f1e6022629cdb57bfae7cccc7",
+ "ref": "refs/tags/v1.0.0",
+ "checkout_sha": "5937ac0a7beb003549fc5fd26fc247adbce4a52e",
+ "user_id": 1,
+ "user_name": "John Smith",
+ "user_avatar": "https://s.gravatar.com/avatar/d4c74594d841139328695756648b6bd6?s=8://s.gravatar.com/avatar/d4c74594d841139328695756648b6bd6?s=80",
+ "project_id": 1,
+ "project":{
+ "name":"Example",
+ "description":"",
+ "web_url":"http://example.com/jsmith/example",
+ "avatar_url":null,
+ "git_ssh_url":"git@example.com:jsmith/example.git",
+ "git_http_url":"http://example.com/jsmith/example.git",
+ "namespace":"Jsmith",
+ "visibility_level":0,
+ "path_with_namespace":"jsmith/example",
+ "default_branch":"master",
+ "homepage":"http://example.com/jsmith/example",
+ "url":"git@example.com:jsmith/example.git",
+ "ssh_url":"git@example.com:jsmith/example.git",
+ "http_url":"http://example.com/jsmith/example.git"
+ },
+ "repository":{
+ "name": "Example",
+ "url": "ssh://git@example.com/jsmith/example.git",
+ "description": "",
+ "homepage": "http://example.com/jsmith/example",
+ "git_http_url":"http://example.com/jsmith/example.git",
+ "git_ssh_url":"git@example.com:jsmith/example.git",
+ "visibility_level":0
+ },
+ "commits": [],
+ "total_commits_count": 0
+}
+```
diff --git a/doc/update/8.2-to-8.3.md b/doc/update/8.2-to-8.3.md
index 2ca4e1f3770..9f5c6c4dc84 100644
--- a/doc/update/8.2-to-8.3.md
+++ b/doc/update/8.2-to-8.3.md
@@ -1,5 +1,14 @@
# From 8.2 to 8.3
+Make sure you view this update guide from the tag (version) of GitLab you would
+like to install. In most cases this should be the highest numbered production
+tag (without rc in it). You can select the tag in the version dropdown at the
+top left corner of GitLab (below the menu bar).
+
+If the highest number stable branch is unclear please check the
+[GitLab Blog](https://about.gitlab.com/blog/archives.html) for installation
+guide links by version.
+
**NOTE:** GitLab 8.0 introduced several significant changes related to
installation and configuration which *are not duplicated here*. Be sure you're
already running a working version of at least 8.0 before proceeding with this
diff --git a/doc/update/8.3-to-8.4.md b/doc/update/8.3-to-8.4.md
index 269deec7a9c..9f6517d9487 100644
--- a/doc/update/8.3-to-8.4.md
+++ b/doc/update/8.3-to-8.4.md
@@ -1,5 +1,14 @@
# From 8.3 to 8.4
+Make sure you view this update guide from the tag (version) of GitLab you would
+like to install. In most cases this should be the highest numbered production
+tag (without rc in it). You can select the tag in the version dropdown at the
+top left corner of GitLab (below the menu bar).
+
+If the highest number stable branch is unclear please check the
+[GitLab Blog](https://about.gitlab.com/blog/archives.html) for installation
+guide links by version.
+
### 1. Stop server
sudo service gitlab stop
diff --git a/doc/update/8.4-to-8.5.md b/doc/update/8.4-to-8.5.md
index 0a9cb5683e7..0cb137a03cc 100644
--- a/doc/update/8.4-to-8.5.md
+++ b/doc/update/8.4-to-8.5.md
@@ -1,5 +1,14 @@
# From 8.4 to 8.5
+Make sure you view this update guide from the tag (version) of GitLab you would
+like to install. In most cases this should be the highest numbered production
+tag (without rc in it). You can select the tag in the version dropdown at the
+top left corner of GitLab (below the menu bar).
+
+If the highest number stable branch is unclear please check the
+[GitLab Blog](https://about.gitlab.com/blog/archives.html) for installation
+guide links by version.
+
### 1. Stop server
sudo service gitlab stop
diff --git a/doc/update/8.5-to-8.6.md b/doc/update/8.5-to-8.6.md
index 024f6e8a433..6267f14eba4 100644
--- a/doc/update/8.5-to-8.6.md
+++ b/doc/update/8.5-to-8.6.md
@@ -1,5 +1,14 @@
# From 8.5 to 8.6
+Make sure you view this update guide from the tag (version) of GitLab you would
+like to install. In most cases this should be the highest numbered production
+tag (without rc in it). You can select the tag in the version dropdown at the
+top left corner of GitLab (below the menu bar).
+
+If the highest number stable branch is unclear please check the
+[GitLab Blog](https://about.gitlab.com/blog/archives.html) for installation
+guide links by version.
+
### 1. Stop server
sudo service gitlab stop
@@ -37,7 +46,7 @@ sudo -u git -H git checkout 8-6-stable-ee
```bash
cd /home/git/gitlab-shell
sudo -u git -H git fetch --all
-sudo -u git -H git checkout v2.6.11
+sudo -u git -H git checkout v2.6.12
```
### 5. Update gitlab-workhorse
@@ -49,11 +58,30 @@ GitLab 8.1.
```bash
cd /home/git/gitlab-workhorse
sudo -u git -H git fetch --all
-sudo -u git -H git checkout 0.6.5
+sudo -u git -H git checkout v0.7.1
sudo -u git -H make
```
-### 6. Install libs, migrations, etc.
+### 6. Updates for PostgreSQL Users
+
+Starting with 8.6 users using GitLab in combination with PostgreSQL are required
+to have the `pg_trgm` extension enabled for all GitLab databases. If you're
+using GitLab's Omnibus packages there's nothing you'll need to do manually as
+this extension is enabled automatically. Users who install GitLab without using
+Omnibus (e.g. by building from source) have to enable this extension manually.
+To enable this extension run the following SQL command as a PostgreSQL super
+user for _every_ GitLab database:
+
+```sql
+CREATE EXTENSION IF NOT EXISTS pg_trgm;
+```
+
+Certain operating systems might require the installation of extra packages for
+this extension to be available. For example, users using Ubuntu will have to
+install the `postgresql-contrib` package in order for this extension to be
+available.
+
+### 7. Install libs, migrations, etc.
```bash
cd /home/git/gitlab
@@ -75,7 +103,7 @@ sudo -u git -H bundle exec rake assets:clean assets:precompile cache:clear RAILS
```
-### 7. Update configuration files
+### 8. Update configuration files
#### New configuration options for `gitlab.yml`
@@ -111,25 +139,6 @@ Ensure you're still up-to-date with the latest init script changes:
sudo cp lib/support/init.d/gitlab /etc/init.d/gitlab
-### 8. Updates for PostgreSQL Users
-
-Starting with 8.6 users using GitLab in combination with PostgreSQL are required
-to have the `pg_trgm` extension enabled for all GitLab databases. If you're
-using GitLab's Omnibus packages there's nothing you'll need to do manually as
-this extension is enabled automatically. Users who install GitLab without using
-Omnibus (e.g. by building from source) have to enable this extension manually.
-To enable this extension run the following SQL command as a PostgreSQL super
-user for _every_ GitLab database:
-
-```sql
-CREATE EXTENSION IF NOT EXISTS pg_trgm;
-```
-
-Certain operating systems might require the installation of extra packages for
-this extension to be available. For example, users using Ubuntu will have to
-install the `postgresql-contrib` package in order for this extension to be
-available.
-
### 9. Start application
sudo service gitlab start
diff --git a/doc/update/8.6-to-8.7.md b/doc/update/8.6-to-8.7.md
new file mode 100644
index 00000000000..bb463d43a7c
--- /dev/null
+++ b/doc/update/8.6-to-8.7.md
@@ -0,0 +1,162 @@
+# From 8.6 to 8.7
+
+Make sure you view this update guide from the tag (version) of GitLab you would
+like to install. In most cases this should be the highest numbered production
+tag (without rc in it). You can select the tag in the version dropdown at the
+top left corner of GitLab (below the menu bar).
+
+If the highest number stable branch is unclear please check the
+[GitLab Blog](https://about.gitlab.com/blog/archives.html) for installation
+guide links by version.
+
+### 1. Stop server
+
+ sudo service gitlab stop
+
+### 2. Backup
+
+```bash
+cd /home/git/gitlab
+sudo -u git -H bundle exec rake gitlab:backup:create RAILS_ENV=production
+```
+
+### 3. Get latest code
+
+```bash
+sudo -u git -H git fetch --all
+sudo -u git -H git checkout -- db/schema.rb # local changes will be restored automatically
+```
+
+For GitLab Community Edition:
+
+```bash
+sudo -u git -H git checkout 8-7-stable
+```
+
+OR
+
+For GitLab Enterprise Edition:
+
+```bash
+sudo -u git -H git checkout 8-7-stable-ee
+```
+
+### 4. Update gitlab-shell
+
+```bash
+cd /home/git/gitlab-shell
+sudo -u git -H git fetch --all --tags
+sudo -u git -H git checkout v2.7.2
+```
+
+### 5. Update gitlab-workhorse
+
+Install and compile gitlab-workhorse. This requires
+[Go 1.5](https://golang.org/dl) which should already be on your system from
+GitLab 8.1.
+
+```bash
+cd /home/git/gitlab-workhorse
+sudo -u git -H git fetch --all
+sudo -u git -H git checkout v0.7.1
+sudo -u git -H make
+```
+
+### 6. Install libs, migrations, etc.
+
+```bash
+cd /home/git/gitlab
+
+# MySQL installations (note: the line below states '--without postgres')
+sudo -u git -H bundle install --without postgres development test --deployment
+
+# PostgreSQL installations (note: the line below states '--without mysql')
+sudo -u git -H bundle install --without mysql development test --deployment
+
+# Optional: clean up old gems
+sudo -u git -H bundle clean
+
+# Run database migrations
+sudo -u git -H bundle exec rake db:migrate RAILS_ENV=production
+
+# Clean up assets and cache
+sudo -u git -H bundle exec rake assets:clean assets:precompile cache:clear RAILS_ENV=production
+
+```
+
+### 7. Update configuration files
+
+#### New configuration options for `gitlab.yml`
+
+There are new configuration options available for [`gitlab.yml`](config/gitlab.yml.example). View them with the command below and apply them manually to your current `gitlab.yml`:
+
+```sh
+git diff origin/8-6-stable:config/gitlab.yml.example origin/8-7-stable:config/gitlab.yml.example
+```
+
+#### Git configuration
+
+Disable `git gc --auto` because GitLab runs `git gc` for us already.
+
+```sh
+sudo -u git -H git config --global gc.auto 0
+```
+
+#### Nginx configuration
+
+Ensure you're still up-to-date with the latest NGINX configuration changes:
+
+```sh
+# For HTTPS configurations
+git diff origin/8-6-stable:lib/support/nginx/gitlab-ssl origin/8-7-stable:lib/support/nginx/gitlab-ssl
+
+# For HTTP configurations
+git diff origin/8-6-stable:lib/support/nginx/gitlab origin/8-7-stable:lib/support/nginx/gitlab
+```
+
+If you are using Apache instead of NGINX please see the updated [Apache templates].
+Also note that because Apache does not support upstreams behind Unix sockets you
+will need to let gitlab-workhorse listen on a TCP port. You can do this
+via [/etc/default/gitlab].
+
+[Apache templates]: https://gitlab.com/gitlab-org/gitlab-recipes/tree/master/web-server/apache
+[/etc/default/gitlab]: https://gitlab.com/gitlab-org/gitlab-ce/blob/8-7-stable/lib/support/init.d/gitlab.default.example#L37
+
+#### Init script
+
+Ensure you're still up-to-date with the latest init script changes:
+
+ sudo cp lib/support/init.d/gitlab /etc/init.d/gitlab
+
+### 8. Start application
+
+ sudo service gitlab start
+ sudo service nginx restart
+
+### 9. Check application status
+
+Check if GitLab and its environment are configured correctly:
+
+ sudo -u git -H bundle exec rake gitlab:env:info RAILS_ENV=production
+
+To make sure you didn't miss anything run a more thorough check:
+
+ sudo -u git -H bundle exec rake gitlab:check RAILS_ENV=production
+
+If all items are green, then congratulations, the upgrade is complete!
+
+## Things went south? Revert to previous version (8.6)
+
+### 1. Revert the code to the previous version
+
+Follow the [upgrade guide from 8.5 to 8.6](8.5-to-8.6.md), except for the
+database migration (the backup is already migrated to the previous version).
+
+### 2. Restore from the backup
+
+```bash
+cd /home/git/gitlab
+sudo -u git -H bundle exec rake gitlab:backup:restore RAILS_ENV=production
+```
+
+If you have more than one backup `*.tar` file(s) please add `BACKUP=timestamp_of_backup` to the command above.
diff --git a/doc/update/8.7-to-8.8.md b/doc/update/8.7-to-8.8.md
new file mode 100644
index 00000000000..32906650f6f
--- /dev/null
+++ b/doc/update/8.7-to-8.8.md
@@ -0,0 +1,162 @@
+# From 8.7 to 8.8
+
+Make sure you view this update guide from the tag (version) of GitLab you would
+like to install. In most cases this should be the highest numbered production
+tag (without rc in it). You can select the tag in the version dropdown at the
+top left corner of GitLab (below the menu bar).
+
+If the highest number stable branch is unclear please check the
+[GitLab Blog](https://about.gitlab.com/blog/archives.html) for installation
+guide links by version.
+
+### 1. Stop server
+
+ sudo service gitlab stop
+
+### 2. Backup
+
+```bash
+cd /home/git/gitlab
+sudo -u git -H bundle exec rake gitlab:backup:create RAILS_ENV=production
+```
+
+### 3. Get latest code
+
+```bash
+sudo -u git -H git fetch --all
+sudo -u git -H git checkout -- db/schema.rb # local changes will be restored automatically
+```
+
+For GitLab Community Edition:
+
+```bash
+sudo -u git -H git checkout 8-8-stable
+```
+
+OR
+
+For GitLab Enterprise Edition:
+
+```bash
+sudo -u git -H git checkout 8-8-stable-ee
+```
+
+### 4. Update gitlab-shell
+
+```bash
+cd /home/git/gitlab-shell
+sudo -u git -H git fetch --all --tags
+sudo -u git -H git checkout v2.7.2
+```
+
+### 5. Update gitlab-workhorse
+
+Install and compile gitlab-workhorse. This requires
+[Go 1.5](https://golang.org/dl) which should already be on your system from
+GitLab 8.1.
+
+```bash
+cd /home/git/gitlab-workhorse
+sudo -u git -H git fetch --all
+sudo -u git -H git checkout v0.7.1
+sudo -u git -H make
+```
+
+### 6. Install libs, migrations, etc.
+
+```bash
+cd /home/git/gitlab
+
+# MySQL installations (note: the line below states '--without postgres')
+sudo -u git -H bundle install --without postgres development test --deployment
+
+# PostgreSQL installations (note: the line below states '--without mysql')
+sudo -u git -H bundle install --without mysql development test --deployment
+
+# Optional: clean up old gems
+sudo -u git -H bundle clean
+
+# Run database migrations
+sudo -u git -H bundle exec rake db:migrate RAILS_ENV=production
+
+# Clean up assets and cache
+sudo -u git -H bundle exec rake assets:clean assets:precompile cache:clear RAILS_ENV=production
+
+```
+
+### 7. Update configuration files
+
+#### New configuration options for `gitlab.yml`
+
+There are new configuration options available for [`gitlab.yml`](config/gitlab.yml.example). View them with the command below and apply them manually to your current `gitlab.yml`:
+
+```sh
+git diff origin/8-7-stable:config/gitlab.yml.example origin/8-8-stable:config/gitlab.yml.example
+```
+
+#### Git configuration
+
+Disable `git gc --auto` because GitLab runs `git gc` for us already.
+
+```sh
+sudo -u git -H git config --global gc.auto 0
+```
+
+#### Nginx configuration
+
+Ensure you're still up-to-date with the latest NGINX configuration changes:
+
+```sh
+# For HTTPS configurations
+git diff origin/8-7-stable:lib/support/nginx/gitlab-ssl origin/8-8-stable:lib/support/nginx/gitlab-ssl
+
+# For HTTP configurations
+git diff origin/8-7-stable:lib/support/nginx/gitlab origin/8-8-stable:lib/support/nginx/gitlab
+```
+
+If you are using Apache instead of NGINX please see the updated [Apache templates].
+Also note that because Apache does not support upstreams behind Unix sockets you
+will need to let gitlab-workhorse listen on a TCP port. You can do this
+via [/etc/default/gitlab].
+
+[Apache templates]: https://gitlab.com/gitlab-org/gitlab-recipes/tree/master/web-server/apache
+[/etc/default/gitlab]: https://gitlab.com/gitlab-org/gitlab-ce/blob/8-8-stable/lib/support/init.d/gitlab.default.example#L37
+
+#### Init script
+
+Ensure you're still up-to-date with the latest init script changes:
+
+ sudo cp lib/support/init.d/gitlab /etc/init.d/gitlab
+
+### 8. Start application
+
+ sudo service gitlab start
+ sudo service nginx restart
+
+### 9. Check application status
+
+Check if GitLab and its environment are configured correctly:
+
+ sudo -u git -H bundle exec rake gitlab:env:info RAILS_ENV=production
+
+To make sure you didn't miss anything run a more thorough check:
+
+ sudo -u git -H bundle exec rake gitlab:check RAILS_ENV=production
+
+If all items are green, then congratulations, the upgrade is complete!
+
+## Things went south? Revert to previous version (8.7)
+
+### 1. Revert the code to the previous version
+
+Follow the [upgrade guide from 8.6 to 8.7](8.6-to-8.7.md), except for the
+database migration (the backup is already migrated to the previous version).
+
+### 2. Restore from the backup
+
+```bash
+cd /home/git/gitlab
+sudo -u git -H bundle exec rake gitlab:backup:restore RAILS_ENV=production
+```
+
+If you have more than one backup `*.tar` file(s) please add `BACKUP=timestamp_of_backup` to the command above.
diff --git a/doc/update/8.8-to-8.9.md b/doc/update/8.8-to-8.9.md
new file mode 100644
index 00000000000..f14046bb4be
--- /dev/null
+++ b/doc/update/8.8-to-8.9.md
@@ -0,0 +1,162 @@
+# From 8.8 to 8.9
+
+Make sure you view this update guide from the tag (version) of GitLab you would
+like to install. In most cases this should be the highest numbered production
+tag (without rc in it). You can select the tag in the version dropdown at the
+top left corner of GitLab (below the menu bar).
+
+If the highest number stable branch is unclear please check the
+[GitLab Blog](https://about.gitlab.com/blog/archives.html) for installation
+guide links by version.
+
+### 1. Stop server
+
+ sudo service gitlab stop
+
+### 2. Backup
+
+```bash
+cd /home/git/gitlab
+sudo -u git -H bundle exec rake gitlab:backup:create RAILS_ENV=production
+```
+
+### 3. Get latest code
+
+```bash
+sudo -u git -H git fetch --all
+sudo -u git -H git checkout -- db/schema.rb # local changes will be restored automatically
+```
+
+For GitLab Community Edition:
+
+```bash
+sudo -u git -H git checkout 8-9-stable
+```
+
+OR
+
+For GitLab Enterprise Edition:
+
+```bash
+sudo -u git -H git checkout 8-9-stable-ee
+```
+
+### 4. Update gitlab-shell
+
+```bash
+cd /home/git/gitlab-shell
+sudo -u git -H git fetch --all --tags
+sudo -u git -H git checkout v3.0.0
+```
+
+### 5. Update gitlab-workhorse
+
+Install and compile gitlab-workhorse. This requires
+[Go 1.5](https://golang.org/dl) which should already be on your system from
+GitLab 8.1.
+
+```bash
+cd /home/git/gitlab-workhorse
+sudo -u git -H git fetch --all
+sudo -u git -H git checkout v0.7.5
+sudo -u git -H make
+```
+
+### 6. Install libs, migrations, etc.
+
+```bash
+cd /home/git/gitlab
+
+# MySQL installations (note: the line below states '--without postgres')
+sudo -u git -H bundle install --without postgres development test --deployment
+
+# PostgreSQL installations (note: the line below states '--without mysql')
+sudo -u git -H bundle install --without mysql development test --deployment
+
+# Optional: clean up old gems
+sudo -u git -H bundle clean
+
+# Run database migrations
+sudo -u git -H bundle exec rake db:migrate RAILS_ENV=production
+
+# Clean up assets and cache
+sudo -u git -H bundle exec rake assets:clean assets:precompile cache:clear RAILS_ENV=production
+
+```
+
+### 7. Update configuration files
+
+#### New configuration options for `gitlab.yml`
+
+There are new configuration options available for [`gitlab.yml`](config/gitlab.yml.example). View them with the command below and apply them manually to your current `gitlab.yml`:
+
+```sh
+git diff origin/8-8-stable:config/gitlab.yml.example origin/8-9-stable:config/gitlab.yml.example
+```
+
+#### Git configuration
+
+Disable `git gc --auto` because GitLab runs `git gc` for us already.
+
+```sh
+sudo -u git -H git config --global gc.auto 0
+```
+
+#### Nginx configuration
+
+Ensure you're still up-to-date with the latest NGINX configuration changes:
+
+```sh
+# For HTTPS configurations
+git diff origin/8-8-stable:lib/support/nginx/gitlab-ssl origin/8-9-stable:lib/support/nginx/gitlab-ssl
+
+# For HTTP configurations
+git diff origin/8-8-stable:lib/support/nginx/gitlab origin/8-9-stable:lib/support/nginx/gitlab
+```
+
+If you are using Apache instead of NGINX please see the updated [Apache templates].
+Also note that because Apache does not support upstreams behind Unix sockets you
+will need to let gitlab-workhorse listen on a TCP port. You can do this
+via [/etc/default/gitlab].
+
+[Apache templates]: https://gitlab.com/gitlab-org/gitlab-recipes/tree/master/web-server/apache
+[/etc/default/gitlab]: https://gitlab.com/gitlab-org/gitlab-ce/blob/8-9-stable/lib/support/init.d/gitlab.default.example#L37
+
+#### Init script
+
+Ensure you're still up-to-date with the latest init script changes:
+
+ sudo cp lib/support/init.d/gitlab /etc/init.d/gitlab
+
+### 8. Start application
+
+ sudo service gitlab start
+ sudo service nginx restart
+
+### 9. Check application status
+
+Check if GitLab and its environment are configured correctly:
+
+ sudo -u git -H bundle exec rake gitlab:env:info RAILS_ENV=production
+
+To make sure you didn't miss anything run a more thorough check:
+
+ sudo -u git -H bundle exec rake gitlab:check RAILS_ENV=production
+
+If all items are green, then congratulations, the upgrade is complete!
+
+## Things went south? Revert to previous version (8.8)
+
+### 1. Revert the code to the previous version
+
+Follow the [upgrade guide from 8.7 to 8.8](8.7-to-8.8.md), except for the
+database migration (the backup is already migrated to the previous version).
+
+### 2. Restore from the backup
+
+```bash
+cd /home/git/gitlab
+sudo -u git -H bundle exec rake gitlab:backup:restore RAILS_ENV=production
+```
+
+If you have more than one backup `*.tar` file(s) please add `BACKUP=timestamp_of_backup` to the command above.
diff --git a/doc/update/README.md b/doc/update/README.md
index 109d5de3fa2..975d72164b4 100644
--- a/doc/update/README.md
+++ b/doc/update/README.md
@@ -1,17 +1,95 @@
-Depending on the installation method and your GitLab version, there are multiple update guides. Choose one that fits your needs.
+# Updating GitLab
+
+Depending on the installation method and your GitLab version, there are multiple
+update guides.
+
+There are currently 3 official ways to install GitLab:
+
+- Omnibus packages
+- Source installation
+- Docker installation
+
+Based on your installation, choose a section below that fits your needs.
+
+---
+
+<!-- START doctoc generated TOC please keep comment here to allow auto update -->
+<!-- DON'T EDIT THIS SECTION, INSTEAD RE-RUN doctoc TO UPDATE -->
+**Table of Contents** *generated with [DocToc](https://github.com/thlorenz/doctoc)*
+
+- [Omnibus Packages](#omnibus-packages)
+- [Installation from source](#installation-from-source)
+- [Installation using Docker](#installation-using-docker)
+- [Upgrading between editions](#upgrading-between-editions)
+ - [Community to Enterprise Edition](#community-to-enterprise-edition)
+ - [Enterprise to Community Edition](#enterprise-to-community-edition)
+- [Miscellaneous](#miscellaneous)
+
+<!-- END doctoc generated TOC please keep comment here to allow auto update -->
## Omnibus Packages
-- [Omnibus update guide](https://gitlab.com/gitlab-org/omnibus-gitlab/blob/master/doc/update.md) contains the steps needed to update a GitLab [package](https://about.gitlab.com/downloads/).
+- The [Omnibus update guide](http://docs.gitlab.com/omnibus/update/README.html)
+ contains the steps needed to update an Omnibus GitLab package.
## Installation from source
-- [The individual upgrade guides](https://gitlab.com/gitlab-org/gitlab-ce/tree/master/doc/update) are for those who have installed GitLab from source.
-- [The CE to EE update guides](https://gitlab.com/subscribers/gitlab-ee/tree/master/doc/update) are for subscribers of the Enterprise Edition only. The steps are very similar to a version upgrade: stop the server, get the code, update config files for the new functionality, install libs and do migrations, update the init script, start the application and check the application status.
-- [Upgrader](upgrader.md) is an automatic ruby script that performs the update for installations from source.
-- [Patch versions](patch_versions.md) guide includes the steps needed for a patch version, eg. 6.2.0 to 6.2.1.
+- [Upgrading Community Edition from source][source-ce] - The individual
+ upgrade guides are for those who have installed GitLab CE from source.
+- [Upgrading Enterprise Edition from source][source-ee] - The individual
+ upgrade guides are for those who have installed GitLab EE from source.
+- [Patch versions](patch_versions.md) guide includes the steps needed for a
+ patch version, eg. 6.2.0 to 6.2.1, and apply to both Community and Enterprise
+ Editions.
+
+## Installation using Docker
+
+GitLab provides official Docker images for both Community and Enterprise
+editions. They are based on the Omnibus package and instructions on how to
+update them are in [a separate document][omnidocker].
+
+## Upgrading between editions
+
+GitLab comes in two flavors: [Community Edition][ce] which is MIT licensed,
+and [Enterprise Edition][ee] which builds on top of the Community Edition and
+includes extra features mainly aimed at organizations with more than 100 users.
+
+Below you can find some guides to help you change editions easily.
+
+### Community to Enterprise Edition
+
+>**Note:**
+The following guides are for subscribers of the Enterprise Edition only.
+
+If you wish to upgrade your GitLab installation from Community to Enterprise
+Edition, follow the guides below based on the installation method:
+
+- [Source CE to EE update guides][source-ee] - Find your version, and follow the
+ `-ce-to-ee.md` guide. The steps are very similar to a version upgrade: stop
+ the server, get the code, update config files for the new functionality,
+ install libraries and do migrations, update the init script, start the
+ application and check its status.
+- [Omnibus CE to EE][omni-ce-ee] - Follow this guide to update your Omnibus
+ GitLab Community Edition to the Enterprise Edition.
+
+### Enterprise to Community Edition
+
+If you need to downgrade your Enterprise Edition installation back to Community
+Edition, you can follow [this guide][ee-ce] to make the process as smooth as
+possible.
## Miscellaneous
-- [MySQL to PostgreSQL](mysql_to_postgresql.md) guides you through migrating your database from MySQL to PostgreSQL.
-- [MySQL installation guide](https://gitlab.com/gitlab-org/gitlab-ce/blob/master/doc/install/database_mysql.md) contains additional information about configuring GitLab to work with a MySQL database.
+- [MySQL to PostgreSQL](mysql_to_postgresql.md) guides you through migrating
+ your database from MySQL to PostgreSQL.
+- [MySQL installation guide](../install/database_mysql.md) contains additional
+ information about configuring GitLab to work with a MySQL database.
+- [Restoring from backup after a failed upgrade](restore_after_failure.md)
+
+[omnidocker]: http://docs.gitlab.com/omnibus/docker/README.html
+[source-ee]: https://gitlab.com/gitlab-org/gitlab-ee/tree/master/doc/update
+[source-ce]: https://gitlab.com/gitlab-org/gitlab-ce/tree/master/doc/update
+[ee-ce]: ../downgrade_ee_to_ce/README.md
+[ce]: https://about.gitlab.com/features/#community
+[ee]: https://about.gitlab.com/features/#enterprise
+[omni-ce-ee]: http://docs.gitlab.com/omnibus/update/README.html#from-community-edition-to-enterprise-edition
diff --git a/doc/update/patch_versions.md b/doc/update/patch_versions.md
index f446ed0a35b..60729316cde 100644
--- a/doc/update/patch_versions.md
+++ b/doc/update/patch_versions.md
@@ -47,7 +47,7 @@ sudo -u git -H git checkout v`cat /home/git/gitlab/GITLAB_SHELL_VERSION` -b v`ca
```bash
cd /home/git/gitlab-workhorse
sudo -u git -H git fetch
-sudo -u git -H git checkout `cat /home/git/gitlab/GITLAB_WORKHORSE_VERSION` -b `cat /home/git/gitlab/GITLAB_WORKHORSE_VERSION`
+sudo -u git -H git checkout v`cat /home/git/gitlab/GITLAB_WORKHORSE_VERSION` -b v`cat /home/git/gitlab/GITLAB_WORKHORSE_VERSION`
sudo -u git -H make
```
diff --git a/doc/update/restore_after_failure.md b/doc/update/restore_after_failure.md
new file mode 100644
index 00000000000..01c52aae7f5
--- /dev/null
+++ b/doc/update/restore_after_failure.md
@@ -0,0 +1,83 @@
+# Restoring from backup after a failed upgrade
+
+Upgrades are usually smooth and restoring from backup is a rare occurrence.
+However, it's important to know how to recover when problems do arise.
+
+## Roll back to an earlier version and restore a backup
+
+In some cases after a failed upgrade, the fastest solution is to roll back to
+the previous version you were using.
+
+First, roll back the code or package. For source installations this involves
+checking out the older version (branch or tag). For Omnibus installations this
+means installing the older .deb or .rpm package. Then, restore from a backup.
+Follow the instructions in the
+[Backup and Restore](../raketasks/backup_restore.md#restore-a-previously-created-backup)
+documentation.
+
+## Potential problems on the next upgrade
+
+When a rollback is necessary it can produce problems on subsequent upgrade
+attempts. This is because some tables may have been added during the failed
+upgrade. If these tables are still present after you restore from the
+older backup it can lead to migration failures on future upgrades.
+
+Starting in GitLab 8.6 we drop all tables prior to importing the backup to
+prevent this problem. If you've restored a backup to a version prior to 8.6 you
+may need to manually correct the problem next time you upgrade.
+
+Example error:
+
+```
+== 20151103134857 CreateLfsObjects: migrating =================================
+-- create_table(:lfs_objects)
+rake aborted!
+StandardError: An error has occurred, this and all later migrations canceled:
+
+PG::DuplicateTable: ERROR: relation "lfs_objects" already exists
+```
+
+Copy the version from the error. In this case the version number is
+`20151103134857`.
+
+>**WARNING:** Use the following steps only if you are certain this is what you
+need to do.
+
+### GitLab 8.6+
+
+Pass the version to a database rake task to manually mark the migration as
+complete.
+
+```
+# Source install
+sudo -u git -H bundle exec rake gitlab:db:mark_migration_complete[20151103134857] RAILS_ENV=production
+
+# Omnibus install
+sudo gitlab-rake gitlab:db:mark_migration_complete[20151103134857]
+```
+
+Once the migration is successfully marked, run the rake `db:migrate` task again.
+You will likely have to repeat this process several times until all failed
+migrations are marked complete.
+
+### GitLab < 8.6
+
+```
+# Source install
+sudo -u git -H bundle exec rails console production
+
+# Omnibus install
+sudo gitlab-rails console
+```
+
+At the Rails console, type the following commands:
+
+```
+ActiveRecord::Base.connection.execute("INSERT INTO schema_migrations (version) VALUES('20151103134857')")
+exit
+```
+
+Once the migration is successfully marked, run the rake `db:migrate` task again.
+You will likely have to repeat this process several times until all failed
+migrations are marked complete.
+
diff --git a/doc/web_hooks/web_hooks.md b/doc/web_hooks/web_hooks.md
index afdf1a682e2..8559b67af04 100644
--- a/doc/web_hooks/web_hooks.md
+++ b/doc/web_hooks/web_hooks.md
@@ -13,6 +13,19 @@ You can configure webhooks to listen for specific events like pushes, issues or
Webhooks can be used to update an external issue tracker, trigger CI builds, update a backup mirror, or even deploy to your production server.
+## Webhook endpoint tips
+
+If you are writing your own endpoint (web server) that will receive
+GitLab webhooks keep in mind the following things:
+
+- Your endpoint should send its HTTP response as fast as possible. If
+ you wait too long, GitLab may decide the hook failed and retry it.
+- Your endpoint should ALWAYS return a valid HTTP response. If you do
+ not do this then GitLab will think the hook failed and retry it.
+ Most HTTP libraries take care of this for you automatically but if
+ you are writing a low-level hook this is important to remember.
+- GitLab ignores the HTTP status code returned by your endpoint.
+
## SSL Verification
By default, the SSL certificate of the webhook endpoint is verified based on
@@ -41,6 +54,7 @@ X-Gitlab-Event: Push Hook
"before": "95790bf891e76fee5e1747ab589903a6a1f80f22",
"after": "da1560886d4f094c3e6c9ef40349f7d38b5d27d7",
"ref": "refs/heads/master",
+ "checkout_sha": "da1560886d4f094c3e6c9ef40349f7d38b5d27d7",
"user_id": 4,
"user_name": "John Smith",
"user_email": "john@example.com",
@@ -58,13 +72,13 @@ X-Gitlab-Event: Push Hook
"path_with_namespace":"mike/diaspora",
"default_branch":"master",
"homepage":"http://example.com/mike/diaspora",
- "url":"git@example.com:mike/diasporadiaspora.git",
+ "url":"git@example.com:mike/diaspora.git",
"ssh_url":"git@example.com:mike/diaspora.git",
"http_url":"http://example.com/mike/diaspora.git"
},
"repository":{
"name": "Diaspora",
- "url": "git@example.com:mike/diasporadiaspora.git",
+ "url": "git@example.com:mike/diaspora.git",
"description": "",
"homepage": "http://example.com/mike/diaspora",
"git_http_url":"http://example.com/mike/diaspora.git",
@@ -113,15 +127,15 @@ Triggered when you create (or delete) tags to the repository.
X-Gitlab-Event: Tag Push Hook
```
-
**Request body:**
```json
{
"object_kind": "tag_push",
- "ref": "refs/tags/v1.0.0",
"before": "0000000000000000000000000000000000000000",
"after": "82b3d5ae55f7080f1e6022629cdb57bfae7cccc7",
+ "ref": "refs/tags/v1.0.0",
+ "checkout_sha": "82b3d5ae55f7080f1e6022629cdb57bfae7cccc7",
"user_id": 1,
"user_name": "John Smith",
"user_avatar": "https://s.gravatar.com/avatar/d4c74594d841139328695756648b6bd6?s=8://s.gravatar.com/avatar/d4c74594d841139328695756648b6bd6?s=80",
@@ -143,7 +157,7 @@ X-Gitlab-Event: Tag Push Hook
"http_url":"http://example.com/jsmith/example.git"
},
"repository":{
- "name": "jsmith",
+ "name": "Example",
"url": "ssh://git@example.com/jsmith/example.git",
"description": "",
"homepage": "http://example.com/jsmith/example",
@@ -478,7 +492,7 @@ X-Gitlab-Event: Note Hook
},
"repository":{
"name":"diaspora",
- "url":"git@example.com:mike/diasporadiaspora.git",
+ "url":"git@example.com:mike/diaspora.git",
"description":"",
"homepage":"http://example.com/mike/diaspora"
},
@@ -681,6 +695,61 @@ X-Gitlab-Event: Merge Request Hook
}
```
+## Wiki Page events
+
+Triggered when a wiki page is created or edited.
+
+**Request Header**:
+
+```
+X-Gitlab-Event: Wiki Page Hook
+```
+
+**Request Body**:
+
+```json
+{
+ "object_kind": "wiki_page",
+ "user": {
+ "name": "Administrator",
+ "username": "root",
+ "avatar_url": "http://www.gravatar.com/avatar/e64c7d89f26bd1972efa854d13d7dd61?s=80\u0026d=identicon"
+ },
+ "project": {
+ "name": "awesome-project",
+ "description": "This is awesome",
+ "web_url": "http://example.com/root/awesome-project",
+ "avatar_url": null,
+ "git_ssh_url": "git@example.com:root/awesome-project.git",
+ "git_http_url": "http://example.com/root/awesome-project.git",
+ "namespace": "root",
+ "visibility_level": 0,
+ "path_with_namespace": "root/awesome-project",
+ "default_branch": "master",
+ "homepage": "http://example.com/root/awesome-project",
+ "url": "git@example.com:root/awesome-project.git",
+ "ssh_url": "git@example.com:root/awesome-project.git",
+ "http_url": "http://example.com/root/awesome-project.git"
+ },
+ "wiki": {
+ "web_url": "http://example.com/root/awesome-project/wikis/home",
+ "git_ssh_url": "git@example.com:root/awesome-project.wiki.git",
+ "git_http_url": "http://example.com/root/awesome-project.wiki.git",
+ "path_with_namespace": "root/awesome-project.wiki",
+ "default_branch": "master"
+ },
+ "object_attributes": {
+ "title": "Awesome",
+ "content": "awesome content goes here",
+ "format": "markdown",
+ "message": "adding an awesome page to the wiki",
+ "slug": "awesome",
+ "url": "http://example.com/root/awesome-project/wikis/awesome",
+ "action": "create"
+ }
+}
+```
+
#### Example webhook receiver
If you want to see GitLab's webhooks in action for testing purposes you can use
diff --git a/doc/workflow/README.md b/doc/workflow/README.md
index 25893f948ea..9efe41308dc 100644
--- a/doc/workflow/README.md
+++ b/doc/workflow/README.md
@@ -20,6 +20,7 @@
- [Milestones](milestones.md)
- [Merge Requests](merge_requests.md)
- [Revert changes](revert_changes.md)
+- [Cherry-pick changes](cherry_pick_changes.md)
- ["Work In Progress" Merge Requests](wip_merge_requests.md)
- [Merge When Build Succeeds](merge_when_build_succeeds.md)
- [Manage large binaries with Git LFS](lfs/manage_large_binaries_with_git_lfs.md)
diff --git a/doc/workflow/award_emoji.md b/doc/workflow/award_emoji.md
new file mode 100644
index 00000000000..70b35c58be6
--- /dev/null
+++ b/doc/workflow/award_emoji.md
@@ -0,0 +1,48 @@
+# Award emojis
+
+>**Note:**
+This feature was [introduced][1825] in GitLab 8.2.
+
+When you're collaborating online, you get fewer opportunities for high-fives
+and thumbs-ups. In order to make virtual celebrations easier, you can now vote
+on issues and merge requests using emoji!
+
+![Award emoji](img/award_emoji_select.png)
+
+This makes it much easier to give and receive feedback, without a long comment
+thread. Any comment that contains only the thumbs up or down emojis is
+converted to a vote and depicted in the emoji area.
+
+You can then use that functionality to sort issues and merge requests based on
+popularity.
+
+## Sort issues and merge requests on vote count
+
+>**Note:**
+This feature was [introduced][2871] in GitLab 8.5.
+
+You can quickly sort the issues or merge requests by the number of votes they
+have received. The sort option can be found in the right dropdown menu.
+
+![Votes sort options](img/award_emoji_votes_sort_options.png)
+
+---
+
+Sort by most popular issues/merge requests.
+
+![Votes sort by most popular](img/award_emoji_votes_most_popular.png)
+
+---
+
+Sort by least popular issues/merge requests.
+
+![Votes sort by least popular](img/award_emoji_votes_least_popular.png)
+
+---
+
+The number of upvotes and downvotes is not summed up. That means that an issue
+with 18 upvotes and 5 downvotes is considered more popular than an issue with
+17 upvotes and no downvotes.
+
+[2871]: https://gitlab.com/gitlab-org/gitlab-ce/merge_requests/2781
+[1825]: https://gitlab.com/gitlab-org/gitlab-ce/merge_requests/1825
diff --git a/doc/workflow/cherry_pick_changes.md b/doc/workflow/cherry_pick_changes.md
new file mode 100644
index 00000000000..4a499009842
--- /dev/null
+++ b/doc/workflow/cherry_pick_changes.md
@@ -0,0 +1,53 @@
+# Cherry-pick changes
+
+>**Note:**
+This feature was [introduced][ce-3514] in GitLab 8.7.
+
+---
+
+GitLab implements Git's powerful feature to [cherry-pick any commit][git-cherry-pick]
+with introducing a **Cherry-pick** button in Merge Requests and commit details.
+
+## Cherry-picking a Merge Request
+
+After the Merge Request has been merged, a **Cherry-pick** button will be available
+to cherry-pick the changes introduced by that Merge Request:
+
+![Cherry-pick Merge Request](img/cherry_pick_changes_mr.png)
+
+---
+
+You can cherry-pick the changes directly into the selected branch or you can opt to
+create a new Merge Request with the cherry-pick changes:
+
+![Cherry-pick Merge Request modal](img/cherry_pick_changes_mr_modal.png)
+
+## Cherry-picking a Commit
+
+You can cherry-pick a Commit from the Commit details page:
+
+![Cherry-pick commit](img/cherry_pick_changes_commit.png)
+
+---
+
+Similar to cherry-picking a Merge Request, you can opt to cherry-pick the changes
+directly into the target branch or create a new Merge Request to cherry-pick the
+changes:
+
+![Cherry-pick commit modal](img/cherry_pick_changes_commit_modal.png)
+
+---
+
+Please note that when cherry-picking merge commits, the mainline will always be the
+first parent. If you want to use a different mainline then you need to do that
+from the command line.
+
+Here is a quick example to cherry-pick a merge commit using the second parent as the
+mainline:
+
+```bash
+git cherry-pick -m 2 7a39eb0
+```
+
+[ce-3514]: https://gitlab.com/gitlab-org/gitlab-ce/merge_requests/3514 "Cherry-pick button Merge Request"
+[git-cherry-pick]: https://git-scm.com/docs/git-cherry-pick "Git cherry-pick documentation"
diff --git a/doc/workflow/gitlab_flow.md b/doc/workflow/gitlab_flow.md
index 1b354bcc0f1..2b2f140f8bf 100644
--- a/doc/workflow/gitlab_flow.md
+++ b/doc/workflow/gitlab_flow.md
@@ -131,7 +131,7 @@ When you feel comfortable with it to be merged you assign it to the person that
There is room for more feedback and after the assigned person feels comfortable with the result the branch is merged.
If the assigned person does not feel comfortable they can close the merge request without merging.
-In GitLab it is common to protect the long-lived branches (e.g. the master branch) so that normal developers [can't modify these protected branches](http://doc.gitlab.com/ce/permissions/permissions.html).
+In GitLab it is common to protect the long-lived branches (e.g. the master branch) so that normal developers [can't modify these protected branches](http://docs.gitlab.com/ce/permissions/permissions.html).
So if you want to merge it into a protected branch you assign it to someone with master authorizations.
## Issues with GitLab flow
@@ -187,7 +187,7 @@ If you have an issue that spans across multiple repositories, the best thing is
![Vim screen showing the rebase view](rebase.png)
With git you can use an interactive rebase (`rebase -i`) to squash multiple commits into one and reorder them.
-In GitLab EE and .com you can also [rebase before merge](http://doc.gitlab.com/ee/workflow/rebase_before_merge.html) from the web interface.
+In GitLab EE and .com you can also [rebase before merge](http://docs.gitlab.com/ee/workflow/rebase_before_merge.html) from the web interface.
This functionality is useful if you made a couple of commits for small changes during development and want to replace them with a single commit or if you want to make the order more logical.
However you should never rebase commits you have pushed to a remote server.
Somebody can have referred to the commits or cherry-picked them.
diff --git a/doc/workflow/groups.md b/doc/workflow/groups.md
index 52bf611dc5e..34ada1774d8 100644
--- a/doc/workflow/groups.md
+++ b/doc/workflow/groups.md
@@ -54,7 +54,7 @@ If necessary, you can increase the access level of an individual user for a spec
## Managing group memberships via LDAP
In GitLab Enterprise Edition it is possible to manage GitLab group memberships using LDAP groups.
-See [the GitLab Enterprise Edition documentation](http://doc.gitlab.com/ee/integration/ldap.html) for more information.
+See [the GitLab Enterprise Edition documentation](http://docs.gitlab.com/ee/integration/ldap.html) for more information.
## Allowing only admins to create groups
diff --git a/doc/workflow/img/award_emoji_select.png b/doc/workflow/img/award_emoji_select.png
new file mode 100644
index 00000000000..fffdfedda5d
--- /dev/null
+++ b/doc/workflow/img/award_emoji_select.png
Binary files differ
diff --git a/doc/workflow/img/award_emoji_votes_least_popular.png b/doc/workflow/img/award_emoji_votes_least_popular.png
new file mode 100644
index 00000000000..2ef5be7154f
--- /dev/null
+++ b/doc/workflow/img/award_emoji_votes_least_popular.png
Binary files differ
diff --git a/doc/workflow/img/award_emoji_votes_most_popular.png b/doc/workflow/img/award_emoji_votes_most_popular.png
new file mode 100644
index 00000000000..5b089730d93
--- /dev/null
+++ b/doc/workflow/img/award_emoji_votes_most_popular.png
Binary files differ
diff --git a/doc/workflow/img/award_emoji_votes_sort_options.png b/doc/workflow/img/award_emoji_votes_sort_options.png
new file mode 100644
index 00000000000..9bbf3f82a0b
--- /dev/null
+++ b/doc/workflow/img/award_emoji_votes_sort_options.png
Binary files differ
diff --git a/doc/workflow/img/cherry_pick_changes_commit.png b/doc/workflow/img/cherry_pick_changes_commit.png
new file mode 100644
index 00000000000..ae91d2cae53
--- /dev/null
+++ b/doc/workflow/img/cherry_pick_changes_commit.png
Binary files differ
diff --git a/doc/workflow/img/cherry_pick_changes_commit_modal.png b/doc/workflow/img/cherry_pick_changes_commit_modal.png
new file mode 100644
index 00000000000..f502f87677a
--- /dev/null
+++ b/doc/workflow/img/cherry_pick_changes_commit_modal.png
Binary files differ
diff --git a/doc/workflow/img/cherry_pick_changes_mr.png b/doc/workflow/img/cherry_pick_changes_mr.png
new file mode 100644
index 00000000000..59c610e620b
--- /dev/null
+++ b/doc/workflow/img/cherry_pick_changes_mr.png
Binary files differ
diff --git a/doc/workflow/img/cherry_pick_changes_mr_modal.png b/doc/workflow/img/cherry_pick_changes_mr_modal.png
new file mode 100644
index 00000000000..96a80f4726d
--- /dev/null
+++ b/doc/workflow/img/cherry_pick_changes_mr_modal.png
Binary files differ
diff --git a/doc/workflow/img/new_branch_from_issue.png b/doc/workflow/img/new_branch_from_issue.png
new file mode 100644
index 00000000000..394c139e17e
--- /dev/null
+++ b/doc/workflow/img/new_branch_from_issue.png
Binary files differ
diff --git a/doc/workflow/importing/import_projects_from_github.md b/doc/workflow/importing/import_projects_from_github.md
index f693f430a42..a7dfac2c120 100644
--- a/doc/workflow/importing/import_projects_from_github.md
+++ b/doc/workflow/importing/import_projects_from_github.md
@@ -1,7 +1,8 @@
# Import your project from GitHub to GitLab
-_**Note:** In order to enable the GitHub import setting, you should first
-enable the [GitHub integration][gh-import] in your GitLab instance._
+>**Note:**
+In order to enable the GitHub import setting, you should first
+enable the [GitHub integration][gh-import] in your GitLab instance.
At its current state, GitHub importer can import:
@@ -10,10 +11,13 @@ At its current state, GitHub importer can import:
- the issues (introduced in GitLab 7.7)
- the pull requests (introduced in GitLab 8.4)
- the wiki pages (introduced in GitLab 8.4)
+- the milestones (introduced in GitLab 8.7)
+- the labels (introduced in GitLab 8.7)
-It is not yet possible to import your labels, milestones and cross-repository
-pull requests (those from forks). We are working on improving this in the near
-future.
+With GitLab 8.7+, references to pull requests and issues are preserved.
+
+It is not yet possible to import your cross-repository pull requests (those from
+forks). We are working on improving this in the near future.
The importer page is visible when you [create a new project][new-project].
Click on the **GitHub** link and you will be redirected to GitHub for
@@ -40,5 +44,5 @@ case the namespace is taken, the project will be imported on the user's
namespace.
[gh-import]: ../../integration/github.md "GitHub integration"
-[ee-gh]: http://doc.gitlab.com/ee/integration/github.html "GitHub integration for GitLab EE"
+[ee-gh]: http://docs.gitlab.com/ee/integration/github.html "GitHub integration for GitLab EE"
[new-project]: ../../gitlab-basics/create-project.md "How to create a new project in GitLab"
diff --git a/doc/workflow/importing/import_projects_from_gitlab_com.md b/doc/workflow/importing/import_projects_from_gitlab_com.md
index 1117db98e7e..dcc00074b75 100644
--- a/doc/workflow/importing/import_projects_from_gitlab_com.md
+++ b/doc/workflow/importing/import_projects_from_gitlab_com.md
@@ -2,7 +2,7 @@
You can import your existing GitLab.com projects to your GitLab instance. But keep in mind that it is possible only if
GitLab support is enabled on your GitLab instance.
-You can read more about GitLab support [here](http://doc.gitlab.com/ce/integration/gitlab.html)
+You can read more about GitLab support [here](http://docs.gitlab.com/ce/integration/gitlab.html)
To get to the importer page you need to go to "New project" page.
![New project page](gitlab_importer/new_project_page.png)
diff --git a/doc/workflow/lfs/lfs_administration.md b/doc/workflow/lfs/lfs_administration.md
index 36cb9da2380..9dc1e9b47e3 100644
--- a/doc/workflow/lfs/lfs_administration.md
+++ b/doc/workflow/lfs/lfs_administration.md
@@ -23,6 +23,10 @@ In `/etc/gitlab/gitlab.rb`:
```ruby
gitlab_rails['lfs_enabled'] = false
+
+# Optionally, change the storage path location. Defaults to
+# `#{gitlab_rails['shared_path']}/lfs-objects`. Which evaluates to
+# `/var/opt/gitlab/gitlab-rails/shared/lfs-objects` by default.
gitlab_rails['lfs_storage_path'] = "/mnt/storage/lfs-objects"
```
diff --git a/doc/workflow/lfs/manage_large_binaries_with_git_lfs.md b/doc/workflow/lfs/manage_large_binaries_with_git_lfs.md
index ba91685a20b..9fe065fa680 100644
--- a/doc/workflow/lfs/manage_large_binaries_with_git_lfs.md
+++ b/doc/workflow/lfs/manage_large_binaries_with_git_lfs.md
@@ -4,7 +4,7 @@ Managing large files such as audio, video and graphics files has always been one
of the shortcomings of Git. The general recommendation is to not have Git repositories
larger than 1GB to preserve performance.
-GitLab already supports [managing large files with git annex](http://doc.gitlab.com/ee/workflow/git_annex.html)
+GitLab already supports [managing large files with git annex](http://docs.gitlab.com/ee/workflow/git_annex.html)
(EE only), however in certain environments it is not always convenient to use
different commands to differentiate between the large files and regular ones.
@@ -44,7 +44,7 @@ check it into your Git repository:
```bash
git clone git@gitlab.example.com:group/project.git
-git lfs init # initialize the Git LFS project project
+git lfs install # initialize the Git LFS project project
git lfs track "*.iso" # select the file extensions that you want to treat as large files
```
@@ -127,7 +127,7 @@ To prevent this from happening, set the lfs url in project Git config:
```bash
-git config --add lfs.url "http://gitlab.example.com/group/project.git/info/lfs/objects/batch"
+git config --add lfs.url "http://gitlab.example.com/group/project.git/info/lfs"
```
### Credentials are always required when pushing an object
@@ -152,4 +152,4 @@ If you are using OS X you can use `osxkeychain` to store and encrypt your creden
For Windows, you can use `wincred` or Microsoft's [Git Credential Manager for Windows](https://github.com/Microsoft/Git-Credential-Manager-for-Windows/releases).
More details about various methods of storing the user credentials can be found
-on [Git Credential Storage documentation](https://git-scm.com/book/en/v2/Git-Tools-Credential-Storage). \ No newline at end of file
+on [Git Credential Storage documentation](https://git-scm.com/book/en/v2/Git-Tools-Credential-Storage).
diff --git a/doc/workflow/merge_requests.md b/doc/workflow/merge_requests.md
index 6d57b5d98cd..d2ec56e6504 100644
--- a/doc/workflow/merge_requests.md
+++ b/doc/workflow/merge_requests.md
@@ -2,6 +2,17 @@
Merge requests allow you to exchange changes you made to source code
+## Only allow merge requests to be merged if the build succeeds
+
+You can prevent merge requests from being merged if their build did not succeed
+in the project settings page.
+
+![only_allow_merge_if_build_succeeds](merge_requests/only_allow_merge_if_build_succeeds.png)
+
+Navigate to project settings page and select the `Only allow merge requests to be merged if the build succeeds` check box.
+
+Please note that you need to have builds configured to enable this feature.
+
## Checkout merge requests locally
Locate the section for your GitLab remote in the `.git/config` file. It looks like this:
@@ -12,9 +23,9 @@ Locate the section for your GitLab remote in the `.git/config` file. It looks li
fetch = +refs/heads/*:refs/remotes/origin/*
```
-Now add the line `fetch = +refs/merge-requests/*/head:refs/remotes/origin/merge-requests/*` to this section.
+Now add the line `fetch = +refs/merge-requests/*/head:refs/remotes/origin/merge-requests/*` to this section.
-It should looks like this:
+It should look like this:
```
[remote "origin"]
@@ -43,7 +54,7 @@ $ git checkout origin/merge-requests/1
![MR diff](merge_requests/merge_request_diff.png)
-It you add `w=1` option to URL, you can see diff without whitespace changes.
+If you click the "Hide whitespace changes" button, you can see the diff without whitespace changes.
![MR diff without whitespace](merge_requests/merge_request_diff_without_whitespace.png)
diff --git a/doc/workflow/merge_requests/commit_compare.png b/doc/workflow/merge_requests/commit_compare.png
index 46b3a56a59b..dfd7ee220f0 100644
--- a/doc/workflow/merge_requests/commit_compare.png
+++ b/doc/workflow/merge_requests/commit_compare.png
Binary files differ
diff --git a/doc/workflow/merge_requests/merge_request_diff.png b/doc/workflow/merge_requests/merge_request_diff.png
index ed08ae91bec..f368423c746 100644
--- a/doc/workflow/merge_requests/merge_request_diff.png
+++ b/doc/workflow/merge_requests/merge_request_diff.png
Binary files differ
diff --git a/doc/workflow/merge_requests/merge_request_diff_without_whitespace.png b/doc/workflow/merge_requests/merge_request_diff_without_whitespace.png
index 67d67a64d12..b2d03bb66f9 100644
--- a/doc/workflow/merge_requests/merge_request_diff_without_whitespace.png
+++ b/doc/workflow/merge_requests/merge_request_diff_without_whitespace.png
Binary files differ
diff --git a/doc/workflow/merge_requests/only_allow_merge_if_build_succeeds.png b/doc/workflow/merge_requests/only_allow_merge_if_build_succeeds.png
new file mode 100644
index 00000000000..18bebf5fe92
--- /dev/null
+++ b/doc/workflow/merge_requests/only_allow_merge_if_build_succeeds.png
Binary files differ
diff --git a/doc/workflow/notifications.md b/doc/workflow/notifications.md
index 80817c98d22..fe4485e148a 100644
--- a/doc/workflow/notifications.md
+++ b/doc/workflow/notifications.md
@@ -4,7 +4,7 @@ GitLab has a notification system in place to notify a user of events that are im
## Notification settings
-Under user profile page you can find the notification settings.
+You can find notification settings under the user profile.
![notification settings](notifications/settings.png)
@@ -20,6 +20,7 @@ Each of these settings have levels of notification:
* Participating - receive notifications from related resources
* Watch - receive notifications from projects or groups user is a member of
* Global - notifications as set at the global settings
+* Custom - user will receive notifications when mentioned, is participant and custom selected events.
#### Global Settings
@@ -55,7 +56,7 @@ Below is the table of events users can be notified of:
| User added to project | User | Sent when user is added to project |
| Project access level changed | User | Sent when user project access level is changed |
| User added to group | User | Sent when user is added to group |
-| Group access level changed | User | Sent when user group access level is changed |
+| Group access level changed | User | Sent when user group access level is changed |
| Project moved | Project members [1] | [1] not disabled |
### Issue / Merge Request events
@@ -69,8 +70,9 @@ In all of the below cases, the notification will be sent to:
...with notification level "Participating" or higher
-- Watchers: project members with notification level "Watch"
+- Watchers: users with notification level "Watch"
- Subscribers: anyone who manually subscribed to the issue/merge request
+- Custom: Users with notification level "custom" who turned on notifications for any of the events present in the table below
| Event | Sent to |
|------------------------|---------|
diff --git a/doc/workflow/notifications/settings.png b/doc/workflow/notifications/settings.png
index e5b50ee2494..7c6857aad1a 100644
--- a/doc/workflow/notifications/settings.png
+++ b/doc/workflow/notifications/settings.png
Binary files differ
diff --git a/doc/workflow/shortcuts.png b/doc/workflow/shortcuts.png
index 83e562d6929..beb6c53ec77 100644
--- a/doc/workflow/shortcuts.png
+++ b/doc/workflow/shortcuts.png
Binary files differ
diff --git a/doc/workflow/web_editor.md b/doc/workflow/web_editor.md
index 4a451d98953..1832567a34c 100644
--- a/doc/workflow/web_editor.md
+++ b/doc/workflow/web_editor.md
@@ -66,6 +66,35 @@ the target branch. Click **Create directory** to finish.
## Create a new branch
+There are multiple ways to create a branch from GitLab's web interface.
+
+### Create a new branch from an issue
+
+>**Note:**
+This feature was [introduced][ce-2808] in GitLab 8.6.
+
+In case your development workflow dictates to have an issue for every merge
+request, you can quickly create a branch right on the issue page which will be
+tied with the issue itself. You can see a **New Branch** button after the issue
+description, unless there is already a branch with the same name or a referenced
+merge request.
+
+![New Branch Button](img/new_branch_from_issue.png)
+
+Once you click it, a new branch will be created that diverges from the default
+branch of your project, by default `master`. The branch name will be based on
+the title of the issue and as suffix it will have its ID. Thus, the example
+screenshot above will yield a branch named
+`2-et-cum-et-sed-expedita-repellat-consequatur-ut-assumenda-numquam-rerum`.
+
+After the branch is created, you can edit files in the repository to fix
+the issue. When a merge request is created based on the newly created branch,
+the description field will automatically display the [issue closing pattern]
+`Closes #ID`, where `ID` the ID of the issue. This will close the issue once the
+merge request is merged.
+
+### Create a new branch from a project's dashboard
+
If you want to make changes to several files before creating a new merge
request, you can create a new branch up front. From a project's files page,
choose **New branch** from the dropdown.
@@ -118,3 +147,6 @@ appear that is labeled **Start a new merge request with these changes**. After
you commit the changes you will be taken to a new merge request form.
![Start a new merge request with these changes](img/web_editor_start_new_merge_request.png)
+
+[ce-2808]: https://gitlab.com/gitlab-org/gitlab-ce/merge_requests/2808
+[issue closing pattern]: ../customization/issue_closing.md