Merge branch 'master' into merge-if-green

# Conflicts:
#	app/views/projects/merge_requests/widget/_heading.html.haml
#	app/views/projects/merge_requests/widget/open/_accept.html.haml

4 files changed, 284 insertions, 4 deletions

diff --git a/doc/api/groups.md b/doc/api/groups.md
index 0b9f6406d8d..808675d8605 100644
--- a/doc/api/groups.md
+++ b/doc/api/groups.md
@@ -1,6 +1,6 @@
 # Groups

 

-## List project groups

+## List groups

 

 Get a list of groups. (As user: my groups, as admin: all groups)

 

@@ -21,6 +21,70 @@ GET /groups
 

 You can search for groups by name or path, see below.

 

+

+## List a group's projects

+

+Get a list of projects in this group.

+

+```

+GET /groups/:id/projects

+```

+

+Parameters:

+

+- `archived` (optional) - if passed, limit by archived status

+- `order_by` (optional) - Return requests ordered by `id`, `name`, `path`, `created_at`, `updated_at` or `last_activity_at` fields. Default is `created_at`

+- `sort` (optional) - Return requests sorted in `asc` or `desc` order. Default is `desc`

+- `search` (optional) - Return list of authorized projects according to a search criteria

+- `ci_enabled_first` - Return projects ordered by ci_enabled flag. Projects with enabled GitLab CI go first

+

+```json

+[

+  {

+    "id": 4,

+    "description": null,

+    "default_branch": "master",

+    "public": false,

+    "visibility_level": 0,

+    "ssh_url_to_repo": "git@example.com:diaspora/diaspora-client.git",

+    "http_url_to_repo": "http://example.com/diaspora/diaspora-client.git",

+    "web_url": "http://example.com/diaspora/diaspora-client",

+    "tag_list": [

+      "example",

+      "disapora client"

+    ],

+    "owner": {

+      "id": 3,

+      "name": "Diaspora",

+      "created_at": "2013-09-30T13: 46: 02Z"

+    },

+    "name": "Diaspora Client",

+    "name_with_namespace": "Diaspora / Diaspora Client",

+    "path": "diaspora-client",

+    "path_with_namespace": "diaspora/diaspora-client",

+    "issues_enabled": true,

+    "merge_requests_enabled": true,

+    "builds_enabled": true,

+    "wiki_enabled": true,

+    "snippets_enabled": false,

+    "created_at": "2013-09-30T13: 46: 02Z",

+    "last_activity_at": "2013-09-30T13: 46: 02Z",

+    "creator_id": 3,

+    "namespace": {

+      "created_at": "2013-09-30T13: 46: 02Z",

+      "description": "",

+      "id": 3,

+      "name": "Diaspora",

+      "owner_id": 1,

+      "path": "diaspora",

+      "updated_at": "2013-09-30T13: 46: 02Z"

+    },

+    "archived": false,

+    "avatar_url": "http://example.com/uploads/project/avatar/4/uploads/avatar.png"

+  }

+]

+```

+

 ## Details of a group

 

 Get all details of a group.

@@ -186,7 +250,7 @@ To get more (up to 100), pass the following as an argument to the API call:
 /groups?per_page=100

 ```

 

-And to switch pages add: 

+And to switch pages add:

 ```

 /groups?per_page=100&page=2

-```
\ No newline at end of file
+```

diff --git a/doc/integration/ldap.md b/doc/integration/ldap.md
index 7e2920b8865..845f588f913 100644
--- a/doc/integration/ldap.md
+++ b/doc/integration/ldap.md
@@ -13,6 +13,12 @@ An LDAP user who is allowed to change their email on the LDAP server can [take o
 
 We recommend against using GitLab LDAP integration if your LDAP users are allowed to change their 'mail', 'email' or 'userPrincipalName'  attribute on the LDAP server.
 
+If a user is deleted from the LDAP server, they will be blocked in GitLab as well.
+Users will be immediately blocked from logging in. However, there is an LDAP check
+cache time of one hour. The means users that are already logged in or are using Git
+over SSH will still be able to access GitLab for up to one hour. Manually block
+the user in the GitLab Admin area to immediately block all access.
+
 ## Configuring GitLab for LDAP integration
 
 To enable GitLab LDAP integration you need to add your LDAP server settings in `/etc/gitlab/gitlab.rb` or `/home/git/gitlab/config/gitlab.yml`.
@@ -192,4 +198,4 @@ Not supported by GitLab's configuration options.
 When setting `method: ssl`, the underlying authentication method used by 
 `omniauth-ldap` is `simple_tls`.  This method establishes TLS encryption with 
 the LDAP server before any LDAP-protocol data is exchanged but no validation of
-the LDAP server's SSL certificate is performed.
\ No newline at end of file
+the LDAP server's SSL certificate is performed.
diff --git a/doc/operations/moving_repositories.md b/doc/operations/moving_repositories.md
new file mode 100644
index 00000000000..39086b7a251
--- /dev/null
+++ b/doc/operations/moving_repositories.md
@@ -0,0 +1,180 @@
+# Moving repositories managed by GitLab
+
+Sometimes you need to move all repositories managed by GitLab to
+another filesystem or another server. In this document we will look
+at some of the ways you can copy all your repositories from
+`/var/opt/gitlab/git-data/repositories` to `/mnt/gitlab/repositories`.
+
+We will look at three scenarios: the target directory is empty, the
+target directory contains an outdated copy of the repositories, and
+how to deal with thousands of repositories.
+
+**Each of the approaches we list can/will overwrite data in the
+target directory `/mnt/gitlab/repositories`. Do not mix up the
+source and the target.**
+
+## Target directory is empty: use a tar pipe
+
+If the target directory `/mnt/gitlab/repositories` is empty the
+simplest thing to do is to use a tar pipe.  This method has low
+overhead and tar is almost always already installed on your system.
+However, it is not possible to resume an interrupted tar pipe:  if
+that happens then all data must be copied again.
+
+```
+# As the git user
+tar -C /var/opt/gitlab/git-data/repositories -cf - -- . |\
+  tar -C /mnt/gitlab/repositories -xf -
+```
+
+If you want to see progress, replace `-xf` with `-xvf`.
+
+### Tar pipe to another server
+
+You can also use a tar pipe to copy data to another server. If your
+'git' user has SSH access to the newserver as 'git@newserver', you
+can pipe the data through SSH.
+
+```
+# As the git user
+tar -C /var/opt/gitlab/git-data/repositories -cf - -- . |\
+  ssh git@newserver tar -C /mnt/gitlab/repositories -xf -
+```
+
+If you want to compress the data before it goes over the network
+(which will cost you CPU cycles) you can replace `ssh` with `ssh -C`.
+
+## The target directory contains an outdated copy of the repositories: use rsync
+
+If the target directory already contains a partial / outdated copy
+of the repositories it may be wasteful to copy all the data again
+with tar. In this scenario it is better to use rsync. This utility
+is either already installed on your system or easily installable
+via apt, yum etc.
+
+```
+# As the 'git' user
+rsync -a --delete /var/opt/gitlab/git-data/repositories/. \
+  /mnt/gitlab/repositories
+```
+
+The `/.` in the command above is very important, without it you can
+easily get the wrong directory structure in the target directory.
+If you want to see progress, replace `-a` with `-av`.
+
+### Single rsync to another server
+
+If the 'git' user on your source system has SSH access to the target
+server you can send the repositories over the network with rsync.
+
+```
+# As the 'git' user
+rsync -a --delete /var/opt/gitlab/git-data/repositories/. \
+  git@newserver:/mnt/gitlab/repositories
+```
+
+## Thousands of Git repositories: use one rsync per repository
+
+Every time you start an rsync job it has to inspect all files in
+the source directory, all files in the target directory, and then
+decide what files to copy or not. If the source or target directory
+has many contents this startup phase of rsync can become a burden
+for your GitLab server. In cases like this you can make rsync's
+life easier by dividing its work in smaller pieces, and sync one
+repository at a time.
+
+In addition to rsync we will use [GNU
+Parallel](http://www.gnu.org/software/parallel/). This utility is
+not included in GitLab so you need to install it yourself with apt
+or yum.  Also note that the GitLab scripts we used below were added
+in GitLab 8.1.
+
+** This process does not clean up repositories at the target location that no
+longer exist at the source. ** If you start using your GitLab instance with
+`/mnt/gitlab/repositories`, you need to run `gitlab-rake gitlab:cleanup:repos`
+after switching to the new repository storage directory.
+
+### Parallel rsync for all repositories known to GitLab
+
+This will sync repositories with 10 rsync processes at a time. We keep
+track of progress so that the transfer can be restarted if necessary.
+
+First we create a new directory, owned by 'git', to hold transfer
+logs. We assume the directory is empty before we start the transfer
+procedure, and that we are the only ones writing files in it.
+
+```
+# Omnibus
+sudo mkdir /var/opt/gitlab/transfer-logs
+sudo chown git:git /var/opt/gitlab/transfer-logs
+
+# Source
+sudo -u git -H mkdir /home/git/transfer-logs
+```
+
+We seed the process with a list of the directories we want to copy.
+
+```
+# Omnibus
+sudo -u git sh -c 'gitlab-rake gitlab:list_repos > /var/opt/gitlab/transfer-logs/all-repos-$(date +%s).txt'
+
+# Source
+cd /home/git/gitlab
+sudo -u git -H sh -c 'bundle exec rake gitlab:list_repos > /home/git/transfer-logs/all-repos-$(date +%s).txt'
+```
+
+Now we can start the transfer. The command below is idempotent, and
+the number of jobs done by GNU Parallel should converge to zero. If it
+does not some repositories listed in all-repos-1234.txt may have been
+deleted/renamed before they could be copied.
+
+```
+# Omnibus
+sudo -u git sh -c '
+cat /var/opt/gitlab/transfer-logs/* | sort | uniq -u |\
+  /usr/bin/env JOBS=10 \
+  /opt/gitlab/embedded/service/gitlab-rails/bin/parallel-rsync-repos \
+    /var/opt/gitlab/transfer-logs/succes-$(date +%s).log \
+    /var/opt/gitlab/git-data/repositories \
+    /mnt/gitlab/repositories
+'
+
+# Source
+cd /home/git/gitlab
+sudo -u git -H sh -c '
+cat /home/git/transfer-logs/* | sort | uniq -u |\
+  /usr/bin/env JOBS=10 \
+  bin/parallel-rsync-repos \
+    /home/git/transfer-logs/succes-$(date +%s).log \
+    /home/git/repositories \
+    /mnt/gitlab/repositories
+`
+```
+
+### Parallel rsync only for repositories with recent activity
+
+Suppose you have already done one sync that started after 2015-10-1 12:00 UTC.
+Then you might only want to sync repositories that were changed via GitLab
+_after_ that time. You can use the 'SINCE' variable to tell 'rake
+gitlab:list_repos' to only print repositories with recent activity.
+
+```
+# Omnibus
+sudo gitlab-rake gitlab:list_repos SINCE='2015-10-1 12:00 UTC' |\
+  sudo -u git \
+  /usr/bin/env JOBS=10 \
+  /opt/gitlab/embedded/service/gitlab-rails/bin/parallel-rsync-repos \
+    succes-$(date +%s).log \
+    /var/opt/gitlab/git-data/repositories \
+    /mnt/gitlab/repositories
+
+# Source
+cd /home/git/gitlab
+sudo -u git -H bundle exec rake gitlab:list_repos SINCE='2015-10-1 12:00 UTC' |\
+  sudo -u git -H \
+  /usr/bin/env JOBS=10 \
+  bin/parallel-rsync-repos \
+    succes-$(date +%s).log \
+    /home/git/repositories \
+    /mnt/gitlab/repositories
+```
diff --git a/doc/raketasks/list_repos.md b/doc/raketasks/list_repos.md
new file mode 100644
index 00000000000..476428eb4f5
--- /dev/null
+++ b/doc/raketasks/list_repos.md
@@ -0,0 +1,30 @@
+# Listing repository directories
+
+You can print a list of all Git repositories on disk managed by
+GitLab with the following command:
+
+```
+# Omnibus
+sudo gitlab-rake gitlab:list_repos
+
+# Source
+cd /home/git/gitlab
+sudo -u git -H bundle exec rake gitlab:list_repos RAILS_ENV=production
+```
+
+If you only want to list projects with recent activity you can pass
+a date with the 'SINCE' environment variable.  The time you specify
+is parsed by the Rails [TimeZone#parse
+function](http://api.rubyonrails.org/classes/ActiveSupport/TimeZone.html#method-i-parse).
+
+```
+# Omnibus
+sudo gitlab-rake gitlab:list_repos SINCE='Sep 1 2015'
+
+# Source
+cd /home/git/gitlab
+sudo -u git -H bundle exec rake gitlab:list_repos RAILS_ENV=production SINCE='Sep 1 2015'
+```
+
+Note that the projects listed are NOT sorted by activity; they use
+the default ordering of the GitLab Rails application.