Merge branch 'master' into per-build-token

14 files changed, 214 insertions, 72 deletions

diff --git a/doc/api/groups.md b/doc/api/groups.md
index a898387eaa2..3e94e1e4efe 100644
--- a/doc/api/groups.md
+++ b/doc/api/groups.md
@@ -288,6 +288,7 @@ Parameters:
 - `path` (required) - The path of the group
 - `description` (optional) - The group's description
 - `visibility_level` (optional) - The group's visibility. 0 for private, 10 for internal, 20 for public.
+- `lfs_enabled` (optional)      - Enable/disable Large File Storage (LFS) for the projects in this group
 
 ## Transfer project to group
 
@@ -317,6 +318,7 @@ PUT /groups/:id
 | `path` | string | no | The path of the group |
 | `description` | string | no | The description of the group |
 | `visibility_level` | integer | no | The visibility level of the group. 0 for private, 10 for internal, 20 for public. |
+| `lfs_enabled` (optional) | boolean | no | Enable/disable Large File Storage (LFS) for the projects in this group |
 
 ```bash
 curl --request PUT --header "PRIVATE-TOKEN: 9koXpg98eAheJpvBs5tK" "https://gitlab.example.com/api/v3/groups/5?name=Experimental"
diff --git a/doc/api/notes.md b/doc/api/notes.md
index 85d140d06ac..572844b8b3f 100644
--- a/doc/api/notes.md
+++ b/doc/api/notes.md
@@ -78,7 +78,8 @@ Parameters:
 
 ### Create new issue note
 
-Creates a new note to a single project issue.
+Creates a new note to a single project issue. If you create a note where the body
+only contains an Award Emoji, you'll receive this object back.
 
 ```
 POST /projects/:id/issues/:issue_id/notes
@@ -204,6 +205,7 @@ Parameters:
 ### Create new snippet note
 
 Creates a new note for a single snippet. Snippet notes are comments users can post to a snippet.
+If you create a note where the body only contains an Award Emoji, you'll receive this object back.
 
 ```
 POST /projects/:id/snippets/:snippet_id/notes
@@ -332,6 +334,8 @@ Parameters:
 ### Create new merge request note
 
 Creates a new note for a single merge request.
+If you create a note where the body only contains an Award Emoji, you'll receive
+this object back.
 
 ```
 POST /projects/:id/merge_requests/:merge_request_id/notes
diff --git a/doc/api/settings.md b/doc/api/settings.md
index a76dad0ebd4..aaa2c99642b 100644
--- a/doc/api/settings.md
+++ b/doc/api/settings.md
@@ -67,7 +67,7 @@ PUT /application/settings
 | `default_snippet_visibility` | integer | no | What visibility level new snippets receive. Can take `0` _(Private)_, `1` _(Internal)_ and `2` _(Public)_ as a parameter. Default is `0`.|
 | `domain_whitelist` | array of strings | no | Force people to use only corporate emails for sign-up. Default is null, meaning there is no restriction. |
 | `domain_blacklist_enabled` | boolean | no | Enable/disable the `domain_blacklist` |
-| `domain_blacklist` | array of strings | yes (if `domain_whitelist_enabled` is `true` | People trying to sign-up with emails from this domain will not be allowed to do so. |
+| `domain_blacklist` | array of strings | yes (if `domain_blacklist_enabled` is `true`) | People trying to sign-up with emails from this domain will not be allowed to do so. |
 | `user_oauth_applications` | boolean | no | Allow users to register any application to use GitLab as an OAuth provider |
 | `after_sign_out_path` | string | no | Where to redirect users after logout |
 | `container_registry_token_expire_delay` | integer | no | Container Registry token duration in minutes |
diff --git a/doc/ci/examples/README.md b/doc/ci/examples/README.md
index 406396deaaa..71670e6247c 100644
--- a/doc/ci/examples/README.md
+++ b/doc/ci/examples/README.md
@@ -16,4 +16,4 @@ Apart from those, here is an collection of tutorials and guides on setting up yo
 - [Repo's with examples for various languages](https://gitlab.com/groups/gitlab-examples)
 - [The .gitlab-ci.yml file for GitLab itself](https://gitlab.com/gitlab-org/gitlab-ce/blob/master/.gitlab-ci.yml)
 
-[gitlab-ci-templates][https://gitlab.com/gitlab-org/gitlab-ci-yml]
+[gitlab-ci-templates]: https://gitlab.com/gitlab-org/gitlab-ci-yml
diff --git a/doc/ci/yaml/README.md b/doc/ci/yaml/README.md
index ff4c8ddc54b..16868554c1f 100644
--- a/doc/ci/yaml/README.md
+++ b/doc/ci/yaml/README.md
@@ -90,8 +90,7 @@ builds, including deploy builds. This can be an array or a multi-line string.
 
 ### after_script
 
->**Note:**
-Introduced in GitLab 8.7 and requires Gitlab Runner v1.2
+> Introduced in GitLab 8.7 and requires Gitlab Runner v1.2
 
 `after_script` is used to define the command that will be run after for all
 builds. This has to be an array or a multi-line string.
@@ -135,8 +134,7 @@ Alias for [stages](#stages).
 
 ### variables
 
->**Note:**
-Introduced in GitLab Runner v0.5.0.
+> Introduced in GitLab Runner v0.5.0.
 
 GitLab CI allows you to add variables to `.gitlab-ci.yml` that are set in the
 build environment. The variables are stored in the Git repository and are meant
@@ -158,8 +156,7 @@ Variables can be also defined on [job level](#job-variables).
 
 ### cache
 
->**Note:**
-Introduced in GitLab Runner v0.7.0.
+> Introduced in GitLab Runner v0.7.0.
 
 `cache` is used to specify a list of files and directories which should be
 cached between builds.
@@ -220,8 +217,7 @@ will be always present. For implementation details, please check GitLab Runner.
 
 #### cache:key
 
->**Note:**
-Introduced in GitLab Runner v1.0.0.
+> Introduced in GitLab Runner v1.0.0.
 
 The `key` directive allows you to define the affinity of caching
 between jobs, allowing to have a single cache for all jobs,
@@ -531,8 +527,7 @@ The above script will:
 
 #### Manual actions
 
->**Note:**
-Introduced in GitLab 8.10.
+> Introduced in GitLab 8.10.
 
 Manual actions are a special type of job that are not executed automatically;
 they need to be explicitly started by a user. Manual actions can be started
@@ -543,17 +538,16 @@ An example usage of manual actions is deployment to production.
 
 ### environment
 
->**Note:**
-Introduced in GitLab 8.9.
+> Introduced in GitLab 8.9.
 
-`environment` is used to define that a job deploys to a specific environment.
+`environment` is used to define that a job deploys to a specific [environment].
 This allows easy tracking of all deployments to your environments straight from
 GitLab.
 
 If `environment` is specified and no environment under that name exists, a new
 one will be created automatically.
 
-The `environment` name must contain only letters, digits, '-' and '_'. Common
+The `environment` name must contain only letters, digits, '-', '_', '/', '$', '{', '}' and spaces. Common
 names are `qa`, `staging`, and `production`, but you can use whatever name works
 with your workflow.
 
@@ -571,6 +565,35 @@ deploy to production:
 The `deploy to production` job will be marked as doing deployment to
 `production` environment.
 
+#### dynamic environments
+
+> [Introduced][ce-6323] in GitLab 8.12 and GitLab Runner 1.6.
+
+`environment` can also represent a configuration hash with `name` and `url`.
+These parameters can use any of the defined CI [variables](#variables)
+(including predefined, secure variables and `.gitlab-ci.yml` variables).
+
+The common use case is to create dynamic environments for branches and use them
+as review apps.
+
+---
+
+**Example configurations**
+
+```
+deploy as review app:
+  stage: deploy
+  script: ...
+  environment:
+    name: review-apps/$CI_BUILD_REF_NAME
+    url: https://$CI_BUILD_REF_NAME.review.example.com/
+```
+
+The `deploy as review app` job will be marked as deployment to dynamically
+create the `review-apps/branch-name` environment.
+
+This environment should be accessible under `https://branch-name.review.example.com/`.
+
 ### artifacts
 
 >**Notes:**
@@ -638,8 +661,7 @@ be available for download in the GitLab UI.
 
 #### artifacts:name
 
->**Note:**
-Introduced in GitLab 8.6 and GitLab Runner v1.1.0.
+> Introduced in GitLab 8.6 and GitLab Runner v1.1.0.
 
 The `name` directive allows you to define the name of the created artifacts
 archive. That way, you can have a unique name for every archive which could be
@@ -702,8 +724,7 @@ job:
 
 #### artifacts:when
 
->**Note:**
-Introduced in GitLab 8.9 and GitLab Runner v1.3.0.
+> Introduced in GitLab 8.9 and GitLab Runner v1.3.0.
 
 `artifacts:when` is used to upload artifacts on build failure or despite the
 failure.
@@ -728,8 +749,7 @@ job:
 
 #### artifacts:expire_in
 
->**Note:**
-Introduced in GitLab 8.9 and GitLab Runner v1.3.0.
+> Introduced in GitLab 8.9 and GitLab Runner v1.3.0.
 
 `artifacts:expire_in` is used to delete uploaded artifacts after the specified
 time. By default, artifacts are stored on GitLab forever. `expire_in` allows you
@@ -764,8 +784,7 @@ job:
 
 ### dependencies
 
->**Note:**
-Introduced in GitLab 8.6 and GitLab Runner v1.1.1.
+> Introduced in GitLab 8.6 and GitLab Runner v1.1.1.
 
 This feature should be used in conjunction with [`artifacts`](#artifacts) and
 allows you to define the artifacts to pass between different builds.
@@ -839,9 +858,8 @@ job:
 
 ## Git Strategy
 
->**Note:**
-Introduced in GitLab 8.9 as an experimental feature. May change in future
-releases or be removed completely.
+> Introduced in GitLab 8.9 as an experimental feature. May change in future
+  releases or be removed completely.
 
 You can set the `GIT_STRATEGY` used for getting recent application code. `clone`
 is slower, but makes sure you have a clean directory before every build. `fetch`
@@ -863,8 +881,7 @@ variables:
 
 ## Shallow cloning
 
->**Note:**
-Introduced in GitLab 8.9 as an experimental feature. May change in future
+> Introduced in GitLab 8.9 as an experimental feature. May change in future
 releases or be removed completely.
 
 You can specify the depth of fetching and cloning using `GIT_DEPTH`. This allows
@@ -894,8 +911,7 @@ variables:
 
 ## Hidden keys
 
->**Note:**
-Introduced in GitLab 8.6 and GitLab Runner v1.1.1.
+> Introduced in GitLab 8.6 and GitLab Runner v1.1.1.
 
 Keys that start with a dot (`.`) will be not processed by GitLab CI. You can
 use this feature to ignore jobs, or use the
@@ -923,8 +939,7 @@ Read more about the various [YAML features](https://learnxinyminutes.com/docs/ya
 
 ### Anchors
 
->**Note:**
-Introduced in GitLab 8.6 and GitLab Runner v1.1.1.
+> Introduced in GitLab 8.6 and GitLab Runner v1.1.1.
 
 YAML also has a handy feature called 'anchors', which let you easily duplicate
 content across your document. Anchors can be used to duplicate/inherit
@@ -1067,3 +1082,5 @@ Visit the [examples README][examples] to see a list of examples using GitLab
 CI with various languages.
 
 [examples]: ../examples/README.md
+[ce-6323]: https://gitlab.com/gitlab-org/gitlab-ce/merge_requests/6323
+[environment]: ../environments.md
diff --git a/doc/container_registry/README.md b/doc/container_registry/README.md
index 047a0b08406..d7740647a91 100644
--- a/doc/container_registry/README.md
+++ b/doc/container_registry/README.md
@@ -78,9 +78,9 @@ delete them.
 > **Note:**
 This feature requires GitLab 8.8 and GitLab Runner 1.2.
 
-Make sure that your GitLab Runner is configured to allow building docker images.
-You have to check the [Using Docker Build documentation](../ci/docker/using_docker_build.md).
-Then see the CI documentation on [Using the GitLab Container Registry](../ci/docker/using_docker_build.md#using-the-gitlab-container-registry).
+Make sure that your GitLab Runner is configured to allow building Docker images by
+following the [Using Docker Build](../ci/docker/using_docker_build.md)
+and [Using the GitLab Container Registry documentation](../ci/docker/using_docker_build.md#using-the-gitlab-container-registry).
 
 ## Limitations
 
diff --git a/doc/development/migration_style_guide.md b/doc/development/migration_style_guide.md
index b8fab3aaff7..295eae0a88e 100644
--- a/doc/development/migration_style_guide.md
+++ b/doc/development/migration_style_guide.md
@@ -111,6 +111,28 @@ class MyMigration < ActiveRecord::Migration
 end
 ```
 
+
+## Integer column type
+
+By default, an integer column can hold up to a 4-byte (32-bit) number. That is
+a max value of 2,147,483,647. Be aware of this when creating a column that will
+hold file sizes in byte units. If you are tracking file size in bytes this
+restricts the maximum file size to just over 2GB.
+
+To allow an integer column to hold up to an 8-byte (64-bit) number, explicitly
+set the limit to 8-bytes. This will allow the column to hold a value up to
+9,223,372,036,854,775,807.
+
+Rails migration example:
+
+```
+add_column_with_default(:projects, :foo, :integer, default: 10, limit: 8)
+
+# or
+
+add_column(:projects, :foo, :integer, default: 10, limit: 8)
+```
+
 ## Testing
 
 Make sure that your migration works with MySQL and PostgreSQL with data. An empty database does not guarantee that your migration is correct.
diff --git a/doc/raketasks/backup_restore.md b/doc/raketasks/backup_restore.md
index 835af5443a3..3f4056dc440 100644
--- a/doc/raketasks/backup_restore.md
+++ b/doc/raketasks/backup_restore.md
@@ -79,6 +79,9 @@ gitlab_rails['backup_upload_connection'] = {
   'region' => 'eu-west-1',
   'aws_access_key_id' => 'AKIAKIAKI',
   'aws_secret_access_key' => 'secret123'
+  # If using an IAM Profile, leave aws_access_key_id & aws_secret_access_key empty
+  # ie. 'aws_access_key_id' => '',
+  # 'use_iam_profile' => 'true'
 }
 gitlab_rails['backup_upload_remote_directory'] = 'my.s3.bucket'
 ```
@@ -95,6 +98,9 @@ For installations from source:
         region: eu-west-1
         aws_access_key_id: AKIAKIAKI
         aws_secret_access_key: 'secret123'
+        # If using an IAM Profile, leave aws_access_key_id & aws_secret_access_key empty
+        # ie. aws_access_key_id: ''
+        # use_iam_profile: 'true'
       # The remote 'directory' to store your backups. For S3, this would be the bucket name.
       remote_directory: 'my.s3.bucket'
       # Turns on AWS Server-Side Encryption with Amazon S3-Managed Keys for backups, this is optional
diff --git a/doc/user/project/builds/artifacts.md b/doc/user/project/builds/artifacts.md
index c93ae1c369c..88f1863dddb 100644
--- a/doc/user/project/builds/artifacts.md
+++ b/doc/user/project/builds/artifacts.md
@@ -101,4 +101,36 @@ inside GitLab that make that possible.
 
     ![Build artifacts browser](img/build_artifacts_browser.png)
 
+## Downloading the latest build artifacts
+
+It is possible to download the latest artifacts of a build via a well known URL
+so you can use it for scripting purposes.
+
+The structure of the URL is the following:
+
+```
+https://example.com/<namespace>/<project>/builds/artifacts/<ref>/download?job=<job_name>
+```
+
+For example, to download the latest artifacts of the job named `rspec 6 20` of
+the `master` branch of the `gitlab-ce` project that belongs to the `gitlab-org`
+namespace, the URL would be:
+
+```
+https://gitlab.com/gitlab-org/gitlab-ce/builds/artifacts/master/download?job=rspec+6+20
+```
+
+The latest builds are also exposed in the UI in various places. Specifically,
+look for the download button in:
+
+- the main project's page
+- the branches page
+- the tags page
+
+If the latest build has failed to upload the artifacts, you can see that
+information in the UI.
+
+![Latest artifacts button](img/build_latest_artifacts_browser.png)
+
+
 [gitlab workhorse]: https://gitlab.com/gitlab-org/gitlab-workhorse "GitLab Workhorse repository"
diff --git a/doc/user/project/builds/img/build_latest_artifacts_browser.png b/doc/user/project/builds/img/build_latest_artifacts_browser.png
new file mode 100644
index 00000000000..d8e9071958c
--- /dev/null
+++ b/doc/user/project/builds/img/build_latest_artifacts_browser.png
diff --git a/doc/workflow/importing/img/import_projects_from_github_importer.png b/doc/workflow/importing/img/import_projects_from_github_importer.png
index b6ed8dd692a..2082de06f47 100644
--- a/doc/workflow/importing/img/import_projects_from_github_importer.png
+++ b/doc/workflow/importing/img/import_projects_from_github_importer.png
diff --git a/doc/workflow/importing/img/import_projects_from_github_new_project_page.png b/doc/workflow/importing/img/import_projects_from_github_new_project_page.png
index c8f35a50f48..6e91c430a33 100644
--- a/doc/workflow/importing/img/import_projects_from_github_new_project_page.png
+++ b/doc/workflow/importing/img/import_projects_from_github_new_project_page.png
diff --git a/doc/workflow/importing/img/import_projects_from_github_select_auth_method.png b/doc/workflow/importing/img/import_projects_from_github_select_auth_method.png
new file mode 100644
index 00000000000..c11863ab10c
--- /dev/null
+++ b/doc/workflow/importing/img/import_projects_from_github_select_auth_method.png
diff --git a/doc/workflow/importing/import_projects_from_github.md b/doc/workflow/importing/import_projects_from_github.md
index 370d885d366..dd38fe0bc01 100644
--- a/doc/workflow/importing/import_projects_from_github.md
+++ b/doc/workflow/importing/import_projects_from_github.md
@@ -1,54 +1,113 @@
 # Import your project from GitHub to GitLab

 

+Import your projects from GitHub to GitLab with minimal effort.

+

+## Overview

+

 >**Note:**

-In order to enable the GitHub import setting, you may also want to

-enable the [GitHub integration][gh-import] in your GitLab instance. This

-configuration is optional, you will be able import your GitHub repositories

-with a Personal Access Token.

+If you are an administrator you can enable the [GitHub integration][gh-import]

+in your GitLab instance sitewide. This configuration is optional, users will be

+able import their GitHub repositories with a [personal access token][gh-token].

 

-At its current state, GitHub importer can import:

+- At its current state, GitHub importer can import:

+  - the repository description (GitLab 7.7+)

+  - the Git repository data (GitLab 7.7+)

+  - the issues (GitLab 7.7+)

+  - the pull requests (GitLab 8.4+)

+  - the wiki pages (GitLab 8.4+)

+  - the milestones (GitLab 8.7+)

+  - the labels (GitLab 8.7+)

+  - the release note descriptions (GitLab 8.12+)

+- References to pull requests and issues are preserved (GitLab 8.7+)

+- Repository public access is retained. If a repository is private in GitHub

+  it will be created as private in GitLab as well.

 

-- the repository description (introduced in GitLab 7.7)

-- the git repository data (introduced in GitLab 7.7)

-- the issues (introduced in GitLab 7.7)

-- the pull requests (introduced in GitLab 8.4)

-- the wiki pages (introduced in GitLab 8.4)

-- the milestones (introduced in GitLab 8.7)

-- the labels (introduced in GitLab 8.7)

-- the release note descriptions (introduced in GitLab 8.12)

+## How it works

 

-With GitLab 8.7+, references to pull requests and issues are preserved.

+When issues/pull requests are being imported, the GitHub importer tries to find

+the GitHub author/assignee in GitLab's database using the GitHub ID. For this

+to work, the GitHub author/assignee should have signed in beforehand in GitLab

+and [**associated their GitHub account**][social sign-in]. If the user is not

+found in GitLab's database, the project creator (most of the times the current

+user that started the import process) is set as the author, but a reference on

+the issue about the original GitHub author is kept.

 

-The importer page is visible when you [create a new project][new-project].

-Click on the **GitHub** link and, if you are logged in via the GitHub

-integration, you will be redirected to GitHub for permission to access your

-projects. After accepting, you'll be automatically redirected to the importer.

+The importer will create any new namespaces (groups) if they don't exist or in

+the case the namespace is taken, the repository will be imported under the user's

+namespace that started the import process.

 

-If you are not using the GitHub integration, you can still perform a one-off

-authorization with GitHub to access your projects.

+## Importing your GitHub repositories

 

-Alternatively, you can also enter a GitHub Personal Access Token. Once you enter

-your token, you'll be taken to the importer.

+The importer page is visible when you create a new project.

 

 ![New project page on GitLab](img/import_projects_from_github_new_project_page.png)

 

----

+Click on the **GitHub** link and the import authorization process will start.

+There are two ways to authorize access to your GitHub repositories:

 

-While at the GitHub importer page, you can see the import statuses of your

-GitHub projects. Those that are being imported will show a _started_ status,

-those already imported will be green, whereas those that are not yet imported

-have an **Import** button on the right side of the table. If you want, you can

-import all your GitHub projects in one go by hitting **Import all projects**

-in the upper left corner.

+1. [Using the GitHub integration][gh-integration] (if it's enabled by your

+   GitLab administrator). This is the preferred way as it's possible to

+   preserve the GitHub authors/assignees. Read more in the [How it works](#how-it-works)

+   section.

+1. [Using a personal access token][gh-token] provided by GitHub.

 

-![GitHub importer page](img/import_projects_from_github_importer.png)

+![Select authentication method](img/import_projects_from_github_select_auth_method.png)

+

+### Authorize access to your repositories using the GitHub integration

 

----

+If the [GitHub integration][gh-import] is enabled by your GitLab administrator,

+you can use it instead of the personal access token.

+

+1. First you may want to connect your GitHub account to GitLab in order for

+   the username mapping to be correct. Follow the [social sign-in] documentation

+   on how to do so.

+1. Once you connect GitHub, click the **List your GitHub repositories** button

+   and you will be redirected to GitHub for permission to access your projects.

+1. After accepting, you'll be automatically redirected to the importer.

+

+You can now go on and [select which repositories to import](#select-which-repositories-to-import).

+

+### Authorize access to your repositories using a personal access token

+

+>**Note:**

+For a proper author/assignee mapping for issues and pull requests, the

+[GitHub integration][gh-integration] should be used instead of the

+[personal access token][gh-token]. If the GitHub integration is enabled by your

+GitLab administrator, it should be the preferred method to import your repositories.

+Read more in the [How it works](#how-it-works) section.

 

-The importer will create any new namespaces if they don't exist or in the

-case the namespace is taken, the project will be imported on the user's

-namespace.

+If you are not using the GitHub integration, you can still perform a one-off

+authorization with GitHub to grant GitLab access your repositories:

+

+1. Go to <https://github.com/settings/tokens/new>.

+1. Enter a token description.

+1. Check the `repo` scope.

+1. Click **Generate token**.

+1. Copy the token hash.

+1. Go back to GitLab and provide the token to the GitHub importer.

+1. Hit the **List your GitHub repositories** button and wait while GitLab reads

+   your repositories' information. Once done, you'll be taken to the importer

+   page to select the repositories to import.

+

+### Select which repositories to import

+

+After you've authorized access to your GitHub repositories, you will be

+redirected to the GitHub importer page.

+

+From there, you can see the import statuses of your GitHub repositories.

+

+- Those that are being imported will show a _started_ status,

+- those already successfully imported will be green with a _done_ status,

+- whereas those that are not yet imported will have an **Import** button on the

+  right side of the table.

+

+If you want, you can import all your GitHub projects in one go by hitting

+**Import all projects** in the upper left corner.

+

+![GitHub importer page](img/import_projects_from_github_importer.png)

 

 [gh-import]: ../../integration/github.md "GitHub integration"

-[ee-gh]: http://docs.gitlab.com/ee/integration/github.html "GitHub integration for GitLab EE"

 [new-project]: ../../gitlab-basics/create-project.md "How to create a new project in GitLab"

+[gh-integration]: #authorize-access-to-your-repositories-using-the-github-integration

+[gh-token]: #authorize-access-to-your-repositories-using-a-personal-access-token

+[social sign-in]: ../../profile/account/social_sign_in.md