Refactor new CI build permissions model docs

2 files changed, 211 insertions, 158 deletions

diff --git a/doc/ci/dependent_projects/README.md b/doc/ci/dependent_projects/README.md
index feb148e31b6..3e91f82e2d4 100644
--- a/doc/ci/dependent_projects/README.md
+++ b/doc/ci/dependent_projects/README.md
@@ -1,67 +1,223 @@
-## Dependent projects
+# New CI build permission model
 
 > Introduced in GitLab 8.12.
 
-GitLab 8.12 introduces a new [Build permissions model](../../user/permissions.md#builds-permissions).
+GitLab 8.12 has a completely redesigned build permission system. You can find
+all discussion and all our concerns when choosing the current approach in issue
+[#18994](https://gitlab.com/gitlab-org/gitlab-ce/issues/18994).
 
-This opens an easy to use a way to access all dependent source codes:
-1. Access project's `submodule`,
-1. Access private Container Images,
-1. Access project's and submodule LFS objects.
+---
 
-### Submodules
+Builds permissions should be tightly integrated with the permissions of a user
+who is triggering a build.
 
-> It often happens that while working on one project, you need to use another project from within it.
-> Perhaps it’s a library that a third party developed or that you’re developing separately and using in multiple parent projects.
-> A common issue arises in these scenarios: you want to be able to treat the two projects as separate yet still be able to use one from within the other.
-> (from https://git-scm.com/book/en/v2/Git-Tools-Submodules)
+The reasons to do it like that are:
+
+- We already have a permissions system in place: group and project membership
+  of users.
+- We already fully know who is triggering a build (using `git push`, using the
+  web UI, executing triggers).
+- We already know what user is allowed to do.
+- We use the user permissions for builds that are triggered by the user.
+- It opens a lot of possibilities to further enforce user permissions, like
+  allowing only specific users to access runners or use secure variables and
+  environments.
+- It is simple and convenient that your build can access everything that you
+  as a user have access to.
+- Short living unique tokens are now used, granting access for time of the build
+  and maximizing security.
+
+With the new behavior, any build that is triggered by the user, is also marked
+with their permissions. When a user does a `git push` or changes files through
+the web UI, a new pipeline will be usually created. This pipeline will be marked
+as created be the pusher (local push or via the UI) and any build created in this
+pipeline will have the permissions of the pusher.
+
+This allows us to make it really easy to evaluate the access for all projects
+that have Git submodules or are using container images that the pusher would
+have access too. **The permission is granted only for time that build is running.
+The access is revoked after the build is finished.**
+
+## Types of users
+
+It is important to note that we have a few types of users:
+
+- **Administrators**: CI builds created by Administrators will not have access
+  to all GitLab projects, but only to projects and container images of projects
+  that the administrator is a member of.That means that if a project is either
+  public or internal users have access anyway, but if a project is private, the
+  Administrator will have to be a member of it in order to have access to it
+  via another project's build.
+
+- **External users**: CI builds created by external users will have access only
+  to projects to which user has at least reporter access. This rules out
+  accessing all internal projects by default,
+
+This allows us to make the CI and permission system more trustworthy.
+Let's consider the following scenario:
+
+1. You are an employee of a company. Your company has a number of internal tools
+   hosted in private repositories and you have multiple CI builds that make use
+   of these repositories.
+
+2. You invite a new user, a visitor, the external user. CI builds created by
+   that user do not have access to internal repositories, because the user also
+   doesn't have the access from within GitLab. You as an employee have to grant
+   explicit access for this user. This allows us to prevent from accidental data
+   leakage.
+
+## Build token
+
+A unique build token is generated for each project and it allows the user to
+access all projects that would be normally accessible to the user creating that
+build.
+
+We try to make sure that this token doesn't leak by:
+
+1. Securing all API endpoints to not expose the build token.
+1. Masking the build token from build logs.
+1. Allowing to use the build token **only** when build is running.
+
+However, this brings a question about the Runners security. To make sure that
+this token doesn't leak, you should also make sure that you configure
+your Runners in the most possible secure way, by avoiding the following:
+
+1. Any usage of Docker's `privileged` mode is risky if the machines are re-used.
+1. Using the `shell` executor since builds run on the same machine.
+
+By using an insecure GitLab Runner configuration, you allow the rogue developers
+to steal the tokens of other builds.
+
+## Debugging problems
+
+With the new permission model in place, there may be times that your build will
+fail. This is most likely because your project tries to access other project's
+sources, and you don't have the appropriate permissions. In the build log look
+for information about 403 or forbidden access messages
+
+As an Administrator, you can verify that the user is a member of the group or
+project they're trying to have access to, and you can impersonate the user to
+retry the failing build in order to verify that everything is correct.
+
+## Before GitLab 8.12
+
+In versions before GitLab 8.12, all CI builds would use the CI Runner's token
+to checkout project sources.
+
+The project's Runner's token was a token that you could find under the
+project's **Settings > CI/CD Pipelines** and was limited to access only that
+project.
+
+The project's Runner's token was used for registering new specific Runners
+assigned to project and to checkout project sources.
+
+The project Runners token could also be used with the GitLab Container Registry
+for that project, allowing pulling and pushing Docker images from within the
+CI build.
+
+GitLab would create a special checkout URL like:
+
+```
+https://gitlab-ci-token:<project-runners-token>/gitlab.com/gitlab-org/gitlab-ce.git
+```
+
+And then the users could also use it in their CI builds all Docker related
+commands to interact with GitLab Container Registry. For example:
+
+```
+docker login -u gitlab-ci-token -p $CI_BUILD_TOKEN registry.gitlab.com
+```
+
+Using single token had multiple security implications
+
+- The token would be readable to anyone who had developer access to a project
+  that could run CI builds, allowing the developer to register any specific
+  Runner for that project.
+- The token would allow to access only the project's sources, forbidding from
+  accessing any other projects.
+- The token was not expiring and was multi-purpose: used for checking out sources,
+  for registering specific runners and for accessing a project's container
+  registry with read-write permissions.
+
+All the above led to a new permission model for builds that was introduced
+with GitLab 8.12.
+
+## Making use of the new CI build permissions model
+
+With the new build permission model, there is now an easy way to access all
+dependent source code in a project. That way, we can:
+
+1. Access a project's Git submodules
+1. Access private container images
+1. Access project's and submodule LFS objects
+
+Let's see how that works with Git submodules and private Docker images hosted on
+the container registry.
+
+## Git submodules
+
+>
+It often happens that while working on one project, you need to use another
+project from within it. Perhaps it’s a library that a third party developed or
+you’re developing a project separately and are using it in multiple parent
+projects.
+A common issue arises in these scenarios: you want to be able to treat the two
+projects as separate yet still be able to use one from within the other.
+>
+_Excerpt from the [Git website][git-scm] about submodules._
+
+If dealing with submodules, your project will probably have a file named
+`.gitmodules`. And this is how it usually looks like:
 
-Your project usually have a file named `.gitmodules`.
-This file usually looks like that:
 ```
 [submodule "tools"]
 	path = tools
 	url = git@gitlab.com/group/tools.git
 ```
 
-Before 8.12 you had to do a multiple workarounds (ex. [SSH keys](../ssh_keys/README.md))
-in order to access the sources of `gitlab.com/group/tools`.
-
-GitLab 8.12 uses your permissions to evaluate what a CI build can access.
-More information about how this system works can be found here: [Build permissions model](../../user/permissions.md#builds-permissions).
+> **Note:**
+If you are not using GitLab 8.12 or higher, you would need to work your way
+around this issue in order to access the sources of `gitlab.com/group/tools`
+(e.g., use [SSH keys](../ssh_keys/README.md)).
+>
+With GitLab 8.12 onward, your permissions are used to evaluate what a CI build
+can access. More information about how this system works can be found in the
+[Build permissions model](../../user/permissions.md#builds-permissions).
 
-To make use of a new changes you have to update your `.gitmodules` file to use a relative URL.
+To make use of the new changes you have to update your `.gitmodules` file to
+use a relative URL.
 
 Let's consider the following example:
 
-1. Your project is located at https://gitlab.com/secret-group/my-project,
-1. To checkout your sources you usually use a SSH address: `git@gitlab.com:secret-group/my-project.git`,
-1. Your project depend on https://gitlab.com/group/tools,
+1. Your project is located at `https://gitlab.com/secret-group/my-project`.
+1. To checkout your sources you usually use an SSH address like
+   `git@gitlab.com:secret-group/my-project.git`.
+1. Your project depends on `https://gitlab.com/group/tools`.
 1. You have the `.gitmodules` file with above content.
 
-Since that you can use a relative URLs for your `.gitmodules` configuration
-it easily allows you to use an HTTP cloning for all your CI builds,
-and SSH clonning for all your local checkouts.
+Since Git allows the usage of relative URLs for your `.gitmodules` configuration,
+this easily allows you to use HTTP for cloning all your CI builds and SSH
+for all your local checkouts.
 
-If you change the `url` of your `tools` dependency:
-```
-git@gitlab.com/group/tools.git => ../../group/tools.git
-```
+For example, if you change the `url` of your `tools` dependency, from
+`git@gitlab.com/group/tools.git` to `../../group/tools.git`, this will instruct
+Git to automatically deduce the URL that should be used when cloning sources.
+Whether you use HTTP or SSH, Git will use that same channel and it will allow
+to make all your CI builds use HTTPS (because GitLab CI uses HTTPS for cloning
+your sources), and all your local clones will continue using SSH.
 
-It will instruct GIT to automatically deduce a URL that should be used when cloning sources.
-Whether you used a HTTP or SSH it will instruct GIT to use the same channel.
-And this will allow to make all your CI builds to use HTTPS (because GitLab CI uses HTTPS for cloning your sources),
-and all your local clones will continue using SSH.
+Given the above explanation, your `.gitmodules` file should eventually look
+like this:
 
-Given the above explanation, your `.gitmodules` file should look like this:
 ```
 [submodule "tools"]
 	path = tools
 	url = ../../group/tools.git
 ```
 
-However, you have to instruct GitLab CI to clone your submodules as this is not done automatically.
-You can achieve that by adding a `before_script` section to your `.gitlab-ci.yml` with `git submodule` command:
+However, you have to explicitly tell GitLab CI to clone your submodules as this
+is not done automatically. You can achieve that by adding a `before_script`
+section to your `.gitlab-ci.yml`:
 
 ```
 before_script:
@@ -72,10 +228,12 @@ test:
     - run-my-tests
 ```
 
-This will make GitLab CI to initialize (fetch) and update (checkout) all your submodules recursively.
+This will make GitLab CI initialize (fetch) and update (checkout) all your
+submodules recursively.
 
-It can happen that your environment or your Docker Image does not have a git installed.
-You have to either ask your Administrator or install the missing dependency yourself:
+In case your environment or your Docker image doesn't have Git installed,
+you have to either ask your Administrator or install the missing dependency
+yourself:
 
 ```
 # Debian / Ubuntu
@@ -97,17 +255,21 @@ before_script:
 
 ### Container Registry
 
-With the update permission model we also extended support for accessing Container Registries for private projects.
+With the update permission model we also extended the support for accessing
+Container Registries for private projects.
+
+> **Note:**
+As GitLab Runner 1.6 doesn't yet incorporate the introduced changes for
+permissions, this makes the `image:` directive to not work with private projects
+automatically. The manual configuration by an Administrator is required to use
+private images. We plan to remove that limitation in one of the upcoming releases.
 
-> Note: As of 1.6 the GitLab Runner doesn't yet incorporate the introduced changes for permissions.
-> This makes a `image:` to not work with private projects automatically.
-> The manual configuration by Administrator is required to use private images.
-> We plan to remove that limitation in one of the upcoming releases.
+Your builds can access all container images that you would normally have access
+to. The only implication is that you can push to the Container Registry of the
+project for which the build is triggered.
 
-Your builds can access all container images that you would normally have access to.
-The only implication is that you can push to Container Registry of project for which the build is triggered.
+This is how an example usage can look like:
 
-This is how the example usage can look like:
 ```
 test:
   script:
@@ -115,3 +277,5 @@ test:
     - docker pull $CI_REGISTRY/group/other-project:latest
     - docker run $CI_REGISTRY/group/other-project:latest
 ```
+
+[git-scm]: https://git-scm.com/book/en/v2/Git-Tools-Submodules
diff --git a/doc/user/permissions.md b/doc/user/permissions.md
index eb5bb43bb55..d1156127a00 100644
--- a/doc/user/permissions.md
+++ b/doc/user/permissions.md
@@ -147,49 +147,6 @@ GitLab 8.12 has completely redesigned build permission system.
 You can find all discussion and all our concerns when choosing the current approach:
 https://gitlab.com/gitlab-org/gitlab-ce/issues/18994
 
-We decided that builds permission should be tightly integrated with a permission
-of a user who is triggering a build.
-
-The reason to do it like that:
-
-- We already have permission system in place: group and project membership of users,
-- We already fully know who is triggering a build (using git push, using web, executing triggers),
-- We already know what user is allowed to do,
-- We use the user permission for builds that are triggered by him,
-- This opens us a lot of possibilities to further enforce user permissions, like:
-  allowing only specific users to access runners, secure variables and environments,
-- It is simple and convenient, that your build can access to everything that you have access to,
-- We choose to short living unique tokens, granting access for time of the build,
-
-Currently, any build that is triggered by the user, it's also signed with his permissions.
-When user do `git push` or changes files through web (**the pusher**),
-we will usually create a new Pipeline.
-The Pipeline will be signed as created be the pusher.
-Any build created in this pipeline will have the permissions of **the pusher**.
-
-This allows us to make it really easy to evaluate access for all dependent projects,
-container images that the pusher would have access too.
-The permission is granted only for time that build is running.
-The access is revoked after the build is finished.
-
-It is important to note that we have a few types of Users:
-
-- Administrators: CI builds created by Administrators would not have access to all GitLab projects,
-  but only to projects and container images of projects that the user is a member of or that are either public, or internal,
-
-- External users: CI builds created by external users will have access only to projects to which user has at least reporter access,
-  this rules out accessing all internal projects by default,
-
-This allows us to make the CI and permission system more trustable.
-Let's consider the following scenario:
-
-1. You are an employee of the company. Your company have number of internal tool repositories.
-   You have multiple CI builds that make use of this repositories.
-
-2. You invite a new user, a visitor, the external user. CI builds created by that user do not have access to internal repositories,
-   because user also doesn't have the access from within GitLab. You as an employee have to grant explicit access for this user.
-   This allows us to prevent from accidental data leakage.
-
 ### Build privileges
 
 This table shows granted privileges for builds triggered by specific types of users:
@@ -212,71 +169,3 @@ This table shows granted privileges for builds triggered by specific types of us
 [^3]: Only if user is not external one.
 [^4]: Only if user is a member of the project.
 
-### Build token
-
-The above gives a question about trustability of build token.
-Unique build token is generated for each project.
-This build token allows to access all projects that would be normally accessible
-to the user creating that build.
-
-We try to make sure that this token doesn't leak.
-We do that by:
-1. Securing all API endpoints to not expose the build token,
-1. Masking the build token from build logs,
-1. Allowing to use the build token only when build is running,
-
-However, this brings a question about runners security.
-To make sure that this token doesn't leak you also make sure that you configure
-your runners in most secure possible way, by avoiding using this configurations:
-1. Any usage of `privileged` mode if the machines are re-used is risky,
-1. Using `shell` executor,
-
-By using in-secure GitLab Runner configuration you allow the rogue developers
-to steal the tokens of other builds.
-
-### Debugging problems
-
-It can happen that some of the users will complain that CI builds do fail for them.
-
-It is most likely that your project access other projects sources,
-and the user doesn't have the permissions.
-In the build log look for information about 403 or forbidden access.
-
-You then as Administrator can verify that the user is a member of the group or project,
-and you when impersonated as the user can retry a failing build
-on behalf of the user to verify that everything is correct.
-
-### Before 8.12
-
-In versions before 8.12 all CI builds would use runners token to checkout project sources.
-
-The project runners token was a token that you would find in
-[CI/CD Pipelines](https://gitlab.com/my-group/my-project/pipelines/settings).
-
-The project runners token was used for registering new specific runners assigned to project
-and to checkout project sources.
-
-The project runners token could also be used to use GitLab Container Registry for that project,
-allowing to pull and push Docker images from within CI build.
-
-This token was limited to access only that project.
-
-GitLab would create an special checkout URL:
-```
-https://gitlab-ci-token:<project-runners-token>/gitlab.com/gitlab-org/gitlab-ce.git
-```
-
-User could also use in his CI builds all docker related commands
-to interact with GitLab Container Registry:
-```
-docker login -u gitlab-ci-token -p $CI_BUILD_TOKEN registry.gitlab.com
-```
-
-Using single token had multiple security implications:
-
-- Token would be readable to anyone who has developer access to project who could run CI builds,
-  allowing to register any specific runner for a project,
-- Token would allow to access only project sources,
-  forbidding to accessing any other projects,
-- Token was not expiring, and multi-purpose: used for checking out sources,
-  for registering specific runners and for accessing project's container registry with read-write permissions