diff options
Diffstat (limited to 'doc')
218 files changed, 2578 insertions, 639 deletions
diff --git a/doc/README.md b/doc/README.md index 5eabc126b95..d4119d35162 100644 --- a/doc/README.md +++ b/doc/README.md @@ -1,5 +1,6 @@ --- toc: false +comments: false --- # GitLab Documentation @@ -170,6 +171,7 @@ have access to GitLab administration tools and settings. - [GitLab performance monitoring with InfluxDB](administration/monitoring/performance/introduction.md): Configure GitLab and InfluxDB for measuring performance metrics. - [GitLab performance monitoring with Prometheus](administration/monitoring/prometheus/index.md): Configure GitLab and Prometheus for measuring performance metrics. - [Monitoring uptime](user/admin_area/monitoring/health_check.md): Check the server status using the health check endpoint. +- [Monitoring GitHub imports](administration/monitoring/github_imports.md) ### Performance diff --git a/doc/administration/auth/README.md b/doc/administration/auth/README.md index 13bd501e397..ee9b9a9466a 100644 --- a/doc/administration/auth/README.md +++ b/doc/administration/auth/README.md @@ -1,3 +1,7 @@ +--- +comments: false +--- + # Authentication and Authorization GitLab integrates with the following external authentication and authorization diff --git a/doc/administration/high_availability/README.md b/doc/administration/high_availability/README.md index 4d3be0ab8f6..a88e67bfeb5 100644 --- a/doc/administration/high_availability/README.md +++ b/doc/administration/high_availability/README.md @@ -53,7 +53,9 @@ or in different cloud availability zones. > **Note:** GitLab recommends against choosing this HA method because of the complexity of managing DRBD and crafting automatic failover. This is - *compatible* with GitLab, but not officially *supported*. + *compatible* with GitLab, but not officially *supported*. If you are + an EE customer, support will help you with GitLab related problems, but if the + root cause is identified as DRBD, we will not troubleshoot further. Components/Servers Required: 2 servers/virtual machines (one active/one passive) diff --git a/doc/administration/integration/plantuml.md b/doc/administration/integration/plantuml.md index 652ca9cf454..93c3642a1f1 100644 --- a/doc/administration/integration/plantuml.md +++ b/doc/administration/integration/plantuml.md @@ -56,29 +56,34 @@ that, login with an Admin account and do following: With PlantUML integration enabled and configured, we can start adding diagrams to our AsciiDoc snippets, wikis and repos using delimited blocks: -``` -[plantuml, format="png", id="myDiagram", width="200px"] --- -Bob->Alice : hello -Alice -> Bob : Go Away --- -``` +- **Markdown** + + ```plantuml + Bob -> Alice : hello + Alice -> Bob : Go Away + ``` -And in Markdown using fenced code blocks: +- **AsciiDoc** - ```plantuml - Bob -> Alice : hello + ``` + [plantuml, format="png", id="myDiagram", width="200px"] + -- + Bob->Alice : hello Alice -> Bob : Go Away + -- ``` -And in reStructuredText using a directive: +- **reStructuredText** -``` -.. plantuml:: + ``` + .. plantuml:: + :caption: Caption with **bold** and *italic* - Bob -> Alice: hello - Alice -> Bob: Go Away -``` + Bob -> Alice: hello + Alice -> Bob: Go Away + ``` + + You can also use the `uml::` directive for compatibility with [sphinxcontrib-plantuml](https://pypi.python.org/pypi/sphinxcontrib-plantuml), but please note that we currently only support the `caption` option. The above blocks will be converted to an HTML img tag with source pointing to the PlantUML instance. If the PlantUML server is correctly configured, this should diff --git a/doc/administration/logs.md b/doc/administration/logs.md index c9ed2d84ccb..debaa2330d0 100644 --- a/doc/administration/logs.md +++ b/doc/administration/logs.md @@ -192,4 +192,13 @@ installations from source. It logs information whenever a [repository check is run][repocheck] on a project. +## Reconfigure Logs + +Reconfigure log files live in `/var/log/gitlab/reconfigure` for Omnibus GitLab +packages. Installations from source don't have reconfigure logs. A reconfigure log +is populated whenever `gitlab-ctl reconfigure` is run manually or as part of an upgrade. + +Reconfigure logs files are named according to the UNIX timestamp of when the reconfigure +was initiated, such as `1509705644.log` + [repocheck]: repository_checks.md diff --git a/doc/administration/monitoring/github_imports.md b/doc/administration/monitoring/github_imports.md new file mode 100644 index 00000000000..5592e0a9e9a --- /dev/null +++ b/doc/administration/monitoring/github_imports.md @@ -0,0 +1,101 @@ +# Monitoring GitHub imports + +>**Note:** +Available since [GitLab 10.2][14731]. + +The GitHub importer exposes various Prometheus metrics that you can use to +monitor the health and progress of the importer. + +## Import Duration Times + +| Name | Type | +|------------------------------------------|-----------| +| `github_importer_total_duration_seconds` | histogram | + +This metric tracks the total time spent (in seconds) importing a project (from +project creation until the import process finishes), for every imported project. + +The name of the project is stored in the `project` label in the format +`namespace/name` (e.g. `gitlab-org/gitlab-ce`). + +## Number of imported projects + +| Name | Type | +|-------------------------------------|---------| +| `github_importer_imported_projects` | counter | + +This metric tracks the total number of projects imported over time. This metric +does not expose any labels. + +## Number of GitHub API calls + +| Name | Type | +|---------------------------------|---------| +| `github_importer_request_count` | counter | + +This metric tracks the total number of GitHub API calls performed over time, for +all projects. This metric does not expose any labels. + +## Rate limit errors + +| Name | Type | +|-----------------------------------|---------| +| `github_importer_rate_limit_hits` | counter | + +This metric tracks the number of times we hit the GitHub rate limit, for all +projects. This metric does not expose any labels. + +## Number of imported issues + +| Name | Type | +|-----------------------------------|---------| +| `github_importer_imported_issues` | counter | + +This metric tracks the number of imported issues across all projects. + +The name of the project is stored in the `project` label in the format +`namespace/name` (e.g. `gitlab-org/gitlab-ce`). + +## Number of imported pull requests + +| Name | Type | +|------------------------------------------|---------| +| `github_importer_imported_pull_requests` | counter | + +This metric tracks the number of imported pull requests across all projects. + +The name of the project is stored in the `project` label in the format +`namespace/name` (e.g. `gitlab-org/gitlab-ce`). + +## Number of imported comments + +| Name | Type | +|----------------------------------|---------| +| `github_importer_imported_notes` | counter | + +This metric tracks the number of imported comments across all projects. + +The name of the project is stored in the `project` label in the format +`namespace/name` (e.g. `gitlab-org/gitlab-ce`). + +## Number of imported pull request review comments + +| Name | Type | +|---------------------------------------|---------| +| `github_importer_imported_diff_notes` | counter | + +This metric tracks the number of imported comments across all projects. + +The name of the project is stored in the `project` label in the format +`namespace/name` (e.g. `gitlab-org/gitlab-ce`). + +## Number of imported repositories + +| Name | Type | +|-----------------------------------------|---------| +| `github_importer_imported_repositories` | counter | + +This metric tracks the number of imported repositories across all projects. This +metric does not expose any labels. + +[14731]: https://gitlab.com/gitlab-org/gitlab-ce/merge_requests/14731 diff --git a/doc/administration/operations/sidekiq_memory_killer.md b/doc/administration/operations/sidekiq_memory_killer.md index b5e78348989..cbffd883774 100644 --- a/doc/administration/operations/sidekiq_memory_killer.md +++ b/doc/administration/operations/sidekiq_memory_killer.md @@ -28,7 +28,7 @@ The MemoryKiller is controlled using environment variables. delayed shutdown is triggered. The default value for Omnibus packages is set [in the omnibus-gitlab repository](https://gitlab.com/gitlab-org/omnibus-gitlab/blob/master/files/gitlab-cookbooks/gitlab/attributes/default.rb). -- `SIDEKIQ_MEMORY_KILLER_GRACE_TIME`: defaults 900 seconds (15 minutes). When +- `SIDEKIQ_MEMORY_KILLER_GRACE_TIME`: defaults to 900 seconds (15 minutes). When a shutdown is triggered, the Sidekiq process will keep working normally for another 15 minutes. - `SIDEKIQ_MEMORY_KILLER_SHUTDOWN_WAIT`: defaults to 30 seconds. When the grace @@ -36,5 +36,3 @@ The MemoryKiller is controlled using environment variables. Existing jobs get 30 seconds to finish. After that, the MemoryKiller tells Sidekiq to shut down, and an external supervision mechanism (e.g. Runit) must restart Sidekiq. -- `SIDEKIQ_MEMORY_KILLER_SHUTDOWN_SIGNAL`: defaults to `SIGKILL`. The name of - the final signal sent to the Sidekiq process when we want it to shut down. diff --git a/doc/administration/repository_storage_types.md b/doc/administration/repository_storage_types.md index fa882bbe28a..21184fed6e9 100644 --- a/doc/administration/repository_storage_types.md +++ b/doc/administration/repository_storage_types.md @@ -9,7 +9,7 @@ mapping structure from the projects URLs: * Project's repository: `#{namespace}/#{project_name}.git` * Project's wiki: `#{namespace}/#{project_name}.wiki.git` - + This structure made simple to migrate from existing solutions to GitLab and easy for Administrators to find where the repository is stored. @@ -25,7 +25,10 @@ Any change in the URL will need to be reflected on disk (when groups / users or of load in big installations, and can be even worst if they are using any type of network based filesystem. Last, for GitLab Geo, this storage type means we have to synchronize the disk state, replicate renames in the correct -order or we may end-up with wrong repository or missing data temporarily. +order or we may end-up with wrong repository or missing data temporarily. + +This pattern also exists in other objects stored in GitLab, like issue Attachments, GitLab Pages artifacts, +Docker Containers for the integrated Registry, etc. ## Hashed Storage @@ -59,11 +62,31 @@ you will never mistakenly restore a repository in the wrong project (considering ### How to migrate to Hashed Storage -In GitLab, go to **Admin > Settings**, find the **Repository Storage** section and select +In GitLab, go to **Admin > Settings**, find the **Repository Storage** section and select "_Create new projects using hashed storage paths_". - + To migrate your existing projects to the new storage type, check the specific [rake tasks]. [ce-28283]: https://gitlab.com/gitlab-org/gitlab-ce/issues/28283 [rake tasks]: raketasks/storage.md#migrate-existing-projects-to-hashed-storage [storage-paths]: repository_storage_types.md + +### Hashed Storage coverage + +We are incrementally moving every storable object in GitLab to the Hashed Storage pattern. You can check the current +coverage status below. + +Note that things stored in an S3 compatible endpoint will not have the downsides mentioned earlier, if they are not +prefixed with `#{namespace}/#{project_name}`, which is true for CI Cache and LFS Objects. + +| Storable Object | Legacy Storage | Hashed Storage | S3 Compatible | GitLab Version | +| --------------- | -------------- | -------------- | ------------- | -------------- | +| Repository | Yes | Yes | - | 10.0 | +| Attachments | Yes | Yes | - | 10.2 | +| Avatars | Yes | No | - | - | +| Pages | Yes | No | - | - | +| Docker Registry | Yes | No | - | - | +| CI Build Logs | No | No | - | - | +| CI Artifacts | No | No | Yes (EEP) | - | +| CI Cache | No | No | Yes | - | +| LFS Objects | Yes | No | Yes (EEP) | - | diff --git a/doc/administration/troubleshooting/debug.md b/doc/administration/troubleshooting/debug.md index 6f1356ddf8f..83a714810c1 100644 --- a/doc/administration/troubleshooting/debug.md +++ b/doc/administration/troubleshooting/debug.md @@ -141,7 +141,7 @@ separate Rails process to debug the issue: 1. Log in to your GitLab account. 1. Copy the URL that is causing problems (e.g. https://gitlab.com/ABC). -1. Obtain the private token for your user (Profile Settings -> Account). +1. Create a Personal Access Token for your user (Profile Settings -> Access Tokens). 1. Bring up the GitLab Rails console. For omnibus users, run: ``` @@ -163,6 +163,34 @@ separate Rails process to debug the issue: 1. In a new window, run `top`. It should show this ruby process using 100% CPU. Write down the PID. 1. Follow step 2 from the previous section on using gdb. +### GitLab: API is not accessible + +This often occurs when gitlab-shell attempts to request authorization via the +internal API (e.g., `http://localhost:8080/api/v4/internal/allowed`), and +something in the check fails. There are many reasons why this may happen: + +1. Timeout connecting to a database (e.g., PostgreSQL or Redis) +1. Error in Git hooks or push rules +1. Error accessing the repository (e.g., stale NFS handles) + +To diagnose this problem, try to reproduce the problem and then see if there +is a unicorn worker that is spinning via `top`. Try to use the `gdb` +techniques above. In addition, using `strace` may help isolate issues: + +```shell +strace -tt -T -f -s 1024 -p <PID of unicorn worker> -o /tmp/unicorn.txt +``` + +If you cannot isolate which Unicorn worker is the issue, try to run `strace` +on all the Unicorn workers to see where the `/internal/allowed` endpoint gets +stuck: + +```shell +ps auwx | grep unicorn | awk '{ print " -p " $2}' | xargs strace -tt -T -f -s 1024 -o /tmp/unicorn.txt +``` + +The output in `/tmp/unicorn.txt` may help diagnose the root cause. + # More information * [Debugging Stuck Ruby Processes](https://blog.newrelic.com/2013/04/29/debugging-stuck-ruby-processes-what-to-do-before-you-kill-9/) diff --git a/doc/api/README.md b/doc/api/README.md index 89ffe9d7868..f226716c3b5 100644 --- a/doc/api/README.md +++ b/doc/api/README.md @@ -50,7 +50,6 @@ following locations: - [Repository Files](repository_files.md) - [Runners](runners.md) - [Services](services.md) -- [Session](session.md) - [Settings](settings.md) - [Sidekiq metrics](sidekiq_metrics.md) - [System Hooks](system_hooks.md) @@ -86,27 +85,10 @@ API requests should be prefixed with `api` and the API version. The API version is defined in [`lib/api.rb`][lib-api-url]. For example, the root of the v4 API is at `/api/v4`. -For endpoints that require [authentication](#authentication), you need to pass -a `private_token` parameter via query string or header. If passed as a header, -the header name must be `PRIVATE-TOKEN` (uppercase and with a dash instead of -an underscore). - -Example of a valid API request: - -``` -GET /projects?private_token=9koXpg98eAheJpvBs5tK -``` - -Example of a valid API request using cURL and authentication via header: +Example of a valid API request using cURL: ```shell -curl --header "PRIVATE-TOKEN: 9koXpg98eAheJpvBs5tK" "https://gitlab.example.com/api/v4/projects" -``` - -Example of a valid API request using cURL and authentication via a query string: - -```shell -curl "https://gitlab.example.com/api/v4/projects?private_token=9koXpg98eAheJpvBs5tK" +curl "https://gitlab.example.com/api/v4/projects" ``` The API uses JSON to serialize data. You don't need to specify `.json` at the @@ -114,15 +96,20 @@ end of an API URL. ## Authentication -Most API requests require authentication via a session cookie or token. For +Most API requests require authentication, or will only return public data when +authentication is not provided. For those cases where it is not required, this will be mentioned in the documentation for each individual endpoint. For example, the [`/projects/:id` endpoint](projects.md). -There are three types of access tokens available: +There are three ways to authenticate with the GitLab API: 1. [OAuth2 tokens](#oauth2-tokens) -1. [Private tokens](#private-tokens) 1. [Personal access tokens](#personal-access-tokens) +1. [Session cookie](#session-cookie) + +For admins who want to authenticate with the API as a specific user, or who want to build applications or scripts that do so, two options are available: +1. [Impersonation tokens](#impersonation-tokens) +2. [Sudo](#sudo) If authentication information is invalid or omitted, an error message will be returned with status code `401`: @@ -133,74 +120,84 @@ returned with status code `401`: } ``` -### Session cookie +### OAuth2 tokens -When signing in to GitLab as an ordinary user, a `_gitlab_session` cookie is -set. The API will use this cookie for authentication if it is present, but using -the API to generate a new session cookie is currently not supported. +You can use an [OAuth2 token](oauth2.md) to authenticate with the API by passing it in either the +`access_token` parameter or the `Authorization` header. -### OAuth2 tokens +Example of using the OAuth2 token in a parameter: -You can use an OAuth 2 token to authenticate with the API by passing it either in the -`access_token` parameter or in the `Authorization` header. +```shell +curl https://gitlab.example.com/api/v4/projects?access_token=OAUTH-TOKEN +``` -Example of using the OAuth2 token in the header: +Example of using the OAuth2 token in a header: ```shell curl --header "Authorization: Bearer OAUTH-TOKEN" https://gitlab.example.com/api/v4/projects ``` -Read more about [GitLab as an OAuth2 client](oauth2.md). +Read more about [GitLab as an OAuth2 provider](oauth2.md). -### Private tokens +### Personal access tokens -Private tokens provide full access to the GitLab API. Anyone with access to -them can interact with GitLab as if they were you. You can find or reset your -private token in your account page (`/profile/account`). +You can use a [personal access token][pat] to authenticate with the API by passing it in either the +`private_token` parameter or the `Private-Token` header. -For examples of usage, [read the basic usage section](#basic-usage). +Example of using the personal access token in a parameter: -### Personal access tokens +```shell +curl https://gitlab.example.com/api/v4/projects?private_token=9koXpg98eAheJpvBs5tK +``` + +Example of using the personal access token in a header: -Instead of using your private token which grants full access to your account, -personal access tokens could be a better fit because of their granular -permissions. +```shell +curl --header "Private-Token: 9koXpg98eAheJpvBs5tK" https://gitlab.example.com/api/v4/projects +``` -Once you have your token, pass it to the API using either the `private_token` -parameter or the `PRIVATE-TOKEN` header. For examples of usage, -[read the basic usage section](#basic-usage). +Read more about [personal access tokens][pat]. + +### Session cookie + +When signing in to the main GitLab application, a `_gitlab_session` cookie is +set. The API will use this cookie for authentication if it is present, but using +the API to generate a new session cookie is currently not supported. -[Read more about personal access tokens.][pat] +The primary user of this authentication method is the web frontend of GitLab itself, +which can use the API as the authenticated user to get a list of their projects, +for example, without needing to explicitly pass an access token. ### Impersonation tokens > [Introduced][ce-9099] in GitLab 9.0. Needs admin permissions. Impersonation tokens are a type of [personal access token][pat] -that can only be created by an admin for a specific user. +that can only be created by an admin for a specific user. They are a great fit +if you want to build applications or scripts that authenticate with the API as a specific user. -They are a better alternative to using the user's password/private token -or using the [Sudo](#sudo) feature which also requires the admin's password -or private token, since the password/token can change over time. Impersonation -tokens are a great fit if you want to build applications or tools which -authenticate with the API as a specific user. +They are an alternative to directly using the user's password or one of their +personal access tokens, and to using the [Sudo](#sudo) feature, since the user's (or admin's, in the case of Sudo) +password/token may not be known or may change over time. For more information, refer to the [users API](users.md#retrieve-user-impersonation-tokens) docs. -For examples of usage, [read the basic usage section](#basic-usage). +Impersonation tokens are used exactly like regular personal access tokens, and can be passed in either the +`private_token` parameter or the `Private-Token` header. ### Sudo > Needs admin permissions. All API requests support performing an API call as if you were another user, -provided your private token is from an administrator account. You need to pass -the `sudo` parameter either via query string or a header with an ID/username of +provided you are authenticated as an administrator with an OAuth or Personal Access Token that has the `sudo` scope. + +You need to pass the `sudo` parameter either via query string or a header with an ID/username of the user you want to perform the operation as. If passed as a header, the -header name must be `SUDO` (uppercase). +header name must be `Sudo`. -If a non administrative `private_token` is provided, then an error message will +If a non administrative access token is provided, an error message will be returned with status code `403`: ```json @@ -209,12 +206,23 @@ be returned with status code `403`: } ``` +If an access token without the `sudo` scope is provided, an error message will +be returned with status code `403`: + +```json +{ + "error": "insufficient_scope", + "error_description": "The request requires higher privileges than provided by the access token.", + "scope": "sudo" +} +``` + If the sudo user ID or username cannot be found, an error message will be returned with status code `404`: ```json { - "message": "404 Not Found: No user id or username for: <id/username>" + "message": "404 User with ID or username '123' Not Found" } ``` @@ -228,7 +236,7 @@ GET /projects?private_token=9koXpg98eAheJpvBs5tK&sudo=username ``` ```shell -curl --header "PRIVATE-TOKEN: 9koXpg98eAheJpvBs5tK" --header "SUDO: username" "https://gitlab.example.com/api/v4/projects" +curl --header "Private-Token: 9koXpg98eAheJpvBs5tK" --header "Sudo: username" "https://gitlab.example.com/api/v4/projects" ``` Example of a valid API call and a request using cURL with sudo request, @@ -239,7 +247,7 @@ GET /projects?private_token=9koXpg98eAheJpvBs5tK&sudo=23 ``` ```shell -curl --header "PRIVATE-TOKEN: 9koXpg98eAheJpvBs5tK" --header "SUDO: 23" "https://gitlab.example.com/api/v4/projects" +curl --header "Private-Token: 9koXpg98eAheJpvBs5tK" --header "Sudo: 23" "https://gitlab.example.com/api/v4/projects" ``` ## Status codes diff --git a/doc/api/custom_attributes.md b/doc/api/custom_attributes.md index 8b26f7093ab..91d1b0e1520 100644 --- a/doc/api/custom_attributes.md +++ b/doc/api/custom_attributes.md @@ -2,17 +2,22 @@ Every API call to custom attributes must be authenticated as administrator. +Custom attributes are currently available on users, groups, and projects, +which will be referred to as "resource" in this documentation. + ## List custom attributes -Get all custom attributes on a user. +Get all custom attributes on a resource. ``` GET /users/:id/custom_attributes +GET /groups/:id/custom_attributes +GET /projects/:id/custom_attributes ``` | Attribute | Type | Required | Description | | --------- | ---- | -------- | ----------- | -| `id` | integer | yes | The ID of a user | +| `id` | integer | yes | The ID of a resource | ```bash curl --header "PRIVATE-TOKEN: 9koXpg98eAheJpvBs5tK" https://gitlab.example.com/api/v4/users/42/custom_attributes @@ -35,15 +40,17 @@ Example response: ## Single custom attribute -Get a single custom attribute on a user. +Get a single custom attribute on a resource. ``` GET /users/:id/custom_attributes/:key +GET /groups/:id/custom_attributes/:key +GET /projects/:id/custom_attributes/:key ``` | Attribute | Type | Required | Description | | --------- | ---- | -------- | ----------- | -| `id` | integer | yes | The ID of a user | +| `id` | integer | yes | The ID of a resource | | `key` | string | yes | The key of the custom attribute | ```bash @@ -61,16 +68,18 @@ Example response: ## Set custom attribute -Set a custom attribute on a user. The attribute will be updated if it already exists, +Set a custom attribute on a resource. The attribute will be updated if it already exists, or newly created otherwise. ``` PUT /users/:id/custom_attributes/:key +PUT /groups/:id/custom_attributes/:key +PUT /projects/:id/custom_attributes/:key ``` | Attribute | Type | Required | Description | | --------- | ---- | -------- | ----------- | -| `id` | integer | yes | The ID of a user | +| `id` | integer | yes | The ID of a resource | | `key` | string | yes | The key of the custom attribute | | `value` | string | yes | The value of the custom attribute | @@ -89,15 +98,17 @@ Example response: ## Delete custom attribute -Delete a custom attribute on a user. +Delete a custom attribute on a resource. ``` DELETE /users/:id/custom_attributes/:key +DELETE /groups/:id/custom_attributes/:key +DELETE /projects/:id/custom_attributes/:key ``` | Attribute | Type | Required | Description | | --------- | ---- | -------- | ----------- | -| `id` | integer | yes | The ID of a user | +| `id` | integer | yes | The ID of a resource | | `key` | string | yes | The key of the custom attribute | ```bash diff --git a/doc/api/environments.md b/doc/api/environments.md index e8deb3e07e9..6e20781f51a 100644 --- a/doc/api/environments.md +++ b/doc/api/environments.md @@ -36,7 +36,7 @@ Creates a new environment with the given name and external_url. It returns `201` if the environment was successfully created, `400` for wrong parameters. ``` -POST /projects/:id/environment +POST /projects/:id/environments ``` | Attribute | Type | Required | Description | diff --git a/doc/api/groups.md b/doc/api/groups.md index 99d200c9c93..c1b5737c247 100644 --- a/doc/api/groups.md +++ b/doc/api/groups.md @@ -9,13 +9,13 @@ Parameters: | Attribute | Type | Required | Description | | --------- | ---- | -------- | ----------- | -| `skip_groups` | array of integers | no | Skip the group IDs passes | -| `all_available` | boolean | no | Show all the groups you have access to | -| `search` | string | no | Return list of authorized groups matching the search criteria | +| `skip_groups` | array of integers | no | Skip the group IDs passed | +| `all_available` | boolean | no | Show all the groups you have access to (defaults to `false` for authenticated users) | +| `search` | string | no | Return the list of authorized groups matching the search criteria | | `order_by` | string | no | Order groups by `name` or `path`. Default is `name` | | `sort` | string | no | Order groups in `asc` or `desc` order. Default is `asc` | | `statistics` | boolean | no | Include group statistics (admins only) | -| `owned` | boolean | no | Limit by groups owned by the current user | +| `owned` | boolean | no | Limit to groups owned by the current user | ``` GET /groups @@ -74,6 +74,55 @@ GET /groups?statistics=true You can search for groups by name or path, see below. +You can filter by [custom attributes](custom_attributes.md) with: + +``` +GET /groups?custom_attributes[key]=value&custom_attributes[other_key]=other_value +``` + +## List a groups's subgroups + +> [Introduced][ce-15142] in GitLab 10.3. + +Get a list of visible direct subgroups in this group. +When accessed without authentication, only public groups are returned. + +Parameters: + +| Attribute | Type | Required | Description | +| --------- | ---- | -------- | ----------- | +| `id` | integer/string | yes | The ID or [URL-encoded path of the group](README.md#namespaced-path-encoding) of the parent group | +| `skip_groups` | array of integers | no | Skip the group IDs passed | +| `all_available` | boolean | no | Show all the groups you have access to (defaults to `false` for authenticated users) | +| `search` | string | no | Return the list of authorized groups matching the search criteria | +| `order_by` | string | no | Order groups by `name` or `path`. Default is `name` | +| `sort` | string | no | Order groups in `asc` or `desc` order. Default is `asc` | +| `statistics` | boolean | no | Include group statistics (admins only) | +| `owned` | boolean | no | Limit to groups owned by the current user | + +``` +GET /groups/:id/subgroups +``` + +```json +[ + { + "id": 1, + "name": "Foobar Group", + "path": "foo-bar", + "description": "An interesting group", + "visibility": "public", + "lfs_enabled": true, + "avatar_url": "http://gitlab.example.com/uploads/group/avatar/1/foo.jpg", + "web_url": "http://gitlab.example.com/groups/foo-bar", + "request_access_enabled": false, + "full_name": "Foobar Group", + "full_path": "foo-bar", + "parent_id": 123 + } +] +``` + ## List a group's projects Get a list of projects in this group. When accessed without authentication, only @@ -466,3 +515,5 @@ And to switch pages add: ``` /groups?per_page=100&page=2 ``` + +[ce-15142]: https://gitlab.com/gitlab-org/gitlab-ce/merge_requests/15142 diff --git a/doc/api/merge_requests.md b/doc/api/merge_requests.md index 50a971102fb..b2e4b6d0955 100644 --- a/doc/api/merge_requests.md +++ b/doc/api/merge_requests.md @@ -15,6 +15,11 @@ given state (`opened`, `closed`, or `merged`) or all of them (`all`). The pagination parameters `page` and `per_page` can be used to restrict the list of merge requests. +**Note**: the `changes_count` value in the response is a string, not an +integer. This is because when an MR has too many changes to display and store, +it will be capped at 1,000. In that case, the API will return the string +`"1000+"` for the changes count. + ``` GET /merge_requests GET /merge_requests?state=opened @@ -53,6 +58,8 @@ Parameters: "project_id": 3, "title": "test1", "state": "opened", + "created_at": "2017-04-29T08:46:00Z", + "updated_at": "2017-04-29T08:46:00Z", "upvotes": 0, "downvotes": 0, "author": { @@ -92,6 +99,7 @@ Parameters: "sha": "8888888888888888888888888888888888888888", "merge_commit_sha": null, "user_notes_count": 1, + "changes_count": "1", "should_remove_source_branch": true, "force_remove_source_branch": false, "web_url": "http://example.com/example/example/merge_requests/1", @@ -130,6 +138,11 @@ will be the same. In the case of a merge request from a fork, `target_project_id` and `project_id` will be the same and `source_project_id` will be the fork project's ID. +**Note**: the `changes_count` value in the response is a string, not an +integer. This is because when an MR has too many changes to display and store, +it will be capped at 1,000. In that case, the API will return the string +`"1000+"` for the changes count. + Parameters: | Attribute | Type | Required | Description | @@ -159,6 +172,8 @@ Parameters: "project_id": 3, "title": "test1", "state": "opened", + "created_at": "2017-04-29T08:46:00Z", + "updated_at": "2017-04-29T08:46:00Z", "upvotes": 0, "downvotes": 0, "author": { @@ -198,6 +213,7 @@ Parameters: "sha": "8888888888888888888888888888888888888888", "merge_commit_sha": null, "user_notes_count": 1, + "changes_count": "1", "should_remove_source_branch": true, "force_remove_source_branch": false, "web_url": "http://example.com/example/example/merge_requests/1", @@ -234,6 +250,8 @@ Parameters: "project_id": 3, "title": "test1", "state": "merged", + "created_at": "2017-04-29T08:46:00Z", + "updated_at": "2017-04-29T08:46:00Z", "upvotes": 0, "downvotes": 0, "author": { @@ -274,6 +292,7 @@ Parameters: "sha": "8888888888888888888888888888888888888888", "merge_commit_sha": "9999999999999999999999999999999999999999", "user_notes_count": 1, + "changes_count": "1", "should_remove_source_branch": true, "force_remove_source_branch": false, "web_url": "http://example.com/example/example/merge_requests/1", @@ -386,6 +405,7 @@ Parameters: "sha": "8888888888888888888888888888888888888888", "merge_commit_sha": null, "user_notes_count": 1, + "changes_count": "1", "should_remove_source_branch": true, "force_remove_source_branch": false, "web_url": "http://example.com/example/example/merge_requests/1", @@ -480,6 +500,7 @@ POST /projects/:id/merge_requests "sha": "8888888888888888888888888888888888888888", "merge_commit_sha": null, "user_notes_count": 0, + "changes_count": "1", "should_remove_source_branch": true, "force_remove_source_branch": false, "web_url": "http://example.com/example/example/merge_requests/1", @@ -565,6 +586,7 @@ Must include at least one non-required attribute from above. "sha": "8888888888888888888888888888888888888888", "merge_commit_sha": null, "user_notes_count": 1, + "changes_count": "1", "should_remove_source_branch": true, "force_remove_source_branch": false, "web_url": "http://example.com/example/example/merge_requests/1", @@ -670,6 +692,7 @@ Parameters: "sha": "8888888888888888888888888888888888888888", "merge_commit_sha": "9999999999999999999999999999999999999999", "user_notes_count": 1, + "changes_count": "1", "should_remove_source_branch": true, "force_remove_source_branch": false, "web_url": "http://example.com/example/example/merge_requests/1", @@ -747,6 +770,7 @@ Parameters: "sha": "8888888888888888888888888888888888888888", "merge_commit_sha": null, "user_notes_count": 1, + "changes_count": "1", "should_remove_source_branch": true, "force_remove_source_branch": false, "web_url": "http://example.com/example/example/merge_requests/1", @@ -822,7 +846,8 @@ Example response when the GitLab issue tracker is used: "created_at" : "2016-01-04T15:31:51.081Z", "iid" : 6, "labels" : [], - "user_notes_count": 1 + "user_notes_count": 1, + "changes_count": "1" }, ] ``` @@ -1077,6 +1102,7 @@ Example response: "sha": "8888888888888888888888888888888888888888", "merge_commit_sha": null, "user_notes_count": 7, + "changes_count": "1", "should_remove_source_branch": true, "force_remove_source_branch": false, "web_url": "http://example.com/example/example/merge_requests/1" diff --git a/doc/api/pages_domains.md b/doc/api/pages_domains.md index 51962595e33..50685f335f7 100644 --- a/doc/api/pages_domains.md +++ b/doc/api/pages_domains.md @@ -4,6 +4,31 @@ Endpoints for connecting custom domain(s) and TLS certificates in [GitLab Pages] The GitLab Pages feature must be enabled to use these endpoints. Find out more about [administering](../administration/pages/index.md) and [using](../user/project/pages/index.md) the feature. +## List all pages domains + +Get a list of all pages domains. The user must have admin permissions. + +```http +GET /pages/domains +``` + +```bash +curl --header "PRIVATE-TOKEN: 9koXpg98eAheJpvBs5tK" https://gitlab.example.com/api/v4/pages/domains +``` + +```json +[ + { + "domain": "ssl.domain.example", + "url": "https://ssl.domain.example", + "certificate": { + "expired": false, + "expiration": "2020-04-12T14:32:00.000Z" + } + } +] +``` + ## List pages domains Get a list of project pages domains. The user must have permissions to view pages domains. diff --git a/doc/api/pipelines.md b/doc/api/pipelines.md index 890945cfc7e..a6631cab8c3 100644 --- a/doc/api/pipelines.md +++ b/doc/api/pipelines.md @@ -57,7 +57,7 @@ GET /projects/:id/pipelines/:pipeline_id | `pipeline_id` | integer | yes | The ID of a pipeline | ``` -curl --header "PRIVATE-TOKEN: 9koXpg98eAheJpvBs5tK" "https://gitlab.example.com/api/v4/projects/1/pipeline/46" +curl --header "PRIVATE-TOKEN: 9koXpg98eAheJpvBs5tK" "https://gitlab.example.com/api/v4/projects/1/pipelines/46" ``` Example of response diff --git a/doc/api/projects.md b/doc/api/projects.md index 07331d05231..5a403f7593a 100644 --- a/doc/api/projects.md +++ b/doc/api/projects.md @@ -192,6 +192,12 @@ GET /projects ] ``` +You can filter by [custom attributes](custom_attributes.md) with: + +``` +GET /projects?custom_attributes[key]=value&custom_attributes[other_key]=other_value +``` + ## List user projects Get a list of visible projects for the given user. When accessed without diff --git a/doc/api/services.md b/doc/api/services.md index 6c8f196fd5c..08df26db3ec 100644 --- a/doc/api/services.md +++ b/doc/api/services.md @@ -490,6 +490,41 @@ Remove all previously JIRA settings from a project. DELETE /projects/:id/services/jira ``` +## Kubernetes + +Kubernetes / Openshift integration + +### Create/Edit Kubernetes service + +Set Kubernetes service for a project. + +``` +PUT /projects/:id/services/kubernetes +``` + +Parameters: + +- `namespace` (**required**) - The Kubernetes namespace to use +- `api_url` (**required**) - The URL to the Kubernetes cluster API, e.g., https://kubernetes.example.com +- `token` (**required**) - The service token to authenticate against the Kubernetes cluster with +- `ca_pem` (optional) - A custom certificate authority bundle to verify the Kubernetes cluster with (PEM format) + +### Delete Kubernetes service + +Delete Kubernetes service for a project. + +``` +DELETE /projects/:id/services/kubernetes +``` + +### Get Kubernetes service settings + +Get Kubernetes service settings for a project. + +``` +GET /projects/:id/services/kubernetes +``` + ## Slack slash commands Ability to receive slash commands from a Slack chat instance. @@ -572,7 +607,7 @@ Parameters: | Attribute | Type | Required | Description | | --------- | ---- | -------- | ----------- | | `token` | string | yes | The Mattermost token | - +| `username` | string | no | The username to use to post the message | ### Delete Mattermost slash command service @@ -582,6 +617,40 @@ Delete Mattermost slash command service for a project. DELETE /projects/:id/services/mattermost-slash-commands ``` +## Packagist + +Update your project on Packagist, the main Composer repository, when commits or tags are pushed to GitLab. + +### Create/Edit Packagist service + +Set Packagist service for a project. + +``` +PUT /projects/:id/services/packagist +``` + +Parameters: + +- `username` (**required**) +- `token` (**required**) +- `server` (optional) + +### Delete Packagist service + +Delete Packagist service for a project. + +``` +DELETE /projects/:id/services/packagist +``` + +### Get Packagist service settings + +Get Packagist service settings for a project. + +``` +GET /projects/:id/services/packagist +``` + ## Pipeline-Emails Get emails for GitLab CI pipelines. diff --git a/doc/api/session.md b/doc/api/session.md deleted file mode 100644 index b97e26f34a2..00000000000 --- a/doc/api/session.md +++ /dev/null @@ -1,55 +0,0 @@ -# Session API - ->**Deprecation notice:** -Starting in GitLab 8.11, this feature has been **disabled** for users with -[two-factor authentication][2fa] turned on. These users can access the API -using [personal access tokens] instead. - -You can login with both GitLab and LDAP credentials in order to obtain the -private token. - -``` -POST /session -``` - -| Attribute | Type | Required | Description | -| ---------- | ------- | -------- | -------- | -| `login` | string | yes | The username of the user| -| `email` | string | yes if login is not provided | The email of the user | -| `password` | string | yes | The password of the user | - -```bash -curl --request POST "https://gitlab.example.com/api/v4/session?login=john_smith&password=strongpassw0rd" -``` - -Example response: - -```json -{ - "name": "John Smith", - "username": "john_smith", - "id": 32, - "state": "active", - "avatar_url": null, - "created_at": "2015-01-29T21:07:19.440Z", - "is_admin": true, - "bio": null, - "skype": "", - "linkedin": "", - "twitter": "", - "website_url": "", - "email": "john@example.com", - "theme_id": 1, - "color_scheme_id": 1, - "projects_limit": 10, - "current_sign_in_at": "2015-07-07T07:10:58.392Z", - "identities": [], - "can_create_group": true, - "can_create_project": true, - "two_factor_enabled": false, - "private_token": "9koXpg98eAheJpvBs5tK" -} -``` - -[2fa]: ../user/profile/account/two_factor_authentication.md -[personal access tokens]: ../user/profile/personal_access_tokens.md diff --git a/doc/api/settings.md b/doc/api/settings.md index 4e24e4bbfc3..b27220f57f4 100644 --- a/doc/api/settings.md +++ b/doc/api/settings.md @@ -79,7 +79,7 @@ PUT /application/settings | `clientside_sentry_enabled` | boolean | no | Enable Sentry error reporting for the client side | | `container_registry_token_expire_delay` | integer | no | Container Registry token duration in minutes | | `default_artifacts_expire_in` | string | no | Set the default expiration time for each job's artifacts | -| `default_branch_protection` | integer | no | Determine if developers can push to master. Can take `0` _(not protected, both developers and masters can push new commits, force push or delete the branch)_, `1` _(partially protected, developers can push new commits, but cannot force push or delete the branch, masters can do anything)_ or `2` _(fully protected, developers cannot push new commits, force push or delete the branch, masters can do anything)_ as a parameter. Default is `2`. | +| `default_branch_protection` | integer | no | Determine if developers can push to master. Can take `0` _(not protected, both developers and masters can push new commits, force push, or delete the branch)_, `1` _(partially protected, developers and masters can push new commits, but cannot force push or delete the branch)_ or `2` _(fully protected, developers cannot push new commits, but masters can; no-one can force push or delete the branch)_ as a parameter. Default is `2`. | | `default_group_visibility` | string | no | What visibility level new groups receive. Can take `private`, `internal` and `public` as a parameter. Default is `private`. | | `default_project_visibility` | string | no | What visibility level new projects receive. Can take `private`, `internal` and `public` as a parameter. Default is `private`. | | `default_projects_limit` | integer | no | Project limit per user. Default is `100000` | diff --git a/doc/api/users.md b/doc/api/users.md index 1643c584244..478d747a50d 100644 --- a/doc/api/users.md +++ b/doc/api/users.md @@ -297,6 +297,7 @@ Parameters: - `location` (optional) - User's location - `admin` (optional) - User is admin - true or false (default) - `can_create_group` (optional) - User can create groups - true or false +- `skip_reconfirmation` (optional) - Skip reconfirmation - true or false (default) - `external` (optional) - Flags the user as external - true or false(default) - `avatar` (optional) - Image file for user's avatar @@ -410,8 +411,7 @@ GET /user "can_create_group": true, "can_create_project": true, "two_factor_enabled": true, - "external": false, - "private_token": "dd34asd13as" + "external": false } ``` diff --git a/doc/ci/README.md b/doc/ci/README.md index ec0ddfbea75..12404eddbe2 100644 --- a/doc/ci/README.md +++ b/doc/ci/README.md @@ -1,3 +1,7 @@ +--- +comments: false +--- + # GitLab Continuous Integration (GitLab CI)  diff --git a/doc/ci/docker/README.md b/doc/ci/docker/README.md index 99669a9272a..b0e01d74f7e 100644 --- a/doc/ci/docker/README.md +++ b/doc/ci/docker/README.md @@ -1,3 +1,7 @@ +--- +comments: false +--- + # Docker integration - [Using Docker Images](using_docker_images.md) diff --git a/doc/ci/docker/using_docker_build.md b/doc/ci/docker/using_docker_build.md index 4586caa457d..0a2419b7ed2 100644 --- a/doc/ci/docker/using_docker_build.md +++ b/doc/ci/docker/using_docker_build.md @@ -31,12 +31,12 @@ There are three methods to enable the use of `docker build` and `docker run` dur The simplest approach is to install GitLab Runner in `shell` execution mode. GitLab Runner then executes job scripts as the `gitlab-runner` user. -1. Install [GitLab Runner](https://gitlab.com/gitlab-org/gitlab-ci-multi-runner/#installation). +1. Install [GitLab Runner](https://gitlab.com/gitlab-org/gitlab-runner/#installation). 1. During GitLab Runner installation select `shell` as method of executing job scripts or use command: ```bash - sudo gitlab-ci-multi-runner register -n \ + sudo gitlab-runner register -n \ --url https://gitlab.com/ \ --registration-token REGISTRATION_TOKEN \ --executor shell \ @@ -93,7 +93,7 @@ In order to do that, follow the steps: mode: ```bash - sudo gitlab-ci-multi-runner register -n \ + sudo gitlab-runner register -n \ --url https://gitlab.com/ \ --registration-token REGISTRATION_TOKEN \ --executor docker \ @@ -178,7 +178,7 @@ In order to do that, follow the steps: 1. Register GitLab Runner from the command line to use `docker` and share `/var/run/docker.sock`: ```bash - sudo gitlab-ci-multi-runner register -n \ + sudo gitlab-runner register -n \ --url https://gitlab.com/ \ --registration-token REGISTRATION_TOKEN \ --executor docker \ diff --git a/doc/ci/docker/using_docker_images.md b/doc/ci/docker/using_docker_images.md index f7493794b6a..ecb8f15c851 100644 --- a/doc/ci/docker/using_docker_images.md +++ b/doc/ci/docker/using_docker_images.md @@ -501,8 +501,8 @@ First start with creating a file named `build_script`: ```bash cat <<EOF > build_script -git clone https://gitlab.com/gitlab-org/gitlab-ci-multi-runner.git /builds/gitlab-org/gitlab-ci-multi-runner -cd /builds/gitlab-org/gitlab-ci-multi-runner +git clone https://gitlab.com/gitlab-org/gitlab-runner.git /builds/gitlab-org/gitlab-runner +cd /builds/gitlab-org/gitlab-runner make EOF ``` diff --git a/doc/ci/enable_or_disable_ci.md b/doc/ci/enable_or_disable_ci.md index b8f9988e3ef..7aa7de97c43 100644 --- a/doc/ci/enable_or_disable_ci.md +++ b/doc/ci/enable_or_disable_ci.md @@ -1,4 +1,4 @@ -## Enable or disable GitLab CI/CD +# How to enable or disable GitLab CI/CD To effectively use GitLab CI/CD, you need a valid [`.gitlab-ci.yml`](yaml/README.md) file present at the root directory of your project and a @@ -21,7 +21,7 @@ individually under each project's settings, or site-wide by modifying the settings in `gitlab.yml` and `gitlab.rb` for source and Omnibus installations respectively. -### Per-project user setting +## Per-project user setting The setting to enable or disable GitLab CI/CD can be found under your project's **Settings > General > Permissions**. Choose one of "Disabled", "Only team members" @@ -29,7 +29,7 @@ or "Everyone with access" and hit **Save changes** for the settings to take effe  -### Site-wide admin setting +## Site-wide admin setting You can disable GitLab CI/CD site-wide, by modifying the settings in `gitlab.yml` and `gitlab.rb` for source and Omnibus installations respectively. diff --git a/doc/ci/examples/README.md b/doc/ci/examples/README.md index f094546c3bd..d05b4db953a 100644 --- a/doc/ci/examples/README.md +++ b/doc/ci/examples/README.md @@ -1,3 +1,7 @@ +--- +comments: false +--- + # GitLab CI Examples A collection of `.gitlab-ci.yml` files is maintained at the [GitLab CI Yml project][gitlab-ci-templates]. diff --git a/doc/ci/examples/deployment/composer-npm-deploy.md b/doc/ci/examples/deployment/composer-npm-deploy.md index b9f0485290e..bed379b0254 100644 --- a/doc/ci/examples/deployment/composer-npm-deploy.md +++ b/doc/ci/examples/deployment/composer-npm-deploy.md @@ -1,4 +1,4 @@ -## Running Composer and NPM scripts with deployment via SCP +# Running Composer and NPM scripts with deployment via SCP in GitLab CI/CD This guide covers the building dependencies of a PHP project while compiling assets via an NPM script. @@ -39,13 +39,13 @@ In this particular case, the `npm deploy` script is a Gulp script that does the All these operations will put all files into a `build` folder, which is ready to be deployed to a live server. -### How to transfer files to a live server? +## How to transfer files to a live server You have multiple options: rsync, scp, sftp and so on. For now, we will use scp. To make this work, you need to add a GitLab Secret Variable (accessible on _gitlab.example/your-project-name/variables_). That variable will be called `STAGING_PRIVATE_KEY` and it's the **private** ssh key of your server. -#### Security tip +### Security tip Create a user that has access **only** to the folder that needs to be updated! @@ -69,7 +69,7 @@ In order, this means that: And this is basically all you need in the `before_script` section. -## How to deploy things? +## How to deploy things As we stated above, we need to deploy the `build` folder from the docker image to our server. To do so, we create a new job: @@ -88,7 +88,7 @@ stage_deploy: - ssh -p22 server_user@server_host "rm -rf htdocs/wp-content/themes/_old" ``` -### What's going on here? +Here's the breakdown: 1. `only:dev` means that this build will run only when something is pushed to the `dev` branch. You can remove this block completely and have everything be ran on every push (but probably this is something you don't want) 2. `ssh-add ...` we will add that private key you added on the web UI to the docker container @@ -99,7 +99,7 @@ stage_deploy: What's the deal with the artifacts? We just tell GitLab CI to keep the `build` directory (later on, you can download that as needed). -#### Why we do it this way? +### Why we do it this way If you're using this only for stage server, you could do this in two steps: @@ -112,7 +112,7 @@ The problem is that there will be a small period of time when you won't have the So we use so many steps because we want to make sure that at any given time we have a functional app in place. -## Where to go next? +## Where to go next Since this was a WordPress project, I gave real life code snippets. Some ideas you can pursuit: diff --git a/doc/ci/examples/php.md b/doc/ci/examples/php.md index f2dd12b67d3..6768a2e012f 100644 --- a/doc/ci/examples/php.md +++ b/doc/ci/examples/php.md @@ -267,10 +267,10 @@ terminal execute: ```bash # Check using docker executor -gitlab-ci-multi-runner exec docker test:app +gitlab-runner exec docker test:app # Check using shell executor -gitlab-ci-multi-runner exec shell test:app +gitlab-runner exec shell test:app ``` ## Example project diff --git a/doc/ci/examples/test-and-deploy-python-application-to-heroku.md b/doc/ci/examples/test-and-deploy-python-application-to-heroku.md index 73aebaf6d7f..a6ed1c54e16 100644 --- a/doc/ci/examples/test-and-deploy-python-application-to-heroku.md +++ b/doc/ci/examples/test-and-deploy-python-application-to-heroku.md @@ -1,10 +1,13 @@ -## Test and Deploy a python application +# Test and Deploy a python application with GitLab CI/CD + This example will guide you how to run tests in your Python application and deploy it automatically as Heroku application. -You can checkout the example [source](https://gitlab.com/ayufan/python-getting-started) and check [CI status](https://gitlab.com/ayufan/python-getting-started/builds?scope=all). +You can checkout the [example source](https://gitlab.com/ayufan/python-getting-started). + +## Configure project -### Configure project This is what the `.gitlab-ci.yml` file looks like for this project: + ```yaml test: script: @@ -41,23 +44,27 @@ This project has three jobs: 2. `staging` - used to automatically deploy staging environment every push to `master` branch 3. `production` - used to automatically deploy production environmnet for every created tag -### Store API keys +## Store API keys + You'll need to create two variables in `Project > Variables`: 1. `HEROKU_STAGING_API_KEY` - Heroku API key used to deploy staging app, 2. `HEROKU_PRODUCTION_API_KEY` - Heroku API key used to deploy production app. Find your Heroku API key in [Manage Account](https://dashboard.heroku.com/account). -### Create Heroku application +## Create Heroku application + For each of your environments, you'll need to create a new Heroku application. You can do this through the [Dashboard](https://dashboard.heroku.com/). -### Create runner +## Create Runner + First install [Docker Engine](https://docs.docker.com/installation/). -To build this project you also need to have [GitLab Runner](https://about.gitlab.com/gitlab-ci/#gitlab-runner). +To build this project you also need to have [GitLab Runner](https://docs.gitlab.com/runner). You can use public runners available on `gitlab.com`, but you can register your own: + ``` -gitlab-ci-multi-runner register \ +gitlab-runner register \ --non-interactive \ --url "https://gitlab.com/" \ --registration-token "PROJECT_REGISTRATION_TOKEN" \ diff --git a/doc/ci/examples/test-and-deploy-ruby-application-to-heroku.md b/doc/ci/examples/test-and-deploy-ruby-application-to-heroku.md index 6fa64a67e82..10fd2616fab 100644 --- a/doc/ci/examples/test-and-deploy-ruby-application-to-heroku.md +++ b/doc/ci/examples/test-and-deploy-ruby-application-to-heroku.md @@ -1,10 +1,13 @@ -## Test and Deploy a ruby application +# Test and Deploy a ruby application with GitLab CI/CD + This example will guide you how to run tests in your Ruby on Rails application and deploy it automatically as Heroku application. You can checkout the example [source](https://gitlab.com/ayufan/ruby-getting-started) and check [CI status](https://gitlab.com/ayufan/ruby-getting-started/builds?scope=all). -### Configure project +## Configure the project + This is what the `.gitlab-ci.yml` file looks like for this project: + ```yaml test: script: @@ -36,23 +39,28 @@ This project has three jobs: 2. `staging` - used to automatically deploy staging environment every push to `master` branch 3. `production` - used to automatically deploy production environment for every created tag -### Store API keys -You'll need to create two variables in `Project > Variables`: +## Store API keys + +You'll need to create two variables in your project's **Settings > CI/CD > Variables**: + 1. `HEROKU_STAGING_API_KEY` - Heroku API key used to deploy staging app, 2. `HEROKU_PRODUCTION_API_KEY` - Heroku API key used to deploy production app. Find your Heroku API key in [Manage Account](https://dashboard.heroku.com/account). -### Create Heroku application +## Create Heroku application + For each of your environments, you'll need to create a new Heroku application. You can do this through the [Dashboard](https://dashboard.heroku.com/). -### Create runner +## Create Runner + First install [Docker Engine](https://docs.docker.com/installation/). To build this project you also need to have [GitLab Runner](https://about.gitlab.com/gitlab-ci/#gitlab-runner). You can use public runners available on `gitlab.com`, but you can register your own: + ``` -gitlab-ci-multi-runner register \ +gitlab-runner register \ --non-interactive \ --url "https://gitlab.com/" \ --registration-token "PROJECT_REGISTRATION_TOKEN" \ @@ -62,6 +70,6 @@ gitlab-ci-multi-runner register \ --docker-postgres latest ``` -With the command above, you create a runner that uses [ruby:2.2](https://hub.docker.com/r/_/ruby/) image and uses [postgres](https://hub.docker.com/r/_/postgres/) database. +With the command above, you create a Runner that uses [ruby:2.2](https://hub.docker.com/r/_/ruby/) image and uses [postgres](https://hub.docker.com/r/_/postgres/) database. To access PostgreSQL database you need to connect to `host: postgres` as user `postgres` without password. diff --git a/doc/ci/examples/test-clojure-application.md b/doc/ci/examples/test-clojure-application.md index 56b746ce025..3b1026d174f 100644 --- a/doc/ci/examples/test-clojure-application.md +++ b/doc/ci/examples/test-clojure-application.md @@ -1,10 +1,10 @@ -## Test a Clojure application +# Test a Clojure application with GitLab CI/CD This example will guide you how to run tests in your Clojure application. You can checkout the example [source](https://gitlab.com/dzaporozhets/clojure-web-application) and check [CI status](https://gitlab.com/dzaporozhets/clojure-web-application/builds?scope=all). -### Configure project +## Configure the project This is what the `.gitlab-ci.yml` file looks like for this project: @@ -23,13 +23,13 @@ before_script: - lein deps - lein migratus migrate -test: - script: +test: + script: - lein test ``` -In before script we install JRE and [Leiningen](http://leiningen.org/). -Sample project uses [migratus](https://github.com/yogthos/migratus) library to manage database migrations. +In before script we install JRE and [Leiningen](http://leiningen.org/). +Sample project uses [migratus](https://github.com/yogthos/migratus) library to manage database migrations. So we added database migration as last step of `before_script` section You can use public runners available on `gitlab.com` for testing your application with such configuration. diff --git a/doc/ci/examples/test-phoenix-application.md b/doc/ci/examples/test-phoenix-application.md index 150698ca04b..f6c81b076bc 100644 --- a/doc/ci/examples/test-phoenix-application.md +++ b/doc/ci/examples/test-phoenix-application.md @@ -1,9 +1,9 @@ -## Test a Phoenix application +# Test a Phoenix application with GitLab CI/CD This example demonstrates the integration of Gitlab CI with Phoenix, Elixir and Postgres. -### Add `.gitlab-ci.yml` file to project +## Add `.gitlab-ci.yml` to project The following `.gitlab-ci.yml` should be added in the root of your repository to trigger CI: @@ -36,7 +36,7 @@ run your migrations. Finally, the test `script` will run your tests. -### Update the Config Settings +## Update the Config Settings In `config/test.exs`, update the database hostname: @@ -45,12 +45,12 @@ config :my_app, MyApp.Repo, hostname: if(System.get_env("CI"), do: "postgres", else: "localhost"), ``` -### Add the Migrations Folder +## Add the Migrations Folder If you do not have any migrations yet, you will need to create an empty `.gitkeep` file in `priv/repo/migrations`. -### Sources +## Sources - https://medium.com/@nahtnam/using-phoenix-on-gitlab-ci-5a51eec81142 - https://davejlong.com/ci-with-phoenix-and-gitlab/ diff --git a/doc/ci/git_submodules.md b/doc/ci/git_submodules.md index 36c6e153d95..286f3dee665 100644 --- a/doc/ci/git_submodules.md +++ b/doc/ci/git_submodules.md @@ -8,7 +8,7 @@ with the use of [SSH keys](ssh_keys/README.md). - With GitLab 8.12 onward, your permissions are used to evaluate what a CI job can access. More information about how this system works can be found in the - [Jobs permissions model](../user/permissions.md#jobs-permissions). + [Jobs permissions model](../user/permissions.md#job-permissions). - The HTTP(S) Git protocol [must be enabled][gitpro] in your GitLab instance. ## Configuring the `.gitmodules` file @@ -61,7 +61,7 @@ correctly with your CI jobs: 1. First, make sure you have used [relative URLs](#configuring-the-gitmodules-file) for the submodules located in the same GitLab server. -1. Next, if you are using `gitlab-ci-multi-runner` v1.10+, you can set the +1. Next, if you are using `gitlab-runner` v1.10+, you can set the `GIT_SUBMODULE_STRATEGY` variable to either `normal` or `recursive` to tell the runner to fetch your submodules before the job: ```yaml @@ -71,7 +71,7 @@ correctly with your CI jobs: See the [`.gitlab-ci.yml` reference](yaml/README.md#git-submodule-strategy) for more details about `GIT_SUBMODULE_STRATEGY`. -1. If you are using an older version of `gitlab-ci-multi-runner`, then use +1. If you are using an older version of `gitlab-runner`, then use `git submodule sync/update` in `before_script`: ```yaml diff --git a/doc/ci/permissions/README.md b/doc/ci/permissions/README.md index 42eb59f84c8..80d8e46f29c 100644 --- a/doc/ci/permissions/README.md +++ b/doc/ci/permissions/README.md @@ -1,3 +1 @@ -# Users Permissions - This document was moved to [user/permissions.md](../../user/permissions.md#gitlab-ci). diff --git a/doc/ci/quick_start/README.md b/doc/ci/quick_start/README.md index 2d56b2540ef..f621bf07251 100644 --- a/doc/ci/quick_start/README.md +++ b/doc/ci/quick_start/README.md @@ -1,4 +1,4 @@ -# Getting started with GitLab CI +# Getting started with GitLab CI/CD >**Note:** Starting from version 8.0, GitLab [Continuous Integration][ci] (CI) is fully integrated into GitLab itself and is [enabled] by default on all diff --git a/doc/ci/runners/README.md b/doc/ci/runners/README.md index 8b51d112a2c..df66810a838 100644 --- a/doc/ci/runners/README.md +++ b/doc/ci/runners/README.md @@ -1,4 +1,4 @@ -# Runners +# Configuring GitLab Runners In GitLab CI, Runners run the code defined in [`.gitlab-ci.yml`](../yaml/README.md). They are isolated (virtual) machines that pick up jobs through the coordinator diff --git a/doc/ci/services/README.md b/doc/ci/services/README.md index 4b79461d55c..d94b472b768 100644 --- a/doc/ci/services/README.md +++ b/doc/ci/services/README.md @@ -1,4 +1,8 @@ -## GitLab CI Services +--- +comments: false +--- + +# GitLab CI Services GitLab CI uses the `services` keyword to define what docker containers should be linked with your base image. Below is a list of examples you may use. diff --git a/doc/ci/services/docker-services.md b/doc/ci/services/docker-services.md index df36ebaf7d4..787c5e462e4 100644 --- a/doc/ci/services/docker-services.md +++ b/doc/ci/services/docker-services.md @@ -1,5 +1,9 @@ -## GitLab CI Services +--- +comments: false +--- -+ [Using MySQL](mysql.md) -+ [Using PostgreSQL](postgres.md) -+ [Using Redis](redis.md) +# GitLab CI Services + +- [Using MySQL](mysql.md) +- [Using PostgreSQL](postgres.md) +- [Using Redis](redis.md) diff --git a/doc/ci/variables/README.md b/doc/ci/variables/README.md index 73568757aaa..a9e6bda9916 100644 --- a/doc/ci/variables/README.md +++ b/doc/ci/variables/README.md @@ -1,4 +1,4 @@ -# Variables +# GitLab CI/CD Variables When receiving a job from GitLab CI, the [Runner] prepares the build environment. It starts by setting a list of **predefined variables** (environment variables) @@ -43,7 +43,7 @@ future GitLab releases.** | **CI_COMMIT_TAG** | 9.0 | 0.5 | The commit tag name. Present only when building tags. | | **CI_CONFIG_PATH** | 9.4 | 0.5 | The path to CI config file. Defaults to `.gitlab-ci.yml` | | **CI_DEBUG_TRACE** | all | 1.7 | Whether [debug tracing](#debug-tracing) is enabled | -| **CI_DISPOSABLE_ENVIRONMENT** | all | 10.1 | Mark that job is executed in a disposable environment (something that is created only for this job and disposed of/destroyed after the execution - all executors except `shell` and `ssh`). If the environment is disposable, it is set to true, otherwise it is not defined at all. | +| **CI_DISPOSABLE_ENVIRONMENT** | all | 10.1 | Marks that the job is executed in a disposable environment (something that is created only for this job and disposed of/destroyed after the execution - all executors except `shell` and `ssh`). If the environment is disposable, it is set to true, otherwise it is not defined at all. | | **CI_ENVIRONMENT_NAME** | 8.15 | all | The name of the environment for this job | | **CI_ENVIRONMENT_SLUG** | 8.15 | all | A simplified version of the environment name, suitable for inclusion in DNS, URLs, Kubernetes labels, etc. | | **CI_ENVIRONMENT_URL** | 9.3 | all | The URL of the environment for this job | @@ -66,6 +66,7 @@ future GitLab releases.** | **CI_PROJECT_PATH** | 8.10 | 0.5 | The namespace with project name | | **CI_PROJECT_PATH_SLUG** | 9.3 | all | `$CI_PROJECT_PATH` lowercased and with everything except `0-9` and `a-z` replaced with `-`. Use in URLs and domain names. | | **CI_PROJECT_URL** | 8.10 | 0.5 | The HTTP address to access project | +| **CI_PROJECT_VISIBILITY** | 10.3 | all | The project visibility (internal, private, public) | | **CI_REGISTRY** | 8.10 | 0.5 | If the Container Registry is enabled it returns the address of GitLab's Container Registry | | **CI_REGISTRY_IMAGE** | 8.10 | 0.5 | If the Container Registry is enabled for the project it returns the address of the registry tied to the specific project | | **CI_REGISTRY_PASSWORD** | 9.0 | all | The password to use to push containers to the GitLab Container Registry | @@ -74,7 +75,7 @@ future GitLab releases.** | **CI_SERVER_NAME** | all | all | The name of CI server that is used to coordinate jobs | | **CI_SERVER_REVISION** | all | all | GitLab revision that is used to schedule jobs | | **CI_SERVER_VERSION** | all | all | GitLab version that is used to schedule jobs | -| **CI_SHARED_ENVIRONMENT** | all | 10.1 | Mark that job is executed in a shared environment (something that is persisted across CI invocations like `shell` or `ssh` executor). If the environment is shared, it is set to true, otherwise it is not defined at all. | +| **CI_SHARED_ENVIRONMENT** | all | 10.1 | Marks that the job is executed in a shared environment (something that is persisted across CI invocations like `shell` or `ssh` executor). If the environment is shared, it is set to true, otherwise it is not defined at all. | | **ARTIFACT_DOWNLOAD_ATTEMPTS** | 8.15 | 1.9 | Number of attempts to download artifacts running a job | | **GET_SOURCES_ATTEMPTS** | 8.15 | 1.9 | Number of attempts to fetch sources running a job | | **GITLAB_CI** | all | all | Mark that job is executed in GitLab CI environment | diff --git a/doc/ci/yaml/README.md b/doc/ci/yaml/README.md index 6ad70707594..f40d2c5e347 100644 --- a/doc/ci/yaml/README.md +++ b/doc/ci/yaml/README.md @@ -124,7 +124,7 @@ stages: 1. First, all jobs of `build` are executed in parallel. 1. If all jobs of `build` succeed, the `test` jobs are executed in parallel. 1. If all jobs of `test` succeed, the `deploy` jobs are executed in parallel. -1. If all jobs of `deploy` succeed, the commit is marked as `success`. +1. If all jobs of `deploy` succeed, the commit is marked as `passed`. 1. If any of the previous jobs fails, the commit is marked as `failed` and no jobs of further stage are executed. diff --git a/doc/development/README.md b/doc/development/README.md index 36096842344..6892838be7f 100644 --- a/doc/development/README.md +++ b/doc/development/README.md @@ -1,3 +1,7 @@ +--- +comments: false +--- + # GitLab development guides ## Get started! @@ -18,6 +22,7 @@ - [UX guide](ux_guide/index.md) for building GitLab with existing CSS styles and elements - [Frontend guidelines](fe_guide/index.md) +- [Emoji guide](fe_guide/emojis.md) ## Backend guides @@ -33,6 +38,7 @@ - [Gotchas](gotchas.md) to avoid - [Issue and merge requests state models](object_state_models.md) - [How to dump production data to staging](db_dump.md) +- [Working with the GitHub importer](github_importer.md) ## Performance guides @@ -65,6 +71,7 @@ - [Iterating tables in batches](iterating_tables_in_batches.md) - [Ordering table columns](ordering_table_columns.md) - [Verifying database capabilities](verifying_database_capabilities.md) +- [Database Debugging and Troubleshooting](database_debugging.md) ## Testing guides diff --git a/doc/development/database_debugging.md b/doc/development/database_debugging.md new file mode 100644 index 00000000000..50eb8005b44 --- /dev/null +++ b/doc/development/database_debugging.md @@ -0,0 +1,55 @@ +# Database Debugging and Troubleshooting + +This section is to help give some copy-pasta you can use as a reference when you +run into some head-banging database problems. + +An easy first step is to search for your error in Slack or google "GitLab <my error>". + +--- + +Available `RAILS_ENV` + + - `production` (generally not for your main GDK db, but you may need this for e.g. omnibus) + - `development` (this is your main GDK db) + - `test` (used for tests like rspec and spinach) + + +## Nuke everything and start over + +If you just want to delete everything and start over with an empty DB (~1 minute): + + - `bundle exec rake db:reset RAILS_ENV=development` + +If you just want to delete everything and start over with dummy data (~40 minutes). This also does `db:reset` and runs DB-specific migrations: + + - `bundle exec rake dev:setup RAILS_ENV=development` + +If your test DB is giving you problems, it is safe to nuke it because it doesn't contain important data: + + - `bundle exec rake db:reset RAILS_ENV=test` + +## Migration wrangling + + - `bundle exec rake db:migrate RAILS_ENV=development`: Execute any pending migrations that you may have picked up from a MR + - `bundle exec rake db:migrate:status RAILS_ENV=development`: Check if all migrations are `up` or `down` + - `bundle exec rake db:migrate:down VERSION=20170926203418 RAILS_ENV=development`: Tear down a migration + - `bundle exec rake db:migrate:up VERSION=20170926203418 RAILS_ENV=development`: Setup a migration + - `bundle exec rake db:migrate:redo VERSION=20170926203418 RAILS_ENV=development`: Re-run a specific migration + + +## Manually access the database + +Access the database via one of these commands (they all get you to the same place) + +``` +gdk psql -d gitlabhq_development +bundle exec rails dbconsole RAILS_ENV=development +bundle exec rails db RAILS_ENV=development +``` + + - `\q`: Quit/exit + - `\dt`: List all tables + - `\d+ issues`: List columns for `issues` table + - `CREATE TABLE board_labels();`: Create a table called `board_labels` + - `SELECT * FROM schema_migrations WHERE version = '20170926203418';`: Check if a migration was run + - `DELETE FROM schema_migrations WHERE version = '20170926203418';`: Manually remove a migration diff --git a/doc/development/doc_styleguide.md b/doc/development/doc_styleguide.md index 798f40eef3d..0e4ffbd7910 100644 --- a/doc/development/doc_styleguide.md +++ b/doc/development/doc_styleguide.md @@ -459,11 +459,11 @@ Rendered example: ### cURL commands - Use `https://gitlab.example.com/api/v4/` as an endpoint. -- Wherever needed use this private token: `9koXpg98eAheJpvBs5tK`. +- Wherever needed use this personal access token: `9koXpg98eAheJpvBs5tK`. - Always put the request first. `GET` is the default so you don't have to include it. - Use double quotes to the URL when it includes additional parameters. -- Prefer to use examples using the private token and don't pass data of +- Prefer to use examples using the personal access token and don't pass data of username and password. | Methods | Description | diff --git a/doc/development/fe_guide/axios.md b/doc/development/fe_guide/axios.md new file mode 100644 index 00000000000..962fe3dcec9 --- /dev/null +++ b/doc/development/fe_guide/axios.md @@ -0,0 +1,68 @@ +# Axios +We use [axios][axios] to communicate with the server in Vue applications and most new code. + +In order to guarantee all defaults are set you *should not use `axios` directly*, you should import `axios` from `axios_utils`. + +## CSRF token +All our request require a CSRF token. +To guarantee this token is set, we are importing [axios][axios], setting the token, and exporting `axios` . + +This exported module should be used instead of directly using `axios` to ensure the token is set. + +## Usage +```javascript + import axios from '~/lib/utils/axios_utils'; + + axios.get(url) + .then((response) => { + // `data` is the response that was provided by the server + const data = response.data; + + // `headers` the headers that the server responded with + // All header names are lower cased + const paginationData = response.headers; + }) + .catch(() => { + //handle the error + }); +``` + +## Mock axios response on tests + +To help us mock the responses we need we use [axios-mock-adapter][axios-mock-adapter] + + +```javascript + import axios from '~/lib/utils/axios_utils'; + import MockAdapter from 'axios-mock-adapter'; + + let mock; + beforeEach(() => { + // This sets the mock adapter on the default instance + mock = new MockAdapter(axios); + // Mock any GET request to /users + // arguments for reply are (status, data, headers) + mock.onGet('/users').reply(200, { + users: [ + { id: 1, name: 'John Smith' } + ] + }); + }); + + afterEach(() => { + mock.reset(); + }); +``` + +### Mock poll requests on tests with axios + +Because polling function requires an header object, we need to always include an object as the third argument: + +```javascript + mock.onGet('/users').reply(200, { foo: 'bar' }, {}); +``` + +[axios]: https://github.com/axios/axios +[axios-instance]: #creating-an-instance +[axios-interceptors]: https://github.com/axios/axios#interceptors +[axios-mock-adapter]: https://github.com/ctimmerm/axios-mock-adapter diff --git a/doc/development/fe_guide/dropdowns.md b/doc/development/fe_guide/dropdowns.md new file mode 100644 index 00000000000..e1660ac5caa --- /dev/null +++ b/doc/development/fe_guide/dropdowns.md @@ -0,0 +1,38 @@ +# Dropdowns + + +## How to style a bootstrap dropdown +1. Use the HTML structure provided by the [docs][bootstrap-dropdowns] +1. Add a specific class to the top level `.dropdown` element + + + ```Haml + .dropdown.my-dropdown + %button{ type: 'button', data: { toggle: 'dropdown' }, 'aria-haspopup': true, 'aria-expanded': false } + %span.dropdown-toggle-text + Toggle Dropdown + = icon('chevron-down') + + %ul.dropdown-menu + %li + %a + item! + ``` + + Or use the helpers + ```Haml + .dropdown.my-dropdown + = dropdown_toggle('Toogle!', { toggle: 'dropdown' }) + = dropdown_content + %li + %a + item! + ``` + +1. Include the mixin in CSS + + ```SCSS + @include new-style-dropdown('.my-dropdown '); + ``` + +[bootstrap-dropdowns]: https://getbootstrap.com/docs/3.3/javascript/#dropdowns diff --git a/doc/development/fe_guide/emojis.md b/doc/development/fe_guide/emojis.md new file mode 100644 index 00000000000..38794c47965 --- /dev/null +++ b/doc/development/fe_guide/emojis.md @@ -0,0 +1,27 @@ +# Emojis + +GitLab supports native unicode emojis and fallsback to image-based emojis selectively +when your platform does not support it. + +# How to update Emojis + + 1. Update the `gemojione` gem + 1. Update `fixtures/emojis/index.json` from [Gemojione](https://github.com/jonathanwiesel/gemojione/blob/master/config/index.json). + In the future, we could grab the file directly from the gem. + We should probably make a PR on the Gemojione project to get access to + all emojis after being parsed or just a raw path to the `json` file itself. + 1. Ensure [`emoji-unicode-version`](https://www.npmjs.com/package/emoji-unicode-version) + is up to date with the latest version. + 1. Run `bundle exec rake gemojione:aliases` + 1. Run `bundle exec rake gemojione:digests` + 1. Run `bundle exec rake gemojione:sprite` + 1. Ensure new sprite sheets generated for 1x and 2x + - `app/assets/images/emoji.png` + - `app/assets/images/emoji@2x.png` + 1. Ensure you see new individual images copied into `app/assets/images/emoji/` + 1. Ensure you can see the new emojis and their aliases in the GFM Autocomplete + 1. Ensure you can see the new emojis and their aliases in the award emoji menu + 1. You might need to add new emoji unicode support checks and rules for platforms + that do not support a certain emoji and we need to fallback to an image. + See `app/assets/javascripts/emoji/support/is_emoji_unicode_supported.js` + and `app/assets/javascripts/emoji/support/unicode_support_map.js` diff --git a/doc/development/fe_guide/icons.md b/doc/development/fe_guide/icons.md index a76e978bd26..b288ee95722 100644 --- a/doc/development/fe_guide/icons.md +++ b/doc/development/fe_guide/icons.md @@ -4,15 +4,17 @@ We are using SVG Icons in GitLab with a SVG Sprite, due to this the icons are on ### Usage in HAML/Rails -To use a sprite Icon in HAML or Rails we use a specific helper function : +To use a sprite Icon in HAML or Rails we use a specific helper function : `sprite_icon(icon_name, size: nil, css_class: '')` -**icon_name** Use the icon_name that you can find in the SVG Sprite (Overview is available under `/assets/sprite.symbol.html`). +**icon_name** Use the icon_name that you can find in the SVG Sprite ([Overview is available here](http://gitlab-org.gitlab.io/gitlab-svgs/)`). + **size (optional)** Use one of the following sizes : 16,24,32,48,72 (this will be translated into a `s16` class) + **css_class (optional)** If you want to add additional css classes -**Example** +**Example** `= sprite_icon('issues', size: 72, css_class: 'icon-danger')` @@ -20,16 +22,34 @@ To use a sprite Icon in HAML or Rails we use a specific helper function : `<svg class="s72 icon-danger"><use xmlns:xlink="http://www.w3.org/1999/xlink" xlink:href="/assets/icons.svg#issues"></use></svg>` +### Usage in Vue + +We have a special Vue component for our sprite icons in `\vue_shared\components\icon.vue`. + +Sample usage : + +`<icon + name="retry" + :size="32" + css-classes="top" + />` + +**name** Name of the Icon in the SVG Sprite ([Overview is available here](http://gitlab-org.gitlab.io/gitlab-svgs/)`). + +**size (optional)** Number value for the size which is then mapped to a specific CSS class (Available Sizes: 8,12,16,18,24,32,48,72 are mapped to `sXX` css classes) + +**css-classes (optional)** Additional CSS Classes to add to the svg tag. + ### Usage in HTML/JS -Please use the following function inside JS to render an icon : +Please use the following function inside JS to render an icon : `gl.utils.spriteIcon(iconName)` ## Adding a new icon to the sprite All Icons and Illustrations are managed in the [gitlab-svgs](https://gitlab.com/gitlab-org/gitlab-svgs) repository which is added as a dev-dependency. -To upgrade to a new SVG Sprite version run `yarn upgrade https://gitlab.com/gitlab-org/gitlab-svgs` and then run `yarn run svg`. This task will copy the svg sprite and all illustrations in the correct folders. +To upgrade to a new SVG Sprite version run `yarn upgrade @gitlab-org/gitlab-svgs` and then run `yarn run svg`. This task will copy the svg sprite and all illustrations in the correct folders. The updated files should be tracked in Git as those are referenced. # SVG Illustrations diff --git a/doc/development/fe_guide/index.md b/doc/development/fe_guide/index.md index 8f956681693..72cb557d054 100644 --- a/doc/development/fe_guide/index.md +++ b/doc/development/fe_guide/index.md @@ -71,12 +71,14 @@ Vue specific design patterns and practices. --- -## [Vue Resource](vue_resource.md) -Vue resource specific practices and gotchas. +## [Axios](axios.md) +Axios specific practices and gotchas. ## [Icons](icons.md) How we use SVG for our Icons. +## [Dropdowns](dropdowns.md) +How we use dropdowns. --- ## Style Guides diff --git a/doc/development/fe_guide/vue.md b/doc/development/fe_guide/vue.md index f88f0753687..6e9f18dd1c3 100644 --- a/doc/development/fe_guide/vue.md +++ b/doc/development/fe_guide/vue.md @@ -178,16 +178,13 @@ itself, please read this guide: [State Management][state-management] The Service is a class used only to communicate with the server. It does not store or manipulate any data. It is not aware of the store or the components. -We use [vue-resource][vue-resource-repo] to communicate with the server. -Refer to [vue resource](vue_resource.md) for more details. +We use [axios][axios] to communicate with the server. +Refer to [axios](axios.md) for more details. -Vue Resource should only be imported in the service file. +Axios instance should only be imported in the service file. ```javascript - import Vue from 'vue'; - import VueResource from 'vue-resource'; - - Vue.use(VueResource); + import axios from 'javascripts/lib/utils/axios_utils'; ``` ### End Result @@ -230,15 +227,14 @@ export default class Store { } // service.js -import Vue from 'vue'; -import VueResource from 'vue-resource'; -import 'vue_shared/vue_resource_interceptor'; - -Vue.use(VueResource); +import axios from 'javascripts/lib/utils/axios_utils' export default class Service { constructor(options) { - this.todos = Vue.resource(endpoint.todosEndpoint); + this.todos = axios.create({ + baseURL: endpoint.todosEndpoint + }); + } getTodos() { @@ -477,50 +473,8 @@ The main return value of a Vue component is the rendered output. In order to tes need to test the rendered output. [Vue][vue-test] guide's to unit test show us exactly that: ### Stubbing API responses -[Vue Resource Interceptors][vue-resource-interceptor] allow us to add a interceptor with -the response we need: - - ```javascript - // Mock the service to return data - const interceptor = (request, next) => { - next(request.respondWith(JSON.stringify([{ - title: 'This is a todo', - body: 'This is the text' - }]), { - status: 200, - })); - }; +Refer to [mock axios](axios.md#mock-axios-response-on-tests) - beforeEach(() => { - Vue.http.interceptors.push(interceptor); - }); - - afterEach(() => { - Vue.http.interceptors = _.without(Vue.http.interceptors, interceptor); - }); - - it('should do something', (done) => { - setTimeout(() => { - // Test received data - done(); - }, 0); - }); - ``` - -1. Headers interceptor -Refer to [this section](vue.md#headers) - -1. Use `$.mount()` to mount the component - -```javascript -// bad -new Component({ - el: document.createElement('div') -}); - -// good -new Component().$mount(); -``` ## Vuex To manage the state of an application you may use [Vuex][vuex-docs]. @@ -721,7 +675,6 @@ describe('component', () => { [component-system]: https://vuejs.org/v2/guide/#Composing-with-Components [state-management]: https://vuejs.org/v2/guide/state-management.html#Simple-State-Management-from-Scratch [one-way-data-flow]: https://vuejs.org/v2/guide/components.html#One-Way-Data-Flow -[vue-resource-interceptor]: https://github.com/pagekit/vue-resource/blob/develop/docs/http.md#interceptors [vue-test]: https://vuejs.org/v2/guide/unit-testing.html [issue-boards-service]: https://gitlab.com/gitlab-org/gitlab-ce/blob/master/app/assets/javascripts/boards/services/board_service.js.es6 [flux]: https://facebook.github.io/flux @@ -729,3 +682,6 @@ describe('component', () => { [vuex-structure]: https://vuex.vuejs.org/en/structure.html [vuex-mutations]: https://vuex.vuejs.org/en/mutations.html [vuex-testing]: https://vuex.vuejs.org/en/testing.html +[axios]: https://github.com/axios/axios +[axios-interceptors]: https://github.com/axios/axios#interceptors + diff --git a/doc/development/fe_guide/vue_resource.md b/doc/development/fe_guide/vue_resource.md deleted file mode 100644 index c376c5c32bf..00000000000 --- a/doc/development/fe_guide/vue_resource.md +++ /dev/null @@ -1,72 +0,0 @@ -# Vue Resouce -In Vue applications we use [vue-resource][vue-resource-repo] to communicate with the server. - -## HTTP Status Codes - -### `.json()` -When making a request to the server, you will most likely need to access the body of the response. -Use `.json()` to convert. Because `.json()` returns a Promise the follwoing structure should be used: - - ```javascript - service.get('url') - .then(resp => resp.json()) - .then((data) => { - this.store.storeData(data); - }) - .catch(() => new Flash('Something went wrong')); - ``` - - -When using `Poll` (`app/assets/javascripts/lib/utils/poll.js`), the `successCallback` needs to handle `.json()` as a Promise: - ```javascript - successCallback: (response) => { - return response.json().then((data) => { - // handle the response - }); - } - ``` - -### 204 -Some endpoints - usually `delete` endpoints - return `204` as the success response. -When handling `204 - No Content` responses, we cannot use `.json()` since it tries to parse the non-existant body content. - -When handling `204` responses, do not use `.json`, otherwise the promise will throw an error and will enter the `catch` statement: - -```javascript - Vue.http.delete('path') - .then(() => { - // success! - }) - .catch(() => { - // handle error - }) -``` - -## Headers -Headers are being parsed into a plain object in an interceptor. -In Vue-resource 1.x `headers` object was changed into an `Headers` object. In order to not change all old code, an interceptor was added. - -If you need to write a unit test that takes the headers in consideration, you need to include an interceptor to parse the headers after your test interceptor. -You can see an example in `spec/javascripts/environments/environment_spec.js`: - ```javascript - import { headersInterceptor } from './helpers/vue_resource_helper'; - - beforeEach(() => { - Vue.http.interceptors.push(myInterceptor); - Vue.http.interceptors.push(headersInterceptor); - }); - - afterEach(() => { - Vue.http.interceptors = _.without(Vue.http.interceptors, myInterceptor); - Vue.http.interceptors = _.without(Vue.http.interceptors, headersInterceptor); - }); - ``` - -## CSRF token -We use a Vue Resource interceptor to manage the CSRF token. -`app/assets/javascripts/vue_shared/vue_resource_interceptor.js` holds all our common interceptors. -Note: You don't need to load `app/assets/javascripts/vue_shared/vue_resource_interceptor.js` -since it's already being loaded by `common_vue.js`. - - -[vue-resource-repo]: https://github.com/pagekit/vue-resource diff --git a/doc/development/file_storage.md b/doc/development/file_storage.md new file mode 100644 index 00000000000..cf00e24e11a --- /dev/null +++ b/doc/development/file_storage.md @@ -0,0 +1,49 @@ +# File Storage in GitLab + +We use the [CarrierWave] gem to handle file upload, store and retrieval. + +There are many places where file uploading is used, according to contexts: + +* System + - Instance Logo (logo visible in sign in/sign up pages) + - Header Logo (one displayed in the navigation bar) +* Group + - Group avatars +* User + - User avatars + - User snippet attachments +* Project + - Project avatars + - Issues/MR Markdown attachments + - Issues/MR Legacy Markdown attachments + - CI Build Artifacts + - LFS Objects + + +## Disk storage + +GitLab started saving everything on local disk. While directory location changed from previous versions, +they are still not 100% standardized. You can see them below: + +| Description | In DB? | Relative path | Uploader class | model_type | +| ------------------------------------- | ------ | ----------------------------------------------------------- | ---------------------- | ---------- | +| Instance logo | yes | uploads/-/system/appearance/logo/:id/:filename | `AttachmentUploader` | Appearance | +| Header logo | yes | uploads/-/system/appearance/header_logo/:id/:filename | `AttachmentUploader` | Appearance | +| Group avatars | yes | uploads/-/system/group/avatar/:id/:filename | `AvatarUploader` | Group | +| User avatars | yes | uploads/-/system/user/avatar/:id/:filename | `AvatarUploader` | User | +| User snippet attachments | yes | uploads/-/system/personal_snippet/:id/:random_hex/:filename | `PersonalFileUploader` | Snippet | +| Project avatars | yes | uploads/-/system/project/avatar/:id/:filename | `AvatarUploader` | Project | +| Issues/MR Markdown attachments | yes | uploads/:project_path_with_namespace/:random_hex/:filename | `FileUploader` | Project | +| Issues/MR Legacy Markdown attachments | no | uploads/-/system/note/attachment/:id/:filename | `AttachmentUploader` | Note | +| CI Artifacts (CE) | yes | shared/artifacts/:year_:month/:project_id/:id | `ArtifactUploader` | Ci::Build | +| LFS Objects (CE) | yes | shared/lfs-objects/:hex/:hex/:object_hash | `LfsObjectUploader` | LfsObject | + +CI Artifacts and LFS Objects behave differently in CE and EE. In CE they inherit the `GitlabUploader` +while in EE they inherit the `ObjectStoreUploader` and store files in and S3 API compatible object store. + +In the case of Issues/MR Markdown attachments, there is a different approach using the [Hashed Storage] layout, +instead of basing the path into a mutable variable `:project_path_with_namespace`, it's possible to use the +hash of the project ID instead, if project migrates to the new approach (introduced in 10.2). + +[CarrierWave]: https://github.com/carrierwaveuploader/carrierwave +[Hashed Storage]: ../administration/repository_storage_types.md diff --git a/doc/development/github_importer.md b/doc/development/github_importer.md new file mode 100644 index 00000000000..0d558583bb8 --- /dev/null +++ b/doc/development/github_importer.md @@ -0,0 +1,209 @@ +# Working with the GitHub importer + +In GitLab 10.2 a new version of the GitHub importer was introduced. This new +importer performs its work in parallel using Sidekiq, greatly reducing the time +necessary to import GitHub projects into a GitLab instance. + +The GitHub importer offers two different types of importers: a sequential +importer and a parallel importer. The Rake task `import:github` uses the +sequential importer, while everything else uses the parallel importer. The +difference between these two importers is quite simple: the sequential importer +does all work in a single thread, making it more useful for debugging purposes +or Rake tasks. The parallel importer on the other hand uses Sidekiq. + +## Requirements + +* GitLab CE 10.2.0 or newer. +* Sidekiq workers that process the `github_importer` and + `github_importer_advance_stage` queues (this is enabled by default). +* Octokit (used for interacting with the GitHub API) + +## Code structure + +The importer's codebase is broken up into the following directories: + +* `lib/gitlab/github_import`: this directory contains most of the code such as + the classes used for importing resources. +* `app/workers/gitlab/github_import`: this directory contains the Sidekiq + workers. +* `app/workers/concerns/gitlab/github_import`: this directory contains a few + modules reused by the various Sidekiq workers. + +## Architecture overview + +When a GitHub project is imported we schedule and execute a job for the +`RepositoryImportworker` worker as all other importers. However, unlike other +importers we don't immediately perform the work necessary. Instead work is +divided into separate stages, with each stage consisting out of a set of Sidekiq +jobs that are executed. Between every stage a job is scheduled that periodically +checks if all work of the current stage is completed, advancing the import +process to the next stage when this is the case. The worker handling this is +called `Gitlab::GithubImport::AdvanceStageWorker`. + +## Stages + +### 1. RepositoryImportWorker + +This worker will kick off the import process by simply scheduling a job for the +next worker. + +### 2. Stage::ImportRepositoryWorker + +This worker will import the repository and wiki, scheduling the next stage when +done. + +### 3. Stage::ImportBaseDataWorker + +This worker will import base data such as labels, milestones, and releases. This +work is done in a single thread since it can be performed fast enough that we +don't need to perform this work in parallel. + +### 4. Stage::ImportPullRequestsWorker + +This worker will import all pull requests. For every pull request a job for the +`Gitlab::GithubImport::ImportPullRequestWorker` worker is scheduled. + +### 5. Stage::ImportIssuesAndDiffNotesWorker + +This worker will import all issues and pull request comments. For every issue we +schedule a job for the `Gitlab::GithubImport::ImportIssueWorker` worker. For +pull request comments we instead schedule jobs for the +`Gitlab::GithubImport::DiffNoteImporter` worker. + +This worker processes both issues and diff notes in parallel so we don't need to +schedule a separate stage and wait for the previous one to complete. + +Issues are imported separately from pull requests because only the "issues" API +includes labels for both issue and pull requests. Importing issues and setting +label links in the same worker removes the need for performing a separate crawl +through the API data, reducing the number of API calls necessary to import a +project. + +### 6. Stage::ImportNotesWorker + +This worker imports regular comments for both issues and pull requests. For +every comment we schedule a job for the +`Gitlab::GithubImport::ImportNoteWorker` worker. + +Regular comments have to be imported at the end since the GitHub API used +returns comments for both issues and pull requests. This means we have to wait +for all issues and pull requests to be imported before we can import regular +comments. + +### 7. Stage::FinishImportWorker + +This worker will wrap up the import process by performing some housekeeping +(such as flushing any caches) and by marking the import as completed. + +## Advancing stages + +Advancing stages is done in one of two ways: + +1. Scheduling the worker for the next stage directly. +2. Scheduling a job for `Gitlab::GithubImport::AdvanceStageWorker` which will + advance the stage when all work of the current stage has been completed. + +The first approach should only be used by workers that perform all their work in +a single thread, while `AdvanceStageWorker` should be used for everything else. + +The way `AdvanceStageWorker` works is fairly simple. When scheduling a job it +will be given a project ID, a list of Redis keys, and the name of the next +stage. The Redis keys (produced by `Gitlab::JobWaiter`) are used to check if the +currently running stage has been completed or not. If the stage has not yet been +completed `AdvanceStageWorker` will reschedule itself. Once a stage finishes +`AdvanceStageworker` will refresh the import JID (more on this below) and +schedule the worker of the next stage. + +To reduce the number of `AdvanceStageWorker` jobs scheduled this worker will +briefly wait for jobs to complete before deciding what the next action should +be. For small projects this may slow down the import process a bit, but it will +also reduce pressure on the system as a whole. + +## Refreshing import JIDs + +GitLab includes a worker called `StuckImportJobsWorker` that will periodically +run and mark project imports as failed if they have been running for more than +15 hours. For GitHub projects this poses a bit of a problem: importing large +projects could take several hours depending on how often we hit the GitHub rate +limit (more on this below), but we don't want `StuckImportJobsWorker` to mark +our import as failed because of this. + +To prevent this from happening we periodically refresh the expiration time of +the import process. This works by storing the JID of the import job in the +database, then refreshing this JID's TTL at various stages throughout the import +process. This is done by calling `Project#refresh_import_jid_expiration`. By +refreshing this TTL we can ensure our import does not get marked as failed so +long we're still performing work. + +## GitHub rate limit + +GitHub has a rate limit of 5 000 API calls per hour. The number of requests +necessary to import a project is largely dominated by the number of unique users +involved in a project (e.g. issue authors). Other data such as issue pages +and comments typically only requires a few dozen requests to import. This is +because we need the Email address of users in order to map them to GitLab users. + +We handle this by doing the following: + +1. Once we hit the rate limit all jobs will automatically reschedule themselves + in such a way that they are not executed until the rate limit has been reset. +2. We cache the mapping of GitHub users to GitLab users in Redis. + +More information on user caching can be found below. + +## Caching user lookups + +When mapping GitHub users to GitLab users we need to (in the worst case) +perform: + +1. One API call to get the user's Email address. +2. Two database queries to see if a corresponding GitLab user exists. One query + will try to find the user based on the GitHub user ID, while the second query + is used to find the user using their GitHub Email address. + +Because this process is quite expensive we cache the result of these lookups in +Redis. For every user looked up we store three keys: + +1. A Redis key mapping GitHub usernames to their Email addresses. +2. A Redis key mapping a GitHub Email addresses to a GitLab user ID. +3. A Redis key mapping a GitHub user ID to GitLab user ID. + +There are two types of lookups we cache: + +1. A positive lookup, meaning we found a GitLab user ID. +2. A negative lookup, meaning we didn't find a GitLab user ID. Caching this + prevents us from performing the same work for users that we know don't exist + in our GitLab database. + +The expiration time of these keys is 24 hours. When retrieving the cache of a +positive lookups we refresh the TTL automatically. The TTL of false lookups is +never refreshed. + +Because of this caching layer it's possible newly registered GitLab accounts +won't be linked to their corresponding GitHub accounts. This however will sort +itself out once the cached keys expire. + +The user cache lookup is shared across projects. This means that the more +projects get imported the fewer GitHub API calls will be needed. + +The code for this resides in: + +* `lib/gitlab/github_import/user_finder.rb` +* `lib/gitlab/github_import/caching.rb` + +## Mapping labels and milestones + +To reduce pressure on the database we do not query it when setting labels and +milestones on issues and merge requests. Instead we cache this data when we +import labels and milestones, then we reuse this cache when assigning them to +issues/merge requests. Similar to the user lookups these cache keys are expired +automatically after 24 hours of not being used. + +Unlike the user lookup caches these label and milestone caches are scoped to the +project that is being imported. + +The code for this resides in: + +* `lib/gitlab/github_import/label_finder.rb` +* `lib/gitlab/github_import/milestone_finder.rb` +* `lib/gitlab/github_import/caching.rb` diff --git a/doc/development/i18n/externalization.md b/doc/development/i18n/externalization.md index 7c38260406d..4b65a0f4a35 100644 --- a/doc/development/i18n/externalization.md +++ b/doc/development/i18n/externalization.md @@ -110,7 +110,7 @@ You can mark that content for translation with: In JavaScript we added the `__()` (double underscore parenthesis) function for translations. -### Updating the PO files with the new content +## Updating the PO files with the new content Now that the new content is marked for translation, we need to update the PO files with the following command: @@ -119,23 +119,20 @@ files with the following command: bundle exec rake gettext:find ``` -This command will update the `locale/**/gitlab.edit.po` file with the -new content that the parser has found. +This command will update the `locale/gitlab.pot` file with the newly externalized +strings and remove any strings that aren't used anymore. You should check this +file in. Once the changes are on master, they will be picked up by +[Crowdin](http://translate.gitlab.com) and be presented for translation. -New translations will be added with their default content and will be marked -fuzzy. To use the translation, look for the `#, fuzzy` mention in `gitlab.edit.po` -and remove it. +The command also updates the translation files for each language: `locale/*/gitlab.po` +These changes can be discarded, the languange files will be updated by Crowdin +automatically. -We need to make sure we remove the `fuzzy` translations before generating the -`locale/**/gitlab.po` file. When they aren't removed, the resulting `.po` will -be treated as a binary file which could overwrite translations that were merged -before the new translations. +Discard all of them at once like this: -When we are just preparing a page to be translated, but not actually adding any -translations. There's no need to generate `.po` files. - -Translations that aren't used in the source code anymore will be marked with -`~#`; these can be removed to keep our translation files clutter-free. +```sh +git checkout locale/*/gitlab.po +``` ### Validating PO files diff --git a/doc/development/licensing.md b/doc/development/licensing.md index 902b1c74a42..274923c2d43 100644 --- a/doc/development/licensing.md +++ b/doc/development/licensing.md @@ -4,11 +4,11 @@ GitLab CE is licensed under the terms of the MIT License. GitLab EE is licensed ## Automated Testing -In order to comply with the terms the libraries we use are licensed under, we have to make sure to check new gems for compatible licenses whenever they're added. To automate this process, we use the [license_finder][license_finder] gem by Pivotal. It runs every time a new commit is pushed and verifies that all gems in the bundle use a license that doesn't conflict with the licensing of either GitLab Community Edition or GitLab Enterprise Edition. +In order to comply with the terms the libraries we use are licensed under, we have to make sure to check new gems for compatible licenses whenever they're added. To automate this process, we use the [license_finder][license_finder] gem by Pivotal. It runs every time a new commit is pushed and verifies that all gems and node modules in the bundle use a license that doesn't conflict with the licensing of either GitLab Community Edition or GitLab Enterprise Edition. -There are some limitations with the automated testing, however. CSS and JavaScript libraries, as well as any Ruby libraries not included by way of Bundler, must be verified manually and independently. Take care whenever one such library is used, as automated tests won't catch problematic licenses from them. +There are some limitations with the automated testing, however. CSS, JavaScript, or Ruby libraries which are not included by way of Bundler, NPM, or Yarn (for instance those manually copied into our source tree in the `vendor` directory), must be verified manually and independently. Take care whenever one such library is used, as automated tests won't catch problematic licenses from them. -Some gems may not include their license information in their `gemspec` file. These won't be detected by License Finder, and will have to be verified manually. +Some gems may not include their license information in their `gemspec` file, and some node modules may not include their license information in their `package.json` file. These won't be detected by License Finder, and will have to be verified manually. ### License Finder commands diff --git a/doc/development/limit_ee_conflicts.md b/doc/development/limit_ee_conflicts.md index 899be9eae4b..ba82babb38a 100644 --- a/doc/development/limit_ee_conflicts.md +++ b/doc/development/limit_ee_conflicts.md @@ -336,6 +336,12 @@ Blocks of code that are EE-specific should be moved to partials as much as possible to avoid conflicts with big chunks of HAML code that that are not fun to resolve when you add the indentation in the equation. +### Assets + +#### gitlab-svgs + +Conflicts in `app/assets/images/icons.json` or `app/assets/images/icons.svg` can be resolved simply by regenerating those assets with [`yarn run svg`](https://gitlab.com/gitlab-org/gitlab-svgs). + --- [Return to Development documentation](README.md) diff --git a/doc/development/migration_style_guide.md b/doc/development/migration_style_guide.md index 9b8ab5da74e..a235dd74909 100644 --- a/doc/development/migration_style_guide.md +++ b/doc/development/migration_style_guide.md @@ -198,7 +198,43 @@ end Keep in mind that this operation can easily take 10-15 minutes to complete on larger installations (e.g. GitLab.com). As a result you should only add default -values if absolutely necessary. +values if absolutely necessary. There is a RuboCop cop that will fail if this +method is used on some tables that are very large on GitLab.com, which would +cause other issues. + +## Updating an existing column + +To update an existing column to a particular value, you can use +`update_column_in_batches` (`add_column_with_default` uses this internally to +fill in the default value). This will split the updates into batches, so we +don't update too many rows at in a single statement. + +This updates the column `foo` in the `projects` table to 10, where `some_column` +is `'hello'`: + +```ruby +update_column_in_batches(:projects, :foo, 10) do |table, query| + query.where(table[:some_column].eq('hello')) +end +``` + +To perform a computed update, the value can be wrapped in `Arel.sql`, so Arel +treats it as an SQL literal. The below example is the same as the one above, but +the value is set to the product of the `bar` and `baz` columns: + +```ruby +update_value = Arel.sql('bar * baz') + +update_column_in_batches(:projects, :foo, update_value) do |table, query| + query.where(table[:some_column].eq('hello')) +end +``` + +Like `add_column_with_default`, there is a RuboCop cop to detect usage of this +on large tables. In the case of `update_column_in_batches`, it may be acceptable +to run on a large table, as long as it is only updating a small subset of the +rows in the table, but do not ignore that without validating on the GitLab.com +staging environment - or asking someone else to do so for you - beforehand. ## Integer column type diff --git a/doc/development/query_recorder.md b/doc/development/query_recorder.md index e0127aaed4c..12e90101139 100644 --- a/doc/development/query_recorder.md +++ b/doc/development/query_recorder.md @@ -22,6 +22,52 @@ As an example you might create 5 issues in between counts, which would cause the > **Note:** In some cases the query count might change slightly between runs for unrelated reasons. In this case you might need to test `exceed_query_limit(control_count + acceptable_change)`, but this should be avoided if possible. +## Finding the source of the query + +It may be useful to identify the source of the queries by looking at the call backtrace. +To enable this, run the specs with the `QUERY_RECORDER_DEBUG` environment variable set. For example: + +``` +QUERY_RECORDER_DEBUG=1 bundle exec rspec spec/requests/api/projects_spec.rb +``` + +This will log calls to QueryRecorder into the `test.log`. For example: + +``` +QueryRecorder SQL: SELECT COUNT(*) FROM "issues" WHERE "issues"."deleted_at" IS NULL AND "issues"."project_id" = $1 AND ("issues"."state" IN ('opened')) AND "issues"."confidential" = $2 + --> /home/user/gitlab/gdk/gitlab/spec/support/query_recorder.rb:19:in `callback' + --> /home/user/.rbenv/versions/2.3.5/lib/ruby/gems/2.3.0/gems/activesupport-4.2.8/lib/active_support/notifications/fanout.rb:127:in `finish' + --> /home/user/.rbenv/versions/2.3.5/lib/ruby/gems/2.3.0/gems/activesupport-4.2.8/lib/active_support/notifications/fanout.rb:46:in `block in finish' + --> /home/user/.rbenv/versions/2.3.5/lib/ruby/gems/2.3.0/gems/activesupport-4.2.8/lib/active_support/notifications/fanout.rb:46:in `each' + --> /home/user/.rbenv/versions/2.3.5/lib/ruby/gems/2.3.0/gems/activesupport-4.2.8/lib/active_support/notifications/fanout.rb:46:in `finish' + --> /home/user/.rbenv/versions/2.3.5/lib/ruby/gems/2.3.0/gems/activesupport-4.2.8/lib/active_support/notifications/instrumenter.rb:36:in `finish' + --> /home/user/.rbenv/versions/2.3.5/lib/ruby/gems/2.3.0/gems/activesupport-4.2.8/lib/active_support/notifications/instrumenter.rb:25:in `instrument' + --> /home/user/.rbenv/versions/2.3.5/lib/ruby/gems/2.3.0/gems/activerecord-4.2.8/lib/active_record/connection_adapters/abstract_adapter.rb:478:in `log' + --> /home/user/.rbenv/versions/2.3.5/lib/ruby/gems/2.3.0/gems/activerecord-4.2.8/lib/active_record/connection_adapters/postgresql_adapter.rb:601:in `exec_cache' + --> /home/user/.rbenv/versions/2.3.5/lib/ruby/gems/2.3.0/gems/activerecord-4.2.8/lib/active_record/connection_adapters/postgresql_adapter.rb:585:in `execute_and_clear' + --> /home/user/.rbenv/versions/2.3.5/lib/ruby/gems/2.3.0/gems/activerecord-4.2.8/lib/active_record/connection_adapters/postgresql/database_statements.rb:160:in `exec_query' + --> /home/user/.rbenv/versions/2.3.5/lib/ruby/gems/2.3.0/gems/activerecord-4.2.8/lib/active_record/connection_adapters/abstract/database_statements.rb:356:in `select' + --> /home/user/.rbenv/versions/2.3.5/lib/ruby/gems/2.3.0/gems/activerecord-4.2.8/lib/active_record/connection_adapters/abstract/database_statements.rb:32:in `select_all' + --> /home/user/.rbenv/versions/2.3.5/lib/ruby/gems/2.3.0/gems/activerecord-4.2.8/lib/active_record/connection_adapters/abstract/query_cache.rb:68:in `block in select_all' + --> /home/user/.rbenv/versions/2.3.5/lib/ruby/gems/2.3.0/gems/activerecord-4.2.8/lib/active_record/connection_adapters/abstract/query_cache.rb:83:in `cache_sql' + --> /home/user/.rbenv/versions/2.3.5/lib/ruby/gems/2.3.0/gems/activerecord-4.2.8/lib/active_record/connection_adapters/abstract/query_cache.rb:68:in `select_all' + --> /home/user/.rbenv/versions/2.3.5/lib/ruby/gems/2.3.0/gems/activerecord-4.2.8/lib/active_record/relation/calculations.rb:270:in `execute_simple_calculation' + --> /home/user/.rbenv/versions/2.3.5/lib/ruby/gems/2.3.0/gems/activerecord-4.2.8/lib/active_record/relation/calculations.rb:227:in `perform_calculation' + --> /home/user/.rbenv/versions/2.3.5/lib/ruby/gems/2.3.0/gems/activerecord-4.2.8/lib/active_record/relation/calculations.rb:133:in `calculate' + --> /home/user/.rbenv/versions/2.3.5/lib/ruby/gems/2.3.0/gems/activerecord-4.2.8/lib/active_record/relation/calculations.rb:48:in `count' + --> /home/user/gitlab/gdk/gitlab/app/services/base_count_service.rb:20:in `uncached_count' + --> /home/user/gitlab/gdk/gitlab/app/services/base_count_service.rb:12:in `block in count' + --> /home/user/.rbenv/versions/2.3.5/lib/ruby/gems/2.3.0/gems/activesupport-4.2.8/lib/active_support/cache.rb:299:in `block in fetch' + --> /home/user/.rbenv/versions/2.3.5/lib/ruby/gems/2.3.0/gems/activesupport-4.2.8/lib/active_support/cache.rb:585:in `block in save_block_result_to_cache' + --> /home/user/.rbenv/versions/2.3.5/lib/ruby/gems/2.3.0/gems/activesupport-4.2.8/lib/active_support/cache.rb:547:in `block in instrument' + --> /home/user/.rbenv/versions/2.3.5/lib/ruby/gems/2.3.0/gems/activesupport-4.2.8/lib/active_support/notifications.rb:166:in `instrument' + --> /home/user/.rbenv/versions/2.3.5/lib/ruby/gems/2.3.0/gems/activesupport-4.2.8/lib/active_support/cache.rb:547:in `instrument' + --> /home/user/.rbenv/versions/2.3.5/lib/ruby/gems/2.3.0/gems/activesupport-4.2.8/lib/active_support/cache.rb:584:in `save_block_result_to_cache' + --> /home/user/.rbenv/versions/2.3.5/lib/ruby/gems/2.3.0/gems/activesupport-4.2.8/lib/active_support/cache.rb:299:in `fetch' + --> /home/user/gitlab/gdk/gitlab/app/services/base_count_service.rb:12:in `count' + --> /home/user/gitlab/gdk/gitlab/app/models/project.rb:1296:in `open_issues_count' +``` + ## See also - [Bullet](profiling.md#Bullet) For finding `N+1` query problems diff --git a/doc/development/rake_tasks.md b/doc/development/rake_tasks.md index bfd80aab6a4..4773b6773e8 100644 --- a/doc/development/rake_tasks.md +++ b/doc/development/rake_tasks.md @@ -122,6 +122,15 @@ they can be easily inspected. bundle exec rake services:doc ``` +## Updating Emoji Aliases + +To update the Emoji aliases file (used for Emoji autocomplete) you must run the +following: + +``` +bundle exec rake gemojione:aliases +``` + ## Updating Emoji Digests To update the Emoji digests file (used for Emoji autocomplete) you must run the @@ -131,6 +140,7 @@ following: bundle exec rake gemojione:digests ``` + This will update the file `fixtures/emojis/digests.json` based on the currently available Emoji. diff --git a/doc/development/testing_guide/best_practices.md b/doc/development/testing_guide/best_practices.md index 7ddd02e6c73..8b7b015427f 100644 --- a/doc/development/testing_guide/best_practices.md +++ b/doc/development/testing_guide/best_practices.md @@ -60,6 +60,35 @@ writing one](testing_levels.md#consider-not-writing-a-system-test)! - It's ok to look for DOM elements but don't abuse it since it makes the tests more brittle +#### Debugging Capybara + +Sometimes you may need to debug Capybara tests by observing browser behavior. + +You can pause Capybara and view the website on the browser by using the +`live_debug` method in your spec. The current page will be automatically opened +in your default browser. +You may need to sign in first (the current user's credentials are displayed in +the terminal). + +To resume the test run, press any key. + +For example: + +``` +$ bin/rspec spec/features/auto_deploy_spec.rb:34 +Running via Spring preloader in process 8999 +Run options: include {:locations=>{"./spec/features/auto_deploy_spec.rb"=>[34]}} + +Current example is paused for live debugging +The current user credentials are: user2 / 12345678 +Press any key to resume the execution of the example! +Back to the example! +. + +Finished in 34.51 seconds (files took 0.76702 seconds to load) +1 example, 0 failures +``` + ### `let` variables GitLab's RSpec suite has made extensive use of `let` variables to reduce diff --git a/doc/development/testing_guide/testing_levels.md b/doc/development/testing_guide/testing_levels.md index 9b9ba0baa71..1cbd4350284 100644 --- a/doc/development/testing_guide/testing_levels.md +++ b/doc/development/testing_guide/testing_levels.md @@ -126,7 +126,7 @@ always in-sync with the codebase. [GitLab Workhorse]: https://gitlab.com/gitlab-org/gitlab-workhorse [Gitaly]: https://gitlab.com/gitlab-org/gitaly [GitLab Pages]: https://gitlab.com/gitlab-org/gitlab-pages -[GitLab Runner]: https://gitlab.com/gitlab-org/gitlab-ci-multi-runner +[GitLab Runner]: https://gitlab.com/gitlab-org/gitlab-runner [GitLab Omnibus]: https://gitlab.com/gitlab-org/omnibus-gitlab [GitLab QA]: https://gitlab.com/gitlab-org/gitlab-qa [part of GitLab Rails]: https://gitlab.com/gitlab-org/gitlab-ce/tree/master/qa diff --git a/doc/development/testing_guide/testing_rake_tasks.md b/doc/development/testing_guide/testing_rake_tasks.md index 5bf185dd7b5..60163f1a230 100644 --- a/doc/development/testing_guide/testing_rake_tasks.md +++ b/doc/development/testing_guide/testing_rake_tasks.md @@ -1,4 +1,4 @@ -## Testing Rake tasks +# Testing Rake tasks To make testing Rake tasks a little easier, there is a helper that can be included in lieu of the standard Spec helper. Instead of `require 'spec_helper'`, use diff --git a/doc/development/ux_guide/components.md b/doc/development/ux_guide/components.md index fa31c496b30..16a811dbc74 100644 --- a/doc/development/ux_guide/components.md +++ b/doc/development/ux_guide/components.md @@ -10,6 +10,7 @@ * [Tables](#tables) * [Blocks](#blocks) * [Panels](#panels) +* [Dialog modals](#dialog-modals) * [Alerts](#alerts) * [Forms](#forms) * [Search box](#search-box) @@ -254,6 +255,38 @@ Skeleton loading can replace any existing UI elements for the period in which th --- +## Dialog modals + +Dialog modals are only used for having a conversation and confirmation with the user. The user is not able to access the features on the main page until closing the modal. + +### Usage + +* When the action is irreversible, dialog modals provide the details and confirm with the user before they take an advanced action. +* When the action will affect privacy or authorization, dialog modals provide advanced information and confirm with the user. + +### Style + +* Dialog modals contain the header, body, and actions. + * **Header(1):** The header title is a question instead of a descriptive phrase. + * **Body(2):** The content in body should never be ambiguous and unclear. It provides specific information. + * **Actions(3):** Contains a affirmative action, a dismissive action, and an extra action. The order of actions from left to right: Dismissive action → Extra action → Affirmative action +* Confirmations regarding labels should keep labeling styling. +* References to commits, branches, and tags should be **monospaced**. + + + +### Placement + +* Dialog modals should always be the center of the screen horizontally and be positioned **72px** from the top. + +| Dialog with 2 actions | Dialog with 3 actions | Special confirmation | +| --------------------- | --------------------- | -------------------- | +|  |  |  | + +> TODO: Special case for dialog modal. + +--- + ## Panels > TODO: Catalog how we are currently using panels and rationalize how they relate to alerts diff --git a/doc/development/ux_guide/img/modals-general-confimation-dialog.png b/doc/development/ux_guide/img/modals-general-confimation-dialog.png Binary files differnew file mode 100644 index 00000000000..00a17374a0b --- /dev/null +++ b/doc/development/ux_guide/img/modals-general-confimation-dialog.png diff --git a/doc/development/ux_guide/img/modals-layout-for-modals.png b/doc/development/ux_guide/img/modals-layout-for-modals.png Binary files differnew file mode 100644 index 00000000000..6c7bc09e750 --- /dev/null +++ b/doc/development/ux_guide/img/modals-layout-for-modals.png diff --git a/doc/development/ux_guide/img/modals-special-confimation-dialog.png b/doc/development/ux_guide/img/modals-special-confimation-dialog.png Binary files differnew file mode 100644 index 00000000000..bf1e56326c5 --- /dev/null +++ b/doc/development/ux_guide/img/modals-special-confimation-dialog.png diff --git a/doc/development/ux_guide/img/modals-three-buttons.png b/doc/development/ux_guide/img/modals-three-buttons.png Binary files differnew file mode 100644 index 00000000000..519439e64e4 --- /dev/null +++ b/doc/development/ux_guide/img/modals-three-buttons.png diff --git a/doc/development/ux_guide/resources.md b/doc/development/ux_guide/resources.md index 2f760c94414..db57258237f 100644 --- a/doc/development/ux_guide/resources.md +++ b/doc/development/ux_guide/resources.md @@ -10,4 +10,8 @@ you can use during GitLab development. ## Design repository All design files are stored in the [gitlab-design](https://gitlab.com/gitlab-org/gitlab-design) -repository and maintained by GitLab UX designers.
\ No newline at end of file +repository and maintained by GitLab UX designers. + +## [brand.ai](https://brand.ai/git-lab/primary-brand) + +We are in the process of capturing our UI paradigms on brand.ai, see https://brand.ai/git-lab/primary-brand diff --git a/doc/development/ux_guide/users.md b/doc/development/ux_guide/users.md index cbd7c17de41..fce882a45f1 100644 --- a/doc/development/ux_guide/users.md +++ b/doc/development/ux_guide/users.md @@ -1,4 +1,5 @@ -## UX Personas +# UX Personas + * [Nazim Ramesh](#nazim-ramesh) - Small to medium size organisations using GitLab CE * [James Mackey](#james-mackey) @@ -7,16 +8,16 @@ * [Karolina Plaskaty](#karolina-plaskaty) - Using GitLab.com for personal/hobby projects - Would like to use GitLab at work - - Working for a medium to large size organisation + - Working for a medium to large size organisation -<hr> +--- -### Nazim Ramesh +## Nazim Ramesh - Small to medium size organisations using GitLab CE <img src="img/nazim-ramesh.png" width="300px"> -#### Demographics +### Demographics - **Age**<br>32 years old - **Location**<br>Germany @@ -26,7 +27,7 @@ - **Frequently used programming languages**<br>JavaScript, SQL, PHP - **Hobbies / interests**<br>Functional programming, open source, gaming, web development and web security. -#### Motivations +### Motivations Nazim works for a software development company which currently hires around 80 people. When Nazim first joined the company, the engineering team were using Subversion (SVN) as their primary form of source control. However, Nazim felt SVN was not flexible enough to work with many feature branches and noticed that developers with less experience of source control struggled with the central-repository nature of SVN. Armed with a wishlist of features, Nazim began comparing source control tools. A search for “self-hosted Git server repository management” returned GitLab. In his own words, Nazim explains why he wanted the engineering team to start using GitLab: > @@ -39,48 +40,48 @@ In his role as a full-stack web developer, Nazim could recommend products that h “The biggest challenge...why should we change anything at all from the status quo? We needed to switch from SVN to Git. They knew they needed to learn Git and a Git workflow...using Git was scary to my colleagues...they thought it was more complex than SVN to use.” > -Undeterred, Nazim decided to migrate a couple of projects across to GitLab. +Undeterred, Nazim decided to migrate a couple of projects across to GitLab. > “Old SVN users couldn’t see the benefits of Git at first. It took a month or two to convince them.” > -Slowly, by showing his colleagues how easy it was to use Git, the majority of the team’s projects were migrated to GitLab. +Slowly, by showing his colleagues how easy it was to use Git, the majority of the team’s projects were migrated to GitLab. -The engineering team have been using GitLab CE for around 2 years now. Nazim credits himself as being entirely responsible for his company’s decision to move to GitLab. +The engineering team have been using GitLab CE for around 2 years now. Nazim credits himself as being entirely responsible for his company’s decision to move to GitLab. -#### Frustrations -##### Adoption to GitLab has been slow +### Frustrations +#### Adoption to GitLab has been slow Not only has the engineering team had to get to grips with Git, they’ve also had to adapt to using GitLab. Due to lack of training and existing skills in other tools, the full feature set of GitLab CE is not being utilised. Nazim sold GitLab to his manager as an ‘all in one’ tool which would replace multiple tools used within the company, thus saving costs. Nazim hasn’t had the time to integrate the legacy tools to GitLab and he’s struggling to convince his peers to change their habits. -##### Missing Features +#### Missing Features Nazim’s company want GitLab to be able to do everything. There isn’t a large budget for software, so they’re selective about what tools are implemented. It needs to add real value to the company. In order for GitLab to be widely adopted and to meet the requirements of different roles within the company, it needs a host of features. When an individual within Nazim’s company wants to know if GitLab has a specific feature or does a particular thing, Nazim is the person to ask. He becomes the point of contact to investigate, build or sometimes just raise the feature request. Nazim gets frustrated when GitLab isn’t able to do what he or his colleagues need it to do. -##### Regressions and bugs +#### Regressions and bugs Nazim often has to calm down his colleagues, when a release contains regressions or new bugs. As he puts it “every new version adds something awesome, but breaks something”. He feels that “old issues for "minor" annoyances get quickly buried in the mass of open issues and linger for a very long time. More generally, I have the feeling that GitLab focus on adding new functionalities, but overlook a bunch of annoying minor regressions or introduced bugs.” Due to limited resource and expertise within the team, not only is it difficult to remain up-to-date with the frequent release cycle, it’s also counterproductive to fix workflows every month. -##### Uses too much RAM and CPU +#### Uses too much RAM and CPU > “Memory usages mean that if we host it from a cloud based host like AWS, we spend almost as much on the instance as what we would pay GitHub” > -##### UI/UX +#### UI/UX GitLab’s interface initially attracted Nazim when he was comparing version control software. He thought it would help his less technical colleagues to adapt to using Git and perhaps, GitLab could be rolled out to other areas of the business, beyond engineering. However, using GitLab’s interface daily has left him frustrated at the lack of personalisation / control over his user experience. He’s also regularly lost in a maze of navigation. Whilst he acknowledges that GitLab listens to its users and that the interface is improving, he becomes annoyed when the changes are too progressive. “Too frequent UI changes. Most of them tend to turn out great after a few cycles of fixes, but the frequency is still far too high for me to feel comfortable to always stay on the current release.” -#### Goals +### Goals * To convince his colleagues to fully adopt GitLab CE, thus improving workflow and collaboration. * To use a feature rich version control platform that covers all stages of the development lifecycle, in order to reduce dependencies on other tools. * To use an intuitive and stable product, so he can spend more time on his core job responsibilities and less time bug-fixing, guiding colleagues, etc. -<hr> +--- -### James Mackey +## James Mackey - Medium to large size organisations using CE or EE - Small organisations using EE <img src="img/james-mackey.png" width="300px"> -#### Demographics +### Demographics - **Age**<br>36 years old - **Location**<br>US @@ -90,7 +91,7 @@ GitLab’s interface initially attracted Nazim when he was comparing version con - **Frequently used programming languages**<br>JavaScript, SQL, Node.js, Java, PHP, Python - **Hobbies / interests**<br>DevOps, open source, web development, science, automation and electronics. -#### Motivations +### Motivations James works for a research company which currently hires around 800 staff. He began using GitLab.com back in 2013 for his own open source, hobby projects and loved “the simplicity of installation, administration and use”. After using GitLab for over a year, he began to wonder about using it at work. James explains: > @@ -99,7 +100,7 @@ James works for a research company which currently hires around 800 staff. He be James and his colleagues also reviewed competitor products including GitHub Enterprise, but they found it “less innovative and with considerable costs...GitLab had the features we wanted at a much lower cost per head than GitHub”. -The company James works for provides employees with a discretionary budget to spend how they want on software, so James and his team decided to upgrade to EE. +The company James works for provides employees with a discretionary budget to spend how they want on software, so James and his team decided to upgrade to EE. James feels partially responsible for his organisation’s decision to start using GitLab. @@ -107,33 +108,33 @@ James feels partially responsible for his organisation’s decision to start usi “It's still up to the teams themselves [to decide] which tools to use. We just had a great experience moving our daily development to GitLab, so other teams have followed the path or are thinking about switching.” > -#### Frustrations -##### Third Party Integration +### Frustrations +#### Third Party Integration Some of GitLab EE’s features are too basic, in particular, issues boards which do not have the level of reporting that James and his team need. Subsequently, they still need to use GitLab EE in conjunction with other tools, such as JIRA. Whilst James feels it isn’t essential for GitLab to meet all his needs (his company are happy for him to use, and pay for, multiple tools), he sometimes isn’t sure what is/isn’t possible with plugins and what level of custom development he and his team will need to do. -##### UX/UI +#### UX/UI James and his team use CI quite heavily for several projects. Whilst they’ve welcomed improvements to the builds and pipelines interface, they still have some difficulty following build process on the different tabs under Pipelines. Some confusion has arisen from not knowing where to find different pieces of information or how to get to the next stages logs from the current stage’s log output screen. They feel more intuitive linking and flow may alleviate the problem. Generally, they feel GitLab’s navigation needs to reviewed and optimised. -##### Permissions +#### Permissions > “There is no granular control over user or group permissions. The permissions for a project are too tightly coupled to the permissions for Gitlab CI/build pipelines.” > -#### Goals +### Goals * To be able to integrate third party tools easily with GitLab EE and to create custom integrations and patches where needed. * To use GitLab EE primarily for code hosting, merge requests, continuous integration and issue management. James and his team want to be able to understand and use these particular features easily. * To able to share one instance of GitLab EE with multiple teams across the business. Advanced user management, the ability to separate permissions on different parts of the source code, etc are important to James. -<hr> +--- -### Karolina Plaskaty +## Karolina Plaskaty - Using GitLab.com for personal/hobby projects - Would like to use GitLab at work -- Working for a medium to large size organisation +- Working for a medium to large size organisation <img src="img/karolina-plaskaty.png" width="300px"> -#### Demographics +### Demographics - **Age**<br>26 years old - **Location**<br>UK @@ -143,22 +144,22 @@ James and his team use CI quite heavily for several projects. Whilst they’ve w - **Frequently used programming languages**<br>JavaScript and SQL - **Hobbies / interests**<br>Web development, mobile development, UX, open source, gaming and travel. -#### Motivations +### Motivations Karolina has been using GitLab.com for around a year. She roughly spends 8 hours every week programming, of that, 2 hours is spent contributing to open source projects. Karolina contributes to open source projects to gain programming experience and to give back to the community. She likes GitLab.com for its free private repositories and range of features which provide her with everything she needs for her personal projects. Karolina is also a massive fan of GitLab’s values and the fact that it isn’t a “behemoth of a company”. She explains that “displaying every single thing (doc, culture, assumptions, development...) in the open gives me greater confidence to choose Gitlab personally and to recommend it at work.” She’s also an avid reader of GitLab’s blog. Karolina works for a software development company which currently hires around 500 people. Karolina would love to use GitLab at work but the company has used GitHub Enterprise for a number of years. She describes management at her company as “old fashioned” and explains that it’s “less of a technical issue and more of a cultural issue” to convince upper management to move to GitLab. Karolina is also relatively new to the company so she’s apprehensive about pushing too hard to change version control platforms. -#### Frustrations -##### Unable to use GitLab at work +### Frustrations +#### Unable to use GitLab at work Karolina wants to use GitLab at work but isn’t sure how to approach the subject with management. In her current role, she doesn’t feel that she has the authority to request GitLab. -##### Performance +#### Performance GitLab.com is frequently slow and unavailable. Karolina has also heard that GitLab is a “memory hog” which has deterred her from running GitLab on her own machine for just hobby / personal projects. -##### UX/UI +#### UX/UI Karolina has an interest in UX and therefore has strong opinions about how GitLab should look and feel. She feels the interface is cluttered, “it has too many links/buttons” and the navigation “feels a bit weird sometimes. I get lost if I don’t pay attention.” As Karolina also enjoys contributing to open-source projects, it’s important to her that GitLab is well designed for public repositories, she doesn’t feel that GitLab currently achieves this. -#### Goals +### Goals * To develop her programming experience and to learn from other developers. * To contribute to both her own and other open source projects. -* To use a fast and intuitive version control platform.
\ No newline at end of file +* To use a fast and intuitive version control platform. diff --git a/doc/gitlab-basics/README.md b/doc/gitlab-basics/README.md index 3d893ba53dd..4e15f7cfd49 100644 --- a/doc/gitlab-basics/README.md +++ b/doc/gitlab-basics/README.md @@ -1,3 +1,7 @@ +--- +comments: false +--- + # GitLab basics Step-by-step guides on the basics of working with Git and GitLab. diff --git a/doc/install/README.md b/doc/install/README.md index 656f8720361..540cb0d3f38 100644 --- a/doc/install/README.md +++ b/doc/install/README.md @@ -1,3 +1,7 @@ +--- +comments: false +--- + # Installation GitLab can be installed via various ways. Check the [installation methods][methods] diff --git a/doc/install/installation.md b/doc/install/installation.md index 2a004152d5e..4efe911b778 100644 --- a/doc/install/installation.md +++ b/doc/install/installation.md @@ -299,9 +299,9 @@ sudo usermod -aG redis git ### Clone the Source # Clone GitLab repository - sudo -u git -H git clone https://gitlab.com/gitlab-org/gitlab-ce.git -b 10-1-stable gitlab + sudo -u git -H git clone https://gitlab.com/gitlab-org/gitlab-ce.git -b 10-2-stable gitlab -**Note:** You can change `10-1-stable` to `master` if you want the *bleeding edge* version, but never install master on a production server! +**Note:** You can change `10-2-stable` to `master` if you want the *bleeding edge* version, but never install master on a production server! ### Configure It diff --git a/doc/install/relative_url.md b/doc/install/relative_url.md index 713d11b75e4..2f5d4142d04 100644 --- a/doc/install/relative_url.md +++ b/doc/install/relative_url.md @@ -1,11 +1,11 @@ -## Install GitLab under a relative URL +# Install GitLab under a relative URL -_**Note:** +NOTE: **Note:** This document describes how to run GitLab under a relative URL for installations from source. If you are using an Omnibus package, [the steps are different][omnibus-rel]. Use this guide along with the [installation guide](installation.md) if you are installing GitLab for the -first time._ +first time. --- @@ -33,7 +33,7 @@ serve GitLab under a relative URL is: After all the changes you need to recompile the assets and [restart GitLab]. -### Relative URL requirements +## Relative URL requirements If you configure GitLab with a relative URL, the assets (JavaScript, CSS, fonts, images, etc.) will need to be recompiled, which is a task which consumes a lot @@ -43,11 +43,11 @@ least 2GB of RAM available on your system, while we recommend 4GB RAM, and 4 or See the [requirements](requirements.md) document for more information. -### Enable relative URL in GitLab +## Enable relative URL in GitLab -_**Note:** +NOTE: **Note:** Do not make any changes to your web server configuration file regarding -relative URL. The relative URL support is implemented by GitLab Workhorse._ +relative URL. The relative URL support is implemented by GitLab Workhorse. --- @@ -115,7 +115,7 @@ Make sure to follow all steps below: 1. [Restart GitLab][] for the changes to take effect. -### Disable relative URL in GitLab +## Disable relative URL in GitLab To disable the relative URL: diff --git a/doc/install/requirements.md b/doc/install/requirements.md index 3d7becd18fc..baecf9455b0 100644 --- a/doc/install/requirements.md +++ b/doc/install/requirements.md @@ -80,13 +80,13 @@ errors during usage. - 256GB RAM supports up to 32,000 users - More users? Run it on [multiple application servers](https://about.gitlab.com/high-availability/) -We recommend having at least 2GB of swap on your server, even if you currently have +We recommend having at least [2GB of swap on your server](https://askubuntu.com/a/505344/310789), even if you currently have enough available RAM. Having swap will help reduce the chance of errors occurring if your available memory changes. We also recommend [configuring the kernel's swappiness setting](https://askubuntu.com/a/103916) to a low value like `10` to make the most of your RAM while still having the swap available when needed. -Notice: The 25 workers of Sidekiq will show up as separate processes in your process overview (such as top or htop) but they share the same RAM allocation since Sidekiq is a multithreaded application. Please see the section below about Unicorn workers for information about how many you need of those. +Notice: The 25 workers of Sidekiq will show up as separate processes in your process overview (such as `top` or `htop`) but they share the same RAM allocation since Sidekiq is a multithreaded application. Please see the section below about Unicorn workers for information about how many you need of those. ## Database @@ -121,7 +121,7 @@ Existing users using GitLab with MySQL/MariaDB are advised to ### PostgreSQL Requirements -As of GitLab 10.0, PostgreSQL 9.6 or newer is required, and earlier versions are +As of GitLab 10.0, PostgreSQL 9.6 or newer (but less than 10) is required, and earlier versions are not supported. We highly recommend users to use PostgreSQL 9.6 as this is the PostgreSQL version used for development and testing. @@ -146,7 +146,7 @@ So for a machine with 2 cores, 3 unicorn workers is ideal. For all machines that have 2GB and up we recommend a minimum of three unicorn workers. If you have a 1GB machine we recommend to configure only two Unicorn workers to prevent excessive swapping. -To change the Unicorn workers when you have the Omnibus package please see [the Unicorn settings in the Omnibus GitLab documentation](https://gitlab.com/gitlab-org/omnibus-gitlab/blob/master/doc/settings/unicorn.md#unicorn-settings). +To change the Unicorn workers when you have the Omnibus package (which defaults to the recommendation above) please see [the Unicorn settings in the Omnibus GitLab documentation](https://gitlab.com/gitlab-org/omnibus-gitlab/blob/master/doc/settings/unicorn.md#unicorn-settings). ## Redis and Sidekiq @@ -184,7 +184,7 @@ Runner. We recommend using a separate machine for each GitLab Runner, if you plan to use the CI features. -[security reasons]: https://gitlab.com/gitlab-org/gitlab-ci-multi-runner/blob/master/docs/security/index.md +[security reasons]: https://gitlab.com/gitlab-org/gitlab-runner/blob/master/docs/security/index.md ## Supported web browsers diff --git a/doc/integration/README.md b/doc/integration/README.md index 09d96bdd338..54e78bdef54 100644 --- a/doc/integration/README.md +++ b/doc/integration/README.md @@ -1,3 +1,7 @@ +--- +comments: false +--- + # GitLab Integration GitLab integrates with multiple third-party services to allow external issue diff --git a/doc/integration/external-issue-tracker.md b/doc/integration/external-issue-tracker.md index 372e1909330..075feaeead9 100644 --- a/doc/integration/external-issue-tracker.md +++ b/doc/integration/external-issue-tracker.md @@ -22,6 +22,7 @@ Visit the links below for details: - [Redmine](../user/project/integrations/redmine.md) - [Jira](../user/project/integrations/jira.md) - [Bugzilla](../user/project/integrations/bugzilla.md) +- [Custom Issue Tracker](../user/project/integrations/custom_issue_tracker.md) ### Service Template diff --git a/doc/intro/README.md b/doc/intro/README.md index 7485912d1a2..d9acc5bdeac 100644 --- a/doc/intro/README.md +++ b/doc/intro/README.md @@ -1,3 +1,7 @@ +--- +comments: false +--- + # Get started with GitLab ## Organize diff --git a/doc/legal/README.md b/doc/legal/README.md index 56d72ae3859..6413f1d645f 100644 --- a/doc/legal/README.md +++ b/doc/legal/README.md @@ -1,3 +1,7 @@ +--- +comments: false +--- + # Legal - [Corporate contributor license agreement](corporate_contributor_license_agreement.md) diff --git a/doc/legal/corporate_contributor_license_agreement.md b/doc/legal/corporate_contributor_license_agreement.md index 7f08188bd65..ebb24ba0a7f 100644 --- a/doc/legal/corporate_contributor_license_agreement.md +++ b/doc/legal/corporate_contributor_license_agreement.md @@ -1,29 +1,2 @@ -# Corporate contributor license agreement - -You accept and agree to the following terms and conditions for Your present and future Contributions submitted to GitLab B.V.. Except for the license granted herein to GitLab B.V. and recipients of software distributed by GitLab B.V., You reserve all right, title, and interest in and to Your Contributions. - -1. Definitions. - - "You" (or "Your") shall mean the copyright owner or legal entity authorized by the copyright owner that is making this Agreement with GitLab B.V.. For legal entities, the entity making a Contribution and all other entities that control, are controlled by, or are under common control with that entity are considered to be a single Contributor. For the purposes of this definition, "control" means (i) the power, direct or indirect, to cause the direction or management of such entity, whether by contract or otherwise, or (ii) ownership of fifty percent (50%) or more of the outstanding shares, or (iii) beneficial ownership of such entity. - - "Contribution" shall mean the code, documentation or other original works of authorship, including any modifications or additions to an existing work, that is submitted by You to GitLab B.V. for inclusion in, or documentation of, any of the products owned or managed by GitLab B.V. (the "Work"). For the purposes of this definition, "submitted" means any form of electronic, verbal, or written communication sent to GitLab B.V. or its representatives, including but not limited to communication on electronic mailing lists, source code control systems, and issue tracking systems that are managed by, or on behalf of, GitLab B.V. for the purpose of discussing and improving the Work, but excluding communication that is conspicuously marked or otherwise designated in writing by You as "Not a Contribution." - -2. Grant of Copyright License. - -Subject to the terms and conditions of this Agreement, You hereby grant to GitLab B.V. and to recipients of software distributed by GitLab B.V. a perpetual, worldwide, non-exclusive, no-charge, royalty-free, irrevocable copyright license to reproduce, prepare derivative works of, publicly display, publicly perform, sublicense, and distribute Your Contributions and such derivative works. - -3. Grant of Patent License. - -Subject to the terms and conditions of this Agreement, You hereby grant to GitLab B.V. and to recipients of software distributed by GitLab B.V. a perpetual, worldwide, non-exclusive, no-charge, royalty-free, irrevocable (except as stated in this section) patent license to make, have made, use, offer to sell, sell, import, and otherwise transfer the Work, where such license applies only to those patent claims licensable by You that are necessarily infringed by Your Contribution(s) alone or by combination of Your Contribution(s) with the Work to which such Contribution(s) was submitted. If any entity institutes patent litigation against You or any other entity (including a cross-claim or counterclaim in a lawsuit) alleging that your Contribution, or the Work to which you have contributed, constitutes direct or contributory patent infringement, then any patent licenses granted to that entity under this Agreement for that Contribution or Work shall terminate as of the date such litigation is filed. - -4. You represent that You are legally entitled to grant the above license. You represent further that each of Your employees is authorized to submit Contributions on Your behalf, but excluding employees that are designated in writing by You as "Not authorized to submit Contributions on behalf of [name of Your corporation here]." Such designations of exclusion for unauthorized employees are to be submitted via email to legal@gitlab.com. - -5. You represent that each of Your Contributions is Your original creation (see section 7 for submissions on behalf of others). - -6. You are not expected to provide support for Your Contributions, except to the extent You desire to provide support. You may provide support for free, for a fee, or not at all. Unless required by applicable law or agreed to in writing, You provide Your Contributions on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied, including, without limitation, any warranties or conditions of TITLE, NON-INFRINGEMENT, MERCHANTABILITY, or FITNESS FOR A PARTICULAR PURPOSE. - -7. Should You wish to submit work that is not Your original creation, You may submit it to GitLab B.V. separately from any Contribution, identifying the complete details of its source and of any license or other restriction (including, but not limited to, related patents, trademarks, and license agreements) of which you are personally aware, and conspicuously marking the work as "Submitted on behalf of a third-party: [named here]". - -8. It is Your responsibility to notify GitLab.com when any change is required to the list of designated employees excluded from submitting Contributions on Your behalf per Section 4. Such notification should be sent via email to legal@gitlab.com. - -This text is licensed under the [Creative Commons Attribution 3.0 License](https://creativecommons.org/licenses/by/3.0/) and the original source is the Google Open Source Programs Office. +This document has been replaced by a Developer Certificate of Origin and License, +as described in [Contributing.md](https://gitlab.com/gitlab-org/gitlab-ce/blob/master/CONTRIBUTING.md).
\ No newline at end of file diff --git a/doc/legal/individual_contributor_license_agreement.md b/doc/legal/individual_contributor_license_agreement.md index 59803aea080..ebb24ba0a7f 100644 --- a/doc/legal/individual_contributor_license_agreement.md +++ b/doc/legal/individual_contributor_license_agreement.md @@ -1,25 +1,2 @@ -# Individual contributor license agreement - -You accept and agree to the following terms and conditions for Your present and future Contributions submitted to GitLab B.V.. Except for the license granted herein to GitLab B.V. and recipients of software distributed by GitLab B.V., You reserve all right, title, and interest in and to Your Contributions. - -1. Definitions. - - "You" (or "Your") shall mean the copyright owner or legal entity authorized by the copyright owner that is making this Agreement with GitLab B.V.. For legal entities, the entity making a Contribution and all other entities that control, are controlled by, or are under common control with that entity are considered to be a single Contributor. For the purposes of this definition, "control" means (i) the power, direct or indirect, to cause the direction or management of such entity, whether by contract or otherwise, or (ii) ownership of fifty percent (50%) or more of the outstanding shares, or (iii) beneficial ownership of such entity. - - "Contribution" shall mean any original work of authorship, including any modifications or additions to an existing work, that is intentionally submitted by You to GitLab B.V. for inclusion in, or documentation of, any of the products owned or managed by GitLab B.V. (the "Work"). For the purposes of this definition, "submitted" means any form of electronic, verbal, or written communication sent to GitLab B.V. or its representatives, including but not limited to communication on electronic mailing lists, source code control systems, and issue tracking systems that are managed by, or on behalf of, GitLab B.V. for the purpose of discussing and improving the Work, but excluding communication that is conspicuously marked or otherwise designated in writing by You as "Not a Contribution." - -2. Grant of Copyright License. Subject to the terms and conditions of this Agreement, You hereby grant to GitLab B.V. and to recipients of software distributed by GitLab B.V. a perpetual, worldwide, non-exclusive, no-charge, royalty-free, irrevocable copyright license to reproduce, prepare derivative works of, publicly display, publicly perform, sublicense, and distribute Your Contributions and such derivative works. - -3. Grant of Patent License. Subject to the terms and conditions of this Agreement, You hereby grant to GitLab B.V. and to recipients of software distributed by GitLab B.V. a perpetual, worldwide, non-exclusive, no-charge, royalty-free, irrevocable (except as stated in this section) patent license to make, have made, use, offer to sell, sell, import, and otherwise transfer the Work, where such license applies only to those patent claims licensable by You that are necessarily infringed by Your Contribution(s) alone or by combination of Your Contribution(s) with the Work to which such Contribution(s) was submitted. If any entity institutes patent litigation against You or any other entity (including a cross-claim or counterclaim in a lawsuit) alleging that your Contribution, or the Work to which you have contributed, constitutes direct or contributory patent infringement, then any patent licenses granted to that entity under this Agreement for that Contribution or Work shall terminate as of the date such litigation is filed. - -4. You represent that you are legally entitled to grant the above license. If your employer(s) has rights to intellectual property that you create that includes your Contributions, you represent that you have received permission to make Contributions on behalf of that employer, that your employer has waived such rights for your Contributions to GitLab B.V., or that your employer has executed a separate Corporate CLA with GitLab B.V.. - -5. You represent that each of Your Contributions is Your original creation (see section 7 for submissions on behalf of others). You represent that Your Contribution submissions include complete details of any third-party license or other restriction (including, but not limited to, related patents and trademarks) of which you are personally aware and which are associated with any part of Your Contributions. - -6. You are not expected to provide support for Your Contributions, except to the extent You desire to provide support. You may provide support for free, for a fee, or not at all. Unless required by applicable law or agreed to in writing, You provide Your Contributions on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied, including, without limitation, any warranties or conditions of TITLE, NON- INFRINGEMENT, MERCHANTABILITY, or FITNESS FOR A PARTICULAR PURPOSE. - -7. Should You wish to submit work that is not Your original creation, You may submit it to GitLab B.V. separately from any Contribution, identifying the complete details of its source and of any license or other restriction (including, but not limited to, related patents, trademarks, and license agreements) of which you are personally aware, and conspicuously marking the work as "Submitted on behalf of a third-party: [insert_name_here]". - -8. You agree to notify GitLab B.V. of any facts or circumstances of which you become aware that would make these representations inaccurate in any respect. - -This text is licensed under the [Creative Commons Attribution 3.0 License](https://creativecommons.org/licenses/by/3.0/) and the original source is the Google Open Source Programs Office. +This document has been replaced by a Developer Certificate of Origin and License, +as described in [Contributing.md](https://gitlab.com/gitlab-org/gitlab-ce/blob/master/CONTRIBUTING.md).
\ No newline at end of file diff --git a/doc/migrate_ci_to_ce/README.md b/doc/migrate_ci_to_ce/README.md index 2e7782736ff..9347a834510 100644 --- a/doc/migrate_ci_to_ce/README.md +++ b/doc/migrate_ci_to_ce/README.md @@ -372,8 +372,10 @@ CREATE TABLE ``` To fix that you need to apply this SQL statement before doing final backup: -``` -# Omnibus + +```sql +## Omnibus GitLab + gitlab-ci-rails dbconsole <<EOF -- ALTER TABLES - DROP DEFAULTS ALTER TABLE ONLY ci_application_settings ALTER COLUMN id DROP DEFAULT; @@ -427,7 +429,8 @@ ALTER TABLE ONLY ci_variables ALTER COLUMN id SET DEFAULT nextval('ci_variables_ ALTER TABLE ONLY ci_web_hooks ALTER COLUMN id SET DEFAULT nextval('ci_web_hooks_id_seq'::regclass); EOF -# Source +## Source installations + cd /home/gitlab_ci/gitlab-ci sudo -u gitlab_ci -H bundle exec rails dbconsole production <<EOF ... COPY SQL STATEMENTS FROM ABOVE ... diff --git a/doc/policy/maintenance.md b/doc/policy/maintenance.md index 7ab56c89014..8d0afa9e692 100644 --- a/doc/policy/maintenance.md +++ b/doc/policy/maintenance.md @@ -19,24 +19,30 @@ For example, for GitLab version 10.5.7: * `5` represents minor version * `7` represents patch number -## Security releases +## Patch releases -The current stable release will receive security patches and bug fixes -(eg. `8.9.0` -> `8.9.1`). +Patch releases usually only include bug fixes and are only done for the current +stable release. That said, in some cases, we may backport it to previous stable +release, depending on the severity of the bug. -Feature releases will mark the next supported stable -release where the minor version is increased numerically by increments of one -(eg. `8.9 -> 8.10`). +For instance, if we release `10.1.1` with a fix for a severe bug introduced in +`10.0.0`, we could backport the fix to a new `10.0.x` patch release. -Our current policy is to support one stable release at any given time. -For medium-level security issues, we may consider backporting to the previous two +### Security releases + +Security releases are a special kind of patch release that only include security +fixes and patches (see below). + +Our current policy is to support one stable release at any given time, but for +medium-level security issues, we may backport security fixes to the previous two monthly releases. -For very serious security issues, there is [precedent](https://about.gitlab.com/2016/05/02/cve-2016-4340-patches/) -to backport security fixes to even more monthly releases of GitLab. This decision -is made on a case-by-case basis. +For very serious security issues, there is +[precedent](https://about.gitlab.com/2016/05/02/cve-2016-4340-patches/) +to backport security fixes to even more monthly releases of GitLab. +This decision is made on a case-by-case basis. -## Version support +## Upgrade recommendations We encourage everyone to run the latest stable release to ensure that you can easily upgrade to the most secure and feature-rich GitLab experience. In order @@ -70,7 +76,6 @@ Please see the table below for some examples: | -------------- | ------------ | ------------------------ | ---------------- | | 9.4.5 | 8.13.4 | `8.13.4` -> `8.17.7` -> `9.4.5` | `8.17.7` is the last version in version `8` | | 10.1.4 | 8.13.4 | `8.13.4` -> `8.17.7` -> `9.5.8` -> `10.1.4` | `8.17.7` is the last version in version `8`, `9.5.8` is the last version in version `9` | -| More information about the release procedures can be found in our [release-tools documentation][rel]. You may also want to read our diff --git a/doc/raketasks/README.md b/doc/raketasks/README.md index 2b81ebc9c59..2f916f5dea7 100644 --- a/doc/raketasks/README.md +++ b/doc/raketasks/README.md @@ -1,3 +1,7 @@ +--- +comments: false +--- + # Rake tasks - [Backup restore](backup_restore.md) diff --git a/doc/raketasks/import.md b/doc/raketasks/import.md index 2b305cb5c99..97e9b36d1a6 100644 --- a/doc/raketasks/import.md +++ b/doc/raketasks/import.md @@ -3,49 +3,47 @@ ## Notes - The owner of the project will be the first admin -- The groups will be created as needed +- The groups will be created as needed, including subgroups - The owner of the group will be the first admin - Existing projects will be skipped +- The existing Git repos will be moved from disk (removed from the original path) ## How to use -### Create a new folder inside the git repositories path. This will be the name of the new group. +### Create a new folder to import your Git repositories from. -- For omnibus-gitlab, it is located at: `/var/opt/gitlab/git-data/repositories` by default, unless you changed -it in the `/etc/gitlab/gitlab.rb` file. -- For installations from source, it is usually located at: `/home/git/repositories` or you can see where -your repositories are located by looking at `config/gitlab.yml` under the `repositories => storages` entries -(you'll usually use the `default` storage path to start). - -New folder needs to have git user ownership and read/write/execute access for git user and its group: +The new folder needs to have git user ownership and read/write/execute access for git user and its group: ``` -sudo -u git mkdir /var/opt/gitlab/git-data/repositories/new_group +sudo -u git mkdir /var/opt/gitlab/git-data/repository-import-<date>/new_group ``` -If you are using an installation from source, replace `/var/opt/gitlab/git-data` -with `/home/git`. - ### Copy your bare repositories inside this newly created folder: +- Any .git repositories found on any of the subfolders will be imported as projects +- Groups will be created as needed, these could be nested folders. Example: + +If we copy the repos to `/var/opt/gitlab/git-data/repository-import-<date>`, and repo A needs to be under the groups G1 and G2, it will +have to be created under those folders: `/var/opt/gitlab/git-data/repository-import-<date>/G1/G2/A.git`. + + ``` -sudo cp -r /old/git/foo.git /var/opt/gitlab/git-data/repositories/new_group/ +sudo cp -r /old/git/foo.git /var/opt/gitlab/git-data/repository-import-<date>/new_group/ # Do this once when you are done copying git repositories -sudo chown -R git:git /var/opt/gitlab/git-data/repositories/new_group/ +sudo chown -R git:git /var/opt/gitlab/git-data/repository-import-<date> ``` `foo.git` needs to be owned by the git user and git users group. -If you are using an installation from source, replace `/var/opt/gitlab/git-data` -with `/home/git`. +If you are using an installation from source, replace `/var/opt/gitlab/` with `/home/git`. ### Run the command below depending on your type of installation: #### Omnibus Installation ``` -$ sudo gitlab-rake gitlab:import:repos +$ sudo gitlab-rake gitlab:import:repos['/var/opt/gitlab/git-data/repository-import-<date>'] ``` #### Installation from source @@ -54,16 +52,21 @@ Before running this command you need to change the directory to where your GitLa ``` $ cd /home/git/gitlab -$ sudo -u git -H bundle exec rake gitlab:import:repos RAILS_ENV=production +$ sudo -u git -H bundle exec rake gitlab:import:repos['/var/opt/gitlab/git-data/repository-import-<date>'] RAILS_ENV=production ``` #### Example output ``` -Processing abcd.git +Processing /var/opt/gitlab/git-data/repository-import-1/a/b/c/blah.git + * Using namespace: a/b/c + * Created blah (a/b/c/blah) + * Skipping repo /var/opt/gitlab/git-data/repository-import-1/a/b/c/blah.wiki.git +Processing /var/opt/gitlab/git-data/repository-import-1/abcd.git * Created abcd (abcd.git) -Processing group/xyz.git - * Created Group group (2) +Processing /var/opt/gitlab/git-data/repository-import-1/group/xyz.git + * Using namespace: group (2) * Created xyz (group/xyz.git) + * Skipping repo /var/opt/gitlab/git-data/repository-import-1/@shared/a/b/abcd.git [...] ``` diff --git a/doc/raketasks/user_management.md b/doc/raketasks/user_management.md index 3ae46019daf..5554a0c8b78 100644 --- a/doc/raketasks/user_management.md +++ b/doc/raketasks/user_management.md @@ -149,18 +149,3 @@ cp config/secrets.yml.bak config/secrets.yml sudo /etc/init.d/gitlab start ``` - -## Clear authentication tokens for all users. Important! Data loss! - -Clear authentication tokens for all users in the GitLab database. This -task is useful if your users' authentication tokens might have been exposed in -any way. All the existing tokens will become invalid, and new tokens are -automatically generated upon sign-in or user modification. - -``` -# omnibus-gitlab -sudo gitlab-rake gitlab:users:clear_all_authentication_tokens - -# installation from source -bundle exec rake gitlab:users:clear_all_authentication_tokens RAILS_ENV=production -``` diff --git a/doc/security/README.md b/doc/security/README.md index 0fea6be8b55..d397ff104ab 100644 --- a/doc/security/README.md +++ b/doc/security/README.md @@ -1,3 +1,7 @@ +--- +comments: false +--- + # Security - [Password length limits](password_length_limits.md) diff --git a/doc/ssh/README.md b/doc/ssh/README.md index 793de9d777c..33a2d7a88a7 100644 --- a/doc/ssh/README.md +++ b/doc/ssh/README.md @@ -1,4 +1,4 @@ -# SSH +# GitLab and SSH keys Git is a distributed version control system, which means you can work locally but you can also share or "push" your changes to other servers. @@ -114,7 +114,7 @@ custom name continue onto the next step. If you manually copied your public SSH key make sure you copied the entire key starting with `ssh-rsa` and ending with your email. - + 1. Optionally you can test your setup by running `ssh -T git@example.com` (replacing `example.com` with your GitLab domain) and verifying that you receive a `Welcome to GitLab` message. @@ -172,7 +172,7 @@ dummy user account. If you are a project master or owner, you can add a deploy key in the project settings under the section 'Repository'. Specify a title for the new deploy key and paste a public SSH key. After this, the machine that uses -the corresponding private SSH key has read-only or read-write (if enabled) +the corresponding private SSH key has read-only or read-write (if enabled) access to the project. You can't add the same deploy key twice using the form. @@ -232,7 +232,7 @@ something is wrong with your SSH setup. - Ensure that you generated your SSH key pair correctly and added the public SSH key to your GitLab profile -- Try manually registering your private SSH key using `ssh-agent` as documented +- Try manually registering your private SSH key using `ssh-agent` as documented earlier in this document - Try to debug the connection by running `ssh -Tv git@example.com` (replacing `example.com` with your GitLab domain) diff --git a/doc/system_hooks/system_hooks.md b/doc/system_hooks/system_hooks.md index a45a4eb9e49..f2a9b1d769b 100644 --- a/doc/system_hooks/system_hooks.md +++ b/doc/system_hooks/system_hooks.md @@ -1,6 +1,24 @@ # System hooks -Your GitLab instance can perform HTTP POST requests on the following events: `project_create`, `project_destroy`, `project_rename`, `project_transfer`, `project_update`, `user_add_to_team`, `user_remove_from_team`, `user_create`, `user_destroy`, `key_create`, `key_destroy`, `group_create`, `group_destroy`, `user_add_to_group` and `user_remove_from_group`. +Your GitLab instance can perform HTTP POST requests on the following events: + +- `project_create` +- `project_destroy` +- `project_rename` +- `project_transfer` +- `project_update` +- `user_add_to_team` +- `user_remove_from_team` +- `user_create` +- `user_destroy` +- `user_rename` +- `key_create` +- `key_destroy` +- `group_create` +- `group_destroy` +- `group_rename` +- `user_add_to_group` +- `user_remove_from_group` The triggers for most of these are self-explanatory, but `project_update` and `project_rename` deserve some clarification: `project_update` is fired any time an attribute of a project is changed (name, description, tags, etc.) *unless* the `path` attribute is also changed. In that case, a `project_rename` is triggered instead (so that, for instance, if all you care about is the repo URL, you can just listen for `project_rename`). @@ -72,6 +90,9 @@ X-Gitlab-Event: System Hook } ``` +Note that `project_rename` is not triggered if the namespace changes. +Please refer to `group_rename` and `user_rename` for that case. + **Project transferred:** ```json @@ -175,6 +196,21 @@ X-Gitlab-Event: System Hook } ``` +**User renamed:** + +```json +{ + "event_name": "user_rename", + "created_at": "2017-11-01T11:21:04Z", + "updated_at": "2017-11-01T14:04:47Z", + "name": "new-name", + "email": "best-email@example.tld", + "user_id": 58, + "username": "new-exciting-name", + "old_username": "old-boring-name" +} +``` + **Key added** ```json @@ -209,13 +245,15 @@ X-Gitlab-Event: System Hook "updated_at": "2012-07-21T07:38:22Z", "event_name": "group_create", "name": "StoreCloud", - "owner_email": "johnsmith@gmail.com", - "owner_name": "John Smith", + "owner_email": null, + "owner_name": null, "path": "storecloud", "group_id": 78 } ``` +`owner_name` and `owner_email` are always `null`. Please see https://gitlab.com/gitlab-org/gitlab-ce/issues/39675. + **Group removed:** ```json @@ -224,13 +262,35 @@ X-Gitlab-Event: System Hook "updated_at": "2012-07-21T07:38:22Z", "event_name": "group_destroy", "name": "StoreCloud", - "owner_email": "johnsmith@gmail.com", - "owner_name": "John Smith", + "owner_email": null, + "owner_name": null, "path": "storecloud", "group_id": 78 } ``` +`owner_name` and `owner_email` are always `null`. Please see https://gitlab.com/gitlab-org/gitlab-ce/issues/39675. + +**Group renamed:** + +```json +{ + "event_name": "group_rename", + "created_at": "2017-10-30T15:09:00Z", + "updated_at": "2017-11-01T10:23:52Z", + "name": "Better Name", + "path": "better-name", + "full_path": "parent-group/better-name", + "group_id": 64, + "owner_name": null, + "owner_email": null, + "old_path": "old-name", + "old_full_path": "parent-group/old-name" +} +``` + +`owner_name` and `owner_email` are always `null`. Please see https://gitlab.com/gitlab-org/gitlab-ce/issues/39675. + **New Group Member:** ```json diff --git a/doc/topics/autodevops/index.md b/doc/topics/autodevops/index.md index 5561784ed0b..28308fc905c 100644 --- a/doc/topics/autodevops/index.md +++ b/doc/topics/autodevops/index.md @@ -144,6 +144,12 @@ has a `.gitlab-ci.yml` or not: All you need to do is remove your existing `.gitlab-ci.yml`, and you can even do that in a branch to test Auto DevOps before committing to `master`. +NOTE: **Note:** +If you are a GitLab Administrator, you can enable Auto DevOps instance wide +in **Admin Area > Settings > Continuous Integration and Deployment**. Doing that, +all the projects that haven't explicitly set an option will have Auto DevOps +enabled by default. + ## Stages of Auto DevOps The following sections describe the stages of Auto DevOps. Read them carefully @@ -315,7 +321,7 @@ Auto DevOps uses [Helm](https://helm.sh/) to deploy your application to Kubernet You can override the Helm chart used by bundling up a chart into your project repo or by specifying a project variable: -- **Bundled chart** - If your project has a `./charts` directory with a `Chart.yaml` +- **Bundled chart** - If your project has a `./chart` directory with a `Chart.yaml` file in it, Auto DevOps will detect the chart and use it instead of the [default one](https://gitlab.com/charts/charts.gitlab.io/tree/master/charts/auto-deploy-app). This can be a great way to control exactly how your application is deployed. @@ -517,7 +523,7 @@ Feature.get(:auto_devops_banner_disabled).enable Or through the HTTP API with an admin access token: ```sh -curl --data "value=true" --header "PRIVATE-TOKEN: private_token" https://gitlab.example.com/api/v4/features/auto_devops_banner_disabled +curl --data "value=true" --header "PRIVATE-TOKEN: personal_access_token" https://gitlab.example.com/api/v4/features/auto_devops_banner_disabled ``` [ce-37115]: https://gitlab.com/gitlab-org/gitlab-ce/issues/37115 diff --git a/doc/university/README.md b/doc/university/README.md index 170582bcd0c..55865ac23e8 100644 --- a/doc/university/README.md +++ b/doc/university/README.md @@ -1,3 +1,7 @@ +--- +comments: false +--- + # GitLab University GitLab University is the best place to learn about **Version Control with Git and GitLab**. @@ -51,10 +55,10 @@ The curriculum is composed of GitLab videos, screencasts, presentations, project #### 1.5. Migrating from other Source Control -1. [Migrating from BitBucket/Stash](https://docs.gitlab.com/ee/workflow/importing/import_projects_from_bitbucket.html) -1. [Migrating from GitHub](https://docs.gitlab.com/ee/workflow/importing/import_projects_from_github.html) -1. [Migrating from SVN](https://docs.gitlab.com/ee/workflow/importing/migrating_from_svn.html) -1. [Migrating from Fogbugz](https://docs.gitlab.com/ee/workflow/importing/import_projects_from_fogbugz.html) +1. [Migrating from BitBucket/Stash](https://docs.gitlab.com/ee/user/project/import/bitbucket.html) +1. [Migrating from GitHub](https://docs.gitlab.com/ee/user/project/import/github.html) +1. [Migrating from SVN](https://docs.gitlab.com/ee/user/project/import/svn.html) +1. [Migrating from Fogbugz](https://docs.gitlab.com/ee/user/project/import/fogbugz.html) #### 1.6. GitLab Inc. @@ -76,13 +80,13 @@ The curriculum is composed of GitLab videos, screencasts, presentations, project - Being part of our Great Community and Contributing to GitLab 1. [Getting Started with the GitLab Development Kit (GDK)](https://about.gitlab.com/2016/06/08/getting-started-with-gitlab-development-kit/) 1. [Contributing Technical Articles to the GitLab Blog](https://about.gitlab.com/2016/01/26/call-for-writers/) -1. [GitLab Training Workshops](https://about.gitlab.com/training) +1. [GitLab Training Workshops](https://docs.gitlab.com/ce/university/training/end-user/) +1. [GitLab Professional Services](https://about.gitlab.com/services/) #### 1.8 GitLab Training Material 1. [Git and GitLab Terminology](glossary/README.md) 1. [Git and GitLab Workshop - Slides](https://docs.google.com/presentation/d/1JzTYD8ij9slejV2-TO-NzjCvlvj6mVn9BORePXNJoMI/edit?usp=drive_web) -1. [Git and GitLab Revision](https://gitlab.com/gitlab-org/gitlab-ce/tree/master/doc/university/training/end-user) --- diff --git a/doc/university/bookclub/booklist.md b/doc/university/bookclub/booklist.md index c4229832e9f..26c3851276b 100644 --- a/doc/university/bookclub/booklist.md +++ b/doc/university/bookclub/booklist.md @@ -1,3 +1,7 @@ +--- +comments: false +--- + # Books List of books and resources, that may be worth reading. diff --git a/doc/university/bookclub/index.md b/doc/university/bookclub/index.md index 022a61f4429..63238685b2b 100644 --- a/doc/university/bookclub/index.md +++ b/doc/university/bookclub/index.md @@ -1,3 +1,7 @@ +--- +comments: false +--- + # The GitLab Book Club The Book Club is a casual meet-up to read and discuss books we like. diff --git a/doc/university/glossary/README.md b/doc/university/glossary/README.md index 9544de41b9a..076fbf6f710 100644 --- a/doc/university/glossary/README.md +++ b/doc/university/glossary/README.md @@ -1,4 +1,8 @@ -## What is the Glossary +--- +comments: false +--- + +# What is the Glossary This contains a simplified list and definitions of some of the terms that you will encounter in your day to day activities when working with GitLab. Please add any terms that you discover that you think would be useful for others. @@ -19,6 +23,10 @@ A Microsoft-based [directory service](https://msdn.microsoft.com/en-us/library/b Building and [delivering software](http://agilemethodology.org/) in phases/parts rather than trying to build everything at once then delivering to the user/client. The latter is known as the WaterFall model. +### Amazon RDS + +External reference: <http://docs.aws.amazon.com/AmazonRDS/latest/UserGuide/CHAP_Storage.html> + ### Application Lifecycle Management (ALM) The entire product lifecycle management process for an application, from requirements management, development, and testing until deployment. GitLab has [advantages](https://docs.google.com/presentation/d/1vCU-NbZWz8NTNK8Vu3y4zGMAHb5DpC8PE5mHtw1PWfI/edit#slide=id.g72f2e4906_2_288) over both legacy and modern ALM tools. @@ -64,7 +72,7 @@ A branch is a parallel version of a repository. This allows you to work on the r Having your own logo on [your GitLab instance login page](https://docs.gitlab.com/ee/customization/branded_login_page.html) instead of the GitLab logo. -### Job triggers +### Job triggers (Build Triggers) These protect your code base against breaks, for instance when a team is working on the same project. Learn about [setting up](https://docs.gitlab.com/ce/ci/triggers/README.html) job triggers. ### CEPH @@ -105,15 +113,15 @@ Atlassian's product for collaboration on documents and projects. ### Continuous Delivery -A [software engineering approach](https://about.gitlab.com/2016/08/05/continuous-integration-delivery-and-deployment-with-gitlab/) in which continuous integration, automated testing, and automated deployment capabilities allow software to be developed and deployed rapidly, reliably and repeatedly with minimal human intervention. Still, the deployment to production is defined strategically and triggered manually. +A [software engineering approach](https://about.gitlab.com/2016/08/05/continuous-integration-delivery-and-deployment-with-gitlab/) in which continuous integration, automated testing, and automated deployment capabilities allow software to be developed and deployed rapidly, reliably and repeatedly with minimal human intervention. Still, the deployment to production is defined strategically and triggered manually. [Amazon moves toward continuous delivery](https://www.youtube.com/watch?v=esEFaY0FDKc) ### Continuous Deployment -A [software development practice](https://about.gitlab.com/2016/08/05/continuous-integration-delivery-and-deployment-with-gitlab/) in which every code change goes through the entire pipeline and is put into production automatically, resulting in many production deployments every day. It does everything that Continuous Delivery does, but the process is fully automated, there's no human intervention at all. +A [software development practice](https://about.gitlab.com/2016/08/05/continuous-integration-delivery-and-deployment-with-gitlab/) in which every code change goes through the entire pipeline and is put into production automatically, resulting in many production deployments every day. It does everything that Continuous Delivery does, but the process is fully automated, there's no human intervention at all. [The difference between Continuous Delivery and Continuous Integration.](https://www.youtube.com/watch?v=igwFj8PPSnw) ### Continuous Integration -A [software development practice](https://about.gitlab.com/2016/08/05/continuous-integration-delivery-and-deployment-with-gitlab/) in which you build and test software every time a developer pushes code to the application, and it happens several times a day. +A [software development practice](https://about.gitlab.com/2016/08/05/continuous-integration-delivery-and-deployment-with-gitlab/) in which you build and test software every time a developer pushes code to the application, and it happens several times a day. [Thoughtworks discusses continuous integration.](https://www.thoughtworks.com/continuous-integration) ### Contributor @@ -123,6 +131,10 @@ Term used for a person contributing to an open source project. A [natural evolution](https://about.gitlab.com/2016/09/14/gitlab-live-event-recap/) of software development that carries a conversation across functional groups throughout the development process, enabling developers to track the full path of development in a cohesive and intuitive way. ConvDev accelerates the development lifecycle by fostering collaboration and knowledge sharing from idea to production. +### Cycle Analytics + +See <https://gitlab.com/gitlab-org/gitlab-ce/issues/22458> + ### Cycle Time The time it takes to move from [idea to production](https://about.gitlab.com/2016/08/05/continuous-integration-delivery-and-deployment-with-gitlab/#from-idea-to-production-with-gitlab). @@ -131,6 +143,10 @@ The time it takes to move from [idea to production](https://about.gitlab.com/201 Atlassian product for High Availability. +### Dependencies + +As in "specify [dependencies](https://gitlab.com/gitlab-org/gitlab-ce/issues/14728) between stages." + ### Deploy Keys A [SSH key](https://docs.gitlab.com/ce/gitlab-basics/create-your-ssh-keys.html)stored on your server that grants access to a single GitLab repository. This is used by a GitLab runner to clone a project's code so that tests can be run against the checked out code. @@ -147,15 +163,17 @@ The intersection of software engineering, quality assurance, and technology oper The difference between two commits, or saved changes. This will also be shown visually after the changes. -#### Directory +### Directory A folder used for storing multiple files. ### Docker Container Registry -A [feature](https://docs.gitlab.com/ce/user/project/container_registry.html) of GitLab projects. Containers wrap up a piece of software in a complete filesystem that contains everything it needs to run: code, runtime, system tools, system libraries – anything you can install on a server. This guarantees that it will always run the same, regardless of the environment it is running in. +A [feature](https://docs.gitlab.com/ce/user/project/container_registry.html) of [GitLab projects](https://about.gitlab.com/2016/05/23/gitlab-container-registry/). Containers wrap up a piece of software in a complete filesystem that contains everything it needs to run: code, runtime, system tools, system libraries – anything you can install on a server. This guarantees that it will always run the same, regardless of the environment it is running in. + +### Dynamic Environment (review apps) -### Dynamic Environment +### EC2 Instance ### ElasticSearch @@ -163,10 +181,26 @@ Elasticsearch is a flexible, scalable and powerful search service. When [enabled ### Emacs +External reference: <https://www.masteringemacs.org/article/mastering-key-bindings-emacs> + +### First Byte + +External reference: <https://en.wikipedia.org/wiki/Time_To_First_Byte> + +First Byte (sometimes referred to as time to first byte or [TTFB](https://en.wikipedia.org/wiki/Time_To_First_Byte)) measures the time between making a request and receiving the first byte of information in return. As a result, First Byte encompasses everything that is the backend as well as network transit issues. It differs from [_Speed Index_](#speed-index) mostly by frontend related issues which are included in Speed Index such as javascript loading, page rendering, and so on. + ### Fork Your [own copy](https://docs.gitlab.com/ce/workflow/forking_workflow.html) of a repository that allows you to make changes to the repository without affecting the original. +### Funnel, or: TOFU, MOFU, BOFU + +External reference: [Blog post](https://www.weidert.com/whole_brain_marketing_blog/bid/113688/ToFu-MoFu-BoFu-Serving-Up-The-Right-Content-for-Lead-Nurturing) + +TOFU: top of funnel +MOFU: middle of funnel +BOFU: bottom of funnel + ### Gerrit A code review [tool](https://www.gerritcodereview.com/) built on top of Git. @@ -179,6 +213,8 @@ A [git attributes file](https://git-scm.com/docs/gitattributes) is a simple text [Scripts](https://git-scm.com/book/en/v2/Customizing-Git-Git-Hooks) you can use to trigger actions at certain points. +Difference between a [webhook](#webhooks) and a git hook: a git hook is local to its repo (usually) while a webhook is not (it can make API or http calls). So for example if you want your linter to fire before you commit, you can set that up with a git hook. If the linter fails, the commit does not go through. A git hook _can_ be configured to go beyond its repo, e.g. by having it make an API call. + ### GitHost.io A single-tenant solution that provides GitLab CE or EE as a managed service. GitLab Inc. is responsible for installing, updating, hosting, and backing up customers' own private and secure GitLab instance. @@ -207,9 +243,20 @@ Our free SaaS for public and private repositories. Allows you to replicate your GitLab instance to other geographical locations as a read-only fully operational version. It [can be used](https://docs.gitlab.com/ee/gitlab-geo/README.html) for cloning and fetching projects, in addition to reading any data. This will make working with large repositories over large distances much faster. +### GitLab High Availability + +### GitLab Master Plan + +Related blog post: <https://about.gitlab.com/2016/09/13/gitlab-master-plan/>. + ### GitLab Pages + These allow you to [create websites](https://gitlab.com/help/pages/README.md) for your GitLab projects, groups, or user account. +### GitLab Runner + +Related project: <https://gitlab.com/gitlab-org/gitlab-runner> + ### Gitolite An [access layer](https://git-scm.com/book/en/v1/Git-on-the-Server-Gitolite) that sits on top of Git. Users are granted access to repos via a simple config file. As an admin, you only need the users' public SSH key and a username. @@ -222,6 +269,10 @@ A web-based hosting service for projects using Git. It was acquired by GitLab an An open source programming [language](https://golang.org/). +### Gogs + +External reference: <https://gogs.io/> + ### GUI/ Git GUI A portable [graphical interface](https://git-scm.com/docs/git-gui) to Git that allows users to make changes to their repository by making new commits, amending existing ones, creating branches, performing local merges, and fetching/pushing to remote repositories. @@ -252,7 +303,7 @@ A [tool](https://docs.gitlab.com/ee/integration/external-issue-tracker.html) use ### Jenkins -An Open Source CI tool written using the Java programming language. [Jenkins](https://jenkins-ci.org/) does the same job as GitLab CI, Bamboo, and Travis CI. It is extremely popular. +An Open Source CI tool written using the Java programming language. [Jenkins](https://jenkins-ci.org/) does the same job as GitLab CI, Bamboo, and Travis CI. It is extremely popular. Related [documentation](https://docs.gitlab.com/ee/integration/jenkins.html). ### Jira @@ -286,6 +337,10 @@ GitLab [integrates](https://docs.gitlab.com/ce/administration/auth/ldap.html) wi Allows you to synchronize the members of a GitLab group with one or more LDAP groups. +### Lint + +Static code analysis for our various file types. For example, we use [scss-lint](https://github.com/brigade/scss-lint) to ensure that a consistent code styling is respected. Similar tools: rubocop / eslint. + ### Load Balancer A [device](https://en.wikipedia.org/wiki/Load_balancing_(computing)) that distributes network or application traffic across multiple servers. @@ -326,6 +381,10 @@ Takes changes from one branch, and [applies them](https://git-scm.com/docs/git-m [Arises](https://about.gitlab.com/2016/09/06/resolving-merge-conflicts-from-the-gitlab-ui/) when a merge can't be performed cleanly between two versions of the same file. +#### Merge Request + +[Takes changes](https://docs.gitlab.com/ce/gitlab-basics/add-merge-request.html) from one branch, and applies them into another branch. + ### Meteor A [platform](https://www.meteor.com) for building javascript apps. @@ -346,6 +405,14 @@ A type of software license. It lets people do anything with your code with prope A free disaster recovery [software](https://help.ubuntu.com/community/MondoMindi). +#### Mount + +External reference: + +As stated on the [wikipedia page](https://en.wikipedia.org/wiki/Mount_(Unix)), "Mounting makes file systems, files, directories, devices and special files available for use and available to the user." + +For example, we have NFS servers where the _git files_ reside. In order for a worker node to "see" or "use" the git files, the NFS server needs to be _mounted_ on the worker; that is, the worker needs to know that the NFS server exists and how to connect to it. Think of it as getting a shared drive to show up in your Finder (on Mac) or Explorer (on Windows). + ### MySQL A relational [database](http://www.mysql.com/) owned by Oracle. Currently only supported if you are using EE. @@ -356,7 +423,7 @@ A set of symbols that are used to organize objects of various kinds so that thes ### Nginx -A web [server](https://www.nginx.com/resources/wiki/) (pronounced "engine x"). It can act as a reverse proxy server for HTTP, HTTPS, SMTP, POP3, and IMAP protocols, as well as a load balancer and an HTTP cache. +A web [server](https://www.nginx.com/resources/wiki/) (pronounced "engine x"). [It can act]((https://gitlab.com/gitlab-org/omnibus-gitlab/blob/master/doc/settings/nginx.md) as a reverse proxy server for HTTP, HTTPS, SMTP, POP3, and IMAP protocols, as well as a load balancer and an HTTP cache. ### OAuth @@ -376,7 +443,11 @@ GitLab's [business model](https://about.gitlab.com/2016/07/20/gitlab-is-open-cor ### Open Source Software -Software for which the original source code is freely [available](https://opensource.org/docs/osd) and may be redistributed and modified. GitLab prioritizes open source [stewardship](https://about.gitlab.com/2016/01/11/being-a-good-open-source-steward/). +Software for which the original source code is freely [available](https://opensource.org/docs/osd) and may be redistributed and modified. GitLab prioritizes open source [stewardship](https://about.gitlab.com/2016/01/11/being-a-good-open-source-steward/). Including to providing access to the source code, open source software must comply with a number of criteria, among them free distribution and no discrimination against persons, groups, or fields of endeavor. + +#### Open Source Stewardship + +[Related blog post](https://about.gitlab.com/2016/01/11/being-a-good-open-source-steward/). ### Owner @@ -426,6 +497,8 @@ A popular DevOps [automation tool](https://puppet.com/product/how-puppet-works). Git [command](https://git-scm.com/docs/git-push) to send commits from the local repository to the remote repository. Read about [advanced push rules](https://gitlab.com/help/pages/README.md) in GitLab. +### Raketasks + ### RE Read Only Permissions to see a file and its contents, but not change it. @@ -434,10 +507,24 @@ Permissions to see a file and its contents, but not change it. In addition to the merge, the [rebase](https://git-scm.com/book/en/v2/Git-Branching-Rebasing) is a main way to integrate changes from one branch into another. +### Regression + +A regression is something that used to work one way in the last release and then we made a **breaking change** and it no longer works the same way. + +_or_ + +A regression is defined as a change that results in a negative impact on the functionality of an existing feature due to recent changes, i.e. the latest release. + +### Remote mirroring + ### (Git) Repository A directory where Git [has been initiatlized](https://git-scm.com/book/en/v2/Git-Basics-Getting-a-Git-Repository) to start version controlling your files. The history of your work is stored here. A remote repository is not on your machine, but usually online (like on GitLab.com, for instance). The main remote repository is usually called "Origin." +##### Remote repository + +A [repository](https://about.gitlab.com/2015/05/18/simple-words-for-a-gitlab-newbie/) that is not-on-your-machine, so it's anything that is not your computer. Usually, it is online, GitLab.com for instance. The main remote repository is usually called “Origin”. + ### Requirements management Gives your distributed teams a single shared repository to collaborate and share requirements, understand their relationship to tests, and evaluate linked defects. It includes multiple, preconfigured requirement types. @@ -456,7 +543,7 @@ A route table contains rules (called routes) that determine where network traffi ### Runners -Actual build machines/containers that [run and execute tests](https://gitlab.com/gitlab-org/gitlab-ci-multi-runner) you have specified to be run on GitLab CI. +Actual build machines/containers that [run and execute tests](https://gitlab.com/gitlab-org/gitlab-runner) you have specified to be run on GitLab CI. ### Sidekiq @@ -470,7 +557,11 @@ Software that is hosted centrally and accessed on-demand (i.e. whenever you want This term is often used by people when they mean "Version Control." -## Scrum +#### SCLAU + +Abbreviation for SQO Count [Large And Up](https://about.gitlab.com/handbook/sales/#market-segmentation). This is the number of opportunities in large and strategic organizations passed from marketing to sales. + +### Scrum An Agile [framework](https://www.scrum.org/Resources/What-is-Scrum) designed to typically help complete complex software projects. It's made up of several parts: product requirements backlog, sprint planning, sprint (development), sprint review, and retrospec (analyzing the sprint). The goal is to end up with potentially shippable products. @@ -480,7 +571,9 @@ The board used to track the status and progress of each of the sprint backlog it ### Shell -Terminal on Mac OSX, GitBash on Windows, or Linux Terminal on Linux. You [use git]() and make changes to GitLab projects in your shell. You [use git](https://docs.gitlab.com/ce/gitlab-basics/start-using-git.html) and make changes to GitLab projects in your shell. +Terminal on Mac OSX, GitBash on Windows, or Linux Terminal on Linux. You [use git](https://docs.gitlab.com/ce/gitlab-basics/start-using-git.html) and make changes to GitLab projects in your shell. You [use git](https://docs.gitlab.com/ce/gitlab-basics/start-using-git.html) and make changes to GitLab projects in your shell. + +### Shell command runner ### Single-tenant @@ -490,6 +583,8 @@ The tenant purchases their own copy of the software and the software can be cust Real time messaging app for teams that is used internally by GitLab team members. GitLab users can enable [Slack integration](https://docs.gitlab.com/ce/project_services/slack.html) to trigger push, issue, and merge request events among others. +### Slash commands + ### Slave Servers Also known as secondary servers, these help to spread the load over multiple machines. They also provide backups when the master/primary server crashes. @@ -498,6 +593,10 @@ Also known as secondary servers, these help to spread the load over multiple mac Program code as typed by a computer programmer (i.e. it has not yet been compiled/translated by the computer to machine language). +### Speed Index + +[Speed Index](https://sites.google.com/a/webpagetest.org/docs/using-webpagetest/metrics/speed-index) is "the average time at which visible parts of the page are displayed". + ### SSH Key A unique identifier of a computer. It is used to identify computers without the need for a password (e.g., On GitLab I have [added the ssh key](https://docs.gitlab.com/ce/gitlab-basics/create-your-ssh-keys.html) of all my work machines so that the GitLab instance knows that it can accept code pushes and pulls from this trusted machines whose keys are I have added.) @@ -538,6 +637,16 @@ An open source version control system. Read about [migrating from SVN](https://d [Represents](https://docs.gitlab.com/ce/api/tags.html) a version of a particular branch at a moment in time. +### Tenancy + +#### Multi-tenant + +A [multi-tenant](http://whatis.techtarget.com/definition/multi-tenancy) GitLab instance can have any number of customers - such as companies or groups of users using it. GitLab.com is an example of a multi-tenant GitLab instance. + +#### Single-tenant + +A [single-tenant](http://searchcloudapplications.techtarget.com/definition/single-tenancy) GitLab instance has only one customer - such as a company - using it. On premise GitLab instances are almost exclusively single-tenant. + ### Tool Stack The set of tools used in a process to achieve a common outcome (e.g. set of tools used in Application Lifecycle Management). @@ -546,9 +655,17 @@ The set of tools used in a process to achieve a common outcome (e.g. set of tool An open source project management and bug tracking web [application](https://trac.edgewall.org/). +### True-Up licensing model + +### Ubuntu + ### Untracked files -New files that Git has not [been told](https://git-scm.com/book/en/v2/Git-Basics-Recording-Changes-to-the-Repository) to track previously. +New files that Git has not [been told](https://git-scm.com/book/en/v2/Git-Basics-Recording-Changes-to-the-Repository) to track previously. Add them by using the command "git add [file path]" + +### Upstream repository vs. GitLab repository + +[External conversation](https://news.ycombinator.com/item?id=12487112) ### User @@ -556,11 +673,11 @@ Anyone interacting with the software. ### Version Control Software (VCS) -Version control is a system that records changes to a file or set of files over time so that you can recall specific versions later. VCS [has evolved](https://docs.google.com/presentation/d/16sX7hUrCZyOFbpvnrAFrg6tVO5_yT98IgdAqOmXwBho/edit#slide=id.gd69537a19_0_32) from local version control systems, to centralized version control systems, to the present distributed version control systems like Git, Mercurial, Bazaar, and Darcs. +Version control is a system that records changes to a file or set of files over time so that you can recall specific versions later. VCS [has evolved](https://docs.google.com/presentation/d/16sX7hUrCZyOFbpvnrAFrg6tVO5_yT98IgdAqOmXwBho/edit#slide=id.gd69537a19_0_32) from local version control systems, to centralized version control systems, to the present [distributed version control systems](https://en.wikipedia.org/wiki/Distributed_version_control) like Git, Mercurial, Bazaar, and Darcs. If any server dies, and these systems were collaborating via it, any of the client repositories can be copied back up to the server to restore it. ### Virtual Private Cloud (VPC) -An on demand configurable pool of shared computing resources allocated within a public cloud environment, providing some isolation between the different users using the resources. GitLab users need to create a new Amazon VPC in order to [setup High Availability](https://docs.gitlab.com/ce/university/high-availability/aws/). +A [VPC](https://docs.gitlab.com/ce/university/glossary/README.html#virtual-private-cloud-vpc) is an on demand configurable pool of shared computing resources allocated within a public cloud environment, providing some isolation between the different users using the resources. GitLab users need to create a new Amazon VPC in order to [setup High Availability](https://docs.gitlab.com/ce/university/high-availability/aws/). ### Virtual private server (VPS) @@ -568,7 +685,7 @@ A [virtual machine](https://en.wikipedia.org/wiki/Virtual_private_server) sold a ### VM Instance -In object-oriented programming, an [instance](http://stackoverflow.com/questions/20461907/what-is-meaning-of-instance-in-programming) is a specific realization of any object. An object may be varied in a number of ways. Each realized variation of that object is an instance. Therefore, a VM instance is an instance of a virtual machine, which is an emulation of a computer system. +In object-oriented programming, an [instance](http://stackoverflow.com/questions/20461907/what-is-meaning-of-instance-in-programming) is a specific realization of any [object](https://cloud.google.com/compute/docs/instances/). An object may be varied in a number of ways. Each realized variation of that object is an instance. Therefore, a VM instance is an instance of a virtual machine, which is an emulation of a computer system. ### Waterfall @@ -582,6 +699,10 @@ A way for for an app to [provide](https://docs.gitlab.com/ce/user/project/integr A [website/system](http://www.wiki.com/) that allows for collaborative editing of its content by the users. In programming, wikis usually contain documentation of how to use the software. +### Working area + +Files that have been modified but are not committed. Check them by using the command "git status". + ### Working Tree [Consists of files](http://stackoverflow.com/questions/3689838/difference-between-head-working-tree-index-in-git) that you are currently working on. diff --git a/doc/university/high-availability/aws/README.md b/doc/university/high-availability/aws/README.md index 6b8f3cd3d1d..54625996dff 100644 --- a/doc/university/high-availability/aws/README.md +++ b/doc/university/high-availability/aws/README.md @@ -1,3 +1,7 @@ +--- +comments: false +--- + # High Availability on AWS diff --git a/doc/university/process/README.md b/doc/university/process/README.md index 04f2d52514f..fdf6224f7f6 100644 --- a/doc/university/process/README.md +++ b/doc/university/process/README.md @@ -1,8 +1,12 @@ --- +comments: false +--- + +--- title: University | Process --- -## Suggesting improvements +# Suggesting improvements If you would like to teach a class or participate or help in any way please submit a merge request and assign it to [Job](https://gitlab.com/u/JobV). diff --git a/doc/university/support/README.md b/doc/university/support/README.md index 567dadb3b47..25d5fe351ca 100644 --- a/doc/university/support/README.md +++ b/doc/university/support/README.md @@ -1,5 +1,9 @@ +--- +comments: false +--- -## Support Boot Camp + +# Support Boot Camp **Goal:** Prepare new Service Engineers at GitLab diff --git a/doc/university/training/end-user/README.md b/doc/university/training/end-user/README.md index 03c62a81b10..a882bf0eb48 100644 --- a/doc/university/training/end-user/README.md +++ b/doc/university/training/end-user/README.md @@ -1,3 +1,7 @@ +--- +comments: false +--- + # Training diff --git a/doc/university/training/gitlab_flow.md b/doc/university/training/gitlab_flow.md index a7db1f2e069..02a6ad48a38 100644 --- a/doc/university/training/gitlab_flow.md +++ b/doc/university/training/gitlab_flow.md @@ -1,3 +1,7 @@ +--- +comments: false +--- + # GitLab Flow - A simplified branching strategy diff --git a/doc/university/training/index.md b/doc/university/training/index.md index 03179ff5a77..14f096b130f 100644 --- a/doc/university/training/index.md +++ b/doc/university/training/index.md @@ -1,3 +1,7 @@ +--- +comments: false +--- + # GitLab Training Material All GitLab training material is stored in markdown format. Slides are diff --git a/doc/university/training/topics/additional_resources.md b/doc/university/training/topics/additional_resources.md index 3ed601625cf..d01634df744 100644 --- a/doc/university/training/topics/additional_resources.md +++ b/doc/university/training/topics/additional_resources.md @@ -1,4 +1,8 @@ -## Additional Resources +--- +comments: false +--- + +# Additional Resources 1. GitLab Documentation [http://docs.gitlab.com](http://docs.gitlab.com/) 2. GUI Clients [http://git-scm.com/downloads/guis](http://git-scm.com/downloads/guis) diff --git a/doc/university/training/topics/agile_git.md b/doc/university/training/topics/agile_git.md index e6e4fea9b51..251af99bed7 100644 --- a/doc/university/training/topics/agile_git.md +++ b/doc/university/training/topics/agile_git.md @@ -1,3 +1,7 @@ +--- +comments: false +--- + # Agile and Git ---------- diff --git a/doc/university/training/topics/bisect.md b/doc/university/training/topics/bisect.md index a60c4365e0c..2d5ab107fe6 100644 --- a/doc/university/training/topics/bisect.md +++ b/doc/university/training/topics/bisect.md @@ -1,3 +1,7 @@ +--- +comments: false +--- + # Bisect ---------- diff --git a/doc/university/training/topics/cherry_picking.md b/doc/university/training/topics/cherry_picking.md index af7a70a2818..df23024b6ee 100644 --- a/doc/university/training/topics/cherry_picking.md +++ b/doc/university/training/topics/cherry_picking.md @@ -1,3 +1,7 @@ +--- +comments: false +--- + # Cherry Pick ---------- diff --git a/doc/university/training/topics/env_setup.md b/doc/university/training/topics/env_setup.md index 8149379b36f..b7bec83ed8a 100644 --- a/doc/university/training/topics/env_setup.md +++ b/doc/university/training/topics/env_setup.md @@ -1,3 +1,7 @@ +--- +comments: false +--- + # Configure your environment ---------- diff --git a/doc/university/training/topics/explore_gitlab.md b/doc/university/training/topics/explore_gitlab.md index b65457728c0..84a1879cd92 100644 --- a/doc/university/training/topics/explore_gitlab.md +++ b/doc/university/training/topics/explore_gitlab.md @@ -1,3 +1,7 @@ +--- +comments: false +--- + # Explore GitLab projects ---------- diff --git a/doc/university/training/topics/feature_branching.md b/doc/university/training/topics/feature_branching.md index 4b34406ea75..0df5f26dbea 100644 --- a/doc/university/training/topics/feature_branching.md +++ b/doc/university/training/topics/feature_branching.md @@ -1,3 +1,7 @@ +--- +comments: false +--- + # Feature branching ---------- diff --git a/doc/university/training/topics/getting_started.md b/doc/university/training/topics/getting_started.md index ec7bb2631aa..153b45fb4da 100644 --- a/doc/university/training/topics/getting_started.md +++ b/doc/university/training/topics/getting_started.md @@ -1,3 +1,7 @@ +--- +comments: false +--- + # Getting Started ---------- diff --git a/doc/university/training/topics/git_add.md b/doc/university/training/topics/git_add.md index 9ffb4b9c859..651366e0d49 100644 --- a/doc/university/training/topics/git_add.md +++ b/doc/university/training/topics/git_add.md @@ -1,3 +1,7 @@ +--- +comments: false +--- + # Git Add ---------- diff --git a/doc/university/training/topics/git_intro.md b/doc/university/training/topics/git_intro.md index ca1ff29d93b..7e502d6dad4 100644 --- a/doc/university/training/topics/git_intro.md +++ b/doc/university/training/topics/git_intro.md @@ -1,3 +1,7 @@ +--- +comments: false +--- + # Git introduction ---------- diff --git a/doc/university/training/topics/git_log.md b/doc/university/training/topics/git_log.md index 32ebceff491..f2709ae3890 100644 --- a/doc/university/training/topics/git_log.md +++ b/doc/university/training/topics/git_log.md @@ -1,3 +1,7 @@ +--- +comments: false +--- + # Git Log ---------- @@ -49,8 +53,8 @@ git log --since=1.month.ago --until=3.weeks.ago ``` cd ~/workspace -git clone git@gitlab.com:gitlab-org/gitlab-ci-multi-runner.git -cd gitlab-ci-multi-runner +git clone git@gitlab.com:gitlab-org/gitlab-runner.git +cd gitlab-runner git log --author="Travis" git log --since=1.month.ago --until=3.weeks.ago git log --since=1.month.ago --until=1.day.ago --author="Travis" diff --git a/doc/university/training/topics/gitlab_flow.md b/doc/university/training/topics/gitlab_flow.md index 8e5d3baf959..b8049b5c80e 100644 --- a/doc/university/training/topics/gitlab_flow.md +++ b/doc/university/training/topics/gitlab_flow.md @@ -1,3 +1,7 @@ +--- +comments: false +--- + # GitLab Flow ---------- diff --git a/doc/university/training/topics/merge_conflicts.md b/doc/university/training/topics/merge_conflicts.md index 77807b3e7ef..9a1ce550868 100644 --- a/doc/university/training/topics/merge_conflicts.md +++ b/doc/university/training/topics/merge_conflicts.md @@ -1,3 +1,7 @@ +--- +comments: false +--- + # Merge conflicts ---------- diff --git a/doc/university/training/topics/merge_requests.md b/doc/university/training/topics/merge_requests.md index 5b446f02f63..4e8c9de85a1 100644 --- a/doc/university/training/topics/merge_requests.md +++ b/doc/university/training/topics/merge_requests.md @@ -1,3 +1,7 @@ +--- +comments: false +--- + # Merge requests ---------- diff --git a/doc/university/training/topics/rollback_commits.md b/doc/university/training/topics/rollback_commits.md index cf647284604..0db1d93d1dc 100644 --- a/doc/university/training/topics/rollback_commits.md +++ b/doc/university/training/topics/rollback_commits.md @@ -1,3 +1,7 @@ +--- +comments: false +--- + # Rollback Commits ---------- diff --git a/doc/university/training/topics/stash.md b/doc/university/training/topics/stash.md index c1bdda32645..5b27ac12f77 100644 --- a/doc/university/training/topics/stash.md +++ b/doc/university/training/topics/stash.md @@ -1,3 +1,7 @@ +--- +comments: false +--- + # Git Stash ---------- diff --git a/doc/university/training/topics/subtree.md b/doc/university/training/topics/subtree.md index 5d869af64c1..b5a892dc17b 100644 --- a/doc/university/training/topics/subtree.md +++ b/doc/university/training/topics/subtree.md @@ -1,8 +1,8 @@ -## Subtree +--- +comments: false +--- ----------- - -## Subtree +# Subtree * Used when there are nested repositories. * Not recommended when the amount of dependencies is too large diff --git a/doc/university/training/topics/tags.md b/doc/university/training/topics/tags.md index e9607b5a875..ab48d52d3c3 100644 --- a/doc/university/training/topics/tags.md +++ b/doc/university/training/topics/tags.md @@ -1,3 +1,7 @@ +--- +comments: false +--- + # Tags ---------- diff --git a/doc/university/training/topics/unstage.md b/doc/university/training/topics/unstage.md index 17dbb64b9e6..fc72949ade9 100644 --- a/doc/university/training/topics/unstage.md +++ b/doc/university/training/topics/unstage.md @@ -1,3 +1,7 @@ +--- +comments: false +--- + # Unstage ---------- diff --git a/doc/university/training/user_training.md b/doc/university/training/user_training.md index 9e38df26b6a..90e1d2ba5e8 100644 --- a/doc/university/training/user_training.md +++ b/doc/university/training/user_training.md @@ -1,3 +1,7 @@ +--- +comments: false +--- + # GitLab Git Workshop --- diff --git a/doc/update/10.0-to-10.1.md b/doc/update/10.0-to-10.1.md index dc14c779026..af815d26a74 100644 --- a/doc/update/10.0-to-10.1.md +++ b/doc/update/10.0-to-10.1.md @@ -1,3 +1,7 @@ +--- +comments: false +--- + # From 10.0 to 10.1 Make sure you view this update guide from the tag (version) of GitLab you would diff --git a/doc/update/10.1-to-10.2.md b/doc/update/10.1-to-10.2.md new file mode 100644 index 00000000000..9e0d8f79522 --- /dev/null +++ b/doc/update/10.1-to-10.2.md @@ -0,0 +1,360 @@ +--- +comments: false +--- + +# From 10.1 to 10.2 + +Make sure you view this update guide from the tag (version) of GitLab you would +like to install. In most cases this should be the highest numbered production +tag (without rc in it). You can select the tag in the version dropdown at the +top left corner of GitLab (below the menu bar). + +If the highest number stable branch is unclear please check the +[GitLab Blog](https://about.gitlab.com/blog/archives.html) for installation +guide links by version. + +### 1. Stop server + +```bash +sudo service gitlab stop +``` + +### 2. Backup + +```bash +cd /home/git/gitlab + +sudo -u git -H bundle exec rake gitlab:backup:create RAILS_ENV=production +``` + +### 3. Update Ruby + +NOTE: GitLab 9.0 and higher only support Ruby 2.3.x and dropped support for Ruby 2.1.x. Be +sure to upgrade your interpreter if necessary. + +You can check which version you are running with `ruby -v`. + +Download and compile Ruby: + +```bash +mkdir /tmp/ruby && cd /tmp/ruby +curl --remote-name --progress https://cache.ruby-lang.org/pub/ruby/2.3/ruby-2.3.5.tar.gz +echo '3247e217d6745c27ef23bdc77b6abdb4b57a118f ruby-2.3.5.tar.gz' | shasum -c - && tar xzf ruby-2.3.5.tar.gz +cd ruby-2.3.5 +./configure --disable-install-rdoc +make +sudo make install +``` + +Install Bundler: + +```bash +sudo gem install bundler --no-ri --no-rdoc +``` + +### 4. Update Node + +GitLab now runs [webpack](http://webpack.js.org) to compile frontend assets and +it has a minimum requirement of node v4.3.0. + +You can check which version you are running with `node -v`. If you are running +a version older than `v4.3.0` you will need to update to a newer version. You +can find instructions to install from community maintained packages or compile +from source at the nodejs.org website. + +<https://nodejs.org/en/download/> + + +Since 8.17, GitLab requires the use of yarn `>= v0.17.0` to manage +JavaScript dependencies. + +```bash +curl --silent --show-error https://dl.yarnpkg.com/debian/pubkey.gpg | sudo apt-key add - +echo "deb https://dl.yarnpkg.com/debian/ stable main" | sudo tee /etc/apt/sources.list.d/yarn.list +sudo apt-get update +sudo apt-get install yarn +``` + +More information can be found on the [yarn website](https://yarnpkg.com/en/docs/install). + +### 5. Update Go + +NOTE: GitLab 9.2 and higher only supports Go 1.8.3 and dropped support for Go +1.5.x through 1.7.x. Be sure to upgrade your installation if necessary. + +You can check which version you are running with `go version`. + +Download and install Go: + +```bash +# Remove former Go installation folder +sudo rm -rf /usr/local/go + +curl --remote-name --progress https://storage.googleapis.com/golang/go1.8.3.linux-amd64.tar.gz +echo '1862f4c3d3907e59b04a757cfda0ea7aa9ef39274af99a784f5be843c80c6772 go1.8.3.linux-amd64.tar.gz' | shasum -a256 -c - && \ + sudo tar -C /usr/local -xzf go1.8.3.linux-amd64.tar.gz +sudo ln -sf /usr/local/go/bin/{go,godoc,gofmt} /usr/local/bin/ +rm go1.8.3.linux-amd64.tar.gz +``` + +### 6. Get latest code + +```bash +cd /home/git/gitlab + +sudo -u git -H git fetch --all +sudo -u git -H git checkout -- db/schema.rb # local changes will be restored automatically +sudo -u git -H git checkout -- locale +``` + +For GitLab Community Edition: + +```bash +cd /home/git/gitlab + +sudo -u git -H git checkout 10-2-stable +``` + +OR + +For GitLab Enterprise Edition: + +```bash +cd /home/git/gitlab + +sudo -u git -H git checkout 10-2-stable-ee +``` + +### 7. Update gitlab-shell + +```bash +cd /home/git/gitlab-shell + +sudo -u git -H git fetch --all --tags +sudo -u git -H git checkout v$(</home/git/gitlab/GITLAB_SHELL_VERSION) +sudo -u git -H bin/compile +``` + +### 8. Update gitlab-workhorse + +Install and compile gitlab-workhorse. GitLab-Workhorse uses +[GNU Make](https://www.gnu.org/software/make/). +If you are not using Linux you may have to run `gmake` instead of +`make` below. + +```bash +cd /home/git/gitlab-workhorse + +sudo -u git -H git fetch --all --tags +sudo -u git -H git checkout v$(</home/git/gitlab/GITLAB_WORKHORSE_VERSION) +sudo -u git -H make +``` + +### 9. Update Gitaly + +#### New Gitaly configuration options required + +In order to function Gitaly needs some additional configuration information. Below we assume you installed Gitaly in `/home/git/gitaly` and GitLab Shell in `/home/git/gitlab-shell`. + +```shell +echo ' +[gitaly-ruby] +dir = "/home/git/gitaly/ruby" + +[gitlab-shell] +dir = "/home/git/gitlab-shell" +' | sudo -u git tee -a /home/git/gitaly/config.toml +``` + +#### Check Gitaly configuration + +Due to a bug in the `rake gitlab:gitaly:install` script your Gitaly +configuration file may contain syntax errors. The block name +`[[storages]]`, which may occur more than once in your `config.toml` +file, should be `[[storage]]` instead. + +```shell +sudo -u git -H sed -i.pre-10.1 's/\[\[storages\]\]/[[storage]]/' /home/git/gitaly/config.toml +``` + +#### Compile Gitaly + +```shell +cd /home/git/gitaly +sudo -u git -H git fetch --all --tags +sudo -u git -H git checkout v$(</home/git/gitlab/GITALY_SERVER_VERSION) +sudo -u git -H make +``` + +### 10. Update MySQL permissions + +If you are using MySQL you need to grant the GitLab user the necessary +permissions on the database: + +```bash +mysql -u root -p -e "GRANT TRIGGER ON \`gitlabhq_production\`.* TO 'git'@'localhost';" +``` + +If you use MySQL with replication, or just have MySQL configured with binary logging, +you will need to also run the following on all of your MySQL servers: + +```bash +mysql -u root -p -e "SET GLOBAL log_bin_trust_function_creators = 1;" +``` + +You can make this setting permanent by adding it to your `my.cnf`: + +``` +log_bin_trust_function_creators=1 +``` + +### 11. Update configuration files + +#### New configuration options for `gitlab.yml` + +There might be configuration options available for [`gitlab.yml`][yaml]. View them with the command below and apply them manually to your current `gitlab.yml`: + +```sh +cd /home/git/gitlab + +git diff origin/10-1-stable:config/gitlab.yml.example origin/10-2-stable:config/gitlab.yml.example +``` + +#### Nginx configuration + +Ensure you're still up-to-date with the latest NGINX configuration changes: + +```sh +cd /home/git/gitlab + +# For HTTPS configurations +git diff origin/10-1-stable:lib/support/nginx/gitlab-ssl origin/10-2-stable:lib/support/nginx/gitlab-ssl + +# For HTTP configurations +git diff origin/10-1-stable:lib/support/nginx/gitlab origin/10-2-stable:lib/support/nginx/gitlab +``` + +If you are using Strict-Transport-Security in your installation to continue using it you must enable it in your Nginx +configuration as GitLab application no longer handles setting it. + +If you are using Apache instead of NGINX please see the updated [Apache templates]. +Also note that because Apache does not support upstreams behind Unix sockets you +will need to let gitlab-workhorse listen on a TCP port. You can do this +via [/etc/default/gitlab]. + +[Apache templates]: https://gitlab.com/gitlab-org/gitlab-recipes/tree/master/web-server/apache +[/etc/default/gitlab]: https://gitlab.com/gitlab-org/gitlab-ce/blob/10-2-stable/lib/support/init.d/gitlab.default.example#L38 + +#### SMTP configuration + +If you're installing from source and use SMTP to deliver mail, you will need to add the following line +to config/initializers/smtp_settings.rb: + +```ruby +ActionMailer::Base.delivery_method = :smtp +``` + +See [smtp_settings.rb.sample] as an example. + +[smtp_settings.rb.sample]: https://gitlab.com/gitlab-org/gitlab-ce/blob/10-2-stable/config/initializers/smtp_settings.rb.sample#L13 + +#### Init script + +There might be new configuration options available for [`gitlab.default.example`][gl-example]. View them with the command below and apply them manually to your current `/etc/default/gitlab`: + +```sh +cd /home/git/gitlab + +git diff origin/10-1-stable:lib/support/init.d/gitlab.default.example origin/10-2-stable:lib/support/init.d/gitlab.default.example +``` + +Ensure you're still up-to-date with the latest init script changes: + +```bash +cd /home/git/gitlab + +sudo cp lib/support/init.d/gitlab /etc/init.d/gitlab +``` + +For Ubuntu 16.04.1 LTS: + +```bash +sudo systemctl daemon-reload +``` + +### 12. Install libs, migrations, etc. + +```bash +cd /home/git/gitlab + +# MySQL installations (note: the line below states '--without postgres') +sudo -u git -H bundle install --without postgres development test --deployment + +# PostgreSQL installations (note: the line below states '--without mysql') +sudo -u git -H bundle install --without mysql development test --deployment + +# Optional: clean up old gems +sudo -u git -H bundle clean + +# Run database migrations +sudo -u git -H bundle exec rake db:migrate RAILS_ENV=production + +# Compile GetText PO files + +sudo -u git -H bundle exec rake gettext:compile RAILS_ENV=production + +# Update node dependencies and recompile assets +sudo -u git -H bundle exec rake yarn:install gitlab:assets:clean gitlab:assets:compile RAILS_ENV=production NODE_ENV=production + +# Clean up cache +sudo -u git -H bundle exec rake cache:clear RAILS_ENV=production +``` + +**MySQL installations**: Run through the `MySQL strings limits` and `Tables and data conversion to utf8mb4` [tasks](../install/database_mysql.md). + +### 13. Start application + +```bash +sudo service gitlab start +sudo service nginx restart +``` + +### 14. Check application status + +Check if GitLab and its environment are configured correctly: + +```bash +cd /home/git/gitlab + +sudo -u git -H bundle exec rake gitlab:env:info RAILS_ENV=production +``` + +To make sure you didn't miss anything run a more thorough check: + +```bash +cd /home/git/gitlab + +sudo -u git -H bundle exec rake gitlab:check RAILS_ENV=production +``` + +If all items are green, then congratulations, the upgrade is complete! + +## Things went south? Revert to previous version (10.0) + +### 1. Revert the code to the previous version + +Follow the [upgrade guide from 9.5 to 10.0](9.5-to-10.0.md), except for the +database migration (the backup is already migrated to the previous version). + +### 2. Restore from the backup + +```bash +cd /home/git/gitlab + +sudo -u git -H bundle exec rake gitlab:backup:restore RAILS_ENV=production +``` + +If you have more than one backup `*.tar` file(s) please add `BACKUP=timestamp_of_backup` to the command above. + +[yaml]: https://gitlab.com/gitlab-org/gitlab-ce/blob/10-2-stable/config/gitlab.yml.example +[gl-example]: https://gitlab.com/gitlab-org/gitlab-ce/blob/10-2-stable/lib/support/init.d/gitlab.default.example diff --git a/doc/update/2.6-to-3.0.md b/doc/update/2.6-to-3.0.md index 97cd277b424..8f18bd93cea 100644 --- a/doc/update/2.6-to-3.0.md +++ b/doc/update/2.6-to-3.0.md @@ -1,3 +1,7 @@ +--- +comments: false +--- + # From 2.6 to 3.0 *Make sure you view this [upgrade guide from the `master` branch](https://gitlab.com/gitlab-org/gitlab-ce/tree/master/doc/update/2.6-to-3.0.md) for the most up to date instructions.* diff --git a/doc/update/2.9-to-3.0.md b/doc/update/2.9-to-3.0.md index a890aa885d5..6a3c2387683 100644 --- a/doc/update/2.9-to-3.0.md +++ b/doc/update/2.9-to-3.0.md @@ -1,3 +1,7 @@ +--- +comments: false +--- + # From 2.9 to 3.0 *Make sure you view this [upgrade guide from the `master` branch](https://gitlab.com/gitlab-org/gitlab-ce/tree/master/doc/update/2.9-to-3.0.md) for the most up to date instructions.* diff --git a/doc/update/3.0-to-3.1.md b/doc/update/3.0-to-3.1.md index e32508745a2..1f25b8265c9 100644 --- a/doc/update/3.0-to-3.1.md +++ b/doc/update/3.0-to-3.1.md @@ -1,3 +1,7 @@ +--- +comments: false +--- + # From 3.0 to 3.1 *Make sure you view this [upgrade guide from the `master` branch](https://gitlab.com/gitlab-org/gitlab-ce/tree/master/doc/update/3.0-to-3.1.md) for the most up to date instructions.* diff --git a/doc/update/3.1-to-4.0.md b/doc/update/3.1-to-4.0.md index b370464390e..1a53ddeb4bd 100644 --- a/doc/update/3.1-to-4.0.md +++ b/doc/update/3.1-to-4.0.md @@ -1,3 +1,7 @@ +--- +comments: false +--- + # From 3.1 to 4.0 *Make sure you view this [upgrade guide from the `master` branch](https://gitlab.com/gitlab-org/gitlab-ce/tree/master/doc/update/3.1-to-4.0.md) for the most up to date instructions.* diff --git a/doc/update/4.0-to-4.1.md b/doc/update/4.0-to-4.1.md index 7124424bb60..40a133e796e 100644 --- a/doc/update/4.0-to-4.1.md +++ b/doc/update/4.0-to-4.1.md @@ -1,3 +1,7 @@ +--- +comments: false +--- + # From 4.0 to 4.1 *Make sure you view this [upgrade guide from the `master` branch](https://gitlab.com/gitlab-org/gitlab-ce/tree/master/doc/update/4.0-to-4.1.md) for the most up to date instructions.* diff --git a/doc/update/4.1-to-4.2.md b/doc/update/4.1-to-4.2.md index 8ed5b333a2e..1fd6c58bda7 100644 --- a/doc/update/4.1-to-4.2.md +++ b/doc/update/4.1-to-4.2.md @@ -1,3 +1,7 @@ +--- +comments: false +--- + # From 4.1 to 4.2 *Make sure you view this [upgrade guide from the `master` branch](https://gitlab.com/gitlab-org/gitlab-ce/tree/master/doc/update/4.1-to-4.2.md) for the most up to date instructions.* diff --git a/doc/update/4.2-to-5.0.md b/doc/update/4.2-to-5.0.md index 1ec39218ba8..311664b2bc1 100644 --- a/doc/update/4.2-to-5.0.md +++ b/doc/update/4.2-to-5.0.md @@ -1,3 +1,7 @@ +--- +comments: false +--- + # From 4.2 to 5.0 *Make sure you view this [upgrade guide from the `master` branch](https://gitlab.com/gitlab-org/gitlab-ce/tree/master/doc/update/4.2-to-5.0.md) for the most up to date instructions.* diff --git a/doc/update/5.0-to-5.1.md b/doc/update/5.0-to-5.1.md index 9c9950fb2c6..7067ea4c40c 100644 --- a/doc/update/5.0-to-5.1.md +++ b/doc/update/5.0-to-5.1.md @@ -1,3 +1,7 @@ +--- +comments: false +--- + # From 5.0 to 5.1 *Make sure you view this [upgrade guide from the `master` branch](https://gitlab.com/gitlab-org/gitlab-ce/tree/master/doc/update/5.0-to-5.1.md) for the most up to date instructions.* diff --git a/doc/update/5.1-to-5.2.md b/doc/update/5.1-to-5.2.md index 2aab47d2d7c..4faf5fa549e 100644 --- a/doc/update/5.1-to-5.2.md +++ b/doc/update/5.1-to-5.2.md @@ -1,3 +1,7 @@ +--- +comments: false +--- + # From 5.1 to 5.2 *Make sure you view this [upgrade guide from the `master` branch](https://gitlab.com/gitlab-org/gitlab-ce/tree/master/doc/update/5.1-to-5.2.md) for the most up to date instructions.* diff --git a/doc/update/5.1-to-5.4.md b/doc/update/5.1-to-5.4.md index e80f1b89c63..212343bac3f 100644 --- a/doc/update/5.1-to-5.4.md +++ b/doc/update/5.1-to-5.4.md @@ -1,3 +1,7 @@ +--- +comments: false +--- + # From 5.1 to 5.4 *Make sure you view this [upgrade guide from the `master` branch](https://gitlab.com/gitlab-org/gitlab-ce/tree/master/doc/update/5.1-to-5.4.md) for the most up to date instructions.* diff --git a/doc/update/5.1-to-6.0.md b/doc/update/5.1-to-6.0.md index 1ee175383da..865d38e0ca4 100644 --- a/doc/update/5.1-to-6.0.md +++ b/doc/update/5.1-to-6.0.md @@ -1,3 +1,7 @@ +--- +comments: false +--- + # From 5.1 to 6.0 *Make sure you view this [upgrade guide from the `master` branch](https://gitlab.com/gitlab-org/gitlab-ce/tree/master/doc/update/5.1-to-6.0.md) for the most up to date instructions.* diff --git a/doc/update/5.2-to-5.3.md b/doc/update/5.2-to-5.3.md index 2ae50510f63..ed4f3ebdd53 100644 --- a/doc/update/5.2-to-5.3.md +++ b/doc/update/5.2-to-5.3.md @@ -1,3 +1,7 @@ +--- +comments: false +--- + # From 5.2 to 5.3 *Make sure you view this [upgrade guide from the `master` branch](https://gitlab.com/gitlab-org/gitlab-ce/tree/master/doc/update/5.2-to-5.3.md) for the most up to date instructions.* diff --git a/doc/update/5.3-to-5.4.md b/doc/update/5.3-to-5.4.md index 842e3bb6791..7277250eb32 100644 --- a/doc/update/5.3-to-5.4.md +++ b/doc/update/5.3-to-5.4.md @@ -1,3 +1,7 @@ +--- +comments: false +--- + # From 5.3 to 5.4 *Make sure you view this [upgrade guide from the `master` branch](https://gitlab.com/gitlab-org/gitlab-ce/tree/master/doc/update/5.3-to-5.4.md) for the most up to date instructions.* diff --git a/doc/update/5.4-to-6.0.md b/doc/update/5.4-to-6.0.md index 44715984f0c..dacdf05cc9c 100644 --- a/doc/update/5.4-to-6.0.md +++ b/doc/update/5.4-to-6.0.md @@ -1,3 +1,7 @@ +--- +comments: false +--- + # From 5.4 to 6.0 *Make sure you view this [upgrade guide from the `master` branch](https://gitlab.com/gitlab-org/gitlab-ce/tree/master/doc/update/5.4-to-6.0.md) for the most up to date instructions.* diff --git a/doc/update/6.0-to-6.1.md b/doc/update/6.0-to-6.1.md index 0c672abeb05..a3c52a1cfb4 100644 --- a/doc/update/6.0-to-6.1.md +++ b/doc/update/6.0-to-6.1.md @@ -1,3 +1,7 @@ +--- +comments: false +--- + # From 6.0 to 6.1 *Make sure you view this [upgrade guide from the `master` branch](https://gitlab.com/gitlab-org/gitlab-ce/tree/master/doc/update/6.0-to-6.1.md) for the most up to date instructions.* diff --git a/doc/update/6.1-to-6.2.md b/doc/update/6.1-to-6.2.md index d3760cf0619..36a395bf01e 100644 --- a/doc/update/6.1-to-6.2.md +++ b/doc/update/6.1-to-6.2.md @@ -1,3 +1,7 @@ +--- +comments: false +--- + # From 6.1 to 6.2 *Make sure you view this [upgrade guide from the `master` branch](https://gitlab.com/gitlab-org/gitlab-ce/tree/master/doc/update/6.1-to-6.2.md) for the most up to date instructions.* diff --git a/doc/update/6.2-to-6.3.md b/doc/update/6.2-to-6.3.md index 91105de2e29..02e87a08b8f 100644 --- a/doc/update/6.2-to-6.3.md +++ b/doc/update/6.2-to-6.3.md @@ -1,3 +1,7 @@ +--- +comments: false +--- + # From 6.2 to 6.3 *Make sure you view this [upgrade guide from the `master` branch](https://gitlab.com/gitlab-org/gitlab-ce/tree/master/doc/update/6.2-to-6.3.md) for the most up to date instructions.* diff --git a/doc/update/6.3-to-6.4.md b/doc/update/6.3-to-6.4.md index 20b58ed8b25..285ed06bdad 100644 --- a/doc/update/6.3-to-6.4.md +++ b/doc/update/6.3-to-6.4.md @@ -1,3 +1,7 @@ +--- +comments: false +--- + # From 6.3 to 6.4 *Make sure you view this [upgrade guide from the `master` branch](https://gitlab.com/gitlab-org/gitlab-ce/tree/master/doc/update/6.3-to-6.4.md) for the most up to date instructions.* diff --git a/doc/update/6.4-to-6.5.md b/doc/update/6.4-to-6.5.md index 5ee0f040b5d..e07c98a5ad4 100644 --- a/doc/update/6.4-to-6.5.md +++ b/doc/update/6.4-to-6.5.md @@ -1,3 +1,7 @@ +--- +comments: false +--- + # From 6.4 to 6.5 *Make sure you view this [upgrade guide from the `master` branch](https://gitlab.com/gitlab-org/gitlab-ce/tree/master/doc/update/6.4-to-6.5.md) for the most up to date instructions.* diff --git a/doc/update/6.5-to-6.6.md b/doc/update/6.5-to-6.6.md index fa3712f83ad..3f79b19644e 100644 --- a/doc/update/6.5-to-6.6.md +++ b/doc/update/6.5-to-6.6.md @@ -1,3 +1,7 @@ +--- +comments: false +--- + # From 6.5 to 6.6 *Make sure you view this [upgrade guide from the `master` branch](https://gitlab.com/gitlab-org/gitlab-ce/tree/master/doc/update/6.5-to-6.6.md) for the most up to date instructions.* diff --git a/doc/update/6.6-to-6.7.md b/doc/update/6.6-to-6.7.md index 9c85ed091c5..a0542d20d49 100644 --- a/doc/update/6.6-to-6.7.md +++ b/doc/update/6.6-to-6.7.md @@ -1,3 +1,7 @@ +--- +comments: false +--- + # From 6.6 to 6.7 *Make sure you view this [upgrade guide from the `master` branch](https://gitlab.com/gitlab-org/gitlab-ce/tree/master/doc/update/6.6-to-6.7.md) for the most up to date instructions.* diff --git a/doc/update/6.7-to-6.8.md b/doc/update/6.7-to-6.8.md index 687c1265d9b..acf004577f1 100644 --- a/doc/update/6.7-to-6.8.md +++ b/doc/update/6.7-to-6.8.md @@ -1,3 +1,7 @@ +--- +comments: false +--- + # From 6.7 to 6.8 *Make sure you view this [upgrade guide from the `master` branch](https://gitlab.com/gitlab-org/gitlab-ce/tree/master/doc/update/6.7-to-6.8.md) for the most up to date instructions.* diff --git a/doc/update/6.8-to-6.9.md b/doc/update/6.8-to-6.9.md index 0205b0c896a..3d7b1e5346b 100644 --- a/doc/update/6.8-to-6.9.md +++ b/doc/update/6.8-to-6.9.md @@ -1,3 +1,7 @@ +--- +comments: false +--- + # From 6.8 to 6.9 *Make sure you view this [upgrade guide from the `master` branch](https://gitlab.com/gitlab-org/gitlab-ce/tree/master/doc/update/6.8-to-6.9.md) for the most up to date instructions.* diff --git a/doc/update/6.9-to-7.0.md b/doc/update/6.9-to-7.0.md index 4b6e3989893..27063948028 100644 --- a/doc/update/6.9-to-7.0.md +++ b/doc/update/6.9-to-7.0.md @@ -1,3 +1,7 @@ +--- +comments: false +--- + # From 6.9 to 7.0 *Make sure you view this [upgrade guide from the `master` branch](https://gitlab.com/gitlab-org/gitlab-ce/tree/master/doc/update/6.9-to-7.0.md) for the most up to date instructions.* diff --git a/doc/update/6.x-or-7.x-to-7.14.md b/doc/update/6.x-or-7.x-to-7.14.md index 1e39fe47ef9..41d0e78b7d8 100644 --- a/doc/update/6.x-or-7.x-to-7.14.md +++ b/doc/update/6.x-or-7.x-to-7.14.md @@ -1,3 +1,7 @@ +--- +comments: false +--- + # From 6.x or 7.x to 7.14 *Make sure you view this [upgrade guide from the `master` branch](https://gitlab.com/gitlab-org/gitlab-ce/tree/master/doc/update/6.x-or-7.x-to-7.14.md) for the most up to date instructions.* diff --git a/doc/update/7.0-to-7.1.md b/doc/update/7.0-to-7.1.md index 2e9457aa142..308e8aeb985 100644 --- a/doc/update/7.0-to-7.1.md +++ b/doc/update/7.0-to-7.1.md @@ -1,3 +1,7 @@ +--- +comments: false +--- + # From 7.0 to 7.1 *Make sure you view this [upgrade guide from the `master` branch](https://gitlab.com/gitlab-org/gitlab-ce/tree/master/doc/update/7.0-to-7.1.md) for the most up to date instructions.* diff --git a/doc/update/7.1-to-7.2.md b/doc/update/7.1-to-7.2.md index e5045b5570f..07f92ac3af6 100644 --- a/doc/update/7.1-to-7.2.md +++ b/doc/update/7.1-to-7.2.md @@ -1,3 +1,7 @@ +--- +comments: false +--- + # From 7.1 to 7.2 *Make sure you view this [upgrade guide from the `master` branch](https://gitlab.com/gitlab-org/gitlab-ce/tree/master/doc/update/7.1-to-7.2.md) for the most up to date instructions.* diff --git a/doc/update/7.10-to-7.11.md b/doc/update/7.10-to-7.11.md index 89213ba7178..39eeefc0e32 100644 --- a/doc/update/7.10-to-7.11.md +++ b/doc/update/7.10-to-7.11.md @@ -1,3 +1,7 @@ +--- +comments: false +--- + # From 7.10 to 7.11 ### 0. Stop server diff --git a/doc/update/7.11-to-7.12.md b/doc/update/7.11-to-7.12.md index 3865186918c..530066e5fdb 100644 --- a/doc/update/7.11-to-7.12.md +++ b/doc/update/7.11-to-7.12.md @@ -1,3 +1,7 @@ +--- +comments: false +--- + # From 7.11 to 7.12 ### 0. Double-check your Git version diff --git a/doc/update/7.12-to-7.13.md b/doc/update/7.12-to-7.13.md index 4c8d8f1f741..8f413a2079a 100644 --- a/doc/update/7.12-to-7.13.md +++ b/doc/update/7.12-to-7.13.md @@ -1,3 +1,7 @@ +--- +comments: false +--- + # From 7.12 to 7.13 ### 0. Double-check your Git version diff --git a/doc/update/7.13-to-7.14.md b/doc/update/7.13-to-7.14.md index 934898da5a1..a8980662855 100644 --- a/doc/update/7.13-to-7.14.md +++ b/doc/update/7.13-to-7.14.md @@ -1,3 +1,7 @@ +--- +comments: false +--- + # From 7.13 to 7.14 ### 0. Double-check your Git version diff --git a/doc/update/7.14-to-8.0.md b/doc/update/7.14-to-8.0.md index 25fa6d93f06..513afccff50 100644 --- a/doc/update/7.14-to-8.0.md +++ b/doc/update/7.14-to-8.0.md @@ -1,3 +1,7 @@ +--- +comments: false +--- + # From 7.14 to 8.0 ### 0. Double-check your Git version diff --git a/doc/update/7.2-to-7.3.md b/doc/update/7.2-to-7.3.md index d3391ddd225..a16f9de54e4 100644 --- a/doc/update/7.2-to-7.3.md +++ b/doc/update/7.2-to-7.3.md @@ -1,3 +1,7 @@ +--- +comments: false +--- + # From 7.2 to 7.3 *Make sure you view this [upgrade guide from the `master` branch](https://gitlab.com/gitlab-org/gitlab-ce/tree/master/doc/update/7.2-to-7.3.md) for the most up to date instructions.* diff --git a/doc/update/7.3-to-7.4.md b/doc/update/7.3-to-7.4.md index 6d632dc3c8e..734c655f1d1 100644 --- a/doc/update/7.3-to-7.4.md +++ b/doc/update/7.3-to-7.4.md @@ -1,3 +1,7 @@ +--- +comments: false +--- + # From 7.3 to 7.4 *Make sure you view this [upgrade guide from the `master` branch](https://gitlab.com/gitlab-org/gitlab-ce/tree/master/doc/update/7.3-to-7.4.md) for the most up to date instructions.* diff --git a/doc/update/7.4-to-7.5.md b/doc/update/7.4-to-7.5.md index ec50706d421..7a3a49ff948 100644 --- a/doc/update/7.4-to-7.5.md +++ b/doc/update/7.4-to-7.5.md @@ -1,3 +1,7 @@ +--- +comments: false +--- + # From 7.4 to 7.5 ### 0. Stop server diff --git a/doc/update/7.5-to-7.6.md b/doc/update/7.5-to-7.6.md index 331f5de080e..f0dfb177b79 100644 --- a/doc/update/7.5-to-7.6.md +++ b/doc/update/7.5-to-7.6.md @@ -1,3 +1,7 @@ +--- +comments: false +--- + # From 7.5 to 7.6 ### 0. Stop server diff --git a/doc/update/7.6-to-7.7.md b/doc/update/7.6-to-7.7.md index 918b10fbd95..85de6b0c546 100644 --- a/doc/update/7.6-to-7.7.md +++ b/doc/update/7.6-to-7.7.md @@ -1,3 +1,7 @@ +--- +comments: false +--- + # From 7.6 to 7.7 ### 0. Stop server diff --git a/doc/update/7.7-to-7.8.md b/doc/update/7.7-to-7.8.md index 84e0464a824..7cee5f79a13 100644 --- a/doc/update/7.7-to-7.8.md +++ b/doc/update/7.7-to-7.8.md @@ -1,3 +1,7 @@ +--- +comments: false +--- + # From 7.7 to 7.8 ### 0. Stop server diff --git a/doc/update/7.8-to-7.9.md b/doc/update/7.8-to-7.9.md index b0dc2ba1dbb..5a8b689dbc1 100644 --- a/doc/update/7.8-to-7.9.md +++ b/doc/update/7.8-to-7.9.md @@ -1,3 +1,7 @@ +--- +comments: false +--- + # From 7.8 to 7.9 ### 0. Stop server diff --git a/doc/update/7.9-to-7.10.md b/doc/update/7.9-to-7.10.md index 8f7f84b41ba..99df51dbb99 100644 --- a/doc/update/7.9-to-7.10.md +++ b/doc/update/7.9-to-7.10.md @@ -1,3 +1,7 @@ +--- +comments: false +--- + # From 7.9 to 7.10 ### 0. Stop server diff --git a/doc/update/8.0-to-8.1.md b/doc/update/8.0-to-8.1.md index 6ee0c0656ee..f612606af68 100644 --- a/doc/update/8.0-to-8.1.md +++ b/doc/update/8.0-to-8.1.md @@ -1,3 +1,7 @@ +--- +comments: false +--- + # From 8.0 to 8.1 **NOTE:** GitLab 8.0 introduced several significant changes related to diff --git a/doc/update/8.1-to-8.2.md b/doc/update/8.1-to-8.2.md index 4c9ff5c5c0a..2d0b19abd74 100644 --- a/doc/update/8.1-to-8.2.md +++ b/doc/update/8.1-to-8.2.md @@ -1,3 +1,7 @@ +--- +comments: false +--- + # From 8.1 to 8.2 **NOTE:** GitLab 8.0 introduced several significant changes related to diff --git a/doc/update/8.10-to-8.11.md b/doc/update/8.10-to-8.11.md index e538983e603..df3e34f5cc6 100644 --- a/doc/update/8.10-to-8.11.md +++ b/doc/update/8.10-to-8.11.md @@ -1,3 +1,7 @@ +--- +comments: false +--- + # From 8.10 to 8.11 Make sure you view this update guide from the tag (version) of GitLab you would diff --git a/doc/update/8.11-to-8.12.md b/doc/update/8.11-to-8.12.md index 604166beb56..9d6a1f42375 100644 --- a/doc/update/8.11-to-8.12.md +++ b/doc/update/8.11-to-8.12.md @@ -1,3 +1,7 @@ +--- +comments: false +--- + # From 8.11 to 8.12 Make sure you view this update guide from the tag (version) of GitLab you would diff --git a/doc/update/8.12-to-8.13.md b/doc/update/8.12-to-8.13.md index d83965131f5..6225dee9802 100644 --- a/doc/update/8.12-to-8.13.md +++ b/doc/update/8.12-to-8.13.md @@ -1,3 +1,7 @@ +--- +comments: false +--- + # From 8.12 to 8.13 Make sure you view this update guide from the tag (version) of GitLab you would diff --git a/doc/update/8.13-to-8.14.md b/doc/update/8.13-to-8.14.md index aaadcec8ac0..d2508e3f980 100644 --- a/doc/update/8.13-to-8.14.md +++ b/doc/update/8.13-to-8.14.md @@ -1,3 +1,7 @@ +--- +comments: false +--- + # From 8.13 to 8.14 Make sure you view this update guide from the tag (version) of GitLab you would diff --git a/doc/update/8.14-to-8.15.md b/doc/update/8.14-to-8.15.md index a68fe3bb605..daf8d0f2ca6 100644 --- a/doc/update/8.14-to-8.15.md +++ b/doc/update/8.14-to-8.15.md @@ -1,3 +1,7 @@ +--- +comments: false +--- + # From 8.14 to 8.15 Make sure you view this update guide from the tag (version) of GitLab you would diff --git a/doc/update/8.15-to-8.16.md b/doc/update/8.15-to-8.16.md index 9f8f0f714d4..3668142edd2 100644 --- a/doc/update/8.15-to-8.16.md +++ b/doc/update/8.15-to-8.16.md @@ -1,3 +1,7 @@ +--- +comments: false +--- + # From 8.15 to 8.16 Make sure you view this update guide from the tag (version) of GitLab you would diff --git a/doc/update/8.16-to-8.17.md b/doc/update/8.16-to-8.17.md index 74ffe0bc846..ee2e31c2aec 100644 --- a/doc/update/8.16-to-8.17.md +++ b/doc/update/8.16-to-8.17.md @@ -1,3 +1,7 @@ +--- +comments: false +--- + # From 8.16 to 8.17 Make sure you view this update guide from the tag (version) of GitLab you would diff --git a/doc/update/8.17-to-9.0.md b/doc/update/8.17-to-9.0.md index baab217b6b7..2e0c26a9092 100644 --- a/doc/update/8.17-to-9.0.md +++ b/doc/update/8.17-to-9.0.md @@ -1,3 +1,7 @@ +--- +comments: false +--- + # From 8.17 to 9.0 Make sure you view this update guide from the tag (version) of GitLab you would diff --git a/doc/update/8.2-to-8.3.md b/doc/update/8.2-to-8.3.md index 4b3c5bf6d64..3a0d647cbfe 100644 --- a/doc/update/8.2-to-8.3.md +++ b/doc/update/8.2-to-8.3.md @@ -1,3 +1,7 @@ +--- +comments: false +--- + # From 8.2 to 8.3 Make sure you view this update guide from the tag (version) of GitLab you would diff --git a/doc/update/8.3-to-8.4.md b/doc/update/8.3-to-8.4.md index 8b89455ca87..f5162dd5ff5 100644 --- a/doc/update/8.3-to-8.4.md +++ b/doc/update/8.3-to-8.4.md @@ -1,3 +1,7 @@ +--- +comments: false +--- + # From 8.3 to 8.4 Make sure you view this update guide from the tag (version) of GitLab you would diff --git a/doc/update/8.4-to-8.5.md b/doc/update/8.4-to-8.5.md index 0eedfaee2db..9e2f98add8d 100644 --- a/doc/update/8.4-to-8.5.md +++ b/doc/update/8.4-to-8.5.md @@ -1,3 +1,7 @@ +--- +comments: false +--- + # From 8.4 to 8.5 Make sure you view this update guide from the tag (version) of GitLab you would diff --git a/doc/update/8.5-to-8.6.md b/doc/update/8.5-to-8.6.md index 851056161bb..55d8178c407 100644 --- a/doc/update/8.5-to-8.6.md +++ b/doc/update/8.5-to-8.6.md @@ -1,3 +1,7 @@ +--- +comments: false +--- + # From 8.5 to 8.6 Make sure you view this update guide from the tag (version) of GitLab you would diff --git a/doc/update/8.6-to-8.7.md b/doc/update/8.6-to-8.7.md index 34c727260aa..49db6f2967c 100644 --- a/doc/update/8.6-to-8.7.md +++ b/doc/update/8.6-to-8.7.md @@ -1,3 +1,7 @@ +--- +comments: false +--- + # From 8.6 to 8.7 Make sure you view this update guide from the tag (version) of GitLab you would diff --git a/doc/update/8.7-to-8.8.md b/doc/update/8.7-to-8.8.md index 6feeb1919de..ee7ec6f7614 100644 --- a/doc/update/8.7-to-8.8.md +++ b/doc/update/8.7-to-8.8.md @@ -1,3 +1,7 @@ +--- +comments: false +--- + # From 8.7 to 8.8 Make sure you view this update guide from the tag (version) of GitLab you would diff --git a/doc/update/8.8-to-8.9.md b/doc/update/8.8-to-8.9.md index 61cdf8854d4..7508443c30a 100644 --- a/doc/update/8.8-to-8.9.md +++ b/doc/update/8.8-to-8.9.md @@ -1,3 +1,7 @@ +--- +comments: false +--- + # From 8.8 to 8.9 Make sure you view this update guide from the tag (version) of GitLab you would diff --git a/doc/update/8.9-to-8.10.md b/doc/update/8.9-to-8.10.md index 42132f690d8..915e7db819a 100644 --- a/doc/update/8.9-to-8.10.md +++ b/doc/update/8.9-to-8.10.md @@ -1,3 +1,7 @@ +--- +comments: false +--- + # From 8.9 to 8.10 Make sure you view this update guide from the tag (version) of GitLab you would diff --git a/doc/update/9.0-to-9.1.md b/doc/update/9.0-to-9.1.md index 6f1870a1366..f60bd92e236 100644 --- a/doc/update/9.0-to-9.1.md +++ b/doc/update/9.0-to-9.1.md @@ -1,3 +1,7 @@ +--- +comments: false +--- + # From 9.0 to 9.1 Make sure you view this update guide from the tag (version) of GitLab you would diff --git a/doc/update/9.1-to-9.2.md b/doc/update/9.1-to-9.2.md index ce72b313031..2fff6544797 100644 --- a/doc/update/9.1-to-9.2.md +++ b/doc/update/9.1-to-9.2.md @@ -1,3 +1,7 @@ +--- +comments: false +--- + # From 9.1 to 9.2 Make sure you view this update guide from the tag (version) of GitLab you would diff --git a/doc/update/9.2-to-9.3.md b/doc/update/9.2-to-9.3.md index 779ced0cf75..1b36cf53f4c 100644 --- a/doc/update/9.2-to-9.3.md +++ b/doc/update/9.2-to-9.3.md @@ -1,3 +1,7 @@ +--- +comments: false +--- + # From 9.2 to 9.3 Make sure you view this update guide from the tag (version) of GitLab you would diff --git a/doc/update/9.3-to-9.4.md b/doc/update/9.3-to-9.4.md index 78d8a6c7de5..210b6eb607d 100644 --- a/doc/update/9.3-to-9.4.md +++ b/doc/update/9.3-to-9.4.md @@ -1,3 +1,7 @@ +--- +comments: false +--- + # From 9.3 to 9.4 Make sure you view this update guide from the tag (version) of GitLab you would diff --git a/doc/update/9.4-to-9.5.md b/doc/update/9.4-to-9.5.md index a7255142ef5..1bfc1167c36 100644 --- a/doc/update/9.4-to-9.5.md +++ b/doc/update/9.4-to-9.5.md @@ -1,3 +1,7 @@ +--- +comments: false +--- + # From 9.4 to 9.5 Make sure you view this update guide from the tag (version) of GitLab you would diff --git a/doc/update/9.5-to-10.0.md b/doc/update/9.5-to-10.0.md index 8581e6511f2..8d1cf0f737b 100644 --- a/doc/update/9.5-to-10.0.md +++ b/doc/update/9.5-to-10.0.md @@ -1,3 +1,7 @@ +--- +comments: false +--- + # From 9.5 to 10.0 Make sure you view this update guide from the tag (version) of GitLab you would diff --git a/doc/update/patch_versions.md b/doc/update/patch_versions.md index b2679d1ff22..e1857ce99c6 100644 --- a/doc/update/patch_versions.md +++ b/doc/update/patch_versions.md @@ -1,3 +1,7 @@ +--- +comments: false +--- + # Universal update guide for patch versions ## Select Version to Install diff --git a/doc/update/upgrader.md b/doc/update/upgrader.md index eb7f14a96d5..746d6bf93e7 100644 --- a/doc/update/upgrader.md +++ b/doc/update/upgrader.md @@ -1,3 +1,7 @@ +--- +comments: false +--- + # GitLab Upgrader (deprecated) *DEPRECATED* We recommend to [switch to the Omnibus package and repository server](https://about.gitlab.com/update/) instead of using this script. diff --git a/doc/user/discussions/img/image_resolved_discussion.png b/doc/user/discussions/img/image_resolved_discussion.png Binary files differindex ed00b5c77fe..ed00b5c77fe 100755..100644 --- a/doc/user/discussions/img/image_resolved_discussion.png +++ b/doc/user/discussions/img/image_resolved_discussion.png diff --git a/doc/user/discussions/img/onion_skin_view.png b/doc/user/discussions/img/onion_skin_view.png Binary files differindex 91c3b396844..91c3b396844 100755..100644 --- a/doc/user/discussions/img/onion_skin_view.png +++ b/doc/user/discussions/img/onion_skin_view.png diff --git a/doc/user/discussions/img/swipe_view.png b/doc/user/discussions/img/swipe_view.png Binary files differindex 82d6e52173c..82d6e52173c 100755..100644 --- a/doc/user/discussions/img/swipe_view.png +++ b/doc/user/discussions/img/swipe_view.png diff --git a/doc/user/discussions/img/two_up_view.png b/doc/user/discussions/img/two_up_view.png Binary files differindex d9e90708e87..d9e90708e87 100755..100644 --- a/doc/user/discussions/img/two_up_view.png +++ b/doc/user/discussions/img/two_up_view.png diff --git a/doc/user/markdown.md b/doc/user/markdown.md index 454988b9b80..fb61e360996 100644 --- a/doc/user/markdown.md +++ b/doc/user/markdown.md @@ -368,6 +368,37 @@ _Be advised that KaTeX only supports a [subset][katex-subset] of LaTeX._ >**Note:** This also works for the asciidoctor `:stem: latexmath`. For details see the [asciidoctor user manual][asciidoctor-manual]. +### Mermaid + +> If this is not rendered correctly, see +https://gitlab.com/gitlab-org/gitlab-ce/blob/master/doc/user/markdown.md#mermaid + +It is possible to generate diagrams and flowcharts from text using [Mermaid][mermaid]. + +In order to generate a diagram or flowchart, you should write your text inside the `mermaid` block. + +Example: + + ```mermaid + graph TD; + A-->B; + A-->C; + B-->D; + C-->D; + ``` + +Becomes: + +```mermaid +graph TD; + A-->B; + A-->C; + B-->D; + C-->D; +``` + +For details see the [Mermaid official page][mermaid]. + ## Standard Markdown ### Headers @@ -814,6 +845,7 @@ A link starting with a `/` is relative to the wiki root. [^2]: This is my awesome footnote. [markdown.md]: https://gitlab.com/gitlab-org/gitlab-ce/blob/master/doc/user/markdown.md +[mermaid]: https://mermaidjs.github.io/ "Mermaid website" [rouge]: http://rouge.jneen.net/ "Rouge website" [redcarpet]: https://github.com/vmg/redcarpet "Redcarpet website" [katex]: https://github.com/Khan/KaTeX "KaTeX website" diff --git a/doc/user/permissions.md b/doc/user/permissions.md index c03700a3501..b9532bf897f 100644 --- a/doc/user/permissions.md +++ b/doc/user/permissions.md @@ -197,6 +197,7 @@ instance and project. In addition, all admins can use the admin interface under |---------------------------------------|-----------------|-------------|----------|--------| | See commits and jobs | ✓ | ✓ | ✓ | ✓ | | Retry or cancel job | | ✓ | ✓ | ✓ | +| Erase job artifacts and trace | | ✓ [^7] | ✓ | ✓ | | Remove project | | | ✓ | ✓ | | Create project | | | ✓ | ✓ | | Change project configuration | | | ✓ | ✓ | @@ -261,5 +262,6 @@ only. [^4]: Not allowed for Guest, Reporter, Developer, Master, or Owner [^5]: Only if user is not external one. [^6]: Only if user is a member of the project. +[^7]: Only if the build was triggered by the user [ce-18994]: https://gitlab.com/gitlab-org/gitlab-ce/issues/18994 [new-mod]: project/new_ci_build_permissions_model.md diff --git a/doc/user/profile/index.md b/doc/user/profile/index.md index 5ebb88bf324..5fcc0501dc1 100644 --- a/doc/user/profile/index.md +++ b/doc/user/profile/index.md @@ -52,7 +52,7 @@ You can edit your account settings by navigating from the up-right corner menu b From there, you can: - Update your personal information -- Manage [private tokens](../../api/README.md#private-tokens), email tokens, [2FA](account/two_factor_authentication.md) +- Manage [2FA](account/two_factor_authentication.md) - Change your username and [delete your account](account/delete_account.md) - Manage applications that can [use GitLab as an OAuth provider](../../integration/oauth_provider.md#introduction-to-oauth) diff --git a/doc/user/profile/personal_access_tokens.md b/doc/user/profile/personal_access_tokens.md index f28c034e74c..9b4fdd65e2f 100644 --- a/doc/user/profile/personal_access_tokens.md +++ b/doc/user/profile/personal_access_tokens.md @@ -2,17 +2,15 @@ > [Introduced][ce-3749] in GitLab 8.8. -Personal access tokens are useful if you need access to the [GitLab API][api]. -Instead of using your private token which grants full access to your account, -personal access tokens could be a better fit because of their -[granular permissions](#limiting-scopes-of-a-personal-access-token). +Personal access tokens are the preferred way for third party applications and scripts to +authenticate with the [GitLab API][api], if using [OAuth2](../../api/oauth2.md) is not practical. You can also use them to authenticate against Git over HTTP. They are the only accepted method of authentication when you have [Two-Factor Authentication (2FA)][2fa] enabled. Once you have your token, [pass it to the API][usage] using either the -`private_token` parameter or the `PRIVATE-TOKEN` header. +`private_token` parameter or the `Private-Token` header. The expiration of personal access tokens happens on the date you define, at midnight UTC. @@ -49,12 +47,14 @@ the following table. |`read_user` | Allows access to the read-only endpoints under `/users`. Essentially, any of the `GET` requests in the [Users API][users] are allowed ([introduced][ce-5951] in GitLab 8.15). | | `api` | Grants complete access to the API (read/write) ([introduced][ce-5951] in GitLab 8.15). Required for accessing Git repositories over HTTP when 2FA is enabled. | | `read_registry` | Allows to read [container registry] images if a project is private and authorization is required ([introduced][ce-11845] in GitLab 9.3). | +| `sudo` | Allows performing API actions as any user in the system (if the authenticated user is an admin) ([introduced][ce-14838] in GitLab 10.2). | [2fa]: ../account/two_factor_authentication.md [api]: ../../api/README.md [ce-3749]: https://gitlab.com/gitlab-org/gitlab-ce/merge_requests/3749 [ce-5951]: https://gitlab.com/gitlab-org/gitlab-ce/merge_requests/5951 [ce-11845]: https://gitlab.com/gitlab-org/gitlab-ce/merge_requests/11845 +[ce-14838]: https://gitlab.com/gitlab-org/gitlab-ce/merge_requests/14838 [container registry]: ../project/container_registry.md [users]: ../../api/users.md -[usage]: ../../api/README.md#basic-usage +[usage]: ../../api/README.md#personal-access-tokens diff --git a/doc/user/profile/preferences.md b/doc/user/profile/preferences.md index f2ad42f21fd..022d6317555 100644 --- a/doc/user/profile/preferences.md +++ b/doc/user/profile/preferences.md @@ -55,9 +55,10 @@ You have 6 options here that you can use for your default dashboard view: The project home page content setting allows you to choose what content you want to see on a project’s home page. -You can choose between 2 options: +You can choose between 3 options: - Show the files and the readme (default) +- Show the readme - Show the project’s activity [rouge]: http://rouge.jneen.net/ "Rouge website" diff --git a/doc/user/project/clusters/index.md b/doc/user/project/clusters/index.md index 7d9e771f570..cf0c7c109a8 100644 --- a/doc/user/project/clusters/index.md +++ b/doc/user/project/clusters/index.md @@ -1,14 +1,15 @@ -# Connecting GitLab with GKE +# Connecting GitLab with a Kubernetes cluster > [Introduced](https://gitlab.com/gitlab-org/gitlab-ce/issues/35954) in 10.1. CAUTION: **Warning:** The Cluster integration is currently in **Beta**. -Connect your project to Google Container Engine (GKE) in a few steps. - With a cluster associated to your project, you can use Review Apps, deploy your -applications, run your pipelines, and much more in an easy way. +applications, run your pipelines, and much more, in an easy way. + +Connect your project to Google Kubernetes Engine (GKE) or your own Kubernetes +cluster in a few steps. NOTE: **Note:** The Cluster integration will eventually supersede the @@ -30,36 +31,58 @@ prerequisites must be met: - You must have Master [permissions] in order to be able to access the **Cluster** page. -If all of the above requirements are met, you can proceed to add a new cluster. +If all of the above requirements are met, you can proceed to add a new GKE +cluster. ## Adding a cluster NOTE: **Note:** You need Master [permissions] and above to add a cluster. +There are two options when adding a new cluster; either use Google Kubernetes +Engine (GKE) or provide the credentials to your own Kubernetes cluster. + To add a new cluster: -1. Navigate to your project's **CI/CD > Cluster** page. -1. Connect your Google account if you haven't done already by clicking the - "Sign-in with Google" button. -1. Fill in the requested values: - - **Cluster name** (required) - The name you wish to give the cluster. - - **GCP project ID** (required) - The ID of the project you created in your GCP - console that will host the Kubernetes cluster. This must **not** be confused - with the project name. Learn more about [Google Cloud Platform projects](https://cloud.google.com/resource-manager/docs/creating-managing-projects). - - **Zone** - The zone under which the cluster will be created. Read more about - [the available zones](https://cloud.google.com/compute/docs/regions-zones/). - - **Number of nodes** - The number of nodes you wish the cluster to have. - - **Machine type** - The machine type of the Virtual Machine instance that - the cluster will be based on. Read more about [the available machine types](https://cloud.google.com/compute/docs/machine-types). - - **Project namespace** - The unique namespace for this project. By default you - don't have to fill it in; by leaving it blank, GitLab will create one for you. -1. Click the **Create cluster** button. - -After a few moments your cluster should be created. If something goes wrong, +1. Navigate to your project's **CI/CD > Cluster** page +1. If you want to let GitLab create a cluster on GKE for you, go through the + following steps, otherwise skip to the next one. + 1. Click on **Create with GKE** + 1. Connect your Google account if you haven't done already by clicking the + **Sign in with Google** button + 1. Fill in the requested values: + - **Cluster name** (required) - The name you wish to give the cluster. + - **GCP project ID** (required) - The ID of the project you created in your GCP + console that will host the Kubernetes cluster. This must **not** be confused + with the project name. Learn more about [Google Cloud Platform projects](https://cloud.google.com/resource-manager/docs/creating-managing-projects). + - **Zone** - The [zone](https://cloud.google.com/compute/docs/regions-zones/) + under which the cluster will be created. + - **Number of nodes** - The number of nodes you wish the cluster to have. + - **Machine type** - The [machine type](https://cloud.google.com/compute/docs/machine-types) + of the Virtual Machine instance that the cluster will be based on. + - **Project namespace** - The unique namespace for this project. By default you + don't have to fill it in; by leaving it blank, GitLab will create one for you. +1. If you want to use your own existing Kubernetes cluster, click on + **Add an existing cluster** and fill in the details as described in the + [Kubernetes integration](../integrations/kubernetes.md) documentation. +1. Finally, click the **Create cluster** button + +After a few moments, your cluster should be created. If something goes wrong, you will be notified. -Now, you can proceed to [enable the Cluster integration](#enabling-or-disabling-the-cluster-integration). +You can now proceed to install some pre-defined applications and then +enable the Cluster integration. + +## Installing applications + +GitLab provides a one-click install for various applications which will be +added directly to your configured cluster. Those applications are needed for +[Review Apps](../../../ci/review_apps/index.md) and [deployments](../../../ci/environments.md). + +| Application | GitLab version | Description | +| ----------- | :------------: | ----------- | +| [Helm Tiller](https://docs.helm.sh/) | 10.2+ | Helm is a package manager for Kubernetes and is required to install all the other applications. It will be automatically installed as a dependency when you try to install a different app. It is installed in its own pod inside the cluster which can run the `helm` CLI in a safe environment. | +| [Ingress](https://kubernetes.io/docs/concepts/services-networking/ingress/) | 10.2+ | Ingress can provide load balancing, SSL termination and name-based virtual hosting. It acts as a web proxy for your applications and is useful if you want to use [Auto DevOps](../../../topics/autodevops/index.md) or deploy your own web apps. | ## Enabling or disabling the Cluster integration diff --git a/doc/user/project/container_registry.md b/doc/user/project/container_registry.md index 2c4dfcff4a6..394aa9209e4 100644 --- a/doc/user/project/container_registry.md +++ b/doc/user/project/container_registry.md @@ -27,7 +27,8 @@ to enable it. 1. First, ask your system administrator to enable GitLab Container Registry following the [administration documentation](../../administration/container_registry.md). If you are using GitLab.com, this is enabled by default so you can start using - the Registry immediately. + the Registry immediately. Currently there is a soft (10GB) size restriction for + registry on GitLab.com, as part of the [repository size limit](repository/index.html#repository-size). 1. Go to your [project's General settings](settings/index.md#sharing-and-permissions) and enable the **Container Registry** feature on your project. For new projects this might be enabled by default. For existing projects diff --git a/doc/user/project/img/label_priority_sort_order.png b/doc/user/project/img/label_priority_sort_order.png Binary files differnew file mode 100644 index 00000000000..21c7a76a322 --- /dev/null +++ b/doc/user/project/img/label_priority_sort_order.png diff --git a/doc/user/project/img/labels_filter_by_priority.png b/doc/user/project/img/labels_filter_by_priority.png Binary files differdeleted file mode 100644 index 419e555e709..00000000000 --- a/doc/user/project/img/labels_filter_by_priority.png +++ /dev/null diff --git a/doc/user/project/img/priority_sort_order.png b/doc/user/project/img/priority_sort_order.png Binary files differnew file mode 100644 index 00000000000..c558ec23b0e --- /dev/null +++ b/doc/user/project/img/priority_sort_order.png diff --git a/doc/user/project/import/github.md b/doc/user/project/import/github.md index 6423beefc77..72def9d1d1d 100644 --- a/doc/user/project/import/github.md +++ b/doc/user/project/import/github.md @@ -24,6 +24,8 @@ constrains of a Sidekiq worker. - the milestones (GitLab 8.7+) - the labels (GitLab 8.7+) - the release note descriptions (GitLab 8.12+) + - the pull request review comments (GitLab 10.2+) + - the regular issue and pull request comments - References to pull requests and issues are preserved (GitLab 8.7+) - Repository public access is retained. If a repository is private in GitHub it will be created as private in GitLab as well. @@ -43,10 +45,13 @@ the case the namespace is taken, the repository will be imported under the user' namespace that started the import process. The importer will also import branches on forks of projects related to open pull -requests. These branches will be imported with a naming scheume similar to +requests. These branches will be imported with a naming scheme similar to GH-SHA-Username/Pull-Request-number/fork-name/branch. This may lead to a discrepency in branches compared to the GitHub Repository. +For a more technical description and an overview of the architecture you can +refer to [Working with the GitHub importer][gh-import-dev-docs]. + ## Importing your GitHub repositories The importer page is visible when you create a new project. @@ -121,7 +126,29 @@ If you want, you can import all your GitHub projects in one go by hitting You can also choose a different name for the project and a different namespace, if you have the privileges to do so. +## Making the import process go faster + +For large projects it may take a while to import all data. To reduce the time +necessary you can increase the number of Sidekiq workers that process the +following queues: + +* `github_importer` +* `github_importer_advance_stage` + +For an optimal experience we recommend having at least 4 Sidekiq processes (each +running a number of threads equal to the number of CPU cores) that _only_ +process these queues. We also recommend that these processes run on separate +servers. For 4 servers with 8 cores this means you can import up to 32 objects +(e.g. issues) in parallel. + +Reducing the time spent in cloning a repository can be done by increasing +network throughput, CPU capacity, and disk performance (e.g. by using high +performance SSDs) of the disks that store the Git repositories (for your GitLab +instance). Increasing the number of Sidekiq workers will _not_ reduce the time +spent cloning repositories. + [gh-import]: ../../../integration/github.md "GitHub integration" [gh-rake]: ../../../administration/raketasks/github_import.md "GitHub rake task" [gh-integration]: #authorize-access-to-your-repositories-using-the-github-integration [gh-token]: #authorize-access-to-your-repositories-using-a-personal-access-token +[gh-import-dev-docs]: ../../../development/github_importer.md "Working with the GitHub importer" diff --git a/doc/user/project/integrations/custom_issue_tracker.md b/doc/user/project/integrations/custom_issue_tracker.md new file mode 100644 index 00000000000..757522c2ae3 --- /dev/null +++ b/doc/user/project/integrations/custom_issue_tracker.md @@ -0,0 +1,20 @@ +# Custom Issue Tracker Service + +To enable the Custom Issue Tracker integration in a project, navigate to the +[Integrations page](project_services.md#accessing-the-project-services), click +the **Customer Issue Tracker** service, and fill in the required details on the page as described +in the table below. + +| Field | Description | +| ----- | ----------- | +| `title` | A title for the issue tracker (to differentiate between instances, for example) | +| `description` | A name for the issue tracker (to differentiate between instances, for example) | +| `project_url` | Currently unused. Will be changed in a future release. | +| `issues_url` | The URL to the issue in the issue tracker project that is linked to this GitLab project. Note that the `issues_url` requires `:id` in the URL. This ID is used by GitLab as a placeholder to replace the issue number. For example, `https://customissuetracker.com/project-name/:id`. | +| `new_issue_url` | Currently unused. Will be changed in a future release. | + + +## Referencing issues + +Issues are referenced with `#<ID>`, where `<ID>` is a number (example `#143`). +So with the example above, `#143` would refer to `https://customissuetracker.com/project-name/143`.
\ No newline at end of file diff --git a/doc/user/project/integrations/img/webhook_logs.png b/doc/user/project/integrations/img/webhook_logs.png Binary files differindex 917068d9398..803678db6b6 100644 --- a/doc/user/project/integrations/img/webhook_logs.png +++ b/doc/user/project/integrations/img/webhook_logs.png diff --git a/doc/user/project/integrations/project_services.md b/doc/user/project/integrations/project_services.md index 51989ccaaea..a0405161495 100644 --- a/doc/user/project/integrations/project_services.md +++ b/doc/user/project/integrations/project_services.md @@ -43,6 +43,7 @@ Click on the service links to see further configuration instructions and details | [Mattermost slash commands](mattermost_slash_commands.md) | Mattermost chat and ChatOps slash commands | | [Mattermost Notifications](mattermost.md) | Receive event notifications in Mattermost | | [Microsoft teams](microsoft_teams.md) | Receive notifications for actions that happen on GitLab into a room on Microsoft Teams using Office 365 Connectors | +| Packagist | Update your project on Packagist, the main Composer repository | | Pipelines emails | Email the pipeline status to a list of recipients | | [Slack Notifications](slack.md) | Send GitLab events (e.g. issue created) to Slack as notifications | | [Slack slash commands](slack_slash_commands.md) | Use slash commands in Slack to control GitLab | diff --git a/doc/user/project/integrations/prometheus_library/kubernetes.md b/doc/user/project/integrations/prometheus_library/kubernetes.md index 518683965e8..a6673fa2a00 100644 --- a/doc/user/project/integrations/prometheus_library/kubernetes.md +++ b/doc/user/project/integrations/prometheus_library/kubernetes.md @@ -13,8 +13,8 @@ integration services must be enabled. | Name | Query | | ---- | ----- | -| Average Memory Usage (MB) | (sum(container_memory_usage_bytes{container_name!="POD",%{environment_filter}}) / count(container_memory_usage_bytes{container_name!="POD",%{environment_filter}})) /1024/1024 | -| Average CPU Utilization (%) | sum(rate(container_cpu_usage_seconds_total{container_name!="POD",%{environment_filter}}[2m])) by (cpu) * 100 | +| Average Memory Usage (MB) | (sum(avg(container_memory_usage_bytes{container_name!="POD",environment="%{ci_environment_slug}"}) without (job))) / count(avg(container_memory_usage_bytes{container_name!="POD",environment="%{ci_environment_slug}"}) without (job)) /1024/1024 | +| Average CPU Utilization (%) | sum(avg(rate(container_cpu_usage_seconds_total{container_name!="POD",environment="%{ci_environment_slug}"}[2m])) without (job)) * 100 | ## Configuring Prometheus to monitor for Kubernetes node metrics diff --git a/doc/user/project/integrations/webhooks.md b/doc/user/project/integrations/webhooks.md index 7abc600a680..5896f8f72a0 100644 --- a/doc/user/project/integrations/webhooks.md +++ b/doc/user/project/integrations/webhooks.md @@ -76,6 +76,7 @@ X-Gitlab-Event: Push Hook "user_avatar": "https://s.gravatar.com/avatar/d4c74594d841139328695756648b6bd6?s=8://s.gravatar.com/avatar/d4c74594d841139328695756648b6bd6?s=80", "project_id": 15, "project":{ + "id": 15, "name":"Diaspora", "description":"", "web_url":"http://example.com/mike/diaspora", @@ -156,6 +157,7 @@ X-Gitlab-Event: Tag Push Hook "user_avatar": "https://s.gravatar.com/avatar/d4c74594d841139328695756648b6bd6?s=8://s.gravatar.com/avatar/d4c74594d841139328695756648b6bd6?s=80", "project_id": 1, "project":{ + "id": 1, "name":"Example", "description":"", "web_url":"http://example.com/jsmith/example", @@ -206,6 +208,7 @@ X-Gitlab-Event: Issue Hook "avatar_url": "http://www.gravatar.com/avatar/e64c7d89f26bd1972efa854d13d7dd61?s=40\u0026d=identicon" }, "project": { + "id": 1, "name":"Gitlab Test", "description":"Aut reprehenderit ut est.", "web_url":"http://example.com/gitlabhq/gitlab-test", @@ -335,6 +338,7 @@ X-Gitlab-Event: Note Hook }, "project_id": 5, "project":{ + "id": 5, "name":"Gitlab Test", "description":"Aut reprehenderit ut est.", "web_url":"http://example.com/gitlabhq/gitlab-test", @@ -414,6 +418,7 @@ X-Gitlab-Event: Note Hook }, "project_id": 5, "project":{ + "id": 5, "name":"Gitlab Test", "description":"Aut reprehenderit ut est.", "web_url":"http://example.com/gitlab-org/gitlab-test", @@ -540,6 +545,7 @@ X-Gitlab-Event: Note Hook }, "project_id": 5, "project":{ + "id": 5, "name":"Gitlab Test", "description":"Aut reprehenderit ut est.", "web_url":"http://example.com/gitlab-org/gitlab-test", @@ -618,6 +624,7 @@ X-Gitlab-Event: Note Hook }, "project_id": 5, "project":{ + "id": 5, "name":"Gitlab Test", "description":"Aut reprehenderit ut est.", "web_url":"http://example.com/gitlab-org/gitlab-test", @@ -692,6 +699,7 @@ X-Gitlab-Event: Merge Request Hook "avatar_url": "http://www.gravatar.com/avatar/e64c7d89f26bd1972efa854d13d7dd61?s=40\u0026d=identicon" }, "project": { + "id": 1, "name":"Gitlab Test", "description":"Aut reprehenderit ut est.", "web_url":"http://example.com/gitlabhq/gitlab-test", @@ -848,6 +856,7 @@ X-Gitlab-Event: Wiki Page Hook "avatar_url": "http://www.gravatar.com/avatar/e64c7d89f26bd1972efa854d13d7dd61?s=80\u0026d=identicon" }, "project": { + "id": 1, "name": "awesome-project", "description": "This is awesome", "web_url": "http://example.com/root/awesome-project", @@ -919,6 +928,7 @@ X-Gitlab-Event: Pipeline Hook "avatar_url": "http://www.gravatar.com/avatar/e32bd13e2add097461cb96824b7a829c?s=80\u0026d=identicon" }, "project":{ + "id": 1, "name": "Gitlab Test", "description": "Atque in sunt eos similique dolores voluptatem.", "web_url": "http://192.168.64.1:3005/gitlab-org/gitlab-test", @@ -1130,6 +1140,18 @@ From this page, you can repeat delivery with the same data by clicking `Resend R >**Note:** If URL or secret token of the webhook were updated, data will be delivered to the new address. +### Receiving duplicate or multiple web hook requests triggered by one event + +When GitLab sends a webhook it expects a response in 10 seconds (set default value). If it does not receive one, it'll retry the webhook. +If the endpoint doesn't send its HTTP response within those 10 seconds, GitLab may decide the hook failed and retry it. + +If you are receiving multiple requests, you can try increasing the default value to wait for the HTTP response after sending the webhook +by uncommenting or adding the following setting to your `/etc/gitlab/gitlab.rb`: + +``` +gitlab_rails['webhook_timeout'] = 10 +``` + ## Example webhook receiver If you want to see GitLab's webhooks in action for testing purposes you can use diff --git a/doc/user/project/issue_board.md b/doc/user/project/issue_board.md index 96a5a23ee13..8c2690ec3b2 100644 --- a/doc/user/project/issue_board.md +++ b/doc/user/project/issue_board.md @@ -166,12 +166,26 @@ board itself.  -## Re-ordering an issue in a list - -> Introduced in GitLab 9.0. - -Issues can be re-ordered inside of lists. This is as simple as dragging and dropping -an issue into the order you want. +## Issue ordering in a list + +When visiting a board, issues appear ordered in any list. You are able to change +that order simply by dragging and dropping the issues. The changed order will be saved +to the system so that anybody who visits the same board later will see the reordering, +with some exceptions. + +The first time a given issue appears in any board (i.e. the first time a user +loads a board containing that issue), it will be ordered with +respect to other issues in that list according to [Priority order][label-priority]. +At that point, that issue will be assigned a relative order value by the system +representing its relative order with respect to the other issues in the list. Any time +you drag-and-drop reorder that issue, its relative order value will change accordingly. +Also, any time that issue appears in any board when it is loaded by a user, +the updated relative order value will be used for the ordering. (It's only the first +time an issue appears that it takes from the Priority order mentioned above.) This means that +if issue `A` is drag-and-drop reordered to be above issue `B` by any user in +a given board inside your GitLab instance, any time those two issues are subsequently +loaded in any board in the same instance (could be a different project board or a different group board, for example), +that ordering will be maintained. ## Filtering issues diff --git a/doc/user/project/labels.md b/doc/user/project/labels.md index 21a2e1213ec..d7eb4bca89c 100644 --- a/doc/user/project/labels.md +++ b/doc/user/project/labels.md @@ -77,15 +77,32 @@ having their priority set to null.  -Now that you have labels prioritized, you can use the 'Priority' and 'Label -priority' filters in the issues or merge requests tracker. +Now that you have labels prioritized, you can use the 'Label priority' and 'Priority' +sort orders in the issues or merge requests tracker. -The 'Label priority' filter puts issues with the highest priority label on top. +In the following, everything applies to both issues and merge requests, but we'll +refer to just issues for brevity. -The 'Priority' filter sorts issues by their soonest milestone due date, then by -label priority. +The 'Label priority' sort order positions issues with higher priority labels +toward the top, and issues with lower priority labels toward the bottom. A non-prioritized +label is considered to have the lowest priority. For a given issue, we _only_ consider the +highest priority label assigned to it in the comparison. ([We are discussing](https://gitlab.com/gitlab-org/gitlab-ce/issues/18554) +including all the labels in a given issue for this comparison.) Given two issues +are equal according to this sort comparison, their relative order is equal, and +therefore it's not guaranteed that one will be always above the other. + + + +The 'Priority' sort order comparison first considers an issue's milestone's due date, +(if the issue is assigned a milestone and the milestone's due date exists), and then +secondarily considers the label priority comparison above. Sooner due dates results +a higher sort order. If an issue doesn't have a milestone due date, it is equivalent to +being assigned to a milestone that has a due date in the infinite future. Given two issues +are equal according to this two-stage sort comparison, their relative order is equal, and +therefore it's not guaranteed that one will be always above the other. + + - ## Subscribe to labels diff --git a/doc/user/project/members/index.md b/doc/user/project/members/index.md index b8dd96087f1..43713855e26 100644 --- a/doc/user/project/members/index.md +++ b/doc/user/project/members/index.md @@ -21,7 +21,7 @@ want to add. --- -Select the user and the [permission level](../../user/permissions.md) +Select the user and the [permission level](../../permissions.md) that you'd like to give the user. Note that you can select more than one user.  diff --git a/doc/user/project/milestones/index.md b/doc/user/project/milestones/index.md index 876b98a4dc5..83adbd8cce2 100644 --- a/doc/user/project/milestones/index.md +++ b/doc/user/project/milestones/index.md @@ -29,7 +29,8 @@ In addition to that you will be able to filter issues or merge requests by group ## Milestone promotion -You will be able to promote a project milestone to a group milestone [in the future](https://gitlab.com/gitlab-org/gitlab-ce/issues/35833). +Project milestones can be promoted to group milestones if its project belongs to a group. When a milestone is promoted all other milestones across the group projects with the same title will be merged into it, which means all milestone's children like issues, merge requests and boards will be moved into the new promoted milestone. +The promote button can be found in the milestone view or milestones list. ## Special milestone filters diff --git a/doc/user/project/new_ci_build_permissions_model.md b/doc/user/project/new_ci_build_permissions_model.md index 271adee7da1..17dcd152363 100644 --- a/doc/user/project/new_ci_build_permissions_model.md +++ b/doc/user/project/new_ci_build_permissions_model.md @@ -230,7 +230,7 @@ test: - docker run $CI_REGISTRY/group/other-project:latest ``` -[job permissions]: ../permissions.md#jobs-permissions +[job permissions]: ../permissions.md#job-permissions [comment]: https://gitlab.com/gitlab-org/gitlab-ce/issues/22484#note_16648302 [ext]: ../permissions.md#external-users [gitsub]: ../../ci/git_submodules.md diff --git a/doc/user/project/pages/getting_started_part_one.md b/doc/user/project/pages/getting_started_part_one.md index 453e10184f0..1e19f422d94 100644 --- a/doc/user/project/pages/getting_started_part_one.md +++ b/doc/user/project/pages/getting_started_part_one.md @@ -62,7 +62,7 @@ which is highly recommendable and much faster than hardcoding. If you set up a GitLab Pages project on GitLab.com, it will automatically be accessible under a -[subdomain of `namespace.pages.io`](introduction.md#gitlab-pages-on-gitlab-com). +[subdomain of `namespace.gitlab.io`](introduction.md#gitlab-pages-on-gitlab-com). The `namespace` is defined by your username on GitLab.com, or the group name you created this project under. diff --git a/doc/user/project/pipelines/job_artifacts.md b/doc/user/project/pipelines/job_artifacts.md index 9ef6f9185c9..f9a268fb789 100644 --- a/doc/user/project/pipelines/job_artifacts.md +++ b/doc/user/project/pipelines/job_artifacts.md @@ -52,7 +52,8 @@ directly in the job artifacts browser without the need to download them. >**Note:** With [GitLab 10.1][ce-14399], HTML files in a public project can be previewed -directly in a new tab without the need to download them. +directly in a new tab without the need to download them when +[GitLab Pages](../../../administration/pages/index.md) is enabled After a job finishes, if you visit the job's specific page, there are three buttons. You can download the artifacts archive or browse its contents, whereas @@ -69,7 +70,8 @@ browse inside them. Below you can see how browsing looks like. In this case we have browsed inside the archive and at this point there is one directory, a couple files, and -one HTML file that you can view directly online (opens in a new tab). +one HTML file that you can view directly online when +[GitLab Pages](../../../administration/pages/index.md) is enabled (opens in a new tab).  diff --git a/doc/user/project/pipelines/schedules.md b/doc/user/project/pipelines/schedules.md index 9ad15a12c3c..eac706be3a7 100644 --- a/doc/user/project/pipelines/schedules.md +++ b/doc/user/project/pipelines/schedules.md @@ -44,7 +44,7 @@ GitLab CI so that they can be used in your `.gitlab-ci.yml` file. To configure that a job can be executed only when the pipeline has been scheduled (or the opposite), you can use -[only and except](../../../ci/yaml/README.md#only-and-except) configuration keywords. +[only and except](../../../ci/yaml/README.md#only-and-except-simplified) configuration keywords. ``` job:on-schedule: diff --git a/doc/user/project/pipelines/settings.md b/doc/user/project/pipelines/settings.md index 56f58fd755a..daa5463d680 100644 --- a/doc/user/project/pipelines/settings.md +++ b/doc/user/project/pipelines/settings.md @@ -115,10 +115,12 @@ pages. Depending on the status of your job, a badge can have the following values: +- pending - running -- success +- passed - failed - skipped +- canceled - unknown You can access a pipeline status badge image using the following link: diff --git a/doc/workflow/README.md b/doc/workflow/README.md index 6b2aba47f54..272f7807ac0 100644 --- a/doc/workflow/README.md +++ b/doc/workflow/README.md @@ -1,3 +1,7 @@ +--- +comments: false +--- + # Workflow - [Automatic issue closing](../user/project/issues/automatic_issue_closing.md) diff --git a/doc/workflow/gitlab_flow.md b/doc/workflow/gitlab_flow.md index 9d466ae1971..23b67310d25 100644 --- a/doc/workflow/gitlab_flow.md +++ b/doc/workflow/gitlab_flow.md @@ -1,6 +1,6 @@  -## Introduction +# Introduction to GitLab Flow Version management with git makes branching and merging much easier than older versioning systems such as SVN. This allows a wide variety of branching strategies and workflows. |
