diff options
Diffstat (limited to 'doc/development/documentation')
| -rw-r--r-- | doc/development/documentation/index.md | 4 | ||||
| -rw-r--r-- | doc/development/documentation/site_architecture/index.md | 145 | ||||
| -rw-r--r-- | doc/development/documentation/site_architecture/release_process.md | 241 | ||||
| -rw-r--r-- | doc/development/documentation/styleguide.md | 19 | ||||
| -rw-r--r-- | doc/development/documentation/workflow.md | 2 |
5 files changed, 374 insertions, 37 deletions
diff --git a/doc/development/documentation/index.md b/doc/development/documentation/index.md index fb0aa5130f8..7d575e9b0b1 100644 --- a/doc/development/documentation/index.md +++ b/doc/development/documentation/index.md @@ -16,7 +16,7 @@ In addition to this page, the following resources can help you craft and contrib ## Source files and rendered web locations -Documentation for GitLab, GitLab Runner, Omnibus GitLab and Charts are published to <https://docs.gitlab.com>. Documentation for GitLab is also published within the application at `/help` on the domain of the GitLab instance. +Documentation for GitLab, GitLab Runner, Omnibus GitLab and Charts is published to <https://docs.gitlab.com>. Documentation for GitLab is also published within the application at `/help` on the domain of the GitLab instance. At `/help`, only help for your current edition and version is included. Help for other versions is available at <https://docs.gitlab.com/archives/>. The source of the documentation exists within the codebase of each GitLab application in the following repository locations: @@ -67,8 +67,6 @@ This document was moved to [another location](path/to/new_doc.md). where `path/to/new_doc.md` is the relative path to the root directory `doc/`. ---- - For example, if you move `doc/workflow/lfs/lfs_administration.md` to `doc/administration/lfs.md`, then the steps would be: diff --git a/doc/development/documentation/site_architecture/index.md b/doc/development/documentation/site_architecture/index.md index f5a12e9c216..bf873995e54 100644 --- a/doc/development/documentation/site_architecture/index.md +++ b/doc/development/documentation/site_architecture/index.md @@ -4,14 +4,16 @@ description: "Learn how GitLab's documentation website is architectured." # Documentation site architecture -Learn how we build and architecture [`gitlab-docs`](https://gitlab.com/gitlab-org/gitlab-docs) -and deploy it to <https://docs.gitlab.com>. +The [`gitlab-docs`](https://gitlab.com/gitlab-org/gitlab-docs) project hosts +the repository which is used to generate the GitLab documentation website and +is deployed to <https://docs.gitlab.com>. It uses the [Nanoc](http://nanoc.ws) +static site generator. -## Repository +## Architecture While the source of the documentation content is stored in GitLab's respective product -repositories, the source that is used to build the documentation site _from that content_ -is located at <https://gitlab.com/gitlab-org/gitlab-docs>. +repositories, the source that is used to build the documentation +site _from that content_ is located at <https://gitlab.com/gitlab-org/gitlab-docs>. The following diagram illustrates the relationship between the repositories from where content is sourced, the `gitlab-docs` project, and the published output. @@ -43,8 +45,23 @@ from where content is sourced, the `gitlab-docs` project, and the published outp G --> L ``` -See the [README there](https://gitlab.com/gitlab-org/gitlab-docs/blob/master/README.md) -for detailed information. +You will not find any GitLab docs content in the `gitlab-docs` repository. +All documentation files are hosted in the respective repository of each +product, and all together are pulled to generate the docs website: + +- [GitLab](https://gitlab.com/gitlab-org/gitlab/tree/master/doc) +- [Omnibus GitLab](https://gitlab.com/gitlab-org/omnibus-gitlab/tree/master/doc) +- [GitLab Runner](https://gitlab.com/gitlab-org/gitlab-runner/tree/master/docs) +- [GitLab Chart](https://gitlab.com/charts/gitlab/tree/master/doc) + +NOTE: **Note:** +In September 2019, we [moved towards a single codebase](https://gitlab.com/gitlab-org/gitlab-ee/issues/2952), +as such the docs for CE and EE are now identical. For historical reasons and +in order not to break any existing links throughout the internet, we still +maintain the CE docs (`https://docs.gitlab.com/ce/`), although it is hidden +from the website, and is now a symlink to the EE docs. When +[Pages supports redirects](https://gitlab.com/gitlab-org/gitlab-pages/issues/24), +we will be able to remove this completely. ## Assets @@ -73,28 +90,112 @@ Read through [the global navigation documentation](global_nav.md) to understand: - How the global navigation is built. - How to add new navigation items. -## Deployment +<!-- +## Helpers -The docs site is deployed to production with GitLab Pages, and previewed in -merge requests with Review Apps. +TBA +--> -The deployment aspects will be soon transferred from the [original document](https://gitlab.com/gitlab-org/gitlab-docs/blob/master/README.md) -to this page. +## Using YAML data files -<!-- -## Repositories +The easiest way to achieve something similar to +[Jekyll's data files](https://jekyllrb.com/docs/datafiles/) in Nanoc is by +using the [`@items`](https://nanoc.ws/doc/reference/variables/#items-and-layouts) +variable. -TBA +The data file must be placed inside the `content/` directory and then it can +be referenced in an ERB template. -## Search engine +Suppose we have the `content/_data/versions.yaml` file with the content: -TBA +```yaml +versions: +- 10.6 +- 10.5 +- 10.4 +``` -## Versions +We can then loop over the `versions` array with something like: -TBA +```erb +<% @items['/_data/versions.yaml'][:versions].each do | version | %> -## Helpers +<h3><%= version %></h3> -TBA ---> +<% end &> +``` + +Note that the data file must have the `yaml` extension (not `yml`) and that +we reference the array with a symbol (`:versions`). + +## Bumping versions of CSS and Javascript + +Whenever the custom CSS and Javascript files under `content/assets/` change, +make sure to bump their version in the frontmatter. This method guarantees that +your changes will take effect by clearing the cache of previous files. + +Always use Nanoc's way of including those files, do not hardcode them in the +layouts. For example use: + +```erb +<script async type="application/javascript" src="<%= @items['/assets/javascripts/badges.*'].path %>"></script> + +<link rel="stylesheet" href="<%= @items['/assets/stylesheets/toc.*'].path %>"> +``` + +The links pointing to the files should be similar to: + +```erb +<%= @items['/path/to/assets/file.*'].path %> +``` + +Nanoc will then build and render those links correctly according with what's +defined in [`Rules`](https://gitlab.com/gitlab-org/gitlab-docs/blob/master/Rules). + +## Linking to source files + +A helper called [`edit_on_gitlab`](https://gitlab.com/gitlab-org/gitlab-docs/blob/master/lib/helpers/edit_on_gitlab.rb) can be used +to link to a page's source file. We can link to both the simple editor and the +web IDE. Here's how you can use it in a Nanoc layout: + +- Default editor: `<a href="<%= edit_on_gitlab(@item, editor: :simple) %>">Simple editor</a>` +- Web IDE: `<a href="<%= edit_on_gitlab(@item, editor: :webide) %>">Web IDE</a>` + +If you don't specify `editor:`, the simple one is used by default. + +## Algolia search engine + +The docs site uses [Algolia docsearch](https://community.algolia.com/docsearch/) +for its search function. This is how it works: + +1. GitLab is a member of the [docsearch program](https://community.algolia.com/docsearch/#join-docsearch-program), + which is the free tier of [Algolia](https://www.algolia.com/). +1. Algolia hosts a [doscsearch config](https://github.com/algolia/docsearch-configs/blob/master/configs/gitlab.json) + for the GitLab docs site, and we've worked together to refine it. +1. That [config](https://community.algolia.com/docsearch/config-file.html) is + parsed by their [crawler](https://community.algolia.com/docsearch/crawler-overview.html) + every 24h and [stores](https://community.algolia.com/docsearch/inside-the-engine.html) + the [docsearch index](https://community.algolia.com/docsearch/how-do-we-build-an-index.html) + on [Algolia's servers](https://community.algolia.com/docsearch/faq.html#where-is-my-data-hosted%3F). +1. On the docs side, we use a [docsearch layout](https://gitlab.com/gitlab-org/gitlab-docs/blob/master/layouts/docsearch.html) which + is present on pretty much every page except <https://docs.gitlab.com/search/>, + which uses its [own layout](https://gitlab.com/gitlab-org/gitlab-docs/blob/master/layouts/instantsearch.html). In those layouts, + there's a javascript snippet which initiates docsearch by using an API key + and an index name (`gitlab`) that are needed for Algolia to show the results. + +NOTE: **For GitLab employees:** +The credentials to access the Algolia dashboard are stored in 1Password. If you +want to receive weekly reports of the search usage, search the Google doc with +title "Email, Slack, and GitLab Groups and Aliases", search for `docsearch`, +and add a comment with your email to be added to the alias that gets the weekly +reports. + +## Monthly release process (versions) + +The docs website supports versions and each month we add the latest one to the list. +For more information, read about the [monthly release process](release_process.md). + +## Review Apps for documentation merge requests + +If you are contributing to GitLab docs read how to [create a Review App with each +merge request](../index.md#previewing-the-changes-live). diff --git a/doc/development/documentation/site_architecture/release_process.md b/doc/development/documentation/site_architecture/release_process.md new file mode 100644 index 00000000000..6f723531f4c --- /dev/null +++ b/doc/development/documentation/site_architecture/release_process.md @@ -0,0 +1,241 @@ +# GitLab Docs monthly release process + +The [`dockerfiles` directory](https://gitlab.com/gitlab-org/gitlab-docs/blob/master/dockerfiles/) +contains all needed Dockerfiles to build and deploy the versioned website. It +is heavily inspired by Docker's +[Dockerfile](https://github.com/docker/docker.github.io/blob/06ed03db13895bfe867761b6fc2ad40acf6026dd/Dockerfile). + +The following Dockerfiles are used. + +| Dockerfile | Docker image | Description | +| ---------- | ------------ | ----------- | +| [`Dockerfile.bootstrap`](https://gitlab.com/gitlab-org/gitlab-docs/blob/master/dockerfiles/Dockerfile.bootstrap) | `gitlab-docs:bootstrap` | Contains all the dependencies that are needed to build the website. If the gems are updated and `Gemfile{,.lock}` changes, the image must be rebuilt. | +| [`Dockerfile.builder.onbuild`](https://gitlab.com/gitlab-org/gitlab-docs/blob/master/dockerfiles/Dockerfile.builder.onbuild) | `gitlab-docs:builder-onbuild` | Base image to build the docs website. It uses `ONBUILD` to perform all steps and depends on `gitlab-docs:bootstrap`. | +| [`Dockerfile.nginx.onbuild`](https://gitlab.com/gitlab-org/gitlab-docs/blob/master/dockerfiles/Dockerfile.nginx.onbuild) | `gitlab-docs:nginx-onbuild` | Base image to use for building documentation archives. It uses `ONBUILD` to perform all required steps to copy the archive, and relies upon its parent `Dockerfile.builder.onbuild` that is invoked when building single documentation achives (see the `Dockerfile` of each branch. | +| [`Dockerfile.archives`](https://gitlab.com/gitlab-org/gitlab-docs/blob/master/dockerfiles/Dockerfile.archives) | `gitlab-docs:archives` | Contains all the versions of the website in one archive. It copies all generated HTML files from every version in one location. | + +## How to build the images + +Although build images are built automatically via GitLab CI/CD, you can build +and tag all tooling images locally: + +1. Make sure you have [Docker installed](https://docs.docker.com/install/). +1. Make sure you're on the `dockerfiles/` directory of the `gitlab-docs` repo. +1. Build the images: + + ```sh + docker build -t registry.gitlab.com/gitlab-org/gitlab-docs:bootstrap -f Dockerfile.bootstrap ../ + docker build -t registry.gitlab.com/gitlab-org/gitlab-docs:builder-onbuild -f Dockerfile.builder.onbuild ../ + docker build -t registry.gitlab.com/gitlab-org/gitlab-docs:nginx-onbuild -f Dockerfile.nginx.onbuild ../ + ``` + +For each image, there's a manual job under the `images` stage in +[`.gitlab-ci.yml`](https://gitlab.com/gitlab-org/gitlab-docs/blob/master/.gitlab-ci.yml) which can be invoked at will. + +## Monthly release process + +When a new GitLab version is released on the 22nd, we need to create the respective +single Docker image, and update some files so that the dropdown works correctly. + +### 1. Add the chart version + +Since the charts use a different version number than all the other GitLab +products, we need to add a +[version mapping](https://docs.gitlab.com/charts/installation/version_mappings.html): + +1. Check that there is a [stable branch created](https://gitlab.com/gitlab-org/charts/gitlab/-/branches) + for the new chart version. If you're unsure or can't find it, drop a line in + the `#g_delivery` channel. +1. Make sure you're on the root path of the `gitlab-docs` repo. +1. Open `content/_data/chart_versions.yaml` and add the new stable branch version using the + version mapping. Note that only the `major.minor` version is needed. +1. Create a new merge request and merge it. + +TIP: **Tip:** +It can be handy to create the future mappings since they are pretty much known. +In that case, when a new GitLab version is released, you don't have to repeat +this first step. + +### 2. Create an image for a single version + +The single docs version must be created before the release merge request, but +this needs to happen when the stable branches for all products have been created. + +1. Make sure you're on the root path of the `gitlab-docs` repo. +1. Run the raketask to create the single version: + + ```sh + ./bin/rake "release:single[12.0]" + ``` + + A new `Dockerfile.12.0` should have been created and committed to a new branch. + +1. Push the newly created branch, but **don't create a merge request**. + Once you push, the `image:docker-singe` job will create a new Docker image + tagged with the branch name you created in the first step. In the end, the + image will be uploaded in the [Container Registry](https://gitlab.com/gitlab-org/gitlab-docs/container_registry) + and it will be listed under the + [`registry` environment folder](https://gitlab.com/gitlab-org/gitlab-docs/environments/folders/registry). + +Optionally, you can test locally by building the image and running it: + +```sh +docker build -t docs:12.0 -f Dockerfile.12.0 . +docker run -it --rm -p 4000:4000 docs:12.0 +``` + +Visit `http://localhost:4000/12.0/` to see if everything works correctly. + +### 3. Create the release merge request + +Now it's time to create the monthly release merge request that adds the new +version and rotates the old one: + +1. Make sure you're on the root path of the `gitlab-docs` repo. +1. Create a branch `release-X-Y`: + + ```sh + git checkout -b release-12-0 + ``` + +1. **Rotate the online and offline versions:** + + At any given time, there are 4 browsable online versions: one pulled from + the upstream master branches (docs for GitLab.com) and the three latest + stable versions. + + Edit `content/_data/versions.yaml` and rotate the versions to reflect the + new changes: + + - `online`: The 3 latest stable versions. + - `offline`: All the previous versions offered as an offline archive. + +1. **Add the new offline version in the 404 page redirect script:** + + Since we're deprecating the oldest version each month, we need to redirect + those URLs in order not to create [404 entries](https://gitlab.com/gitlab-org/gitlab-docs/issues/221). + There's a temporary hack for now: + + 1. Edit `content/404.html`, making sure all offline versions under + `content/_data/versions.yaml` are in the Javascript snippet at the end of + the document. + +1. **Update the `:latest` and `:archives` Docker images:** + + The following two Dockerfiles need to be updated: + + 1. `dockerfiles/Dockerfile.archives` - Add the latest version at the top of + the list. + 1. `Dockerfile.master` - Rotate the versions (oldest gets removed and latest + is added at the top of the list). + +1. In the end, there should be four files in total that have changed. + Commit and push to create the merge request using the "Release" template: + + ```sh + git add content/ Dockerfile.master dockerfiles/Dockerfile.archives + git commit -m "Release 12.0" + git push origin release-12-0 + ``` + +### 4. Update the dropdown for all online versions + +The versions dropdown is in a way "hardcoded". When the site is built, it looks +at the contents of `content/_data/versions.yaml` and based on that, the dropdown +is populated. So, older branches will have different content, which means the +dropdown will be one or more releases behind. Remember that the new changes of +the dropdown are included in the unmerged `release-X-Y` branch. + +The content of `content/_data/versions.yaml` needs to change for all online +versions: + +1. Before creating the merge request, [disable the scheduled pipeline](https://gitlab.com/gitlab-org/gitlab-docs/pipeline_schedules/228/edit) + by unchecking the "Active" option. Since all steps must run in sequence, we need + to do this to avoid race conditions in the event some previous versions are + updated before the release merge request is merged. +1. Run the raketask that will create all the respective merge requests needed to + update the dropdowns and will be set to automatically be merged when their + pipelines succeed. The `release-X-Y` branch needs to be present locally, + otherwise the raketask will fail: + + ```sh + ./bin/rake release:dropdowns + ``` + +Once all are merged, proceed to the following and final step. + +TIP: **Tip:** +In case a pipeline fails, see [troubleshooting](#troubleshooting). + +### 5. Merge the release merge request + +The dropdown merge requests should have now been merged into their respective +version (stable branch), which will trigger another pipeline. At this point, +you need to only babysit the pipelines and make sure they don't fail: + +1. Check the [pipelines page](https://gitlab.com/gitlab-org/gitlab-docs/pipelines) + and make sure all stable branches have green pipelines. +1. After all the pipelines of the online versions succeed, merge the release merge request. +1. Finally, re-activate the [scheduled pipeline](https://gitlab.com/gitlab-org/gitlab-docs/pipeline_schedules/228/edit), + save it, and hit the play button to get it started. + +Once the scheduled pipeline succeeds, the docs site will be deployed with all +new versions online. + +## Update an old Docker image with new upstream docs content + +If there are any changes to any of the stable branches of the products that are +not included in the single Docker image, just +[rerun the pipeline](https://gitlab.com/gitlab-org/gitlab-docs/pipelines/new) +for the version in question. + +## Porting new website changes to old versions + +CAUTION: **Warning:** +Porting changes to older branches can have unintended effects as we're constantly +changing the backend of the website. Use only when you know what you're doing +and make sure to test locally. + +The website will keep changing and being improved. In order to consolidate +those changes to the stable branches, we'd need to pick certain changes +from time to time. + +If this is not possible or there are many changes, merge master into them: + +```sh +git branch 12.0 +git fetch origin master +git merge origin/master +``` + +## Troubleshooting + +Releasing a new version is a long process that involves many moving parts. + +### `test_internal_links_and_anchors` failing on dropdown merge requests + +When [updating the dropdown for the stable versions](#4-update-the-dropdown-for-all-online-versions), +there may be cases where some links might fail. The process of how the +dropdown MRs are created have a caveat, and that is that the tests run by +pulling the master branches of all products, instead of the respective stable +ones. + +In a real world scenario, the [Update 12.2 dropdown to match that of 12.4](https://gitlab.com/gitlab-org/gitlab-docs/merge_requests/604) +merge request failed because of the [`test_internal_links_and_anchors` test](https://gitlab.com/gitlab-org/gitlab-docs/-/jobs/328042431). + +This happened because there has been a rename of a product (`gitlab-monitor` to `gitlab-exporter`) +and the old name was still referenced in the 12.2 docs. If the respective stable +branches for 12.2 were used, this wouldn't have failed, but as we can see from +the [`compile_dev` job](https://gitlab.com/gitlab-org/gitlab-docs/-/jobs/328042427), +the `master` branches were pulled. + +To fix this, you need to [re-run the pipeline](https://gitlab.com/gitlab-org/gitlab-docs/pipelines/new) +for the `update-12-2-for-release-12-4` branch, by including the following environment variables: + +- `BRANCH_CE` set to `12-2-stable` +- `BRANCH_EE` set to `12-2-stable-ee` +- `BRANCH_OMNIBUS` set to `12-2-stable` +- `BRANCH_RUNNER` set to `12-2-stable` +- `BRANCH_CHARTS` set to `2-2-stable` + +This should make the MR pass. diff --git a/doc/development/documentation/styleguide.md b/doc/development/documentation/styleguide.md index b6ec7a858fa..e6d666473c3 100644 --- a/doc/development/documentation/styleguide.md +++ b/doc/development/documentation/styleguide.md @@ -604,9 +604,6 @@ Inside the document: - Always use a proper description for what the image is about. That way, when a browser fails to show the image, this text will be used as an alternative description. -- If there are consecutive images with little text between them, always add - three dashes (`---`) between the image and the text to create a horizontal - line for better clarity. - If a heading is placed right after an image, always add three dashes (`---`) between the image and the heading. @@ -1182,12 +1179,12 @@ Rendered example: - Prefer to use examples using the personal access token and don't pass data of username and password. -| Methods | Description | -|:-------------------------------------------|:------------------------------------------------------| -| `-H "PRIVATE-TOKEN: <your_access_token>"` | Use this method as is, whenever authentication needed | -| `-X POST` | Use this method when creating new objects | -| `-X PUT` | Use this method when updating existing objects | -| `-X DELETE` | Use this method when removing existing objects | +| Methods | Description | +|:------------------------------------------- |:------------------------------------------------------| +| `--header "PRIVATE-TOKEN: <your_access_token>"` | Use this method as is, whenever authentication needed | +| `--request POST` | Use this method when creating new objects | +| `--request PUT` | Use this method when updating existing objects | +| `--request DELETE` | Use this method when removing existing objects | ### cURL Examples @@ -1209,9 +1206,9 @@ Create a new project under the authenticated user's namespace: curl --request POST --header "PRIVATE-TOKEN: <your_access_token>" "https://gitlab.example.com/api/v4/projects?name=foo" ``` -#### Post data using cURL's --data +#### Post data using cURL's `--data` -Instead of using `-X POST` and appending the parameters to the URI, you can use +Instead of using `--request POST` and appending the parameters to the URI, you can use cURL's `--data` option. The example below will create a new project `foo` under the authenticated user's namespace. diff --git a/doc/development/documentation/workflow.md b/doc/development/documentation/workflow.md index 24399391b1a..c373b976453 100644 --- a/doc/development/documentation/workflow.md +++ b/doc/development/documentation/workflow.md @@ -208,7 +208,7 @@ code reviewer have ensured: Documentation [is required](../contributing/merge_request_workflow.html#definition-of-done) for a milestone when: -- A new or enhanced feature is shipped that impacts the user of administrator experience. +- A new or enhanced feature is shipped that impacts the user or administrator experience. - There are changes to the UI or API. - A process, workflow, or previously documented feature is changed. - A feature is deprecated or removed. |