diff options
author | GitLab Bot <gitlab-bot@gitlab.com> | 2020-06-18 14:18:50 +0300 |
---|---|---|
committer | GitLab Bot <gitlab-bot@gitlab.com> | 2020-06-18 14:18:50 +0300 |
commit | 8c7f4e9d5f36cff46365a7f8c4b9c21578c1e781 (patch) | |
tree | a77e7fe7a93de11213032ed4ab1f33a3db51b738 /doc/topics/offline | |
parent | 00b35af3db1abfe813a778f643dad221aad51fca (diff) |
Add latest changes from gitlab-org/gitlab@13-1-stable-ee
Diffstat (limited to 'doc/topics/offline')
-rw-r--r-- | doc/topics/offline/index.md | 138 | ||||
-rw-r--r-- | doc/topics/offline/quick_start_guide.md | 157 |
2 files changed, 295 insertions, 0 deletions
diff --git a/doc/topics/offline/index.md b/doc/topics/offline/index.md new file mode 100644 index 00000000000..6d4c486d350 --- /dev/null +++ b/doc/topics/offline/index.md @@ -0,0 +1,138 @@ +# Offline GitLab + +Computers in an offline environment are isolated from the public internet as a security measure. This +page lists all the information available for running GitLab in an offline environment. + +## Quick start + +If you plan to deploy a GitLab instance on a physically-isolated and offline network, see the +[quick start guide](quick_start_guide.md) for configuration steps. + +## Features + +Follow these best practices to use GitLab's features in an offline environment: + +- [Operating the GitLab Secure scanners in an offline environment](../../user/application_security/offline_deployments/index.md). + +## Loading Docker images onto your offline host + +To use many GitLab features, including +[security scans](../../user/application_security/index.md#working-in-an-offline-environment) +and [Auto DevOps](../autodevops/), the GitLab Runner must be able to fetch the +relevant Docker images. + +The process for making these images available without direct access to the public internet +involves downloading the images then packaging and transferring them to the offline host. Here's an +example of such a transfer: + +1. Download Docker images from public internet. +1. Package Docker images as tar archives. +1. Transfer images to offline environment. +1. Load transferred images into offline Docker registry. + +### Using the official GitLab template + +GitLab provides a [vendored template](../../ci/yaml/README.md#includetemplate) +to ease this process. + +This template should be used in a new, empty project, with a `gitlab-ci.yml` file containing: + +```yaml +include: + - template: Secure-Binaries.gitlab-ci.yml +``` + +The pipeline downloads the Docker images needed for the Security Scanners and saves them as +[job artifacts](../../ci/pipelines/job_artifacts.md) or pushes them to the [Container Registry](../../user/packages/container_registry/index.md) +of the project where the pipeline is executed. These archives can be transferred to another location +and [loaded](https://docs.docker.com/engine/reference/commandline/load/) in a Docker daemon. +This method requires a GitLab Runner with access to both `gitlab.com` (including +`registry.gitlab.com`) and the local offline instance. This runner must run in +[privileged mode](https://docs.gitlab.com/runner/executors/docker.html#use-docker-in-docker-with-privileged-mode) +to be able to use the `docker` command inside the jobs. This runner can be installed in a DMZ or on +a bastion, and used only for this specific project. + +#### Scheduling the updates + +By default, this project's pipeline will run only once, when the `.gitlab-ci.yml` is added to the +repo. To update the GitLab security scanners and signatures, it's necessary to run this pipeline +regularly. GitLab provides a way to [schedule pipelines](../../ci/pipelines/schedules.md). For +example, you can set this up to download and store the Docker images every week. + +Some images can be updated more frequently than others. For example, the [vulnerability database](https://hub.docker.com/r/arminc/clair-db/tags) +for Container Scanning is updated daily. To update this single image, create a new Scheduled +Pipeline that runs daily and set `SECURE_BINARIES_ANALYZERS` to `clair-vulnerabilities-db`. Only +this job will be triggered, and the image will be updated daily and made available in the project +registry. + +#### Using the secure bundle created + +The project using the `Secure-Binaries.gitlab-ci.yml` template should now host all the required +images and resources needed to run GitLab Security features. + +Next, you must tell the offline instance to use these resources instead of the default ones on +GitLab.com. To do so, set the environment variable `SECURE_ANALYZERS_PREFIX` with the URL of the +project [container registry](../../user/packages/container_registry/index.md). + +You can set this variable in the projects' `.gitlab-ci.yml`, or +in the GitLab UI at the project or group level. See the [GitLab CI/CD environment variables page](../../ci/variables/README.md#custom-environment-variables) +for more information. + +#### Variables + +The following table shows which variables you can use with the `Secure-Binaries.gitlab-ci.yml` +template: + +| VARIABLE | Description | Default value | +|-------------------------------------------|-----------------------------------------------|-----------------------------------| +| `SECURE_BINARIES_ANALYZERS` | Comma-separated list of analyzers to download | `"bandit, brakeman, gosec, and so on..."` | +| `SECURE_BINARIES_DOWNLOAD_IMAGES` | Used to disable jobs | `"true"` | +| `SECURE_BINARIES_PUSH_IMAGES` | Push files to the project registry | `"true"` | +| `SECURE_BINARIES_SAVE_ARTIFACTS` | Also save image archives as artifacts | `"false"` | +| `SECURE_BINARIES_ANALYZER_VERSION` | Default analyzer version (Docker tag) | `"2"` | + +### Alternate way without the official template + +If it's not possible to follow the above method, the images can be transferred manually instead: + +#### Example image packager script + +```shell +#!/bin/bash +set -ux + +# Specify needed analyzer images +analyzers=${SAST_ANALYZERS:-"bandit eslint gosec"} +gitlab=registry.gitlab.com/gitlab-org/security-products/analyzers/ + +for i in "${analyzers[@]}" +do + tarname="${i}_2.tar" + docker pull $gitlab$i:2 + docker save $gitlab$i:2 -o ./analyzers/${tarname} + chmod +r ./analyzers/${tarname} +done +``` + +#### Example image loader script + +This example loads the images from a bastion host to an offline host. In certain configurations, +physical media may be needed for such a transfer: + +```shell +#!/bin/bash +set -ux + +# Specify needed analyzer images +analyzers=${SAST_ANALYZERS:-"bandit eslint gosec"} +registry=$GITLAB_HOST:4567 + +for i in "${analyzers[@]}" +do + tarname="${i}_2.tar" + scp ./analyzers/${tarname} ${GITLAB_HOST}:~/${tarname} + ssh $GITLAB_HOST "sudo docker load -i ${tarname}" + ssh $GITLAB_HOST "sudo docker tag $(sudo docker images | grep $i | awk '{print $3}') ${registry}/analyzers/${i}:2" + ssh $GITLAB_HOST "sudo docker push ${registry}/analyzers/${i}:2" +done +``` diff --git a/doc/topics/offline/quick_start_guide.md b/doc/topics/offline/quick_start_guide.md new file mode 100644 index 00000000000..0abdd08ffcf --- /dev/null +++ b/doc/topics/offline/quick_start_guide.md @@ -0,0 +1,157 @@ +# Getting started with an offline GitLab Installation + +This is a step-by-step guide that helps you install, configure, and use a self-managed GitLab +instance entirely offline. + +## Installation + +NOTE: **Note:** +This guide assumes the server is Ubuntu 18.04. Instructions for other servers may vary. + +NOTE: **Note:** +This guide assumes the server host resolves as `my-host`, which you should replace with your +server's name. + +Follow the installation instructions [as outlined in the omnibus install +guide](https://about.gitlab.com/install/#ubuntu), but make sure to specify an `http` +URL for the `EXTERNAL_URL` installation step. Once installed, we will manually +configure the SSL ourselves. + +It is strongly recommended to setup a domain for IP resolution rather than bind +to the server's IP address. This better ensures a stable target for our certs' CN +and will make long-term resolution simpler. + +```shell +sudo EXTERNAL_URL="http://my-host.internal" install gitlab-ee +``` + +## Enabling SSL + +Follow these steps to enable SSL for your fresh instance. Note that these steps reflect those for +[manually configuring SSL in Omnibus's NGINX configuration](https://docs.gitlab.com/omnibus/settings/nginx.html#manually-configuring-https): + +1. Make the following changes to `/etc/gitlab/gitlab.rb`: + + ```ruby + # Update external_url from "http" to "https" + external_url "https://example.gitlab.com" + + # Set Let's Encrypt to false + letsencrypt['enable'] = false + ``` + +1. Create the following directories with the appropriate permissions for generating self-signed + certificates: + + ```shell + sudo mkdir -p /etc/gitlab/ssl + sudo chmod 755 /etc/gitlab/ssl + sudo openssl req -x509 -nodes -days 365 -newkey rsa:2048 -keyout /etc/gitlab/ssl/my-host.internal.key -out /etc/gitlab/ssl/my-host.internal.crt + ``` + +1. Reconfigure your instance to apply the changes: + + ```shell + sudo gitlab-ctl reconfigure + ``` + +## Enabling the GitLab Container Registry + +Follow these steps to enable the container registry. Note that these steps reflect those for +[configuring the container registry under an existing domain](../../administration/packages/container_registry.md#configure-container-registry-under-an-existing-gitlab-domain): + +1. Make the following changes to `/etc/gitlab/gitlab.rb`: + + ```ruby + # Change external_registry_url to match external_url, but append the port 4567 + external_url "https://example.gitlab.com" + registry_external_url "https://example.gitlab.com:4567" + ``` + +1. Reconfigure your instance to apply the changes: + + ```shell + sudo gitlab-ctl reconfigure + ``` + +## Allow the Docker daemon to trust the registry and GitLab Runner + +Provide your Docker daemon with your certs by +[following the steps for using trusted certificates with your registry](../../administration/packages/container_registry.md#using-self-signed-certificates-with-container-registry): + +```shell +sudo mkdir -p /etc/docker/certs.d/my-host.internal:5000 + +sudo cp /etc/gitlab/ssl/my-host.internal.crt /etc/docker/certs.d/my-host.internal:5000/ca.crt +``` + +Provide your GitLab Runner (to be installed next) with your certs by +[following the steps for using trusted certificates with your Runner](https://docs.gitlab.com/runner/install/docker.html#installing-trusted-ssl-server-certificates): + +```shell +sudo mkdir -p /etc/gitlab-runner/certs + +sudo cp /etc/gitlab/ssl/my-host.internal.crt /etc/gitlab-runner/certs/ca.crt +``` + +## Enabling GitLab Runner + +[Following a similar process to the steps for installing our GitLab Runner as a +Docker service](https://docs.gitlab.com/runner/install/docker.html#docker-image-installation), we must first register our Runner: + +```shell +$ sudo docker run --rm -it -v /etc/gitlab-runner:/etc/gitlab-runner gitlab/gitlab-runner register +Updating CA certificates... +Runtime platform arch=amd64 os=linux pid=7 revision=1b659122 version=12.8.0 +Running in system-mode. + +Please enter the gitlab-ci coordinator URL (e.g. https://gitlab.com/): +https://my-host.internal +Please enter the gitlab-ci token for this runner: +XXXXXXXXXXX +Please enter the gitlab-ci description for this runner: +[eb18856e13c0]: +Please enter the gitlab-ci tags for this runner (comma separated): + +Registering runner... succeeded runner=FSMwkvLZ +Please enter the executor: custom, docker, virtualbox, kubernetes, docker+machine, docker-ssh+machine, docker-ssh, parallels, shell, ssh: +docker +Please enter the default Docker image (e.g. ruby:2.6): +ruby:2.6 +Runner registered successfully. Feel free to start it, but if it's running already the config should be automatically reloaded! +``` + +Now we must add some additional configuration to our runner: + +Make the following changes to `/etc/gitlab-runner/config.toml`: + +- Add Docker socket to volumes `volumes = ["/var/run/docker.sock:/var/run/docker.sock", "/cache"]` +- Add `pull_policy = "if-not-present"` to the executor configuration + +Now we can start our Runner: + +```shell +sudo docker run -d --restart always --name gitlab-runner -v /etc/gitlab-runner:/etc/gitlab-runner -v /var/run/docker.sock:/var/run/docker.sock gitlab/gitlab-runner:latest +90646b6587127906a4ee3f2e51454c6e1f10f26fc7a0b03d9928d8d0d5897b64 +``` + +### Authenticating the registry against the host OS + +As noted in [Docker's registry authentication documentation](https://docs.docker.com/registry/insecure/#docker-still-complains-about-the-certificate-when-using-authentication), +certain versions of Docker require trusting the certificate chain at the OS level. + +In the case of Ubuntu, this involves using `update-ca-certificates`: + +```shell +sudo cp /etc/docker/certs.d/my-host.internal\:5000/ca.crt /usr/local/share/ca-certificates/my-host.internal.crt + +sudo update-ca-certificates +``` + +If all goes well, this is what you should see: + +```plaintext +1 added, 0 removed; done. +Running hooks in /etc/ca-certificates/update.d... +done. +``` |