diff options
Diffstat (limited to 'doc')
| -rw-r--r-- | doc/administration/gitaly/index.md | 2 | ||||
| -rw-r--r-- | doc/administration/reference_architectures/troubleshooting.md | 2 | ||||
| -rw-r--r-- | doc/administration/troubleshooting/debug.md | 6 | ||||
| -rw-r--r-- | doc/api/graphql/reference/index.md | 3 | ||||
| -rw-r--r-- | doc/api/merge_request_approvals.md | 29 | ||||
| -rw-r--r-- | doc/ci/docker/using_docker_build.md | 95 | ||||
| -rw-r--r-- | doc/development/architecture.md | 2 | ||||
| -rw-r--r-- | doc/development/internal_api.md | 10 | ||||
| -rw-r--r-- | doc/development/maintenance_mode.md | 2 | ||||
| -rw-r--r-- | doc/topics/autodevops/upgrading_auto_deploy_dependencies.md | 2 | ||||
| -rw-r--r-- | doc/user/clusters/agent/index.md | 75 |
11 files changed, 153 insertions, 75 deletions
diff --git a/doc/administration/gitaly/index.md b/doc/administration/gitaly/index.md index 7bb99bd1018..c442fd6cda2 100644 --- a/doc/administration/gitaly/index.md +++ b/doc/administration/gitaly/index.md @@ -1198,7 +1198,7 @@ Confirm the following are all true: successfully creates the project, but doesn't create the README. - When [tailing the logs](https://docs.gitlab.com/omnibus/settings/logs.html#tail-logs-in-a-console-on-the-server) on a Gitaly client and reproducing the error, you get `401` errors - when reaching the `/api/v4/internal/allowed` endpoint: + when reaching the [`/api/v4/internal/allowed`](../../development/internal_api.md) endpoint: ```shell # api_json.log diff --git a/doc/administration/reference_architectures/troubleshooting.md b/doc/administration/reference_architectures/troubleshooting.md index cab45a99ee4..42d9651ca91 100644 --- a/doc/administration/reference_architectures/troubleshooting.md +++ b/doc/administration/reference_architectures/troubleshooting.md @@ -301,7 +301,7 @@ Confirm the following are all true: - Creating a new project and [initializing it with a README](../../user/project/working_with_projects.md#blank-projects) successfully creates the project but doesn't create the README. - When [tailing the logs](https://docs.gitlab.com/omnibus/settings/logs.html#tail-logs-in-a-console-on-the-server) on an app node and reproducing the error, you get `401` errors - when reaching the `/api/v4/internal/allowed` endpoint: + when reaching the [`/api/v4/internal/allowed`](../../development/internal_api.md) endpoint: ```shell # api_json.log diff --git a/doc/administration/troubleshooting/debug.md b/doc/administration/troubleshooting/debug.md index a6073e34d58..5a8ee1c5c94 100644 --- a/doc/administration/troubleshooting/debug.md +++ b/doc/administration/troubleshooting/debug.md @@ -255,7 +255,7 @@ separate Rails process to debug the issue: ### GitLab: API is not accessible This often occurs when GitLab Shell attempts to request authorization via the -internal API (e.g., `http://localhost:8080/api/v4/internal/allowed`), and +[internal API](../../development/internal_api.md) (e.g., `http://localhost:8080/api/v4/internal/allowed`), and something in the check fails. There are many reasons why this may happen: 1. Timeout connecting to a database (e.g., PostgreSQL or Redis) @@ -271,8 +271,8 @@ strace -ttTfyyy -s 1024 -p <PID of unicorn worker> -o /tmp/unicorn.txt ``` If you cannot isolate which Unicorn worker is the issue, try to run `strace` -on all the Unicorn workers to see where the `/internal/allowed` endpoint gets -stuck: +on all the Unicorn workers to see where the +[`/internal/allowed`](../../development/internal_api.md) endpoint gets stuck: ```shell ps auwx | grep unicorn | awk '{ print " -p " $2}' | xargs strace -ttTfyyy -s 1024 -o /tmp/unicorn.txt diff --git a/doc/api/graphql/reference/index.md b/doc/api/graphql/reference/index.md index e329da46509..0e9750094d9 100644 --- a/doc/api/graphql/reference/index.md +++ b/doc/api/graphql/reference/index.md @@ -427,7 +427,8 @@ Describes an alert from the project's Alert Management. | `eventCount` | Int | Number of events of this alert. | | `hosts` | String! => Array | List of hosts the alert came from. | | `iid` | ID! | Internal ID of the alert. | -| `issueIid` | ID | Internal ID of the GitLab issue attached to the alert. | +| `issue` | Issue | Issue attached to the alert. | +| `issueIid` **{warning-solid}** | ID | **Deprecated:** Use issue field. Deprecated in 13.10. | | `metricsDashboardUrl` | String | URL for metrics embed for the alert. | | `monitoringTool` | String | Monitoring tool the alert came from. | | `notes` | NoteConnection! | All notes on this noteable. | diff --git a/doc/api/merge_request_approvals.md b/doc/api/merge_request_approvals.md index 86bc56b7149..bf0fed81aec 100644 --- a/doc/api/merge_request_approvals.md +++ b/doc/api/merge_request_approvals.md @@ -627,6 +627,35 @@ POST /projects/:id/external_approval_rules | `external_url` | string | yes | URL of external approval resource | | `protected_branch_ids` | array<Integer> | no | The ids of protected branches to scope the rule by | +### Delete external approval rule **(ULTIMATE)** + +You can delete an external approval rule for a project using the following endpoint: + +```plaintext +DELETE /projects/:id/external_approval_rules/:rule_id +``` + +| Attribute | Type | Required | Description | +|------------------------|----------------|----------|----------------------------------------------------| +| `rule_id` | integer | yes | The ID of an approval rule | +| `id` | integer | yes | The ID of a project | + +### Update external approval rule **(ULTIMATE)** + +You can update an existing external approval rule for a project using the following endpoint: + +```plaintext +PATCH /projects/:id/external_approval_rules/:rule_id +``` + +| Attribute | Type | Required | Description | +|------------------------|----------------|----------|----------------------------------------------------| +| `id` | integer | yes | The ID of a project | +| `rule_id` | integer | yes | The ID of an external approval rule | +| `name` | string | no | Display name of approval rule | +| `external_url` | string | no | URL of external approval resource | +| `protected_branch_ids` | array<Integer> | no | The ids of protected branches to scope the rule by | + ### Enable or disable External Project-level MR approvals **(ULTIMATE SELF)** Enable or disable External Project-level MR approvals is under development and not ready for production use. It is diff --git a/doc/ci/docker/using_docker_build.md b/doc/ci/docker/using_docker_build.md index a3ee1c7daa0..2091a80bdf2 100644 --- a/doc/ci/docker/using_docker_build.md +++ b/doc/ci/docker/using_docker_build.md @@ -7,32 +7,16 @@ type: concepts, howto # Use Docker to build Docker images -You can use GitLab CI/CD with Docker to build and test Docker images. - -For example, you might want to: - -1. Create a Docker image of your application. -1. Run tests against the image. -1. Push the image to a remote registry. -1. Use the image to deploy your application to a server. - -Or, if your application already has a `Dockerfile`, you can -use it to create and test an image: - -```shell -docker build -t my-image dockerfiles/ -docker run my-image /script/to/run/tests -docker tag my-image my-registry:5000/my-image -docker push my-registry:5000/my-image -``` +You can use GitLab CI/CD with Docker to create Docker images. +For example, you can create a Docker image of your application, +test it, and publish it to a container registry. To run Docker commands in your CI/CD jobs, you must configure GitLab Runner to support `docker` commands. ## Enable Docker commands in your CI/CD jobs -There are three ways to enable the use of `docker build` and `docker run` -during jobs, each with their own tradeoffs. You can use: +To enable Docker commands for your CI/CD jobs, you can use: - [The shell executor](#use-the-shell-executor) - [The Docker executor with the Docker image (Docker-in-Docker)](#use-the-docker-executor-with-the-docker-image-docker-in-docker) @@ -47,12 +31,9 @@ to learn more about how these runners are configured. ### Use the shell executor -One way to configure GitLab Runner for `docker` support is to use the -`shell` executor. - -After you register a runner and select the `shell` executor, -your job scripts are executed as the `gitlab-runner` user. -This user needs permission to run Docker commands. +You can include Docker commands in your CI/CD jobs if your runner is configured to +use the `shell` executor. The `gitlab-runner` user runs the Docker commands, but +needs permission to run them. 1. [Install](https://gitlab.com/gitlab-org/gitlab-runner/#installation) GitLab Runner. 1. [Register](https://docs.gitlab.com/runner/register/) a runner. @@ -100,9 +81,11 @@ Learn more about the [security of the `docker` group](https://blog.zopyx.com/on- ### Use the Docker executor with the Docker image (Docker-in-Docker) -Another way to configure GitLab Runner for `docker` support is to -register a runner with the Docker executor and use the [Docker image](https://hub.docker.com/_/docker/) -to run your job scripts. This configuration is referred to as "Docker-in-Docker." +You can use "Docker-in-Docker" to run commands in your CI/CD jobs: + +- Register a runner that uses the Docker executor. +- Use the [Docker image](https://hub.docker.com/_/docker/) provided by Docker to + run the jobs that need Docker commands. The Docker image has all of the `docker` tools installed and can run the job script in context of the image in privileged mode. @@ -111,14 +94,18 @@ The `docker-compose` command is not available in this configuration by default. To use `docker-compose` in your job scripts, follow the `docker-compose` [installation instructions](https://docs.docker.com/compose/install/). +An example project that uses this approach can be found here: <https://gitlab.com/gitlab-examples/docker>. + WARNING: When you enable `--docker-privileged`, you are effectively disabling all of the security mechanisms of containers and exposing your host to privilege -escalation which can lead to container breakout. For more information, check +escalation. Doing this can lead to container breakout. For more information, check out the official Docker documentation on [runtime privilege and Linux capabilities](https://docs.docker.com/engine/reference/run/#runtime-privilege-and-linux-capabilities). -Docker-in-Docker works well, and is the recommended configuration, but it is +#### Limitations of Docker-in-Docker + +Docker-in-Docker is the recommended configuration, but it is not without its own challenges: - When using Docker-in-Docker, each job is in a clean environment without the past @@ -144,8 +131,6 @@ not without its own challenges: - docker run -v "$MOUNT_POINT:/mnt" my-docker-image ``` -An example project using this approach can be found here: <https://gitlab.com/gitlab-examples/docker>. - In the examples below, we are using Docker images tags to specify a specific version, such as `docker:19.03.12`. If tags like `docker:stable` are used, you have no control over what version is used. This can lead to @@ -373,9 +358,8 @@ build: ### Use Docker socket binding -Another way to configure GitLab Runner for `docker` support is to -bind-mount `/var/run/docker.sock` into the -container so that Docker is available in the context of the image. +To use Docker commands in your CI/CD jobs, you can bind-mount `/var/run/docker.sock` into the +container. Docker is then available in the context of the image. NOTE: If you bind the Docker socket and you are @@ -478,13 +462,10 @@ services: > [Introduced](https://gitlab.com/gitlab-org/gitlab-runner/-/issues/27173) in GitLab Runner 13.6. -If you are an administrator of GitLab Runner and you have the `dind` -service defined for the [Docker -executor](https://docs.gitlab.com/runner/configuration/advanced-configuration.html#the-runnersdockerservices-section), -or the [Kubernetes -executor](https://docs.gitlab.com/runner/executors/kubernetes.html#using-services) -you can specify the `command` to configure the registry mirror for the -Docker daemon. +If you are a GitLab Runner administrator, you can specify the `command` to configure the registry mirror +for the Docker daemon. The `dind` service must be defined for the +[Docker](https://docs.gitlab.com/runner/configuration/advanced-configuration.html#the-runnersdockerservices-section) +or [Kubernetes executor](https://docs.gitlab.com/runner/executors/kubernetes.html#using-services). Docker: @@ -516,11 +497,10 @@ Kubernetes: ##### Docker executor inside GitLab Runner configuration -If you are an administrator of GitLab Runner and you want to use -the mirror for every `dind` service, update the +If you are a GitLab Runner administrator, you can use +the mirror for every `dind` service. Update the [configuration](https://docs.gitlab.com/runner/configuration/advanced-configuration.html) -to specify a [volume -mount](https://docs.gitlab.com/runner/configuration/advanced-configuration.html#volumes-in-the-runnersdocker-section). +to specify a [volume mount](https://docs.gitlab.com/runner/configuration/advanced-configuration.html#volumes-in-the-runnersdocker-section). For example, if you have a `/opt/docker/daemon.json` file with the following content: @@ -552,11 +532,10 @@ picked up by the `dind` service. > [Introduced](https://gitlab.com/gitlab-org/gitlab-runner/-/issues/3223) in GitLab Runner 13.6. -If you are an administrator of GitLab Runner and you want to use -the mirror for every `dind` service, update the +If you are a GitLab Runner administrator, you can use +the mirror for every `dind` service. Update the [configuration](https://docs.gitlab.com/runner/configuration/advanced-configuration.html) -to specify a [ConfigMap volume -mount](https://docs.gitlab.com/runner/executors/kubernetes.html#using-volumes). +to specify a [ConfigMap volume mount](https://docs.gitlab.com/runner/executors/kubernetes.html#using-volumes). For example, if you have a `/tmp/daemon.json` file with the following content: @@ -602,7 +581,7 @@ The configuration is picked up by the `dind` service. When you use Docker-in-Docker, the [normal authentication methods](using_docker_images.html#define-an-image-from-a-private-container-registry) -won't work because a fresh Docker daemon is started with the service. +don't work because a fresh Docker daemon is started with the service. ### Option 1: Run `docker login` @@ -634,14 +613,14 @@ empty or remove it. If you are an administrator for GitLab Runner, you can mount a file with the authentication configuration to `~/.docker/config.json`. -Then every job that the runner picks up will be authenticated already. If you +Then every job that the runner picks up is authenticated already. If you are using the official `docker:19.03.13` image, the home directory is under `/root`. If you mount the configuration file, any `docker` command that modifies the `~/.docker/config.json` (for example, `docker login`) fails, because the file is mounted as read-only. Do not change it from -read-only, because other problems will occur. +read-only, because problems occur. Here is an example of `/opt/.docker/config.json` that follows the [`DOCKER_AUTH_CONFIG`](using_docker_images.md#determining-your-docker_auth_config-data) @@ -743,8 +722,8 @@ build: When using Docker-in-Docker, Docker downloads all layers of your image every time you create a build. Recent versions of Docker (Docker 1.13 and above) can -use a pre-existing image as a cache during the `docker build` step, considerably -speeding up the build process. +use a pre-existing image as a cache during the `docker build` step. This considerably +speeds up the build process. ### How Docker caching works @@ -754,8 +733,8 @@ any changes. Change in one layer causes all subsequent layers to be recreated. You can specify a tagged image to be used as a cache source for the `docker build` command by using the `--cache-from` argument. Multiple images can be specified -as a cache source by using multiple `--cache-from` arguments. Keep in mind that -any image that's used with the `--cache-from` argument must first be pulled +as a cache source by using multiple `--cache-from` arguments. Any image that's used +with the `--cache-from` argument must first be pulled (using `docker pull`) before it can be used as a cache source. ### Using Docker caching diff --git a/doc/development/architecture.md b/doc/development/architecture.md index b577b4e93b7..7754d9f0b3c 100644 --- a/doc/development/architecture.md +++ b/doc/development/architecture.md @@ -899,7 +899,7 @@ in Rails, scheduled to run whenever an SSH key is modified by a user. instead of keys. In this case, `AuthorizedKeysCommand` is replaced with an `AuthorizedPrincipalsCommand`. This extracts a username from the certificate without using the Rails internal API, which is used instead of `key_id` in the -`/api/internal/allowed` call later. +[`/api/internal/allowed`](internal_api.md) call later. GitLab Shell also has a few operations that do not involve Gitaly, such as resetting two-factor authentication codes. These are handled in the same way, diff --git a/doc/development/internal_api.md b/doc/development/internal_api.md index 78b59940613..4710ae4606c 100644 --- a/doc/development/internal_api.md +++ b/doc/development/internal_api.md @@ -35,12 +35,12 @@ This is called by [Gitaly](https://gitlab.com/gitlab-org/gitaly) and [GitLab Shell](https://gitlab.com/gitlab-org/gitlab-shell) to check access to a repository. -When called from GitLab Shell no changes are passed and the internal -API replies with the information needed to pass the request on to -Gitaly. +- **When called from GitLab Shell**: No changes are passed, and the internal + API replies with the information needed to pass the request on to Gitaly. +- **When called from Gitaly in a `pre-receive` hook**: The changes are passed + and validated to determine if the push is allowed. -When called from Gitaly in a `pre-receive` hook the changes are passed -and those are validated to determine if the push is allowed. +Calls are limited to 50 seconds each. ```plaintext POST /internal/allowed diff --git a/doc/development/maintenance_mode.md b/doc/development/maintenance_mode.md index 6b5a6045bb9..f05a731a331 100644 --- a/doc/development/maintenance_mode.md +++ b/doc/development/maintenance_mode.md @@ -13,7 +13,7 @@ GitLab Maintenance Mode **only** blocks writes from HTTP and SSH requests at the - [the read-only database method](https://gitlab.com/gitlab-org/gitlab/blob/2425e9de50c678413ceaad6ee3bf66f42b7e228c/ee/lib/ee/gitlab/database.rb#L13), which toggles special behavior when we are not allowed to write to the database. [Search the codebase for `Gitlab::Database.read_only?`.](https://gitlab.com/search?utf8=%E2%9C%93&search=Gitlab%3A%3ADatabase.read_only%3F&group_id=9970&project_id=278964&scope=blobs&search_code=false&snippets=false&repository_ref=) - [the read-only middleware](https://gitlab.com/gitlab-org/gitlab/-/blob/master/ee/lib/ee/gitlab/middleware/read_only/controller.rb), where HTTP requests that cause database writes are blocked, unless explicitly allowed. -- [Git push access via SSH is denied](https://gitlab.com/gitlab-org/gitlab/-/blob/2425e9de50c678413ceaad6ee3bf66f42b7e228c/ee/lib/ee/gitlab/git_access.rb#L13) by returning 401 when `gitlab-shell` POSTs to `/internal/allowed` to [check if access is allowed](internal_api.md#git-authentication). +- [Git push access via SSH is denied](https://gitlab.com/gitlab-org/gitlab/-/blob/2425e9de50c678413ceaad6ee3bf66f42b7e228c/ee/lib/ee/gitlab/git_access.rb#L13) by returning 401 when `gitlab-shell` POSTs to [`/internal/allowed`](internal_api.md) to [check if access is allowed](internal_api.md#git-authentication). - [Container registry authentication service](https://gitlab.com/gitlab-org/gitlab/-/blob/2425e9de50c678413ceaad6ee3bf66f42b7e228c/ee/app/services/ee/auth/container_registry_authentication_service.rb#L12), where updates to the container registry are blocked. The database itself is not in read-only mode (except in a Geo secondary site) and can be written by sources other than the ones blocked. diff --git a/doc/topics/autodevops/upgrading_auto_deploy_dependencies.md b/doc/topics/autodevops/upgrading_auto_deploy_dependencies.md index 1fff935880c..0dabb80204a 100644 --- a/doc/topics/autodevops/upgrading_auto_deploy_dependencies.md +++ b/doc/topics/autodevops/upgrading_auto_deploy_dependencies.md @@ -5,7 +5,7 @@ info: To determine the technical writer assigned to the Stage/Group associated w type: reference --- -# Upgrading deployments for newer Auto Deploy dependencies (Auto Deploy template, auto-deploy-image and auto-deploy-app chart) +# Upgrading deployments for newer Auto Deploy dependencies [Auto Deploy](stages.md#auto-deploy) is a feature that deploys your application to a Kubernetes cluster. It consists of several dependencies: diff --git a/doc/user/clusters/agent/index.md b/doc/user/clusters/agent/index.md index 9ead60183de..34249ac8fde 100644 --- a/doc/user/clusters/agent/index.md +++ b/doc/user/clusters/agent/index.md @@ -281,7 +281,7 @@ example [`resources.yml` file](#example-resourcesyml-file) in the following ways after you install the `gitlab-kas` sub-chart, or enable `gitlab-kas` for Omnibus GitLab. When using the sub-chart, you must set `wss://kas.host.tld:443` as `kas-address`, where `host.tld` is the domain you've setup for your GitLab installation. - When using Omnibus GitLab, you must set `wss://GitLab.host.tld:443/-/kubernetes-agent` as + When using Omnibus GitLab, you must set `wss://GitLab.host.tld:443/-/kubernetes-agent/` as `kas-address`, where `GitLab.host.tld` is your GitLab hostname. - When using the sub-chart, specify the `ws` scheme (such as `ws://kas.host.tld:80`) to use an unencrypted WebSockets connection. @@ -346,7 +346,7 @@ spec: - --token-file=/config/token - --kas-address - wss://kas.host.tld:443 # change this line for the one below if using Omnibus GitLab - # - wss://gitlab.host.tld:443/-/kubernetes-agent + # - wss://gitlab.host.tld:443/-/kubernetes-agent/ volumeMounts: - name: token-volume mountPath: /config @@ -569,7 +569,7 @@ This error is shown if there are some connectivity issues between the address specified as `kas-address`, and your Agent pod. To fix it, make sure that you specified the `kas-address` correctly. -### Agent logs - ValidationError(Deployment.metadata +### Agent logs - ValidationError(Deployment.metadata) ```plaintext {"level":"info","time":"2020-10-30T08:56:54.329Z","msg":"Synced","project_id":"root/kas-manifest001","resource_key":"apps/Deployment/kas-test001/nginx-deployment","sync_result":"error validating data: [ValidationError(Deployment.metadata): unknown field \"replicas\" in io.k8s.apimachinery.pkg.apis.meta.v1.ObjectMeta, ValidationError(Deployment.metadata): unknown field \"selector\" in io.k8s.apimachinery.pkg.apis.meta.v1.ObjectMeta, ValidationError(Deployment.metadata): unknown field \"template\" in io.k8s.apimachinery.pkg.apis.meta.v1.ObjectMeta]"} @@ -603,3 +603,72 @@ issue is in progress, directly edit the deployment with the This error is shown if the version of the agent is newer that the version of KAS. To fix it, make sure that both `agentk` and KAS use the same versions. + +### Agent logs - Certificate signed by unknown authority + +```plaintext +{"level":"error","time":"2021-02-25T07:22:37.158Z","msg":"Reverse tunnel","mod_name":"reverse_tunnel","error":"Connect(): rpc error: code = Unavailable desc = connection error: desc = \"transport: Error while dialing failed to WebSocket dial: failed to send handshake request: Get \\\"https://GitLabhost.tld:443/-/kubernetes-agent/\\\": x509: certificate signed by unknown authority\""} +``` + +This error is shown if your GitLab instance is using a certificate signed by an internal CA that +is unknown to the agent. One approach to fixing it is to present the CA certificate file to the agent +via a Kubernetes `configmap` and mount the file in the agent `/etc/ssl/certs` directory from where it +will be picked up automatically. + +For example, if your internal CA certifciate is `myCA.pem`: + +```plaintext +kubectl -n gitlab-agent create configmap ca-pemstore --from-file=myCA.pem +``` + +Then in `resources.yml`: + +```plaintext + spec: + serviceAccountName: gitlab-agent + containers: + - name: agent + image: "registry.gitlab.com/gitlab-org/cluster-integration/gitlab-agent/agentk:latest" + args: + - --token-file=/config/token + - --kas-address + - wss://kas.host.tld:443 # change this line for the one below if using Omnibus GitLab + # - wss://gitlab.host.tld:443/-/kubernetes-agent + volumeMounts: + - name: token-volume + mountPath: /config + - name: ca-pemstore-volume + mountPath: /etc/ssl/certs/myCA.pem + subPath: myCA.pem + volumes: + - name: token-volume + secret: + secretName: gitlab-agent-token + - name: ca-pemstore-volume + configMap: + name: ca-pemstore + items: + - key: myCA.pem + path: myCA.pem +``` + +Alternatively, you can mount the certificate file at a different location and include it using the +`--ca-cert-file` agent parameter: + +```plaintext + containers: + - name: agent + image: "registry.gitlab.com/gitlab-org/cluster-integration/gitlab-agent/agentk:latest" + args: + - --ca-cert-file=/tmp/myCA.pem + - --token-file=/config/token + - --kas-address + - wss://kas.host.tld:443 # change this line for the one below if using Omnibus GitLab + # - wss://gitlab.host.tld:443/-/kubernetes-agent + volumeMounts: + - name: token-volume + mountPath: /config + - name: ca-pemstore-volume + mountPath: /tmp/myCA.pem + subPath: myCA.pem +``` |