diff options
author | GitLab Bot <gitlab-bot@gitlab.com> | 2022-03-18 23:02:30 +0300 |
---|---|---|
committer | GitLab Bot <gitlab-bot@gitlab.com> | 2022-03-18 23:02:30 +0300 |
commit | 41fe97390ceddf945f3d967b8fdb3de4c66b7dea (patch) | |
tree | 9c8d89a8624828992f06d892cd2f43818ff5dcc8 /doc/user/clusters/agent | |
parent | 0804d2dc31052fb45a1efecedc8e06ce9bc32862 (diff) |
Add latest changes from gitlab-org/gitlab@14-9-stable-eev14.9.0-rc42
Diffstat (limited to 'doc/user/clusters/agent')
-rw-r--r-- | doc/user/clusters/agent/ci_cd_tunnel.md | 264 | ||||
-rw-r--r-- | doc/user/clusters/agent/gitops.md | 140 | ||||
-rw-r--r-- | doc/user/clusters/agent/gitops/secrets_management.md | 61 | ||||
-rw-r--r-- | doc/user/clusters/agent/index.md | 200 | ||||
-rw-r--r-- | doc/user/clusters/agent/install/index.md | 276 | ||||
-rw-r--r-- | doc/user/clusters/agent/repository.md | 539 | ||||
-rw-r--r-- | doc/user/clusters/agent/runner.md | 9 | ||||
-rw-r--r-- | doc/user/clusters/agent/troubleshooting.md | 13 | ||||
-rw-r--r-- | doc/user/clusters/agent/vulnerabilities.md | 113 |
9 files changed, 837 insertions, 778 deletions
diff --git a/doc/user/clusters/agent/ci_cd_tunnel.md b/doc/user/clusters/agent/ci_cd_tunnel.md index 5fe772d9686..73a8470e025 100644 --- a/doc/user/clusters/agent/ci_cd_tunnel.md +++ b/doc/user/clusters/agent/ci_cd_tunnel.md @@ -4,60 +4,254 @@ group: Configure info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/engineering/ux/technical-writing/#assignments --- -# CI/CD Tunnel **(FREE)** +# Using a GitLab CI/CD workflow for Kubernetes **(FREE)** > - [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/327409) in GitLab 14.1. > - The pre-configured `KUBECONFIG` was [introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/324275) in GitLab 14.2. > - The ability to authorize groups was [introduced](https://gitlab.com/groups/gitlab-org/-/epics/5784) in GitLab 14.3. > - [Moved](https://gitlab.com/groups/gitlab-org/-/epics/6290) to GitLab Free in 14.5. > - Support for Omnibus installations was [introduced](https://gitlab.com/gitlab-org/omnibus-gitlab/-/merge_requests/5686) in GitLab 14.5. +> - The ability to switch between certificate-based clusters and agents was [introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/335089) in GitLab 14.9. The certificate-based cluster context is always called `gitlab-deploy`. -To use GitLab CI/CD to safely deploy your application to a cluster, you can use the CI/CD Tunnel. +You can use a GitLab CI/CD workflow to safely deploy to and update your Kubernetes clusters. -You can authorize multiple projects to access the same cluster, so you -can keep your application's codebase in one repository and configure -your cluster in another. This method is scalable and can save you resources. +To do so, you must first [install an agent in your cluster](install/index.md). When done, you have a Kubernetes context and can +run Kubernetes API commands in your GitLab CI/CD pipeline. -To ensure access to your cluster is safe, only the projects you -authorize can access your Agent through the CI/CD Tunnel. +To ensure access to your cluster is safe: -## Prerequisites +- Each agent has a separate context (`kubecontext`). +- Only the project where the agent is configured, and any additional projects you authorize, can access the agent in your cluster. -To use the CI/CD Tunnel, you need an existing Kubernetes cluster connected to GitLab through the -[GitLab Agent](install/index.md#install-the-agent-onto-the-cluster). +You do not need to have a runner in the cluster with the agent. -To run your CI/CD jobs using the CI/CD Tunnel, you do not need to have a runner in the same cluster. +## GitLab CI/CD workflow steps -## How the CI/CD Tunnel works +To update a Kubernetes cluster by using GitLab CI/CD, complete the following steps. -When you authorize a project to use an Agent, the Tunnel automatically -injects a `KUBECONFIG` variable into its CI/CD jobs. This way, you can -run `kubectl` commands from GitLab CI/CD scripts that belong to the -authorized project. +1. Ensure you have a working Kubernetes cluster and the manifests are in a GitLab project. +1. In the same GitLab project, [register and install the GitLab agent](install/index.md). +1. [Update your `.gitlab-ci.yml` file](#update-your-gitlab-ciyml-file-to-run-kubectl-commands) to + select the agent's Kubernetes context and run the Kubernetes API commands. +1. Run your pipeline to deploy to or update the cluster. -When you authorize a group, all the projects that belong to that group -become authorized to access the selected Agent. +If you have multiple GitLab projects that contain Kubernetes manifests: -An Agent can only authorize projects or groups in the same group -hierarchy as the Agent's configuration project. You can authorize -up to 100 projects and 100 groups per Agent. +1. [Install the GitLab agent](install/index.md) in its own project, or in one of the + GitLab projects where you keep Kubernetes manifests. +1. [Authorize the agent](#authorize-the-agent) to access your GitLab projects. +1. Optional. For added security, [use impersonation](#use-impersonation-to-restrict-project-and-group-access). +1. [Update your `.gitlab-ci.yml` file](#update-your-gitlab-ciyml-file-to-run-kubectl-commands) to + select the agent's Kubernetes context and run the Kubernetes API commands. +1. Run your pipeline to deploy to or update the cluster. -Also, each Agent has a separate context (`kubecontext`). -The Tunnel uses this information to safely allow access to the cluster from -jobs running in the projects you authorized. +## Authorize the agent -### `~/.kube/cache` permissions - -`kubectl` and other tools based on the same libraries (such as Helm, `kpt`, and `kustomize`) cache information about +You must authorize the agent to access the project where you keep your Kubernetes manifests. +You can authorize the agent to access individual projects, or authorize a group or subgroup, +so all projects within have access. For added security, you can also +[use impersonation](#use-impersonation-to-restrict-project-and-group-access). + +### Authorize the agent to access your projects + +> [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/327850) in GitLab 14.4. + +To authorize the agent to access the GitLab project where you keep Kubernetes manifests: + +1. On the top bar, select **Menu > Projects** and find the project that contains the agent configuration file (`config.yaml`). +1. Edit the file. Under the `ci_access` keyword, add the `projects` attribute. +1. For the `id`, add the path: + + ```yaml + ci_access: + projects: + - id: path/to/project + ``` + + - The Kubernetes projects must be in the same group hierarchy as the project where the agent's configuration is. + - You can authorize up to 100 projects. + +All CI/CD jobs now include a `KUBECONFIG` with contexts for every shared agent connection. +Choose the context to run `kubectl` commands from your CI/CD scripts. + +### Authorize the agent to access projects in your groups + +> [Introduced](https://gitlab.com/groups/gitlab-org/-/epics/5784) in GitLab 14.3. + +To authorize the agent to access all of the GitLab projects in a group or subgroup: + +1. On the top bar, select **Menu > Projects** and find the project that contains the agent configuration file (`config.yaml`). +1. Edit the file. Under the `ci_access` keyword, add the `groups` attribute. +1. For the `id`, add the path: + + ```yaml + ci_access: + groups: + - id: path/to/group/subgroup + ``` + + - The Kubernetes projects must be in the same group hierarchy as the project where the agent's configuration is. + - You can authorize up to 100 groups. + +All the projects that belong to the group are now authorized to access the agent. +All CI/CD jobs now include a `KUBECONFIG` with contexts for every shared agent connection. +Choose the context to run `kubectl` commands from your CI/CD scripts. + +## Update your `.gitlab-ci.yml` file to run `kubectl` commands + +In the project where you want to run Kubernetes commands, edit your project's `.gitlab-ci.yml` file. + +In the first command under the `script` keyword, set your agent's context. +Use the format `path/to/agent/repository:agent-name`. For example: + +```yaml + deploy: + image: + name: bitnami/kubectl:latest + entrypoint: [""] + script: + - kubectl config get-contexts + - kubectl config use-context path/to/agent/repository:agent-name + - kubectl get pods +``` + +If you are not sure what your agent's context is, open a terminal and connect to your cluster. +Run `kubectl config get-contexts`. + +### Environments with both certificate-based and agent-based connections + +When you deploy to an environment that has both a [certificate-based +cluster](../../infrastructure/clusters/index.md) (deprecated) and an agent connection: + +- The certificate-based cluster's context is called `gitlab-deploy`. This context + is always selected by default. +- In GitLab 14.9 and later, agent contexts are included in the + `KUBECONFIG`. You can select them by using `kubectl config use-context + path/to/agent/repository:agent-name`. +- In GitLab 14.8 and earlier, you can still use agent connections, but for environments that + already have a certificate-based cluster, the agent connections are not included in the `KUBECONFIG`. + +To use an agent connection when certificate-based connections are present, you can manually configure a new `kubectl` +configuration context. For example: + + ```yaml + deploy: + variables: + KUBE_CONTEXT: my-context # The name to use for the new context + AGENT_ID: 1234 # replace with your agent's numeric ID + K8S_PROXY_URL: wss://kas.gitlab.com/k8s-proxy/ # replace with your agent server (KAS) Kubernetes proxy URL + # ... any other variables you have configured + before_script: + - kubectl config set-credentials agent:$AGENT_ID --token="ci:${AGENT_ID}:${CI_JOB_TOKEN}" + - kubectl config set-cluster gitlab --server="${K8S_PROXY_URL}" + - kubectl config set-context "$KUBE_CONTEXT" --cluster=gitlab --user="agent:${AGENT_ID}" + - kubectl config use-context "$KUBE_CONTEXT" + # ... rest of your job configuration + ``` + +## Use impersonation to restrict project and group access **(PREMIUM)** + +> [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/345014) in GitLab 14.5. + +By default, your CI/CD job inherits all the permissions from the service account used to install the +agent in the cluster. +To restrict access to your cluster, you can use [impersonation](https://kubernetes.io/docs/reference/access-authn-authz/authentication/#user-impersonation). + +To specify impersonations, use the `access_as` attribute in your agent configuration file and use Kubernetes RBAC rules to manage impersonated account permissions. + +You can impersonate: + +- The agent itself (default). +- The CI/CD job that accesses the cluster. +- A specific user or system account defined within the cluster. + +### Impersonate the agent + +The agent is impersonated by default. You don't need to do anything to impersonate it. + +### Impersonate the CI/CD job that accesses the cluster + +To impersonate the CI/CD job that accesses the cluster, under the `access_as` key, add the `ci_job: {}` key-value. + +When the agent makes the request to the actual Kubernetes API, it sets the +impersonation credentials in the following way: + +- `UserName` is set to `gitlab:ci_job:<job id>`. Example: `gitlab:ci_job:1074499489`. +- `Groups` is set to: + - `gitlab:ci_job` to identify all requests coming from CI jobs. + - The list of IDs of groups the project is in. + - The project ID. + - The slug of the environment this job belongs to. + + Example: for a CI job in `group1/group1-1/project1` where: + + - Group `group1` has ID 23. + - Group `group1/group1-1` has ID 25. + - Project `group1/group1-1/project1` has ID 150. + - Job running in a prod environment. + + Group list would be `[gitlab:ci_job, gitlab:group:23, gitlab:group:25, gitlab:project:150, gitlab:project_env:150:prod]`. + +- `Extra` carries extra information about the request. The following properties are set on the impersonated identity: + +| Property | Description | +| -------- | ----------- | +| `agent.gitlab.com/id` | Contains the agent ID. | +| `agent.gitlab.com/config_project_id` | Contains the agent's configuration project ID. | +| `agent.gitlab.com/project_id` | Contains the CI project ID. | +| `agent.gitlab.com/ci_pipeline_id` | Contains the CI pipeline ID. | +| `agent.gitlab.com/ci_job_id` | Contains the CI job ID. | +| `agent.gitlab.com/username` | Contains the username of the user the CI job is running as. | +| `agent.gitlab.com/environment_slug` | Contains the slug of the environment. Only set if running in an environment. | + +Example to restrict access by the CI/CD job's identity: + +```yaml +ci_access: + projects: + - id: path/to/project + access_as: + ci_job: {} +``` + +### Impersonate a static identity + +For a given connection, you can use a static identity for the impersonation. + +Under the `access_as` key, add the `impersonate` key to make the request using the provided identity. + +The identity can be specified with the following keys: + +- `username` (required) +- `uid` +- `groups` +- `extra` + +See the [official Kubernetes documentation for details](https://kubernetes.io/docs/reference/access-authn-authz/authentication/#user-impersonation). + +## Troubleshooting + +### `kubectl` commands not supported + +The commands `kubectl exec`, `kubectl cp`, and `kubectl attach` are not supported. +Anything that uses these API endpoints does not work, because they use the deprecated +SPDY protocol. +[An issue exists](https://gitlab.com/gitlab-org/gitlab/-/issues/346248) to add support for these commands. + +### Grant write permissions to `~/.kube/cache` + +Tools like `kubectl`, Helm, `kpt`, and `kustomize` cache information about the cluster in `~/.kube/cache`. If this directory is not writable, the tool fetches information on each invocation, -making interactions slower and creating unnecessary load on the cluster. Make sure that this directory in the container image -you use is writable for the best experience. +making interactions slower and creating unnecessary load on the cluster. For the best experience, in the +image you use in your .`gitlab-ci.yml` file, ensure this directory is writable. + +### Enable TLS -## Configure the CI/CD Tunnel +If you are on a self-managed GitLab instance, ensure your instance is configured with Transport Layer Security (TLS). -The CI/CD Tunnel is configured directly through the -Agent's configuration file ([`config.yaml`](repository.md)) to: +If you attempt to use `kubectl` without TLS, you might get an error like: -- Authorize [projects](repository.md#authorize-projects-to-use-an-agent) and [groups](repository.md#authorize-groups-to-use-an-agent) to use the same Agent. -- [Run `kubectl` commands using the CI/CD Tunnel](repository.md#run-kubectl-commands-using-the-cicd-tunnel). -- [Restrict access of authorized projects and groups through impersonation strategies](repository.md#use-impersonation-to-restrict-project-and-group-access). +```shell +$ kubectl get pods +error: You must be logged in to the server (the server has asked for the client to provide credentials) +``` diff --git a/doc/user/clusters/agent/gitops.md b/doc/user/clusters/agent/gitops.md new file mode 100644 index 00000000000..8f0e2255121 --- /dev/null +++ b/doc/user/clusters/agent/gitops.md @@ -0,0 +1,140 @@ +--- +stage: Configure +group: Configure +info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/engineering/ux/technical-writing/#assignments +--- + +# Using a GitOps workflow for Kubernetes **(PREMIUM)** + +> [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/259669) in GitLab 13.7. + +With GitOps, you can manage containerized clusters and applications from a Git repository that: + +- Is the single source of truth of your system. +- Is the single place where you operate your system. + +By combining GitLab, Kubernetes, and GitOps, you can have: + +- GitLab as the GitOps operator. +- Kubernetes as the automation and convergence system. +- GitLab CI/CD for Continuous Integration and the agent for Continuous Deployment. + +This diagram shows the repositories and main actors in a GitOps deployment: + +```mermaid +sequenceDiagram + participant D as Developer + participant A as Application code repository + participant M as Manifest repository + participant K as GitLab agent + participant C as Agent configuration repository + loop Regularly + K-->>C: Grab the configuration + end + D->>+A: Pushing code changes + A->>M: Updating manifest + loop Regularly + K-->>M: Watching changes + M-->>K: Pulling and applying changes + end +``` + +For details, view the [architecture documentation](https://gitlab.com/gitlab-org/cluster-integration/gitlab-agent/-/blob/master/doc/architecture.md#high-level-architecture). + +## GitOps workflow steps + +To update a Kubernetes cluster by using GitOps, complete the following steps. + +1. Ensure you have a working Kubernetes cluster, and that the manifests are in a GitLab project. +1. In the same project, [register and install the GitLab agent](install/index.md). +1. Configure the agent configuration file so that the agent monitors the project for changes to the Kubernetes manifests. + Use the [GitOps configuration reference](#gitops-configuration-reference) for guidance. + +Any time you commit updates to your Kubernetes manifests, the agent updates the cluster. + +## GitOps configuration reference + +The following snippet shows an example of the possible keys and values for the GitOps section of an agent configuration file. + +```yaml +gitops: + manifest_projects: + - id: gitlab-org/cluster-integration/gitlab-agent + default_namespace: my-ns + paths: + # Read all YAML files from this directory. + - glob: '/team1/app1/*.yaml' + # Read all .yaml files from team2/apps and all subdirectories. + - glob: '/team2/apps/**/*.yaml' + # If 'paths' is not specified or is an empty list, the configuration below is used. + - glob: '/**/*.{yaml,yml,json}' + reconcile_timeout: 3600s + dry_run_strategy: none + prune: true + prune_timeout: 3600s + prune_propagation_policy: foreground + inventory_policy: must_match +``` + +| Keyword | Description | +|--|--| +| `manifest_projects` | Projects where your Kubernetes manifests are stored. The agent monitors the files in the repositories in these projects. When manifest files change, the agent deploys the changes to the cluster. | +| `id` | Required. Path to a Git repository that has Kubernetes manifests in YAML or JSON format. No authentication mechanisms are currently supported. | +| `default_namespace` | Namespace to use if not set explicitly in object manifest. Also used for inventory `ConfigMap` objects. | +| `paths` | Repository paths to scan for manifest files. Directories with names that start with a dot `(.)` are ignored. | +| `paths[].glob` | Required. See [doublestar](https://github.com/bmatcuk/doublestar#about) and [the match function](https://pkg.go.dev/github.com/bmatcuk/doublestar/v2#Match) for globbing rules. | +| `reconcile_timeout` | Determines whether the applier should wait until all applied resources have been reconciled, and if so, how long to wait. Default is 3600 seconds (1 hour). | +| `dry_run_strategy` | Determines whether changes [should be performed](https://github.com/kubernetes-sigs/cli-utils/blob/d6968048dcd80b1c7b55d9e4f31fc25f71c9b490/pkg/common/common.go#L68-L89). Can be: `none`, `client`, or `server`. Default is `none`.| +| `prune` | Determines whether pruning of previously applied objects should happen after apply. Default is `true`. | +| `prune_timeout` | Determines whether to wait for all resources to be fully deleted after pruning, and if so, how long to wait. Default is 3600 seconds (1 hour). | +| `prune_propagation_policy` | The deletion propagation policy that [should be used for pruning](https://github.com/kubernetes/apimachinery/blob/44113beed5d39f1b261a12ec398a356e02358307/pkg/apis/meta/v1/types.go#L456-L470). Can be: `orphan`, `background`, or `foreground`. Default is `foreground`. | +| `inventory_policy` | Determines whether an inventory object can take over objects that belong to another inventory object or don't belong to any inventory object. This is done by determining if the apply/prune operation can go through for a resource based on comparison of the `inventory-id` value in the package and the `owning-inventory` annotation (`config.k8s.io/owning-inventory`) [in the live object](https://github.com/kubernetes-sigs/cli-utils/blob/d6968048dcd80b1c7b55d9e4f31fc25f71c9b490/pkg/inventory/policy.go#L12-L66). Can be: `must_match`, `adopt_if_no_inventory`, or `adopt_all`. Default is `must_match`. | + +## Additional resources + +The following documentation and examples can help you get started with a GitOps workflow. + +- [Managing Kubernetes secrets in a GitOps workflow](gitops/secrets_management.md) +- [Application and manifest repository example](https://gitlab.com/gitlab-examples/ops/gitops-demo/hello-world-service-gitops) + +## Troubleshooting + +### Avoiding conflicts when you have multiple projects + +The agent watches each glob pattern set under a project's `paths` section independently, and makes updates to the cluster concurrently. +If changes are found at multiple paths, when the agent attempts to update the cluster, +a conflict can occur. + +To prevent this from happening, consider storing a logical group of manifests in a single place and reference them only once to avoid overlapping globs. + +For example, both of these globs match `*.yaml` files in the root directory +and could cause conflicts: + +```yaml +gitops: + manifest_projects: + - id: project1 + paths: + - glob: '/**/*.yaml' + - glob: '/*.yaml' +``` + +Instead, specify a single glob that matches all `*.yaml` files recursively: + +```yaml +gitops: + manifest_projects: + - id: project1 + paths: + - glob: '/**/*.yaml' +``` + +### Use multiple agents or projects + +If you store your Kubernetes manifests in separate GitLab projects, +update your agent configuration file with the location of these projects. + +WARNING: +The project with the agent's +configuration file can be private or public. Other projects with Kubernetes manifests must be public. Support for private manifest projects is tracked +in [this issue](https://gitlab.com/gitlab-org/gitlab/-/issues/283885). diff --git a/doc/user/clusters/agent/gitops/secrets_management.md b/doc/user/clusters/agent/gitops/secrets_management.md new file mode 100644 index 00000000000..cf520c881bf --- /dev/null +++ b/doc/user/clusters/agent/gitops/secrets_management.md @@ -0,0 +1,61 @@ +--- +stage: Configure +group: Configure +info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/engineering/ux/technical-writing/#assignments +--- + +# Managing Kubernetes secrets in a GitOps workflow + +You should never store Kubernetes secrets in unencrypted form in a `git` repository. If you use a GitOps workflow, you can follow these steps to securely manage your secrets. + +1. Set up the Sealed Secrets controller to manage secrets. +1. Deploy Docker credentials so the cluster can pull images from the GitLab Container Registry. + +## Prerequisites + +This setup requires: + +- A [GitLab agent for Kubernetes configured for the GitOps workflow](../gitops.md). +- Access to the cluster to finish the setup. + +## Set up the Sealed Secrets controller to manage secrets + +You can use the [Sealed Secrets controller](https://github.com/bitnami-labs/sealed-secrets) to store encrypted secrets securely in a `git` repository. The controller decrypts the secret into a standard Kubernetes `Secret` kind resource. + +1. Go to [the Sealed Secrets release page](https://github.com/bitnami-labs/sealed-secrets/releases) and download the most recent `controller.yaml` file. +1. In GitLab, go to the project that contains your Kubernetes manifests and upload the `controller.yaml` file. +1. Open the agent configuration file (`config.yaml`) and if needed, update the `paths.glob` pattern to match the Sealed Secrets manifest. +1. Commit and push the changes to GitLab. +1. Confirm that the Sealed Secrets controller was installed successfully: + + ```shell + kubectl get pods -lname=sealed-secrets-controller -n kube-system + ``` + +1. Install the `kubeseal` command line utility by following [the Sealed Secrets instructions](https://github.com/bitnami-labs/sealed-secrets#homebrew). +1. Get the public key you need to encrypt secrets without direct access to the cluster: + + ```shell + kubeseal --fetch-cert > public.pem + ``` + +1. Commit the public key to the repository. + +For more details on how the Sealed Secrets controller works, view [the usage instructions](https://github.com/bitnami-labs/sealed-secrets/blob/main/README.md#usage). + +## Deploy Docker credentials + +To deploy containers from the GitLab Container Registry, you must configure the cluster with the proper Docker registry credentials. You can achieve this by deploying a `docker-registry` type secret. + +1. Generate a GitLab token with at least `read-registry` rights. The token can be either a Personal or a Project Access Token. +1. Create a Kubernetes secret manifest YAML file. Update the values as needed: + + ```shell + kubectl create secret docker-registry gitlab-credentials --docker-server=registry.gitlab.example.com --docker-username=<gitlab-username> --docker-password=<gitlab-token> --docker-email=<gitlab-user-email> -n <namespace> --dry-run=client -o yaml > gitlab-credentials.yaml + ``` + +1. Encrypt the secret into a `SealedSecret` manifest: + + ```shell + kubeseal --format=yaml --cert=public.pem < gitlab-credentials.yaml > gitlab-credentials.sealed.yaml + ``` diff --git a/doc/user/clusters/agent/index.md b/doc/user/clusters/agent/index.md index 3fb141e402f..a8ad19df2e1 100644 --- a/doc/user/clusters/agent/index.md +++ b/doc/user/clusters/agent/index.md @@ -8,23 +8,36 @@ info: To determine the technical writer assigned to the Stage/Group associated w > - [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/223061) in GitLab 13.4. > - Support for `grpcs` [introduced](https://gitlab.com/gitlab-org/cluster-integration/gitlab-agent/-/issues/7) in GitLab 13.6. -> - Agent Server [introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/300960) on GitLab.com under `wss://kas.gitlab.com` through an Early Adopter Program in GitLab 13.10. +> - Agent server [introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/300960) on GitLab.com under `wss://kas.gitlab.com` through an Early Adopter Program in GitLab 13.10. > - The agent became available to every project on GitLab.com in GitLab 13.11. > - [Moved](https://gitlab.com/groups/gitlab-org/-/epics/6290) from GitLab Premium to GitLab Free in 14.5. > - [Renamed](https://gitlab.com/groups/gitlab-org/-/epics/7167) from "GitLab Kubernetes Agent" to "GitLab agent for Kubernetes" in GitLab 14.6. You can connect your Kubernetes cluster with GitLab to deploy, manage, -and monitor your cloud-native solutions. You can choose from two primary workflows. +and monitor your cloud-native solutions. -In a **GitOps workflow**, you keep your Kubernetes manifests in GitLab. You install a GitLab agent in your cluster, and +To connect a Kubernetes cluster to GitLab, you must first [install an agent in your cluster](install/index.md). + +The agent runs in the cluster, and you can use it to: + +- Communicate with a cluster, which is behind a firewall or NAT. +- Access API endpoints in a cluster in real time. +- Push information about events happening in the cluster. +- Enable a cache of Kubernetes objects, which are kept up-to-date with very low latency. + +For more details about the agent's purpose and architecture, see the [architecture documentation](https://gitlab.com/gitlab-org/cluster-integration/gitlab-agent/-/blob/master/doc/architecture.md). + +## Workflows + +You can choose from two primary workflows. + +In a [**GitOps** workflow](gitops.md), you keep your Kubernetes manifests in GitLab. You install a GitLab agent in your cluster, and any time you update your manifests, the agent updates the cluster. This workflow is fully driven with Git and is considered pull-based, because the cluster is pulling updates from your GitLab repository. -In a **CI/CD** workflow, you use GitLab CI/CD to query and update your cluster by using the Kubernetes API. +In a [**CI/CD** workflow](ci_cd_tunnel.md), you use GitLab CI/CD to query and update your cluster by using the Kubernetes API. This workflow is considered push-based, because GitLab is pushing requests from GitLab CI/CD to your cluster. -Both of these workflows require you to [install an agent in your cluster](install/index.md). - ## Supported cluster versions GitLab supports the following Kubernetes versions. You can upgrade your @@ -32,8 +45,6 @@ Kubernetes version to a supported version at any time: - 1.20 (support ends on July 22, 2022) - 1.19 (support ends on February 22, 2022) -- 1.18 (support ends on November 22, 2021) -- 1.17 (support ends on September 22, 2021) GitLab supports at least two production-ready Kubernetes minor versions at any given time. GitLab regularly reviews the supported versions and @@ -47,174 +58,15 @@ version. The list of supported versions is based on: Some GitLab features might work on versions not listed here. -## Using Kubernetes with GitOps **(PREMIUM)** - -With GitOps, you can manage containerized clusters and applications from a Git repository that: - -- Is the single source of truth of your system. -- Is the single place where you operate your system. - -By combining GitLab, Kubernetes, and GitOps, you can have: - -- GitLab as the GitOps operator. -- Kubernetes as the automation and convergence system. -- GitLab CI/CD for Continuous Integration and the agent for Continuous Deployment. - -Beyond that, you can use all the features offered by GitLab as -the all-in-one DevOps platform for your product and your team. - -### GitOps workflow **(PREMIUM)** - -The agent uses multiple GitLab projects to provide a flexible workflow -that can suit various needs. This diagram shows these repositories and the main -The agent uses multiple GitLab projects to provide a flexible workflow. -This diagram shows these repositories and the main -actors involved in a deployment: - -```mermaid -sequenceDiagram - participant D as Developer - participant A as Application code repository - participant M as Manifest repository - participant K as GitLab agent - participant C as Agent configuration repository - loop Regularly - K-->>C: Grab the configuration - end - D->>+A: Pushing code changes - A->>M: Updating manifest - loop Regularly - K-->>M: Watching changes - M-->>K: Pulling and applying changes - end -``` - -For details, view the [architecture documentation](https://gitlab.com/gitlab-org/cluster-integration/gitlab-agent/-/blob/master/doc/architecture.md#high-level-architecture). - -To perform GitOps deployments, you need: - -- A properly-configured Kubernetes cluster where the GitLab agent is running. -- A project that contains the agent's configuration file (`config.yaml`) in the repository. - This file tells the agent which repositories to synchronize with the cluster. -- A project that contains Kubernetes manifests. Any changes to manifests are applied to the cluster. - -You can keep the agent's configuration file and Kubernetes manifests in one project, or you can use multiple. - -- One GitLab project (recommended): When you use one project for both the Kubernetes manifests - and the agent's configuration file, the projects can be either private or public. -- Two GitLab projects: When you use two different GitLab projects (one for Kubernetes - manifests and another for the agent's configuration file), the project with Kubernetes manifests must - be public. The project with the agent's configuration file can be either private or public. - -Support for separate private projects is tracked in [this issue](https://gitlab.com/gitlab-org/gitlab/-/issues/283885). - -## Remove an agent - -You can remove an agent by using the [GitLab UI](#remove-an-agent-through-the-gitlab-ui) or the [GraphQL API](#remove-an-agent-with-the-gitlab-graphql-api). - -### Remove an agent through the GitLab UI - -> [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/323055) in GitLab 14.7. - -To remove an agent from the UI: - -1. On the top bar, select **Menu > Projects** and find the project that contains the agent's configuration file. -1. From the left sidebar, select **Infrastructure > Kubernetes clusters**. -1. In the table, in the row for your agent, in the **Options** column, select the vertical ellipsis (**{ellipsis_v}**). -1. Select **Delete agent**. - -### Remove an agent with the GitLab GraphQL API - -1. Get the `<cluster-agent-token-id>` from a query in the interactive GraphQL explorer. - - For GitLab.com, go to <https://gitlab.com/-/graphql-explorer> to open GraphQL Explorer. - - For self-managed GitLab, go to `https://gitlab.example.com/-/graphql-explorer`, replacing `gitlab.example.com` with your instance's URL. - - ```graphql - query{ - project(fullPath: "<full-path-to-agent-configuration-project>") { - clusterAgent(name: "<agent-name>") { - id - tokens { - edges { - node { - id - } - } - } - } - } - } - ``` - -1. Remove an agent record with GraphQL by deleting the `clusterAgentToken`. - - ```graphql - mutation deleteAgent { - clusterAgentDelete(input: { id: "<cluster-agent-id>" } ) { - errors - } - } - - mutation deleteToken { - clusterAgentTokenDelete(input: { id: "<cluster-agent-token-id>" }) { - errors - } - } - ``` - -1. Verify whether the removal occurred successfully. If the output in the Pod logs includes `unauthenticated`, it means that the agent was successfully removed: - - ```json - { - "level": "warn", - "time": "2021-04-29T23:44:07.598Z", - "msg": "GetConfiguration.Recv failed", - "error": "rpc error: code = Unauthenticated desc = unauthenticated" - } - ``` - -1. Delete the agent in your cluster: - - ```shell - kubectl delete -n gitlab-kubernetes-agent -f ./resources.yml - ``` - -## Migrating to the agent from the legacy certificate-based integration - -Find out how to [migrate to the agent for Kubernetes](../../infrastructure/clusters/migrate_to_gitlab_agent.md) from the certificate-based integration. - -## Kubernetes network security alerts **(ULTIMATE)** - -> [Deprecated](https://gitlab.com/groups/gitlab-org/-/epics/7476) in GitLab 14.8, and planned for [removal](https://gitlab.com/groups/gitlab-org/-/epics/7477) in GitLab 15.0. - -WARNING: -Cilium integration is in its end-of-life process. It's [deprecated](https://gitlab.com/groups/gitlab-org/-/epics/7476) -for use in GitLab 14.8, and planned for [removal](https://gitlab.com/groups/gitlab-org/-/epics/7477) -in GitLab 15.0. - -The agent for Kubernetes also provides an integration with Cilium. This integration provides a simple way to -generate network policy-related alerts and to surface those alerts in GitLab. - -Several components work in concert for the agent to generate the alerts: - -- A working Kubernetes cluster. -- Cilium integration through either of these options: - - Installation through [cluster management template](../../project/clusters/protect/container_network_security/quick_start_guide.md#use-the-cluster-management-template-to-install-cilium). - - Enablement of [hubble-relay](https://docs.cilium.io/en/v1.8/concepts/overview/#hubble) on an - existing installation. -- One or more network policies through any of these options: - - Use the [Container Network Policy editor](../../application_security/policies/index.md#container-network-policy-editor) to create and manage policies. - - Use an [AutoDevOps](../../application_security/policies/index.md#container-network-policy) configuration. - - Add the required labels and annotations to existing network policies. -- A configuration repository with [Cilium configured in `config.yaml`](repository.md#surface-network-security-alerts-from-cluster-to-gitlab) +## Migrate to the agent from the legacy certificate-based integration -The setup process follows the same [agent's installation steps](install/index.md), -with the following differences: - -- When you define a configuration repository, you must do so with [Cilium settings](repository.md#surface-network-security-alerts-from-cluster-to-gitlab). -- You do not need to specify the `gitops` configuration section. +Read about how to [migrate to the agent for Kubernetes](../../infrastructure/clusters/migrate_to_gitlab_agent.md) from the certificate-based integration. ## Related topics +- [GitOps workflow](gitops.md) +- [GitLab CI/CD workflow](ci_cd_tunnel.md) +- [Install the agent](install/index.md) +- [Work with the agent](repository.md) - [Troubleshooting](troubleshooting.md) -- [Contribute to the GitLab agent's development](https://gitlab.com/gitlab-org/cluster-integration/gitlab-agent/-/tree/master/doc) +- [Contribute to the agent's development](https://gitlab.com/gitlab-org/cluster-integration/gitlab-agent/-/tree/master/doc) diff --git a/doc/user/clusters/agent/install/index.md b/doc/user/clusters/agent/install/index.md index 4d196e57f8f..fca80a4a291 100644 --- a/doc/user/clusters/agent/install/index.md +++ b/doc/user/clusters/agent/install/index.md @@ -4,92 +4,94 @@ group: Configure info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/engineering/ux/technical-writing/#assignments --- -# Install the GitLab Agent **(FREE)** +# Installing the agent for Kubernetes **(FREE)** > - [Moved](https://gitlab.com/groups/gitlab-org/-/epics/6290) from GitLab Premium to GitLab Free in 14.5. > - [Introduced](https://gitlab.com/gitlab-org/cluster-integration/gitlab-agent/-/merge_requests/594) multi-arch images in GitLab 14.8. The first multi-arch release is `v14.8.1`. It supports AMD64 and ARM64 architectures. +> - [Introduced](https://gitlab.com/gitlab-org/cluster-integration/gitlab-agent/-/merge_requests/603) ARM architecture support in GitLab 14.9. -To connect a cluster to GitLab, you need to install the GitLab Agent -onto your cluster. +To connect a Kubernetes cluster to GitLab, you must install an agent in your cluster. ## Prerequisites -- An existing Kubernetes cluster. If you don't have a cluster yet, you can create a new cluster on cloud providers, such as: +Before you can install the agent in your cluster, you need: + +- An existing Kubernetes cluster. If you don't have a cluster, you can create one on a cloud provider, like: - [Google Kubernetes Engine (GKE)](https://cloud.google.com/kubernetes-engine/docs/quickstart) - [Amazon Elastic Kubernetes Service (EKS)](https://docs.aws.amazon.com/eks/latest/userguide/getting-started.html) - [Digital Ocean](https://docs.digitalocean.com/products/kubernetes/quickstart/) -- On self-managed GitLab instances, a GitLab administrator needs to set up the [GitLab Agent Server (KAS)](../../../../administration/clusters/kas.md). +- On self-managed GitLab instances, a GitLab administrator must set up the [agent server](../../../../administration/clusters/kas.md). + On GitLab.com, the agent server is available at `wss://kas.gitlab.com`. ## Installation steps -To install the GitLab Agent on your cluster: +To install the agent in your cluster: -1. [Define a configuration repository](#define-a-configuration-repository). -1. [Register the Agent with GitLab](#register-the-agent-with-gitlab). -1. [Install the Agent onto the cluster](#install-the-agent-onto-the-cluster). +1. [Register the agent with GitLab](#register-the-agent-with-gitlab). +1. [Install the agent in your cluster](#install-the-agent-in-the-cluster). -<i class="fa fa-youtube-play youtube" aria-hidden="true"></i> Watch a GitLab 14.2 [walking-through video](https://www.youtube.com/watch?v=XuBpKtsgGkE) with this process. +<i class="fa fa-youtube-play youtube" aria-hidden="true"></i> Watch a GitLab 14.2 [walk-through of this process](https://www.youtube.com/watch?v=XuBpKtsgGkE). -When you complete the installation process, you can -[view your Agent's status and activity information](#view-your-agents). -You can also [configure](#configure-the-agent) it to your needs. +### Register the agent with GitLab -### Define a configuration repository +> - [Introduced](https://gitlab.com/groups/gitlab-org/-/epics/5786) in GitLab 14.1, you can create a new agent record directly from the GitLab UI. +> - [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/347240) in GitLab 14.9, the agent can be registered without creating an agent configuration file. -> - [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/259669) in GitLab 13.7, the Agent manifest configuration can be added to multiple directories (or subdirectories) of its repository. -> - Group authorization was [introduced](https://gitlab.com/groups/gitlab-org/-/epics/5784) in GitLab 14.3. +You must register an agent with GitLab. -To create an Agent, you need a GitLab repository to hold its -configuration file. If you already have a repository holding your -cluster's manifest files, you can use it to store your -Agent's configuration file and sync them with no further steps. +Prerequisites: -#### Create the Agent's configuration file +- For a [GitLab CI/CD workflow](../ci_cd_tunnel.md), ensure that + [GitLab CI/CD is enabled](../../../../ci/enable_or_disable_ci.md#enable-cicd-in-a-project). -To create an Agent, go to the repository where you want to store -it and add the Agent's configuration file under: +To register an agent with GitLab: -```plaintext -.gitlab/agents/<agent-name>/config.yaml -``` +1. On the top bar, select **Menu > Projects** and find your project. +1. From the left sidebar, select **Infrastructure > Kubernetes clusters**. +1. Select **Actions**. +1. From the **Select an agent** dropdown list: + - If you want to create a configuration with CI/CD defaults, type a name for the agent. + - If you already have an [agent configuration file](#create-an-agent-configuration-file), select it from the list. +1. Select **Register an agent**. +1. GitLab generates an access token for the agent. Securely store this token. You need it to install the agent in your cluster and to [update the agent](#update-the-agent-version) to another version. +1. Copy the command under **Recommended installation method**. You need it when you use the one-liner installation method to install the agent in your cluster. -You **don't have to add any content** to this file at the moment you -create it. The fact that the file exists tells GitLab that this is -an Agent. You can edit it later to [configure the Agent](#configure-the-agent). +### Create an agent configuration file -When creating this file, pay special attention to: +> - [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/259669) in GitLab 13.7, the agent configuration file can be added to multiple directories (or subdirectories) of the repository. +> - Group authorization was [introduced](https://gitlab.com/groups/gitlab-org/-/epics/5784) in GitLab 14.3. -- The [Agent's naming format](https://gitlab.com/gitlab-org/cluster-integration/gitlab-agent/-/blob/master/doc/identity_and_auth.md#agent-identity-and-name). -- The file extension: use the `.yaml` extension (`config.yaml`). The `.yml` extension is **not** recognized. +You can use an agent configuration file to specify details about your implementation. +Creating a file is optional but is needed if: -### Register the Agent with GitLab +- You use [a GitOps workflow](../gitops.md#gitops-configuration-reference) and you want a more advanced configuration. +- You use a GitLab CI/CD workflow. In that workflow, you must [authorize the agent](../ci_cd_tunnel.md#authorize-the-agent). -> [Introduced](https://gitlab.com/groups/gitlab-org/-/epics/5786) in GitLab 14.1, you can create a new Agent record directly from the GitLab UI. +To create an agent configuration file, go to the GitLab project. In the repository, create a file called `config.yaml` at this path: -Now that you've created your Agent's configuration file, register it -with GitLab. -When you register the Agent, GitLab generates a token that you need for -installing the Agent onto your cluster. +```plaintext +.gitlab/agents/<agent-name>/config.yaml +``` -In GitLab, go to the project where you added your Agent's configuration -file and: +- Ensure the agent name follows the [naming convention](https://gitlab.com/gitlab-org/cluster-integration/gitlab-agent/-/blob/master/doc/identity_and_auth.md#agent-identity-and-name). +- Ensure the filename has the `.yaml` file extension (`config.yaml`). The `.yml` extension is not accepted. +- Add content to the `config.yaml` file: + - For a GitOps workflow, view [the configuration reference](../gitops.md#gitops-configuration-reference) for details. + - For a GitLab CI/CD workflow, you can leave the file blank for now. -1. Ensure that [GitLab CI/CD is enabled in your project](../../../../ci/enable_or_disable_ci.md#enable-cicd-in-a-project). -1. From your project's sidebar, select **Infrastructure > Kubernetes clusters**. -1. Select **Actions**. -1. From the **Select an Agent** dropdown list, select the Agent you want to register and select **Register an Agent**. -1. GitLab generates a registration token for this Agent. Securely store this secret token, as you need it to install the Agent onto your cluster and to [update the Agent](#update-the-agent-version) to another version. -1. Copy the command under **Recommended installation method**. You need it to install the Agent onto your cluster through the one-liner installation method. +The agent bootstraps with the GitLab installation URL and an access token, +and you provide the rest of the configuration in your repository, following +Infrastructure as Code (IaaC) best practices. -### Install the Agent onto the cluster +### Install the agent in the cluster -To connect your cluster to GitLab, install the registered Agent -onto your cluster. To install it, you can use either: +To connect your cluster to GitLab, install the registered agent +in your cluster. To install it, you can use either: - [The one-liner installation method](#one-liner-installation). - [The advanced installation method](#advanced-installation). -You can use the one-liner installation for trying to use the Agent for the first time, to do internal setups with +You can use the one-liner installation for trying to use the agent for the first time, to do internal setups with high trust, and to quickly get started. For long-term production usage, you may want to use the advanced installation method to benefit from more configuration options. @@ -99,35 +101,45 @@ The one-liner installation is the simplest process, but you need Docker installed locally. If you don't have it, you can either install it or opt to the [advanced installation method](#advanced-installation). -To install the Agent on your cluster using the one-liner installation: +To install the agent on your cluster using the one-liner installation: -1. In your computer, open the terminal and connect to your cluster. +1. In your computer, open a terminal and [connect to your cluster](https://kubernetes.io/docs/tasks/access-application-cluster/access-cluster/). 1. Run the command you copied when registering your cluster in the previous step. Optionally, you can [customize the one-liner installation command](#customize-the-one-liner-installation). ##### Customize the one-liner installation -The one-liner command generated by GitLab: +By default, the one-liner command generated by GitLab: -- Creates a namespace for the deployment (`gitlab-kubernetes-agent`). +- Creates a namespace for the deployment (`gitlab-agent`). - Sets up a service account with `cluster-admin` rights (see [how to restrict this service account](#customize-the-permissions-for-the-agentk-service-account)). -- Creates a `Secret` resource for the Agent's registration token. +- Creates a `Secret` resource for the agent's access token. - Creates a `Deployment` resource for the `agentk` pod. -You can edit these parameters according to your needs to customize the -one-liner installation command at the command line. To find all available -options, run in your terminal: +You can edit these parameters to customize the one-liner installation command. +To view all available options, open a terminal and run this command: ```shell docker run --pull=always --rm registry.gitlab.com/gitlab-org/cluster-integration/gitlab-agent/cli:stable generate --help + +Usage: + cli generate [flags] + +Flags: + --agent-token string Access token registered for agent + --agent-version string Version of the agentk image to use (default "v14.8.1") + -h, --help help for generate + --kas-address string GitLab agent server for Kubernetes address + --name-prefix string The prefix to use for names of Kubernetes objects + --namespace string Kubernetes namespace to create resources in (default "gitlab-agent") + --no-rbac Do not include corresponding Roles and RoleBindings for the agent service account ``` WARNING: -`--agent-version stable` can be used to refer to the latest stable -release at the time when the command runs. It's fine for testing -purposes but for production please make sure to specify a matching -version explicitly. +Use `--agent-version stable` to refer to the latest stable +release at the time when the command runs. For production, however, +you should explicitly specify a matching version. #### Advanced installation @@ -135,105 +147,76 @@ For advanced installation options, use [the `kpt` installation method](https://g ##### Customize the permissions for the `agentk` service account -The GitLab Agent allows you to fully own your cluster and grant GitLab -the permissions you want. Still, to facilitate the process, by default the -generated manifests provide `cluster-admin` rights to the Agent. +You own your cluster and can grant GitLab the permissions you want. +By default, however, the generated manifests provide `cluster-admin` rights to the agent. -You can restrict the Agent's access rights using Kustomize overlays. [An example is commented out](https://gitlab.com/gitlab-org/cluster-integration/gitlab-agent/-/blob/master/build/deployment/gitlab-agent/cluster/kustomization.yaml) in the `kpt` package you retrieved as part of the installation. +You can restrict the agent's access rights by using Kustomize overlays. [An example is commented out](https://gitlab.com/gitlab-org/cluster-integration/gitlab-agent/-/blob/master/build/deployment/gitlab-agent/cluster/kustomization.yaml) in the `kpt` package you retrieved as part of the installation. -To create restricted permissions: +To restrict permissions: 1. Copy the `cluster` directory. 1. Edit the `kustomization.yaml` and `components/*` files based on your requirements. 1. Run `kustomize build <your copied directory> | kubectl apply -f -` to apply your configuration. -The above setup allows you to regularly update from the upstream package using `kpt pkg update gitlab-agent --strategy resource-merge` and maintain your customizations at the same time. +#### Update the advanced installation base layer -## Configure the Agent +Now you can update from the upstream package by using `kpt pkg update gitlab-agent --strategy resource-merge`. +When the advanced installation setup changes, you will not need to change your custom overlays. -When successfully installed, you can [configure the Agent](../repository.md) -by editing its configuration file. -When you update the configuration file, GitLab transmits the -information to the cluster automatically without downtime. +## Install multiple agents in your cluster -## View your Agents +For total separation between teams, you might need to run multiple `agentk` instances in your cluster. +You might want multiple agents so you can restrict RBAC for every `agentk` deployment. -> The version of installed `agentk` shown on the Agent tab [introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/340882) in GitLab 14.8. -If you have at least the Developer role, you can access the Agent's -configuration repository and view the Agent's list: +To install multiple agents, follow the +[advanced installation steps](https://gitlab.com/gitlab-org/cluster-integration/gitlab-agent/-/tree/master/build/deployment/gitlab-agent) +a second time and: -1. On the left sidebar, select **Infrastructure > Kubernetes clusters**. -1. Select **Agent** tab to view clusters connected to GitLab through the Agent. +1. Change the agent name and create a new configuration file. +1. Register the new agent. You receive a new access token. Each token should be used only with one agent. +1. Change the namespace or prefix you use for the installation. -On this page, you can view: +You should also change the RBAC for the installed `agentk`. -- All the registered Agents for the current project. -- The connection status. -- The version of `agentk` installed on your cluster. -- The path to each Agent's configuration file. - -Furthermore, if you select one of the Agents on your list, you can view its -[activity information](#view-the-agents-activity-information). - -### View the Agent's activity information - -> [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/277323) in GitLab 14.6. - -The activity logs help you to identify problems and get the information -you need for troubleshooting. You can see events from a week before the -current date. To access an Agent's activity: - -1. In your Agent's repository, go to the Agents list as described [above](#view-your-agents). -1. Select the Agent you want to see the activity. - -The activity list includes: - -- Agent registration events: when a new token is **created**. -- Connection events: when an Agent is successfully **connected** to a cluster. - -Note that the connection status is logged when you connect an Agent for -the first time or after more than an hour of inactivity. - -To check what else is planned for the Agent's UI and provide feedback, -see the [related epic](https://gitlab.com/groups/gitlab-org/-/epics/4739). +## Example projects -### View vulnerabilities in cluster images **(ULTIMATE)** +The following example projects can help you get started with the agent. -> [Introduced](https://gitlab.com/groups/gitlab-org/-/epics/6346) in GitLab 14.8 [with a flag](../../../../administration/feature_flags.md) named `cluster_vulnerabilities`. Enabled by default. +- [Configuration repository with minimal manifests](https://gitlab.com/gitlab-examples/ops/gitops-demo/k8s-agents) +- [Distinct application and manifest repository example](https://gitlab.com/gitlab-examples/ops/gitops-demo/hello-world-service-gitops) +- [Auto DevOps setup that uses the CI/CD workflow](https://gitlab.com/gitlab-examples/ops/gitops-demo/hello-world-service) +- [Cluster management project template example that uses the CI/CD workflow](https://gitlab.com/gitlab-examples/ops/gitops-demo/cluster-management) -Users with at least the [Developer role](../../../permissions.md) -can view cluster vulnerabilities. You can access them through the [vulnerability report](../../../application_security/vulnerabilities/index.md) -or in your cluster's image through the following process: +## Upgrades and version compatibility -1. Configure [cluster image scanning](../../../application_security/cluster_image_scanning/index.md) - to your build process. -1. Go to your Agent's configuration repository. -1. On the left sidebar, select **Infrastructure > Kubernetes clusters**. -1. Select the **Agent** tab. -1. Select the Agent you want to see the vulnerabilities for. +The agent has two major components: `agentk` and `kas`. +GitLab provides `kas` installers built into the various GitLab installation methods. +The required `kas` version corresponds to the GitLab `major.minor` (X.Y) versions. -![Cluster Agent security tab UI](../../img/cluster_agent_security_tab_v14_8.png) +At the same time, `agentk` and `kas` can differ by 1 minor version in either direction. For example, +`agentk` 14.4 supports `kas` 14.3, 14.4, and 14.5 (regardless of the patch). -## Create multiple Agents +A feature introduced in a given GitLab minor version might work with other `agentk` or `kas` versions. +To ensure it works, use at least the same `agentk` and `kas` minor version. For example, +if your GitLab version is 14.2, use at least `agentk` 14.2 and `kas` 14.2. -You can create and install multiple Agents using the same process -documented above. Give each Agent's configuration file a unique name -and you're good to go. You can create multiple Agents, for example: +We recommend upgrading your `kas` installations together with GitLab instances' upgrades, and to +[upgrade the `agentk` installations](#update-the-agent-version) after upgrading GitLab. -- To reach your cluster from different projects. -- To connect multiple clusters to GitLab. +The available `agentk` and `kas` versions are available in +[the Container Registry](https://gitlab.com/gitlab-org/cluster-integration/gitlab-agent/container_registry/). -## Update the Agent version +### Update the agent version -> [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/340882) in GitLab 14.8, GitLab warns you on the Agent's list page to update the Agent version installed on your cluster. +> [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/340882) in GitLab 14.8, GitLab warns you on the agent's list page to update the agent version installed on your cluster. -To update the Agent's version on your cluster, you need to re-run the [installation command](#install-the-agent-onto-the-cluster) +To update the agent's version, re-run the [installation command](#install-the-agent-in-the-cluster) with a newer `--agent-version`. Make sure to specify the other required parameters: `--kas-address`, `--namespace`, and `--agent-token`. -You can find the available `agentk` versions in [the container registry](https://gitlab.com/gitlab-org/cluster-integration/gitlab-agent/container_registry/1223205?sort=desc). +The available `agentk` versions are in [the Container Registry](https://gitlab.com/gitlab-org/cluster-integration/gitlab-agent/container_registry/1223205?sort=desc). -If you don't have access to your Agent's token, you can retrieve it from your cluster: +If you don't have access to your agent's access token, you can retrieve it from your cluster: -1. On your computer, open the terminal and connect to your cluster. +1. Open a terminal and connect to your cluster. 1. To retrieve the namespace, run: ```shell @@ -246,33 +229,8 @@ If you don't have access to your Agent's token, you can retrieve it from your cl kubectl -n <namespace> get secrets ``` -1. To retrieve the token, run: +1. To retrieve the access token, run: ```shell kubectl -n <namespace> get secret <secret-name> --template={{.data.token}} | base64 --decode ``` - -## Example projects - -The following example projects can help you get started with the Agent. - -- [Configuration repository](https://gitlab.com/gitlab-org/configure/examples/kubernetes-agent) -- This basic GitOps example deploys NGINX: [Manifest repository](https://gitlab.com/gitlab-org/configure/examples/gitops-project) - -## Upgrades and version compatibility - -The Agent is comprised of two major components: `agentk` and `kas`. -As we provide `kas` installers built into the various GitLab installation methods, the required `kas` version corresponds to the GitLab `major.minor` (X.Y) versions. - -At the same time, `agentk` and `kas` can differ by 1 minor version in either direction. For example, -`agentk` 14.4 supports `kas` 14.3, 14.4, and 14.5 (regardless of the patch). - -A feature introduced in a given GitLab minor version might work with other `agentk` or `kas` versions. -To make sure that it works, use at least the same `agentk` and `kas` minor version. For example, -if your GitLab version is 14.2, use at least `agentk` 14.2 and `kas` 14.2. - -We recommend upgrading your `kas` installations together with GitLab instances' upgrades, and to -[upgrade the `agentk` installations](#update-the-agent-version) after upgrading GitLab. - -The available `agentk` and `kas` versions can be found in -[the container registry](https://gitlab.com/gitlab-org/cluster-integration/gitlab-agent/container_registry/). diff --git a/doc/user/clusters/agent/repository.md b/doc/user/clusters/agent/repository.md index a9d9fd1c13d..3f743d34e0a 100644 --- a/doc/user/clusters/agent/repository.md +++ b/doc/user/clusters/agent/repository.md @@ -1,335 +1,169 @@ --- stage: Configure group: Configure -info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/engineering/ux/technical-writing/#designated-technical-writers +info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/engineering/ux/technical-writing/#assignments --- -# Agent configuration repository **(FREE)** +# Working with the agent for Kubernetes **(FREE)** > - [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/259669) in GitLab 13.7. -> - [Introduced](https://gitlab.com/groups/gitlab-org/-/epics/3834) in GitLab 13.11, the GitLab Agent became available on GitLab.com. +> - [Introduced](https://gitlab.com/groups/gitlab-org/-/epics/3834) in GitLab 13.11, the GitLab agent became available on GitLab.com. > - [Introduced](https://gitlab.com/groups/gitlab-org/-/epics/5784) the `ci_access` attribute in GitLab 14.3. > - [Moved](https://gitlab.com/groups/gitlab-org/-/epics/6290) from GitLab Premium to GitLab Free in 14.5. > - [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/332227) in GitLab 14.0, the `resource_inclusions` and `resource_exclusions` attributes were removed and `reconcile_timeout`, `dry_run_strategy`, `prune`, `prune_timeout`, `prune_propagation_policy`, and `inventory_policy` attributes were added. -The [GitLab Agent](index.md) supports hosting your configuration for -multiple agents in a single repository. These agents can be running -in the same cluster or in multiple clusters, and potentially with more than one agent per cluster. +## View your agents -The Agent bootstraps with the GitLab installation URL and an authentication token, -and you provide the rest of the configuration in your repository, following -Infrastructure as Code (IaaC) best practices. +> [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/340882) in GitLab 14.8, the installed `agentk` version is displayed on the **Agent** tab. -A minimal repository layout looks like this, with `my-agent-1` as the name -of your Agent: +Prerequisite: -```plaintext -|- .gitlab - |- agents - |- my-agent-1 - |- config.yaml -``` - -Make sure that `<agent-name>` conforms to the [Agent's naming format](https://gitlab.com/gitlab-org/cluster-integration/gitlab-agent/-/blob/master/doc/identity_and_auth.md#agent-identity-and-name). +- You must have at least the Developer role. -## Synchronize manifest projects **(PREMIUM)** +To view the list of agents: -Your `config.yaml` file contains a `gitops` section, which contains a `manifest_projects` -section. Each `id` in the `manifest_projects` section is the path to a Git repository -with Kubernetes resource definitions in YAML or JSON format. The Agent monitors -each project you declare, and when the project changes, GitLab deploys the changes -using the Agent. +1. On the top bar, select **Menu > Projects** and find the project that contains your agent configuration file. +1. On the left sidebar, select **Infrastructure > Kubernetes clusters**. +1. Select **Agent** tab to view clusters connected to GitLab through the agent. -WARNING: -When using separate GitLab projects for manifest files and configuration repository, the manifests project must be public. +On this page, you can view: -To use multiple YAML files, specify a `paths` attribute in the `gitops.manifest_projects` section. +- All the registered agents for the current project. +- The connection status. +- The version of `agentk` installed on your cluster. +- The path to each agent configuration file. -```yaml -gitops: - # Manifest projects are watched by the agent. Whenever a project changes, - # GitLab deploys the changes using the agent. - manifest_projects: - # No authentication mechanisms are currently supported. - # The `id` is a path to a Git repository with Kubernetes resource definitions - # in YAML or JSON format. - - id: gitlab-org/cluster-integration/gitlab-agent - # Namespace to use if not set explicitly in object manifest. - # Also used for inventory ConfigMap objects. - default_namespace: my-ns - # Paths inside of the repository to scan for manifest files. - # Directories with names starting with a dot are ignored. - paths: - # Read all .yaml files from team1/app1 directory. - # See https://github.com/bmatcuk/doublestar#about and - # https://pkg.go.dev/github.com/bmatcuk/doublestar/v2#Match for globbing rules. - - glob: '/team1/app1/*.yaml' - # Read all .yaml files from team2/apps and all subdirectories - - glob: '/team2/apps/**/*.yaml' - # If 'paths' is not specified or is an empty list, the configuration below is used - - glob: '/**/*.{yaml,yml,json}' - # Reconcile timeout defines whether the applier should wait - # until all applied resources have been reconciled, and if so, - # how long to wait. - reconcile_timeout: 3600s # 1 hour by default - # Dry run strategy defines whether changes should actually be performed, - # or if it is just talk and no action. - # https://github.com/kubernetes-sigs/cli-utils/blob/d6968048dcd80b1c7b55d9e4f31fc25f71c9b490/pkg/common/common.go#L68-L89 - # Can be: none, client, server - dry_run_strategy: none # 'none' by default - # Prune defines whether pruning of previously applied - # objects should happen after apply. - prune: true # enabled by default - # Prune timeout defines whether we should wait for all resources - # to be fully deleted after pruning, and if so, how long we should - # wait. - prune_timeout: 3600s # 1 hour by default - # Prune propagation policy defines the deletion propagation policy - # that should be used for pruning. - # https://github.com/kubernetes/apimachinery/blob/44113beed5d39f1b261a12ec398a356e02358307/pkg/apis/meta/v1/types.go#L456-L470 - # Can be: orphan, background, foreground - prune_propagation_policy: foreground # 'foreground' by default - # Inventory policy defines if an inventory object can take over - # objects that belong to another inventory object or don't - # belong to any inventory object. - # This is done by determining if the apply/prune operation - # can go through for a resource based on the comparison - # the inventory-id value in the package and the owning-inventory - # annotation (config.k8s.io/owning-inventory) in the live object. - # https://github.com/kubernetes-sigs/cli-utils/blob/d6968048dcd80b1c7b55d9e4f31fc25f71c9b490/pkg/inventory/policy.go#L12-L66 - # Can be: must_match, adopt_if_no_inventory, adopt_all - inventory_policy: must_match # 'must_match' by default -``` +## View an agent's activity information -### Using multiple manifest projects +> [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/277323) in GitLab 14.6. -Storing Kubernetes manifests in more than one repository can be handy, for example: +The activity logs help you to identify problems and get the information +you need for troubleshooting. You can see events from a week before the +current date. To view an agent's activity: -- You may store manifests for different applications in separate repositories. -- Different teams can work on manifests of independent projects in separate repositories. +1. On the top bar, select **Menu > Projects** and find the project that contains your agent configuration file. +1. On the left sidebar, select **Infrastructure > Kubernetes clusters**. +1. Select the agent you want to see activity for. -To use multiple repositories as the source of Kubernetes manifests, specify them in the list of -`manifest_projects` in your `config.yaml`: +The activity list includes: -```yaml -gitops: - manifest_projects: - - id: group1/project1 - - id: group2/project2 -``` +- Agent registration events: When a new token is **created**. +- Connection events: When an agent is successfully **connected** to a cluster. -Note that repositories are synchronized **concurrently** and **independently** from each other, -which means that, ideally, there should **not** be any dependencies shared by these repositories. -Storing a logical group of manifests in a single repository may work better than distributing it across several -repositories. +The connection status is logged when you connect an agent for +the first time or after more than an hour of inactivity. -You cannot use a single repository as a source for multiple concurrent synchronization -operations. If such functionality is needed, you may use multiple agents reading -manifests from the same repository. +View and provide feedback about the UI in [this epic](https://gitlab.com/groups/gitlab-org/-/epics/4739). -Ensure not to specify "overlapping" globs to avoid synchronizing the same files more than once. -This is detected by the Agent and leads to an error. +## Debug the agent -INCORRECT - both globs match `*.yaml` files in the root directory: +To debug the cluster-side component (`agentk`) of the agent, set the log +level according to the available options: -```yaml -gitops: - manifest_projects: - - id: project1 - paths: - - glob: '/**/*.yaml' - - glob: '/*.yaml' -``` +- `off` +- `warning` +- `error` +- `info` +- `debug` -CORRECT - single globs matches all `*.yaml` files recursively: +The log level defaults to `info`. You can change it by using a top-level `observability` +section in the configuration file, for example: ```yaml -gitops: - manifest_projects: - - id: project1 - paths: - - glob: '/**/*.yaml' +observability: + logging: + level: debug ``` -## Authorize projects and groups to use an Agent - -With the [CI/CD Tunnel](ci_cd_tunnel.md), you can authorize [projects](#authorize-projects-to-use-an-agent) -and [groups](#authorize-groups-to-use-an-agent) to use an Agent. - -Then, you can reach your cluster from authorized projects and [run Kubernetes commands from GitLab CI/CD scripts](#run-kubectl-commands-using-the-cicd-tunnel) -in these projects. - -### Authorize projects to use an Agent - -> [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/327850) in GitLab 14.4. - -To grant projects access to the Agent through the CI/CD Tunnel: - -1. Go to your Agent's configuration repository. -1. Edit the Agent's configuration file (`config.yaml`). -1. Add the `projects` attribute into `ci_access`. -1. Identify the project through its path: - - ```yaml - ci_access: - projects: - - id: path/to/project +## Reset the agent token + +> [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/327152) in GitLab 14.9. + +To reset the agent token without downtime: + +1. Create a new token: + 1. On the top bar, select **Menu > Projects** and find your project. + 1. On the left sidebar, select **Infrastructure > Kubernetes clusters**. + 1. Select the agent you want to create a token for. + 1. On the **Tokens** tab, select **Create token**. + 1. Enter token's name and description (optional) and select **Create token**. +1. Securely store the generated token. +1. Use the token to [install the agent in your cluster](install/index.md#install-the-agent-in-the-cluster) and to [update the agent](install/index.md#update-the-agent-version) to another version. +1. Delete the token you're no longer using. + +## Remove an agent + +You can remove an agent by using the [GitLab UI](#remove-an-agent-through-the-gitlab-ui) or the +[GraphQL API](#remove-an-agent-with-the-gitlab-graphql-api). The agent and any associated tokens +are removed from GitLab, but no changes are made in your Kubernetes cluster. You must +clean up those resources manually. + +### Remove an agent through the GitLab UI + +> [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/323055) in GitLab 14.7. + +To remove an agent from the UI: + +1. On the top bar, select **Menu > Projects** and find the project that contains the agent configuration file. +1. From the left sidebar, select **Infrastructure > Kubernetes clusters**. +1. In the table, in the row for your agent, in the **Options** column, select the vertical ellipsis (**{ellipsis_v}**). +1. Select **Delete agent**. + +### Remove an agent with the GitLab GraphQL API + +1. Get the `<cluster-agent-token-id>` from a query in the interactive GraphQL explorer. + - For GitLab.com, go to <https://gitlab.com/-/graphql-explorer> to open GraphQL Explorer. + - For self-managed GitLab, go to `https://gitlab.example.com/-/graphql-explorer`, replacing `gitlab.example.com` with your instance's URL. + + ```graphql + query{ + project(fullPath: "<full-path-to-agent-configuration-project>") { + clusterAgent(name: "<agent-name>") { + id + tokens { + edges { + node { + id + } + } + } + } + } + } ``` -### Authorize groups to use an Agent +1. Remove an agent record with GraphQL by deleting the `clusterAgentToken`. -> [Introduced](https://gitlab.com/groups/gitlab-org/-/epics/5784) in GitLab 14.3. + ```graphql + mutation deleteAgent { + clusterAgentDelete(input: { id: "<cluster-agent-id>" } ) { + errors + } + } -To grant access to all projects within a group: - -1. Go to your Agent's configuration repository. -1. Edit the Agent's configuration file (`config.yaml`). -1. Add the `groups` attribute into `ci_access`. -1. Identify the group or subgroup through its path: - - ```yaml - ci_access: - groups: - - id: path/to/group/subgroup + mutation deleteToken { + clusterAgentTokenDelete(input: { id: "<cluster-agent-token-id>" }) { + errors + } + } ``` -## Run `kubectl` commands using the CI/CD Tunnel - -After you authorize your project or group to use the Agent, you need to -configure the project's `.gitlab-ci.yaml` file to access the Agent. -This makes it possible to deploy applications to your cluster and run -any Kubernetes-specific commands from the authorized project. - -First, configure your Agent: - -1. Go to your Agent's configuration repository. -1. Edit your Agent's `config.yaml` file authorizing the [project](#authorize-projects-to-use-an-agent) or [group](#authorize-groups-to-use-an-agent) you want to run Kubernetes commands from. - -Then, configure the other project: - -1. Go to the project where you want to run Kubernetes commands from. -1. Edit your project's `.gitlab-ci.yml` file. -1. Set your Agent's context in the first command of the script with the format `path/to/agent/repository:agent-name`. -1. Run Kubernetes commands. - -For example: - -```yaml - deploy: - image: - name: bitnami/kubectl:latest - entrypoint: [""] - script: - - kubectl config use-context path/to/agent/repository:agent-name - - kubectl get pods -``` - -When you use the Agent, KubeContexts are named as `path/to/agent/repository:agent-name`. - -To get the list of available contexts: - -1. Open your terminal and connect to your cluster. -1. Run `kubectl config get-contexts`. - -### `kubectl` commands not supported - -The commands `kubectl exec`, `kubectl cp`, and `kubectl attach` are not supported by the CI/CD tunnel. -Anything else that uses the same API endpoints does not work either as they use the deprecated -SPDY protocol. -We [plan to add support for these features](https://gitlab.com/gitlab-org/gitlab/-/issues/346248) -in a future version of GitLab. - -### `kubectl` requires TLS - -`kubectl` would never send credentials over an unencrypted connection. Self-managed users should ensure that their -GitLab instance is configured with TLS for the CI/CD tunnel feature to work. Trying to use it without TLS -would produce errors: - -```shell -$ kubectl get pods -error: You must be logged in to the server (the server has asked for the client to provide credentials) -``` - -## Use impersonation to restrict project and group access **(PREMIUM)** - -> [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/345014) in GitLab 14.5. - -By default, the [CI/CD Tunnel](ci_cd_tunnel.md) inherits all the permissions from the service account used to install the -Agent in the cluster. -To restrict access to your cluster, you can use [impersonation](https://kubernetes.io/docs/reference/access-authn-authz/authentication/#user-impersonation). - -To specify impersonations, use the `access_as` attribute in your Agent's configuration file and use Kubernetes RBAC rules to manage impersonated account permissions. - -You can impersonate: - -- The Agent itself (default). -- The CI job that accesses the cluster. -- A specific user or system account defined within the cluster. - -### Impersonate the Agent - -The Agent is impersonated by default. You don't need to do anything to impersonate it. - -### Impersonate the CI job that accesses the cluster - -To impersonate the CI job that accesses the cluster, add the `ci_job: {}` key-value -under the `access_as` key. - -When the agent makes the request to the actual Kubernetes API, it sets the -impersonation credentials in the following way: - -- `UserName` is set to `gitlab:ci_job:<job id>`. Example: `gitlab:ci_job:1074499489`. -- `Groups` is set to: - - `gitlab:ci_job` to identify all requests coming from CI jobs. - - The list of IDs of groups the project is in. - - The project ID. - - The slug of the environment this job belongs to. +1. Verify whether the removal occurred successfully. If the output in the Pod logs includes `unauthenticated`, it means that the agent was successfully removed: - Example: for a CI job in `group1/group1-1/project1` where: - - - Group `group1` has ID 23. - - Group `group1/group1-1` has ID 25. - - Project `group1/group1-1/project1` has ID 150. - - Job running in a prod environment. - - Group list would be `[gitlab:ci_job, gitlab:group:23, gitlab:group:25, gitlab:project:150, gitlab:project_env:150:prod]`. - -- `Extra` carries extra information about the request. The following properties are set on the impersonated identity: - -| Property | Description | -| -------- | ----------- | -| `agent.gitlab.com/id` | Contains the agent ID. | -| `agent.gitlab.com/config_project_id` | Contains the agent's configuration project ID. | -| `agent.gitlab.com/project_id` | Contains the CI project ID. | -| `agent.gitlab.com/ci_pipeline_id` | Contains the CI pipeline ID. | -| `agent.gitlab.com/ci_job_id` | Contains the CI job ID. | -| `agent.gitlab.com/username` | Contains the username of the user the CI job is running as. | -| `agent.gitlab.com/environment_slug` | Contains the slug of the environment. Only set if running in an environment. | - -Example to restrict access by the CI job's identity: - -```yaml -ci_access: - projects: - - id: path/to/project - access_as: - ci_job: {} -``` - -### Impersonate a static identity - -For the given CI/CD Tunnel connection, you can use a static identity for the impersonation. - -Add the `impersonate` key under the `access_as` key to make the request using the provided identity. - -The identity can be specified with the following keys: + ```json + { + "level": "warn", + "time": "2021-04-29T23:44:07.598Z", + "msg": "GetConfiguration.Recv failed", + "error": "rpc error: code = Unauthenticated desc = unauthenticated" + } + ``` -- `username` (required) -- `uid` -- `groups` -- `extra` +1. Delete the agent in your cluster: -See the [official Kubernetes documentation for more details](https://kubernetes.io/docs/reference/access-authn-authz/authentication/#user-impersonation) on the usage of these keys. + ```shell + kubectl delete -n gitlab-kubernetes-agent -f ./resources.yml + ``` ## Surface network security alerts from cluster to GitLab **(ULTIMATE)** @@ -340,7 +174,28 @@ Cilium integration is in its end-of-life process. It's [deprecated](https://gitl for use in GitLab 14.8, and planned for [removal](https://gitlab.com/groups/gitlab-org/-/epics/7477) in GitLab 15.0. -The GitLab Agent provides an [integration with Cilium](index.md#kubernetes-network-security-alerts). +The agent for Kubernetes also provides an integration with Cilium. This integration provides a simple way to +generate network policy-related alerts and to surface those alerts in GitLab. + +Several components work in concert for the agent to generate the alerts: + +- A working Kubernetes cluster. +- Cilium integration through either of these options: + - Installation through [cluster management template](../../project/clusters/protect/container_network_security/quick_start_guide.md#use-the-cluster-management-template-to-install-cilium). + - Enablement of [hubble-relay](https://docs.cilium.io/en/v1.8/concepts/overview/#hubble) on an + existing installation. +- One or more network policies through any of these options: + - Use the [Container Network Policy editor](../../application_security/policies/index.md#container-network-policy-editor) to create and manage policies. + - Use an [AutoDevOps](../../application_security/policies/index.md#container-network-policy) configuration. + - Add the required labels and annotations to existing network policies. +- A configuration repository with [Cilium configured in `config.yaml`](repository.md#surface-network-security-alerts-from-cluster-to-gitlab) + +The setup process follows the same [agent's installation steps](install/index.md), +with the following differences: + +- When you define a configuration repository, you must do so with [Cilium settings](repository.md#surface-network-security-alerts-from-cluster-to-gitlab). +- You do not need to specify the `gitops` configuration section. + To integrate, add a top-level `cilium` section to your `config.yml` file. Currently, the only configuration option is the Hubble relay address: @@ -357,107 +212,3 @@ you can use `hubble-relay.gitlab-managed-apps.svc.cluster.local:80` as the addre cilium: hubble_relay_address: "hubble-relay.gitlab-managed-apps.svc.cluster.local:80" ``` - -## Scan your container images for vulnerabilities **(ULTIMATE)** - -You can use [cluster image scanning](../../application_security/cluster_image_scanning/index.md) -to scan container images in your cluster for security vulnerabilities. - -To begin scanning all resources in your cluster, add a `starboard` -configuration block to your agent's `config.yaml` with no `filters`: - -```yaml -starboard: - vulnerability_report: - filters: [] -``` - -The namespaces that are able to be scanned depend on the [Starboard Operator install mode](https://aquasecurity.github.io/starboard/latest/operator/configuration/#install-modes). -By default, the Starboard Operator only scans resources in the `default` namespace. To change this -behavior, edit the `STARBOARD_OPERATOR` environment variable in the `starboard-operator` deployment -definition. - -By adding filters, you can limit scans by: - -- Resource name -- Kind -- Container name -- Namespace - -```yaml -starboard: - vulnerability_report: - filters: - - namespaces: - - staging - - production - kinds: - - Deployment - - DaemonSet - containers: - - ruby - - postgres - - nginx - resources: - - my-app-name - - postgres - - ingress-nginx -``` - -A resource is scanned if the resource matches any of the given names and all of the given filter -types (`namespaces`, `kinds`, `containers`, `resources`). If a filter type is omitted, then all -names are scanned. In this example, a resource isn't scanned unless it has a container named `ruby`, -`postgres`, or `nginx`, and it's a `Deployment`: - -```yaml -starboard: - vulnerability_report: - filters: - - kinds: - - Deployment - containers: - - ruby - - postgres - - nginx -``` - -There is also a global `namespaces` field that applies to all filters: - -```yaml -starboard: - vulnerability_report: - namespaces: - - production - filters: - - kinds: - - Deployment - - kinds: - - DaemonSet - resources: - - log-collector -``` - -In this example, the following resources are scanned: - -- All deployments (`Deployment`) in the `production` namespace -- All daemon sets (`DaemonSet`) named `log-collector` in the `production` namespace - -## Debugging - -To debug the cluster-side component (`agentk`) of the Agent, set the log -level according to the available options: - -- `off` -- `warning` -- `error` -- `info` -- `debug` - -The log level defaults to `info`. You can change it by using a top-level `observability` -section in the configuration file, for example: - -```yaml -observability: - logging: - level: debug -``` diff --git a/doc/user/clusters/agent/runner.md b/doc/user/clusters/agent/runner.md deleted file mode 100644 index c40733bd7a5..00000000000 --- a/doc/user/clusters/agent/runner.md +++ /dev/null @@ -1,9 +0,0 @@ ---- -redirect_to: 'https://docs.gitlab.com/runner/install/kubernetes-agent.html' -remove_date: '2022-02-01' ---- - -This document was moved to [another location](https://docs.gitlab.com/runner/install/kubernetes-agent.html). - -<!-- This redirect file can be deleted after <2022-02-01>. --> -<!-- Before deletion, see: https://docs.gitlab.com/ee/development/documentation/#move-or-rename-a-page -->
\ No newline at end of file diff --git a/doc/user/clusters/agent/troubleshooting.md b/doc/user/clusters/agent/troubleshooting.md index 2c9f98b7c45..a5e568837ad 100644 --- a/doc/user/clusters/agent/troubleshooting.md +++ b/doc/user/clusters/agent/troubleshooting.md @@ -4,9 +4,9 @@ group: Configure info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/engineering/ux/technical-writing/#assignments --- -# Troubleshooting the GitLab Agent for Kubernetes +# Troubleshooting the GitLab agent for Kubernetes -When you are using the GitLab Agent for Kubernetes, you might experience issues you need to troubleshoot. +When you are using the GitLab agent for Kubernetes, you might experience issues you need to troubleshoot. You can start by viewing the service logs: @@ -14,7 +14,7 @@ You can start by viewing the service logs: kubectl logs -f -l=app=gitlab-agent -n gitlab-kubernetes-agent ``` -If you are a GitLab administrator, you can also view the [GitLab Agent Server logs](../../../administration/clusters/kas.md#troubleshooting). +If you are a GitLab administrator, you can also view the [GitLab agent server logs](../../../administration/clusters/kas.md#troubleshooting). ## Transport: Error while dialing failed to WebSocket dial @@ -28,7 +28,7 @@ If you are a GitLab administrator, you can also view the [GitLab Agent Server lo ``` This error is shown if there are some connectivity issues between the address -specified as `kas-address`, and your Agent pod. To fix it, make sure that you +specified as `kas-address`, and your agent pod. To fix it, make sure that you specified the `kas-address` correctly. ```json @@ -188,6 +188,5 @@ Alternatively, you can mount the certificate file at a different location and in } ``` -This error is shown if the manifest project is not public. To fix it, -[make sure your manifest project is public](repository.md#synchronize-manifest-projects) or your manifest files -are stored in the Agent's configuration repository. +This error is shown if the manifest project is not public. To fix it, make sure your manifest project is public or your manifest files +are stored in the agent's configuration repository. diff --git a/doc/user/clusters/agent/vulnerabilities.md b/doc/user/clusters/agent/vulnerabilities.md new file mode 100644 index 00000000000..480b09ff2ab --- /dev/null +++ b/doc/user/clusters/agent/vulnerabilities.md @@ -0,0 +1,113 @@ +--- +stage: Configure +group: Configure +info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/engineering/ux/technical-writing/#assignments +--- + +# Container vulnerability scanning **(ULTIMATE)** + +> [Introduced](https://gitlab.com/groups/gitlab-org/-/epics/6346) in GitLab 14.8. + +To view cluster vulnerabilities, you can view the [vulnerability report](../../application_security/vulnerabilities/index.md). +You can also configure your agent so the vulnerabilities are displayed with other agent information in GitLab. + +## View cluster vulnerabilities + +Prerequisite: + +- You must have at least the Developer role. +- [Cluster image scanning](../../application_security/cluster_image_scanning/index.md) + must be part of your build process. + +To view vulnerability information in GitLab: + +1. On the top bar, select **Menu > Projects** and find the project that contains the agent configuration file. +1. On the left sidebar, select **Infrastructure > Kubernetes clusters**. +1. Select the **Agent** tab. +1. Select the agent you want to see the vulnerabilities for. + +![Cluster agent security tab UI](../img/cluster_agent_security_tab_v14_8.png) + +## Enable cluster vulnerability scanning **(ULTIMATE)** + +You can use [cluster image scanning](../../application_security/cluster_image_scanning/index.md) +to scan container images in your cluster for security vulnerabilities. + +To begin scanning all resources in your cluster, add a `starboard` +configuration block to your agent configuration file with no `filters`: + +```yaml +starboard: + vulnerability_report: + filters: [] +``` + +The namespaces that are able to be scanned depend on the [Starboard Operator install mode](https://aquasecurity.github.io/starboard/latest/operator/configuration/#install-modes). +By default, the Starboard Operator only scans resources in the `default` namespace. To change this +behavior, edit the `STARBOARD_OPERATOR` environment variable in the `starboard-operator` deployment +definition. + +By adding filters, you can limit scans by: + +- Resource name +- Kind +- Container name +- Namespace + +```yaml +starboard: + vulnerability_report: + filters: + - namespaces: + - staging + - production + kinds: + - Deployment + - DaemonSet + containers: + - ruby + - postgres + - nginx + resources: + - my-app-name + - postgres + - ingress-nginx +``` + +A resource is scanned if the resource matches any of the given names and all of the given filter +types (`namespaces`, `kinds`, `containers`, `resources`). If a filter type is omitted, then all +names are scanned. In this example, a resource isn't scanned unless it has a container named `ruby`, +`postgres`, or `nginx`, and it's a `Deployment`: + +```yaml +starboard: + vulnerability_report: + filters: + - kinds: + - Deployment + containers: + - ruby + - postgres + - nginx +``` + +There is also a global `namespaces` field that applies to all filters: + +```yaml +starboard: + vulnerability_report: + namespaces: + - production + filters: + - kinds: + - Deployment + - kinds: + - DaemonSet + resources: + - log-collector +``` + +In this example, the following resources are scanned: + +- All deployments (`Deployment`) in the `production` namespace. +- All daemon sets (`DaemonSet`) named `log-collector` in the `production` namespace. |