diff options
Diffstat (limited to 'doc/user/clusters/agent/gitops')
-rw-r--r-- | doc/user/clusters/agent/gitops/example_repository_structure.md | 78 | ||||
-rw-r--r-- | doc/user/clusters/agent/gitops/flux.md | 36 | ||||
-rw-r--r-- | doc/user/clusters/agent/gitops/flux_tutorial.md | 192 | ||||
-rw-r--r-- | doc/user/clusters/agent/gitops/helm.md | 10 | ||||
-rw-r--r-- | doc/user/clusters/agent/gitops/secrets_management.md | 10 |
5 files changed, 316 insertions, 10 deletions
diff --git a/doc/user/clusters/agent/gitops/example_repository_structure.md b/doc/user/clusters/agent/gitops/example_repository_structure.md new file mode 100644 index 00000000000..93c05e40bfd --- /dev/null +++ b/doc/user/clusters/agent/gitops/example_repository_structure.md @@ -0,0 +1,78 @@ +--- +stage: Deploy +group: Environments +info: An example of how to structure a repository for GitOps deployments +--- + +# Example GitOps repository structure **(FREE)** + +This page describes an example structure for a project that builds and deploys an application +to a Kubernetes cluster with [GitOps](https://about.gitlab.com/topics/gitops) and the +[GitLab agent for Kubernetes](../../agent/gitops.md). + +You can find an example project that uses this structure +[in this GitLab repository](https://gitlab.com/tigerwnz/minimal-gitops-app). You can use the example project +as a starting point to create your own deployment project. + +## Deployment workflow + +The default branch is the single source of truth for your application and the +Kubernetes manifests that deploy it. To be reflected in a Kubernetes cluster, +a code or configuration change must exist in the default branch. + +A GitLab agent for Kubernetes is installed in every Kubernetes cluster. The agent +is configured to sync manifests from a corresponding branch in the repository. +These branches represent the state of each cluster, and contain only commits that +exist in the default branch. + +Changes are deployed by merging the default branch into the branch of a cluster. +The agent that watches the branch picks up the change and syncs it to the cluster. + +For the actual deployment, the example project uses the GitLab agent for Kubernetes, +but you can also use other GitOps tools. + +### Review apps + +Ephemeral environments such as [review apps](../../../../ci/review_apps/index.md) +are deployed differently. Their configuration does not exist on the default branch, +and the changes are not meant to be deployed to a permanent environment. Review app +manifests are generated and deployed in a merge request feature branch, which is removed +when the MR is merged. + +## Example deployment + +The example project deploys to two permanent environments, staging and production, +which each have a dedicated Kubernetes cluster. A third cluster is used for ephemeral +review apps. + +Each cluster has a corresponding branch that represents the current state of the cluster: +`_gitlab/agents/staging`, `_gitlab/agents/production` and `_gitlab/agents/review`. Each branch is +[protected](../../../../user/project/protected_branches.md) and +a [project access token](../../../../user/project/settings/project_access_tokens.md) +is created for each branch with a configuration that allows only the corresponding token to push to the branch. +This ensures that environment branches are updated only through the configured process. + +Deployment branches are updated by CI/CD jobs. The access token that allows pushing to each +branch is configured as a [CI/CD variable](../../../../ci/variables/index.md). These variables +are protected, and only available to pipelines running on a protected branch. +The CI/CD job merges the default branch `main` into the deployment branch, and pushes +the deployment branch back to the repository using the provided token. To preserve the +commit history between both branches, the CI/CD job uses a fast-forward merge. + +Each cluster has an agent for Kubernetes, and each agent is configured to +[sync manifests from the branch corresponding to its cluster](../gitops.md#gitops-configuration-reference). +In your own project, you can different GitOps tool like Flux, or use the same configuration to deploy +to virtual machines with GitLab CI/CD. + +### Application changes + +The example project follows this process to deploy an application change: + +1. A new feature branch is created with the desired changes. The pipeline builds an image, + runs the test suite, and deploy the changes to a review app in the `review` cluster. +1. The feature branch is merged to `main` and the review app is removed. +1. Manifests are updated on `main` (either directly or via merge request) to point to an updated + version of the deployed image. The pipeline automatically merges `main` into the `_gitlab/agents/staging` + branch, which updates the `staging` cluster. +1. The `production` job is triggered manually, and merges `main` into the `_gitlab/agents/production` branch, + deploying to the `production` cluster. diff --git a/doc/user/clusters/agent/gitops/flux.md b/doc/user/clusters/agent/gitops/flux.md new file mode 100644 index 00000000000..98840080716 --- /dev/null +++ b/doc/user/clusters/agent/gitops/flux.md @@ -0,0 +1,36 @@ +--- +stage: Deploy +group: Environments +info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments +--- + +# Flux (Beta) **(FREE)** + +Flux is a GitOps tool that helps you manage your Kubernetes clusters. +You can use Flux to: + +- Keep your clusters in sync with your Git repositories. +- Reconcile code changes with your deployments. +- Manage your Flux installation itself with a bootstrap. + +To get started, see the [Flux installation documentation](https://fluxcd.io/flux/installation). + +Support for Flux is in [Beta](../../../../policy/alpha-beta-support.md#beta). + +## Bootstrap installation + +Use the Flux command [`bootstrap gitlab`](https://fluxcd.io/flux/installation/#gitlab-and-gitlab-enterprise) +to configure a Kubernetes cluster to manage itself from a Git repository. + +You must authenticate your installation with either: + +- Recommended. [A project access token](../../../project/settings/project_access_tokens.md). +- A [group access token](../../../group/settings/group_access_tokens.md). +- A [personal access token](../../../profile/personal_access_tokens.md). + +Some Flux features like [automated image updates](https://fluxcd.io/flux/guides/image-update/) require +write access to the source repositories. + +## GitOps repository structure + +You should organize your repositories to meet the needs of your team. For detailed recommendations, see the Flux [repository structure documentation](https://fluxcd.io/flux/guides/repository-structure/). diff --git a/doc/user/clusters/agent/gitops/flux_tutorial.md b/doc/user/clusters/agent/gitops/flux_tutorial.md new file mode 100644 index 00000000000..bdb772ac14e --- /dev/null +++ b/doc/user/clusters/agent/gitops/flux_tutorial.md @@ -0,0 +1,192 @@ +--- +stage: Deploy +group: Environments +info: A tutorial using Flux +--- + +# Tutorial: Set up Flux for GitOps **(FREE)** + +This tutorial teaches you how to set up Flux for GitOps. You'll set up a sample project, +complete a bootstrap Flux installation, and authenticate your installation with a +[project deploy token](../../../project/deploy_tokens/index.md). + +You can find the fully configured tutorial project [in this GitLab repository](https://gitlab.com/gitlab-org/configure/examples/flux/flux-config). +It works in conjunction with [this repository](https://gitlab.com/gitlab-org/configure/examples/flux/web-app-manifests/-/tree/main), which contains the example Kubernetes manifests. + +To set up Flux for GitOps: + +1. [Create a personal access token](#create-a-personal-access-token) +1. [Create the Flux repository](#create-the-flux-repository) +1. [Create the Kubernetes manifest repository](#create-the-kubernetes-manifest-repository) +1. [Configure Flux to sync your manifests](#configure-flux-to-sync-your-manifests) +1. [Verify your configuration](#verify-your-configuration) + +Prerequisites: + +- You must have a Kubernetes cluster running. + +## Create a personal access token + +To authenticate with the Flux CLI, you must create a personal access token +with the `api` scope: + +1. In the upper-right corner, select your avatar. +1. Select **Edit profile**. +1. On the left sidebar, select **Access Tokens**. +1. Enter a name and optional expiry date for the token. +1. Select the `api` scope. +1. Select **Create personal access token**. + +You can also use a [project](../../../project/settings/project_access_tokens.md) or [group access token](../../../group/settings/group_access_tokens.md) with the `api` scope. + +## Create the Flux repository + +Create a Git repository, install Flux, and authenticate Flux with your repo: + +1. Make sure your `kubectl` is configured to access your cluster. +1. [Install the Flux CLI](https://fluxcd.io/flux/installation/#install-the-flux-cli). You must install Flux v2 or higher. +1. In GitLab, create a new empty project called `flux-config`. +1. From your shell, export a `GITLAB_TOKEN` environment variable with the value of your personal access token. + For example, `export GITLAB_TOKEN=<personal-access-token>`. +1. Run the `bootstrap` command. The exact command depends on whether you are + creating the Flux repository under a GitLab user, group, or subgroup. For more information, + see the [Flux bootstrap documentation](https://fluxcd.io/flux/installation/#gitlab-and-gitlab-enterprise). + + In this tutorial, you're working with a public project in a subgroup. The bootstrap command looks like this: + + ```shell + flux bootstrap gitlab \ + --owner=gitlab-org/configure/examples/flux \ + --repository=flux-config \ + --branch=main \ + --path=clusters/my-cluster \ + --deploy-token-auth + ``` + + This command installs Flux on the Kubernetes cluster and configures it to manage itself from the repository `flux-config`. + The command also automatically creates the project deploy token required to access the `flux-config` repository. + +Great work! You now have a repository bootstrapped with a Flux configuration. Any updates to your repository are automatically synced to the cluster. + +## Create the Kubernetes manifest repository + +Next, create a repository for your Kubernetes manifests: + +1. In GitLab, create a new repository called `web-app-manifests`. +1. Add a file to `web-app-manifests` named `nginx-deployment.yaml` with the following contents: + + ```yaml + apiVersion: apps/v1 + + kind: Deployment + + metadata: + name: nginx-deployment + labels: + app: nginx + spec: + replicas: 3 + selector: + matchLabels: + app: nginx + template: + metadata: + labels: + app: nginx + spec: + containers: + - name: nginx + image: nginx:1.14.2 + ports: + - containerPort: 80 + ``` + +1. In the new repository, [create a deploy token](../../../project/deploy_tokens/index.md#create-a-deploy-token) with only the `read_repository` scope. +1. Store your deploy token username and password somewhere safe. +1. In Flux CLI, create a secret with your deploy token and point the secret to the new repository. For example: + + ```shell + flux create secret git flux-deploy-authentication \ + --url=https://gitlab.com/gitlab-org/configure/examples/flux/web-app-manifests \ + --namespace=default \ + --username=<token-user-name> \ + --password=<token-password> + ``` + +1. To check if your secret was generated successfully, run: + + ```shell + kubectl -n default get secrets flux-deploy-authentication -o yaml + ``` + + Under `data`, you should see base64-encoded values associated with your token username and password. + +Congratulations! You now have a manifest repository, a deploy token, and a secret generated directly on your cluster. + +## Configure Flux to sync your manifests + +Next, tell `flux-config` to sync with the `web-app-manifests` repository. + +To do so, create a [`GitRepository`](https://fluxcd.io/flux/components/source/gitrepositories/) resource: + +1. Clone the `flux-config` repo to your machine. +1. In your local clone of `flux-config`, add the `GitRepository` file `clusters/my-cluster/web-app-manifests-source.yaml`: + + ```yaml + --- + apiVersion: source.toolkit.fluxcd.io/v1beta2 + kind: GitRepository + metadata: + name: web-app-manifests + namespace: default + spec: + interval: 1m0s + ref: + branch: main + secretRef: + name: flux-deploy-authentication + url: https://gitlab.com/gitlab-org/configure/examples/flux/web-app-manifests + ``` + + This file uses `secretRef` to refer back to the deploy token secret you created in the last step. + +1. In your local clone of `flux-config`, add the `GitRepository` file `clusters/my-cluster/web-app-manifests-kustomization.yaml`: + + ```yaml + --- + apiVersion: kustomize.toolkit.fluxcd.io/v1beta2 + kind: Kustomization + metadata: + name: nginx-source-kustomization + namespace: default + spec: + interval: 1m0s + path: ./ + prune: true + sourceRef: + kind: GitRepository + name: web-app-manifests + namespace: default + targetNamespace: default + ``` + + This file adds a [`Kustomization`](https://fluxcd.io/flux/components/kustomize/kustomization/) resource that tells Flux to sync the manifests from + `web-app-manifests` with `kustomize`. + +1. Commit the new files and push. + +## Verify your configuration + +You should see a newly created `nginx-deployment` pod in your cluster. + +To check whether the `nginx-deployment` pod is running in the default namespace, run the following: + +```shell +kubectl -n default get pods -n default +``` + +If you want to see the deployment sync again, try updating the number of replicas in the +`nginx-deployment.yaml` file and push to your `main` branch. If all is working well, it +should sync to the cluster. + +Excellent work! You've successfully set up a complete Flux project. diff --git a/doc/user/clusters/agent/gitops/helm.md b/doc/user/clusters/agent/gitops/helm.md index b9a59d37f5d..182fd87e6f9 100644 --- a/doc/user/clusters/agent/gitops/helm.md +++ b/doc/user/clusters/agent/gitops/helm.md @@ -1,10 +1,10 @@ --- -stage: Configure -group: Configure +stage: Deploy +group: Environments info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments --- -# Using Helm charts to update a Kubernetes cluster (Alpha) **(FREE)** +# Using Helm charts to update a Kubernetes cluster (Experiment) **(FREE)** > - [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/371019) in GitLab 15.4. > - Specifying a branch, tag, or commit reference to fetch the Kubernetes manifest files [introduced](https://gitlab.com/groups/gitlab-org/-/epics/4516) in GitLab 15.7. @@ -13,11 +13,11 @@ You can deploy Helm charts to your Kubernetes cluster and keep the resources in with your charts and values. To do this, you use the pull-based GitOps features of the agent for Kubernetes. -This feature is in Alpha and [an epic exists](https://gitlab.com/groups/gitlab-org/-/epics/7938) +This feature is an Experiment and [an epic exists](https://gitlab.com/groups/gitlab-org/-/epics/7938) to track future work. Tell us about your use cases by leaving comments in the epic. NOTE: -This feature is Alpha. In future releases, to accommodate new features, the configuration format might change without notice. +This feature is an Experiment. In future releases, to accommodate new features, the configuration format might change without notice. ## GitOps workflow steps diff --git a/doc/user/clusters/agent/gitops/secrets_management.md b/doc/user/clusters/agent/gitops/secrets_management.md index dc1cbe3009d..6e1b7da9c6c 100644 --- a/doc/user/clusters/agent/gitops/secrets_management.md +++ b/doc/user/clusters/agent/gitops/secrets_management.md @@ -1,6 +1,6 @@ --- -stage: Configure -group: Configure +stage: Deploy +group: Environments info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments --- @@ -50,9 +50,9 @@ To deploy containers from the GitLab Container Registry, you must configure the 1. Generate a GitLab token with at least `read-registry` rights. The token can be either a Personal or a Project Access Token. 1. Create a Kubernetes secret manifest YAML file. Update the values as needed: - ```shell - kubectl create secret docker-registry gitlab-credentials --docker-server=registry.gitlab.example.com --docker-username=<gitlab-username> --docker-password=<gitlab-token> --docker-email=<gitlab-user-email> -n <namespace> --dry-run=client -o yaml > gitlab-credentials.yaml - ``` + ```shell + kubectl create secret docker-registry gitlab-credentials --docker-server=registry.gitlab.example.com --docker-username=<gitlab-username> --docker-password=<gitlab-token> --docker-email=<gitlab-user-email> -n <namespace> --dry-run=client -o yaml > gitlab-credentials.yaml + ``` 1. Encrypt the secret into a `SealedSecret` manifest: |