diff options
Diffstat (limited to 'doc/user/clusters/agent/gitops')
| -rw-r--r-- | doc/user/clusters/agent/gitops/agent.md | 2 | ||||
| -rw-r--r-- | doc/user/clusters/agent/gitops/example_repository_structure.md | 237 | ||||
| -rw-r--r-- | doc/user/clusters/agent/gitops/flux_oci_tutorial.md | 154 | ||||
| -rw-r--r-- | doc/user/clusters/agent/gitops/flux_tutorial.md | 42 |
4 files changed, 359 insertions, 76 deletions
diff --git a/doc/user/clusters/agent/gitops/agent.md b/doc/user/clusters/agent/gitops/agent.md index 07ed2b3a691..a4e83916acb 100644 --- a/doc/user/clusters/agent/gitops/agent.md +++ b/doc/user/clusters/agent/gitops/agent.md @@ -4,7 +4,7 @@ group: Environments info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments --- -# Using GitOps with the agent for Kubernetes (deprecated) **(FREE)** +# Using GitOps with the agent for Kubernetes (deprecated) **(FREE ALL)** > - [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/259669) in GitLab 13.7. > - [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/332227) in GitLab 14.0, the `resource_inclusions` and `resource_exclusions` attributes were removed and `reconcile_timeout`, `dry_run_strategy`, `prune`, `prune_timeout`, `prune_propagation_policy`, and `inventory_policy` attributes were added. diff --git a/doc/user/clusters/agent/gitops/example_repository_structure.md b/doc/user/clusters/agent/gitops/example_repository_structure.md index a5bc3b153fe..02eea3300af 100644 --- a/doc/user/clusters/agent/gitops/example_repository_structure.md +++ b/doc/user/clusters/agent/gitops/example_repository_structure.md @@ -1,78 +1,179 @@ --- stage: Deploy group: Environments -info: An example of how to structure a repository for GitOps deployments +info: A tutorial for deploying a GitLab repository using Flux --- -# Example GitOps repository structure **(FREE)** +# Tutorial: Deploy a Git repository using Flux **(FREE ALL)** -This page describes an example structure for a project that builds and deploys an application -to a Kubernetes cluster with [GitOps](https://about.gitlab.com/topics/gitops) and the -[GitLab agent for Kubernetes](../../agent/gitops.md). +In this tutorial, you'll create a GitLab project that builds and deploys an application +to a Kubernetes cluster using Flux. You'll set up a sample manifest project, configure it to +push manifests to a deployment branch, and configure Flux to sync the deployment branch. With this +setup, you can run additional steps in GitLab pipelines before Flux picks up the changes +from the repository. + +This tutorial deploys an application from a public project. If you want to add a non-public project, you should create a [project deploy token](../../../project/deploy_tokens/index.md). + +To set up a repository for GitOps deployments: + +1. [Create the Kubernetes manifest repository](#create-the-kubernetes-manifest-repository) +1. [Create a deployment branch](#create-a-deployment-branch) +1. [Configure GitLab CI/CD to push to your branch](#configure-gitlab-cicd-to-push-to-your-branch) +1. [Configure Flux to sync your manifests](#configure-flux-to-sync-your-manifests) +1. [Verify your configuration](#verify-your-configuration) + +Prerequisites: + +- You have a Flux repository connected to a Kubernetes cluster. + If you're starting from scratch, see [Set up Flux for GitOps](flux_tutorial.md). -You can find an example project that uses this structure -[in this GitLab repository](https://gitlab.com/tigerwnz/minimal-gitops-app). You can use the example project -as a starting point to create your own deployment project. +## Create the Kubernetes manifest repository -## Deployment workflow +First, create a repository for your Kubernetes manifests: -The default branch is the single source of truth for your application and the -Kubernetes manifests that deploy it. To be reflected in a Kubernetes cluster, -a code or configuration change must exist in the default branch. +1. In GitLab, create a new repository called `web-app-manifests`. +1. In `web-app-manifests`, add a file named `src/nginx-deployment.yaml` with the following contents: -A GitLab agent for Kubernetes is installed in every Kubernetes cluster. The agent -is configured to sync manifests from a corresponding branch in the repository. -These branches represent the state of each cluster, and contain only commits that -exist in the default branch. - -Changes are deployed by merging the default branch into the branch of a cluster. -The agent that watches the branch picks up the change and syncs it to the cluster. - -For the actual deployment, the example project uses the GitLab agent for Kubernetes, -but you can also use other GitOps tools. - -### Review apps - -Ephemeral environments such as [review apps](../../../../ci/review_apps/index.md) -are deployed differently. Their configuration does not exist on the default branch, -and the changes are not meant to be deployed to a permanent environment. Review app -manifests are generated and deployed in a merge request feature branch, which is removed -when the MR is merged. - -## Example deployment - -The example project deploys to two permanent environments, staging and production, -which each have a dedicated Kubernetes cluster. A third cluster is used for ephemeral -review apps. - -Each cluster has a corresponding branch that represents the current state of the cluster: -`_gitlab/agents/staging`, `_gitlab/agents/production` and `_gitlab/agents/review`. Each branch is -[protected](../../../../user/project/protected_branches.md) and -a [project access token](../../../../user/project/settings/project_access_tokens.md) -is created for each branch with a configuration that allows only the corresponding token to push to the branch. -This ensures that environment branches are updated only through the configured process. - -Deployment branches are updated by CI/CD jobs. The access token that allows pushing to each -branch is configured as a [CI/CD variable](../../../../ci/variables/index.md). These variables -are protected, and only available to pipelines running on a protected branch. -The CI/CD job merges the default branch `main` into the deployment branch, and pushes -the deployment branch back to the repository using the provided token. To preserve the -commit history between both branches, the CI/CD job uses a fast-forward merge. - -Each cluster has an agent for Kubernetes, and each agent is configured to -sync manifests from the branch corresponding to its cluster. -In your own project, you can different GitOps tool like Flux, or use the same configuration to deploy -to virtual machines with GitLab CI/CD. - -### Application changes - -The example project follows this process to deploy an application change: - -1. A new feature branch is created with the desired changes. The pipeline builds an image, - runs the test suite, and deploy the changes to a review app in the `review` cluster. -1. The feature branch is merged to `main` and the review app is removed. -1. Manifests are updated on `main` (either directly or via merge request) to point to an updated - version of the deployed image. The pipeline automatically merges `main` into the `_gitlab/agents/staging` - branch, which updates the `staging` cluster. -1. The `production` job is triggered manually, and merges `main` into the `_gitlab/agents/production` branch, - deploying to the `production` cluster. + ```yaml + apiVersion: apps/v1 + kind: Deployment + metadata: + name: nginx + spec: + replicas: 1 + template: + spec: + containers: + - name: nginx + image: nginx:1.14.2 + ports: + - containerPort: 80 + ``` + +1. In `web-app-manifests`, add a file named `src/kustomization.yaml` with the following contents: + + ```yaml + apiVersion: kustomize.config.k8s.io/v1beta1 + kind: Kustomization + resources: + - nginx-deployment.yaml + commonLabels: + app: flux-branches-tutorial + ``` + +## Create a deployment branch + +Next, create a branch to reflect the current state of your cluster. + +In this workflow, the default branch is the single source of truth for your application. +To be reflected in a Kubernetes cluster, a code or configuration change must exist in the default branch. +In a later step, you'll configure CI/CD to merge changes from the default branch into the deployment branch. + +To create a deployment branch: + +1. In `web-app-manifests`, create a branch named `_gitlab/deploy/example` from the default branch. The branch name in this example is chosen to + differentiate the deployment branch from feature branches, but this is not required. You can name the deployment branch whatever you like. +1. Create a [project](../../../../user/project/settings/project_access_tokens.md), + [group](../../../../user/group/settings/group_access_tokens.md) or + [personal access token](../../../../user/profile/personal_access_tokens.md) with the `write_repository` scope. +1. Create a [CI/CD variable](../../../../ci/variables/index.md) with a token value named `DEPLOYMENT_TOKEN`. + Remember to [mask](../../../../ci/variables/index.md#mask-a-cicd-variable) the value so that it won't show in + job logs. +1. Add a rule to [protect](../../../../user/project/protected_branches.md) + your deployment branch with the following values: + + - Allowed to merge: No one. + - Allowed to push and merge: Select the token you created in the previous step, or your user if you created + a personal access token. + - Allowed to force push: Turn off the toggle. + - Require approval from code owners: Turn off the toggle. + +This configuration ensures that only the corresponding token can push to the branch. + +You've successfully created a repository with a protected deployment branch! + +## Configure GitLab CI/CD to push to your branch + +Next, you'll configure CI/CD to merge changes from the default branch to your deployment branch. + +In the root of `web-app-manifests`, create and push a [`.gitlab-ci.yml`](../../../../ci/yaml/gitlab_ci_yaml.md) file with the following contents: + + ```yaml + deploy: + stage: deploy + environment: production + variables: + DEPLOYMENT_BRANCH: _gitlab/deploy/example + script: + - | + git config user.name "Deploy Example Bot" + git config user.email "test@example.com" + git fetch origin $DEPLOYMENT_BRANCH + git checkout $DEPLOYMENT_BRANCH + git merge $CI_COMMIT_SHA --ff-only + git push https://deploy:$DEPLOYMENT_TOKEN@$CI_SERVER_HOST/$CI_PROJECT_PATH.git HEAD:$DEPLOYMENT_BRANCH + resource_group: $CI_ENVIRONMENT_SLUG + ``` + +This creates a CI/CD pipeline with a single `deploy` job that: + +1. Checks out your deployment branch. +1. Merges new changes from the default branch into the deployment branch. +1. Pushes the changes to your repository with the configured token. + +## Configure Flux to sync your manifests + +Next, configure your Flux repository to sync the deployment branch in by the `web-app-manifests` repository. + +To configure, create a [`GitRepository`](https://fluxcd.io/flux/components/source/gitrepositories/) resource: + +1. In your local clone of your Flux repository, add a file named `clusters/my-cluster/web-app-manifests-source.yaml` + with the following contents: + + ```yaml + apiVersion: source.toolkit.fluxcd.io/v1 + kind: GitRepository + metadata: + name: web-app-manifests + namespace: flux-system + spec: + interval: 5m0s + url: https://gitlab.com/gitlab-org/configure/examples/flux/web-app-manifests-branches + ref: + branch: _gitlab/deploy/example + ``` + + You will need to substitute the `url` with the URL of your `web-app-manifests` project. + +1. In your local clone of your Flux repository, add a file named `clusters/my-cluster/web-app-manifests-kustomization.yaml` + with the following contents: + + ```yaml + apiVersion: kustomize.toolkit.fluxcd.io/v1 + kind: Kustomization + metadata: + name: nginx-source-kustomization + namespace: flux-system + spec: + interval: 1m0s + path: ./src + prune: true + sourceRef: + kind: GitRepository + name: web-app-manifests + targetNamespace: default + ``` + + This file adds a [Kustomization](https://fluxcd.io/flux/components/kustomize/kustomization/) resource that tells Flux to sync the manifests in the artifact fetched from the registry. + +1. Commit the new files and push. + +## Verify your configuration + +After the pipeline completes, you should see a newly created `nginx` pod in your cluster. + +If you want to see the deployment sync again, try updating the number of replicas in the +`src/nginx-deployment.yaml` file and push to the default branch. If all is working well, the change +will sync to the cluster when the pipeline has finished. + +Congratulations! You successfully configured a project to deploy an application and synchronize your changes! diff --git a/doc/user/clusters/agent/gitops/flux_oci_tutorial.md b/doc/user/clusters/agent/gitops/flux_oci_tutorial.md new file mode 100644 index 00000000000..b970c818a72 --- /dev/null +++ b/doc/user/clusters/agent/gitops/flux_oci_tutorial.md @@ -0,0 +1,154 @@ +--- +stage: Deploy +group: Environments +info: A tutorial for deploying an OCI artifact using Flux +--- + +# Tutorial: Deploy an OCI artifact using Flux **(FREE ALL)** + +This tutorial teaches you how to package your Kubernetes manifests into an [OCI](https://opencontainers.org/) +artifact and deploy them to your cluster using Flux. You'll set up a sample manifest project, configure it to +store manifests as an artifact in the project's Container Registry, and configure Flux to sync the artifact. With this +setup, you can run additional steps in GitLab pipelines before Flux picks up the changes +from the OCI image. + +This tutorial deploys an application from a public project. If you want to add a non-public project, you should create a [project deploy token](../../../project/deploy_tokens/index.md). + +To deploy an OCI artifact using Flux: + +1. [Create the Kubernetes manifest repository](#create-the-kubernetes-manifest-repository) +1. [Configure the manifest repository to create an OCI artifact](#configure-the-manifest-repository-to-create-an-oci-artifact) +1. [Configure Flux to sync your artifact](#configure-flux-to-sync-your-artifact) +1. [Verify your configuration](#verify-your-configuration) + +Prerequisites: + +- You have a Flux repository connected to a Kubernetes cluster. + If you're starting from scratch, see [Set up Flux for GitOps](flux_tutorial.md). + +## Create the Kubernetes manifest repository + +First, create a repository for your Kubernetes manifests: + +1. In GitLab, create a new repository called `web-app-manifests`. +1. In `web-app-manifests`, add a file named `src/nginx-deployment.yaml` with the following contents: + + ```yaml + apiVersion: apps/v1 + kind: Deployment + metadata: + name: nginx + spec: + replicas: 1 + template: + spec: + containers: + - name: nginx + image: nginx:1.14.2 + ports: + - containerPort: 80 + ``` + +1. In `web-app-manifests`, add a file named `src/kustomization.yaml` with the following contents: + + ```yaml + apiVersion: kustomize.config.k8s.io/v1beta1 + kind: Kustomization + resources: + - nginx-deployment.yaml + commonLabels: + app: flux-oci-tutorial + ``` + +## Configure the manifest repository to create an OCI artifact + +Next, configure [GitLab CI/CD](../../../../ci/index.md) to package your manifests into an OCI artifact, +and push the artifact to the [GitLab Container Registry](../../../packages/container_registry/index.md): + +1. In the root of `web-app-manifests`, create and push a [`.gitlab-ci.yml`](../../../../ci/yaml/gitlab_ci_yaml.md) file with the following contents: + + ```yaml + package: + stage: deploy + image: + name: fluxcd/flux-cli:v2.0.0-rc.1 + entrypoint: [""] + script: + - mkdir -p manifests + - kubectl kustomize ./src --output ./manifests + - | + flux push artifact oci://$CI_REGISTRY_IMAGE:latest \ + --path="./manifests" \ + --source="$CI_REPOSITORY_URL" \ + --revision="$CI_COMMIT_SHORT_SHA" \ + --creds="$CI_REGISTRY_USER:$CI_REGISTRY_PASSWORD" \ + --annotations="org.opencontainers.image.url=$CI_PROJECT_URL" \ + --annotations="org.opencontainers.image.title=$CI_PROJECT_NAME" \ + --annotations="com.gitlab.job.id=$CI_JOB_ID" \ + --annotations="com.gitlab.job.url=$CI_JOB_URL" + ``` + + When the file is pushed to GitLab, a CI/CD pipeline with a single `package` job is created. This job: + + - Uses `kustomization.yaml` to render your final Kubernetes manifests. + - Packages your manifests into an OCI artifact. + - Pushes the OCI artifact to the Container Registry. + + After the pipeline has completed, you can check your OCI artifact with the Container Registry UI. + +## Configure Flux to sync your artifact + +Next, configure your Flux repository to sync the artifact produced by the `web-app-manifests` repository. + +To configure, create an [`OCIRepository`](https://fluxcd.io/flux/components/source/ocirepositories/) resource: + +1. In your local clone of your Flux repository, add a file named `clusters/my-cluster/web-app-manifests-source.yaml` + with the following contents: + + ```yaml + apiVersion: source.toolkit.fluxcd.io/v1 + kind: OCIRepository + metadata: + name: web-app-manifests + namespace: flux-system + spec: + interval: 1m0s + url: oci://registry.gitlab.com/gitlab-org/configure/examples/flux/web-app-manifests-oci + ref: + tag: latest + ``` + + You will need to substitute the `url` with the URL of your `web-app-manifests` project's container registry. + +1. In your local clone of your Flux repository, add a file named `clusters/my-cluster/web-app-manifests-kustomization.yaml` + with the following contents: + + ```yaml + apiVersion: kustomize.toolkit.fluxcd.io/v1 + kind: Kustomization + metadata: + name: nginx-source-kustomization + namespace: flux-system + spec: + interval: 1m0s + path: ./ + prune: true + sourceRef: + kind: OCIRepository + name: web-app-manifests + targetNamespace: default + ``` + + This file adds a [Kustomization](https://fluxcd.io/flux/components/kustomize/kustomization/) resource that tells Flux to sync the manifests in the artifact fetched from the registry. + +1. Commit the new files and push. + +## Verify your configuration + +You should see a newly created `nginx` pod in your cluster. + +If you want to see the deployment sync again, try updating the number of replicas in the +`src/nginx-deployment.yaml` file and push to the default branch. If all is working well, the change +should sync to the cluster when the pipeline has finished. + +Congratulations! You successfully configured a project to deploy an application and synchronize your changes! diff --git a/doc/user/clusters/agent/gitops/flux_tutorial.md b/doc/user/clusters/agent/gitops/flux_tutorial.md index 8aee0c01d65..f2d65b900f0 100644 --- a/doc/user/clusters/agent/gitops/flux_tutorial.md +++ b/doc/user/clusters/agent/gitops/flux_tutorial.md @@ -4,7 +4,7 @@ group: Environments info: A tutorial using Flux --- -# Tutorial: Set up Flux for GitOps **(FREE)** +# Tutorial: Set up Flux for GitOps **(FREE ALL)** This tutorial teaches you how to set up Flux for GitOps. You'll complete a bootstrap installation, install `agentk` in your cluster, and deploy a simple `nginx` application. @@ -83,8 +83,19 @@ You must register `agentk` before you install it in your cluster. To register `agentk`: -- Complete the steps in [Register the agent with GitLab](../install/index.md#register-the-agent-with-gitlab). - Be sure to save the agent registration token and `kas` address. +1. On the left sidebar, at the top, select **Search GitLab** (**{search}**) to find your project. + If you have an [agent configuration file](../install/index.md#create-an-agent-configuration-file), + it must be in this project. Your cluster manifest files should also be in this project. +1. Select **Operate > Kubernetes clusters**. +1. Select **Connect a cluster (agent)**. + - If you want to create a configuration with CI/CD defaults, type a name. + - If you already have an agent configuration file, select it from the list. +1. Select **Register an agent**. +1. Securely store the agent access token and `kasAddress` for later. + +The agent is registered for your project. You don't need to run any commands yet. + +In the next step, you'll use Flux to install `agentk` in your cluster. ## Install `agentk` @@ -103,10 +114,24 @@ To install `agentk`: name: gitlab ``` -1. Apply the agent registration token as a secret in the cluster: +1. Create a file called `secret.yaml` that contains your agent access token as a secret: + + ```yaml + apiVersion: v1 + kind: Secret + metadata: + name: gitlab-agent-token + type: Opaque + stringData: + values.yaml: |- + config: + token: "<your-token-here>" + ``` + +1. Apply `secret.yaml` to your cluster: ```shell - kubectl create secret generic gitlab-agent-token -n gitlab --from-literal=token=YOUR-TOKEN-HERE + kubectl apply -f secret.yaml -n gitlab ``` Although this step does not follow GitOps principles, it simplifies configuration for new Flux users. @@ -147,8 +172,11 @@ To install `agentk`: interval: 1h0m0s values: config: - kasAddress: "wss://kas.gitlab.example.com" - secretName: "gitlab-agent-token" + kasAddress: "wss://kas.gitlab.com" + valuesFrom: + - kind: Secret + name: gitlab-agent-token + valuesKey: values.yaml ``` 1. To verify that `agentk` is installed and running in the cluster, run the following command: |