diff options
Diffstat (limited to 'doc/user/infrastructure')
16 files changed, 873 insertions, 140 deletions
diff --git a/doc/user/infrastructure/clusters/manage/management_project_applications/apparmor.md b/doc/user/infrastructure/clusters/manage/management_project_applications/apparmor.md new file mode 100644 index 00000000000..7fbbbac866c --- /dev/null +++ b/doc/user/infrastructure/clusters/manage/management_project_applications/apparmor.md @@ -0,0 +1,30 @@ +--- +stage: Protect +group: Container Security +info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/engineering/ux/technical-writing/#assignments +--- + +# Install AppArmor with a cluster management project + +> [Introduced](https://gitlab.com/gitlab-org/project-templates/cluster-management/-/merge_requests/5) in GitLab 14.0. + +Assuming you already have a [Cluster management project](../../../../../user/clusters/management_project.md) created from a +[management project template](../../../../../user/clusters/management_project_template.md), to install AppArmor you should +uncomment this line from your `helmfile.yaml`: + +```yaml + - path: applications/apparmor/helmfile.yaml +``` + +You can define one or more AppArmor profiles by adding them into +`applications/apparmor/values.yaml` as the following: + +```yaml +profiles: + profile-one: |- + profile profile-one { + file, + } +``` + +Refer to the [AppArmor chart](https://gitlab.com/gitlab-org/charts/apparmor) for more information on this chart. diff --git a/doc/user/infrastructure/clusters/manage/management_project_applications/certmanager.md b/doc/user/infrastructure/clusters/manage/management_project_applications/certmanager.md new file mode 100644 index 00000000000..3bf79ea90c7 --- /dev/null +++ b/doc/user/infrastructure/clusters/manage/management_project_applications/certmanager.md @@ -0,0 +1,56 @@ +--- +stage: Configure +group: Configure +info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/engineering/ux/technical-writing/#assignments +--- + +# Install cert-manager with a cluster management project + +> [Introduced](https://gitlab.com/gitlab-org/project-templates/cluster-management/-/merge_requests/5) in GitLab 14.0. + +Assuming you already have a [Cluster management project](../../../../../user/clusters/management_project.md) created from a +[management project template](../../../../../user/clusters/management_project_template.md), to install cert-manager you should +uncomment this line from your `helmfile.yaml`: + +```yaml + - path: applications/cert-manager/helmfile.yaml +``` + +cert-manager: + +- Is installed by default into the `gitlab-managed-apps` namespace of your cluster. +- Can be installed with or without a default + [Let's Encrypt `ClusterIssuer`](https://cert-manager.io/docs/configuration/acme/), which requires an + email address to be specified. The email address is used by Let's Encrypt to + contact you about expiring certificates and issues related to your account. + +The following configuration in your `applications/cert-manager/helmfile.yaml` is required to install cert-manager: + +```yaml +certManager: + installed: true + letsEncryptClusterIssuer: + installed: true + email: "user@example.com" +``` + +Or without the default `ClusterIssuer`: + +```yaml +certManager: + installed: true + letsEncryptClusterIssuer: + installed: false +``` + +You can customize the installation of cert-manager by defining a +`.gitlab/managed-apps/cert-manager/values.yaml` file in your cluster +management project. Refer to the +[chart](https://github.com/jetstack/cert-manager) for the +available configuration options. + +Support for installing the Cert Manager managed application is provided by the +GitLab Configure group. If you run into unknown issues, +[open a new issue](https://gitlab.com/gitlab-org/gitlab/-/issues/new), and ping at +least 2 people from the +[Configure group](https://about.gitlab.com/handbook/product/categories/#configure-group). diff --git a/doc/user/infrastructure/clusters/manage/management_project_applications/cilium.md b/doc/user/infrastructure/clusters/manage/management_project_applications/cilium.md new file mode 100644 index 00000000000..4e84f2c5ef4 --- /dev/null +++ b/doc/user/infrastructure/clusters/manage/management_project_applications/cilium.md @@ -0,0 +1,128 @@ +--- +stage: Protect +group: Container Security +info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/engineering/ux/technical-writing/#assignments +--- + +# Install Cilium with a cluster management project + +> [Introduced](https://gitlab.com/gitlab-org/project-templates/cluster-management/-/merge_requests/5) in GitLab 14.0. + +[Cilium](https://cilium.io/) is a networking plugin for Kubernetes that you can use to implement +support for [NetworkPolicy](https://kubernetes.io/docs/concepts/services-networking/network-policies/) +resources. For more information, see [Network Policies](../../../../../topics/autodevops/stages.md#network-policy). + +<i class="fa fa-youtube-play youtube" aria-hidden="true"></i> +For an overview, see the +[Container Network Security Demo for GitLab 12.8](https://www.youtube.com/watch?v=pgUEdhdhoUI). + +Assuming you already have a [Cluster management project](../../../../../user/clusters/management_project.md) created from a +[management project template](../../../../../user/clusters/management_project_template.md), to install cilium you should +uncomment this line from your `helmfile.yaml`: + +```yaml + - path: applications/cilium/helmfile.yaml +``` + +and update the `applications/cilium/values.yaml` to set the `clusterType`: + +```yaml +# possible values are gke or eks +clusterType: gke +``` + +The `clusterType` variable enables the recommended Helm variables for a corresponding cluster type. +You can check the recommended variables for each cluster type in the official documentation: + +- [Google GKE](https://docs.cilium.io/en/v1.8/gettingstarted/k8s-install-gke/#deploy-cilium) +- [AWS EKS](https://docs.cilium.io/en/v1.8/gettingstarted/k8s-install-eks/#deploy-cilium) + +Do not use `clusterType` for sandbox environments like [Minikube](https://minikube.sigs.k8s.io/docs/). + +You can customize Cilium's Helm variables by defining the +`applications/cilium/values.yaml` file in your cluster +management project. Refer to the +[Cilium chart](https://github.com/cilium/cilium/tree/master/install/kubernetes/cilium) +for the available configuration options. + +You can check Cilium's installation status on the cluster management page: + +- [Project-level cluster](../../../../project/clusters/index.md): Navigate to your project's + **Infrastructure > Kubernetes clusters** page. +- [Group-level cluster](../../../../group/clusters/index.md): Navigate to your group's + **Kubernetes** page. + +WARNING: +Installation and removal of the Cilium requires a **manual** +[restart](https://docs.cilium.io/en/stable/gettingstarted/k8s-install-helm/#restart-unmanaged-pods) +of all affected pods in all namespaces to ensure that they are +[managed](https://docs.cilium.io/en/v1.8/operations/troubleshooting/#ensure-managed-pod) +by the correct networking plugin. Whenever Hubble is enabled, its related pod might require a +restart depending on whether it started prior to Cilium. For more information, see +[Failed Deployment](https://kubernetes.io/docs/concepts/workloads/controllers/deployment/#failed-deployment) +in the Kubernetes docs. + +NOTE: +Major upgrades might require additional setup steps. For more information, see +the official [upgrade guide](https://docs.cilium.io/en/v1.8/operations/upgrade/). + +By default, Cilium's +[audit mode](https://docs.cilium.io/en/v1.8/gettingstarted/policy-creation/#enable-policy-audit-mode) +is enabled. In audit mode, Cilium doesn't drop disallowed packets. You +can use `policy-verdict` log to observe policy-related decisions. You +can disable audit mode by adding the following to +`applications/cilium/values.yaml`: + +```yaml +config: + policyAuditMode: false + +agent: + monitor: + eventTypes: ["drop"] +``` + +The Cilium monitor log for traffic is logged out by the +`cilium-monitor` sidecar container. You can check these logs with the following command: + +```shell +kubectl -n gitlab-managed-apps logs -l k8s-app=cilium -c cilium-monitor +``` + +You can disable the monitor log in `.gitlab/managed-apps/cilium/values.yaml`: + +```yaml +agent: + monitor: + enabled: false +``` + +The [Hubble](https://github.com/cilium/hubble) monitoring daemon is enabled by default +and it's set to collect per namespace flow metrics. This metrics are accessible on the +[Threat Monitoring](../../../../application_security/threat_monitoring/index.md) +dashboard. You can disable Hubble by adding the following to +`applications/cilium/values.yaml`: + +```yaml +global: + hubble: + enabled: false +``` + +You can also adjust Helm values for Hubble by using +`applications/cilium/values.yaml`: + +```yaml +global: + hubble: + enabled: true + metrics: + enabled: + - 'flow:sourceContext=namespace;destinationContext=namespace' +``` + +Support for installing the Cilium managed application is provided by the +GitLab Container Security group. If you run into unknown issues, +[open a new issue](https://gitlab.com/gitlab-org/gitlab/-/issues/new), and ping at +least 2 people from the +[Container Security group](https://about.gitlab.com/handbook/product/categories/#container-security-group). diff --git a/doc/user/infrastructure/clusters/manage/management_project_applications/elasticstack.md b/doc/user/infrastructure/clusters/manage/management_project_applications/elasticstack.md new file mode 100644 index 00000000000..3d2b3caf0af --- /dev/null +++ b/doc/user/infrastructure/clusters/manage/management_project_applications/elasticstack.md @@ -0,0 +1,34 @@ +--- +stage: Monitor +group: Monitor +info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/engineering/ux/technical-writing/#assignments +--- + +# Install Elastic Stack with a cluster management project + +> [Introduced](https://gitlab.com/gitlab-org/project-templates/cluster-management/-/merge_requests/5) in GitLab 14.0. + +Assuming you already have a [Cluster management project](../../../../../user/clusters/management_project.md) created from a +[management project template](../../../../../user/clusters/management_project_template.md), to install Elastic Stack you should +uncomment this line from your `helmfile.yaml`: + +```yaml + - path: applications/elastic-stack/helmfile.yaml +``` + +Elastic Stack is installed by default into the `gitlab-managed-apps` namespace of your cluster. + +You can check the default +[`values.yaml`](https://gitlab.com/gitlab-org/project-templates/cluster-management/-/blob/master/applications/elastic-stack/values.yaml) +we set for this chart. + +You can customize the installation of Elastic Stack by updating the +`applications/elastic-stack/values.yaml` file in your cluster +management project. Refer to the +[chart](https://gitlab.com/gitlab-org/charts/elastic-stack) for all +available configuration options. + +Support for installing the Elastic Stack managed application is provided by the +GitLab APM group. If you run into unknown issues, +[open a new issue](https://gitlab.com/gitlab-org/gitlab/-/issues/new), and ping at +least 2 people from the [APM group](https://about.gitlab.com/handbook/product/categories/#apm-group). diff --git a/doc/user/infrastructure/clusters/manage/management_project_applications/falco.md b/doc/user/infrastructure/clusters/manage/management_project_applications/falco.md new file mode 100644 index 00000000000..dff0c3bd7bc --- /dev/null +++ b/doc/user/infrastructure/clusters/manage/management_project_applications/falco.md @@ -0,0 +1,101 @@ +--- +stage: Protect +group: Container Security +info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/engineering/ux/technical-writing/#assignments +--- + +# Install Falco with a cluster management project + +> [Introduced](https://gitlab.com/gitlab-org/project-templates/cluster-management/-/merge_requests/5) in GitLab 14.0. + +GitLab Container Host Security Monitoring uses [Falco](https://falco.org/) +as a runtime security tool that listens to the Linux kernel using eBPF. Falco parses system calls +and asserts the stream against a configurable rules engine in real-time. For more information, see +[Falco's Documentation](https://falco.org/docs/). + +Assuming you already have a [Cluster management project](../../../../../user/clusters/management_project.md) created from a +[management project template](../../../../../user/clusters/management_project_template.md), to install Falco you should +uncomment this line from your `helmfile.yaml`: + +```yaml + - path: applications/falco/helmfile.yaml +``` + +You can customize Falco's Helm variables by defining the +`applications/falco/values.yaml` file in your cluster +management project. Refer to the +[Falco chart](https://github.com/falcosecurity/charts/tree/master/falco) +for the available configuration options. + +WARNING: +By default eBPF support is enabled and Falco uses an +[eBPF probe](https://falco.org/docs/event-sources/drivers/#using-the-ebpf-probe) +to pass system calls to user space. If your cluster doesn't support this, you can +configure it to use Falco kernel module instead by adding the following to +`applications/falco/values.yaml`: + +```yaml +ebpf: + enabled: false +``` + +In rare cases where probe installation on your cluster isn't possible and the kernel/probe +isn't pre-compiled, you may need to manually prepare the kernel module or eBPF probe with +[`driverkit`](https://github.com/falcosecurity/driverkit#against-a-kubernetes-cluster) +and install it on each cluster node. + +By default, Falco is deployed with a limited set of rules. To add more rules, add +the following to `applications/falco/values.yaml` (you can get examples from +[Cloud Native Security Hub](https://securityhub.dev/)): + +```yaml +customRules: + file-integrity.yaml: |- + - rule: Detect New File + desc: detect new file created + condition: > + evt.type = chmod or evt.type = fchmod + output: > + File below a known directory opened for writing (user=%user.name + command=%proc.cmdline file=%fd.name parent=%proc.pname pcmdline=%proc.pcmdline gparent=%proc.aname[2]) + priority: ERROR + tags: [filesystem] + - rule: Detect New Directory + desc: detect new directory created + condition: > + mkdir + output: > + File below a known directory opened for writing (user=%user.name + command=%proc.cmdline file=%fd.name parent=%proc.pname pcmdline=%proc.pcmdline gparent=%proc.aname[2]) + priority: ERROR + tags: [filesystem] +``` + +By default, Falco only outputs security events to logs as JSON objects. To set it to output to an +[external API](https://falco.org/docs/alerts/#https-output-send-alerts-to-an-https-end-point) +or [application](https://falco.org/docs/alerts/#program-output), +add the following to `applications/falco/values.yaml`: + +```yaml +falco: + programOutput: + enabled: true + keepAlive: false + program: mail -s "Falco Notification" someone@example.com + + httpOutput: + enabled: true + url: http://some.url +``` + +You can check these logs with the following command: + +```shell +kubectl -n gitlab-managed-apps logs -l app=falco +``` + +Support for installing the Falco managed application is provided by the +GitLab Container Security group. If you run into unknown issues, +[open a new issue](https://gitlab.com/gitlab-org/gitlab/-/issues/new), and ping at +least 2 people from the +[Container Security group](https://about.gitlab.com/handbook/product/categories/#container-security-group). diff --git a/doc/user/infrastructure/clusters/manage/management_project_applications/fluentd.md b/doc/user/infrastructure/clusters/manage/management_project_applications/fluentd.md new file mode 100644 index 00000000000..bf05f8f87d8 --- /dev/null +++ b/doc/user/infrastructure/clusters/manage/management_project_applications/fluentd.md @@ -0,0 +1,36 @@ +--- +stage: Protect +group: Container Security +info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/engineering/ux/technical-writing/#assignments +--- + +# Install Fluentd with a cluster management project + +> [Introduced](https://gitlab.com/gitlab-org/project-templates/cluster-management/-/merge_requests/5) in GitLab 14.0. + +Assuming you already have a [Cluster management project](../../../../../user/clusters/management_project.md) created from a +[management project template](../../../../../user/clusters/management_project_template.md), to install Fluentd you should +uncomment this line from your `helmfile.yaml`: + +```yaml + - path: applications/fluentd/helmfile.yaml +``` + +You can also review the default values set for this chart in the +[`values.yaml`](https://github.com/helm/charts/blob/master/stable/fluentd/values.yaml) file. + +You can customize the installation of Fluentd by defining +`applications/fluentd/values.yaml` file in your cluster management +project. Refer to the +[configuration chart](https://github.com/helm/charts/tree/master/stable/fluentd#configuration) +for the current development release of Fluentd for all available configuration options. + +The configuration chart link points to the current development release, which +may differ from the version you have installed. To ensure compatibility, switch +to the specific branch or tag you are using. + +Support for installing the Fluentd managed application is provided by the +GitLab Container Security group. If you run into unknown issues, +[open a new issue](https://gitlab.com/gitlab-org/gitlab/-/issues/new), and ping at +least 2 people from the +[Container Security group](https://about.gitlab.com/handbook/product/categories/#container-security-group). diff --git a/doc/user/infrastructure/clusters/manage/management_project_applications/ingress.md b/doc/user/infrastructure/clusters/manage/management_project_applications/ingress.md new file mode 100644 index 00000000000..4f17dbab11b --- /dev/null +++ b/doc/user/infrastructure/clusters/manage/management_project_applications/ingress.md @@ -0,0 +1,31 @@ +--- +stage: Configure +group: Configure +info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/engineering/ux/technical-writing/#assignments +--- + +# Install Ingress with a cluster management project + +> [Introduced](https://gitlab.com/gitlab-org/project-templates/cluster-management/-/merge_requests/5) in GitLab 14.0. + +Assuming you already have a [Cluster management project](../../../../../user/clusters/management_project.md) created from a +[management project template](../../../../../user/clusters/management_project_template.md), to install Ingress you should +uncomment this line from your `helmfile.yaml`: + +```yaml + - path: applications/ingress/helmfile.yaml +``` + +Ingress is installed by default into the `gitlab-managed-apps` namespace +of your cluster. + +You can customize the installation of Ingress by updating the +`applications/ingress/values.yaml` file in your cluster +management project. Refer to the +[chart](https://github.com/helm/charts/tree/master/stable/nginx-ingress) +for the available configuration options. + +Support for installing the Ingress managed application is provided by the GitLab Configure group. +If you run into unknown issues, [open a new issue](https://gitlab.com/gitlab-org/gitlab/-/issues/new), +and ping at least 2 people from the +[Configure group](https://about.gitlab.com/handbook/product/categories/#configure-group). diff --git a/doc/user/infrastructure/clusters/manage/management_project_applications/prometheus.md b/doc/user/infrastructure/clusters/manage/management_project_applications/prometheus.md new file mode 100644 index 00000000000..19e6c76d133 --- /dev/null +++ b/doc/user/infrastructure/clusters/manage/management_project_applications/prometheus.md @@ -0,0 +1,32 @@ +--- +stage: Monitor +group: Monitor +info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/engineering/ux/technical-writing/#assignments +--- + +# Install Prometheus with a cluster management project + +> [Introduced](https://gitlab.com/gitlab-org/project-templates/cluster-management/-/merge_requests/5) in GitLab 14.0. + +[Prometheus](https://prometheus.io/docs/introduction/overview/) is an +open-source monitoring and alerting system for supervising your +deployed applications. + +Assuming you already have a [Cluster management project](../../../../../user/clusters/management_project.md) created from a +[management project template](../../../../../user/clusters/management_project_template.md), to install Prometheus you should +uncomment this line from your `helmfile.yaml`: + +```yaml + - path: applications/prometheus/helmfile.yaml +``` + +You can customize the installation of Prometheus by updating the +`applications/prometheus/values.yaml` file in your cluster +management project. Refer to the +[Configuration section](https://github.com/helm/charts/tree/master/stable/prometheus#configuration) +of the Prometheus chart's README for the available configuration options. + +Support for installing the Prometheus managed application is provided by the +GitLab APM group. If you run into unknown issues, +[open a new issue](https://gitlab.com/gitlab-org/gitlab/-/issues/new), and ping at +least 2 people from the [APM group](https://about.gitlab.com/handbook/product/categories/#apm-group). diff --git a/doc/user/infrastructure/clusters/manage/management_project_applications/runner.md b/doc/user/infrastructure/clusters/manage/management_project_applications/runner.md new file mode 100644 index 00000000000..56e01be68cb --- /dev/null +++ b/doc/user/infrastructure/clusters/manage/management_project_applications/runner.md @@ -0,0 +1,48 @@ +--- +stage: Verify +group: Runner +info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/engineering/ux/technical-writing/#assignments +--- + +# Install GitLab Runner with a cluster management project + +> [Introduced](https://gitlab.com/gitlab-org/project-templates/cluster-management/-/merge_requests/5) in GitLab 14.0. + +Assuming you already have a [Cluster management project](../../../../../user/clusters/management_project.md) created from a +[management project template](../../../../../user/clusters/management_project_template.md), to install GitLab Runner you should +uncomment this line from your `helmfile.yaml`: + +```yaml + - path: applications/gitlab-runner/helmfile.yaml +``` + +GitLab Runner is installed by default into the `gitlab-managed-apps` namespace of your cluster. + +For GitLab Runner to function, you _must_ specify the following in your +`applications/gitlab-runner/values.yaml.gotmpl` file: + +- `gitlabUrl`: The GitLab server full URL (for example, `https://gitlab.example.com`) + to register the Runner against. +- `runnerRegistrationToken`: The registration token for adding new runners to GitLab. + This must be [retrieved from your GitLab instance](../../../../../ci/runners/index.md). + +These values can be specified using [CI/CD variables](../../../../../ci/variables/index.md): + +- `GITLAB_RUNNER_GITLAB_URL` is used for `gitlabUrl`. +- `GITLAB_RUNNER_REGISTRATION_TOKEN` is used for `runnerRegistrationToken` + +The methods of specifying these values are mutually exclusive. Either specify variables `GITLAB_RUNNER_REGISTRATION_TOKEN` and `GITLAB_RUNNER_TOKEN` as CI variables (recommended) or provide values for `runnerRegistrationToken:` and `runnerToken:` in `applications/gitlab-runner/values.yaml.gotmpl`. + +The runner registration token allows connection to a project by a runner and therefore should be treated as a secret to prevent malicious use and code exfiltration through a runner. For this reason, we recommend that you specify the runner registration token as a [protected variable](../../../../../ci/variables/index.md#protect-a-cicd-variable) and [masked variable](../../../../../ci/variables/index.md#mask-a-cicd-variable) and do not commit them to the Git repository in the `values.yaml.gotmpl` file. + +You can customize the installation of GitLab Runner by defining +`applications/gitlab-runner/values.yaml.gotmpl` file in your cluster +management project. Refer to the +[chart](https://gitlab.com/gitlab-org/charts/gitlab-runner) for the +available configuration options. + +Support for installing the GitLab Runner managed application is provided by the +GitLab Runner group. If you run into unknown issues, +[open a new issue](https://gitlab.com/gitlab-org/gitlab/-/issues/new), and ping at +least 2 people from the +[Runner group](https://about.gitlab.com/handbook/product/categories/#runner-group). diff --git a/doc/user/infrastructure/clusters/manage/management_project_applications/sentry.md b/doc/user/infrastructure/clusters/manage/management_project_applications/sentry.md new file mode 100644 index 00000000000..4d82fe389d2 --- /dev/null +++ b/doc/user/infrastructure/clusters/manage/management_project_applications/sentry.md @@ -0,0 +1,76 @@ +--- +stage: Monitor +group: Monitor +info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/engineering/ux/technical-writing/#assignments +--- + +# Install Sentry with a cluster management project + +> [Introduced](https://gitlab.com/gitlab-org/project-templates/cluster-management/-/merge_requests/5) in GitLab 14.0. + +The Sentry Helm chart [recommends](https://github.com/helm/charts/blob/f6e5784f265dd459c5a77430185d0302ed372665/stable/sentry/values.yaml#L284-L285) +at least 3 GB of available RAM for database migrations. + +Assuming you already have a [Cluster management project](../../../../../user/clusters/management_project.md) created from a +[management project template](../../../../../user/clusters/management_project_template.md), to install Sentry you should +uncomment this line from your `helmfile.yaml`: + +```yaml + - path: applications/sentry/helmfile.yaml +``` + +Sentry is installed by default into the `gitlab-managed-apps` namespace +of your cluster. + +You can customize the installation of Sentry by defining +`applications/sentry/values.yaml` file in your cluster +management project. Refer to the +[chart](https://github.com/helm/charts/tree/master/stable/sentry) +for the available configuration options. + +We recommend you pay close attention to the following configuration options: + +- `email`. Needed to invite users to your Sentry instance and to send error emails. +- `user`. Where you can set the login credentials for the default administrator user. +- `postgresql`. For a PostgreSQL password that can be used when running future updates. + +When upgrading, it's important to provide the existing PostgreSQL password (given +using the `postgresql.postgresqlPassword` key) to avoid authentication errors. +Read the [PostgreSQL chart documentation](https://github.com/helm/charts/tree/master/stable/postgresql#upgrade) +for more information. + +Here is an example configuration for Sentry: + +```yaml +# Admin user to create +user: + # Indicated to create the admin user or not, + # Default is true as the initial installation. + create: true + email: "<your email>" + password: "<your password>" + +email: + from_address: "<your from email>" + host: smtp + port: 25 + use_tls: false + user: "<your email username>" + password: "<your email password>" + enable_replies: false + +ingress: + enabled: true + hostname: "<sentry.example.com>" + +# Needs to be here between runs. +# See https://github.com/helm/charts/tree/master/stable/postgresql#upgrade for more info +postgresql: + postgresqlPassword: example-postgresql-password +``` + +Support for installing the Sentry managed application is provided by the +GitLab Health group. If you run into unknown issues, +[open a new issue](https://gitlab.com/gitlab-org/gitlab/-/issues/new), and ping at +least 2 people from the +[Health group](https://about.gitlab.com/handbook/product/categories/#health-group). diff --git a/doc/user/infrastructure/clusters/manage/management_project_applications/vault.md b/doc/user/infrastructure/clusters/manage/management_project_applications/vault.md new file mode 100644 index 00000000000..291321963d0 --- /dev/null +++ b/doc/user/infrastructure/clusters/manage/management_project_applications/vault.md @@ -0,0 +1,108 @@ +--- +stage: Release +group: Release +info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/engineering/ux/technical-writing/#assignments +--- + +# Install Vault with a cluster management project + +> [Introduced](https://gitlab.com/gitlab-org/project-templates/cluster-management/-/merge_requests/5) in GitLab 14.0. + +[HashiCorp Vault](https://www.vaultproject.io/) is a secrets management solution which +can be used to safely manage and store passwords, credentials, certificates, and more. A Vault +installation could be leveraged to provide a single secure data store for credentials +used in your applications, GitLab CI/CD jobs, and more. It could also serve as a way of +providing SSL/TLS certificates to systems and deployments in your infrastructure. Leveraging +Vault as a single source for all these credentials allows greater security by having +a single source of access, control, and auditability around all your sensitive +credentials and certificates. This feature requires giving GitLab the highest level of access and +control. Therefore, if GitLab is compromised, the security of this Vault instance is as well. To +avoid this security risk, GitLab recommends using your own HashiCorp Vault to leverage +[external secrets with CI](../../../../../ci/secrets/index.md). + +Assuming you already have a [Cluster management project](../../../../../user/clusters/management_project.md) created from a +[management project template](../../../../../user/clusters/management_project_template.md), to install Vault you should +uncomment this line from your `helmfile.yaml`: + +```yaml + - path: applications/vault/helmfile.yaml +``` + +By default you receive a basic Vault setup with no scalable storage backend. This +is enough for simple testing and small-scale deployments, though has limits +to how much it can scale, and as it's a single instance deployment, upgrading the +Vault application causes downtime. + +To optimally use Vault in a production environment, it's ideal to have a good understanding +of the internals of Vault and how to configure it. This can be done by reading +the [Vault Configuration guide](../../../../../ci/secrets/#configure-your-vault-server), +the [Vault documentation](https://www.vaultproject.io/docs/internals) and +the Vault Helm chart [`values.yaml` file](https://github.com/hashicorp/vault-helm/blob/v0.3.3/values.yaml). + +At a minimum, most users set up: + +- A [seal](https://www.vaultproject.io/docs/configuration/seal) for extra encryption + of the main key. +- A [storage backend](https://www.vaultproject.io/docs/configuration/storage) that's + suitable for environment and storage security requirements. +- [HA Mode](https://www.vaultproject.io/docs/concepts/ha). +- The [Vault UI](https://www.vaultproject.io/docs/configuration/ui). + +The following is an example values file (`applications/vault/values.yaml`) +that configures Google Key Management Service for auto-unseal, using a Google Cloud Storage backend, enabling +the Vault UI, and enabling HA with 3 pod replicas. The `storage` and `seal` stanzas +below are examples and should be replaced with settings specific to your environment. + +```yaml +# Enable the Vault WebUI +ui: + enabled: true +server: + # Disable the built in data storage volume as it's not safe for High Availability mode + dataStorage: + enabled: false + # Enable High Availability Mode + ha: + enabled: true + # Configure Vault to listen on port 8200 for normal traffic and port 8201 for inter-cluster traffic + config: | + listener "tcp" { + tls_disable = 1 + address = "[::]:8200" + cluster_address = "[::]:8201" + } + # Configure Vault to store its data in a GCS Bucket backend + storage "gcs" { + path = "gcs://my-vault-storage/vault-bucket" + ha_enabled = "true" + } + # Configure Vault to unseal storage using a GKMS key + seal "gcpckms" { + project = "vault-helm-dev-246514" + region = "global" + key_ring = "vault-helm-unseal-kr" + crypto_key = "vault-helm-unseal-key" + } +``` + +After you have successfully installed Vault, you must +[initialize the Vault](https://learn.hashicorp.com/tutorials/vault/getting-started-deploy#initializing-the-vault) +and obtain the initial root token. You need access to your Kubernetes cluster that +Vault has been deployed into in order to do this. To initialize the Vault, get a +shell to one of the Vault pods running inside Kubernetes (typically this is done +by using the `kubectl` command line tool). After you have a shell into the pod, +run the `vault operator init` command: + +```shell +kubectl -n gitlab-managed-apps exec -it vault-0 sh +/ $ vault operator init +``` + +This should give you your unseal keys and initial root token. Make sure to note these down +and keep these safe, as they're required to unseal the Vault throughout its lifecycle. + +Support for installing the Vault managed application is provided by the +GitLab Release Management group. If you run into unknown issues, +[open a new issue](https://gitlab.com/gitlab-org/gitlab/-/issues/new), and ping at +least 2 people from the +[Release Management group](https://about.gitlab.com/handbook/product/categories/#release-management-group). diff --git a/doc/user/infrastructure/iac/index.md b/doc/user/infrastructure/iac/index.md new file mode 100644 index 00000000000..5b44490f78d --- /dev/null +++ b/doc/user/infrastructure/iac/index.md @@ -0,0 +1,162 @@ +--- +stage: Configure +group: Configure +info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/engineering/ux/technical-writing/#assignments +--- + +# Infrastructure as Code with Terraform and GitLab **(FREE)** + +## Motivation + +The Terraform integration features in GitLab enable your GitOps / Infrastructure-as-Code (IaC) +workflows to tie into GitLab authentication and authorization. These features focus on +lowering the barrier to entry for teams to adopt Terraform, collaborate effectively in +GitLab, and support Terraform best practices. + +## Quick Start + +Use the following `.gitlab-ci.yml` to set up a basic Terraform project integration +for GitLab versions 14.0 and later: + +```yaml +include: + - template: Terraform.gitlab-ci.yml + +variables: + # If not using GitLab's HTTP backend, remove this line and specify TF_HTTP_* variables + TF_STATE_NAME: default + TF_CACHE_KEY: default + # If your terraform files are in a subdirectory, set TF_ROOT accordingly + # TF_ROOT: terraform/production +``` + +This template includes some opinionated decisions, which you can override: + +- Including the latest [GitLab Terraform Image](https://gitlab.com/gitlab-org/terraform-images). +- Using the [GitLab managed Terraform State](#gitlab-managed-terraform-state) as + the Terraform state storage backend. +- Creating [four pipeline stages](https://gitlab.com/gitlab-org/gitlab/-/blob/master/lib/gitlab/ci/templates/Terraform.gitlab-ci.yml): + `init`, `validate`, `build`, and `deploy`. These stages + [run the Terraform commands](https://gitlab.com/gitlab-org/gitlab/-/blob/master/lib/gitlab/ci/templates/Terraform/Base.gitlab-ci.yml) + `init`, `validate`, `plan`, `plan-json`, and `apply`. The `apply` command only runs on the default branch. + +This video from January 2021 walks you through all the GitLab Terraform integration features: + +<div class="video-fallback"> + See the video: <a href="https://www.youtube.com/watch?v=iGXjUrkkzDI">Terraform with GitLab</a>. +</div> +<figure class="video-container"> + <iframe src="https://www.youtube.com/embed/iGXjUrkkzDI" frameborder="0" allowfullscreen="true"> </iframe> +</figure> + +## GitLab Managed Terraform state + +[Terraform remote backends](https://www.terraform.io/docs/language/settings/backends/index.html) +enable you to store the state file in a remote, shared store. GitLab uses the +[Terraform HTTP backend](https://www.terraform.io/docs/language/settings/backends/http.html) +to securely store the state files in local storage (the default) or +[the remote store of your choice](../../../administration/terraform_state.md). + +The GitLab managed Terraform state backend can store your Terraform state easily and +securely. It spares you from setting up additional remote resources like +Amazon S3 or Google Cloud Storage. Its features include: + +- Supporting encryption of the state file both in transit and at rest. +- Locking and unlocking state. +- Remote Terraform plan and apply execution. + +Read more on setting up and [using GitLab Managed Terraform states](../terraform_state.md) + +## Terraform module registry + +GitLab can be used as a [Terraform module registry](../../packages/terraform_module_registry/index.md) +to create and publish Terraform modules to a private registry specific to your +top-level namespace. + +## Terraform integration in Merge Requests + +Collaborating around Infrastructure as Code (IaC) changes requires both code changes +and expected infrastructure changes to be checked and approved. GitLab provides a +solution to help collaboration around Terraform code changes and their expected +effects using the Merge Request pages. This way users don't have to build custom +tools or rely on 3rd party solutions to streamline their IaC workflows. + +Read more on setting up and [using the merge request integrations](../mr_integration.md). + +## The GitLab Terraform provider + +WARNING: +The GitLab Terraform provider is released separately from GitLab. +We are working on migrating the GitLab Terraform provider for GitLab.com. + +You can use the [GitLab Terraform provider](https://github.com/gitlabhq/terraform-provider-gitlab) +to manage various aspects of GitLab using Terraform. The provider is an open source project, +owned by GitLab, where everyone can contribute. + +The [documentation of the provider](https://registry.terraform.io/providers/gitlabhq/gitlab/latest/docs) +is available as part of the official Terraform provider documentations. + +## Create a new cluster through IaC + +Learn how to [create a new cluster on Google Kubernetes Engine (GKE)](../clusters/connect/new_gke_cluster.md). + +## Troubleshooting + +### `gitlab_group_share_group` resources not detected when subgroup state is refreshed + +The GitLab Terraform provider can fail to detect existing `gitlab_group_share_group` resources +due to the issue ["User with permissions cannot retrieve `share_with_groups` from the API"](https://gitlab.com/gitlab-org/gitlab/-/issues/328428). +This results in an error when running `terraform apply` because Terraform attempts to recreate an +existing resource. + +For example, consider the following group/subgroup configuration: + +```plaintext +parent-group +├── subgroup-A +└── subgroup-B +``` + +Where: + +- User `user-1` creates `parent-group`, `subgroup-A`, and `subgroup-B`. +- `subgroup-A` is shared with `subgroup-B`. +- User `terraform-user` is member of `parent-group` with inherited `owner` access to both subgroups. + +When the Terraform state is refreshed, the API query `GET /groups/:subgroup-A_id` issued by the provider does not return the +details of `subgroup-B` in the `shared_with_groups` array. This leads to the error. + +To workaround this issue, make sure to apply one of the following conditions: + +1. The `terraform-user` creates all subgroup resources. +1. Grant Maintainer or Owner role to the `terraform-user` user on `subgroup-B`. +1. The `terraform-user` inherited access to `subgroup-B` and `subgroup-B` contains at least one project. + +### Invalid CI/CD syntax error when using the "latest" base template + +On GitLab 14.2 and later, you might get a CI/CD syntax error when using the +`latest` Base Terraform template: + +```yaml +include: + - template: Terraform/Base.latest.gitlab-ci.yml + +my-Terraform-job: + extends: .init +``` + +The base template's [jobs were renamed](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/67719/) +with better Terraform-specific names. To resolve the syntax error, you can: + +- Use the stable `Terraform/Base.gitlab-ci.yml` template, which has not changed. +- Update your pipeline configuration to use the new job names in + `https://gitlab.com/gitlab-org/gitlab/-/tree/master/lib/gitlab/ci/templates/Terraform/Base.latest.gitlab-ci.yml`. + For example: + + ```yaml + include: + - template: Terraform/Base.latest.gitlab-ci.yml + + my-Terraform-job: + extends: .terraform:init # The updated name. + ``` diff --git a/doc/user/infrastructure/img/terraform_list_view_actions_v13_8.png b/doc/user/infrastructure/img/terraform_list_view_actions_v13_8.png Binary files differdeleted file mode 100644 index 7d619b6ad7e..00000000000 --- a/doc/user/infrastructure/img/terraform_list_view_actions_v13_8.png +++ /dev/null diff --git a/doc/user/infrastructure/index.md b/doc/user/infrastructure/index.md index bdaae4b8225..b2d75a22615 100644 --- a/doc/user/infrastructure/index.md +++ b/doc/user/infrastructure/index.md @@ -4,136 +4,11 @@ group: Configure info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/engineering/ux/technical-writing/#assignments --- -# Infrastructure as code with Terraform and GitLab **(FREE)** +# Infrastructure management **(FREE)** -## Motivation +GitLab provides you with great solutions to help you manage your +infrastructure: -The Terraform integration features in GitLab enable your GitOps / Infrastructure-as-Code (IaC) -workflows to tie into GitLab authentication and authorization. These features focus on -lowering the barrier to entry for teams to adopt Terraform, collaborate effectively in -GitLab, and support Terraform best practices. - -## Quick Start - -Use the following `.gitlab-ci.yml` to set up a basic Terraform project integration -for GitLab versions 14.0 and later: - -```yaml -include: - - template: Terraform.gitlab-ci.yml - -variables: - # If not using GitLab's HTTP backend, remove this line and specify TF_HTTP_* variables - TF_STATE_NAME: default - TF_CACHE_KEY: default - # If your terraform files are in a subdirectory, set TF_ROOT accordingly - # TF_ROOT: terraform/production -``` - -This template includes some opinionated decisions, which you can override: - -- Including the latest [GitLab Terraform Image](https://gitlab.com/gitlab-org/terraform-images). -- Using the [GitLab managed Terraform State](#gitlab-managed-terraform-state) as - the Terraform state storage backend. -- Creating [four pipeline stages](https://gitlab.com/gitlab-org/gitlab/-/blob/master/lib/gitlab/ci/templates/Terraform.gitlab-ci.yml): - `init`, `validate`, `build`, and `deploy`. These stages - [run the Terraform commands](https://gitlab.com/gitlab-org/gitlab/-/blob/master/lib/gitlab/ci/templates/Terraform/Base.gitlab-ci.yml) - `init`, `validate`, `plan`, `plan-json`, and `apply`. The `apply` command only runs on the default branch. - -This video from January 2021 walks you through all the GitLab Terraform integration features: - -<div class="video-fallback"> - See the video: <a href="https://www.youtube.com/watch?v=iGXjUrkkzDI">Terraform with GitLab</a>. -</div> -<figure class="video-container"> - <iframe src="https://www.youtube.com/embed/iGXjUrkkzDI" frameborder="0" allowfullscreen="true"> </iframe> -</figure> - -## GitLab Managed Terraform state - -[Terraform remote backends](https://www.terraform.io/docs/language/settings/backends/index.html) -enable you to store the state file in a remote, shared store. GitLab uses the -[Terraform HTTP backend](https://www.terraform.io/docs/language/settings/backends/http.html) -to securely store the state files in local storage (the default) or -[the remote store of your choice](../../administration/terraform_state.md). - -The GitLab managed Terraform state backend can store your Terraform state easily and -securely. It spares you from setting up additional remote resources like -Amazon S3 or Google Cloud Storage. Its features include: - -- Supporting encryption of the state file both in transit and at rest. -- Locking and unlocking state. -- Remote Terraform plan and apply execution. - -Read more on setting up and [using GitLab Managed Terraform states](terraform_state.md) - -WARNING: -Like any other job artifact, Terraform plan data is [viewable by anyone with Guest access](../permissions.md) to the repository. -Neither Terraform nor GitLab encrypts the plan file by default. If your Terraform plan -includes sensitive data such as passwords, access tokens, or certificates, GitLab strongly -recommends encrypting plan output or modifying the project visibility settings. - -## Terraform module registry - -GitLab can be used as a [Terraform module registry](../packages/terraform_module_registry/index.md) -to create and publish Terraform modules to a private registry specific to your -top-level namespace. - -## Terraform integration in Merge Requests - -Collaborating around Infrastructure as Code (IaC) changes requires both code changes -and expected infrastructure changes to be checked and approved. GitLab provides a -solution to help collaboration around Terraform code changes and their expected -effects using the Merge Request pages. This way users don't have to build custom -tools or rely on 3rd party solutions to streamline their IaC workflows. - -Read more on setting up and [using the merge request integrations](mr_integration.md). - -## The GitLab Terraform provider - -WARNING: -The GitLab Terraform provider is released separately from GitLab. -We are working on migrating the GitLab Terraform provider for GitLab.com. - -You can use the [GitLab Terraform provider](https://github.com/gitlabhq/terraform-provider-gitlab) -to manage various aspects of GitLab using Terraform. The provider is an open source project, -owned by GitLab, where everyone can contribute. - -The [documentation of the provider](https://registry.terraform.io/providers/gitlabhq/gitlab/latest/docs) -is available as part of the official Terraform provider documentations. - -## Create a new cluster through IaC - -Learn how to [create a new cluster on Google Kubernetes Engine (GKE)](clusters/connect/new_gke_cluster.md). - -## Troubleshooting - -### `gitlab_group_share_group` resources not detected when subgroup state is refreshed - -The GitLab Terraform provider can fail to detect existing `gitlab_group_share_group` resources -due to the issue ["User with permissions cannot retrieve `share_with_groups` from the API"](https://gitlab.com/gitlab-org/gitlab/-/issues/328428). -This results in an error when running `terraform apply` because Terraform attempts to recreate an -existing resource. - -For example, consider the following group/subgroup configuration: - -```plaintext -parent-group -├── subgroup-A -└── subgroup-B -``` - -Where: - -- User `user-1` creates `parent-group`, `subgroup-A`, and `subgroup-B`. -- `subgroup-A` is shared with `subgroup-B`. -- User `terraform-user` is member of `parent-group` with inherited `owner` access to both subgroups. - -When the Terraform state is refreshed, the API query `GET /groups/:subgroup-A_id` issued by the provider does not return the -details of `subgroup-B` in the `shared_with_groups` array. This leads to the error. - -To workaround this issue, make sure to apply one of the following conditions: - -1. The `terraform-user` creates all subgroup resources. -1. Grant Maintainer or Owner role to the `terraform-user` user on `subgroup-B`. -1. The `terraform-user` inherited access to `subgroup-B` and `subgroup-B` contains at least one project. +- [Infrastructure as Code and GitOps](iac/index.md) +- [Kubernetes clusters](../project/clusters/index.md) +- [Runbooks](../project/clusters/runbooks/index.md) diff --git a/doc/user/infrastructure/mr_integration.md b/doc/user/infrastructure/mr_integration.md index 66e00bab6ce..29bf218b109 100644 --- a/doc/user/infrastructure/mr_integration.md +++ b/doc/user/infrastructure/mr_integration.md @@ -15,12 +15,17 @@ you can expose details from `terraform plan` runs directly into a merge request enabling you to see statistics about the resources that Terraform creates, modifies, or destroys. -## Setup +WARNING: +Like any other job artifact, Terraform Plan data is [viewable by anyone with Guest access](../permissions.md) to the repository. +Neither Terraform nor GitLab encrypts the plan file by default. If your Terraform Plan +includes sensitive data such as passwords, access tokens, or certificates, we strongly +recommend encrypting plan output or modifying the project visibility settings. -NOTE: -GitLab ships with a [pre-built CI template](index.md#quick-start) that uses GitLab Managed Terraform state and integrates Terraform changes into merge requests. We recommend customizing the pre-built image and relying on the `gitlab-terraform` helper provided within for a quick setup. +## Configure Terraform report artifacts -To manually configure a GitLab Terraform Report artifact requires the following steps: +GitLab ships with a [pre-built CI template](iac/index.md#quick-start) that uses GitLab Managed Terraform state and integrates Terraform changes into merge requests. We recommend customizing the pre-built image and relying on the `gitlab-terraform` helper provided within for a quick setup. + +To manually configure a GitLab Terraform Report artifact: 1. For simplicity, let's define a few reusable variables to allow us to refer to these files multiple times: diff --git a/doc/user/infrastructure/terraform_state.md b/doc/user/infrastructure/terraform_state.md index 57db2b47de4..179f9677b96 100644 --- a/doc/user/infrastructure/terraform_state.md +++ b/doc/user/infrastructure/terraform_state.md @@ -14,6 +14,11 @@ enable you to store the state file in a remote, shared store. GitLab uses the to securely store the state files in local storage (the default) or [the remote store of your choice](../../administration/terraform_state.md). +WARNING: +Using local storage (the default) on clustered deployments of GitLab will result in +a split state across nodes, making subsequent executions of Terraform inconsistent. +You are highly advised to use a remote storage in that case. + The GitLab managed Terraform state backend can store your Terraform state easily and securely, and spares you from setting up additional remote resources like Amazon S3 or Google Cloud Storage. Its features include: @@ -83,6 +88,14 @@ local machine, this is a simple way to get started: -backend-config="retry_wait_min=5" ``` +If you already have a GitLab-managed Terraform state, you can use the `terraform init` command +with the prepopulated parameters values: + +1. On the top bar, select **Menu > Projects** and find your project. +1. On the left sidebar, select **Infrastructure > Terraform**. +1. Next to the environment you want to use, select the [Actions menu](#managing-state-files) + **{ellipsis_v}** and select **Copy Terraform init command**. + You can now run `terraform plan` and `terraform apply` as you normally would. ### Get started using GitLab CI @@ -222,7 +235,7 @@ An example setup is shown below: ```plaintext example_remote_state_address=https://gitlab.com/api/v4/projects/<TARGET-PROJECT-ID>/terraform/state/<TARGET-STATE-NAME> example_username=<GitLab username> - example_access_token=<GitLab Personal Acceess Token> + example_access_token=<GitLab Personal Access Token> ``` 1. Define the data source by adding the following code block in a `.tf` file (such as `data.tf`): @@ -362,10 +375,8 @@ contains these fields: state file is locked. - **Pipeline**: A link to the most recent pipeline and its status. - **Details**: Information about when the state file was created or changed. -- **Actions**: Actions you can take on the state file, including downloading, - locking, unlocking, or [removing](#remove-a-state-file) the state file and versions: - -  +- **Actions**: Actions you can take on the state file, including copying the `terraform init` command, + downloading, locking, unlocking, or [removing](#remove-a-state-file) the state file and versions. NOTE: Additional improvements to the |
