diff options
Diffstat (limited to 'doc/topics')
-rw-r--r-- | doc/topics/authentication/index.md | 2 | ||||
-rw-r--r-- | doc/topics/autodevops/cloud_deployments/auto_devops_with_eks.md | 301 | ||||
-rw-r--r-- | doc/topics/autodevops/cloud_deployments/auto_devops_with_gke.md | 2 | ||||
-rw-r--r-- | doc/topics/autodevops/index.md | 1 | ||||
-rw-r--r-- | doc/topics/autodevops/upgrading_postgresql.md | 87 | ||||
-rw-r--r-- | doc/topics/awesome_co.md | 70 | ||||
-rw-r--r-- | doc/topics/git/bisect.md | 79 | ||||
-rw-r--r-- | doc/topics/git/feature_branching.md | 34 | ||||
-rw-r--r-- | doc/topics/git/git_log.md | 63 | ||||
-rw-r--r-- | doc/topics/git/index.md | 8 | ||||
-rw-r--r-- | doc/topics/git/merge_conflicts.md | 11 | ||||
-rw-r--r-- | doc/topics/git/partial_clone.md | 4 | ||||
-rw-r--r-- | doc/topics/git/subtree.md | 53 | ||||
-rw-r--r-- | doc/topics/git/tags.md | 44 | ||||
-rw-r--r-- | doc/topics/git/troubleshooting_git.md | 16 | ||||
-rw-r--r-- | doc/topics/git/unstage.md | 14 | ||||
-rw-r--r-- | doc/topics/git/useful_git_commands.md | 6 | ||||
-rw-r--r-- | doc/topics/offline/quick_start_guide.md | 86 | ||||
-rw-r--r-- | doc/topics/set_up_organization.md | 2 | ||||
-rw-r--r-- | doc/topics/your_work.md | 5 |
20 files changed, 567 insertions, 321 deletions
diff --git a/doc/topics/authentication/index.md b/doc/topics/authentication/index.md index c1d0a69e1f4..55a58377b76 100644 --- a/doc/topics/authentication/index.md +++ b/doc/topics/authentication/index.md @@ -12,7 +12,7 @@ This page gathers all the resources for the topic **Authentication** within GitL - [SSH](../../user/ssh.md) - [Two-factor authentication](../../user/profile/account/two_factor_authentication.md) -- [Why do you keep getting signed out?](../../user/profile/index.md#why-do-you-keep-getting-signed-out) +- [Stay signed in indefinitely](../../user/profile/index.md#stay-signed-in-indefinitely) - **Articles:** - [Support for Universal 2nd Factor Authentication - YubiKeys](https://about.gitlab.com/blog/2016/06/22/gitlab-adds-support-for-u2f/) - [Security Webcast with Yubico](https://about.gitlab.com/blog/2016/08/31/gitlab-and-yubico-security-webcast/) diff --git a/doc/topics/autodevops/cloud_deployments/auto_devops_with_eks.md b/doc/topics/autodevops/cloud_deployments/auto_devops_with_eks.md new file mode 100644 index 00000000000..30cbe06ab95 --- /dev/null +++ b/doc/topics/autodevops/cloud_deployments/auto_devops_with_eks.md @@ -0,0 +1,301 @@ +--- +stage: Configure +group: Configure +info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments +--- + +# Use Auto DevOps to deploy an application to Amazon Elastic Kubernetes Service (EKS) + +In this tutorial, we'll help you to get started with [Auto DevOps](../index.md) +through an example of how to deploy an application to Amazon Elastic Kubernetes Service (EKS). + +The tutorial uses the GitLab native Kubernetes integration, so you don't need +to create a Kubernetes cluster manually using the AWS console. + +You can also follow this tutorial on a self-managed instance. +Ensure your own [runners are configured](../../../ci/runners/index.md). + +To deploy a project to EKS: + +1. [Configure your Amazon account](#configure-your-amazon-account) +1. [Create a Kubernetes cluster and deploy the agent](#create-a-kubernetes-cluster) +1. [Create a new project from a template](#create-an-application-project-from-a-template) +1. [Configure the agent](#configure-the-agent) +1. [Install Ingress](#install-ingress) +1. [Configure Auto DevOps](#configure-auto-devops) +1. [Enable Auto DevOps and run the pipeline](#enable-auto-devops-and-run-the-pipeline) +1. [Deploy the application](#deploy-the-application) + +## Configure your Amazon account + +Before you create and connect your Kubernetes cluster to your GitLab project, +you need an [Amazon Web Services account](https://https://aws.amazon.com/). +Sign in with an existing Amazon account or create a new one. + +## Create a Kubernetes cluster + +To create an new cluster on Amazon EKS: + +- Follow the steps in [Create an Amazon EKS cluster](../../../user/infrastructure/clusters/connect/new_eks_cluster.md). + +If you prefer, you can also create a cluster manually using `eksctl`. + +## Create an application project from a template + +Use a GitLab project template to get started. As the name suggests, +those projects provide a bare-bones application built on some well-known frameworks. + +WARNING: +Create the application project in the group hierarchy at the same level or below the project for cluster management. Otherwise, it fails to [authorize the agent](../../../user/clusters/agent/ci_cd_workflow.md#authorize-the-agent). + +1. On the top bar in GitLab, select the plus icon (**{plus-square}**), and select + **New project/repository**. +1. Select **Create from template**. +1. Select the **Ruby on Rails** template. +1. Give your project a name, optionally a description, and make it public so that + you can take advantage of the features available in the + [GitLab Ultimate plan](https://about.gitlab.com/pricing/). +1. Select **Create project**. + +Now you have an application project you are going to deploy to the EKS cluster. + +## Configure the agent + +Next, we'll configure the GitLab agent for Kubernetes so we can use it to deploy the application project. + +1. Go to the project [we created to manage the cluster](#create-a-kubernetes-cluster). +1. Navigate to the [agent configuration file](../../../user/clusters/agent/install/index.md#create-an-agent-configuration-file) (`.gitlab/agents/eks-agent/config.yaml`) and edit it. +1. Configure `ci_access:projects` attribute. Use the application project path as `id`: + +```yaml +ci_access: + projects: + - id: path/to/application-project +``` + +## Install Ingress + +After your cluster is running, you must install NGINX Ingress Controller as a +load balancer to route traffic from the internet to your application. +Install the NGINX Ingress Controller +through the GitLab [Cluster management project template](../../../user/clusters/management_project_template.md), +or manually via the command line: + +1. Ensure you have `kubectl` and Helm installed on your machine. +1. Create an IAM role to access the cluster. +1. Create an access token to access the cluster. +1. Use `kubectl` to connect to your cluster: + + ```shell + helm upgrade --install ingress-nginx ingress-nginx \ + --repo https://kubernetes.github.io/ingress-nginx \ + --namespace gitlab-managed-apps --create-namespace + + # Check that the ingress controller is installed successfully + kubectl get service ingress-nginx-controller -n gitlab-managed-apps + ``` + +## Configure Auto DevOps + +Follow these steps to configure the base domain and other settings required for Auto DevOps. + +1. A few minutes after you install NGINX, the load balancer obtains an IP address, and you can + get the external IP address with the following command: + + ```shell + kubectl get all -n gitlab-managed-apps --selector app.kubernetes.io/instance=ingress-nginx + ``` + + Replace `gitlab-managed-apps` if you have overwritten your namespace. + + Next, find the actual external IP address for your cluster with the following command: + + ```shell + nslookup [External IP] + ``` + + Where the `[External IP]` is the hostname found with the previous command. + + The IP address might be listed in the `Non-authoritative answer:` section of the response. + + Copy this IP address, as you need it in the next step. + +1. Go back to the application project. +1. On the left sidebar, select **Settings > CI/CD** and expand **Variables**. + - Add a key called `KUBE_INGRESS_BASE_DOMAIN` with the application deployment domain as the value. For this example, use the domain `<IP address>.nip.io`. + - Add a key called `KUBE_NAMESPACE` with a value of the Kubernetes namespace for your deployments to target. You can use different namespaces per environment. Configure the environment, use the environment scope. + - Add a key called `KUBE_CONTEXT` with a value like `path/to/agent/project:eks-agent`. Select the environment scope of your choice. + - Select **Save changes**. + +## Enable Auto DevOps and run the pipeline + +While Auto DevOps is enabled by default, Auto DevOps can be disabled at both +the instance level (for self-managed instances) and the group level. Complete +these steps to enable Auto DevOps if it's disabled: + +1. On the top bar, select **Main menu > Projects** and find the application project. +1. On the left sidebar, select **Settings > CI/CD**. +1. Expand **Auto DevOps**. +1. Select **Default to Auto DevOps pipeline** to display more options. +1. In **Deployment strategy**, select your desired [continuous deployment strategy](../requirements.md#auto-devops-deployment-strategy) + to deploy the application to production after the pipeline successfully runs on the default branch. +1. Select **Save changes**. +1. Edit `.gitlab-ci.yml` file to include the Auto DevOps template and commit the change to the default branch: + + ```yaml + include: + - template: Auto-DevOps.gitlab-ci.yml + ``` + +The commit should trigger a pipeline. In the next section, we explain what each job does in the pipeline. + +## Deploy the application + +When your pipeline runs, what is it doing? + +To view the jobs in the pipeline, select the pipeline's status badge. The +**{status_running}** icon displays when pipeline jobs are running, and updates +without refreshing the page to **{status_success}** (for success) or +**{status_failed}** (for failure) when the jobs complete. + +The jobs are separated into stages: + +![Pipeline stages](img/guide_pipeline_stages_v13_0.png) + +- **Build** - The application builds a Docker image and uploads it to your project's + [Container Registry](../../../user/packages/container_registry/index.md) ([Auto Build](../stages.md#auto-build)). +- **Test** - GitLab runs various checks on the application, but all jobs except `test` + are allowed to fail in the test stage: + + - The `test` job runs unit and integration tests by detecting the language and + framework ([Auto Test](../stages.md#auto-test-deprecated)) + - The `code_quality` job checks the code quality and is allowed to fail + ([Auto Code Quality](../stages.md#auto-code-quality)) + - The `container_scanning` job checks the Docker container if it has any + vulnerabilities and is allowed to fail ([Auto Container Scanning](../stages.md#auto-container-scanning)) + - The `dependency_scanning` job checks if the application has any dependencies + susceptible to vulnerabilities and is allowed to fail + ([Auto Dependency Scanning](../stages.md#auto-dependency-scanning)) + - Jobs suffixed with `-sast` run static analysis on the current code to check for potential + security issues, and are allowed to fail ([Auto SAST](../stages.md#auto-sast)) + - The `secret-detection` job checks for leaked secrets and is allowed to fail ([Auto Secret Detection](../stages.md#auto-secret-detection)) + - The `license_scanning` job searches the application's dependencies to determine each of their + licenses and is allowed to fail + ([Auto License Compliance](../stages.md#auto-license-compliance)) + +- **Review** - Pipelines on the default branch include this stage with a `dast_environment_deploy` job. + To learn more, see [Dynamic Application Security Testing (DAST)](../../../user/application_security/dast/index.md). + +- **Production** - After the tests and checks finish, the application deploys in + Kubernetes ([Auto Deploy](../stages.md#auto-deploy)). + +- **Performance** - Performance tests are run on the deployed application + ([Auto Browser Performance Testing](../stages.md#auto-browser-performance-testing)). + +- **Cleanup** - Pipelines on the default branch include this stage with a `stop_dast_environment` job. + +After running a pipeline, you should view your deployed website and learn how +to monitor it. + +### Monitor your project + +After successfully deploying your application, you can view its website and check +on its health on the **Environments** page by navigating to +**Deployments > Environments**. This page displays details about +the deployed applications, and the right-hand column displays icons that link +you to common environment tasks: + +![Environments](img/guide_environments_v12_3.png) + +- **Open live environment** (**{external-link}**) - Opens the URL of the application deployed in production +- **Monitoring** (**{chart}**) - Opens the metrics page where Prometheus collects data + about the Kubernetes cluster and how the application + affects it in terms of memory usage, CPU usage, and latency +- **Deploy to** (**{play}** **{chevron-lg-down}**) - Displays a list of environments you can deploy to +- **Terminal** (**{terminal}**) - Opens a [web terminal](../../../ci/environments/index.md#web-terminals-deprecated) + session inside the container where the application is running +- **Re-deploy to environment** (**{repeat}**) - For more information, see + [Retrying and rolling back](../../../ci/environments/index.md#retry-or-roll-back-a-deployment) +- **Stop environment** (**{stop}**) - For more information, see + [Stopping an environment](../../../ci/environments/index.md#stop-an-environment) + +GitLab displays the [deploy board](../../../user/project/deploy_boards.md) below the +environment's information, with squares representing pods in your +Kubernetes cluster, color-coded to show their status. Hovering over a square on +the deploy board displays the state of the deployment, and selecting the square +takes you to the pod's logs page. + +Although the example shows only one pod hosting the application at the moment, you can add +more pods by defining the [`REPLICAS` CI/CD variable](../cicd_variables.md) +in **Settings > CI/CD > Variables**. + +### Work with branches + +Following the [GitLab flow](../../gitlab_flow.md#working-with-feature-branches), +you should next create a feature branch to add content to your application: + +1. In your project's repository, go to the following file: `app/views/welcome/index.html.erb`. + This file should only contain a paragraph: `<p>You're on Rails!</p>`. +1. Open the GitLab [Web IDE](../../../user/project/web_ide/index.md) to make the change. +1. Edit the file so it contains: + + ```html + <p>You're on Rails! Powered by GitLab Auto DevOps.</p> + ``` + +1. Stage the file. Add a commit message, then create a new branch and a merge request + by selecting **Commit**. + + ![Web IDE commit](img/guide_ide_commit_v12_3.png) + +After submitting the merge request, GitLab runs your pipeline, and all the jobs +in it, as [described previously](#deploy-the-application), in addition to +a few more that run only on branches other than the default branch. + +After a few minutes a test fails, which means a test was +'broken' by your change. Select the failed `test` job to see more information +about it: + +```plaintext +Failure: +WelcomeControllerTest#test_should_get_index [/app/test/controllers/welcome_controller_test.rb:7]: +<You're on Rails!> expected but was +<You're on Rails! Powered by GitLab Auto DevOps.>.. +Expected 0 to be >= 1. + +bin/rails test test/controllers/welcome_controller_test.rb:4 +``` + +To fix the broken test: + +1. Return to your merge request. +1. In the upper right corner, select **Code**, then select **Open in Gitpod**. +1. In the left-hand directory of files, find the `test/controllers/welcome_controller_test.rb` + file, and select it to open it. +1. Change line 7 to say `You're on Rails! Powered by GitLab Auto DevOps.` +1. Select **Commit**. +1. In the left-hand column, under **Unstaged changes**, select the checkmark icon + (**{stage-all}**) to stage the changes. +1. Write a commit message, and select **Commit**. + +Return to the **Overview** page of your merge request, and you should not only +see the test passing, but also the application deployed as a +[review application](../stages.md#auto-review-apps). You can visit it by selecting +the **View app** **{external-link}** button to see your changes deployed. + +After merging the merge request, GitLab runs the pipeline on the default branch, +and then deploys the application to production. + +## Conclusion + +After implementing this project, you should have a solid understanding of the basics of Auto DevOps. +You started from building and testing, to deploying and monitoring an application +all in GitLab. Despite its automatic nature, Auto DevOps can also be configured +and customized to fit your workflow. Here are some helpful resources for further reading: + +1. [Auto DevOps](../index.md) +1. [Multiple Kubernetes clusters](../multiple_clusters_auto_devops.md) +1. [Incremental rollout to production](../cicd_variables.md#incremental-rollout-to-production) +1. [Disable jobs you don't need with CI/CD variables](../cicd_variables.md) +1. [Use your own buildpacks to build your application](../customize.md#custom-buildpacks) +1. [Prometheus monitoring](../../../user/project/integrations/prometheus.md) diff --git a/doc/topics/autodevops/cloud_deployments/auto_devops_with_gke.md b/doc/topics/autodevops/cloud_deployments/auto_devops_with_gke.md index 9bd1d30e1b1..06b772cc455 100644 --- a/doc/topics/autodevops/cloud_deployments/auto_devops_with_gke.md +++ b/doc/topics/autodevops/cloud_deployments/auto_devops_with_gke.md @@ -274,7 +274,7 @@ bin/rails test test/controllers/welcome_controller_test.rb:4 To fix the broken test: 1. Return to your merge request. -1. In the upper right corner, select **Code**, then select **Open in Gitpod**. +1. In the upper-right corner, select **Code**, then select **Open in Gitpod**. 1. In the left-hand directory of files, find the `test/controllers/welcome_controller_test.rb` file, and select it to open it. 1. Change line 7 to say `You're on Rails! Powered by GitLab Auto DevOps.` diff --git a/doc/topics/autodevops/index.md b/doc/topics/autodevops/index.md index 588be855659..2a0cc7f39b1 100644 --- a/doc/topics/autodevops/index.md +++ b/doc/topics/autodevops/index.md @@ -185,6 +185,7 @@ and clear the **Default to Auto DevOps pipeline** checkbox. ### Deploy your app to a cloud provider - [Use Auto DevOps to deploy to a Kubernetes cluster on Google Kubernetes Engine (GKE)](cloud_deployments/auto_devops_with_gke.md) +- [Use Auto DevOps to deploy to a Kubernetes cluster on Amazon Elastic Kubernetes Service (EKS)](cloud_deployments/auto_devops_with_eks.md) - [Use Auto DevOps to deploy to EC2](cloud_deployments/auto_devops_with_ec2.md) - [Use Auto DevOps to deploy to ECS](cloud_deployments/auto_devops_with_ecs.md) diff --git a/doc/topics/autodevops/upgrading_postgresql.md b/doc/topics/autodevops/upgrading_postgresql.md index f18d5c49be5..daf1d5341d3 100644 --- a/doc/topics/autodevops/upgrading_postgresql.md +++ b/doc/topics/autodevops/upgrading_postgresql.md @@ -49,12 +49,12 @@ being modified after the database dump is created. 1. Get the Kubernetes namespace for the environment. It typically looks like `<project-name>-<project-id>-<environment>`. In our example, the namespace is called `minimal-ruby-app-4349298-production`. - ```shell - $ kubectl get ns + ```shell + $ kubectl get ns - NAME STATUS AGE - minimal-ruby-app-4349298-production Active 7d14h - ``` + NAME STATUS AGE + minimal-ruby-app-4349298-production Active 7d14h + ``` 1. For ease of use, export the namespace name: @@ -64,20 +64,20 @@ being modified after the database dump is created. 1. Get the deployment name for your application with the following command. In our example, the deployment name is `production`. - ```shell - $ kubectl get deployment --namespace "$APP_NAMESPACE" - NAME READY UP-TO-DATE AVAILABLE AGE - production 2/2 2 2 7d21h - production-postgres 1/1 1 1 7d21h - ``` + ```shell + $ kubectl get deployment --namespace "$APP_NAMESPACE" + NAME READY UP-TO-DATE AVAILABLE AGE + production 2/2 2 2 7d21h + production-postgres 1/1 1 1 7d21h + ``` 1. To prevent the database from being modified, set replicas to 0 for the deployment with the following command. We use the deployment name from the previous step (`deployments/<DEPLOYMENT_NAME>`). - ```shell - $ kubectl scale --replicas=0 deployments/production --namespace "$APP_NAMESPACE" - deployment.extensions/production scaled - ``` + ```shell + $ kubectl scale --replicas=0 deployments/production --namespace "$APP_NAMESPACE" + deployment.extensions/production scaled + ``` 1. You must also set replicas to zero for workers if you have any. @@ -85,26 +85,26 @@ being modified after the database dump is created. 1. Get the service name for PostgreSQL. The name of the service should end with `-postgres`. In our example the service name is `production-postgres`. - ```shell - $ kubectl get svc --namespace "$APP_NAMESPACE" - NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE - production-auto-deploy ClusterIP 10.30.13.90 <none> 5000/TCP 7d14h - production-postgres ClusterIP 10.30.4.57 <none> 5432/TCP 7d14h - ``` + ```shell + $ kubectl get svc --namespace "$APP_NAMESPACE" + NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE + production-auto-deploy ClusterIP 10.30.13.90 <none> 5000/TCP 7d14h + production-postgres ClusterIP 10.30.4.57 <none> 5432/TCP 7d14h + ``` 1. Get the pod name for PostgreSQL with the following command. In our example, the pod name is `production-postgres-5db86568d7-qxlxv`. - ```shell - $ kubectl get pod --namespace "$APP_NAMESPACE" -l app=production-postgres - NAME READY STATUS RESTARTS AGE - production-postgres-5db86568d7-qxlxv 1/1 Running 0 7d14h - ``` + ```shell + $ kubectl get pod --namespace "$APP_NAMESPACE" -l app=production-postgres + NAME READY STATUS RESTARTS AGE + production-postgres-5db86568d7-qxlxv 1/1 Running 0 7d14h + ``` 1. Connect to the pod with: - ```shell - kubectl exec -it production-postgres-5db86568d7-qxlxv --namespace "$APP_NAMESPACE" -- bash - ``` + ```shell + kubectl exec -it production-postgres-5db86568d7-qxlxv --namespace "$APP_NAMESPACE" -- bash + ``` 1. Once, connected, create a dump file with the following command. @@ -114,20 +114,20 @@ being modified after the database dump is created. - When prompted for the database password, the default is `testing-password`. - ```shell - ## Format is: - # pg_dump -h SERVICE_NAME -U USERNAME DATABASE_NAME > /tmp/backup.sql + ```shell + ## Format is: + # pg_dump -h SERVICE_NAME -U USERNAME DATABASE_NAME > /tmp/backup.sql - pg_dump -h production-postgres -U user production > /tmp/backup.sql - ``` + pg_dump -h production-postgres -U user production > /tmp/backup.sql + ``` 1. Once the backup dump is complete, exit the Kubernetes exec process with `Control-D` or `exit`. 1. Download the dump file with the following command: - ```shell - kubectl cp --namespace "$APP_NAMESPACE" production-postgres-5db86568d7-qxlxv:/tmp/backup.sql backup.sql - ``` + ```shell + kubectl cp --namespace "$APP_NAMESPACE" production-postgres-5db86568d7-qxlxv:/tmp/backup.sql backup.sql + ``` ## Retain persistent volumes @@ -184,8 +184,7 @@ You can also 1. Set `AUTO_DEVOPS_POSTGRES_DELETE_V1` to a non-empty value. This flag is a safeguard to prevent accidental deletion of databases. <!-- DO NOT REPLACE when upgrading GitLab's supported version. This is NOT related to GitLab's PostgreSQL version support, but the one deployed by Auto DevOps. --> -1. If you have a `POSTGRES_VERSION` set, make sure it is set to `9.6.16` *or -higher*. This is the +1. If you have a `POSTGRES_VERSION` set, make sure it is set to `9.6.16` *or higher*. This is the minimum PostgreSQL version supported by Auto DevOps. See also the list of [tags available](https://hub.docker.com/r/bitnami/postgresql/tags). 1. Set `PRODUCTION_REPLICAS` to `0`. For other environments, use @@ -205,11 +204,11 @@ higher*. This is the 1. Get the pod name for the new PostgreSQL, in our example, the pod name is `production-postgresql-0`: - ```shell - $ kubectl get pod --namespace "$APP_NAMESPACE" -l app=postgresql - NAME READY STATUS RESTARTS AGE - production-postgresql-0 1/1 Running 0 19m - ```` + ```shell + $ kubectl get pod --namespace "$APP_NAMESPACE" -l app=postgresql + NAME READY STATUS RESTARTS AGE + production-postgresql-0 1/1 Running 0 19m + ```` 1. Copy the dump file from the backup steps to the pod: diff --git a/doc/topics/awesome_co.md b/doc/topics/awesome_co.md index ff3c81a1b90..ffda564cd91 100644 --- a/doc/topics/awesome_co.md +++ b/doc/topics/awesome_co.md @@ -141,3 +141,73 @@ create(:project, name: 'Throws Error', namespace: create(:group, name: 'Some Gro create(:project, name: 'No longer throws error', owner: @owner, namespace: create(:group, name: 'Some Group')) create(:epic, group: create(:group), author: @owner) ``` + +## YAML Factories + +### Generator to generate _n_ amount of records + +### [Group Labels](https://gitlab.com/gitlab-org/gitlab/-/blob/master/spec/factories/labels.rb) + +```yaml +group_labels: + # Group Label with Name and a Color + - name: Group Label 1 + group_id: <%= @group.id %> + color: "#FF0000" +``` + +### [Group Milestones](https://gitlab.com/gitlab-org/gitlab/-/blob/master/spec/factories/milestones.rb) + +```yaml +group_milestones: + # Past Milestone + - name: Past Milestone + group_id: <%= @group.id %> + group: + start_date: <%= 1.month.ago %> + due_date: <%= 1.day.ago %> + + # Ongoing Milestone + - name: Ongoing Milestone + group_id: <%= @group.id %> + group: + start_date: <%= 1.day.ago %> + due_date: <%= 1.month.from_now %> + + # Future Milestone + - name: Ongoing Milestone + group_id: <%= @group.id %> + group: + start_date: <%= 1.month.from_now %> + due_date: <%= 2.months.from_now %> +``` + +#### Quirks + +- You _must_ specify `group:` and have it be empty. This is because the Milestones factory will manipulate the factory in an `after(:build)`. If this is not present, the Milestone will not be associated properly with the Group. + +### [Epics](https://gitlab.com/gitlab-org/gitlab/-/blob/master/ee/spec/factories/epics.rb) + +```yaml +epics: + # Simple Epic + - title: Simple Epic + group_id: <%= @group.id %> + author_id: <%= @owner.id %> + + # Epic with detailed Markdown description + - title: Detailed Epic + group_id: <%= @group.id %> + author_id: <%= @owner.id %> + description: | + # Markdown + + **Description** + + # Epic with dates + - title: Epic with dates + group_id: <%= @group.id %> + author_id: <%= @owner.id %> + start_date: <%= 1.day.ago %> + due_date: <%= 1.month.from_now %> +``` diff --git a/doc/topics/git/bisect.md b/doc/topics/git/bisect.md index a7f8b2a846c..eaf619ce36f 100644 --- a/doc/topics/git/bisect.md +++ b/doc/topics/git/bisect.md @@ -1,76 +1,11 @@ --- -stage: Create -group: Source Code -info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments -comments: false +redirect_to: 'index.md' +remove_date: '2023-06-09' --- -# Bisect **(FREE)** +This document was moved to [another location](index.md). -- Find a commit that introduced a bug -- Works through a process of elimination -- Specify a known good and bad revision to begin - -## Bisect sample workflow - -1. Start the bisect process -1. Enter the bad revision (usually latest commit) -1. Enter a known good revision (commit/branch) -1. Run code to see if bug still exists -1. Tell bisect the result -1. Repeat the previous 2 items until you find the offending commit - -## Setup - -```shell - mkdir bisect-ex - cd bisect-ex - touch index.html - git add -A - git commit -m "starting out" - vi index.html - # Add all good - git add -A - git commit -m "second commit" - vi index.html - # Add all good 2 - git add -A - git commit -m "third commit" - vi index.html -``` - -```shell - # Add all good 3 - git add -A - git commit -m "fourth commit" - vi index.html - # This looks bad - git add -A - git commit -m "fifth commit" - vi index.html - # Really bad - git add -A - git commit -m "sixth commit" - vi index.html - # again just bad - git add -A - git commit -m "seventh commit" -``` - -## Commands - -```shell - git bisect start - # Test your code - git bisect bad - git bisect next - # Say yes to the warning - # Test - git bisect good - # Test - git bisect bad - # Test - git bisect good - # done - git bisect reset -``` +<!-- This redirect file can be deleted after <2023-06-09>. --> +<!-- Redirects that point to other docs in the same project expire in three months. --> +<!-- Redirects that point to docs in a different project or site (for example, link is not relative and starts with `https:`) expire in one year. --> +<!-- Before deletion, see: https://docs.gitlab.com/ee/development/documentation/redirects.html --> diff --git a/doc/topics/git/feature_branching.md b/doc/topics/git/feature_branching.md index 73de351fde2..eaf619ce36f 100644 --- a/doc/topics/git/feature_branching.md +++ b/doc/topics/git/feature_branching.md @@ -1,31 +1,11 @@ --- -stage: Create -group: Source Code -info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments -comments: false +redirect_to: 'index.md' +remove_date: '2023-06-09' --- -# Feature branching **(FREE)** +This document was moved to [another location](index.md). -- Efficient parallel workflow for teams -- Develop each feature in a branch -- Keeps changes isolated -- Consider a 1-to-1 link to issues -- Push branches to the server frequently - - Hint: Pushing branches is a cheap backup for your work-in-progress code. - -## Feature branching sample workflow - -1. Create a new feature branch called `squash_some_bugs` -1. Edit '`bugs.rb`' and remove all the bugs. -1. Commit -1. Push - -```shell -git checkout -b squash_some_bugs -# Edit `bugs.rb` -git status -git add bugs.rb -git commit -m 'Fix some buggy code' -git push origin squash_some_bugs -``` +<!-- This redirect file can be deleted after <2023-06-09>. --> +<!-- Redirects that point to other docs in the same project expire in three months. --> +<!-- Redirects that point to docs in a different project or site (for example, link is not relative and starts with `https:`) expire in one year. --> +<!-- Before deletion, see: https://docs.gitlab.com/ee/development/documentation/redirects.html --> diff --git a/doc/topics/git/git_log.md b/doc/topics/git/git_log.md index e2e9c0e8804..eaf619ce36f 100644 --- a/doc/topics/git/git_log.md +++ b/doc/topics/git/git_log.md @@ -1,60 +1,11 @@ --- -stage: Create -group: Source Code -info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments -comments: false +redirect_to: 'index.md' +remove_date: '2023-06-09' --- -# Git Log **(FREE)** +This document was moved to [another location](index.md). -Git log lists commit history. It allows searching and filtering. - -- Initiate log: - - ```shell - git log - ``` - -- Retrieve set number of records: - - ```shell - git log -n 2 - ``` - -- Search commits by author. Allows user name or a regular expression. - - ```shell - git log --author="user_name" - ``` - -- Search by comment message: - - ```shell - git log --grep="<pattern>" - ``` - -- Search by date: - - ```shell - git log --since=1.month.ago --until=3.weeks.ago - ``` - -## Git Log Workflow - -1. Change to workspace directory -1. Clone the multi runner projects -1. Change to project dir -1. Search by author -1. Search by date -1. Combine - -## Commands - -```shell -cd ~/workspace -git clone git@gitlab.com:gitlab-org/gitlab-runner.git -cd gitlab-runner -git log --author="Travis" -git log --since=1.month.ago --until=3.weeks.ago -git log --since=1.month.ago --until=1.day.ago --author="Travis" -``` +<!-- This redirect file can be deleted after <2023-06-09>. --> +<!-- Redirects that point to other docs in the same project expire in three months. --> +<!-- Redirects that point to docs in a different project or site (for example, link is not relative and starts with `https:`) expire in one year. --> +<!-- Before deletion, see: https://docs.gitlab.com/ee/development/documentation/redirects.html --> diff --git a/doc/topics/git/index.md b/doc/topics/git/index.md index b47a34fa7b2..fa802952b32 100644 --- a/doc/topics/git/index.md +++ b/doc/topics/git/index.md @@ -32,7 +32,6 @@ The following resources can help you get started with Git: - [How to install Git](how_to_install_git/index.md) - [Git terminology](terminology.md) - [Start using Git on the command line](../../gitlab-basics/start-using-git.md) -- [Edit files through the command line](../../gitlab-basics/command-line-commands.md) - [GitLab Git Cheat Sheet (download)](https://about.gitlab.com/images/press/git-cheat-sheet.pdf) - Commits: - [Revert a commit](../../user/project/merge_requests/revert_changes.md#revert-a-commit) @@ -43,7 +42,7 @@ The following resources can help you get started with Git: - [Git stash](stash.md) - [Git file blame](../../user/project/repository/git_blame.md) - [Git file history](../../user/project/repository/git_history.md) -- [Git tags](tags.md) +- [Git tags](../../user/project/repository/tags/index.md) ### Concepts @@ -58,16 +57,11 @@ The following are resources on version control concepts: You can do many Git tasks from the command line: -- [Bisect](bisect.md). - [Cherry-pick](cherry_picking.md). -- [Feature branching](feature_branching.md). - [Getting started with Git](getting_started.md). - [Git add](git_add.md). -- [Git log](git_log.md). - [Git stash](stash.md). -- [Merge conflicts](merge_conflicts.md). - [Rollback commits](rollback_commits.md). -- [Subtree](subtree.md). - [Unstage](unstage.md). ## Git tips diff --git a/doc/topics/git/merge_conflicts.md b/doc/topics/git/merge_conflicts.md deleted file mode 100644 index ea37afe1b31..00000000000 --- a/doc/topics/git/merge_conflicts.md +++ /dev/null @@ -1,11 +0,0 @@ ---- -redirect_to: '../../user/project/merge_requests/conflicts.md#resolve-conflicts-from-the-command-line' -remove_date: '2023-02-01' ---- - -This document was moved to [another location](../../user/project/merge_requests/conflicts.md#resolve-conflicts-from-the-command-line). - -<!-- This redirect file can be deleted after <2023-02-01>. --> -<!-- Redirects that point to other docs in the same project expire in three months. --> -<!-- Redirects that point to docs in a different project or site (for example, link is not relative and starts with `https:`) expire in one year. --> -<!-- Before deletion, see: https://docs.gitlab.com/ee/development/documentation/redirects.html --> diff --git a/doc/topics/git/partial_clone.md b/doc/topics/git/partial_clone.md index f5f11b17675..de0547ae49d 100644 --- a/doc/topics/git/partial_clone.md +++ b/doc/topics/git/partial_clone.md @@ -94,9 +94,7 @@ Updating files: 100% (28/28), done. $ cd www-gitlab-com -$ git sparse-checkout init --cone - -$ git sparse-checkout add data +$ git sparse-checkout set data --cone remote: Enumerating objects: 301, done. remote: Counting objects: 100% (301/301), done. remote: Compressing objects: 100% (292/292), done. diff --git a/doc/topics/git/subtree.md b/doc/topics/git/subtree.md index a8a665d4e13..eaf619ce36f 100644 --- a/doc/topics/git/subtree.md +++ b/doc/topics/git/subtree.md @@ -1,50 +1,11 @@ --- -stage: Create -group: Source Code -info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments -comments: false +redirect_to: 'index.md' +remove_date: '2023-06-09' --- -# Subtree **(FREE)** +This document was moved to [another location](index.md). -- Used when there are nested repositories. -- Not recommended when the amount of dependencies is too large. -- For these cases we need a dependency control system. -- Command are painfully long so aliases are necessary. - -## Subtree Aliases - -- Add: `git subtree add --prefix <target-folder> <url> <branch> --squash` -- Pull: `git subtree pull --prefix <target-folder> <url> <branch> --squash` -- Push: `git subtree push --prefix <target-folder> <url> <branch>` - -```shell - # Add an alias - # Add - git config alias.sba 'subtree add --prefix st / - git@gitlab.com:balameb/subtree-nested-example.git master --squash' - # Pull - git config alias.sbpl 'subtree pull --prefix st / - git@gitlab.com:balameb/subtree-nested-example.git master --squash' - # Push - git config alias.sbph 'subtree push --prefix st / - git@gitlab.com:balameb/subtree-nested-example.git master' - - # Adding this subtree adds a st dir with a readme - git sba - vi st/README.md - # Edit file - git status shows differences - -``` - -```shell - # Adding, or committing won't change the sub repo at remote - # even if we push - git add -A - git commit -m "Adding to subtree readme" - - # Push to subtree repo - git sbph - # now we can check our remote sub repo -``` +<!-- This redirect file can be deleted after <2023-06-09>. --> +<!-- Redirects that point to other docs in the same project expire in three months. --> +<!-- Redirects that point to docs in a different project or site (for example, link is not relative and starts with `https:`) expire in one year. --> +<!-- Before deletion, see: https://docs.gitlab.com/ee/development/documentation/redirects.html --> diff --git a/doc/topics/git/tags.md b/doc/topics/git/tags.md index ab196f409f7..c66c97717f2 100644 --- a/doc/topics/git/tags.md +++ b/doc/topics/git/tags.md @@ -1,41 +1,11 @@ --- -stage: Create -group: Source Code -info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments +redirect_to: '../../user/project/repository/tags/index.md' +remove_date: '2023-05-27' --- -# Tags **(FREE)** +This document was moved to [another location](../../user/project/repository/tags/index.md). -Tags help you mark certain deployments and releases for later -reference. Git supports two types of tags: - -- Annotated tags: An unchangeable part of Git history. -- Lightweight (soft) tags: Tags that can be set and removed as needed. - -Many projects combine an annotated release tag with a stable branch. Consider -setting deployment or release tags automatically. - -## Tags sample workflow - -1. Create a lightweight tag. -1. Create an annotated tag. -1. Push the tags to the remote repository. - -```shell -git checkout master - -# Lightweight tag -git tag my_lightweight_tag - -# Annotated tag -git tag -a v1.0 -m 'Version 1.0' - -# Show list of the existing tags -git tag - -git push origin --tags -``` - -## Related topics - -- [Tagging](https://git-scm.com/book/en/v2/Git-Basics-Tagging) Git reference page +<!-- This redirect file can be deleted after <2023-05-27>. --> +<!-- Redirects that point to other docs in the same project expire in three months. --> +<!-- Redirects that point to docs in a different project or site (for example, link is not relative and starts with `https:`) expire in one year. --> +<!-- Before deletion, see: https://docs.gitlab.com/ee/development/documentation/redirects.html --> diff --git a/doc/topics/git/troubleshooting_git.md b/doc/topics/git/troubleshooting_git.md index a116f6cc978..37450319350 100644 --- a/doc/topics/git/troubleshooting_git.md +++ b/doc/topics/git/troubleshooting_git.md @@ -221,11 +221,17 @@ apply more than one: 1. Modify the GitLab instance's [`gitlab.rb`](https://gitlab.com/gitlab-org/omnibus-gitlab/-/blob/13.5.1+ee.0/files/gitlab-config-template/gitlab.rb.template#L1435-1455) file: - ```shell - gitaly['gitconfig'] = [ - # Set the http.postBuffer size, in bytes - {key: "http.postBuffer", value: "524288000"}, - ] + ```ruby + gitaly['configuration'] = { + # ... + git: { + # ... + config: [ + # Set the http.postBuffer size, in bytes + {key: "http.postBuffer", value: "524288000"}, + ], + }, + } ``` 1. After applying this change, apply the configuration change: diff --git a/doc/topics/git/unstage.md b/doc/topics/git/unstage.md index 142a80a9ad9..aad5d0f44f8 100644 --- a/doc/topics/git/unstage.md +++ b/doc/topics/git/unstage.md @@ -5,15 +5,20 @@ info: To determine the technical writer assigned to the Stage/Group associated w comments: false --- -# Unstage **(FREE)** +# Unstage a file in Git **(FREE)** -- To remove files from stage use reset HEAD where HEAD is the last commit of the current branch. This unstages the file but maintain the modifications. +When you _stage_ a file in Git, you instruct Git to track changes to the file in +preparation for a commit. To instruct Git to disregard changes to a file, and not +include it in your next commit, _unstage_ the file. + +- To remove files from stage use `reset HEAD`, where HEAD is the last commit of + the current branch. This unstages the file but maintains the modifications. ```shell git reset HEAD <file> ``` -- To revert the file back to the state it was in before the changes we can use: +- To revert the file back to the state it was in before the changes: ```shell git checkout -- <file> @@ -26,7 +31,8 @@ comments: false git rm -r <dirname> ``` -- If we want to remove a file from the repository but keep it on disk, say we forgot to add it to our `.gitignore` file then use `--cache`: +- To keep a file on disk but remove it from the repository (such as a file you want + to add to `.gitignore`), use the `rm` command with the `--cache` flag: ```shell git rm <filename> --cache diff --git a/doc/topics/git/useful_git_commands.md b/doc/topics/git/useful_git_commands.md index 235412f511a..22548be2e8b 100644 --- a/doc/topics/git/useful_git_commands.md +++ b/doc/topics/git/useful_git_commands.md @@ -130,12 +130,6 @@ Use this to check the Git history of the file: git log -- <file> ``` -### Find the tags that contain a particular SHA - -```shell -git tag --contains <sha> -``` - ### Check the content of each change to a file ```shell diff --git a/doc/topics/offline/quick_start_guide.md b/doc/topics/offline/quick_start_guide.md index 80ce703f7db..b2767f2e5eb 100644 --- a/doc/topics/offline/quick_start_guide.md +++ b/doc/topics/offline/quick_start_guide.md @@ -214,3 +214,89 @@ and Praefect servers so they can use an accessible NTP server. On offline instances, the [GitLab Geo check Rake task](../../administration/geo/replication/troubleshooting.md#can-geo-detect-the-current-site-correctly) always fails because it uses `pool.ntp.org`. This error can be ignored but you can [read more about how to work around it](../../administration/geo/replication/troubleshooting.md#message-machine-clock-is-synchronized--exception). + +## Enabling the package metadata database + +Enabling the package metadata database is required to enable [license scanning of CycloneDX files](../../user/compliance/license_scanning_of_cyclonedx_files). +This process will require usage of the GitLab License Database, which is licensed under the [EE License](https://storage.googleapis.com/prod-export-license-bucket-1a6c642fc4de57d4/v1/LICENSE). +Please note the following in relation to use of the License Database: + +- We may change or discontinue all or any part of the License Database, at any time and without notice, at our sole discretion. +- The License Database may contain links to third-party websites or resources. We provide these links only as a convenience and are not responsible for any third-party data, content, products, or services from those websites or resources or links displayed on such websites. +- The License Database is based in part on information made available by third parties, and GitLab is not responsible for the accuracy or completeness of content made available. + +### Using the gsutil tool to download the package metadata exports + +1. Install the [`gsutil`](https://cloud.google.com/storage/docs/gsutil_install) tool. +1. Find the root of the GitLab Rails directory. + + ```shell + export GITLAB_RAILS_ROOT_DIR="$(gitlab-rails runner 'puts Rails.root.to_s')" + echo $GITLAB_RAILS_ROOT_DIR + ``` + +1. Download the package metadata exports. + + ```shell + # To download the package metadata exports, an outbound connection to Google Cloud Storage bucket must be allowed. + mkdir $GITLAB_RAILS_ROOT_DIR/vendor/package_metadata_db/ + gsutil -m rsync -r -d gs://prod-export-license-bucket-1a6c642fc4de57d4 $GITLAB_RAILS_ROOT_DIR/vendor/package_metadata_db/ + + # Alternatively, if the GitLab instance is not allowed to connect to the Google Cloud Storage bucket, the package metadata + # exports can be downloaded using a machine with the allowed access, and then copied to the root of the GitLab Rails directory. + rsync rsync://example_username@gitlab.example.com/package_metadata_db $GITLAB_RAILS_ROOT_DIR/vendor/package_metadata_db/ + ``` + +### Using the Google Cloud Storage REST API to download the package metadata exports + +The package metadata exports can also be downloaded using the Google Cloud Storage API. The contents are available at [https://storage.googleapis.com/storage/v1/b/prod-export-license-bucket-1a6c642fc4de57d4/o](https://storage.googleapis.com/storage/v1/b/prod-export-license-bucket-1a6c642fc4de57d4/o). The following is an example of how this can be downloaded using [cURL](https://curl.se/) and [jq](https://stedolan.github.io/jq/). + +```shell +#!/bin/bash + +set -euo pipefail + +GITLAB_RAILS_ROOT_DIR="$(gitlab-rails runner 'puts Rails.root.to_s')" +PKG_METADATA_DIR="$GITLAB_RAILS_ROOT_DIR/vendor/package_metadata_db" +PKG_METADATA_MANIFEST_OUTPUT_FILE="/tmp/license_db_export_manifest.json" +PKG_METADATA_DOWNLOADS_OUTPUT_FILE="/tmp/license_db_object_links.tsv" + +# Download the contents of the bucket +curl --silent --show-error --request GET "https://storage.googleapis.com/storage/v1/b/prod-export-license-bucket-1a6c642fc4de57d4/o" > "$PKG_METADATA_MANIFEST_OUTPUT_FILE" + +# Parse the links and names for the bucket objects and output them into a tsv file +jq -r '.items[] | [.name, .mediaLink] | @tsv' "$PKG_METADATA_MANIFEST_OUTPUT_FILE" > "$PKG_METADATA_DOWNLOADS_OUTPUT_FILE" + +echo -e "Saving package metadata exports to $PKG_METADATA_DIR\n" + +# Track how many objects will be downloaded +INDEX=1 +TOTAL_OBJECT_COUNT="$(wc -l $PKG_METADATA_DOWNLOADS_OUTPUT_FILE | awk '{print $1}')" + +# Download the objects +while IFS= read -r line; do + FILE="$(echo -n $line | awk '{print $1}')" + URL="$(echo -n $line | awk '{print $2}')" + OUTPUT_DIR="$(dirname $PKG_METADATA_DIR/$FILE)" + OUTPUT_PATH="$PKG_METADATA_DIR/$FILE" + + echo "Downloading $FILE" + + curl --progress-bar --create-dirs --output "$OUTPUT_PATH" --request "GET" "$URL" + + echo -e "$INDEX of $TOTAL_OBJECT_COUNT objects downloaded\n" + + let INDEX=(INDEX+1) +done < "$PKG_METADATA_DOWNLOADS_OUTPUT_FILE" + +echo "All objects saved to $PKG_METADATA_DIR" +``` + +### Automatic synchronization + +Your GitLab instance is synchronized [every hour](https://gitlab.com/gitlab-org/gitlab/-/blob/d4331343d26d6e2a81fadd8f7ecd72f7cb74d04d/config/initializers/1_settings.rb#L831-832) with the contents of the `package_metadata_db` directory. +To automatically update your local copy with the upstream changes, a cron job can be added to periodically download new exports. For example, the following crontabs can be added to setup a cron job that runs every 30 minutes. + +```plaintext +*/30 * * * * gsutil -m rsync -r -d gs://prod-export-license-bucket-1a6c642fc4de57d4 $GITLAB_RAILS_ROOT_DIR/vendor/package_metadata_db/ +``` diff --git a/doc/topics/set_up_organization.md b/doc/topics/set_up_organization.md index 855b4962960..cf0a0ae4ab7 100644 --- a/doc/topics/set_up_organization.md +++ b/doc/topics/set_up_organization.md @@ -11,7 +11,7 @@ and give everyone access to the projects they need. - [Namespaces](../user/namespace/index.md) - [Members](../user/project/members/index.md) -- [Organization](../user/workspace/index.md) _(In development)_ +- [Organization](../user/organization/index.md) _(In development)_ - [Groups](../user/group/index.md) - [User account options](../user/profile/index.md) - [SSH keys](../user/ssh.md) diff --git a/doc/topics/your_work.md b/doc/topics/your_work.md index 862f9ae8430..268a6c4df5b 100644 --- a/doc/topics/your_work.md +++ b/doc/topics/your_work.md @@ -16,3 +16,8 @@ The **Your work** left sidebar provides access to your: - [Merge requests](../user/project/merge_requests/index.md) - [To-do List](../user/todos.md) - [Milestones](../user/project/milestones/index.md) +- [Snippets](../user/snippets.md#snippets) +- [Activity](../user/profile/index.md#view-your-activity) +- [Environments dashboard](../ci/environments/environments_dashboard.md) +- [Operations dashboard](../user/operations_dashboard/index.md) +- [Security center](../user/application_security/security_dashboard/index.md#security-center) |