diff options
Diffstat (limited to 'doc/ci/cloud_deployment')
-rw-r--r-- | doc/ci/cloud_deployment/ecs/deploy_to_aws_ecs.md | 253 | ||||
-rw-r--r-- | doc/ci/cloud_deployment/ecs/quick_start_guide.md | 253 | ||||
-rw-r--r-- | doc/ci/cloud_deployment/index.md | 377 |
3 files changed, 392 insertions, 491 deletions
diff --git a/doc/ci/cloud_deployment/ecs/deploy_to_aws_ecs.md b/doc/ci/cloud_deployment/ecs/deploy_to_aws_ecs.md new file mode 100644 index 00000000000..9af5218e058 --- /dev/null +++ b/doc/ci/cloud_deployment/ecs/deploy_to_aws_ecs.md @@ -0,0 +1,253 @@ +--- +stage: Release +group: Release +info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/engineering/ux/technical-writing/#assignments +--- + +# Deploy to Amazon Elastic Container Service **(FREE)** + +This step-by-step guide helps you deploy a project hosted on GitLab.com to +the Amazon [Elastic Container Service (ECS)](https://aws.amazon.com/ecs/). + +In this guide, you begin by creating an ECS cluster manually using the AWS console. You create and +deploy a simple application that you create from a GitLab template. + +These instructions work for both SaaS and self-managed GitLab instances. +Ensure your own [runners are configured](../../runners/index.md). + +## Prerequisites + +- An [AWS account](https://aws.amazon.com/premiumsupport/knowledge-center/create-and-activate-aws-account/). + Sign in with an existing AWS account or create a new one. +- In this guide, you create an infrastructure in [`us-east-2` region](https://docs.aws.amazon.com/AWSEC2/latest/UserGuide/using-regions-availability-zones.html). + You can use any region, but do not change it after you begin. + +## Create an infrastructure and initial deployment on AWS + +For deploying an application from GitLab, you must first create an infrastructure and initial +deployment on AWS. +This includes an [ECS cluster](https://docs.aws.amazon.com/AmazonECS/latest/developerguide/clusters.html) +and related components, such as +[ECS task definitions](https://docs.aws.amazon.com/AmazonECS/latest/developerguide/task_definitions.html), +[ECS services](https://docs.aws.amazon.com/AmazonECS/latest/developerguide/ecs_services.html), +and containerized application image. + +For the first step here, you create a demo application from a project template. + +### Create a new project from a template + +Use a GitLab project template to get started. As the name suggests, these projects provide a +bare-bones application built on some well-known frameworks. + +1. In GitLab, select the plus icon (**{plus-square}**) at the top of the navigation bar, and select + **New project**. + +1. Select **Create from template**, where you can choose from a Ruby on Rails, Spring, or + NodeJS Express project. For this guide, use the Ruby on Rails template. + + ![Select project template](img/rails-template.png) + +1. Give your project a name. In this example, it's named `ecs-demo`. Make it public so that you can + take advantage of the features available in the + [GitLab Ultimate plan](https://about.gitlab.com/pricing/). + +1. Select **Create project**. + +Now that you created a demo project, you must containerize the application and push it to the +container registry. + +### Push a containerized application image to GitLab Container Registry + +[ECS](https://aws.amazon.com/ecs/) is a container orchestration service, meaning that you must +provide a containerized application image during the infrastructure build. To do so, you can use +GitLab [Auto Build](../../../topics/autodevops/stages.md#auto-build) +and [Container Registry](../../../user/packages/container_registry/index.md). + +1. Go to **ecs-demo** project on GitLab. +1. Select **Setup up CI/CD**. It brings you to a `.gitlab-ci.yml` + creation form. +1. Copy and paste the following content into the empty `.gitlab-ci.yml`. This defines + a pipeline for continuous deployment to ECS. + + ```yaml + include: + - template: AWS/Deploy-ECS.gitlab-ci.yml + ``` + +1. Select **Commit Changes**. It automatically triggers a new pipeline. In this pipeline, the `build` + job containerizes the application and pushes the image to [GitLab Container Registry](../../../user/packages/container_registry/index.md). + + ![Create project](img/initial-pipeline.png) + +1. Visit **Packages & Registries > Container Registry**. Make sure the application image has been + pushed. + + ![Create project](img/registry.png) + +Now you have a containerized application image that can be pulled from AWS. Next, you define the +spec of how this application image is used in AWS. + +Note that the `production_ecs` job fails because ECS Cluster is not connected yet. You'll fix this +later. + +### Create an ECS task definition + +[ECS Task definitions](https://docs.aws.amazon.com/AmazonECS/latest/developerguide/task_definitions.html) +is a specification about how the application image is started by an [ECS service](https://docs.aws.amazon.com/AmazonECS/latest/developerguide/ecs_services.html). + +1. Go to **ECS > Task Definitions** on [AWS console](https://aws.amazon.com/). +1. Select **Create new Task Definition**. + + ![Create project](img/ecs-task-definitions.png) + +1. Choose **EC2** as the launch type. Select **Next Step**. +1. Set `ecs_demo` to **Task Definition Name**. +1. Set `512` to **Task Size > Task memory** and **Task CPU**. +1. Select **Container Definitions > Add container**. This opens a container registration form. +1. Set `web` to **Container name**. +1. Set `registry.gitlab.com/<your-namespace>/ecs-demo/master:latest` to **Image**. + Alternatively, you can copy and paste the image path from the [GitLab Container Registry page](#push-a-containerized-application-image-to-gitlab-container-registry). + + ![Create project](img/container-name.png) + +1. Add a port mapping. Set `80` to **Host Port** and `5000` to **Container port**. + + ![Create project](img/container-port-mapping.png) + +1. Select **Create**. + +Now you have the initial task definition. Next, you create an actual infrastructure to run the +application image. + +### Create an ECS cluster + +An [ECS cluster](https://docs.aws.amazon.com/AmazonECS/latest/developerguide/clusters.html) +is a virtual group of [ECS services](https://docs.aws.amazon.com/AmazonECS/latest/developerguide/ecs_services.html). +It's also associated with EC2 or Fargate as the computation resource. + +1. Go to **ECS > Clusters** on [AWS console](https://aws.amazon.com/). +1. Select **Create Cluster**. +1. Select **EC2 Linux + Networking** as the cluster template. Select **Next Step**. +1. Set `ecs-demo` to **Cluster Name**. +1. Choose the default [VPC](https://aws.amazon.com/vpc/?vpc-blogs.sort-by=item.additionalFields.createdDate&vpc-blogs.sort-order=desc) + in **Networking**. If there are no existing VPCs, you can leave it as-is to create a new one. +1. Set all available subnets of the VPC to **Subnets**. +1. Select **Create**. +1. Make sure that the ECS cluster has been successfully created. + + ![Create project](img/ecs-launch-status.png) + +Now you can register an ECS service to the ECS cluster in the next step. + +Note the following: + +- Optionally, you can set a SSH key pair in the creation form. This allows you to SSH to the EC2 + instance for debugging. +- If you don't choose an existing VPC, it creates a new VPC by default. This could cause an error if + it reaches the maximum allowed number of internet gateways on your account. +- The cluster requires an EC2 instance, meaning it costs you [according to the instance-type](https://aws.amazon.com/ec2/pricing/on-demand/). + +### Create an ECS Service + +[ECS service](https://docs.aws.amazon.com/AmazonECS/latest/developerguide/ecs_services.html) +is a daemon to create an application container based on the [ECS task definition](#create-an-ecs-task-definition). + +1. Go to **ECS > Clusters > ecs-demo > Services** on the [AWS console](https://aws.amazon.com/) +1. Select **Deploy**. This opens a service creation form. +1. Select `EC2` in **Launch Type**. +1. Set `ecs_demo` to **Task definition**. This corresponds to [the task definition you created above](#create-an-ecs-task-definition). +1. Set `ecs_demo` to **Service name**. +1. Set `1` to **Desired tasks**. + + ![Create project](img/service-parameter.png) + +1. Select **Deploy**. +1. Make sure that the created service is active. + + ![Create project](img/service-running.png) + +Note that AWS's console UI changes from time to time. If you can't find a relevant component in the +instructions, select the closest one. + +### View the demo application + +Now, the demo application is accessible from the internet. + +1. Go to **EC2 > Instances** on the [AWS console](https://aws.amazon.com/) +1. Search by `ECS Instance` to find the corresponding EC2 instance that [the ECS cluster created](#create-an-ecs-cluster). +1. Select the ID of the EC2 instance. This brings you to the instance detail page. +1. Copy **Public IPv4 address** and paste it in the browser. Now you can see the demo application + running. + + ![Create project](img/view-running-app.png) + +In this guide, HTTPS/SSL is **not** configured. You can access to the application through HTTP only +(for example, `http://<ec2-ipv4-address>`). + +## Setup Continuous Deployment from GitLab + +Now that you have an application running on ECS, you can set up continuous deployment from GitLab. + +### Create a new IAM user as a deployer + +For GitLab to access the ECS cluster, service, and task definition that you created above, You must +create a deployer user on AWS: + +1. Go to **IAM > Users** on [AWS console](https://aws.amazon.com/). +1. Select **Add user**. +1. Set `ecs_demo` to **User name**. +1. Enable **Programmatic access** checkbox. Select **Next: Permissions**. +1. Select `Attach existing policies directly` in **Set permissions**. +1. Select `AmazonECS_FullAccess` from the policy list. Select **Next: Tags** and **Next: Review**. + + ![Create project](img/ecs-policy.png) + +1. Select **Create user**. +1. Take note of the **Access key ID** and **Secret access key** of the created user. + +NOTE: +Do not share the secret access key in a public place. You must save it in a secure place. + +### Setup credentials in GitLab to let pipeline jobs access to ECS + +You can register the access information in [GitLab Environment Variables](../../variables/index.md#custom-cicd-variables). +These variables are injected into the pipeline jobs and can access the ECS API. + +1. Go to **ecs-demo** project on GitLab. +1. Go to **Settings > CI/CD > Variables**. +1. Select **Add Variable** and set the following key-value pairs. + + |Key|Value|Note| + |---|---|---| + |`AWS_ACCESS_KEY_ID`|`<Access key ID of the deployer>`| For authenticating `aws` CLI. | + |`AWS_SECRET_ACCESS_KEY`|`<Secret access key of the deployer>`| For authenticating `aws` CLI. | + |`AWS_DEFAULT_REGION`|`us-east-2`| For authenticating `aws` CLI. | + |`CI_AWS_ECS_CLUSTER`|`ecs-demo`| The ECS cluster is accessed by `production_ecs` job. | + |`CI_AWS_ECS_SERVICE`|`ecs_demo`| The ECS service of the cluster is updated by `production_ecs` job. | + |`CI_AWS_ECS_TASK_DEFINITION`|`ecs_demo`| The ECS task definition is updated by `production_ecs` job. | + +### Make a change to the demo application + +Change a file in the project and see if it's reflected in the demo application on ECS: + +1. Go to **ecs-demo** project on GitLab. +1. Open the file at **app > views > welcome > `index.html.erb`**. +1. Select **Edit**. +1. Change the text to `You're on ECS!`. +1. Select **Commit Changes**. This automatically triggers a new pipeline. Wait until it finishes. +1. [Access the running application on the ECS cluster](#view-the-demo-application). You should see + this: + + ![Create project](img/view-running-app-2.png) + +Congratulations! You successfully set up continuous deployment to ECS. + +NOTE: +ECS deploy jobs wait for the rollout to complete before exiting. To disable this behavior, +set `CI_AWS_ECS_WAIT_FOR_ROLLOUT_COMPLETE_DISABLED` to a non-empty value. + +## Further reading + +- If you're interested in more of the continuous deployments to clouds, see [cloud deployments](../index.md). +- If you want to quickly set up DevSecOps in your project, see [Auto DevOps](../../../topics/autodevops/index.md). +- If you want to quickly set up the production-grade environment, see [the 5 Minute Production App](https://gitlab.com/gitlab-org/5-minute-production-app/deploy-template/-/blob/master/README.md). diff --git a/doc/ci/cloud_deployment/ecs/quick_start_guide.md b/doc/ci/cloud_deployment/ecs/quick_start_guide.md index 5571c51ef27..7522bc80d0d 100644 --- a/doc/ci/cloud_deployment/ecs/quick_start_guide.md +++ b/doc/ci/cloud_deployment/ecs/quick_start_guide.md @@ -1,250 +1,11 @@ --- -stage: Release -group: Release -info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/engineering/ux/technical-writing/#assignments +redirect_to: 'deploy_to_aws_ecs.md' +remove_date: '2022-09-18' --- -# Getting started with Continuous Deployment to AWS Elastic Container Service **(FREE)** +This document was moved to [another location](deploy_to_aws_ecs.md). -This step-by-step guide helps you use [Continuous Deployment to ECS](../index.md#deploy-your-application-to-the-aws-elastic-container-service-ecs) -that deploys a project hosted on GitLab.com to [Elastic Container Service](https://aws.amazon.com/ecs/) -(ECS) on AWS. - -In this guide, you begin by creating an ECS cluster manually using the AWS console. You create and -deploy a simple application that you create from a GitLab template. - -These instructions work for both SaaS and self-managed GitLab instances. -Ensure your own [runners are configured](../../runners/index.md). - -## Prerequisites - -- An [AWS account](https://aws.amazon.com/premiumsupport/knowledge-center/create-and-activate-aws-account/). - Sign in with an existing AWS account or create a new one. -- In this guide, you create an infrastructure in [`us-east-2` region](https://docs.aws.amazon.com/AWSEC2/latest/UserGuide/using-regions-availability-zones.html). - You can use any region, but do not change it after you begin. - -## Create an infrastructure and initial deployment on AWS - -For deploying an application from GitLab, you must first create an infrastructure and initial -deployment on AWS. -This includes an [ECS cluster](https://docs.aws.amazon.com/AmazonECS/latest/developerguide/clusters.html) -and related components, such as -[ECS task definitions](https://docs.aws.amazon.com/AmazonECS/latest/developerguide/task_definitions.html), -[ECS services](https://docs.aws.amazon.com/AmazonECS/latest/developerguide/ecs_services.html), -and containerized application image. - -For the first step here, you create a demo application from a project template. - -### Create a new project from a template - -Use a GitLab project template to get started. As the name suggests, these projects provide a -bare-bones application built on some well-known frameworks. - -1. In GitLab, select the plus icon (**{plus-square}**) at the top of the navigation bar, and select - **New project**. - -1. Select **Create from template**, where you can choose from a Ruby on Rails, Spring, or - NodeJS Express project. For this guide, use the Ruby on Rails template. - - ![Select project template](img/rails-template.png) - -1. Give your project a name. In this example, it's named `ecs-demo`. Make it public so that you can - take advantage of the features available in the - [GitLab Ultimate plan](https://about.gitlab.com/pricing/). - -1. Select **Create project**. - -Now that you created a demo project, you must containerize the application and push it to the -container registry. - -### Push a containerized application image to GitLab Container Registry - -[ECS](https://aws.amazon.com/ecs/) is a container orchestration service, meaning that you must -provide a containerized application image during the infrastructure build. To do so, you can use -GitLab [Auto Build](../../../topics/autodevops/stages.md#auto-build) -and [Container Registry](../../../user/packages/container_registry/index.md). - -1. Go to **ecs-demo** project on GitLab. -1. Select **Setup up CI/CD**. It brings you to a `.gitlab-ci.yml` - creation form. -1. Copy and paste the following content into the empty `.gitlab-ci.yml`. This defines - [a pipeline for continuous deployment to ECS](../index.md#deploy-your-application-to-the-aws-elastic-container-service-ecs). - - ```yaml - include: - - template: AWS/Deploy-ECS.gitlab-ci.yml - ``` - -1. Select **Commit Changes**. It automatically triggers a new pipeline. In this pipeline, the `build` - job containerizes the application and pushes the image to [GitLab Container Registry](../../../user/packages/container_registry/index.md). - - ![Create project](img/initial-pipeline.png) - -1. Visit **Packages & Registries > Container Registry**. Make sure the application image has been - pushed. - - ![Create project](img/registry.png) - -Now you have a containerized application image that can be pulled from AWS. Next, you define the -spec of how this application image is used in AWS. - -Note that the `production_ecs` job fails because ECS Cluster is not connected yet. You'll fix this -later. - -### Create an ECS task definition - -[ECS Task definitions](https://docs.aws.amazon.com/AmazonECS/latest/developerguide/task_definitions.html) -is a specification about how the application image is started by an [ECS service](https://docs.aws.amazon.com/AmazonECS/latest/developerguide/ecs_services.html). - -1. Go to **ECS > Task Definitions** on [AWS console](https://aws.amazon.com/). -1. Select **Create new Task Definition**. - - ![Create project](img/ecs-task-definitions.png) - -1. Choose **EC2** as the launch type. Select **Next Step**. -1. Set `ecs_demo` to **Task Definition Name**. -1. Set `512` to **Task Size > Task memory** and **Task CPU**. -1. Select **Container Definitions > Add container**. This opens a container registration form. -1. Set `web` to **Container name**. -1. Set `registry.gitlab.com/<your-namespace>/ecs-demo/master:latest` to **Image**. - Alternatively, you can copy and paste the image path from the [GitLab Container Registry page](#push-a-containerized-application-image-to-gitlab-container-registry). - - ![Create project](img/container-name.png) - -1. Add a port mapping. Set `80` to **Host Port** and `5000` to **Container port**. - - ![Create project](img/container-port-mapping.png) - -1. Select **Create**. - -Now you have the initial task definition. Next, you create an actual infrastructure to run the -application image. - -### Create an ECS cluster - -An [ECS cluster](https://docs.aws.amazon.com/AmazonECS/latest/developerguide/clusters.html) -is a virtual group of [ECS services](https://docs.aws.amazon.com/AmazonECS/latest/developerguide/ecs_services.html). -It's also associated with EC2 or Fargate as the computation resource. - -1. Go to **ECS > Clusters** on [AWS console](https://aws.amazon.com/). -1. Select **Create Cluster**. -1. Select **EC2 Linux + Networking** as the cluster template. Select **Next Step**. -1. Set `ecs-demo` to **Cluster Name**. -1. Choose the default [VPC](https://aws.amazon.com/vpc/?vpc-blogs.sort-by=item.additionalFields.createdDate&vpc-blogs.sort-order=desc) - in **Networking**. If there are no existing VPCs, you can leave it as-is to create a new one. -1. Set all available subnets of the VPC to **Subnets**. -1. Select **Create**. -1. Make sure that the ECS cluster has been successfully created. - - ![Create project](img/ecs-launch-status.png) - -Now you can register an ECS service to the ECS cluster in the next step. - -Note the following: - -- Optionally, you can set a SSH key pair in the creation form. This allows you to SSH to the EC2 - instance for debugging. -- If you don't choose an existing VPC, it creates a new VPC by default. This could cause an error if - it reaches the maximum allowed number of internet gateways on your account. -- The cluster requires an EC2 instance, meaning it costs you [according to the instance-type](https://aws.amazon.com/ec2/pricing/on-demand/). - -### Create an ECS Service - -[ECS service](https://docs.aws.amazon.com/AmazonECS/latest/developerguide/ecs_services.html) -is a daemon to create an application container based on the [ECS task definition](#create-an-ecs-task-definition). - -1. Go to **ECS > Clusters > ecs-demo > Services** on the [AWS console](https://aws.amazon.com/) -1. Select **Deploy**. This opens a service creation form. -1. Select `EC2` in **Launch Type**. -1. Set `ecs_demo` to **Task definition**. This corresponds to [the task definition you created above](#create-an-ecs-task-definition). -1. Set `ecs_demo` to **Service name**. -1. Set `1` to **Desired tasks**. - - ![Create project](img/service-parameter.png) - -1. Select **Deploy**. -1. Make sure that the created service is active. - - ![Create project](img/service-running.png) - -Note that AWS's console UI changes from time to time. If you can't find a relevant component in the -instructions, select the closest one. - -### View the demo application - -Now, the demo application is accessible from the internet. - -1. Go to **EC2 > Instances** on the [AWS console](https://aws.amazon.com/) -1. Search by `ECS Instance` to find the corresponding EC2 instance that [the ECS cluster created](#create-an-ecs-cluster). -1. Select the ID of the EC2 instance. This brings you to the instance detail page. -1. Copy **Public IPv4 address** and paste it in the browser. Now you can see the demo application - running. - - ![Create project](img/view-running-app.png) - -In this guide, HTTPS/SSL is **NOT** configured. You can access to the application through HTTP only -(for example, `http://<ec2-ipv4-address>`). - -## Setup Continuous Deployment from GitLab - -Now that you have an application running on ECS, you can set up continuous deployment from GitLab. - -### Create a new IAM user as a deployer - -For GitLab to access the ECS cluster, service, and task definition that you created above, You must -create a deployer user on AWS: - -1. Go to **IAM > Users** on [AWS console](https://aws.amazon.com/). -1. Select **Add user**. -1. Set `ecs_demo` to **User name**. -1. Enable **Programmatic access** checkbox. Select **Next: Permissions**. -1. Select `Attach existing policies directly` in **Set permissions**. -1. Select `AmazonECS_FullAccess` from the policy list. Select **Next: Tags** and **Next: Review**. - - ![Create project](img/ecs-policy.png) - -1. Select **Create user**. -1. Take note of the **Access key ID** and **Secret access key** of the created user. - -NOTE: -Do not share the secret access key in a public place. You must save it in a secure place. - -### Setup credentials in GitLab to let pipeline jobs access to ECS - -You can register the access information in [GitLab Environment Variables](../../variables/index.md#custom-cicd-variables). -These variables are injected into the pipeline jobs and can access the ECS API. - -1. Go to **ecs-demo** project on GitLab. -1. Go to **Settings > CI/CD > Variables**. -1. Select **Add Variable** and set the following key-value pairs. - - |Key|Value|Note| - |---|---|---| - |`AWS_ACCESS_KEY_ID`|`<Access key ID of the deployer>`| For authenticating `aws` CLI. | - |`AWS_SECRET_ACCESS_KEY`|`<Secret access key of the deployer>`| For authenticating `aws` CLI. | - |`AWS_DEFAULT_REGION`|`us-east-2`| For authenticating `aws` CLI. | - |`CI_AWS_ECS_CLUSTER`|`ecs-demo`| The ECS cluster is accessed by `production_ecs` job. | - |`CI_AWS_ECS_SERVICE`|`ecs_demo`| The ECS service of the cluster is updated by `production_ecs` job. | - |`CI_AWS_ECS_TASK_DEFINITION`|`ecs_demo`| The ECS task definition is updated by `production_ecs` job. | - -### Make a change to the demo application - -Change a file in the project and see if it's reflected in the demo application on ECS: - -1. Go to **ecs-demo** project on GitLab. -1. Open the file at **app > views > welcome > `index.html.erb`**. -1. Select **Edit**. -1. Change the text to `You're on ECS!`. -1. Select **Commit Changes**. This automatically triggers a new pipeline. Wait until it finishes. -1. [Access the running application on the ECS cluster](#view-the-demo-application). You should see - this: - - ![Create project](img/view-running-app-2.png) - -Congratulations! You successfully set up continuous deployment to ECS. - -## Further reading - -- If you're interested in more of the continuous deployments to clouds, see [cloud deployments](../index.md). -- If you want to quickly set up DevSecOps in your project, see [Auto DevOps](../../../topics/autodevops/index.md). -- If you want to quickly set up the production-grade environment, see [the 5 Minute Production App](https://gitlab.com/gitlab-org/5-minute-production-app/deploy-template/-/blob/master/README.md). +<!-- This redirect file can be deleted after <2022-09-18>. --> +<!-- Redirects that point to other docs in the same project expire in three months. --> +<!-- Redirects that point to docs in a different project or site (for example, link is not relative and starts with `https:`) expire in one year. --> +<!-- Before deletion, see: https://docs.gitlab.com/ee/development/documentation/redirects.html -->
\ No newline at end of file diff --git a/doc/ci/cloud_deployment/index.md b/doc/ci/cloud_deployment/index.md index c89ce2675e4..c5be2328264 100644 --- a/doc/ci/cloud_deployment/index.md +++ b/doc/ci/cloud_deployment/index.md @@ -5,165 +5,101 @@ info: To determine the technical writer assigned to the Stage/Group associated w type: howto --- -# Cloud deployment **(FREE)** +# Deploy to AWS from GitLab CI/CD **(FREE)** -Interacting with a major cloud provider may have become a much needed task that's -part of your delivery process. With GitLab you can -[deploy your application anywhere](https://about.gitlab.com/stages-devops-lifecycle/deploy-targets/). +GitLab provides Docker images with the libraries and tools you need to deploy +to AWS. You can reference these images in your CI/CD pipeline. -For some specific deployment targets, GitLab makes this process less painful by providing Docker -images with the needed libraries and tools pre-installed. By referencing them in your -CI/CD pipeline, you can interact with your chosen cloud provider more easily. +If you're using GitLab.com and deploying to the [Amazon Elastic Container Service](https://aws.amazon.com/ecs/) (ECS), +read about [deploying to ECS](ecs/quick_start_guide.md). -## AWS +## Authenticate GitLab with AWS -GitLab provides Docker images that you can use to [run AWS commands from GitLab CI/CD](#run-aws-commands-from-gitlab-cicd), and a template to make -it easier to [deploy to AWS](#deploy-your-application-to-the-aws-elastic-container-service-ecs). +To use GitLab CI/CD to connect to AWS, you must authenticate. +After you set up authentication, you can configure CI/CD to deploy. -### Quick start +1. Sign on to your AWS account. +1. Create [an IAM user](https://console.aws.amazon.com/iam/home#/home). +1. Select your user to access its details. Go to **Security credentials > Create a new access key**. +1. Note the **Access key ID** and **Secret access key**. +1. In your GitLab project, go to **Settings > CI/CD**. Set the following + [CI/CD variables](../variables/index.md): -If you're using GitLab.com, see the [quick start guide](ecs/quick_start_guide.md) -for setting up Continuous Deployment to [AWS Elastic Container Service](https://aws.amazon.com/ecs/) (ECS). + | Environment variable name | Value | + |:-------------------------------|:------------------------| + | `AWS_ACCESS_KEY_ID` | Your Access key ID. | + | `AWS_SECRET_ACCESS_KEY` | Your secret access key. | + | `AWS_DEFAULT_REGION` | Your region code. You might want to confirm that the AWS service you intend to use is [available in the chosen region](https://aws.amazon.com/about-aws/global-infrastructure/regional-product-services/). | -### Run AWS commands from GitLab CI/CD +1. Variables are [protected by default](../variables/index.md#protected-cicd-variables). + To use GitLab CI/CD with branches or tags that are not protected, + clear the **Protect variable** checkbox. -> [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/31167) in GitLab 12.6. +## Use an image to run AWS commands -The GitLab AWS Docker image provides the [AWS Command Line Interface](https://aws.amazon.com/cli/), -which enables you to run `aws` commands. As part of your deployment strategy, you can run `aws` commands directly from -`.gitlab-ci.yml` by specifying the [GitLab AWS Docker image](https://gitlab.com/gitlab-org/cloud-deploy). +If an image contains the [AWS Command Line Interface](https://aws.amazon.com/cli/), +you can reference the image in your project's `.gitlab-ci.yml` file. Then you can run +`aws` commands in your CI/CD jobs. -Some credentials are required to be able to run `aws` commands: +For example: -1. Sign up for [an AWS account](https://docs.aws.amazon.com/IAM/latest/UserGuide/getting-set-up.html) if you don't have one yet. -1. Log in onto the console and create [a new IAM user](https://console.aws.amazon.com/iam/home#/home). -1. Select your newly created user to access its details. Navigate to **Security credentials > Create a new access key**. - - NOTE: - A new **Access key ID** and **Secret access key** are generated. Please take a note of them right away. - -1. In your GitLab project, go to **Settings > CI/CD**. Set the following as - [CI/CD variables](../variables/index.md) - (see table below): - - - Access key ID. - - Secret access key. - - Region code. You can check the [list of AWS regional endpoints](https://docs.aws.amazon.com/general/latest/gr/rande.html#regional-endpoints). - You might want to check if the AWS service you intend to use is - [available in the chosen region](https://aws.amazon.com/about-aws/global-infrastructure/regional-product-services/). - - | Environment variable name | Value | - |:-------------------------------|:-----------------------| - | `AWS_ACCESS_KEY_ID` | Your Access key ID | - | `AWS_SECRET_ACCESS_KEY` | Your Secret access key | - | `AWS_DEFAULT_REGION` | Your region code | - - NOTE: - When you create a variable it's set to be [protected by default](../variables/index.md#protected-cicd-variables). If you want to use the `aws` commands on branches or tags that are not protected, make sure to uncheck the **Protect variable** checkbox. - -1. You can now use `aws` commands in the `.gitlab-ci.yml` file of this project: - - ```yaml - deploy: - stage: deploy - image: registry.gitlab.com/gitlab-org/cloud-deploy/aws-base:latest # see the note below - script: - - aws s3 ... - - aws create-deployment ... - ``` +```yaml +deploy: + stage: deploy + image: registry.gitlab.com/gitlab-org/cloud-deploy/aws-base:latest + script: + - aws s3 ... + - aws create-deployment ... +``` - NOTE: - The image used in the example above - (`registry.gitlab.com/gitlab-org/cloud-deploy/aws-base:latest`) is hosted on the [GitLab - Container Registry](../../user/packages/container_registry/index.md) and is - ready to use. Alternatively, replace the image with one hosted on AWS ECR. +GitLab provides a Docker image that includes the AWS CLI: -### Use an AWS Elastic Container Registry (ECR) image in your CI/CD +- Images are hosted in the GitLab Container Registry. The latest image is + `registry.gitlab.com/gitlab-org/cloud-deploy/aws-base:latest`. +- [Images are stored in a GitLab repository](https://gitlab.com/gitlab-org/cloud-deploy/-/tree/master/aws). -Instead of referencing an image hosted on the GitLab Registry, you can -reference an image hosted on any third-party registry, such as the -[Amazon Elastic Container Registry (ECR)](https://aws.amazon.com/ecr/). +Alternately, you can use an [Amazon Elastic Container Registry (ECR)](https://aws.amazon.com/ecr/) image. +[Learn how to push an image to your ECR repository](https://docs.aws.amazon.com/AmazonECR/latest/userguide/docker-push-ecr-image.html). -To do so, [push your image into your ECR -repository](https://docs.aws.amazon.com/AmazonECR/latest/userguide/docker-push-ecr-image.html). -Then reference it in your `.gitlab-ci.yml` file and replace the `image` -path to point to your ECR image. +You can also use an image from any third-party registry. -### Deploy your application to the AWS Elastic Container Service (ECS) +## Deploy your application to ECS > - [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/207962) in GitLab 12.9. > - The `Deploy-ECS.gitlab-ci.yml` template was [moved](https://gitlab.com/gitlab-org/gitlab/-/issues/220821) to `AWS/Deploy-ECS.gitlab-ci.yml` in GitLab 13.2. -GitLab provides a series of [CI templates that you can include in your project](../yaml/index.md#include). -To automate deployments of your application to your [Amazon Elastic Container Service](https://aws.amazon.com/ecs/) (AWS ECS) -cluster, you can `include` the `AWS/Deploy-ECS.gitlab-ci.yml` template in your `.gitlab-ci.yml` file. +You can automate deployments of your application to your [Amazon ECS](https://aws.amazon.com/ecs/) +cluster. -GitLab also provides [Docker images](https://gitlab.com/gitlab-org/cloud-deploy/-/tree/master/aws) that can be used in your `.gitlab-ci.yml` file to simplify working with AWS: +Prerequisites: -- Use `registry.gitlab.com/gitlab-org/cloud-deploy/aws-base:latest` to use AWS CLI commands. -- Use `registry.gitlab.com/gitlab-org/cloud-deploy/aws-ecs:latest` to deploy your application to AWS ECS. +- [Authenticate AWS with GitLab](#authenticate-gitlab-with-aws). +- Create a cluster on Amazon ECS. +- Create related components, like an ECS service, a database on Amazon RDS, and so on. +- Create an ECS task definition, where the value for the `containerDefinitions[].name` attribute is + the same as the `Container name` defined in your targeted ECS service. The task definition can be: + - An existing task definition in ECS. + - [In GitLab 13.3 and later](https://gitlab.com/gitlab-org/gitlab/-/issues/222618), + a JSON file in your GitLab project. Use the + [template in the AWS documentation](https://docs.aws.amazon.com/AmazonECS/latest/developerguide/create-task-definition.html#task-definition-template) + and save the file in your project. For example `<project-root>/ci/aws/task-definition.json`. -Before getting started with this process, you need a cluster on AWS ECS, as well as related -components, like an ECS service, ECS task definition, a database on Amazon RDS, and so on. -[Read more about AWS ECS](https://docs.aws.amazon.com/AmazonECS/latest/developerguide/Welcome.html). - -The ECS task definition can be: - -- An existing task definition in AWS ECS -- A JSON file containing a task definition. Create the JSON file by using the template provided in - the [AWS documentation](https://docs.aws.amazon.com/AmazonECS/latest/developerguide/create-task-definition.html#task-definition-template). - Copy the task definition into a new file in your project, for example `<project-root>/ci/aws/task-definition.json`. - [Available](https://gitlab.com/gitlab-org/gitlab/-/issues/222618) in GitLab 13.3 and later. - -After you have these prerequisites ready, follow these steps: - -1. Make sure your AWS credentials are set up as CI/CD variables for your - project. You can follow [the steps above](#run-aws-commands-from-gitlab-cicd) to complete this setup. -1. Add these variables to your project's `.gitlab-ci.yml` file, or in the project's - [CI/CD settings](../variables/index.md#custom-cicd-variables): - - - `CI_AWS_ECS_CLUSTER`: The name of the AWS ECS cluster that you're targeting for your deployments. - - `CI_AWS_ECS_SERVICE`: The name of the targeted service tied to your AWS ECS cluster. - - `CI_AWS_ECS_TASK_DEFINITION`: The name of an existing task definition in ECS tied - to the service mentioned above. - - ```yaml - variables: - CI_AWS_ECS_CLUSTER: my-cluster - CI_AWS_ECS_SERVICE: my-service - CI_AWS_ECS_TASK_DEFINITION: my-task-definition - ``` - - You can find these names after selecting the targeted cluster on your [AWS ECS dashboard](https://console.aws.amazon.com/ecs/home): - - ![AWS ECS dashboard](../img/ecs_dashboard_v12_9.png) - - Alternatively, if you want to use a task definition defined in a JSON file, use - `CI_AWS_ECS_TASK_DEFINITION_FILE` instead: - - ```yaml - variables: - CI_AWS_ECS_CLUSTER: my-cluster - CI_AWS_ECS_SERVICE: my-service - CI_AWS_ECS_TASK_DEFINITION_FILE: ci/aws/my_task_definition.json - ``` +To deploy to your ECS cluster: - You can create your `CI_AWS_ECS_TASK_DEFINITION_FILE` variable as a - [file-typed CI/CD variable](../variables/index.md#cicd-variable-types) instead of a - regular CI/CD variable. If you choose to do so, set the variable value to be the full contents of - the JSON task definition. You can then remove the JSON file from your project. +1. In your GitLab project, go to **Settings > CI/CD**. Set the following + [CI/CD variables](../variables/index.md). You can find these names by + selecting the targeted cluster on your [Amazon ECS dashboard](https://console.aws.amazon.com/ecs/home). - In both cases, make sure that the value for the `containerDefinitions[].name` attribute is - the same as the `Container name` defined in your targeted ECS service. + | Environment variable name | Value | + |:-------------------------------|:------------------------| + | `CI_AWS_ECS_CLUSTER` | The name of the AWS ECS cluster that you're targeting for your deployments. | + | `CI_AWS_ECS_SERVICE` | The name of the targeted service tied to your AWS ECS cluster. | + | `CI_AWS_ECS_TASK_DEFINITION` | If the task definition is in ECS, the name of the task definition tied to the service. | + | `CI_AWS_ECS_TASK_DEFINITION_FILE` | If the task definition is a JSON file in GitLab, the filename, including the path. For example, `ci/aws/my_task_definition.json`. If the name of the task definition in your JSON file is the same name as an existing task definition in ECS, then a new revision is created when CI/CD runs. Otherwise, a brand new task definition is created, starting at revision 1. | WARNING: - `CI_AWS_ECS_TASK_DEFINITION_FILE` takes precedence over `CI_AWS_ECS_TASK_DEFINITION` if both these - variables are defined within your project. - - NOTE: - If the name of the task definition you wrote in your JSON file is the same name - as an existing task definition on AWS, then a new revision is created for it. - Otherwise, a brand new task definition is created, starting at revision 1. + If you define both `CI_AWS_ECS_TASK_DEFINITION_FILE` and `CI_AWS_ECS_TASK_DEFINITION`, + `CI_AWS_ECS_TASK_DEFINITION_FILE` takes precedence. 1. Include this template in `.gitlab-ci.yml`: @@ -172,86 +108,78 @@ After you have these prerequisites ready, follow these steps: - template: AWS/Deploy-ECS.gitlab-ci.yml ``` - The `AWS/Deploy-ECS` template ships with GitLab and is available [on - GitLab.com](https://gitlab.com/gitlab-org/gitlab/-/blob/master/lib/gitlab/ci/templates/AWS/Deploy-ECS.gitlab-ci.yml). + The `AWS/Deploy-ECS` template ships with GitLab and is available + [on GitLab.com](https://gitlab.com/gitlab-org/gitlab/-/blob/master/lib/gitlab/ci/templates/AWS/Deploy-ECS.gitlab-ci.yml). -1. Commit and push your updated `.gitlab-ci.yml` to your project's repository, and you're done! +1. Commit and push your updated `.gitlab-ci.yml` to your project's repository. - Your application Docker image is rebuilt and pushed to the GitLab registry. - If your image is located in a private registry, make sure your task definition is - [configured with a `repositoryCredentials` attribute](https://docs.aws.amazon.com/AmazonECS/latest/developerguide/private-auth.html). +Your application Docker image is rebuilt and pushed to the GitLab Container Registry. +If your image is located in a private registry, make sure your task definition is +[configured with a `repositoryCredentials` attribute](https://docs.aws.amazon.com/AmazonECS/latest/developerguide/private-auth.html). - Then the targeted task definition is updated with the location of the new - Docker image, and a new revision is created in ECS as result. +The targeted task definition is updated with the location of the new +Docker image, and a new revision is created in ECS as result. - Finally, your AWS ECS service is updated with the new revision of the - task definition, making the cluster pull the newest version of your - application. +Finally, your AWS ECS service is updated with the new revision of the +task definition, making the cluster pull the newest version of your +application. + +NOTE: +ECS deploy jobs wait for the rollout to complete before exiting. To disable this behavior, +set `CI_AWS_ECS_WAIT_FOR_ROLLOUT_COMPLETE_DISABLED` to a non-empty value. WARNING: The [`AWS/Deploy-ECS.gitlab-ci.yml`](https://gitlab.com/gitlab-org/gitlab/-/blob/master/lib/gitlab/ci/templates/AWS/Deploy-ECS.gitlab-ci.yml) -template includes both the [`Jobs/Build.gitlab-ci.yml`](https://gitlab.com/gitlab-org/gitlab/-/blob/master/lib/gitlab/ci/templates/Jobs/Build.gitlab-ci.yml) -and [`Jobs/Deploy/ECS.gitlab-ci.yml`](https://gitlab.com/gitlab-org/gitlab/-/blob/master/lib/gitlab/ci/templates/Jobs/Deploy/ECS.gitlab-ci.yml) -"sub-templates". Do not include these "sub-templates" on their own, and only include the main -`AWS/Deploy-ECS.gitlab-ci.yml` template. The "sub-templates" are designed to only be -used along with the main template. They may move or change unexpectedly causing your -pipeline to fail if you didn't include the main template. Also, the job names within -these templates may change. Do not override these jobs names in your own pipeline, -as the override stops working when the name changes. - -Alternatively, if you don't wish to use the `AWS/Deploy-ECS.gitlab-ci.yml` template -to deploy to AWS ECS, you can always use our -`aws-base` Docker image to run your own [AWS CLI commands for ECS](https://docs.aws.amazon.com/cli/latest/reference/ecs/index.html#cli-aws-ecs). +template includes two templates: [`Jobs/Build.gitlab-ci.yml`](https://gitlab.com/gitlab-org/gitlab/-/blob/master/lib/gitlab/ci/templates/Jobs/Build.gitlab-ci.yml) +and [`Jobs/Deploy/ECS.gitlab-ci.yml`](https://gitlab.com/gitlab-org/gitlab/-/blob/master/lib/gitlab/ci/templates/Jobs/Deploy/ECS.gitlab-ci.yml). Do not include these templates on their own. Only include the +`AWS/Deploy-ECS.gitlab-ci.yml` template. These other templates are designed to be +used only with the main template. They may move or change unexpectedly. Also, the job names within +these templates may change. Do not override these job names in your own pipeline, +because the override stops working when the name changes. -```yaml -deploy: - stage: deploy - image: registry.gitlab.com/gitlab-org/cloud-deploy/aws-base:latest - script: - - aws ecs register-task-definition ... -``` - -### Provision and deploy to your AWS Elastic Compute Cloud (EC2) +## Deploy your application to EC2 > [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/201742) in GitLab 13.5. -You can use the `AWS/CF-Provision-and-Deploy-EC2` CI template to perform the -following actions within the same pipeline: +GitLab provides a template, called `AWS/CF-Provision-and-Deploy-EC2`, +to assist you in deploying to Amazon EC2. -1. **Create stack**: Provision your own infrastructure by leveraging the [AWS CloudFormation](https://aws.amazon.com/cloudformation/) API. -1. **Push to S3**: Push your previously-built artifact to an [AWS S3](https://aws.amazon.com/s3/) bucket. -1. **Deploy to EC2**: Deploy this pushed content onto an [AWS EC2](https://aws.amazon.com/ec2/) instance. +When you configure related JSON objects and use the template, the pipeline: -![CF-Provision-and-Deploy-EC2 diagram](../img/cf_ec2_diagram_v13_5.png) +1. **Creates the stack**: Your infrastructure is provisioned by using + the [AWS CloudFormation](https://aws.amazon.com/cloudformation/) API. +1. **Pushes to an S3 bucket**: When your build runs, it creates an artifact. + The artifact is pushed to an [AWS S3](https://aws.amazon.com/s3/) bucket. +1. **Deploys to EC2**: The content is deployed on an [AWS EC2](https://aws.amazon.com/ec2/) instance. -#### Run the `AWS/CF-Provision-and-Deploy-EC2.gitlab-ci.yml` template +![CF-Provision-and-Deploy-EC2 diagram](../img/cf_ec2_diagram_v13_5.png) -To run the `AWS/CF-Provision-and-Deploy-EC2.gitlab-ci.yml` template, you must -pass three JSON input objects, based on existing templates: +### Configure the template and JSON -1. The AWS documentation provides templates for the _Create stack_ and _Deploy to EC2_ steps (links - below). We provide the template for the remaining step, _Push to S3_: +To deploy to EC2, complete the following steps. - - [Template for the _Create stack_ step on AWS](https://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/template-anatomy.html). - - Template for the _Push to S3_ step. Note that `source` is where a preceding `build` job built - your application, exporting the build through [`artifacts:paths`](../yaml/index.md#artifactspaths): +1. Create JSON for your stack. Use the [AWS template](https://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/template-anatomy.html). +1. Create JSON to push to S3. Include the following details. - ```json - { - "applicationName": "string", - "source": "string", - "s3Location": "s3://your/bucket/project_built_file...]" - } - ``` + ```json + { + "applicationName": "string", + "source": "string", + "s3Location": "s3://your/bucket/project_built_file...]" + } + ``` - - [Template for the _Deploy to EC2_ step on AWS](https://docs.aws.amazon.com/codedeploy/latest/APIReference/API_CreateDeployment.html). + The `source` is the location where a `build` job built your application. + The build is saved to [`artifacts:paths`](../yaml/index.md#artifactspaths). -1. After you have completed these three templates based on your requirements, you - have two ways to pass in these JSON objects: +1. Create JSON to deploy to EC2. Use the [AWS template](https://docs.aws.amazon.com/codedeploy/latest/APIReference/API_CreateDeployment.html). +1. Make the JSON objects accessible to your pipeline: + - If you want these JSON objects saved in your repository, save the objects as three + separate files. - - They can be three actual files located in your project. You must specify their path relative to - your project root in your `.gitlab-ci.yml` file, using the following CI/CD variables. For example, if - your files are in a `<project_root>/aws` folder: + In your `.gitlab-ci.yml` file, add [CI/CD variables](../variables/index.md) + that point to the file paths relative to the project root. For example, + if your JSON files are in a `<project_root>/aws` folder: ```yaml variables: @@ -260,71 +188,30 @@ pass three JSON input objects, based on existing templates: CI_AWS_EC2_DEPLOYMENT_FILE: 'aws/create_deployment.json' ``` - - Alternatively, you can provide these JSON objects as [file-typed CI/CD variables](../variables/index.md#cicd-variable-types). - In your project, go to **Settings > CI/CD > Variables** and add - the three variables listed above as file-typed CI/CD variables. - For each variable, set the value to its corresponding JSON object. + - If you do not want these JSON objects saved in your repository, add each object + as a separate [file type CI/CD variable](../variables/index.md#cicd-variable-types) + in the project settings. Use the same variable names as above. -1. Provide the name of the stack you're creating and/or targeting, as a CI/CD variable: +1. In your `.gitlab-ci.yml` file, create a CI/CD variable for the name of the stack. For example: ```yaml variables: CI_AWS_CF_STACK_NAME: 'YourStackName' ``` -1. Add this CI template to your `.gitlab-ci.yml`: +1. In your `.gitlab-ci.yml` file, add the CI template: ```yaml include: - template: AWS/CF-Provision-and-Deploy-EC2.gitlab-ci.yml ``` -When running your project pipeline at this point: - -- Your AWS CloudFormation stack is created based on the content of your - `CI_AWS_CF_CREATE_STACK_FILE` file/variable. - If your stack already exists, this step is skipped, but the `provision` job it belongs to still - runs. -- Your built application is pushed to your S3 bucket then and deployed to your EC2 instance, based - on the related JSON object's content. The deployment job finishes whenever the deployment to EC2 - is done or has failed. - -#### Custom build job for Auto DevOps - -> [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/216008) in GitLab 13.6. - -To leverage [Auto DevOps](../../topics/autodevops/index.md) for your project when deploying to -AWS EC2, first you must define [your AWS credentials as CI/CD variables](#run-aws-commands-from-gitlab-cicd). - -Next, define a job for the `build` stage. To do so, you must reference the -`Auto-DevOps.gitlab-ci.yml` template and include a job named `build_artifact` in your -`.gitlab-ci.yml` file. For example: - -```yaml -# .gitlab-ci.yml - -include: - - template: Auto-DevOps.gitlab-ci.yml - -variables: - AUTO_DEVOPS_PLATFORM_TARGET: EC2 - -build_artifact: - stage: build - script: - - <your build script goes here> - artifacts: - paths: - - <built artifact> -``` - -<i class="fa fa-youtube-play youtube" aria-hidden="true"></i> -For a video walkthrough of this configuration process, see [Auto Deploy to EC2](https://www.youtube.com/watch?v=4B-qSwKnacA). - -### Deploy to Amazon EKS - -- [How to deploy your application to a GitLab-managed Amazon EKS cluster with Auto DevOps](https://about.gitlab.com/blog/2020/05/05/deploying-application-eks/) - -## Deploy to Google Cloud +1. Run the pipeline. -- [Deploying with GitLab on Google Cloud](https://about.gitlab.com/partners/technology-partners/google-cloud-platform/) + - Your AWS CloudFormation stack is created based on the content of your + `CI_AWS_CF_CREATE_STACK_FILE` variable. + If your stack already exists, this step is skipped, but the `provision` + job it belongs to still runs. + - Your built application is pushed to your S3 bucket then and deployed to your EC2 instance, based + on the related JSON object's content. The deployment job finishes when the deployment to EC2 + is done or has failed. |