diff options
Diffstat (limited to 'doc/administration/object_storage.md')
| -rw-r--r-- | doc/administration/object_storage.md | 84 |
1 files changed, 72 insertions, 12 deletions
diff --git a/doc/administration/object_storage.md b/doc/administration/object_storage.md index 49716883310..39365ffe404 100644 --- a/doc/administration/object_storage.md +++ b/doc/administration/object_storage.md @@ -18,6 +18,7 @@ GitLab has been tested on a number of object storage providers: - [Digital Ocean Spaces](https://www.digitalocean.com/products/spaces/) - [Oracle Cloud Infrastructure](https://docs.cloud.oracle.com/en-us/iaas/Content/Object/Tasks/s3compatibleapi.htm) - [Openstack Swift](https://docs.openstack.org/swift/latest/s3_compat.html) +- [Azure Blob storage](https://docs.microsoft.com/en-us/azure/storage/blobs/storage-blobs-introduction) - On-premises hardware and appliances from various storage vendors. - MinIO. We have [a guide to deploying this](https://docs.gitlab.com/charts/advanced/external-object-storage/minio.html) within our Helm Chart documentation. @@ -50,12 +51,17 @@ Using the consolidated object storage configuration has a number of advantages: - It enables the use of [encrypted S3 buckets](#encrypted-s3-buckets). - It [uploads files to S3 with proper `Content-MD5` headers](https://gitlab.com/gitlab-org/gitlab-workhorse/-/issues/222). -NOTE: **Note:** -Only AWS S3-compatible providers and Google are -supported at the moment since [direct upload -mode](../development/uploads.md#direct-upload) must be used. Background -upload is not supported in this mode. We recommend direct upload mode because -it does not require a shared folder, and [this setting may become the default](https://gitlab.com/gitlab-org/gitlab/-/issues/27331). +Because [direct upload mode](../development/uploads.md#direct-upload) +must be enabled, only the following providers can be used: + +- [Amazon S3-compatible providers](#s3-compatible-connection-settings) +- [Google Cloud Storage](#google-cloud-storage-gcs) +- [Azure Blob storage](#azure-blob-storage) + +Background upload is not supported with the consolidated object storage +configuration. We recommend enabling direct upload mode because it does +not require a shared folder, and [this setting may become the +default](https://gitlab.com/gitlab-org/gitlab/-/issues/27331). NOTE: **Note:** Consolidated object storage configuration cannot be used for @@ -112,7 +118,7 @@ See the section on [ETag mismatch errors](#etag-mismatch) for more details. AWS access key and secret access key/value pairs. For example: ```ruby - gitlab_rails['object_store_connection'] = { + gitlab_rails['object_store']['connection'] = { 'provider' => 'AWS', 'region' => '<eu-central-1>', 'use_iam_profile' => true @@ -158,7 +164,6 @@ See the section on [ETag mismatch errors](#etag-mismatch) for more details. ```toml [object_storage] - enabled = true provider = "AWS" [object_storage.s3] @@ -272,6 +277,61 @@ gitlab_rails['object_store']['connection'] = { } ``` +#### Azure Blob storage + +> [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/25877) in GitLab 13.4. + +Although Azure uses the word `container` to denote a collection of +blobs, GitLab standardizes on the term `bucket`. Be sure to configure +Azure container names in the `bucket` settings. + +The following are the valid connection parameters for Azure. Read the +[Azure Blob storage documentation](https://docs.microsoft.com/en-us/azure/storage/blobs/storage-blobs-introduction) +to learn more. + +| Setting | Description | Example | +|---------|-------------|---------| +| `provider` | Provider name | `AzureRM` | +| `azure_storage_account_name` | Name of the Azure Blob Storage account used to access the storage | `azuretest` | +| `azure_storage_access_key` | Storage account access key used to access the container. This is typically a secret, 512-bit encryption key encoded in base64. | `czV2OHkvQj9FKEgrTWJRZVRoV21ZcTN0Nnc5eiRDJkYpSkBOY1JmVWpYbjJy\nNHU3eCFBJUQqRy1LYVBkU2dWaw==\n` | +| `azure_storage_domain` | Domain name used to contact the Azure Blob Storage API (optional). Defaults to `blob.core.windows.net`. Set this if you are using Azure China, Azure Germany, Azure US Government, or some other custom Azure domain. | `blob.core.windows.net` | + +##### Azure example (consolidated form) + +For Omnibus installations, this is an example of the `connection` setting: + +```ruby +gitlab_rails['object_store']['connection'] = { + 'provider' => 'AzureRM', + 'azure_storage_account_name' => '<AZURE STORAGE ACCOUNT NAME>', + 'azure_storage_access_key' => '<AZURE STORAGE ACCESS KEY>', + 'azure_storage_domain' => '<AZURE STORAGE DOMAIN>', +} +``` + +###### Azure Workhorse settings (source installs only) + +NOTE: **Note:** +For source installations, Workhorse needs to be configured with the +Azure credentials as well. This is not needed in Omnibus installs because +the Workhorse settings are populated from the settings above. + +1. Edit `/home/git/gitlab-workhorse/config.toml` and add or amend the following lines: + + ```toml + [object_storage] + provider = "AzureRM" + + [object_storage.azurerm] + azure_storage_account_name = "<AZURE STORAGE ACCOUNT NAME>" + azure_storage_access_key = "<AZURE STORAGE ACCESS KEY>" + ``` + +If you are using a custom Azure storage domain, note that +`azure_storage_domain` does **not** have to be set in the Workhorse +configuration. This information is exchanged in an API call between +GitLab Rails and Workhorse. + #### OpenStack-compatible connection settings NOTE: **Note:** @@ -279,7 +339,7 @@ This is not compatible with the consolidated object storage form. OpenStack Swift is only supported with the storage-specific form. See the [S3 settings](#s3-compatible-connection-settings) if you want to use the consolidated form. -While OpenStack Swift provides S3 compatibliity, some users may want to use the +While OpenStack Swift provides S3 compatibility, some users may want to use the [Swift API](https://docs.openstack.org/swift/latest/api/object_api_v1_overview.html). Here are the valid connection settings below for the Swift API, provided by [fog-openstack](https://github.com/fog/fog-openstack): @@ -445,15 +505,15 @@ supported by consolidated configuration form, refer to the following guides: | [Backups](../raketasks/backup_restore.md#uploading-backups-to-a-remote-cloud-storage)|No| | [Job artifacts](job_artifacts.md#using-object-storage) and [incremental logging](job_logs.md#new-incremental-logging-architecture) | Yes | | [LFS objects](lfs/index.md#storing-lfs-objects-in-remote-object-storage) | Yes | -| [Uploads](uploads.md#using-object-storage-core-only) | Yes | +| [Uploads](uploads.md#using-object-storage) | Yes | | [Container Registry](packages/container_registry.md#use-object-storage) (optional feature) | No | | [Merge request diffs](merge_request_diffs.md#using-object-storage) | Yes | | [Mattermost](https://docs.mattermost.com/administration/config-settings.html#file-storage)| No | | [Packages](packages/index.md#using-object-storage) (optional feature) **(PREMIUM ONLY)** | Yes | | [Dependency Proxy](packages/dependency_proxy.md#using-object-storage) (optional feature) **(PREMIUM ONLY)** | Yes | | [Pseudonymizer](pseudonymizer.md#configuration) (optional feature) **(ULTIMATE ONLY)** | No | -| [Autoscale Runner caching](https://docs.gitlab.com/runner/configuration/autoscale.html#distributed-runners-caching) (optional for improved performance) | No | -| [Terraform state files](terraform_state.md#using-object-storage-core-only) | Yes | +| [Autoscale runner caching](https://docs.gitlab.com/runner/configuration/autoscale.html#distributed-runners-caching) (optional for improved performance) | No | +| [Terraform state files](terraform_state.md#using-object-storage) | Yes | ### Other alternatives to filesystem storage |