diff options
author | GitLab Bot <gitlab-bot@gitlab.com> | 2020-04-01 21:07:56 +0300 |
---|---|---|
committer | GitLab Bot <gitlab-bot@gitlab.com> | 2020-04-01 21:07:56 +0300 |
commit | 33aa02e7a38d8dfc5e470dd5d776c8d4ce5b2dd5 (patch) | |
tree | 8cd2bc4711d3a017d839760c7fbea267e2ba4951 /doc/administration | |
parent | 1219a9dce91f4edbc135dfc08299b4122b4825a8 (diff) |
Add latest changes from gitlab-org/gitlab@master
Diffstat (limited to 'doc/administration')
-rw-r--r-- | doc/administration/auth/ldap-ee.md | 20 | ||||
-rw-r--r-- | doc/administration/auth/ldap-troubleshooting.md | 49 | ||||
-rw-r--r-- | doc/administration/git_annex.md | 2 | ||||
-rw-r--r-- | doc/administration/instance_limits.md | 12 | ||||
-rw-r--r-- | doc/administration/job_artifacts.md | 79 | ||||
-rw-r--r-- | doc/administration/lfs/img/lfs-icon.png | bin | 4317 -> 0 bytes | |||
-rw-r--r-- | doc/administration/lfs/lfs_administration.md | 2 | ||||
-rw-r--r-- | doc/administration/lfs/manage_large_binaries_with_git_lfs.md | 272 | ||||
-rw-r--r-- | doc/administration/lfs/migrate_from_git_annex_to_git_lfs.md | 255 | ||||
-rw-r--r-- | doc/administration/operations/fast_ssh_key_lookup.md | 6 | ||||
-rw-r--r-- | doc/administration/repository_storage_types.md | 2 |
11 files changed, 129 insertions, 570 deletions
diff --git a/doc/administration/auth/ldap-ee.md b/doc/administration/auth/ldap-ee.md index 655e9df6f76..62b515f1a3f 100644 --- a/doc/administration/auth/ldap-ee.md +++ b/doc/administration/auth/ldap-ee.md @@ -43,7 +43,13 @@ The process will execute the following access checks: - Ensure the user is still present in LDAP. - If the LDAP server is Active Directory, ensure the user is active (not blocked/disabled state). This will only be checked if - `active_directory: true` is set in the LDAP configuration. [^1] + `active_directory: true` is set in the LDAP configuration. + +NOTE: **Note:** +In Active Directory, a user is marked as disabled/blocked if the user +account control attribute (`userAccountControl:1.2.840.113556.1.4.803`) +has bit 2 set. See <https://ctogonewild.com/2009/09/03/bitmask-searches-in-ldap/> +for more information. The user will be set to `ldap_blocked` state in GitLab if the above conditions fail. This means the user will not be able to login or push/pull code. @@ -93,7 +99,7 @@ following. EOS ``` -1. [Reconfigure GitLab][reconfigure] for the changes to take effect. +1. [Apply your changes to GitLab](../restart_gitlab.md#omnibus-gitlab-reconfigure). **Source configuration** @@ -108,7 +114,7 @@ following. group_base: ou=groups,dc=example,dc=com ``` -1. [Restart GitLab][restart] for the changes to take effect. +1. [Restart GitLab](../restart_gitlab.md#installations-from-source) for the changes to take effect. To take advantage of group sync, group owners or maintainers will need to [create one or more LDAP group links](#adding-group-links). @@ -183,7 +189,7 @@ group, as opposed to the full DN. EOS ``` -1. [Reconfigure GitLab][reconfigure] for the changes to take effect. +1. [Apply your changes to GitLab](../restart_gitlab.md#omnibus-gitlab-reconfigure). **Source configuration** @@ -199,7 +205,7 @@ group, as opposed to the full DN. admin_group: my_admin_group ``` -1. [Restart GitLab][restart] for the changes to take effect. +1. [Restart GitLab](../restart_gitlab.md#installations-from-source) for the changes to take effect. ## Global group memberships lock @@ -318,7 +324,7 @@ task. EOS ``` -1. [Reconfigure GitLab][reconfigure] for the changes to take effect. +1. [Apply your changes to GitLab](../restart_gitlab.md#omnibus-gitlab-reconfigure). **Source configuration** @@ -333,7 +339,7 @@ task. external_groups: ['interns', 'contractors'] ``` -1. [Restart GitLab][restart] for the changes to take effect. +1. [Restart GitLab](../restart_gitlab.md#installations-from-source) for the changes to take effect. ## Group sync technical details diff --git a/doc/administration/auth/ldap-troubleshooting.md b/doc/administration/auth/ldap-troubleshooting.md index ded5153133d..01c22711efc 100644 --- a/doc/administration/auth/ldap-troubleshooting.md +++ b/doc/administration/auth/ldap-troubleshooting.md @@ -84,7 +84,7 @@ following is most likely true: - The `bind_dn` user doesn't have enough permissions to traverse the user tree. - The user(s) don't fall under the [configured `base`](ldap.md#configuration). -- The [configured `user_filter`][user-filter] blocks access to the user(s). +- The [configured `user_filter`](ldap.md#using-an-ldap-filter-to-limit-access-to-your-gitlab-server) blocks access to the user(s). In this case, you con confirm which of the above is true using [ldapsearch](#ldapsearch) with the existing LDAP configuration in your @@ -97,7 +97,7 @@ here are some questions to ask yourself: - Does the user fall under the [configured `base`](ldap.md#configuration) in LDAP? The user must fall under this `base` to login. -- Does the user pass through the [configured `user_filter`][user-filter]? +- Does the user pass through the [configured `user_filter`](ldap.md#using-an-ldap-filter-to-limit-access-to-your-gitlab-server)? If one is not configured, this question can be ignored. If it is, then the user must also pass through this filter to be allowed to login. - Refer to our docs on [debugging the `user_filter`](#debug-ldap-user-filter). @@ -132,7 +132,7 @@ are true for the user in question: #### Email has already been taken A user tries to login with the correct LDAP credentials, is denied access, -and the [production.log][production-log] shows an error that looks like this: +and the [production.log](../logs.md#productionlog) shows an error that looks like this: ```plaintext (LDAP) Error saving user <USER DN> (email@example.com): ["Email has already been taken"] @@ -168,7 +168,7 @@ profile](../../user/profile/index.md#user-profile) or an admin can do it. #### Debug LDAP user filter [`ldapsearch`](#ldapsearch) allows you to test your configured -[user filter][user-filter] +[user filter](ldap.md#using-an-ldap-filter-to-limit-access-to-your-gitlab-server) to confirm that it returns the users you expect it to return. ```shell @@ -184,7 +184,7 @@ ldapsearch -H ldaps://$host:$port -D "$bind_dn" -y bind_dn_password.txt -b "$ba #### Sync all users **(STARTER ONLY)** -The output from a manual [user sync][user-sync] can show you what happens when +The output from a manual [user sync](ldap-ee.md#user-sync) can show you what happens when GitLab tries to sync its users against LDAP. Enter the [rails console](#rails-console) and then run: @@ -297,9 +297,9 @@ LDAP group sync, but for some reason it's not happening. There are several things to check to debug the situation. - Ensure LDAP configuration has a `group_base` specified. - [This configuration][group-sync] is required for group sync to work properly. + [This configuration](ldap-ee.md#group-sync) is required for group sync to work properly. - Ensure the correct [LDAP group link is added to the GitLab - group][group-links]. + group](ldap-ee.md#adding-group-links). - Check that the user has an LDAP identity: 1. Sign in to GitLab as an administrator user. 1. Navigate to **Admin area -> Users**. @@ -312,7 +312,7 @@ things to check to debug the situation. interval](ldap-ee.md#adjusting-ldap-group-sync-schedule) for the group to sync. To speed up the process, either go to the GitLab group **Settings -> Members** and press **Sync now** (sync one group) or [run the group sync Rake - task][group-sync-rake] (sync all groups). + task](../raketasks/ldap.md#run-a-group-sync) (sync all groups). If all of the above looks good, jump in to a little more advanced debugging in the rails console. @@ -352,9 +352,9 @@ GitLab syncs the `admin_group`. NOTE: **NOTE:** To sync all groups manually when debugging is unnecessary, [use the Rake -task][group-sync-rake] instead. +task](../raketasks/ldap.md#run-a-group-sync) instead. -The output from a manual [group sync][group-sync] can show you what happens +The output from a manual [group sync](ldap-ee.md#group-sync) can show you what happens when GitLab syncs its LDAP group memberships against LDAP. ```ruby @@ -449,7 +449,7 @@ this line will indicate the sync is finished: Finished syncing admin users for 'ldapmain' provider ``` -If [admin sync][admin-sync] is not configured, you'll see a message +If [admin sync](ldap-ee.md#administrator-sync) is not configured, you'll see a message stating as such: ```shell @@ -541,7 +541,7 @@ for each of these users. ### LDAP check -The [Rake task to check LDAP][ldap-check] is a valuable tool +The [Rake task to check LDAP](../raketasks/ldap.md#check) is a valuable tool to help determine whether GitLab can successfully establish a connection to LDAP and can get so far as to even read users. @@ -561,11 +561,11 @@ users, [see what to do when no users are found](#no-users-are-found). ### GitLab logs If a user account is blocked or unblocked due to the LDAP configuration, a -message will be [logged to `application.log`][application-log]. +message will be [logged to `application.log`](../logs.md#applicationlog). If there is an unexpected error during an LDAP lookup (configuration error, timeout), the login is rejected and a message will be [logged to -`production.log`][production-log]. +`production.log`](../logs.md#productionlog). ### ldapsearch @@ -649,24 +649,3 @@ console](#rails-console) and run: ```ruby Rails.logger.level = Logger::DEBUG ``` - -<!-- LINK REFERENCES --> - -[tail-logs]: https://docs.gitlab.com/omnibus/settings/logs.html#tail-logs-in-a-console-on-the-server -[production-log]: ../logs.md#productionlog -[application-log]: ../logs.md#applicationlog -[reconfigure]: ../restart_gitlab.md#omnibus-gitlab-reconfigure -[restart]: ../restart_gitlab.md#installations-from-source -[ldap-check]: ../raketasks/ldap.md#check -[group-sync-rake]: ../raketasks/ldap.md#run-a-group-sync -[user-filter]: ldap.md#using-an-ldap-filter-to-limit-access-to-your-gitlab-server -[user-sync]: ldap-ee.md#user-sync -[group-sync]: ldap-ee.md#group-sync -[admin-sync]: ldap-ee.md#administrator-sync -[config]: ldap.md#configuration -[group-links]: ldap-ee.md#adding-group-links - -[^1]: In Active Directory, a user is marked as disabled/blocked if the user - account control attribute (`userAccountControl:1.2.840.113556.1.4.803`) - has bit 2 set. See <https://ctogonewild.com/2009/09/03/bitmask-searches-in-ldap/> - for more information. diff --git a/doc/administration/git_annex.md b/doc/administration/git_annex.md index 8f285cd9bcb..4ac70e7fac2 100644 --- a/doc/administration/git_annex.md +++ b/doc/administration/git_annex.md @@ -236,7 +236,7 @@ push origin [deprecate-annex-issue]: https://gitlab.com/gitlab-org/gitlab/issues/1648 [git-annex]: https://git-annex.branchable.com/ "git-annex website" [gitlab shell]: https://gitlab.com/gitlab-org/gitlab-shell "GitLab Shell repository" -[guide]: lfs/migrate_from_git_annex_to_git_lfs.html +[guide]: ../topics/git/lfs/migrate_from_git_annex_to_git_lfs.md [issue]: https://git-annex.branchable.com/forum/Error_from_git-annex-shell_on_creation_of_gcrypt_special_remote/ "git-annex issue" [reconfigure GitLab]: restart_gitlab.md#omnibus-gitlab-reconfigure [restart GitLab]: restart_gitlab.md#installations-from-source diff --git a/doc/administration/instance_limits.md b/doc/administration/instance_limits.md index 034d80be90b..99abf5d3864 100644 --- a/doc/administration/instance_limits.md +++ b/doc/administration/instance_limits.md @@ -19,6 +19,18 @@ will fail. - **Max limit:** 5.000 comments +## Size of comments and descriptions of issues, merge requests, and epics + +> [Introduced](https://gitlab.com/gitlab-org/gitlab-foss/issues/61974) in GitLab 12.2. + +There is a limit to the size of comments and descriptions of issues, merge requests, and epics. +Attempting to add a body of text larger than the limit will result in an error, and the +item will not be created. + +It's possible that this limit will be changed to a lower number in the future. + +- **Max size:** ~1 million characters / ~1 MB + ## Number of pipelines per Git push > [Introduced](https://gitlab.com/gitlab-org/gitlab-foss/issues/51401) in GitLab 11.10. diff --git a/doc/administration/job_artifacts.md b/doc/administration/job_artifacts.md index cc1d744d73b..c003761299e 100644 --- a/doc/administration/job_artifacts.md +++ b/doc/administration/job_artifacts.md @@ -198,6 +198,85 @@ JUnit test report artifact (`junit.xml.gz`) migration [is not supported](https://gitlab.com/gitlab-org/gitlab/issues/27698) by the `gitlab:artifacts:migrate` script. +### OpenStack compatible connection settings + +The connection settings match those provided by [Fog](https://github.com/fog), and are as follows: + +| Setting | Description | Default | +|---------|-------------|---------| +| `provider` | Always `OpenStack` for compatible hosts | OpenStack | +| `openstack_username` | OpenStack username | | +| `openstack_api_key` | OpenStack API key | | +| `openstack_temp_url_key` | OpenStack key for generating temporary urls | | +| `openstack_auth_url` | OpenStack authentication endpont | | +| `openstack_region` | OpenStack region | | +| `openstack_tenant_id` | OpenStack tenant ID | + +**In Omnibus installations:** + +_The uploads are stored by default in +`/var/opt/gitlab/gitlab-rails/shared/artifacts`._ + +1. Edit `/etc/gitlab/gitlab.rb` and add the following lines by replacing with + the values you want: + + ```ruby + gitlab_rails['artifacts_enabled'] = true + gitlab_rails['artifacts_object_store_enabled'] = true + gitlab_rails['artifacts_object_store_remote_directory'] = "artifacts" + gitlab_rails['artifacts_object_store_connection'] = { + 'provider' => 'OpenStack', + 'openstack_username' => 'OS_USERNAME', + 'openstack_api_key' => 'OS_PASSWORD', + 'openstack_temp_url_key' => 'OS_TEMP_URL_KEY', + 'openstack_auth_url' => 'https://auth.cloud.ovh.net', + 'openstack_region' => 'GRA', + 'openstack_tenant_id' => 'OS_TENANT_ID', + } + ``` + +1. Save the file and [reconfigure GitLab](restart_gitlab.md#omnibus-gitlab-reconfigure) for the changes to take effect. +1. Migrate any existing local artifacts to the object storage: + + ```shell + gitlab-rake gitlab:artifacts:migrate + ``` + +--- + +**In installations from source:** + +_The uploads are stored by default in +`/home/git/gitlab/shared/artifacts`._ + +1. Edit `/home/git/gitlab/config/gitlab.yml` and add or amend the following + lines: + + ```yaml + uploads: + object_store: + enabled: true + direct_upload: false + background_upload: true + proxy_download: false + remote_directory: "artifacts" + connection: + provider: OpenStack + openstack_username: OS_USERNAME + openstack_api_key: OS_PASSWORD + openstack_temp_url_key: OS_TEMP_URL_KEY + openstack_auth_url: 'https://auth.cloud.ovh.net' + openstack_region: GRA + openstack_tenant_id: OS_TENANT_ID + ``` + +1. Save the file and [restart GitLab](restart_gitlab.md#installations-from-source) for the changes to take effect. +1. Migrate any existing local artifacts to the object storage: + + ```shell + sudo -u git -H bundle exec rake gitlab:artifacts:migrate RAILS_ENV=production + ``` + ### Migrating from object storage to local storage **In Omnibus installations:** diff --git a/doc/administration/lfs/img/lfs-icon.png b/doc/administration/lfs/img/lfs-icon.png Binary files differdeleted file mode 100644 index eef9a14187a..00000000000 --- a/doc/administration/lfs/img/lfs-icon.png +++ /dev/null diff --git a/doc/administration/lfs/lfs_administration.md b/doc/administration/lfs/lfs_administration.md index a7ff624e1c6..10ff15b1ff4 100644 --- a/doc/administration/lfs/lfs_administration.md +++ b/doc/administration/lfs/lfs_administration.md @@ -4,7 +4,7 @@ disqus_identifier: 'https://docs.gitlab.com/ee/workflow/lfs/lfs_administration.h # GitLab Git LFS Administration -Documentation on how to use Git LFS are under [Managing large binary files with Git LFS doc](manage_large_binaries_with_git_lfs.md). +Documentation on how to use Git LFS are under [Managing large binary files with Git LFS doc](../../topics/git/lfs/index.md). ## Requirements diff --git a/doc/administration/lfs/manage_large_binaries_with_git_lfs.md b/doc/administration/lfs/manage_large_binaries_with_git_lfs.md index 27cf27fddc9..4656bccf5e6 100644 --- a/doc/administration/lfs/manage_large_binaries_with_git_lfs.md +++ b/doc/administration/lfs/manage_large_binaries_with_git_lfs.md @@ -1,273 +1,5 @@ --- -disqus_identifier: 'https://docs.gitlab.com/ee/workflow/lfs/manage_large_binaries_with_git_lfs.html' +redirect_to: '../../topics/git/lfs/index.md' --- -# Git LFS - -Managing large files such as audio, video and graphics files has always been one -of the shortcomings of Git. The general recommendation is to not have Git repositories -larger than 1GB to preserve performance. - -![Git LFS tracking status](img/lfs-icon.png) - -An LFS icon is shown on files tracked by Git LFS to denote if a file is stored -as a blob or as an LFS pointer. - -## How it works - -Git LFS client talks with the GitLab server over HTTPS. It uses HTTP Basic Authentication -to authorize client requests. Once the request is authorized, Git LFS client receives -instructions from where to fetch or where to push the large file. - -## GitLab server configuration - -Documentation for GitLab instance administrators is under [LFS administration doc](lfs_administration.md). - -## Requirements - -- Git LFS is supported in GitLab starting with version 8.2 -- Git LFS must be enabled under project settings -- [Git LFS client](https://git-lfs.github.com) version 1.0.1 and up - -## Known limitations - -- Git LFS v1 original API is not supported since it was deprecated early in LFS - development -- When SSH is set as a remote, Git LFS objects still go through HTTPS -- Any Git LFS request will ask for HTTPS credentials to be provided so a good Git - credentials store is recommended -- Git LFS always assumes HTTPS so if you have GitLab server on HTTP you will have - to add the URL to Git config manually (see [troubleshooting](#troubleshooting)) - -NOTE: **Note:** -With 8.12 GitLab added LFS support to SSH. The Git LFS communication -still goes over HTTP, but now the SSH client passes the correct credentials -to the Git LFS client, so no action is required by the user. - -## Using Git LFS - -Lets take a look at the workflow when you need to check large files into your Git -repository with Git LFS. For example, if you want to upload a very large file and -check it into your Git repository: - -```shell -git clone git@gitlab.example.com:group/project.git -git lfs install # initialize the Git LFS project -git lfs track "*.iso" # select the file extensions that you want to treat as large files -``` - -Once a certain file extension is marked for tracking as a LFS object you can use -Git as usual without having to redo the command to track a file with the same extension: - -```shell -cp ~/tmp/debian.iso ./ # copy a large file into the current directory -git add . # add the large file to the project -git commit -am "Added Debian iso" # commit the file meta data -git push origin master # sync the git repo and large file to the GitLab server -``` - -**Make sure** that `.gitattributes` is tracked by Git. Otherwise Git -LFS will not be working properly for people cloning the project: - -```shell -git add .gitattributes -``` - -Cloning the repository works the same as before. Git automatically detects the -LFS-tracked files and clones them via HTTP. If you performed the `git clone` -command with a SSH URL, you have to enter your GitLab credentials for HTTP -authentication. - -```shell -git clone git@gitlab.example.com:group/project.git -``` - -If you already cloned the repository and you want to get the latest LFS object -that are on the remote repository, eg. for a branch from origin: - -```shell -git lfs fetch origin master -``` - -### Migrate an existing repo to Git LFS - -Read the documentation on how to [migrate an existing Git repo with Git LFS](../../topics/git/migrate_to_git_lfs/index.md). - -### Removing objects from LFS - -To remove objects from LFS: - -1. Use [BFG-Cleaner](../../user/project/repository/reducing_the_repo_size_using_git.md#using-the-bfg-repo-cleaner) or [filter-branch](../../user/project/repository/reducing_the_repo_size_using_git.md#using-git-filter-branch) to remove the objects from the repository. -1. Delete the relevant LFS lines for the objects you have removed from your `.gitattributes` file and commit those changes. - -## File Locking - -> [Introduced](https://gitlab.com/gitlab-org/gitlab-foss/issues/35856) in GitLab 10.5. - -The first thing to do before using File Locking is to tell Git LFS which -kind of files are lockable. The following command will store PNG files -in LFS and flag them as lockable: - -```shell -git lfs track "*.png" --lockable -``` - -After executing the above command a file named `.gitattributes` will be -created or updated with the following content: - -```shell -*.png filter=lfs diff=lfs merge=lfs -text lockable -``` - -You can also register a file type as lockable without using LFS -(In order to be able to lock/unlock a file you need a remote server that implements the LFS File Locking API), -in order to do that you can edit the `.gitattributes` file manually: - -```shell -*.pdf lockable -``` - -After a file type has been registered as lockable, Git LFS will make -them readonly on the file system automatically. This means you will -need to lock the file before editing it. - -### Managing Locked Files - -Once you're ready to edit your file you need to lock it first: - -```shell -git lfs lock images/banner.png -Locked images/banner.png -``` - -This will register the file as locked in your name on the server: - -```shell -git lfs locks -images/banner.png joe ID:123 -``` - -Once you have pushed your changes, you can unlock the file so others can -also edit it: - -```shell -git lfs unlock images/banner.png -``` - -You can also unlock by id: - -```shell -git lfs unlock --id=123 -``` - -If for some reason you need to unlock a file that was not locked by you, -you can use the `--force` flag as long as you have a `maintainer` access on -the project: - -```shell -git lfs unlock --id=123 --force -``` - -## Troubleshooting - -### error: Repository or object not found - -There are a couple of reasons why this error can occur: - -- You don't have permissions to access certain LFS object - -Check if you have permissions to push to the project or fetch from the project. - -- Project is not allowed to access the LFS object - -LFS object you are trying to push to the project or fetch from the project is not -available to the project anymore. Probably the object was removed from the server. - -- Local Git repository is using deprecated LFS API - -### Invalid status for `<url>` : 501 - -Git LFS will log the failures into a log file. -To view this log file, while in project directory: - -```shell -git lfs logs last -``` - -If the status `error 501` is shown, it is because: - -- Git LFS is not enabled in project settings. Check your project settings and - enable Git LFS. - -- Git LFS support is not enabled on the GitLab server. Check with your GitLab - administrator why Git LFS is not enabled on the server. See - [LFS administration documentation](lfs_administration.md) for instructions - on how to enable LFS support. - -- Git LFS client version is not supported by GitLab server. Check your Git LFS - version with `git lfs version`. Check the Git config of the project for traces - of deprecated API with `git lfs -l`. If `batch = false` is set in the config, - remove the line and try to update your Git LFS client. Only version 1.0.1 and - newer are supported. - -### getsockopt: connection refused - -If you push a LFS object to a project and you receive an error similar to: -`Post <URL>/info/lfs/objects/batch: dial tcp IP: getsockopt: connection refused`, -the LFS client is trying to reach GitLab through HTTPS. However, your GitLab -instance is being served on HTTP. - -This behaviour is caused by Git LFS using HTTPS connections by default when a -`lfsurl` is not set in the Git config. - -To prevent this from happening, set the lfs url in project Git config: - -```shell -git config --add lfs.url "http://gitlab.example.com/group/project.git/info/lfs" -``` - -### Credentials are always required when pushing an object - -NOTE: **Note:** -With 8.12 GitLab added LFS support to SSH. The Git LFS communication -still goes over HTTP, but now the SSH client passes the correct credentials -to the Git LFS client, so no action is required by the user. - -Given that Git LFS uses HTTP Basic Authentication to authenticate the user pushing -the LFS object on every push for every object, user HTTPS credentials are required. - -By default, Git has support for remembering the credentials for each repository -you use. This is described in [Git credentials man pages](https://git-scm.com/docs/gitcredentials). - -For example, you can tell Git to remember the password for a period of time in -which you expect to push the objects: - -```shell -git config --global credential.helper 'cache --timeout=3600' -``` - -This will remember the credentials for an hour after which Git operations will -require re-authentication. - -If you are using OS X you can use `osxkeychain` to store and encrypt your credentials. -For Windows, you can use `wincred` or Microsoft's [Git Credential Manager for Windows](https://github.com/Microsoft/Git-Credential-Manager-for-Windows/releases). - -More details about various methods of storing the user credentials can be found -on [Git Credential Storage documentation](https://git-scm.com/book/en/v2/Git-Tools-Credential-Storage). - -### LFS objects are missing on push - -GitLab checks files to detect LFS pointers on push. If LFS pointers are detected, GitLab tries to verify that those files already exist in LFS on GitLab. - -Verify that LFS is installed locally and consider a manual push with `git lfs push --all`. - -If you are storing LFS files outside of GitLab you can disable LFS on the project by setting `lfs_enabled: false` with the [projects API](../../api/projects.md#edit-project). - -### Hosting LFS objects externally - -It is possible to host LFS objects externally by setting a custom LFS url with `git config -f .lfsconfig lfs.url https://example.com/<project>.git/info/lfs`. - -You might choose to do this if you are using an appliance like a Sonatype Nexus to store LFS data. If you choose to use an external LFS store, -GitLab will not be able to verify LFS objects which means that pushes will fail if you have GitLab LFS support enabled. - -To stop push failure, LFS support can be disabled in the [Project settings](../../user/project/settings/index.md). This means you will lose GitLab LFS value-adds (Verifying LFS objects, UI integration for LFS). +This document was moved to [another location](../../topics/git/lfs/index.md). diff --git a/doc/administration/lfs/migrate_from_git_annex_to_git_lfs.md b/doc/administration/lfs/migrate_from_git_annex_to_git_lfs.md index bfe18f697bf..dd98a50e353 100644 --- a/doc/administration/lfs/migrate_from_git_annex_to_git_lfs.md +++ b/doc/administration/lfs/migrate_from_git_annex_to_git_lfs.md @@ -1,254 +1,5 @@ -# Migration guide from Git Annex to Git LFS - ->**Note:** -Git Annex support [has been removed][issue-remove-annex] in GitLab Enterprise -Edition 9.0 (2017/03/22). - -Both [Git Annex][] and [Git LFS][] are tools to manage large files in Git. - -## History - -Git Annex [was introduced in GitLab Enterprise Edition 7.8][post-3], at a time -where Git LFS didn't yet exist. A few months later, GitLab brought support for -Git LFS in [GitLab 8.2][post-2] and is available for both Community and -Enterprise editions. - -## Differences between Git Annex and Git LFS - -Some items below are general differences between the two protocols and some are -ones that GitLab developed. - -- Git Annex works only through SSH, whereas Git LFS works both with SSH and HTTPS - (SSH support was added in GitLab 8.12). -- Annex files are stored in a sub-directory of the normal repositories, whereas - LFS files are stored outside of the repositories in a place you can define. -- Git Annex requires a more complex setup, but has much more options than Git - LFS. You can compare the commands each one offers by running `man git-annex` - and `man git-lfs`. -- Annex files cannot be browsed directly in GitLab's interface, whereas LFS - files can. - -## Migration steps - ->**Note:** -Since Git Annex files are stored in a sub-directory of the normal repositories -(`.git/annex/objects`) and LFS files are stored outside of the repositories, -they are not compatible as they are using a different scheme. Therefore, the -migration has to be done manually per repository. - -There are basically two steps you need to take in order to migrate from Git -Annex to Git LFS. - -### TL; DR - -If you know what you are doing and want to skip the reading, this is what you -need to do (we assume you have [git-annex enabled](../git_annex.md#using-gitlab-git-annex) in your -repository and that you have made backups in case something goes wrong). -Fire up a terminal, navigate to your Git repository and: - -1. Disable `git-annex`: - - ```shell - git annex sync --content - git annex direct - git annex uninit - git annex indirect - ``` - -1. Enable `git-lfs`: - - ```shell - git lfs install - git lfs track <files> - git add . - git commit -m "commit message" - git push - ``` - -### Disabling Git Annex in your repo - -Before changing anything, make sure you have a backup of your repository first. -There are a couple of ways to do that, but you can simply clone it to another -local path and maybe push it to GitLab if you want a remote backup as well. -Here you'll find a guide on -[how to back up a **git-annex** repository to an external hard drive][bkp-ext-drive]. - -Since Annex files are stored as objects with symlinks and cannot be directly -modified, we need to first remove those symlinks. - -NOTE: **Note:** -Make sure the you read about the [`direct` mode][annex-direct] as it contains -useful information that may fit in your use case. Note that `annex direct` is -deprecated in Git Annex version 6, so you may need to upgrade your repository -if the server also has Git Annex 6 installed. Read more in the -[Git Annex troubleshooting tips](../git_annex.md#troubleshooting-tips) section. - -1. Backup your repository - - ```shell - cd repository - git annex sync --content - cd .. - git clone repository repository-backup - cd repository-backup - git annex get - cd .. - ``` - -1. Use `annex direct`: - - ```shell - cd repository - git annex direct - ``` - - The output should be similar to this: - - ```shell - commit - On branch master - Your branch is up-to-date with 'origin/master'. - nothing to commit, working tree clean - ok - direct debian.iso ok - direct ok - ``` - -1. Disable Git Annex with [`annex uninit`][uninit]: - - ```shell - git annex uninit - ``` - - The output should be similar to this: - - ```shell - unannex debian.iso ok - Deleted branch git-annex (was 2534d2c). - ``` - - This will `unannex` every file in the repository, leaving the original files. - -1. Switch back to `indirect` mode: - - ```shell - git annex indirect - ``` - - The output should be similar to this: - - ```shell - (merging origin/git-annex into git-annex...) - (recording state in git...) - commit (recording state in git...) - - ok - (recording state in git...) - [master fac3194] commit before switching to indirect mode - 1 file changed, 1 deletion(-) - delete mode 120000 alpine-virt-3.4.4-x86_64.iso - ok - indirect ok - ok - ``` - +--- +redirect_to: '../../topics/git/lfs/migrate_from_git_annex_to_git_lfs.md' --- -At this point, you have two options. Either add, commit and push the files -directly back to GitLab or switch to Git LFS. We will tackle the LFS switch in -the next section. - -### Enabling Git LFS in your repo - -Git LFS is enabled by default on all GitLab products (GitLab CE, GitLab EE, -GitLab.com), therefore, you don't need to do anything server-side. - -1. First, make sure you have `git-lfs` installed locally: - - ```shell - git lfs help - ``` - - If the terminal doesn't prompt you with a full response on `git-lfs` commands, - [install the Git LFS client][install-lfs] first. - -1. Inside the repo, run the following command to initiate LFS: - - ```shell - git lfs install - ``` - -1. Enable `git-lfs` for the group of files you want to track. You - can track specific files, all files containing the same extension, or an - entire directory: - - ```shell - git lfs track images/01.png # per file - git lfs track **/*.png # per extension - git lfs track images/ # per directory - ``` - - Once you do that, run `git status` and you'll see `.gitattributes` added - to your repo. It collects all file patterns that you chose to track via - `git-lfs`. - -1. Add the files, commit and push them to GitLab: - - ```shell - git add . - git commit -m "commit message" - git push - ``` - - If your remote is set up with HTTP, you will be asked to enter your login - credentials. If you have [2FA enabled](../../user/profile/account/two_factor_authentication.md), make sure to use a - [personal access token](../../user/profile/account/two_factor_authentication.md#personal-access-tokens) - instead of your password. - -## Removing the Git Annex branches - -After the migration finishes successfully, you can remove all `git-annex` -related branches from your repository. - -On GitLab, navigate to your project's **Repository ➔ Branches** and delete all -branches created by Git Annex: `git-annex`, and all under `synced/`. - -![repository branches](img/git-annex-branches.png) - -You can also do this on the command line with: - -```shell -git branch -d synced/master -git branch -d synced/git-annex -git push origin :synced/master -git push origin :synced/git-annex -git push origin :git-annex -git remote prune origin -``` - -If there are still some Annex objects inside your repository (`.git/annex/`) -or references inside `.git/config`, run `annex uninit` again: - -```shell -git annex uninit -``` - -## Further Reading - -- (Blog Post) [Getting Started with Git FLS][post-1] -- (Blog Post) [Announcing LFS Support in GitLab][post-2] -- (Blog Post) [GitLab Annex Solves the Problem of Versioning Large Binaries with Git][post-3] -- (GitLab Docs) [Git Annex](../git_annex.md) -- (GitLab Docs) [Git LFS](manage_large_binaries_with_git_lfs.md) - -[annex-direct]: https://git-annex.branchable.com/direct_mode/ -[bkp-ext-drive]: https://www.thomas-krenn.com/en/wiki/Git-annex_Repository_on_an_External_Hard_Drive -[Git Annex]: http://git-annex.branchable.com/ -[Git LFS]: https://git-lfs.github.com/ -[install-lfs]: https://git-lfs.github.com/ -[issue-remove-annex]: https://gitlab.com/gitlab-org/gitlab/issues/1648 -[lfs-track]: https://about.gitlab.com/blog/2017/01/30/getting-started-with-git-lfs-tutorial/#tracking-files-with-lfs -[post-1]: https://about.gitlab.com/blog/2017/01/30/getting-started-with-git-lfs-tutorial/ -[post-2]: https://about.gitlab.com/blog/2015/11/23/announcing-git-lfs-support-in-gitlab/ -[post-3]: https://about.gitlab.com/blog/2015/02/17/gitlab-annex-solves-the-problem-of-versioning-large-binaries-with-git/ -[uninit]: https://git-annex.branchable.com/git-annex-uninit/ +This document was moved to [another location](../../topics/git/lfs/migrate_from_git_annex_to_git_lfs.md). diff --git a/doc/administration/operations/fast_ssh_key_lookup.md b/doc/administration/operations/fast_ssh_key_lookup.md index 8e0e60e64b0..2d1e1c5bda8 100644 --- a/doc/administration/operations/fast_ssh_key_lookup.md +++ b/doc/administration/operations/fast_ssh_key_lookup.md @@ -4,9 +4,9 @@ > - [Available in](https://gitlab.com/gitlab-org/gitlab/issues/3953) GitLab Community Edition 10.4. NOTE: **Note:** This document describes a drop-in replacement for the -`authorized_keys` file for normal (non-deploy key) users. Consider -using [SSH certificates](ssh_certificates.md), they are even faster, -but are not a drop-in replacement. +`authorized_keys` file. For normal (non-deploy key) users, consider using +[SSH certificates](ssh_certificates.md). They are even faster, but are not a +drop-in replacement. Regular SSH operations become slow as the number of users grows because OpenSSH searches for a key to authorize a user via a linear search. In the worst case, diff --git a/doc/administration/repository_storage_types.md b/doc/administration/repository_storage_types.md index f92106a69ca..29b4179a3f7 100644 --- a/doc/administration/repository_storage_types.md +++ b/doc/administration/repository_storage_types.md @@ -235,7 +235,7 @@ CI Artifacts are S3 compatible since **9.4** (GitLab Premium), and available in ##### LFS Objects -[LFS Objects in GitLab](lfs/manage_large_binaries_with_git_lfs.md) implement a similar +[LFS Objects in GitLab](../topics/git/lfs/index.md) implement a similar storage pattern using 2 chars, 2 level folders, following Git's own implementation: ```ruby |