1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
|
---
stage: Create
group: Source Code
info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/engineering/ux/technical-writing/#assignments
disqus_identifier: 'https://docs.gitlab.com/ee/workflow/repository_mirroring.html'
---
# Repository mirroring **(FREE)**
Repository mirroring allows for the mirroring of repositories to and from external sources. You
can use it to mirror branches, tags, and commits between repositories. It helps you use
a repository outside of GitLab.
A repository mirror at GitLab updates automatically. You can also manually trigger an update:
- At most once every five minutes on GitLab.com.
- According to a [limit set by the administrator](../../../../administration/instance_limits.md#pull-mirroring-interval)
on self-managed instances.
There are two kinds of repository mirroring supported by GitLab:
- [Push](push.md): for mirroring a GitLab repository to another location.
- [Pull](pull.md): for mirroring a repository from another location to GitLab.
When the mirror repository is updated, all new branches, tags, and commits are visible in the
project's activity feed.
Users with the [Maintainer role](../../../permissions.md) for the project can also force an
immediate update, unless:
- The mirror is already being updated.
- The [limit for pull mirroring interval seconds](../../../../administration/instance_limits.md#pull-mirroring-interval) has not elapsed after its last update.
For security reasons, the URL to the original repository is only displayed to users with the
[Maintainer role](../../../permissions.md) or the [Owner role](../../../permissions.md) for the mirrored
project.
## Use cases
The following are some possible use cases for repository mirroring:
- You migrated to GitLab but still must keep your project in another source. In that case, you
can set it up to mirror to GitLab (pull) and all the essential history of commits, tags,
and branches are available in your GitLab instance. **(PREMIUM)**
- You have old projects in another source that you don't use actively anymore, but don't want to
remove for archiving purposes. In that case, you can create a push mirror so that your active
GitLab repository can push its changes to the old location.
- You are a GitLab self-managed user for privacy reasons and your instance is closed to the public,
but you still have certain software components that you want open sourced. In this case, utilizing
GitLab to be your primary repository which is closed from the public, and using push mirroring to a
GitLab.com repository that's public, allows you to open source specific projects and contribute back
to the open source community.
## Mirror only protected branches **(PREMIUM)**
> Moved to GitLab Premium in 13.9.
Based on the mirror direction that you choose, you can opt to mirror only the
[protected branches](../../protected_branches.md) in the mirroring project,
either from or to your remote repository. For pull mirroring, non-protected branches in
the mirroring project are not mirrored and can diverge.
To use this option, check the **Only mirror protected branches** box when
creating a repository mirror. **(PREMIUM)**
## SSH authentication
> [Introduced](https://gitlab.com/gitlab-org/gitlab-foss/-/merge_requests/22982) in GitLab 11.6 for Push mirroring.
SSH authentication is mutual:
- You have to prove to the server that you're allowed to access the repository.
- The server also has to prove to *you* that it's who it claims to be.
You provide your credentials as a password or public key. The server that the
other repository resides on provides its credentials as a "host key", the
fingerprint of which needs to be verified manually.
If you're mirroring over SSH (using an `ssh://` URL), you can authenticate using:
- Password-based authentication, just as over HTTPS.
- Public key authentication. This is often more secure than password authentication,
especially when the other repository supports [deploy keys](../../deploy_keys/index.md).
To get started:
1. In your project, go to **Settings > Repository**, and then expand the **Mirroring repositories** section.
1. Enter an `ssh://` URL for mirroring.
NOTE:
SCP-style URLs (that is, `git@example.com:group/project.git`) are not supported at this time.
Entering the URL adds two buttons to the page:
- **Detect host keys**.
- **Input host keys manually**.
If you select the:
- **Detect host keys** button, GitLab fetches the host keys from the server and display the fingerprints.
- **Input host keys manually** button, a field is displayed where you can paste in host keys.
Assuming you used the former, you now must verify that the fingerprints are
those you expect. GitLab.com and other code hosting sites publish their
fingerprints in the open for you to check:
- [AWS CodeCommit](https://docs.aws.amazon.com/codecommit/latest/userguide/regions.html#regions-fingerprints)
- [Bitbucket](https://support.atlassian.com/bitbucket-cloud/docs/configure-ssh-and-two-step-verification/)
- [GitHub](https://docs.github.com/en/authentication/keeping-your-account-and-data-secure/githubs-ssh-key-fingerprints)
- [GitLab.com](../../../gitlab_com/index.md#ssh-host-keys-fingerprints)
- [Launchpad](https://help.launchpad.net/SSHFingerprints)
- [Savannah](http://savannah.gnu.org/maintenance/SshAccess/)
- [SourceForge](https://sourceforge.net/p/forge/documentation/SSH%20Key%20Fingerprints/)
Other providers vary. If you're running self-managed GitLab, or otherwise
have access to the server for the other repository, you can securely gather the
key fingerprints:
```shell
$ cat /etc/ssh/ssh_host*pub | ssh-keygen -E md5 -l -f -
256 MD5:f4:28:9f:23:99:15:21:1b:bf:ed:1f:8e:a0:76:b2:9d root@example.com (ECDSA)
256 MD5:e6:eb:45:8a:3c:59:35:5f:e9:5b:80:12:be:7e:22:73 root@example.com (ED25519)
2048 MD5:3f:72:be:3d:62:03:5c:62:83:e8:6e:14:34:3a:85:1d root@example.com (RSA)
```
NOTE:
You must exclude `-E md5` for some older versions of SSH.
When mirroring the repository, GitLab checks that at least one of the
stored host keys matches before connecting. This can prevent malicious code from
being injected into your mirror, or your password being stolen.
### SSH public key authentication
To use SSH public key authentication, you must also choose that option
from the **Authentication method** dropdown. When the mirror is created,
GitLab generates a 4096-bit RSA key that can be copied by selecting the **Copy SSH public key** button.
You then must add the public SSH key to the other repository's configuration:
- If the other repository is hosted on GitLab, you should add the public SSH key
as a [deploy key](../../../project/deploy_keys/index.md).
- If the other repository is hosted elsewhere, you must add the key to
your user's `authorized_keys` file. Paste the entire public SSH key into the
file on its own line and save it.
If you must change the key at any time, you can remove and re-add the mirror
to generate a new key. Update the other repository with the new
key to keep the mirror running.
NOTE:
The generated keys are stored in the GitLab database, not in the file system. Therefore,
SSH public key authentication for mirrors cannot be used in a pre-receive hook.
## Force an update **(FREE)**
While mirrors are scheduled to update automatically, you can always force an update by using the
update button which is available on the **Mirroring repositories** section of the **Repository Settings** page.
## Resources
- Configure a [Pull Mirroring Interval](../../../../administration/instance_limits.md#pull-mirroring-interval)
- [Disable mirrors for a project](../../../admin_area/settings/visibility_and_access_controls.md#enable-project-mirroring)
- [Secrets file and mirroring](../../../../raketasks/backup_restore.md#when-the-secrets-file-is-lost)
## Troubleshooting
Should an error occur during a push, GitLab displays an **Error** highlight for that repository. Details on the error can then be seen by hovering over the highlight text.
### 13:Received RST_STREAM with error code 2 with GitHub
If you receive a "13:Received RST_STREAM with error code 2" message while mirroring to a GitHub repository,
your GitHub settings might be set to block pushes that expose your email address used in commits. Either
set your email address on GitHub to be public, or disable the [Block command line pushes that expose my email](https://github.com/settings/emails) setting.
### 4:Deadline Exceeded
When upgrading to GitLab 11.11.8 or newer, a change in how usernames are represented means that you
must update your mirroring username and password to ensure that `%40` characters are replaced with `@`.
### Connection blocked because server only allows public key authentication
As the error indicates, the connection is getting blocked between GitLab and the remote repository. Even if a
[TCP Check](../../../../administration/raketasks/maintenance.md#check-tcp-connectivity-to-a-remote-site) is successful,
you must check any networking components in the route from GitLab to the remote Server to ensure there's no blockage.
For example, we've seen this error when a Firewall was performing a `Deep SSH Inspection` on outgoing packets.
### Could not read username: terminal prompts disabled
If you receive this error after creating a new project using
[GitLab CI/CD for external repositories](../../../../ci/ci_cd_for_external_repos/):
```plaintext
"2:fetch remote: "fatal: could not read Username for 'https://bitbucket.org': terminal prompts disabled\n": exit status 128."
```
Check if the repository owner is specified in the URL of your mirrored repository:
1. Go to your project.
1. On the left sidebar, select **Settings > Repository**.
1. Select **Mirroring repositories**.
1. If no repository owner is specified, delete and add the URL again in this format:
```plaintext
https://**<repo_owner>**@bitbucket.org/<accountname>/<reponame>.git
```
The repository owner is needed for Bitbucket to connect to the repository for mirroring.
### Pull mirror is missing LFS files
In some cases, pull mirroring does not transfer LFS files. This issue occurs when:
- You use an SSH repository URL. The workaround is to use an HTTPS repository URL instead.
There is [an issue to fix this for SSH URLs](https://gitlab.com/gitlab-org/gitlab/-/issues/11997).
- You're using GitLab 14.0 or older, and the source repository is a public Bitbucket URL.
This was [fixed in GitLab 14.0.6](https://gitlab.com/gitlab-org/gitlab/-/issues/335123).
- You mirror an external repository using object storage.
There is [an issue to fix this](https://gitlab.com/gitlab-org/gitlab/-/issues/335495).
|
