1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
|
---
stage: Create
group: Source Code
info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://handbook.gitlab.com/handbook/product/ux/technical-writing/#assignments
---
# Troubleshooting repository mirroring **(FREE ALL)**
When mirroring fails, project maintainers can see a link similar to **{warning-solid}** **Pull mirroring failed 1 hour ago.**
on the project details page. Select this link to go directly to the mirroring settings,
where GitLab displays an **Error** badge for the mirrored repository. You can hover your mouse cursor
over the badge to display the text of the error:
## Received RST_STREAM with error code 2 with GitHub
If you receive this message while mirroring to a GitHub repository:
```plaintext
13:Received RST_STREAM with error code 2
```
One of these issues might be occurring:
1. Your GitHub settings might be set to block pushes that expose your email address
used in commits. To fix this problem, either:
- Set your GitHub email address to public.
- Disable the [Block command line pushes that expose my email](https://github.com/settings/emails)
setting.
1. Your repository exceeds GitHub's file size limit of 100 MB. To fix this problem,
check the file size limit configured for on GitHub, and consider using
[Git Large File Storage](https://git-lfs.github.com) to manage large files.
## Deadline Exceeded
When upgrading GitLab, a change in how usernames are represented means that you
must update your mirroring username and password to ensure that `%40` characters are replaced with `@`.
## Connection blocked because server only allows public key authentication
The connection between GitLab and the remote repository is blocked. Even if a
[TCP Check](../../../../administration/raketasks/maintenance.md#check-tcp-connectivity-to-a-remote-site)
is successful, you must check any networking components in the route from GitLab
to the remote server for blockage.
This error can occur when a firewall performs a `Deep SSH Inspection` on outgoing packets.
## Could not read username: terminal prompts disabled
If you receive this error after creating a new project using
[GitLab CI/CD for external repositories](../../../../ci/ci_cd_for_external_repos/index.md):
- In Bitbucket Cloud:
```plaintext
"2:fetch remote: "fatal: could not read Username for 'https://bitbucket.org':
terminal prompts disabled\n": exit status 128."
```
- In Bitbucket Server (self-managed):
```plaintext
"2:fetch remote: "fatal: could not read Username for 'https://lab.example.com':
terminal prompts disabled\n": exit status 128.
```
Check if the repository owner is specified in the URL of your mirrored repository:
1. On the left sidebar, select **Search or go to** and find your project.
1. Select **Settings > Repository**.
1. Expand **Mirroring repositories**.
1. If no repository owner is specified, delete and add the URL again in this format,
replacing `OWNER`, `ACCOUNTNAME`, `PATH_TO_REPO`, and `REPONAME` with your values:
- In Bitbucket Cloud:
```plaintext
https://OWNER@bitbucket.org/ACCOUNTNAME/REPONAME.git
```
- In Bitbucket Server (self-managed):
```plaintext
https://OWNER@lab.example.com/PATH_TO_REPO/REPONAME.git
```
When connecting to the Cloud or self-managed Bitbucket repository for mirroring, the repository owner is required in the string.
## Pull mirror is missing LFS files
In some cases, pull mirroring does not transfer LFS files. This issue occurs when:
- You use an SSH repository URL. The workaround is to use an HTTPS repository URL instead.
An issue exists [to fix this problem for SSH URLs](https://gitlab.com/gitlab-org/gitlab/-/issues/11997).
- You're using GitLab 14.0 or older, and the source repository is a public Bitbucket URL.
[Fixed](https://gitlab.com/gitlab-org/gitlab/-/issues/335123) in GitLab 14.0.6.
- You mirror an external repository using object storage.
An issue exists [to fix this problem](https://gitlab.com/gitlab-org/gitlab/-/issues/335495).
## Pull mirroring is not triggering pipelines
Pipelines might not run for multiple reasons:
- [Trigger pipelines for mirror updates](pull.md#trigger-pipelines-for-mirror-updates)
might not be enabled. This setting can only be enabled when initially
[configuring pull mirroring](pull.md#configure-pull-mirroring). The status
[is not displayed](https://gitlab.com/gitlab-org/gitlab/-/issues/346630)
when checking the project afterwards.
When mirroring is set up using [CI/CD for external repositories](../../../../ci/ci_cd_for_external_repos/index.md)
this setting is enabled by default. If repository mirroring is manually reconfigured, triggering pipelines
is off by default and this could be why pipelines stop running.
- [`rules`](../../../../ci/yaml/index.md#rules) configuration prevents any jobs from
being added to the pipeline.
- Pipelines are triggered using [the account that set up the pull mirror](https://gitlab.com/gitlab-org/gitlab/-/issues/13697).
If the account is no longer valid, pipelines do not run.
- [Branch protection](../../protected_branches.md#run-pipelines-on-protected-branches)
might prevent the account that set up mirroring from running pipelines.
## `The repository is being updated`, but neither fails nor succeeds visibly
In rare cases, mirroring slots on Redis can become exhausted,
possibly because Sidekiq workers are reaped due to out-of-memory (OoM) events.
When this occurs, mirroring jobs start and complete quickly, but they neither
fail nor succeed. They also do not leave a clear log. To check for this problem:
1. Enter the [Rails console](../../../../administration/operations/rails_console.md)
and check Redis' mirroring capacity:
```ruby
current = Gitlab::Redis::SharedState.with { |redis| redis.scard('MIRROR_PULL_CAPACITY') }.to_i
maximum = Gitlab::CurrentSettings.mirror_max_capacity
available = maximum - current
```
1. If the mirroring capacity is `0` or very low, you can drain all stuck jobs with:
```ruby
Gitlab::Redis::SharedState.with { |redis| redis.smembers('MIRROR_PULL_CAPACITY') }.each do |pid|
Gitlab::Redis::SharedState.with { |redis| redis.srem('MIRROR_PULL_CAPACITY', pid) }
end
```
1. After you run the command, the [background jobs page](../../../../administration/admin_area.md#background-jobs)
should show new mirroring jobs being scheduled, especially when
[triggered manually](index.md#update-a-mirror).
## Invalid URL
If you receive this error while setting up mirroring over [SSH](index.md#ssh-authentication), make sure the URL is in a valid format.
Mirroring does not support the short version of SSH clone URLs (`git@gitlab.com:gitlab-org/gitlab.git`)
and requires the full version including the protocol (`ssh://git@gitlab.com/gitlab-org/gitlab.git`).
Make sure that host and project path are separated using `/` instead of `:`.
## Host key verification failed
This error is returned when the target host public SSH key changes.
Public SSH keys rarely, if ever, change. If host key verification fails,
but you suspect the key is still valid, you can refresh the key's information.
Prerequisites:
- You must have at least the Maintainer role for a project.
To resolve the issue:
1. [Verify the host key](index.md#verify-a-host-key).
1. On the left sidebar, select **Search or go to** and find your project.
1. Select **Settings > Repository**.
1. Expand **Mirroring repositories**.
1. To refresh the keys, either:
- Select **Detect host keys** for GitLab to fetch the host keys from the server, and display the fingerprints.
- Select **Input host keys manually**, and enter the host key into the **SSH host key** field.
- Select **Mirror repository**.
## Transfer mirror users and tokens to a single service account in Rails console
This requires access to the [GitLab Rails console](../../../../administration/operations/rails_console.md#starting-a-rails-console-session).
Use case: If you have multiple users using their own GitHub credentials to set up
repository mirroring, mirroring breaks when people leave the company. Use this
script to migrate disparate mirroring users and tokens into a single service account:
WARNING:
Commands that change data can cause damage if not run correctly or under the right conditions. Always run commands in a test environment first and have a backup instance ready to restore.
```ruby
svc_user = User.find_by(username: 'ourServiceUser')
token = 'githubAccessToken'
Project.where(mirror: true).each do |project|
import_url = project.import_url
# The url we want is https://token@project/path.git
repo_url = if import_url.include?('@')
# Case 1: The url is something like https://23423432@project/path.git
import_url.split('@').last
elsif import_url.include?('//')
# Case 2: The url is something like https://project/path.git
import_url.split('//').last
end
next unless repo_url
final_url = "https://#{token}@#{repo_url}"
project.mirror_user = svc_user
project.import_url = final_url
project.username_only_import_url = final_url
project.save
end
```
|
