1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
263
264
265
266
267
268
269
270
271
272
273
274
275
276
277
278
279
280
281
282
283
284
285
286
287
288
289
290
291
292
293
294
295
296
297
298
299
300
301
302
303
304
305
306
307
308
309
310
311
312
313
314
315
316
317
318
319
320
321
322
323
324
325
326
327
328
329
330
331
332
333
334
335
336
337
338
339
340
341
342
343
344
345
346
347
348
349
350
351
352
353
354
355
356
357
358
359
360
361
362
363
364
365
366
367
368
369
370
371
372
373
374
375
376
377
378
379
380
381
382
383
384
385
386
387
388
389
390
391
392
393
394
395
396
397
398
399
400
401
402
403
404
405
406
407
408
409
410
411
412
|
---
stage: Systems
group: Distribution
info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments
---
# GitLab Mattermost
You can run a [GitLab Mattermost](https://gitlab.com/gitlab-org/gitlab-mattermost)
service on your GitLab server. Mattermost is not part of the single application that GitLab is. There is a good integration between [Mattermost and GitLab](https://mattermost.com/solutions/mattermost-gitlab/), and our Linux package allows you to install it. **However, Mattermost is a separate application from a separate company.** GitLab Support cannot help you with Mattermost-specific questions beyond the integration with GitLab. If you need help with Mattermost itself, see the [community support resources](#community-support-resources).
## Prerequisites
Each release of GitLab Mattermost is compiled and manually tested on an AMD 64 chipset for Linux. ARM chipsets and operating systems, like Raspberry Pi, are not supported.
## Getting started
GitLab Mattermost expects to run on its own virtual host. In your DNS settings, you need
two entries pointing to the same machine. For example, `gitlab.example.com` and
`mattermost.example.com`.
GitLab Mattermost is disabled by default. To enable it:
1. Edit `/etc/gitlab/gitlab.rb` and add the Mattermost external URL:
```ruby
mattermost_external_url 'https://mattermost.example.com'
```
1. Reconfigure GitLab:
```shell
sudo gitlab-ctl reconfigure
```
1. Confirm that GitLab Mattermost is reachable at `https://mattermost.example.com` and authorized to connect to GitLab. Authorizing Mattermost with GitLab allows users to use GitLab as an SSO provider.
The Linux package attempts to automatically authorize GitLab Mattermost with GitLab if the applications are running on the same server.
Automatic authorization requires access to the GitLab database. If the GitLab database is not available
you need to manually authorize GitLab Mattermost for access to GitLab using the process described in the [Authorize GitLab Mattermost section](#authorize-gitlab-mattermost).
## Configuring Mattermost
Starting in GitLab 11.0, Mattermost can be configured using the Mattermost System Console. An extensive list of
Mattermost settings and where they can be set is available [in the Mattermost documentation](https://docs.mattermost.com/administration/config-settings.html).
While using the System Console is recommended, you can also configure Mattermost using one of the following options:
1. Edit the Mattermost configuration directly through `/var/opt/gitlab/mattermost/config.json`.
1. Specify environment variables used to run Mattermost by changing the `mattermost['env']` setting in `gitlab.rb`. Any settings configured in this way are disabled from the System Console and cannot be changed without restarting Mattermost.
## Running GitLab Mattermost with HTTPS
Place the SSL certificate and SSL certificate key inside `/etc/gitlab/ssl`. If the directory doesn't exist, create it:
```shell
sudo mkdir -p /etc/gitlab/ssl
sudo chmod 755 /etc/gitlab/ssl
sudo cp mattermost.gitlab.example.key mattermost.gitlab.example.crt /etc/gitlab/ssl/
```
In `/etc/gitlab/gitlab.rb` specify the following configuration:
```ruby
mattermost_external_url 'https://mattermost.gitlab.example'
mattermost_nginx['redirect_http_to_https'] = true
```
If you haven't named your certificate and key `mattermost.gitlab.example.crt`
and `mattermost.gitlab.example.key` then you need to also add the full paths
as shown below.
```ruby
mattermost_nginx['ssl_certificate'] = "/etc/gitlab/ssl/mattermost-nginx.crt"
mattermost_nginx['ssl_certificate_key'] = "/etc/gitlab/ssl/mattermost-nginx.key"
```
where `mattermost-nginx.crt` is the SSL certificate and `mattermost-nginx.key` is the SSL key.
Once the configuration is set, run `sudo gitlab-ctl reconfigure` to apply the changes.
## Running GitLab Mattermost on its own server
If you want to run GitLab and GitLab Mattermost on two separate servers the GitLab services are still set up on your GitLab Mattermost server, but they do not accept user requests or
consume system resources. You can use the following settings and configuration details on the GitLab Mattermost server to effectively disable the GitLab service bundled into the Linux package.
```ruby
mattermost_external_url 'http://mattermost.example.com'
# Shut down GitLab services on the Mattermost server
gitlab_rails['enable'] = false
redis['enable'] = false
postgres_exporter['enable'] = false
grafana['enable'] = false
```
Then follow the appropriate steps in the [Authorize GitLab Mattermost section](#authorize-gitlab-mattermost). Last, to enable
integrations with GitLab add the following on the GitLab Server:
```ruby
gitlab_rails['mattermost_host'] = "https://mattermost.example.com"
```
By default GitLab Mattermost requires all users to sign up with GitLab and disables the sign-up by email option. See Mattermost [documentation on GitLab SSO](https://docs.mattermost.com/deployment/sso-gitlab.html).
## Manually (re)authorizing GitLab Mattermost with GitLab
### Reauthorize GitLab Mattermost
To reauthorize GitLab Mattermost, you first need to revoke the existing
authorization. This can be done in the **Settings > Applications** area of GitLab. Then follow the steps below to complete authorization.
### Authorize GitLab Mattermost
Navigate to the **Settings > Applications** area in GitLab. Create a new application and for the **Redirect URI** use the following (replace `http` with `https` if you use HTTPS):
```plaintext
http://mattermost.example.com/signup/gitlab/complete
http://mattermost.example.com/login/gitlab/complete
```
Make sure to select the **Trusted** and **Confidential** settings. Under **Scopes**, select `read_user`. Then, choose **Save application**.
Once the application is created you are provided with an `Application ID` and `Secret`. One other piece of information needed is the URL of GitLab instance.
Return to the server running GitLab Mattermost and edit the `/etc/gitlab/gitlab.rb` configuration file as follows using the values you received above:
```ruby
mattermost['gitlab_enable'] = true
mattermost['gitlab_id'] = "12345656"
mattermost['gitlab_secret'] = "123456789"
mattermost['gitlab_scope'] = "read_user"
mattermost['gitlab_auth_endpoint'] = "http://gitlab.example.com/oauth/authorize"
mattermost['gitlab_token_endpoint'] = "http://gitlab.example.com/oauth/token"
mattermost['gitlab_user_api_endpoint'] = "http://gitlab.example.com/api/v4/user"
```
Save the changes and then run `sudo gitlab-ctl reconfigure`. If there are no errors your GitLab and GitLab Mattermost should be configured correctly.
## Specify numeric user and group identifiers
The Linux pacakage creates a user and group `mattermost`. You can specify the
numeric identifiers for these users in `/etc/gitlab/gitlab.rb` as follows:
```ruby
mattermost['uid'] = 1234
mattermost['gid'] = 1234
```
Run `sudo gitlab-ctl reconfigure` to apply the changes.
## Setting custom environment variables
If necessary you can set custom environment variables to be used by Mattermost
via `/etc/gitlab/gitlab.rb`. This can be useful if the Mattermost server
is operated behind a corporate internet proxy. In `/etc/gitlab/gitlab.rb`
supply a `mattermost['env']` with a hash value. For example:
```ruby
mattermost['env'] = {"HTTP_PROXY" => "my_proxy", "HTTPS_PROXY" => "my_proxy", "NO_PROXY" => "my_no_proxy"}
```
Run `sudo gitlab-ctl reconfigure` to apply the changes.
## Connecting to the bundled PostgreSQL database
If you need to connect to the bundled PostgreSQL database and are using the default Linux package database configuration, you can connect as
the PostgreSQL superuser:
```shell
sudo gitlab-psql -d mattermost_production
```
## Back up GitLab Mattermost
GitLab Mattermost is not included in the regular [Linux package backup](../../administration/backup_restore/index.md) Rake task.
The general Mattermost [backup and disaster recovery](https://docs.mattermost.com/deploy/backup-disaster-recovery.html) documentation can be used as a guide
on what needs to be backed up.
### Back up the bundled PostgreSQL database
If you need to back up the bundled PostgreSQL database and are using the default Linux package database configuration, you can back up using this command:
```shell
sudo -i -u gitlab-psql -- /opt/gitlab/embedded/bin/pg_dump -h /var/opt/gitlab/postgresql mattermost_production | gzip > mattermost_dbdump_$(date --rfc-3339=date).sql.gz
```
### Back up the `data` directory and `config.json`
Mattermost has a `data` directory and `config.json` file that need to be backed up as well:
```shell
sudo tar -zcvf mattermost_data_$(date --rfc-3339=date).gz -C /var/opt/gitlab/mattermost data config.json
```
## Restore GitLab Mattermost
If you have previously [created a backup of GitLab Mattermost](#back-up-gitlab-mattermost), you can run the following commands to restore it:
```shell
# Stop Mattermost so we don't have any open database connections
sudo gitlab-ctl stop mattermost
# Drop the Mattermost database
sudo -u gitlab-psql /opt/gitlab/embedded/bin/dropdb -U gitlab-psql -h /var/opt/gitlab/postgresql -p 5432 mattermost_production
# Create the Mattermost database
sudo -u gitlab-psql /opt/gitlab/embedded/bin/createdb -U gitlab-psql -h /var/opt/gitlab/postgresql -p 5432 mattermost_production
# Perform the database restore
# Replace /tmp/mattermost_dbdump_2021-08-05.sql.gz with your backup
sudo -u mattermost sh -c "zcat /tmp/mattermost_dbdump_2021-08-05.sql.gz | /opt/gitlab/embedded/bin/psql -U gitlab_mattermost -h /var/opt/gitlab/postgresql -p 5432 mattermost_production"
# Restore the data directory and config.json
# Replace /tmp/mattermost_data_2021-08-09.gz with your backup
sudo tar -xzvf /tmp/mattermost_data_2021-08-09.gz -C /var/opt/gitlab/mattermost
# Fix permissions if required
sudo chown -R mattermost:mattermost /var/opt/gitlab/mattermost/data
sudo chown mattermost:mattermost /var/opt/gitlab/mattermost/config.json
# Start Mattermost
sudo gitlab-ctl start mattermost
```
## Mattermost Command Line Tools (CLI)
[`mmctl`](https://docs.mattermost.com/manage/mmctl-command-line-tool.html) is a CLI tool for the Mattermost server which is installed locally and uses the Mattermost API, but may also be used remotely. You must configure Mattermost either for local connections or authenticate as an administrator with local login credentials (not through GitLab SSO). The executable is located at `/opt/gitlab/embedded/bin/mmctl`.
### Use `mmctl` through a local connection
For local connections, the `mmctl` binary and Mattermost must be run from the same server. To enable the local socket:
1. Edit `/var/opt/gitlab/mattermost/config.json`, and add the following lines:
```json
{
"ServiceSettings": {
...
"EnableLocalMode": true,
"LocalModeSocketLocation": "/var/tmp/mattermost_local.socket",
...
}
}
```
1. Restart Mattermost:
```shell
sudo gitlab-ctl restart mattermost
```
You can then use `sudo /opt/gitlab/embedded/bin/mmctl --local` to run `mmctl` commands
on your Mattermost instance.
For example, to show the list of users:
```shell
$ sudo /opt/gitlab/embedded/bin/mmctl --local user list
13dzo5bmg7fu8rdox347hbfxde: appsbot (appsbot@localhost)
tbnkwjdug3dejcoddboo4yuomr: boards (boards@localhost)
wd3g5zpepjgbfjgpdjaas7yj6a: feedbackbot (feedbackbot@localhost)
8d3zzgpurp85zgf1q88pef73eo: playbooks (playbooks@localhost)
There are 4 users on local instance
```
### Use `mmctl` through a remote connection
For remote connections or local connections where the socket cannot be used,
create a non-SSO user and give that user administrator privileges. Those credentials
can then be used to authenticate `mmctl`:
```shell
$ /opt/gitlab/embedded/bin/mmctl auth login http://mattermost.example.com
Connection name: test
Username: local-user
Password:
credentials for "test": "local-user@http://mattermost.example.com" stored
```
## Configuring GitLab and Mattermost integrations
You can use the plugin to subscribe Mattermost to receive notifications about issues, merge requests, and pull requests as well as personal notifications regarding merge request reviews, unread messages, and task assignments. If you want to use slash commands to perform actions
such as creating and viewing issues, or to trigger deployments use GitLab [Mattermost slash commands](../../user/project/integrations/mattermost_slash_commands.md).
The plugin and slash commands can be used together or individually.
## Email Notifications
### Setting up SMTP for GitLab Mattermost
These settings are configured through the Mattermost System Console by the System Administrator.
On the **Environment > SMTP** tab of the **System Console**, you can enter the SMTP credentials given by your SMTP provider, or `127.0.0.1` and port `25` to use `sendmail`. More information on the specific settings
that are needed is available in the [Mattermost documentation](https://docs.mattermost.com/install/smtp-email-setup.html).
These settings can also be configured in `/var/opt/gitlab/mattermost/config.json`.
### Email batching
Enabling this feature allows users to control how often they receive email notifications.
Email batching can be enabled in the Mattermost **System Console** by navigating to the **Environment > SMTP** tab, and setting the **Enable Email Batching** setting to **True**.
This setting can also be configured in `/var/opt/gitlab/mattermost/config.json`.
## Upgrading GitLab Mattermost
Below is a list of Mattermost versions for GitLab 14.0 and later:
| GitLab version | Mattermost version |
|:---------------|:-------------------|
| 16.2 | 7.10 |
| 16.1 | 7.10 |
| 16.0 | 7.10 |
| 15.11 | 7.9 |
| 15.10 | 7.8 |
| 15.9 | 7.7 |
| 15.8 | 7.5 |
| 15.7 | 7.5 |
| 15.6 | 7.4 |
| 15.5 | 7.3 |
| 15.4 | 7.2 |
| 15.3 | 7.1 |
| 15.2 | 7.0 |
| 15.1 | 6.7 |
| 15.0 | 6.6 |
| 14.10 | 6.5 |
| 14.9 | 6.4 |
| 14.8 | 6.3 |
| 14.7 | 6.2 |
| 14.6 | 6.1 |
| 14.5 | 5.39 |
| 14.4 | 5.39 |
| 14.3 | 5.38 |
| 14.2 | 5.37 |
| 14.1 | 5.36 |
| 14.0 | 5.35 |
- GitLab 14.5 remained on Mattermost 5.39.
- GitLab 14.6 updates to Mattermost 6.1 instead of 6.0.
- GitLab 15.8 remained on Mattermost 7.5.
- GitLab 16.1 and 16.2 remained on Mattermost 7.10.
NOTE:
When upgrading the Mattermost version, it is essential to check the
[Important Upgrade Notes](https://docs.mattermost.com/administration/important-upgrade-notes.html)
for Mattermost to address any changes or migrations that need to be performed.
GitLab Mattermost can be upgraded through the regular Linux package update process. When upgrading previous versions of
GitLab, the update process can only be used if Mattermost configuration settings have not been changed outside of GitLab. That is, no changes to the Mattermost `config.json`
file have been made - either directly or via the Mattermost **System Console**, which saves changes to `config.json`.
If you have only configured Mattermost using `gitlab.rb`, you can upgrade GitLab using the Linux package and then run `gitlab-ctl reconfigure` to upgrade GitLab Mattermost to the latest version.
If this is not the case, there are two options:
1. Update [`gitlab.rb`](https://gitlab.com/gitlab-org/omnibus-gitlab/blob/master/files/gitlab-config-template/gitlab.rb.template#L706)
with the changes done to `config.json`. This might require adding some parameters as not all
settings in `config.json` are available in `gitlab.rb`. Once complete, the Linux package should be
able to upgrade GitLab Mattermost from one version to the next.
1. Migrate Mattermost outside of the directory controlled by the Linux package so it can be administered
and upgraded independently. Follow the [Mattermost Migration Guide](https://docs.mattermost.com/administration/migrating.html)
to move your Mattermost configuration settings and data to another directory or server independent
from the Linux package.
For a complete list of upgrade notices and special considerations for older versions, see the [Mattermost documentation](https://docs.mattermost.com/administration/important-upgrade-notes.html).
## Upgrading GitLab Mattermost to 14.6
GitLab 14.6 ships with Mattermost 6.1 including potentially long running database migrations for Mattermost 6.0. For information about upgrading and for ways to reduce the downtime caused by those migrations, read the [Important Upgrade Notes](https://docs.mattermost.com/administration/important-upgrade-notes.html) for both versions. If you need to perform any manual migrations, [connect to the bundled PostgreSQL database](#connecting-to-the-bundled-postgresql-database).
NOTE:
The Mattermost upgrade notes refer to different impacts when used with a PostgreSQL versus a MySQL database. The GitLab Mattermost included with the Linux package uses a PostgreSQL database.
## OAuth 2.0 sequence diagram
The following image is a sequence diagram for how GitLab works as an OAuth 2.0
provider for Mattermost. You can use this to troubleshoot errors
in getting the integration to work:
```mermaid
sequenceDiagram
User->>Mattermost: GET https://mm.domain.com
Note over Mattermost, GitLab: Obtain access code
Mattermost->>GitLab: GET https://gitlab.domain.com/oauth/authorize
Note over User, GitLab: GitLab user signs in (if necessary)
Note over GitLab: GitLab verifies client_id matches an OAuth application
GitLab->>User: GitLab asks user to authorize Mattermost OAuth app
User->>GitLab: User selects 'Allow'
Note over GitLab: GitLab verifies redirect_uri matches list of valid URLs
GitLab->>User: 302 redirect: https://mm.domain.com/signup/gitlab/complete
User->>Mattermost: GET https://mm.domain.com/signup/gitlab/complete
Note over Mattermost, GitLab: Exchange access code for access token
Mattermost->>GitLab: POST http://gitlab.domain.com/oauth/token
GitLab->>GitLab: Doorkeeper::TokensController#35;create
GitLab->>Mattermost: Access token
Note over Mattermost, GitLab: Mattermost looks up GitLab user
Mattermost->>GitLab: GET https://gitlab.domain.com/api/v4/user
GitLab->>Mattermost: User details
Mattermost->>User: Mattermost/GitLab user ready
```
## Community support resources
For help and support around your GitLab Mattermost deployment, see:
- [Troubleshooting Mattermost issues](https://docs.mattermost.com/install/troubleshooting.html).
- [Mattermost GitLab Issues Support Handbook](https://docs.mattermost.com/process/support.html?highlight=omnibus#gitlab-issues).
- [GitLab Mattermost issue tracker](https://gitlab.com/gitlab-org/gitlab-mattermost/-/issues) for verified bugs with repro steps.
|