Add latest changes from gitlab-org/gitlab@13-11-stable-eev13.11.0-rc43

1 files changed, 45 insertions, 16 deletions

diff --git a/doc/administration/nfs.md b/doc/administration/nfs.md
index 59b18f7a74a..52948732766 100644
--- a/doc/administration/nfs.md
+++ b/doc/administration/nfs.md
@@ -14,17 +14,19 @@ Pages](https://gitlab.com/gitlab-org/gitlab-pages/-/issues/196).
 For data objects such as LFS, Uploads, Artifacts, etc., an [Object Storage service](object_storage.md)
 is recommended over NFS where possible, due to better performance.
 
-WARNING:
-From GitLab 13.0, using NFS for Git repositories is deprecated.
-From GitLab 14.0, technical support for NFS for Git repositories
-will no longer be provided. Upgrade to [Gitaly Cluster](gitaly/praefect.md)
-as soon as possible.
-
 File system performance can impact overall GitLab performance, especially for
 actions that read or write to Git repositories. For steps you can use to test
 file system performance, see
 [File system Performance Benchmarking](operations/filesystem_benchmarking.md).
 
+## Gitaly and NFS deprecation
+
+WARNING:
+From GitLab 14.0, enhancements and bug fixes for NFS for Git repositories will no longer be
+considered and customer technical support will be considered out of scope.
+[Read more about Gitaly and NFS](gitaly/index.md#nfs-deprecation-notice) and
+[the correct mount options to use](#upgrade-to-gitaly-cluster-or-disable-caching-if-experiencing-data-loss).
+
 ## Known kernel version incompatibilities
 
 RedHat Enterprise Linux (RHEL) and CentOS v7.7 and v7.8 ship with kernel
@@ -169,11 +171,11 @@ apt-get install nfs-common
 Here is an example snippet to add to `/etc/fstab`:
 
 ```plaintext
-10.1.0.1:/var/opt/gitlab/.ssh /var/opt/gitlab/.ssh nfs4 defaults,vers=4.1,hard,rsize=1048576,wsize=1048576,noatime,nofail,lookupcache=positive 0 2
-10.1.0.1:/var/opt/gitlab/gitlab-rails/uploads /var/opt/gitlab/gitlab-rails/uploads nfs4 defaults,vers=4.1,hard,rsize=1048576,wsize=1048576,noatime,nofail,lookupcache=positive 0 2
-10.1.0.1:/var/opt/gitlab/gitlab-rails/shared /var/opt/gitlab/gitlab-rails/shared nfs4 defaults,vers=4.1,hard,rsize=1048576,wsize=1048576,noatime,nofail,lookupcache=positive 0 2
-10.1.0.1:/var/opt/gitlab/gitlab-ci/builds /var/opt/gitlab/gitlab-ci/builds nfs4 defaults,vers=4.1,hard,rsize=1048576,wsize=1048576,noatime,nofail,lookupcache=positive 0 2
-10.1.0.1:/var/opt/gitlab/git-data /var/opt/gitlab/git-data nfs4 defaults,vers=4.1,hard,rsize=1048576,wsize=1048576,noatime,nofail,lookupcache=positive 0 2
+10.1.0.1:/var/opt/gitlab/.ssh /var/opt/gitlab/.ssh nfs4 defaults,vers=4.1,hard,rsize=1048576,wsize=1048576,noatime,nofail,_netdev,lookupcache=positive 0 2
+10.1.0.1:/var/opt/gitlab/gitlab-rails/uploads /var/opt/gitlab/gitlab-rails/uploads nfs4 defaults,vers=4.1,hard,rsize=1048576,wsize=1048576,noatime,nofail,_netdev,lookupcache=positive 0 2
+10.1.0.1:/var/opt/gitlab/gitlab-rails/shared /var/opt/gitlab/gitlab-rails/shared nfs4 defaults,vers=4.1,hard,rsize=1048576,wsize=1048576,noatime,nofail,_netdev,lookupcache=positive 0 2
+10.1.0.1:/var/opt/gitlab/gitlab-ci/builds /var/opt/gitlab/gitlab-ci/builds nfs4 defaults,vers=4.1,hard,rsize=1048576,wsize=1048576,noatime,nofail,_netdev,lookupcache=positive 0 2
+10.1.0.1:/var/opt/gitlab/git-data /var/opt/gitlab/git-data nfs4 defaults,vers=4.1,hard,rsize=1048576,wsize=1048576,noatime,nofail,_netdev,lookupcache=positive 0 2
 ```
 
 You can view information and options set for each of the mounted NFS file
@@ -187,6 +189,8 @@ Note there are several options that you should consider using:
 | `nofail` | Don't halt boot process waiting for this mount to become available
 | `lookupcache=positive` | Tells the NFS client to honor `positive` cache results but invalidates any `negative` cache results. Negative cache results cause problems with Git. Specifically, a `git push` can fail to register uniformly across all NFS clients. The negative cache causes the clients to 'remember' that the files did not exist previously.
 | `hard` | Instead of `soft`. [Further details](#soft-mount-option).
+| `cto` | `cto` is the default option, which you should use. Do not use `nocto`. [Further details](#nocto-mount-option).
+| `_netdev` | Wait to mount filesystem until network is online. See also the [`high_availability['mountpoint']`](https://docs.gitlab.com/omnibus/settings/configuration.html#only-start-omnibus-gitlab-services-after-a-given-file-system-is-mounted) option.
 
 #### `soft` mount option
 
@@ -223,6 +227,25 @@ the mount point. Use `SIGKILL` (`kill -9`) to deal with hung processes.
 The `intr` option
 [stopped working in the 2.6 kernel](https://access.redhat.com/solutions/157873).
 
+#### `nocto` mount option
+
+Do not use `nocto`. Instead, use `cto`, which is the default.
+
+When using `nocto`, the dentry cache is always used, up to `acdirmax` seconds (attribute cache time) from the time it's created.
+
+This results in stale dentry cache issues with multiple clients, where each client can see a different (cached)
+version of a directory.
+
+From the [Linux man page](https://linux.die.net/man/5/nfs), the important parts:
+
+> If the nocto option is specified, the client uses a non-standard heuristic to determine when files on the server have changed.
+>
+> Using the nocto option may improve performance for read-only mounts, but should be used only if the data on the server changes only occasionally.
+
+We have noticed this behavior in an issue about [refs not found after a push](https://gitlab.com/gitlab-org/gitlab/-/issues/326066),
+where newly added loose refs can be seen as missing on a different client with a local dentry cache, as
+[described in this issue](https://gitlab.com/gitlab-org/gitlab/-/issues/326066#note_539436931).
+
 ### A single NFS mount
 
 It's recommended to nest all GitLab data directories within a mount, that allows automatic
@@ -346,12 +369,18 @@ sudo ufw allow from <client_ip_address> to any port nfs
 ### Upgrade to Gitaly Cluster or disable caching if experiencing data loss
 
 WARNING:
-From GitLab 13.0, using NFS for Git repositories is deprecated. In GitLab 14.0,
-support for NFS for Git repositories is scheduled to be removed. Upgrade to
-[Gitaly Cluster](gitaly/praefect.md) as soon as possible.
+From GitLab 13.0, using NFS for Git repositories is deprecated.
+As of GitLab 14.0, NFS-related issues with Gitaly will no longer be addressed. Read
+more about [Gitaly and NFS deprecation](gitaly/index.md#nfs-deprecation-notice).
 
 Customers and users have reported data loss on high-traffic repositories when using NFS for Git repositories.
-For example, we have seen [inconsistent updates after a push](https://gitlab.com/gitlab-org/gitaly/-/issues/2589). The problem may be partially mitigated by adjusting caching using the following NFS client mount options:
+For example, we have seen:
+
+- [Inconsistent updates after a push](https://gitlab.com/gitlab-org/gitaly/-/issues/2589).
+- `git ls-remote` [returning the wrong (or no branches)](https://gitlab.com/gitlab-org/gitaly/-/issues/3083)
+causing Jenkins to intermittently re-run all pipelines for a repository.
+
+The problem may be partially mitigated by adjusting caching using the following NFS client mount options:
 
 | Setting | Description |
 | ------- | ----------- |
@@ -362,7 +391,7 @@ For example, we have seen [inconsistent updates after a push](https://gitlab.com
 WARNING:
 The `actimeo=0` and `noac` options both result in a significant reduction in performance, possibly leading to timeouts.
 You may be able to avoid timeouts and data loss using `actimeo=0` and `lookupcache=positive` _without_ `noac`, however
-we expect the performance reduction will still be significant. As noted above, we strongly recommend upgrading to
+we expect the performance reduction will still be significant. Upgrade to
 [Gitaly Cluster](gitaly/praefect.md) as soon as possible.
 
 ### Avoid using cloud-based file systems