diff options
| author | Marcia Ramos <virtua.creative@gmail.com> | 2018-06-28 20:44:05 +0300 |
|---|---|---|
| committer | Marcia Ramos <virtua.creative@gmail.com> | 2018-06-28 20:44:05 +0300 |
| commit | aa98bb823e3a41b510e8fd757cc0e7fb93841657 (patch) | |
| tree | 6230741e04d54f8700c83336cbf4c18fe81379ac /content/robots.txt.erb | |
| parent | 8979e7f8507929425db80d0deec855d51bc787ce (diff) | |
fix robots.txt to prevent bots from indexing versioned docs
Diffstat (limited to 'content/robots.txt.erb')
| -rw-r--r-- | content/robots.txt.erb | 7 |
1 files changed, 7 insertions, 0 deletions
diff --git a/content/robots.txt.erb b/content/robots.txt.erb index e228e9f8..cb837894 100644 --- a/content/robots.txt.erb +++ b/content/robots.txt.erb @@ -6,6 +6,13 @@ is_hidden: true sitemap: https://docs.gitlab.com/sitemap.xml user-agent: * disallow: /ce/ +disallow: /11.0/ +disallow: /10.8/ +disallow: /10.7/ +disallow: /10.6/ +disallow: /10.5/ +disallow: /10.4/ +disallow: /10.3/ <% else %> # Ban all spiders from the entire site when not on master branch user-agent: * |