Welcome to mirror list, hosted at ThFree Co, Russian Federation.

github.com/gohugoio/gohugoioTheme2.git - Unnamed repository; edit this file 'description' to name the repository.
summaryrefslogtreecommitdiff
diff options
context:
space:
mode:
Diffstat (limited to 'exampleSite/content/en/documentation/explanation/templates/robots.md')
-rw-r--r--exampleSite/content/en/documentation/explanation/templates/robots.md51
1 files changed, 51 insertions, 0 deletions
diff --git a/exampleSite/content/en/documentation/explanation/templates/robots.md b/exampleSite/content/en/documentation/explanation/templates/robots.md
new file mode 100644
index 0000000..e953dd5
--- /dev/null
+++ b/exampleSite/content/en/documentation/explanation/templates/robots.md
@@ -0,0 +1,51 @@
+---
+title: Robots.txt File
+linktitle: Robots.txt
+description: Hugo can generate a customized robots.txt in the same way as any other template.
+date: 2017-02-01
+publishdate: 2017-02-01
+lastmod: 2017-02-01
+categories: [templates]
+keywords: [robots,search engines]
+
+weight: 165
+sections_weight: 165
+draft: false
+aliases: [/extras/robots-txt/]
+toc: false
+---
+
+To create your robots.txt as a template, first set the `enableRobotsTXT` value to `true` in your [configuration file][config]. By default, this option generates a robots.txt with the following content, which tells search engines that they are allowed to crawl everything:
+
+```
+User-agent: *
+```
+
+## Robots.txt Template Lookup Order
+
+The [lookup order][lookup] for the `robots.txt` template is as follows:
+
+* `/layouts/robots.txt`
+* `/themes/<THEME>/layouts/robots.txt`
+
+{{% note %}}
+If you do not want Hugo to create a default `robots.txt` or leverage the `robots.txt` template, you can hand code your own and place the file in `static`. Remember that everything in the [static directory](/getting-started/directory-structure/) is copied over as-is when Hugo builds your site.
+{{% /note %}}
+
+## Robots.txt Template Example
+
+The following is an example `robots.txt` layout:
+
+{{< code file="layouts/robots.txt" download="robots.txt" >}}
+User-agent: *
+
+{{range .Pages}}
+Disallow: {{.RelPermalink}}
+{{end}}
+{{< /code >}}
+
+This template disallows all the pages of the site by creating one `Disallow` entry for each page.
+
+[config]: /getting-started/configuration/
+[lookup]: /templates/lookup-order/
+[robots]: https://www.robotstxt.org/