diff options
| author | XhmikosR <xhmikosr@gmail.com> | 2021-07-17 18:42:30 +0300 |
|---|---|---|
| committer | XhmikosR <xhmikosr@gmail.com> | 2021-07-19 21:22:23 +0300 |
| commit | 2c112ad2f805fedc655c9e9f5d6d3fd8208a069c (patch) | |
| tree | 6e5bc1d6752e835f51ba2eb4877f8bfaef5c9ef5 | |
| parent | a7e08e839e8f7dc91b29bfe0a2541a505d51e93e (diff) | |
robots.txt: disable crawling if `CF_PAGES` is set
| -rw-r--r-- | src/robots.txt | 2 |
1 files changed, 1 insertions, 1 deletions
diff --git a/src/robots.txt b/src/robots.txt index aef3fa1..76fb36f 100644 --- a/src/robots.txt +++ b/src/robots.txt @@ -5,5 +5,5 @@ # Allow crawling of all content User-agent: * -Disallow: +Disallow:{% if jekyll.environment == "CF_PAGES" %} /{% endif %} Sitemap: {{ site.url }}/sitemap.xml |