gitlab.com/gitlab-org/gitlab-foss.git - Unnamed repository; edit this file 'description' to name the repository.

Age	Commit message (Collapse)	Author
2017-01-18	correct User_agent placement in robots.txt	eric sabelhaus

2016-09-23	update robots.txt disallow	Matt Harrison
	Allows projects in groups starting with "s" while still disallowing the snippets short urls.
2016-05-16	Disallow search engines from indexing uploads from a GitLab project.	Connor Shea
	This can sometimes include sensitive information from private projects and confidential issues. It shouldn't be indexed. Resolves #15551.
2015-10-05	allow crawling of commit page but not patch/diffs	Ben Bodenmiller
	Commit page has valuable information that search engines should be allowed to crawl however the .patch and .diff pages have no new information that is not on commit page
2015-08-18	disallow irrelevant pages by default in robots	Ben Bodenmiller
	Update default robots.txt rules to disallow irrelevant pages that search engines should not care about. This will still allow important pages like the files, commit details, merge requests, issues, comments, etc. to be crawled.
2011-10-09	init commit	gitlabhq