NOTE: If you are a developer, please use a private wiki based on foswiki/trunk on a daily base ...or use trunk.foswiki.org to view this page for some minimal testing.
Use Item11383 for general documentation changes for release 1.1.5. Use Item9693 for docu changes for release 2.0.

You are here: Foswiki>Tasks Web>Item2427 (29 Nov 2009, OliverKrueger)

robots.txt does not exist, though it should

Priority:	CurrentState:	AppliesTo:	Component:	WaitingFor:
Normal	Closed	Web Site

On second thought, apart from the robots.txt file, maybe we could also add a sitemap.xml file (should be placed in the root directory as well and can also be referenced by the former).

The Disallow directives should also help to weed out 'redundant' (i.e., merely reformatted) content
Using a sitemap, we could give WebHome topics a higher priority (>0.5)

Note that we need one robots.txt[ / sitemap.xml pair] per subdomain!

-- MarkusUeberall - 28 Nov 2009

Added an Alias /robots.txt "/path/to/robots.txt" to the apache config file for foswiki.org as suggested in ApacheConfigGenerator and restarted the daemon.

-- OliverKrueger - 29 Nov 2009

ItemTemplate edit

Summary	http://foswiki.org/robots.txt does not exist, though it should
ReportedBy	MarkusUeberall
Codebase
SVN Range	Foswiki-1.0.7, Sun, 20 Sep 2009, build 5061
AppliesTo	Web Site
Component
Priority	Normal
CurrentState	Closed
WaitingFor
Checkins
TargetRelease	n/a
ReleasedIn

Topic revision: r2 - 29 Nov 2009 - 01:40:20 - OliverKrueger

Tasks

Submit and Query

Developer Tasks

Non Developer Tasks

Developer Tools

Tools

The copyright of the content on this website is held by the contributing authors, except where stated elsewhere. see CopyrightStatement.