mvhub-dev team mailing list archive

[Dan MacNeil <dan@xxxxxxxxxx>]

The danger of having bad data indexed by google/bing is greater than danger of production not indexed.

robots.txt has to be at /

but yes it would be possible to move static pages outside version control. 

However for some things, it is nice to have it in ---sharing menus. 

A more eligant solution is to redirect robots.txt to a script that spits out output depending on a 'allow_robots' flag in conf or database. See bug with 'robots.txt' in title


-- 
https://code.launchpad.net/~omacneil/mvhub/add_robots.txt/+merge/21205
Your team mvhub-dev is subscribed to branch lp:mvhub.