Robots.txt - General Support - ProcessWire
Hey guys,. It's been a while! Any rules of thumb as to exactly what are the minimum directories that need to be made open in robots.txt so ...
Environment-specific robots.txt - General Support - ProcessWire
Is there any way with PW to do environment-specific robots.txt, i.e. to block robots from staging sites without having to manually edit ...
TV Series on DVD
Old Hard to Find TV Series on DVD
Need help blocking a crawler - General Support - ProcessWire
I tried with robots.txt. but this crawler does not seem to respect what is configured. in robots.txt. I need something more powerfull with a ...
htaccess rewrite runs into processwire error - General Support
Hi, I have a site that has other ccTLDs, such as .co.za, .co.il, etc., I am trying to make a robots.txt for each of the different domains by ...
Hide dev site form google - General Support - ProcessWire
Use a robots.txt or the meta tags as you describe. I prefer option 1 since it's a matter of keeping everyone out except clients with an username ...
Editing .htaccess – OK or NOT? - General Support - ProcessWire
The ErrorDocument stuff is just an optimization to keep the favicon.ico and robots.txt requests from getting sent to PW. It is fine to remove ...
Forumer - Diecastcollecti
Robots.txt - General Support - ProcessWire. Hey guys,. It's been a while! Any rules of thumb as to exactly what are the minimum directories that need to be ...
Forumer - Thepirateserver
Environment-specific robots.txt - General Support - ProcessWire. Create a new template, call it robots, and set its URLs > Should page URLs end with a slash ...
Googlebot & Images - General Support - ProcessWire
Hi there,. Using an Image field, PW stores the images in 'site/assets/files'. For a photography site, all of the "public" images should be crawled and ...
Greg Lumley's Content - ProcessWire Support Forums
... favicon.ico was not found.