Posted - November 23 2005 : 18:14:12
| I use 2 shoppage_header files for various pages. Both have robots meta tags. 1 has index,follow and the other has noindex,nofollow. I apply them to whatever page is appropriate.
I also have a robots.txt file in the main directory tree that disallows what ever pages I want.
The robots don't have to follow the instructions, but none of the disallowed pages have been indexed.
Here is what my robots.txt file looks like;
Edited by - bme on November 23 2005 22:24:36