how to disallow spiders from certain pages

[lsvf]

How do I exclude spiders from crawling and caching certain individual pages? For example, a "thank you" page. In a static html page I would simply exclude all spiders in the header with a meta tag but I don't know what to do on a Soho page.

L

[Ruben]

i believe you can do that by uploading a robots.txt textfile to your root.

check this site /page http://www.robotstxt.org/wc/norobots.html#format

[lsvf]

i believe you can do that by uploading a robots.txt textfile to your root.

check this site /page http://www.robotstxt.org/wc/norobots.html#format

Thank you, Ruben. I am aware of the robots.txt. But, which page to exclude? I have created a page called Thank_You. Do I exclude the .htm redirect pages? What about the Thank_You.php page it creates -- where does it reside? Should I exclude that also?

Also, do you have a Robots.txt that you could share with us? I love this program but it is confusing the way the files are setup (at least for this old woman). Would be nice to have a map of which files affect which -- know what I mean?

Lots of confusion on my end. But maybe one day I'll know it all and be able to answer other's questions :)

Linda

[Ruben]

I don't have a robots.txt file, sorry.

I think you have to ask some others who know more about this, but he... Google is your friend. :)

I understand the confusion. search engines is a big confusion for me also some times.

when i find more info i will share it off course.

[Cameron Allen]

Another way to exclude a page from search engine navigation is to logon to your site. Go into edit page, page properties, then delete CON from the Gateway HTML filed. Also make sure there are no descriptions or keywords entered for that page. Next login to your sites docroot using ftp. Now find the page_name.html file of the page that you don't want to be searched and delete it. This deletes the gateway page that search engines look at. I hope this helps.

[Ruben]

sounds like a cool feature, but isn't it so, when editting the same page, it places this again in the docroot?

[Cameron Allen]

It will write another gateway HTML page but with the CON removed from page properties it writes a blank page.