Is "robots" content="all" the best for me to use?

	Home Register Search Help Login

Sponsors
Frontpage website templates
Creative Website Templates for FrontPage, Dreamweaver, Flash, SwishMax
Shopping Cart Software
Ecommerce software integrated into Frontpage, Dreamweaver and Golive templates. No monthly fees and available in ASP and PHP versions.
Website Templates
We also have a wide selection of Dreamweaver, Expression Web and Frontpage templates as well as webmaster tools and CSS layouts.

Recent Posts
Todays Posts
Most Active posts
Posts since last visit
My Recent Posts
Mark posts read

Is "robots" content="all" the best for me to use?

View related threads: (in this forum | in all forums)

Logged in as: Guest

Users viewing this topic: none

Printable Version

All Forums >> Web Development >> Search Engine Optimization and Web Business >> Is "robots" content="all" the best for me to use?

Page: [1]


mr_tin Posts: 10 Joined: 2/1/2007 Status: offline	Is "robots" content="all" the best ... - 2/23/2007 9:58:59 Thanks everyone for all the help I have received on this website. I'm almost finished my first FP2003 website and I am now turning my attention to SEO. If I want robots to see my site is the tag "robots" content="all" the best one to use? Thanks - Martin


womble Posts: 6285 Joined: 3/14/2005 From: Living on the edge Status: offline	RE: Is "robots" content="all" the b... - 2/23/2007 10:06:20 <meta name="robots" content="index, follow" /> _____________________________ ~~ "A cruel god ain't no god at all" ~~ ~~ Erase hate. Practice love. ~~ (in reply to mr_tin)


jurgen Posts: 446 Joined: 1/9/2007 From: Castle Rock, Colorado Status: offline	RE: Is "robots" content="all" the b... - 2/23/2007 11:46:31 You might want to use a robots.txt file as well. (in reply to womble)


mr_tin Posts: 10 Joined: 2/1/2007 Status: offline	RE: Is "robots" content="all" the b... - 2/23/2007 13:24:18 What is a robots. txt file composed of? (in reply to jurgen)


womble Posts: 6285 Joined: 3/14/2005 From: Living on the edge Status: offline	RE: Is "robots" content="all" the b... - 2/23/2007 14:21:43 http://www.robotstxt.org/ _____________________________ ~~ "A cruel god ain't no god at all" ~~ ~~ Erase hate. Practice love. ~~ (in reply to mr_tin)


mr_tin Posts: 10 Joined: 2/1/2007 Status: offline	RE: Is "robots" content="all" the b... - 2/24/2007 17:50:54 Not sure I understand the robots.txt file thing. If I use Notepad to create a robots.txt file that has the following two lines in it saying: User-Agent: * Disallow: What do I do with the txt file then? How do I get it on my index page? or do I put the two lines on the index page as meta tags or something? (in reply to womble)


jurgen Posts: 446 Joined: 1/9/2007 From: Castle Rock, Colorado Status: offline	RE: Is "robots" content="all" the b... - 2/24/2007 20:49:17 First, the robots.txt file should be in your root directory. It has nothing to do with you actual pages or html code. In your initial post you ask for the meta tag "robots". Yes you want your site spidered, but maybe some of the files you don't want to show up in the search engine. For an example something like Disallow: /style/ Disallow: /images/ This would be the directories you instruct the robots NOT to index. You also can asign certain file as well. With " User-Agent: * " you basically tell ALL spider to follow the rules of "disallow: whatever". You can disallow "user-agents" like googlebot, where this spider would not index your 'style' and 'image' directory. You will have some control of certain crawler to do certain things. But on the contrary, "bad" and "nasty" crawlers just don't give a dung and do whatever they have been programed for. For the most part it will do the job. If you put an empty "robots.txt" file in your root directory means you have noe restrictions to any robots, but at least you won't get the anouying "file not found" in your error log.... (in reply to mr_tin)


mr_tin Posts: 10 Joined: 2/1/2007 Status: offline	RE: Is "robots" content="all" the b... - 3/4/2007 19:47:19 I'm having problems with this concept...where is my root directory? I am already getting the error "File does not exist: /home/evanoff/public_html/robots.txt" I just don't see the root directory in my published website, am I not seeing the forest because of the trees? Here's my website - www.evanoff.ca I would appreciate any help on this, I would like to get my site picked up by search engines. Thanks - Martin (in reply to jurgen)


treetopsranch Posts: 1651 From: Cottage Grove, OR, USA Status: offline	RE: Is "robots" content="all" the b... - 3/4/2007 20:38:17 Martin, are you using FP to publish or are you using a FTP program? If you use FP just put the robots. txt file in the same directory as your index file on your home computer. When you use FP to publish it will put that file in the correct spot on the server. If you use a FTP program just put it in the same directory as your index file on the server. (the main index, the one that opens your home page) And don't forget to validate that robots.txt file here http://tool.motoricerca.info/robots-checker.phtml _____________________________ Don from TreeTops Ranch, Oregon "I've got a taste for quality and luxury" (in reply to mr_tin)


mr_tin Posts: 10 Joined: 2/1/2007 Status: offline	RE: Is "robots" content="all" the b... - 3/5/2007 18:40:18 Thanks - that worked. I finally realized it just went in the root directory. Martin <edit>Edited link</edit> < Message edited by Reflect -- 3/7/2007 10:53:44 > (in reply to treetopsranch)

Page: [1]

All Forums >> Web Development >> Search Engine Optimization and Web Business >> Is "robots" content="all" the best for me to use?

Page: [1]

Jump to: 1

New Messages	No New Messages
Hot Topic w/ New Messages	Hot Topic w/o New Messages
Locked w/ New Messages	Locked w/o New Messages

Post New Thread

Reply to Message

Post New Poll

Submit Vote

Delete My Own Post

Delete My Own Thread

Rate Posts