urllist.txt - yahoo (Full Version)

All Forums >> [Web Development] >> Search Engine Optimization and Web Business



Message


J-man -> urllist.txt - yahoo (12/5/2005 3:52:09)

for anyone who does not know this (maybe do a search on it):

create the file then goto http://submit.search.yahoo.com/free/request

My question: I have a folder on the web that has 300 files in it, what is the easiest way to create a url list of these files (which have no name convention standards)

thanks




Reflect -> RE: urllist.txt - yahoo (12/5/2005 10:55:08)

Ahhh, Yahoo!s' comeback for Google. Guess it had to happen.

Me I use my site search tool as it lists all pages and allows an export to Excel. FDSE by he way is the name.

Anyway, I woyld use a site ripper, most have an export to spread sheet function.

Here isw onw that I just found for you...

http://www.newprosoft.com/web_spider.htm

http://www.freedownloadscenter.com/Network_and_Internet/Web_Searching_Tools/Web_Scraper_Plus_.html

Take care,

Brian




J-man -> RE: urllist.txt - yahoo (12/5/2005 16:03:06)

couldn't get the compression method to work, anyone simuliar ?

...thanks Brian




J-man -> RE: urllist.txt - yahoo (12/5/2005 18:42:45)

Edited - see below




Reflect -> RE: urllist.txt - yahoo (12/6/2005 11:48:28)

J-man, question on your last post. How do you use that? Do I make a file in the root of my web and just call it up in a browser?

I tried pushing it out as an ASP page but recevied an error on for each x in fo.files, stated it expected end of statement.

Thank you,

Brian




J-man -> RE: urllist.txt - yahoo (12/6/2005 14:42:57)

Strange when i copied an pasted from note pad the spacing got messed up, this is better and i modified it to produce more of what we may be looking for:

You can put the file anywhere as long as the GetFolder path is direct at the folder you want a list of.
<%
dim fs,fo,x
set fs=Server.CreateObject("Scripting.FileSystemObject")
set fo=fs.GetFolder("d:\webs\site\folder\folder")
' //use Server.MapPath above to resolve physical path to folder you wish to get a list from
for each x in fo.files
'Print the name of all files in the folder with the http://www.company.com/
Response.write("http://www.company.com/maybeAnotherFolderHere/" & x.Name & "<br />")
next
set fo=nothing
set fs=nothing
%>


Question: i use some Flash applications on my site and I was wondering if it matters if i try to get the Y! bots to crawl .swf files - actually i'm not sure how swf files are indexed; independetly or if they are even indexed? Should i strip them out of the list or keep them in?

Thanks

BTW - maybe we can have a group effort to take this script to the next level? have a button that creates a txt file with the data, plus the option in the asp code to comment or uncomment certain file format extensions so the results don't or can be included - things like PDF, SWF, images, ets... And have the script beable to display the urls of any folder we ask it to. Any thoughts on this?




Reflect -> RE: urllist.txt - yahoo (12/7/2005 9:59:01)

Actually I was thinking on sort of the same level. I thought, in this thread, if that did give an output it would greatly help a few people on the forums. I really can't add to what you have though [X(] as I am not the right person for the job.

If you have Pythin available Goolge allready has a solution. Alas I do not [:@].

https://www.google.com/webmasters/sitemaps/docs/en/sitemap-generator.html

Take care,

Brian




J-man -> RE: urllist.txt - yahoo (12/9/2005 21:11:45)

no python [&:]




Kitka -> RE: urllist.txt - yahoo (12/9/2005 22:11:28)

quote:

If you have Pythin available Goolge allready has a solution. Alas I do not


Not having Python doesn't mean you can't have Google site maps. There are other tools:

http://code.google.com/sm_thirdparty.html




Page: [1]

Valid CSS!




Forum Software © ASPPlayground.NET Advanced Edition 2.4.5 ANSI
0.046875