A big topic in the forums is how people can generate a sitemap when they have dynamically created website such as xoog. this is easy, i dont understand why people have had so much trouble.
The route I have gone down is to submit a text file listing each page on a new line.
This isnt the best way of doing things, although google indexes the pages, so as far as im concerned, my way is perfectly fine.
What you do is go into sitemaps and tell google where your text file is located, and it spiders the text file so it knows what pages you have.
googlebot doesnt seem to spider as deep as some other search engines, right? i think it just gets bored. but i still need it to find every unique page of mine.
My Method
what i do is use REQUEST_URI on every page that the user visits and log it into a database table. I make sure that every entry is unique.
I then use a combination of php to output a file, and htaccess to make the file appear as a .txt (or .xml if you swing that way) file.
this way my users (and the deeper-crawling bots) generate my sitemap file for me :) Plus every time google checks my sitemap file it will always be updated.
This method relies on users and bots clicking away through your site, however if they arent doing this... Is there any point in your site ever being indexed in google anyway? No!
Subscribe to:
Post Comments (Atom)
No comments:
Post a Comment