That's really not correct any more - the belief is a holdover from the days when search engines had trouble with dynamic URLs. Today's spiders do just fine:
Google Facts & Fiction Quote:
| Fiction: Sites are not included in Google's index if they use ASP (or some other non-html file-type.)
Fact: At Google, we are able to index most types of pages and files with very few exceptions. File types we are able to index include: pdf, asp, jsp, hdml, shtml, xml, cfm, doc, xls, ppt, rtf, wks, lwp, wri. |
Google and Dynamic Pages Quote:
| If you decide to use dynamic pages (i.e., the URL contains a "?" character), be aware that not every search engine spider crawls dynamic pages as well as static pages. It helps to keep the parameters short and the number of them few.
Allow search bots to crawl your sites without session IDs or arguments that track their path through the site. These techniques are useful for tracking individual user behavior, but the access pattern of bots is entirely different. Using these techniques may result in incomplete indexing of your site, as bots may not be able to eliminate URLs that look different but actually point to the same page.
Don't use "&id=" as a parameter in your URLs, as we don't include these pages in our index. |
It's just a matter of removing session IDs for "Guests", i.e., non-logged-in visitors, which includes spiders.
Not only is the mod_rewrite function unnecessary, it has two drawbacks:
1. it may increase server load
2. it may result in a duplicate content penalty