Message:

<p>[QUOTE=&quot;Merc Crazy, post: 1394172, member: 27417&quot;]No offense intended, but that is spoken like a dinosaur.</p><p><br /></p><p>The way these bots work is they go to a webpage and crawl every page of the site, building databases out of key words they find on those pages.&nbsp; So if there&#039;s a 40 page thread, that bot will crawl over the whole thread, every page, every word, sticking the key words into a database somewhere associated with that page.&nbsp; The burden of search bots on a forums traffic can be fairly heavy if it&#039;s a popular and very active forum, as is this one, hence why most search engines will allow you to opt out of showing up on their results.&nbsp; If you opt out, the bots will no longer access pages on your domain.</p><p><br /></p><p>How do I know?&nbsp; As admin of a forum with about 500 monthly users, I can tell you that prior to opting out of the search engine results, we were seeing somewhere around 100-150 bots per day crawling all open forums.[/QUOTE]</p><p><br /></p>

Your name or email address:

Do you already have an account?

• No, create an account now.
• Yes, my password is:
• Forgot your password?

Stay logged in