arfn24
Posts : 44 Join date : 2008-07-16
| Subject: Yahoo Bots Thu Jul 17, 2008 10:34 pm | |
| There was a Yahoo bot here earlier... They say their bots search for stuff to put in the search engine. This could be bad if we want to stay a secret. | |
|
jrdnxxhero Admin
Posts : 173 Join date : 2008-07-16 Location : Las Vegas
| Subject: Re: Yahoo Bots Thu Jul 17, 2008 10:44 pm | |
| - arfn24 wrote:
- There was a Yahoo bot here earlier...
They say their bots search for stuff to put in the search engine. This could be bad if we want to stay a secret. Im sure people won't search for Hero Mod. P.S. I'm compiling a bot to track IPs. all Huge websites like google, yahoo, msn, have more than one server and IP. so if it detects more than one IP, it blocks it. | |
|
arfn24
Posts : 44 Join date : 2008-07-16
| Subject: We're not in the search Thu Jul 17, 2008 10:46 pm | |
| Were not in the search Not yet at least. All their bots are at 66.77.73.xx | |
|
coolguy5678
Posts : 82 Join date : 2008-07-17
| Subject: Re: Yahoo Bots Fri Jul 18, 2008 4:22 am | |
| Is there any way you can place a robots.txt to stop spiders from going on this site?
Otherwise, you should make all the forums password protected. | |
|
arfn24
Posts : 44 Join date : 2008-07-16
| Subject: Re: Yahoo Bots Fri Jul 18, 2008 11:10 am | |
| That's exactly what yahoo says to do. | |
|