« previous next »
Pages: [1]

Topic: Web Robots  (Read 1869 times)

0 Members and 1 Guest are viewing this topic.

Offline Bonk

  • Commodore
  • *
  • Posts: 13298
  • You don't have to live like a refugee.
Web Robots
« on: August 13, 2005, 09:54:55 am »
Logged

Offline toasty0

  • Application.Quit();
  • Captain
  • *
  • Posts: 8045
  • Gender: Male
Re: Web Robots
« Reply #1 on: August 13, 2005, 10:02:09 am »
I remember reading somewhere that Google and possible Yahoo ignore that protocol now. Not sure how accurate that info is though...
Logged
MCTS: SQL Server 2005 | MCP: Windows Server 2003 | MCTS: Microsoft Certified Technology Specialist | MCT: Microsoft Certified Trainer | MOS: Microsoft Office Specialist 2003 | VSP: VMware Sales Professional | MCTS: Vista

Offline Bonk

  • Commodore
  • *
  • Posts: 13298
  • You don't have to live like a refugee.
Re: Web Robots
« Reply #2 on: August 13, 2005, 03:15:52 pm »
Google and Yahoo both are still looking for robots.txt, not sure if they are heeding it or not (could test relatively easily).
Logged

Offline toasty0

  • Application.Quit();
  • Captain
  • *
  • Posts: 8045
  • Gender: Male
Re: Web Robots
« Reply #3 on: August 13, 2005, 10:01:50 pm »
If you do let me know, but I think it was one of those ranting blogs that included gloom and doom forcasts for the future of net.
Logged
MCTS: SQL Server 2005 | MCP: Windows Server 2003 | MCTS: Microsoft Certified Technology Specialist | MCT: Microsoft Certified Trainer | MOS: Microsoft Office Specialist 2003 | VSP: VMware Sales Professional | MCTS: Vista

Offline Just plain old Punisher

  • Vice Admiral
  • *
  • Posts: 36927
  • Gender: Male
  • I'm not facist, I just like wearing jackboots
Re: Web Robots
« Reply #4 on: August 23, 2005, 03:05:23 am »
Ummm doom and gloom?

Can I commence profiteering and hoarding?
Logged

"Sex is a lot like pizza.  If you're not careful you can blister your tongue". -Dracho
Pages: [1]
« previous next »