Can Anyone Explain InternetRobots?

[Pat Coyle]

We feel so sheltered. We've heard about crawlers, but we had never heard of Bots such as the onses used on IM serices like AIM. What makes bots different from widgets? Are bots just RSS feeds with a GUIs? Can anyone explain why these mini Web applications are considered a unique species of technology?

[Taulbee Jackson]

The way I understand it . . . (and I'm not a developer, so I may be wrong about this) - a bot is just a script that works on the back end of a site or application, not in the presentation layer (what the user sees), the way a Widget works. Bots gather info from an app or page and actually do something with it, i.e. interact with an application, input info, etc. whereas a Spider just gathers and reports data. One common use of Bots is in internal SEO, where you can specifically tell "good" bots like the ones Google uses that they have permission to crawl your site and do what they need to do to determine relevance. Here's more on "bot herding" as it relates to internal SEO:

http://www.searchandsocial.com/seo-blog/link-shaping-and-bot-herding/

The AIMbots seem to be more like widgets, with bots on the back end that can react to your input, rather than just streaming an RSS feed like many widgets out there. So, rather than the bots being scripts written to perform some action on a page, they are reacting based on your input in a way that gives the illusion of a more human interaction. Like a really smart widget that can talk to you in plain English, like ObamaBuddy here: http://gallery.aim.com/detail/312. What makes them unique is their ability to react to you. SmarterChild is the first app I remember seeing that does this http://gallery.aim.com/detail/284

FWIW, most of the AIM bots I saw do seem to be pretty basic info publishing/RSS driven widgets. I think the AIMbots (tm) terminology is probably more about the marketing than the technology behind it. : )

Hope this helps - surely there's a programmer on SI that can tell me how far off I am?

[Douglas Karr]

How about a widget with a bot? Confusing stuff, yes!

Widgets and Gadgets: widgets are simply compact objects that you can easily insert on your desktop or your web page. The back-end technology can vary widely but they're basically built so you can copy and paste rather than program.

Bots:, such as the AIM bot, are interactive querying engines. You can pose a question or query information and they will respond appropriately. Some of them are becoming quite advanced at natural language queries!

Webbots (website crawling robots, crawlers, spiders, or bots): Taulbee points out the other 'bot' which are really 'spiders'. Whoa, another term! Spiders (like the Google 'bot') crawl your website to index your content appropriately.

[Taulbee Jackson]

Ahhhh. Clarity. : )