Weird Companions

Please explain the terms, spider, crawler, bot. Who are they?

I think they’re software programs that roam the net getting information to provide search engines with.

Zoot is exactly right.

Spider and crawler refer to the same thing, and they are both a type of bot, which is any program that navigates the internet autonomously. They can be used for more than just search indexing, but most of the ones that visit ILP are search indexers.

There are several open source versions if you really want to get to know them.