Barbarians at the gate: Excluding Bing via robots.txt

This isn’t about the variety of cherry. If you haven’t heard of Bing.com, it’s Microsoft’s recent attempt at a search engine. (If you’re curious, Google it.)

If you are not particularly pleased with the idea of a company like Microsoft making money from your creative work, then my strong suggestion is to create a robots.txt file in the root of your web-space, with contents not unlike:

User-agent: msnbot
Disallow: /

User-agent: *
Disllaow:

The robots.txt is a voluntary standard which allows web page authors to exclude search engines from part or all of their sites. There’s a helpful website that has details about why you might want to do this, and how to go about it.

Of course, there have been some allegations that Bing isn’t honoring the robots.txt standard. But announcing that they’re unwelcome is a fine symbolic act, even should they fail to honor your wishes.

By dhenke

Email: dhenke@mythopoeic.org

Leave a comment

Your email address will not be published. Required fields are marked *