The Industry

Search Engines… Do they work?

Some of them do.

Well, one anyway.

Weirdnesses abound.

Cuil was downloading our robots.txt about 1000 times for every actual page they downloaded. Due diligence is great, and, yes, you never know when we might change the robots.txt file and block search engines from something, …but… Well, we hardly see Twiceler at all anymore - but at least they got us to make the fetching of robots.txt as minimally taxing on our resources as we possibly could [which is pretty minimal]

Bing has indexed about 500 pages, last we checked, out of several million in our sitemap. The SEO experts say Bing focuses on and ranks by the age of a website - that 11 year-old sites beat out measly old 10 year-old sites. Which just means that Bing will be irrelevant with respect to Mattters.com [which is new] for the remainder of my lifetime [and I’m old, but not THAT old]. I understand the 80-20 rule, and letting Google do the heavy work and index EVERYTHING while Bing goes for the cream, but this particular approach is flawed [certainly in our case! :-)].

Yahoo, besides Slurp’s love affair with 404s [some kind of 404 fixation issues, numerology experts please step up] kind of sucking because it has to go all the way to our backend each time, their site registration is broken. But no worries - they are about to merge with Bing, with some of the best WebMaster information-tracking control panels around…. [ :-), that is irony AND sarcasm - just FYI].

Meanwhile, AOL seems to be actually working albeit on a reduced scale, as do numerous small fry. And, of course, Google. One of the goals of Mattters is to replace search engines for various specific tasks related to people following and enjoying their interests as opposed to, oh, I don’t know… Writing a paper for school or something. I think we have a chance here.