The Future of Search

Search Engine Honesty

Search engines are relatively recent technology. The future of search engines depends somewhat on how the courts eventually decide to classify search engines.  Are search engines publishers or service providers?

Search Engines are Publishers

If the courts definitively (e.g. Supreme Court decision) decide that search engines are publishers, publishing what amounts to editorial opinions on their results pages, then search engines could more openly impose any desired editorial policy on search results including outright censoring, site-unique ranking bias, etc.  Eventually it would become widely known by users that search engines were "editorial" in nature and delivered "edited" results..

One of the obvious consequences would almost certainly be the imposition of fees on commercial sites wanting to be included in "organic" search results. At one time a listing in the Yahoo Directory was free.  A directory listing for a commercial site now costs $299 per year.  Non-commercial listings are still free but have lower priority in the review process. 

A similar approach could easily be adopted by search engines.  While many webmasters might initially decry this development, it could well be that the imposition of a modest fee would lend some degree of stability and certainty to the present essentially chaotic situation surrounding organic listings.  Many, probably most commercial sites would be willing to pay a small fee to avoid or at least reduce random censoring and enormous random swings in ranking.  Such a development would presumably require somewhat firmer definition and documentation by search engines: What constitutes "commercial"?  What constitutes "acceptable"?  A site paying a fee would presumably be entitled to expect at least somewhat better support in fixing any problems at the search engine/web site interface.  Fee-paying web site owners would be "customers" of search engines, though no doubt on a lower tier than advertising purchasers.

Another consequence might be restraints on competition.  Google might want to be less competitive.  The "publisher" designation probably would cause major problems if one search engine became a virtual monopoly.  Could we afford to be in a permanent situation in which nearly all the information on the Internet was filtered through a single publisher's editorial policy?  (See The Web Czars.)  In the past there have been restrictions imposed to prevent just that sort of "monopolization of information" such as restrictions on the number of radio or TV stations that could be owned by a single entity.  A logical extension of the sort of logic that led to a desire to "break up" Microsoft might also apply to Google.

Another possible outcome is a sort of search diversification.  There could be left-leaning search engines, right-wing search engines, search engines for various special interest groups (teens, women, etc.)  Just as a single publisher can publish dozens of different magazines, each having a different editorial viewpoint and appealing to a different target audience, a single search system publisher could provide dozens of different editorial slants.  

Search engines currently appear to be mainly using outright censoring as opposed to site unique bias to suppress access by their users to sites they don't like.  Outright censoring creates many problems with unhappy site owners but allows search engines to claim that their search algorithms are fairly applied to everybody.  If it becomes legally and socially acceptable for search engines to have editorial policies, outright censoring will probably be replaced by site unique bias. 

Search Engines are Service Providers

The courts might decide for reasons outlined in this site, that search engines are service providers that provide the capability for users to connect to web sites, an essential service.  As such, search engines would not be allowed to make arbitrary and undisclosed editorial decisions regarding censoring of particular web sites.  Rules defining the requirements on web sites in order to be reachable via a particular search service would need to be well defined and fairly enforced by that search service.

Some search engine proponents claim that such a decision would be unworkable in that search engines need secrecy in order to combat spam.  This argument is ridiculous -- see the National Security Argument

Others claim that the workings of a search engine are necessarily arbitrary.  Some aspects of engine operation are indeed subject to various vagaries of the technology.  An "editorial policy" does not, however, hinge on these vagaries but is instead implemented by taking positive, proactive, steps such as "manual" censoring or site unique bias. Therefore, a search engine could be free of "editorial" control and still have technical vagaries.

Others say that the "quality" of search engine results would be reduced if search engines were not imposing editorial selection.  There is no doubt some truth to this claim.  However, it is also true that an edited directory (e.g. Yahoo Directory), or even a newspaper or magazine has much higher "quality" (defined in the same way) than the best search engine results.  So why would anyone use a search engine?  Apparently people use search engines to get access to the largest amount of fresh data with the least amount of editorial filtering.  They are willing to give up some "quality" in order to get better access.  Objective quality guidelines that were publicly defined and fairly applied would not constitute an "editorial policy" and would be acceptable in a service provider.

Some search engines might even welcome such a reform as long as it was imposed on all the search engines.  None will likely implement such a reform unilaterally.

The Bottom Line

No matter what the courts decide, search engines are here to stay.  In either event the relationship between search engines and websites should be better defined.  If search engines were secure in their ability to impose arbitrary editorial policies, and there was less to lose by admitting to such a policy, search engines could be more forthright regarding deception, configuration issues, etc.  There would be less need to hide things.  Search engines could be more honest.  If search engines were required to provide a fair service, there would be "APIs" defining all the rules.  Either way there is less need for SEO, less confusion, and smoother, more effective search.

Google seems to have overwhelming advantages conveyed by economies of scale, superior technology, and the tracking advantages associated with their AdSense network. We can expect Google's dominance to increase.

Search Engine Honesty

Copyright 2006 Azinet LLC