“It’s not information overload we are facing. It’s filter failure.”
If we feel that we’re overwhelmed with information, that means our filters aren’t working. The solution is to fix our filters, especially social filters that rely upon the aggregated judgments of those in our social networks.
We shouldn’t freak out about information overload because we’ve always been overloaded, in one way or another. The new filtering techniques are disruptive, especially when it comes to the authority of knowledge. Old knowledge institutions like newspapers, encyclopedias, and textbooks got much of their authority from the fact that they filtered information for the rest of us. If our social networks are our new filters, then authority is shifting from experts in faraway offices to the network of people we know, like, and respect.
If you are on the Acquisitions Committee of your town library, you are responsible for choosing the trickle of books to buy from the torrent published each year. Thanks to you, and to the expert sources to whom you look, such as journals that preview forthcoming titles, library patrons don’t see the weird cookbooks and the badly written personal reminiscences that didn’t make the cut, just as the readers of newspapers don’t see the crazy letters to the editor written in crayon. But many of the decisions are harder. There just isn’t room for every worthwhile book, even if you had the budget. That’s the way traditional physical filters have worked: They separate a pile into two or more piles, each physically distinct.
The new filters of the online world, on the other hand, remove clicks, not content. The chaff that doesn’t make it through the digital filter is still the same number of clicks away from you, but what makes it through is now a single click away. Even if a paper is the millionth result in a Google search, a different search might pop it to the top, and you may well find it through an email from a friend or on someone else’s top-ten list.
Filters no longer filter out. They filter forward, bringing their results to the front. What doesn’t make it through a filter is still visible and available in the background.
Compare that to your local library’s strategy. In the United States, 275,232 books were published in 2008, a thirty-fold increase in volume from 1900. But it’s highly unlikely that a library got hundreds of times bigger during those past 110 years to accommodate that growth curve. Instead, your library adopted the only realistic tactic, each year ignoring a higher and higher percentage of the available volumes. The filters your town used kept the enormous growth in book-based knowledge out of sight. As a result, library users’ experience of the amount of available knowledge didn’t keep up with its actual growth. But on the Net, search engines answer even our simplest questions with more results than the total number of books in our local library. Every link we see now leads to another set of links in a multi-exponential cascade that fans out from wherever we happen to be standing. Google lists over 3 million hits on the phrase “information overload.”
There was always too much to know, but now that fact is thrown in our faces at every turn. Now we know that there’s too much for us to know. And that has consequences.
First, it’s unavoidably obvious that our old institutions are not up to the task because the task is just too large: How many people would you have to put on your library’s Acquisitions Committee to filter the Web’s trillion pages? We need new filtering techniques that don’t rely on forcing the ocean of information through one little kitchen strainer. The most successful so far use some form of social filtering, relying upon the explicit or implicit choices our social networks make as a guide to what will be most useful and interesting for us. These range from Facebook’s simple “Like” button (or Google’s “+1” button) that enables your friends to alert you to items they recommend, to personalized searches performed by Bing based on information about you on Facebook, to Amazon’s complex algorithms for recommending books based on how your behavior on its site matches the patterns created by everyone else’s behavior.
Second, the abundance revealed to us by our every encounter with the Net tells us that no filter, no matter how social and newfangled, is going to reveal the complete set of knowledge that we need. There’s just too much good stuff.
Third, there’s also way too much bad stuff. We can now see every idiotic idea put forward seriously and every serious idea treated idiotically. What we make of this is, of course, up to us, but it’s hard to avoid at least some level of despair as the traditional authorities lose their grip and before new tools and types of authority have fully settled in. The Internet may not be making me and you stupid, but it sure looks like it’s making a whole bunch of other people stupid.
Fourth, we can see—or at least are led to suspect—that every idea is contradicted somewhere on the Web. We are never all going to agree, even when agreement is widespread, except perhaps on some of the least interesting facts. Just as information overload has become a fact of our environment, so is the fact of perpetual disagreement. We may also conclude that even the ideas we ourselves hold most firmly are subject to debate, although there’s evidence (which we will consider later) that the Net may be driving us to hold to our positions more tightly.
Fifth, there is an odd consequence of the Net’s filtering to the front. The old library Acquisitions Committee did its work behind closed doors. The results were visible to the public only in terms of the books on the shelves, except when an occasional controversy forced the filters themselves into the public eye: Why aren’t there more books in Spanish, or why are so many of the biographies about men? On the Net, the new filters are themselves part of the content. At their most basic, the new filters are links. Links are not just visible on the Net, they are crucial pieces of information. Google ranks results based largely on who’s linking to what. What a blogger links to helps define her. Filters are content.
Sixth, filters are particularly crucial content. The information that the filters add—“These are the important pages if you’re studying hypercomputation and cognitive science”—is itself publicly available and may get linked up with other pages and other filters. The result of the new filtering to the front is an increasingly smart network, with more and more hooks and ties by which we can find our way through it and make sense of what we find.
So, filters have been turned inside out. Instead of reducing information and hiding what does not make it through, filters now increase information and reveal the whole deep sea. Even our techniques for managing knowledge overload show us just how much there is to know that escapes our best attempts. There is no hiding from knowledge overload any more.