Photo Credit: Brian Oberkirch
There has been a lot of discussion among bloggers about web 3.0 (Dare I say).. and how search is broken. People are very passionate about what they believe in and so is Jason Calacanis when is he vehement that search is broken and has to be fixed. Here are some of the facts:
- Google provides decent results for a majority of people.
- Is it perfect? Absolutely not!
- Has the quality of these results diminished over the years ? Sure!
- As search has evolved, people have come up with other solutions.
- Vertical search engines do a much better job than google.
- Google recognizes that vertical search engines do better and have come up with their own versions (case in point blog search)
Search is not broken, it needs to be bettered! Google is currently tinkering it! Tinkering is a short term solution!
Lets take an example of the wild fires in california:
- How many people know to use twitter to get their updates? Very few hundred (According to the number of followers on twitter its about 400+)
- Given that fact that CNN mentioned twitter.. May be a few hundred or thousand more would join in..
- Google doesn’t show the twitter updates in the first page for “California wildfires“, even though it indexes Los Angeles Fire Department LAFD’s twitter updates.
- The key to getting good results from google is knowing which key words to search.. so if I search for “california wildfires” I get a lot more news results and if I search for “LAFD updates”, the twitter feed is right on top. Google doesn’t make the connection.
The question of how good search is really dependent on which vantage point you are looking from! If you are an end user who is technology agnostic you think search is pretty good (Not many end users understand S.E.O and fewer technologists really get S.E.O). But if you are “a power user” of search engines than its pretty apparent to you that you use google as your first line of defense and then venture out on your own. This is Jason’s point and it is well taken .. A year of so from now will “normal people” (as in my uncle or aunt) be as happy with google as they are now? As more people start using S.E.O to game google, the more impact it has on the quality of the results. Its only a matter of time before this happens! What then? Where do we go from there?
Do we use a human search engine like Mahalo.com? or
Do we use out to collaborative search engines like the ones from delicious.com, stumbleupon.com? or
Do we use to generic semantic search engines like powerset.com and Hakia.com? or
A combination of all of the above?
This is where the debate is focussed on! This is where web 3.0 discussions come in play. There is one thing that we all agree upon..
Web 3.0 will provide “better information discovery”
The question is how? I believe it will be a combination of these technologies. A layering of these technologies and techniques!
Human discovery will be better than an automated solution (as it stands today). What Mahalo does is layering “human refinement” over automated search (using google, etc). Mahalo.com shows the fires right on top of the news items. The guide notes definitely are great! The page has news links which is nice and other information, no twitter links. Why not? No links to the LAFD’s site? Also where is a link to the flickr photos or clusters? The page is as good as the person who creates it. Now can you call this an expert page? I don’t think so! Its better for an end user who need not worry about honing it for the right keywords. Wikipedia seems to have an over load of information on the fires.
Will this be sufficient? In terms of quality definitely! In terms of the breadth of things to look for .. No way! As per Jason’s calculations its about 18K terms/SERPs. Thats a drop in the ocean. Jason believes that paying people who provide good information is the way to go. I don’t disagree with the approach, but you are going to need a human army! A really big one! Here’s a suggestion to Jason, there a lot of well educated women in india who are home taking care of their families that would love a part time job. Just sign them up! I wonder why Mahalo isn’t using Amazon Mechanical trunk. Seems like a match made in heaven. There is one thing I disagree with Jason – he extrapolates how Mahalo solves the search betterment problem. But one cant blame him for it, its his passion! I just wish he tempers his argument and his point would be well taken. Lets be honest, Mahalo does not solve this problem. It rather provides one of the pillars for search betterment. And Jason is candid about this:
“One solution isn’t going to solve all these issues, but we’ve done over 18,000 pages so far at Mahalo, each of which serves at least 20 search terms. Over time we’re going to be able to solve the fatigue problem for the first 1/4 to 1/3rd of search terms. If we can do that we will have a really big business that helps a LOT of people–a LOT!”
Lets look at collaborative search engines. Delicious seems to have a good set of links for “California Wildfires” but not very useful for an end user. Moreover this information is not up-to date. Delicious seems to be good at discovering sites on links for topics. I use it extensively to track trends (for example ruby-on-rails). Mahalo does link to a particular search string “California Fires”, but people might be using other terms to bookmark them. It is rather sad that delcious doesn’t have same tag clusters that flickr has. There is lot of work that still needs to be done.
Semantic search engines: Lets look at Hakia. (I don’t have access to Powerset, if you can hook me up please let me know! I have heard a lot of hype about this product but have seen very little. Not a good thing! Trust me I speak from personal experience!) As for Hakia, lets hope that this is not how a semantic search engine would look! Results are disastrous! If this the quality of search results that Hakia or Powerset is going to product then lets all buy google stock! We would have more money saved for a few more generations! Results from 2003?? Are you kidding me? WTF! I would bereally scared if I was a investor! Come On!!! Technologies exist today that do a very good job of understanding semantics! I would have expected Hakia to tell me about the locations, organizations, people and updates. So in the instance of the wildfires I would have expected it to recognize LAFD and hence show me the twitter page for it! These bozos are not even indexing twitter! Hopeless! Lets hope Powerset does a much better job otherwise i may be tempted to throw my hat in!
Web 2.0 has done an awesome job is crowd-sourcing data! Be it videos(from youtube), locations(from wikimapia.org), or pictures (from flickr)! Meanwhile semantic algorithms have made great strides! Simultaneous work has been done on opening data and categorizing them. There is wealth of data in the crowd-sourced data! It is time to enrich this data with these technologies.
“Better Information discovery” will be about combining these technologies and patterns and packaging it together!
This is my take on the next phase, you are entitled to your views. I would be happy to engage in a good healthy debate. Please voice your thoughts below. I have purposely not touched upon the mobile revolution, I will save it for an other day!
If you are tired of reading such a long post (like I usually get), please subscribe to a podcast version of this blog.
Click here to subscribe to this feed with iTunes
Technorati Tags: hakia, mahalo, powerset, semantic, search, Technology, web20, web30
No bad… two descent posts! I feel pretty darn good!