As we start making the transition from Web 2.0 to Web 3.0, I can’t help but notice the hype around real-time web. Everyone from the VCs to the end-users are going gaga over it and money is being poured in, but no one really knows what the outcome will be and whether real-time data ‘really’ impacts our lives (i.e consumers).

The  joke in the entrepreneurial corridors today is that if you have a business idea make sure you end the mission statement with “in real-time” , then use the word at-least 50 times in the business plan and you’ll get funded!

Several funds and investors (prominent among them – Ron Conway and John Borthwick) are focusing on real-time startups. $23.27 million has been invested into pure twitter based startups and many more millions are being invested in other real-time startups. The  recent acquisition of Friendfeed by Facebook and Summize by Twitter has fueled more interest in this area. Even companies like Google and Microsoft are feeling the pressure and are improving their search engines to index data quickly (read/view Larry Page’s comments) while many of the smaller real-time startups hope to get bought out.

However, the question is whether real-time data “really” makes a difference in anyone’s lives. Shouldn’t startups focus on On-Demand web instead?

No latent need for real-time data

Does real-time mean getting data every minute? every second? or every nano-second? Every startup competes to improve on this statistic when it does not matter to the end user… unless they are watching the updates stream in continuously. My guess is that most people who have jobs  and other activities cannot and will not have the time for this. However on the other hand, it is in the vested interests of the real-time websites to keep you hooked (how else will they make money?).

And how critical are these real-time feeds? 99% of the data you see in the feeds is not critical or life-changing. It hardly matters whether you see the update instantaneously or after 5 hours.

Temporal nature – real-time or no data at all

Today, in 5 hrs you will miss tons of updates. If you don’t see the data real-time, you cannot see it at all. Real-time data is not persistent. Have you ever tried looking for information posted a week ago or even a 24hrs ago? Do you know what your own status update was exactly a week ago? You can’t find it easily.

What about applications that rely on real-time data?

Maybe, it is useful for stock-trading, reporting (because media competes to get the news out first) or if you are monitoring your shuttle in space. But only a very very small fraction of users fall into this category.

And corporations monitoring consumer data and customer feedback?

Well, if your customers are dissing your product, there is not much you can do to instantly to change the perception or the quality of the product. While ‘knowing’ instantly may be useful, ‘acting’ on the data quickly is more important. You cannot act on the data quickly because you take a week or two to understand the data.

Sifting through real-time junk isn’t easy…

The biggest problem that users, companies and even search engines like Google and Bing face today is sifting through the ‘information junk’ (90% of  posts). There is no history, relevance or weight attached to the real-time posts. And imagine how much of this junk any algorithm will have to sift through and map… With each user making at least 20 status updates and posts everyday and with 275 million people it turns out to be 5.5 billion updates a day!

And add global warming concerns to it…

More data centers, more processing, more power and probably more jobs but with impact on global warming for sure. Companies like AMD and Intel are burning midnight oil to develop next generation multi-core processors and compilers to keep up with the processing challenges. While  these companies add to the climate problem others are working on technologies like smart grids, smart homes etc to solve it.

So why is on-demand better and useful?

Because that is data you want  and when you want it. Yes, we still need to capture the real-time data but the emphasis will be more on making sense of that data and presenting it in a meaningful and useful way. It does not involve spending hours and hours staring at screens, waiting for the next meaningless update from a supposed friend whom we don’t even care about or know well!

What we need today is a Tivo for real-time data. Something that allows us to leave the laptops and mobile phones behind and enjoy the real things that life and this world has to offer while at the same time experience the benefits of a technology when we need to!

On the other hand, technology companies with vested interests will push their own agenda. They will convince and get you addicted to anything that you may not really need. They will work on getting the real-time data updated every nano-second, so you are more glued to the laptop (and push Ads in the same stream), market a TV with much higher resolution when the fact is that a human eye cannot distinguish between a 720p or 1080p from 6-7 feet away, and sell you the next generation audio equipment with 24bit+ sampling although the human ear can hear only up to 24Khz.   If they don’t do that how else will the economy grow?

Whether these technologies are improving our lives or changing how we live, I am not sure yet. Whatever it is, on-demand web definitely makes more sense than real-time web as we move to web 3.0!

Update: As we think about Tivos for real-time data, here is an hilarious Tivo commercial that probably drives home the point of skipping information we really don’t need!