The key point here is that real-time monitoring of data is the next big thing. I agree wholeheartedly that it is one of the biggest next things coming up. That and data visualization so that people make sense and understand what the data is saying.
Calling it Web 3.0 is just rubbish. There is real value in services that can produce info/knowledge/data in real time, and we certainly don't need a silly label for it.
<sarcasm>Maybe now live.com will become useful for something </sarcasm>
i'm starting to become increasingly skeptical that with all of the free-flow data out there, there is no algorithm that can magically "put it together" in a way that shows you exactly what you want to see.
the problem? what you want to see is mixed in with a whole lot more things you don't want to see, and there is no metadata available that enables anyone to automagically figure out what to show you -- the data's just not there, even for facebook, and if facebook can't do it, twitter doesn't have a chance in hell.
absent of that magic sorting algorithm, "Real Time Data" becomes too time consuming to sort through.
No one said it was easy, and we have a good model of how the industry might evolve: the problem of "search".
We had things like Altavista, Yahoo!, etc, and then came Google with a better algo. It made sense of the data out there without any real use of meta data (meta tags in HTML notwithstanding, at least initially).
And even to this day, we're seeing interesting experiments in solving the problem of finding information on the web, and how to present it. All I'm saying is that the next step is doing this in real time.
i certainly won't exclude that it's in the realm of possibilities... just that, lately, i'm finding it more and more unlikely.
twitter "authority" seems to me to be an even more difficult problem than blog authority, and even that was easily game-able.
besides, twitter doesn't even have the same social graph for me as facebook does, and facebook still has a pretty difficult time separating the wheat from the chaff.
if that breakthrough doesn't show up, it will be revolutionary. if it doesn't, the "real time web" might just be overhyped.
Remember we've only just recently managed to accumulate all this data in a mainstream way. It will take time for that breakthrough to happen, but it will happen.
Example: when AOL released their "anonymous" data, it took the NYT a few days to personally identify a user. It took humans, working on it for a good few hours, being creative, and produced only one data point (one person), but it still happened. We need to find ways to make deep data mining happen much faster... in real time :)
Emphasis on mainstream, by the way, because data mining is not a new field by any measure.
From my point of view.. what whats wrong with waiting anymore. There has not been one event, that has occurred since twitter came out, that.. I had to absolutely follow and read, the thousands of 140 character messages. Usually the hashtag search is enough as it works now. Example, playoffs, i find out who wins and then I wait for espn to come out with a detailed review. I would rather wait 2 or 3 hours, and read a full article on whatever is going on.
I agree. Generations might not be the best term, even though generations of people (grandparents -> grandkids) can definitely co-exist and learn from each other. Maybe a better term is waves (as in one wave rolling into another)
1953 Math. Tables & Other Aids to Computation VII. 73 With the advent of large-scale high-speed digital computers, there arises the question of their possible use in the solution of problems in ‘real time’, i.e., in conjunction with instruments receiving and responding to stimuli from the external environment. The criteria for satisfactory operation in such real-time service are different from those generally encountered.
Thanks for the correction. Anyway the discussion of real-time vs. batch has been going on since (before) I grew up. I would even argue that the term "scalability" bandied about these days is really a reversion to batch processing - even caching algorithms are a form of batch processing.
Personally, when it comes to investing, I'd rather be a contrarian. Out of these 40 to 50 investments, most will not fly unless they get acquired. "Hope we get bought, fingers crossed" is not a business model. It only works if you invest across a broad section and one of them hits big (which Ron is doing).
I'd rather invest in 40 to 50 simple cash-producing businesses on the web (affiliate networks, lead generation, sticky / viral apps in closed platforms, etc.)
There has always been important real time data like stock market prices, weather, air traffic control etc. None of this changes a bit with the proliferation of internet and web apps, so it is hard to expect any new value here.
The new real time data is generated by real time user interaction - between users and computers and among users (using computers as intermediary). The real time user interaction itself has value and this has been demonstrated: we have continuously invented new kinds of social media built on computer networks. For social media instant is certainly better than fast.
In fact, focusing on data seems wrong here, you'd better focus on the interaction. Even if the actual value is the data, this is where the data would be both generated and used.
"Real time" to me means an interaction delay in the range of milliseconds. If the delay is more in the range of seconds like the web application i am working on then "live" would probably be a better term.
The traditional meaning of 'real time' (at least in embedded stuff) doesn't really specify just how fast, just that things have to be done in a given amount of time. Generally, you're right that those things happen quickly too.
There is a range of timespans that Real Time applies to. In data acquisition, say a medical signal such as an electrocardiographic signal or an EEG signal, we are talking milliseconds.
I wouldn't think of web transactions as real-time, even though there is user expectation of response within a certain small number of seconds.
In the context of the investments mentioned, these seem like the data comes in asynchronously, not matched with a specific request, and that the interesting work to be done is to make sense out of this non-fielded data.
As someone else said here in this thread, Google made search interesting again using essentially unstructured data--no real metadata to get your ERD around. Similarly, the other sources we are thinking of also don't really have that much structure--no tag saying HERE-IS-WHAT-YOU-ARE-LOOKING-FOR. To me, this is why these are interesting problems.
I think event-driven (as opposed to
request driven) is probably a better term to describe the conceptual difference being discussed here. Obviously you could then have different latency requirements for event delivery depending on the application.
I think that real time data is indeed going to be the next big thing. One question which remains is are there any data sources outside of twitter which can be aggregated to make something greater than the sum of the parts?
I know this is a generic question that's a bit off-topic, but how do you pitch to someone like this? How do you get introduced (especially if you're in Europe)?
Not easy. Connections, basically. Better yet, build something on your own that gets traction. Then Angels / VCs will come knocking on your door.
He's the top Angel in the valley. It's like asking, how do I pitch "Speilberg" when I'm in Europe. Instead, do your own Indie film, get it in film festivals, get traction, and the right people will find you.
The "Real" real time data mining will happen when credit card companies/banks offer an API. Otherwise, this whole real time on the web is not that big of a deal for most people.
Calling it Web 3.0 is just rubbish. There is real value in services that can produce info/knowledge/data in real time, and we certainly don't need a silly label for it.
<sarcasm>Maybe now live.com will become useful for something </sarcasm>