Skip to content

The Data Decade

January 13, 2010

vcmike

You heard it here first: the 20-10’s are going to be the decade of data.

The troves of data being generated by the web are increasingly being captured, structured, and harnessed to add insight, intelligence and value to all kinds of business and personal activity.

I’ve written about my excitement in the emerging data economy, at the time thinking primarily about web advertising. But I think over the next 10 years similar advances in the use of data to create business value are going to occur in a number of different verticals.

The underlying factor for this, of course, is the rising universality of the web itself, creating a trail of data behind human activity that has never before existed.  But the advancement of a number of other technologies allows that data to be captured, analyzed, and put to use in ways not possible just a few years ago, as my conversation with Brad Cross yesterday made clear to me.

The existence of the cloud allows entrepreneurs to handle compute-intensive tasks that scaled data storage and analysis require; open source software like Hadoop also makes possible data-intensive distributed applications; and dynamic programming languages like Clojure simplifies multithreaded programming.

Let’s check back in 2020 and see where we’ve come.

14 Comments

Post a comment
  1. January 13, 2010

    Completely agree that this will be the data decade as it’s becoming easier than ever to capture, save and use data to plan and take action. And, we think that data will become the core of a new interaction model between consumers, publishers and brands. As an example, here’s Gary Vaynerchuk’s CrushIt! book as an automatically enhanced data object: http://sprng.me/chn Gary has made it easy for anyone to capture, personalize, comment, share, use and buy this book.

    (Full disclosure, I’m a Springpad co-founder)

  2. January 13, 2010

    Hi Mike,

    I’m excited that other people see this trend.

    The outstanding question I have is whether we will start to see more research-driven startups.

    The current mode of operation seems to be that a startup like twitter experiences success, gathers a ton of data, then hires people to come do cool stuff with the data.

    This path seems to work well in cases where doing stuff with the data is an afterthought to achieving product market fit. However, I think we will start to see more companies where research is at the core of the business from the beginning, and research and product evolve together.

  3. January 13, 2010

    A bit of context has to be injected at source into a lot of the data that is currently being generated. I would argue that many of the analysis will fall flat on its face otherwise.

  4. January 13, 2010

    As a corollary to this I think what we’ll really see is the making sense of the droves of data that is already been captured. At the second half of the past decade the web did an incredible job capturing a ton of new data sources, but we have still very early on in making any sense of this.

    It’s the filters, aggregators, curators, and dashboards that are going to be the really winners this decade.

  5. January 13, 2010

    I agree with Sachin … it will not be just data analysis, but aggregation, filtering and presentation of data in new ways, we are only beginning to move toward a NEW DATA world. Should be intersting times ahead and glad to see others humming tunes from the same songbook.

  6. Ethan #
    January 13, 2010

    Completely agree. Thanks, Mike.

    So what did Brad Cross say?

  7. January 14, 2010

    I would very much like to see advances that help boutique brick & mortar businesses communicate their data. Many of the small shops out there are technology-blind, disconnected from the power of the net services that exist. Most small-shop owners can’t, for instance, run sales reports from home – they have to physically go to the location. As these small businesses are able to find solutions that are inexpensive and easy to use, we’ll see even more economic data feeding the beast… and I can’t wait.

    Of course, having facts is very different that gaining understanding.

  8. Mark Stoneham #
    January 14, 2010

    Agreed, Mike. The capturing, structuring and harnessing will continue and analysis engines will improve dramatically. And it will take a while but we’ll eventually see two ends meet up – people looking for stuff (search) and stuff looking for people (using data intelligence and user profiles) will converge and create something that is neither search nor targeting.

    But all the data is being produced out of the actions of individuals, and the individuals have very little control over who captures it, how it’s captured, and what is done with it. As there is quite clearly value in it, data ownership will become a prominent issue. Throw in some regulation and it will be a really interesting space.

  9. January 14, 2010

    Great post and spot on. The data decade is well underway. Check out the cool hack Joe Hughes (@joooe)is developing that converts open transit data into useful web services.

    http://headwayblog.com/2009/12/14/telling-time-with-openrealtime-data/

  10. January 14, 2010

    I think the data will be a huge focus as well, but personally I think we are entering the decade of attention…it’s not the information age anymore, it’s the attention age…IMHO the massive collection and analysis of data is really boiling down to getting, focusing, and sharing attention.

  11. January 14, 2010

    It’s hard to disagree with “data” as it’s hard to imagine new things using less of it, but you could say the same that it will be the decade of bandwidth, or the decade of CPUs (processing all that data).

  12. January 15, 2010

    I think the data will be a huge focus as well, but personally I think we are entering the decade of attention…it’s not the information age anymore, it’s the attention age…

  13. January 15, 2010

    COOL CAREER = DATA MINING

    Great post and I completely agree. If you look at the job postings, there are tons of openings for people with data mining skills. As the economist Hal Varian said in a Mckinsey article: the cool sexy job in the next ten years will be statisticians ( http://bit.ly/5GlNIG ).

Trackbacks & Pingbacks

  1. The Data Decade – The flipside of Platforms – A Frog in the Valley

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s