Big Data and Data Inequality: Research Is Just The Beginning

There was a recent hoo-ha at a scientific conference in France, when Bernardo Huberman was furious when researchers from Google and a contributing university presenting results of social data analysis declined to share the data.

John Markoff, Big Data Troves Stay Forbidden to Social Scientists via  NYTimes.com

The issue came to a boil last month at a scientific conference in Lyon, France, when three scientists from Google and the University of Cambridge declined to release data they had compiled for a paper on the popularity of YouTube videos in different countries.

The chairman of the conference panel — Bernardo A. Huberman, a physicist who directs the social computing group at HP Labs here — responded angrily. In the future, he said, the conference should not accept papers from authors who did not make their data public. He was greeted by applause from the audience.

In February, Dr. Huberman had published a letter in the journal Nature warning that privately held data was threatening the very basis of scientific research. “If another set of data does not validate results obtained with private data,” he asked, “how do we know if it is because they are not universal or the authors made a mistake?”

He added that corporate control of data could give preferential access to an elite group of scientists at the largest corporations. “If this trend continues,” he wrote, “we’ll see a small group of scientists with access to private data repositories enjoy an unfair amount of attention in the community at the expense of equally talented researchers whose only flaw is the lack of right ‘connections’ to private data.”

Facebook and Microsoft declined to comment on the issue. Hal Varian, Google’s chief economist, said he sympathized with the idea of open data but added that the privacy issues were significant.

“This is one of the reasons the general pattern at Google is to try to release data to everyone or no one,” he said. “I have been working to get companies to release more data about their industries. The idea is that you can provide proprietary data aggregated in a way that poses no threats to privacy.”

The debate will only intensify as large companies with deep pockets do more research about their users. “In the Internet era,” said Andreas Weigend, a physicist and former chief scientist at Amazon, “research has moved out of the universities to the Googles, Amazons and Facebooks of the world.”

And of course, big data is worth big money — leaving aside the privacy concerns — and controlling access to that data is central to the aspirations of companies like Google, Facebook, and others.

Research is just the first place where the latent data inequality of the post normal world will come to light. We will each of us — as individuals — be divided from the inherent value of information about our activities and the inferences that can be made about them. As a society, we will find corporations that do not have our interests at heart working to exploit the potential value of our aggregated data exhaust. We are an exploitable resource — like the oceans of fish or the oil beneath the ground — and these companies plan to harvest all the value without our involvement.

We will find that we don’t own the information about ourselves anymore than we own our DNA. (Yes, others can patent your genes: see The Tissue-Industrial Complex.)

F.C.C. Weighs Treating Video Sites Like Cable Companies - Brian Stelter via NYTimes.com

The FCC is likely to let the genii out of th bottle, and redefine who is a Multichannel Video Programming Distributor, or MVPDs, now effectively limited to the linear TV players like Comcast and DirecTV. If the rules are changed to include streaming video services like Hulu and YouTube, the landscape of TV will never be the same:

Brian Stelter via NYTimes.com

Major distributors like Comcast and Time Warner Cable want the definition of M.V.P.D. to remain rather narrow, to include only those who provide the transmission path for programming, like themselves.

Some broadcasters, however, want the definition to be broadened to include online video sites, because then the sites would be subject to the same rules as cable operators, called retransmission consent, and would have to pay fees for their station signals. A number of online TV start-ups, including the Barry Diller-backed Aereo, are trying to sidestep these rules.

Jack Perry of Syncbak, which helps stations simulcast their signals on the Web, said his company would be able to grow more rapidly if the F.C.C. adopted a “21st-century definition of M.V.P.D.’s.”

“The impact could be huge,” he said. Still other stakeholders, including trade groups that represent giants like Google, Microsoft, Amazon and Netflix, have said that the F.C.C. should take more time before deciding.

Yeah, some large players want to avoid paying fees for rebroadcasting, and to possibly limit the entrance of new start-ups.

And the cable and satellite operators want to freeze time, and delay the inevitable, which will turn those companies’ product into a single commodity: basically bringing the Internet to our homes, through which we will be able to access whatever streaming content we want from whatever sources we want: ‘over the top’ TV. Comcast and Time Warner Cable do not want to be competing directly with Apple, Amazon, and Google, but it is in the best interest of the average person is the FCC allows this change to happen.

The Social Operating System: A Reader

For the sake of my pal Valdis Krebs, I am collating a list of posts I’ve made in recent years on the idea of a social operating system. The basic notion:

Stowe Boyd, Rockmelt: Why The Social Browser Won’t Matter

The next generation of operating systems will be social at the core.We won’t be fooling with files and folders. We will be connecting with others, reading streams from our friends, and tossing observations and hopes and insights into the wake we leave behind, spreading out to all that think we matter.

So here’s some links to pieces I’ve written mentioning the idea:

Please send along any references to other people writing on the subject.

Google+ Is A Ghost Town

Google continues to say that others’ analysis of public use of Google+ is a distortion of actual use — since much of the use of G+ is supposedly private — but Google will not share the data:

Exclusive: New Google Study Reveals Minimal Social Activity, Weak User Engagement - Austin Carr via Fast Company

This week, the data analytics firm [RJ Metrics] provided Fast Company with exclusive new insights on Google+. The findings paint a very poor picture of the search giant’s social network—a picture of waning interest, weak user engagement, and minimal social activity. Google calls the study flawed—we’ll explain why in a second—and has boasted that more than 170 million people have “upgraded” to the network. RJ Metrics’ report, on the other hand, is yet another indicator that Google+ might indeed just be a “virtual ghost town,” as some have argued.

Let’s start with the findings. For its study, RJ Metrics (RJM) selected a sample of 40,000 random Google+ users. RJM then downloaded and analyzed every sample users’ public timeline, which contains all publicly available activity. One important caveat: RJM was only able to look at public data, which as it points out, “is not necessarily reflective of the entire population of users,” since some users are private or at least have private activity. That said, the stats are eye-opening:

  • According to RJM’s report, the average post on Google+ has less than one +1, less than one reply, and less than one re-share
  • Roughly 30% of users who make a public post never make a second one
  • Even after making five public posts, there is a 15% chance that a user will not post publicly again
  • Among users who make publicly viewable posts, there is an average of 12 days between each post
  • After a member makes a public post, the average number of public posts they make in each subsequent month declines steadily, a trend that is not improving
[…]

Part of the reason there have been so many reports on the so-called Google+ “ghost town” is because Google has refused to provide clear figures and metrics for its social network’s active user base. The company has said there are 170 million people who have “upgraded” to Google+, which is just a confusing way to say that 170 million people have signed up for the service (which takes about a click or two if you are already a Gmail user).

The company has been asked repeatedly for monthly active users, and it’s repeatedly denied such requests, essentially calling them irrelevant. The closest we’ve seen of active usership was when the company explained how many Google+ users were engaging with Google Plus-enhanced or -related products. The problem is that Google Plus-enhanced products include YouTube and Google.com, meaning if you are engaging with basically any Google property (there are 120 Google+ integrations thus far) while signed up with Google+, Google is basically counting this as engagement with Google+, which is incredibly misleading, as some have argued

Google has continuously fudged its numbers and dodged specifics around Google+, as search guru Danny Sullivan has recorded in his brilliant rundown of Google’s lack of transparency on the subject. To confuse things all the more, Larry Page recently said in an earnings call that “there are 2 parts to the Google+ experience: the part that is the social spine, and the other part that’s the social destination part of Google+ exclusively. Both of these are growing fast, but the social destination part of Google+ is growing as a new product with very healthy growth.”

There’s a simple way to solve this problem: Just provide the number of active monthly users on Google+ (proper). Facebook does it. Google even does it with YouTube, which, as Larry Page boasted recently, has 800 million monthly users. But when I made a request for such figures, Google did not provide them.

They won’t provide them. It’s a Ponzi scheme: Google is hoping that somehow, someway Google+ will catch on and that they can use that success to render the horrible reality of today irrelevant. However, it is likely to fall to pieces.

Is Page concerned that this latest gamble in the social marketplace might be his last? After Wave, is Google+ his last chance?

Is Bezos Crazy Like a Fox, Or Just Crazy?

Nobody Seems to Understand What Jeff Bezos is Doing. Does He? - Farhad Manjoo via PandoDaily

Jeff Bezos once famously declared that, in the service of innovation and its long-term success, Amazon is “willing to be misunderstood for long periods of time.” He was being a bit modest there; Amazon is not merely “willing” to be misunderstood, it often tries to actively sow widespread misunderstanding. This works it its advantage; if competitors don’t know what Amazon is up to, if they can’t even figure out where and how it aims to make money, they’ll have a harder time beating it.

But all this misunderstanding can’t be an unalloyed good. Amazon is so opaque, with so many mysterious businesses and revenue streams, that you’ve got to wonder whether the people who work there even understand what it’s up to. In business, simplicity often wins. Selling me a device to get me to buy a membership in order to get a book for free. Is Bezos crazy like a fox? Or is he just plain crazy? We have no idea.

But Bezos is involved in a land grab: he wants people to use Kindle and buy books from Amazon long enough to become a default standard. If he has to extract value from the publishers and authors of books to do so, he will.

Bezos is looking over his should at Apple (and more distantly at Google) who are developing the most dominant mobile devices on the planet, and he knows it is all converging. People — given their druthers — would rather have a single mobile device to do everything: read books, surf the web, write email, blog, social network (yes, I am using that as a verb).

So the only question is, why doesn’t he put a phone on the Kindle? It’s already a (bad) browsing device with an embedded whispernet data connection, so perhaps he is planning to give away phone service to Amazon Prime subscribers, too.

The Next Big Thing Is Eating The Lunch Of Something That Was Big A Decade Ago

Someone who hasn’t fallen for George Orwell’s trope ‘whoever is winning now will always seem to be invincible.’

Here’s Why Google and Facebook Might Completely Disappear in the Next 5 Years - Eric Jackson via Forbes

In the tech Internet world, we’ve really had 3 generations:

  1. Web 1.0 (companies founded from 1994 – 2001, including Netscape, Yahoo! (YHOO), AOL (AOL), Google (GOOG), Amazon (AMZN) and eBay (EBAY)),
  2. Web 2.0 or Social (companies founded from 2002 – 2009, including Facebook (FB), LinkedIn (LNKD), and Groupon (GRPN)),
  3. and now Mobile (from 2010 – present, including Instagram).
We will never have Web 3.0, because the Web’s dead.

With each succeeding generation in tech the Internet, it seems the prior generation can’t quite wrap its head around the subtle changes that the next generation brings.  Web 1.0 companies did a great job of aggregating data and presenting it in an easy to digest portal fashion.  Google did a good job organizing the chaos of the Web better than AltaVista, Excite, Lycos and all the other search engines that preceded it.  Amazon did a great job of centralizing the chaos of e-commerce shopping and putting all you needed in one place.

When Web 2.0 companies began to emerge, they seemed to gravitate to the importance of social connections.   MySpace built a network of people with a passion for music initially.  Facebook got college students.  LinkedIn got the white collar professionals.  Digg, Reddit, and StumbleUpon showed how users could generate content themselves and make the overall community more valuable.

Yet, Web 1.0 companies never really seemed to be able to grasp the importance of building a social community and tapping into the backgrounds of those users.  Even when it seems painfully obvious to everyone, there just doesn’t seem to be the capacity of these older companies to shift to a new paradigm.  Why has Amazon done so little in social?  And Google?  Even as they pour billions at the problem, their primary business model which made them successful in the first place seems to override their expansion into some new way of thinking.

Social companies born since 2010 have a very different view of the world.  These companies – and Instagram is the most topical example at the moment – view the mobile smartphone as the primary (and oftentimes exclusive) platform for their application.  They don’t even think of launching via a web site.  They assume, over time, people will use their mobile applications almost entirely instead of websites.

We will never have Web 3.0, because the Web’s dead.

Web 1.0 and 2.0 companies still seem unsure how to adapt to this new paradigm.  Facebook is the triumphant winner of social companies.  It will go public in a few weeks and probably hit $140 billion in market capitalization.  Yet, it loses money in mobile and has rather simple iPhone and iPad versions of its desktop experience.  It is just trying to figure out how to make money on the web – as it only had $3.7 billion in revenues in 2011 and its revenues actually decelerated in Q1 of this year relative to Q4 of last year.  It has no idea how it will make money in mobile.

The failed history of Web 1.0 companies adapting to the world of social suggests that Facebook will be as woeful at adapting to social mobile as Google has been with its “ghost town” Google+ initiative last year.

The organizational ecologists talked about the “liability of obsolescence” which is a growing mismatch between an organization’s inherent product strategy and its operating environment over time.  This probably is a good explanation for what we’re seeing in the tech world today.

Are companies like Google, Amazon, and Yahoo! obsolete?  They’re still growing.  They still have enormous audiences.  They also have very talented managers.

But with each new paradigm shift (first to social, now to mobile, and next to whatever else), the older generations get increasingly out of touch and likely closer to their significant decline.  What’s more, the tech world in which we live in seems to be speeding up.

People forget how indomitable AOL seemed, and the promise of Netscape and MySpace, before they fell into the dustbin. As I have said before, Facebook is the new AOL, although Johnson is making a different case for that. I have been presaging the rise of social operating systems — which would invalidate Facebook’s near-monopoly on people’s social inclinations — while he points to the rise of mobile, and says

Considering how long Facebook dragged its feet to get into mobile in the first place, the data suggests they will be exactly as slow to change as Google was to social.

And that’s is not a good place to be.

I agree with Jackson: the rate of change is not slowing, so the monopolies of today are likely to be shorter-lived than those of even a decade ago. And the new world beaters are possibly companies that don’t even exist yet, but whenever they crop up we will first notice them when they start stealing users, market, and attention from the formerly indomitable killer apps of the preceding era.

Google and Facebook Grow Comfortable and Complacent - Nick Bilton via NYTimes.com

Nick Bilton thinks Facebook and Google are slow to get mobile — several meanings of ‘get’ intended — because the engineers and managers there are relatively sessile (go look it up):

Nick Bilton via NYTimes.com

I have a theory on why they both have been slow to capitalize on the shift to mobile.

It’s that working at these companies is like going to work on an all-inclusive cruise ship. The analogy is apt in terms of the luxury — and the isolation.

An employee’s day often begins with a comfy shuttle bus whisking him or her to work in Silicon Valley. The buses have Wi-Fi, so laptops are put to work before anyone arrives on the sprawling campuses.

Once there, dozens of free breakfast options await. Free buffet lunches break the monotony of the day. There is free dinner, too. There are free snacks for those peckish between meals. (The stuff that’s bad for you is on the hard-to-reach lower shelves.)

All of this is wonderful for the employees — and of course well deserved — but these perks could be stultifying. At some of these Silicon Valley businesses, there is no reason to leave the office.

There are on-campus gyms. Day care. Massages. Dry cleaning. Car rentals. (At the Google offices, some of the toilets even have heated seats.)

Sadly, this isn’t how the rest of the world works.

Most people actually have to leave their offices to get coffee. While wandering out into the real world, we unfortunates tend to do a lot with our mobile phones.

We look for new restaurants, check in with location-based apps, share short pithy updates about things we’ve seen in this outside world, and take pictures of food and sunsets.

I’m betting that the Googlers and Facebookers don’t see as much outside, since all these perks are meant to keep people working as long as possible.

Perhaps there is something even more powerful at work, here: the self-centered, self-important mindset that is engendered in these world-beater companies tends to encourage a strong tie to the period of time when the companies became successful, which is three to five years ago. These companies — like Microsoft and Yahoo before them — became mired in the past, like mammoths and saber-tooth tigers sinking in the La Brea tar pits.

Payback With Square

parislemon:

It’s bad enough that Google Wallet co-founding engineer Rob von Behren left the Google Wallet team (following several others out the door). It’s worse that he left Google. But subsequently joining Square is the ultimate kick in the nuts.

As is von Behren’s entire comment to NFC Times:

When I left the Google Wallet project in January, I fully expected to stop working in payments but to remain at Google. After meeting the team at Square, however, I decided to do the opposite. Square is doing some great things in the payment space. They have a strong leadership team and a culture that fosters innovation.

The unstated implication there is that Google is not doing some great things in the payment space. Nor do they have a strong leadership team. Nor do they have a culture that fosters innovation.

This just keeps getting worse and worse for Google.

Google+ solves Google’s big problems, at least in theory. It delivers a social network—arguably better constructed Facebook—that lets it understand the connections between people. It also lets Google tap into a stream of real-time data, and build a search system around that without having to worry that it will ever be left at the altar. And it does so much more, too! It has real time photos, like Instagram. It has a video chat service, like Skype. It lets you see which businesses your friends recommend, like Yelp. It’s a one size fits all solution, and what’s more it’s on the open Web. Perfect!

One problem: People don’t really want to use it. They’re already entrenched in other stuff. Many of Google’s recent actions can be explained by understanding that dilemma. Google wants to know things about you that you aren’t already telling it so you will continue asking it questions and it can continue serving ads against the questions you ask it. So, it feels like it has to herd people into using Google+ whether they want to go there or not.

This explains why Google has been driving privacy advocates crazy and polluting its search results. It explains why now, on the Google homepage, there’s a big ugly black bar across the top that reminds you of all its properties. It explains the glaring red box with the meaningless numbers that so desperately begs you to come see what’s happening in its anti-social network. It explains why Google is being a bully. It explains why Google broke search: Because to remain relevant it has to give real-world answers.

Google has to get you under its tent, and break down all the silos between its individual products once you’re there. It needs you to reveal your location, your friends, your history, your desires, your finances; nothing short of your essence. And it needs to combine all that knowledge together. That’s Search Plus Your World. “Your World” is not just your friends, or your location. It’s your everything. The breadth of information Google wants to collect and collate is the stuff of goosebumps.

- Mat Honan,  The Case Against Google

Google would like to be the next great thing, the bridge into the future, but instead they are failing to deliver breakthrough technology, despite all their edvantages and huge cash reserves. And realizing it, they have become so desperate, and cavalier about a world dominated by user experience that they will wind up chasing us away.

Google is the new Microsoft.