stoweboyd and the /messengers

In today’s online world, what your mother told you is true, only more so: people really can judge you by your friends.

Google Wave is not catching on, according to Hitwise (via Dan Frommer):

Google Wave has little traction since it doesn't do something well defined that people believe they need a tool for. It's an amorphous mess.

There is a place for an open alternative to email, but it would need to be much simpler than Wave.

Matthew Ingram recently chatted with Craig Newmark, the founder of Craigslist, and asked him what was the web's next big problem. His answer?

The question of who to trust online, according to Newmark. To solve it, he believes that what the web needs is a “distributed trust network” that allows us to manage our online relationships and reputations.

[...]

Newmark called some form of distributed trust system “the killingest of killer apps” for the web over the next decade (he said he wasn’t sure that was the best way to describe it, but was trying out to see how it sounded). He talked about “reputation and trust ruling the web, just the way it does in real life,” and how he was looking to big players such as Google, Facebook and Amazon as the kinds of entities that would have the scale to handle such a distributed trust or reputation management network. And he said that despite some occasional missteps by both Google and Facebook when it came to privacy (Google Buzz and Facebook Beacon, respectively), he believed that both were acting in good faith and had a policy of “not being evil.”

[...]

Newmark said that as a society we needed to “get our act together and make this happen,” adding with a wink that the idea for the distributed trust network was all part of his “hidden agenda to move ahead on the web to try and save the world.”

The Web may be the only hope that the world has, and if some new, distributed model of trust is going to be developed it will certainly have to be coming from the web.

How might such a system work? What would it be like?

A few thoughts: the open follower model (a la Twitter) is likely to be the dominant social motif of most web apps, at least for the next decade. It is a loose and distribuable model, and the work on interoperable activity streams suggests that open APIs across many different social tools could lead to a distributed sort of trust.

Imagine how it might work. Someone, Carla Botts, writes a piece about her Malibu. Others who read it could use whatever tools provided by the technology in which the link or post is delivered, like Tumblr or Google Reader. Someone at Chevrolet wants to know if Botts is a trusted source of information, and runs some application that gathers information from various systems -- Twitter, Google Reader, Facebook, Tumblr, and so on -- and calculates some trust metric.

Here's the hitch: while all the tools involved can run independently -- although in an interoperable way -- the algorithm for calculating trust should be completely open, or completely closed.

In the case of going completely open, the factors involved in trust metrics would be made public. This allows anyone anywhere to build trust measures into tools and services, but in such a way that the trust metric is dependable. Of course, people might try to 'game' an open trust system, just as people have gamed Digg and other 'democratic' voting based tools. Again, just as with Digg, a great deal of time and energy would have to be spent on discovering people fabricating trust. This approach is analogous to the Jabber XMPP protocol, which merely defines how cooperating servers talk to each other and authenticate identities.

The alternative is to create a closed trust metric, where the factors involved are concealed except in the most general sense. A server -- or a network of trust servers -- could constantly be updating trust metrics on individuals, in a way analogous to the way that search engines -- like Google -- are constantly recalculating page rank of indexed pages.

Both of these approaches have their merits, and their negatives. But I agree with Newmark that it would be useful -- and potentially radical -- to have a trusted trust framework that is not controlled by a market player -- like Google or Yahoo -- or any government. Newmark suggests there might be a role for government in such a system, but in a public-private setting, where checks and balances involved non-governmental groups.

Study Highlights:

Twitter & FaceBook:
- 48% of people check/update FaceBook and/or Twitter after they go to bed

- 18% of people under 25 years old can't go more than a couple hours without checking in on FaceBook

- 61% of people under 25 have to check in on FaceBook at least once a day

- 11% of people over 25 years old can't go more than a couple hours without checking in on FaceBook

- 55% of people over 25 have to check in on FaceBook at least once a day

- 16% of people under 25 years old rely on Twitter and/or FaceBook for the morning "news"

iPhone Users:
- 28% of iPhone users check/update Twitter before they get out of bed

- 26% of iPhone users check/update Twitter before they turn on their TV

- 23% of iPhone users rely on Twitter for their morning news

Electronic Messages:
- 11% of people under 25 years old can be interrupted by an electronic message during sex (The number drops to 6% of people over 25 years old)

- 24% of people under 25 can be interrupted by an electronic message while in the bathroom. (This number drops to 12% of people over age 25)

- 49% of people under 25 years old can be interrupted by an electronic message during a meal. (27% for people over 25 years of age)

- 22% of people under 25 years old can be interrupted by an electronic message during a meeting. (11% of people over 25 years of age)

So, the presumption is that we shouldn't do these things, and by doing them we are participating in a pattern of behavior that is immoral, immature, or at best questionable. This is, once again, the War On Flow, where various arbiters of public morality will point their fingers and call us names.

And here you can see how they lump together the examples which are titillating or disgusting, or supposedly unacceptable in the business context. What about the percentage of people that will respond to a text message while waiting in line at the Subway? That interruption is acceptable, but you are NOT supposed to take the message while eating your Subway?

There is an implicit etiquette going on here: one that is left unsaid. We are supposed to already know which of these contexts are 'correct' for responding to a text message, or how often and in what context we are supposed to be in to check Twitter. I guess it is verboten to check Twitter in bed, and especially while having sex.

I am completely opposed to this sanctimonious, pseudo-moralistic mumbo jumbo.

They will consistently devalue the actions that keep us connected -- taking the text message, responding to the tweets -- and they rate our behavior based on the context we are in when the message is responded to, but not the level of our connection with the person at the other end of the relationship. It's all form and no substance.

In a rambling grabbag of an article in the NY Times, Steve Lohr mentions in passing that Netflix has shelved a second contest to improve its recomendation capability based on privacy concerns:

How Privacy Vanishes Online

On Friday, Netflix said that it was shelving plans for a second contest — bowing to privacy concerns raised by the F.T.C. and a private litigant. In 2008, a pair of researchers at the University of Texas showed that the customer data released for that first contest, despite being stripped of names and other direct identifying information, could often be “de-anonymized” by statistically analyzing an individual’s distinctive pattern of movie ratings and recommendations.

The movie data that was supposedly 'anonymized' -- stripped of indications of the identity of the people involved -- but researchers were able to reconstruct identities buried in the 'micro-data' associated with them:

Robust De-anonymization of Large Datasets (How to Break Anonymity of the Netflix Prize Dataset), Arvind Narayanan and Vitaly Shmatikov, The University of Texas at Austin, February 5, 2008

We present a new class of statistical de-anonymization attacks against high-dimensional micro-data, such as individual preferences, recommendations, transaction records and so on. Our techniques are robust to perturbation in the data and tolerate some mistakes in the adversary’s background knowledge.

We apply our de-anonymization methodology to the Netflix Prize dataset, which contains anonymous movie ratings of 500,000 subscribers of Netflix, the world’s largest online movie rental service. We demonstrate that an adversary who knows only a little bit about an individual subscriber can easily identify this subscriber’s record in the dataset. Using the Internet Movie Database as the source of background knowledge, we successfully identified the Netflix records of known users, uncovering their apparent political preferences and other potentially sensitive information.

[...]

Datasets containing “micro-data,” that is, information about specific individuals, are increasingly becoming public—both in response to “open government” laws, and to support data mining research. Some datasets include legally protected information such as health histories; others contain individual preferences, pur- chases, and transactions, which many people may view as private or sensitive.

Privacy risks of publishing micro-data are well-known. Even if identifying information such as names, addresses, and Social Security numbers has been removed, the adversary can use contextual and back- ground knowledge, as well as cross-correlation with publicly available databases, to re-identify individual data records. Famous re-identification attacks include de-anonymization of a Massachusetts hospital dis- charge database by joining it with with a public voter database [22], de-anonymization of individual DNA sequences [19], and privacy breaches caused by (ostensibly anonymized) AOL search data [12].

Micro-data are characterized by high dimensionality and sparsity. Informally, micro-data records contain many attributes, each of which can be viewed as a dimension (an attribute can be thought of as a column in a database schema). Sparsity means that a pair of random records are located far apart in the multi-dimensional space defined by the attributes. This sparsity is empirically well-established [6, 4, 16] and related to the “fat tail” phenomenon: individual transaction and preference records tend to include statistically rare attributes.

[references are provided in the pdf.]

Netflix had become embroiled in a lawsuit brought by the contest, and bowing to pressure from the Federal Trade Commission the company decided to drop the second contest:

- Neil Hunt, Netflix Chief Product Officer, Netflix Prize Update

We have reached an understanding with the FTC and have settled the lawsuit with plaintiffs. The resolution to both matters involves certain parameters for how we use Netflix data in any future research programs.

In light of all this, we have decided to not pursue the Netflix Prize sequel that we announced on August 6, 2009.

We will continue to explore ways to collaborate with the research community and improve our recommendations system so we can constantly improve the movie recommendations we make for you. So stay tuned.

So, the upshot is that companies who gather data about us that is implicitly private -- like our movie viewing habits -- are probably not going to be able to publish this data in some hypothetically anonymized fashion.

On the other hand, if users opt into a publicy-based system -- where their movie viewing habits are shared and published openly -- it may seem that Netflix will have no problems. I doubt that Netflix needs all users to do so in order to improve the recommendation service.

But the researchers make a stronger case, saying that other hypothetically private information about users -- like sexual preferences and political orientation -- can be inferred from the datasets, not just determining the users' identities:

Does privacy of Netflix ratings matter? The privacy question is not “Does the average Netflix subscriber care about the privacy of his movie viewing history?,” but “Are there any Netflix subscribers whose privacy can be compromised by analyzing the Netflix Prize dataset?” The answer to the latter question is, undoubtedly, yes. As shown by our experiments with cross-correlating non-anonymous records from the Internet Movie Database with anonymized Netflix records (see below), it is possible to learn sensitive non-public information about a person’s political or even sexual preferences. We assert that even if the vast majority of Netflix subscribers did not care about the privacy of their movie ratings (which is not obvious by any means), our analysis would still indicate serious privacy issues with the Netflix Prize dataset.

Moreover, the linkage between an individual and her movie viewing history has implications for her future privacy. In network security, “forward secrecy” is important: even if the attacker manages to compro- mise a session key, this should not help him much in compromising the keys of future sessions. Similarly, one may state the “forward privacy” property: if someone’s privacy is breached (e.g., her anonymous online records have been linked to her real identity), future privacy breaches should not become easier. Now consider a Netflix subscriber Alice whose entire movie viewing history has been revealed. Even if in the future Alice creates a brand-new virtual identity (call her Ecila), Ecila will never be able to disclose any non-trivial information about the movies that she had rated within Netflix because any such information can be traced back to her real identity via the Netflix Prize dataset. In general, once any piece of data has been linked to a person’s real identity, any association between this data and a virtual identity breaks anonymity of the latter.

It also appears that Netflix might be in violation of its own stated privacy policy. According to this policy, “Personal information means information that can be used to identify and contact you, specifically your name, postal delivery address, e-mail address, payment method (e.g., credit card or debit card) and telephone number, as well as other information when such information is combined with your personal information. [...] We also provide analyses of our users in the aggregate to prospective partners, advertisers and other third parties. We may also disclose and otherwise use, on an anonymous basis, movie ratings, commentary, reviews and other non-personal information about customers.” The simple-minded division of information into personal and non-personal is a false dichotomy.

If someone wants to analyze the correlation between my movie choices and my political leanings, go ahead. But, of course, I live and watch movies in the US and not some repressive country that would jail me for enjoying 'Breaking Away' or 'Breakfast Club', and I live a very public life.

Narayanan and Shmatikov have conclusively demonstrated that there cannot be a separation between personal and non-personal: algorithms like theirs make this intuitive distinction meaningless. So any software solution -- like Netflix -- that has reserved the right to share, distribute or publish anonymized data based on large populations of users will likely be blocked in any attempt to actually use that data in that way. Even if the sharing is not done in a fully open way -- as Netflix did when it opened its dataset up for the contest -- there is the distinct possibility that senstive inferences can be associated with specific identities in the user base, and this will peirce the veils of privacy and secrecy.

Users have granted the right to Netflix to manage information about their viewing habits, and to use it in specific ways to make recommendations. But if it came to light that a part of their internal algorithm inferred explicitly what users' sexual preferences are, for example, and stored that data somewhere -- even if only for the length of a session -- wouldn't that be problematic, too? The possibility that such information exists would lead to the potential of all sorts of troublesome identity and privacy issues.

The whole Four Hour Work Week is obvious bullshit, but I hadn't realized what a jerk Ferriss is. Penelope Trunk has witnessed this all first hand: the way he slimed his way into having coffee with her, his annoying email 'etiquette', his worker bees spamming blogs with comments that link back to his book's site, etc.

She ends her post, 5 Time management tricks I learned from years of hating Tim Ferriss, this way: 

The idea of time management only matters in relation to how important the stuff is that's competing for your time. The stuff that makes time management the most difficult is relationships. Which Tim does not excel in.

Fine. Not everyone has to be good at making real connections.

But Tim runs around telling people who have lots of relationships competing for their time how to think about work/not work, forgetting that in the real world, where people are not assholes, time management is not an equation or a semantic game because relationships really matter. And figuring out how to judge time in terms of competing values is the hardest thing of all.

Tim is all about time management for achievement and winning. But there are not trophies or measurements for relationships. There is only that feeling that someone is kind. And good. And truly connected.

And Tim is not.

Robin Chase is the founder and former CEO of Zipcar, thinks that the strnaglehold that car manufacturers have on car data makes no sense:

Driving By the Numbers

Current federal law requires annual emissions and safety inspections for all cars. A mechanic plugs an electronic reader into what’s known as the onboard diagnostic unit, a computer that sits under your dashboard, monitoring data on acceleration, emissions, fuel levels and engine problems. The mechanic can then download the data to his own computer and analyze it.

Because carmakers believe such diagnostic data to be their property, much of it is accessible only by the manufacturer and authorized dealers and their mechanics. And even then, only a small amount of the data is available — most cars’ computers don’t store data, they only monitor it. Though newer Toyotas have data recorders that gather information in the moments before an air bag is deployed, the carmaker has been frustratingly vague about what kind of data is collected (other manufacturers have been more forthcoming).

But what if a car’s entire data stream was made available to drivers in real time? You could use, for instance, a hypothetical “analyze-my-drive” application for your smart phone to tell you when it was time to change the oil or why your “check engine” light was on. The application could tell you how many miles you were getting to the gallon, and how much yesterday’s commute cost you in time, fuel and emissions. It could even tell you, say, that your spouse’s trips to the grocery store were 20 percent more fuel-efficient than yours.

Carmakers could collect the data, too. Aberrant engine and driving behavior would leap out of the carmakers’ now-large data set, allowing them, if necessary, to conduct recalls much earlier. And, in exchange for your contribution of anonymous data, carmakers could send you driving benchmarks aggregated from your peers; then your app could tell you how your driving compares with the average of all drivers of the same car.

Having such readily accessible data streaming from your car might raise fears of a Big Brother scenario, in which carmakers would know where you are and how you are using (or misusing) your vehicle. But you would still decide whether you wanted to tap into the data, how you would use it and with whom you’d share it.

Allowing drivers and carmakers access to real-time performance data wouldn’t prevent every future mechanical failure. But it would allow carmakers and entrepreneurs to develop analytical tools to help catch developing problems in both individual cars and entire model lines. Cars would continue to break down and even cause accidents, but it wouldn’t take a Congressional hearing to figure out why.

Every car could have something like a Twitter account with a history of all of it's actions and stats. This is the Bruce Sterline notion of a spime: dumb devices stream their internal state changes into the cosmic data stream.

Obviously this would be of benefit to all of us, and -- by the way -- would obviate the need for 'black boxes' in cars for after the fact examination of collisions or other bad driving. Of course, the publicy/privacy duynamic is deeply embedded in a context like this. People's geographic location in their cars would need to shared in safe ways, but cars levels of emissions are supposedly regulated. It would be better to send a message to someone whose car is belching out illegal levels of smog than to find out at next year's emission test, obviously. And people who drive dangerously, by racing through red lights or speeding, have no right to privacy in such cases. That is why we install video camera/speed sensors at dangerous intersections.

William James once wrote, "You can judge a man's intelligence by how well he agrees with you." In that regard, David Gelernter is my main man. In a recent but oddly titled piece at Edge.org (what a nice name!), Gelernter could have been reading one of my recent slide shows:

- David Gelernter, Time To Start Taking The Internet Seriously

 

13. The traditional web site is static, but the Internet specializes in flowing, changing information. The "velocity of information" is important — not just the facts but their rate and direction of flow. Today's typical website is like a stained glass window, many small panels leaded together. There is no good way to change stained glass, and no one expects it to change. So it's not surprising that the Internet is now being overtaken by a different kind of cyberstructure.

14. The structure called a cyberstream or lifestream is better suited to the Internet than a conventional website because it shows information-in-motion, a rushing flow of fresh information instead of a stagnant pool.

15. Every month, more and more information surges through the Cybersphere in lifestreams — some called blogs, "feeds," "activity streams," "event streams," Twitter streams. All these streams are specialized examples of the cyberstructure we called a lifestream in the mid-1990s: a stream made of all sorts of digital documents, arranged by time of creation or arrival, changing in realtime; a stream you can focus and thus turn into a different stream; a stream with a past, present and future. The future flows through the present into the past at the speed of time.

16. Your own information — all your communications, documents, photos, videos — including "cross network" information — phone calls, voice messages, text messages — will be stored in a lifestream in the Cloud.

17. There is no clear way to blend two standard websites together, but it's obvious how to blend two streams. You simply shuffle them together like two decks of cards, maintaining time-order — putting the earlier document first. Blending is important because we must be able to add and subtract in the Cybersphere. We add streams together by blending them. Because it's easy to blend any group of streams, it's easy to integrate stream-structured sites so we can treat the group as a unit, not as many separate points of activity; and integration is important to solving the information overload problem. We subtract streams by searching or focusing. Searching a stream for "snow" means that I subtract every stream-element that doesn't deal with snow. Subtracting the "not snow" stream from the mainstream yields a "snow" stream. Blending streams and searching them are the addition and subtraction of the new Cybersphere.

18. Nearly all flowing, changing information on the Internet will move through streams. You will be able to gather and blend together all the streams that interest you. Streams of world news or news about your friends, streams that describe prices or auctions or new findings in any field, or traffic, weather, markets — they will all be gathered and blended into one stream. Then your own personal lifestream will be added. The result is your mainstream: different from all others; a fast-moving river of all the digital information you care about.

19. You can turn a knob and slow down your mainstream: less-important stream-elements will flow past invisibly and won't distract you, but will remain in the stream and appear when you search for them. You can rewind your lifestream and review the past. If an important-looking document or message sails past and you have no time to deal with it now, you can copy the document or message into the future (copy it to "this evening at 10," say); when the future arrives, the document appears again. You can turn a different knob to make your fast-flowing stream spread out into several slower streams, if you have space enough on your screen to watch them all. And you can gather those separate streams back together whenever you like.

20. Sometimes you will want to listen to your stream instead of watching it (perhaps while you're driving, or sitting through a boring meeting or lecture). Software will read text aloud, and eventually will describe pictures too. When you watch your high-definition TV, you might let the stream trickle down one side of the screen, so you can stay in touch with your life.

21. It's simple for the software that runs your Lifestream to learn about your habits; simple to figure out which emails (for example), or social updates, or news stories, you are likely to find important and interesting. It will therefore be easy for software to highlight the stream elements you're apt to find important, and let the others rush by quickly without drawing your attention.

22. Lifestreams will make it even easier than it is today for software to learn the details of your life and predict your future actions. The potential damage to privacy is too large and important a problem to discuss here. Briefly, the question is whether the crushing blows to privacy from many sources over the last few decades will make us crumple and surrender, or fight harder to protect what remains.

23. The Internet's future is not Web 2.0 or 200.0 but the post-Web, where time instead of space is the organizing principle — instead of many stained-glass windows, instead of information laid out in space, like vegetables at a market — the Net will be many streams of information flowing through time. The Cybersphere as a whole equals every stream in the Internet blended together: the whole world telling its own story. (But the world's own story is full of private information — and so, unfortunately, no human being is allowed to hear it.)

[emphasis mine]

Time is the new space.

I call it the Post-Everything Economy istead of the post-Web, but same same, all same.

How have I never met this guy?