Quant versus Qual In The News Debate

A slow motion debate — slowed by people’s vacation schedules — has been taking place at Steve Buttry’s blog in the comments to a post entitled “Newspapers’ Original Sin”. Buttry presented an argument that newspapers have failed to innovate in business models and supporting technology, while others have suggested other rationales for the current woes in the industry. A greate debate has ensued involving Buttry and others, including Tim O’Brien and Chris O’Brien.

In particular, I’d like to pull out some paragraphs that Chris O’Brien tossed onto the table, there, regarding the way that different groups are perceiving the problem space:

[via Newspapers’ Original Sin: Not failing to charge but failing to innovate.]

In trying to think differently about how to deal with the ongoing news business crisis, over the past two years I’ve taken an approach that is intentionally anecdotal and subjective. I won’t even try to deliver the data that Tim seeks because I simply don’t believe that any amount of data is going to solve this industry’s problems. As I’ve worked on various newsroom reinvention and research projects over the past two years, I’ve come around to believe that the quantitative approach — putting our trust in massive reader surveys, polling data, whatever — has failed us.

Instead, I’m convinced that we need to take a qualitative approach. We need to take a fundamentally different approach to understanding the behavior, patterns and needs of our community when it comes to news and information. So if Tim needs the comfort of some cold, hard facts then I’ll just say straight up that I don’t have them and wouldn’t even try to get them. And even if I did, we’d probably still argue over what they really meant (as we are with Fine’s data).

When conducting research, weighing the quantitative versus the qualitative approach is hardly new or revolutionary. Having just recently spent a weekend at a major sociology conference in San Francisco, I can see how that academic field is split between those who spend time gathering large data sets to get at abstract truths and those who spend time observing and interviewing select subjects. Both approaches provided interesting insight. But more and more, I’ve been finding the qualitative approach has more value for me.

Why? Without listing every single study undertaken and tallying all the money spent, I think I can safely assert that over the past two decades, the news industry has spent millions of dollars accumulating data about readers and what they supposedly want. And our industry has responded by altering its products and newsrooms to produce the things that they thought the data told them that readers really wanted. Today, metro newspapers write shorter stories, with faster ledes, and publish more pictures about fluffier stuff. Our leaders have steadily used this data to make decisions that have made newspapers worse every year. Somehow, no one has stopped to consider that no industry has ever solved its problems by making its main product worse. Instead, management points to the data from readers’ survey to insist they’re doing what people say they want. The result is that we’re worse off than ever. [emphasis mine.]

My bet is that this difference in beliefs is what is effectively creating the two camps of this debate.

On one hand we have the advocates for a qualitative, user-centered approach who believe that what today’s and tomorrow’s newspaper reader wants is difficult to know, and figuring it out requires thinking about future newspapers like other product designs. Said differently, a qualitative perspective looks at the context in which the product is going to exist first, and attempts to characterize user needs in that context and how to satisfy them.

On the other hand, there are those that favor analysis of quantitative data about current users and their attitudes about current products with the goal of making changes in the product to meet the trend curves of the analysis. This is an incrementalist sort of design approach, and may never get around to big changes in the product, and doesn’t really seek to characterize users except as segments in markets. This is a product-centric view, in which users are viewed as a mass market.

You might say the user-centric is bottom-up and future-oriented thinking, less tied to the legacy model of the companies involved in providing the product and more attuned to deeply meeting the needs of a well-defined user.

The product-centric view is top-down and past-oriented thinking, in essence seeking to make the fewest changes while maximizing the size of the user community by providing a product that meets the common needs of a mass market.

As you can guess, I favor user-centric and future-oriented design in general and in this specific case, so I agree totally with Chris O’Brien, although he may not agree with my characterization of the distinction.

My belief is that the quant approach will show the past and some hypothetical near present based on the assumption of only incremental changes in the product and the market context in which it exists, but it can’t say much about user response to radically new products or a context in which radically new products might exist.