On the value (or not) of replication
Where Communication needs to go when studying the future of journalism and democracy...(or reflections on Week 2 of my Econometrics immersion)
So as many of you know, part of the whole motivating force of this Mellon grant for me is the underlying dissatisfaction I often have in response to articles by people outside the field of Communication writing on the latest moral panic associated with Communication (e.g. disinformation, right-wing radicalization “caused” by the interwebs, social media “addiction,” screen time, violent video games, etc.). In this missive, I want to back up a little bit and explain more about what’s driving my time in Barcelona at the Barcelona School of Economics. I will then offer some thoughts about Week 2’s dive into cross-sectional analysis and the excellent instruction from Albrecht Glitz in a later post.
For anyone new to this little Substack, the short story is I am a mid-career Communication scholar who started my career as an ethnographer, have written a few books and many, many journal articles, and have increasingly morphed into using the insights gained from qualitative data to push back on power through more replicable quantitative approaches, although until now, this has always been done with help from friends.
Among my worst frustrations since starting my PhD in 2006 is having to defer to economists who take simple premises, analyze (generally) already existing data, and then get all the credit for knowing everything about fields they lack any subject specific knowledge about. My advisor, Larry Gross, gave me a warning about drinking econometrics kool aid, and he put it so well:
Economists specialize in making predictions that don't turn out correct and then explaining how they now know why their predictions were wrong, and then proceed to make more predictions that will be wrong. Simply put, their models can't account for the complexity and unpredictability of key variables; including but not limited to, um, human behavior.
Larry, a social psychologist at his core, goes on to critique behavioral economists for swallowing “the methods/perspectives of experimental social psych and end up finding "effects" in artificial/trivial situations that fail to predict behavior in real/important situations…” — and in truth, I couldn’t agree more, though I do think that Kahneman would argue that fight or flight risk assessments are super important when flying a fighter jet (I would 100% recommend Michael Lewis’ Undoing Project for an example of excellent intellectual history on behavioral economics).
Yes, I get that many in our field believe it has no center, but talk to a sociologist or anthropologist or even religious studies scholar and you will hear a similar complaint - trying to study people, their beliefs and motivations, and the impact of the external world on shaping how they think and act is HARD work - and you have to put a stake in the ground from which to proceed - which absolutely may not be aligned with others who share your disciplinary association, even. Silvio Waisbord, a Communication scholar trained as a sociologist, actually wrote a book called Communication: A Post-Discipline that really thinks about using this chaotic, diasporic lens as a strength and departure for inquiry rather than a weakness.
Look, I’m so darn sick of writing sentences like this one (from article forthcoming in the International Journal of Press Politics w/ Sanghoon Kim-Leffingwell):
I find this a bit ironic that economics has done more of late to look beyond voting or political polarization to consider other signs that our current crisis of newspaper journalism is problematic for democracy. (*Edit: just the day after I publish this, a new NBER paper on how much individual welfare goes down when a newspaper closes, which is inefficient because it’s more than the price of a newspaper subscription? REALLY?*) Maybe it’s not though, given that economist Anthony Downs in his 1957 “An Economic Theory of Democracy” was one of the first social scientists to really think about using information as a choice, with not everyone having the same needs for that information or the same ability to access the information they might need. Why wasn’t this from Communication? Arguably, we weren’t really an established academic discipline in 1957, but I don’t know if I totally buy this - as the first PhD programs were in place, and mass comm research had enjoyed quite the federally-funded WWII boost.
To wit, though: why isn’t there a plethora of good studies from Communication that shows a very clear, recent connection between declines in journalism and some impact on corruption in today’s super tech tech universe? At the broadest level, political economy of media books help lay out how power, money, and politics have undermined what might be an ideal form of civic information, spotlighting historical choices that have had significant downstream consequences (For this, see sociologist Paul Starr’s The Creation of the Media, and from our own field, most recently, Victor Pickard’s Democracy Without Journalism).
Is American democracy increasingly corrupt because we have lost news? Or how much does this loss of news impact larger trends that result in what is presumably (we don’t know) less government accountability? Otherwise this is just whiggish John Dewey three-cheers for democracy rhetoric. I don’t think my forthcoming paper (draft SSRC version here) should be so lonely in this department from our field.
The best work we have by and large comes from an economist who has decamped to our field, Stanford’s Jay Hamilton, who has twice produced essential, blockbuster findings: in his first book on media, All the News that’s Fit to Sell (OA link!), he actually models how economics can help predict news production, and reminds us that news is an information good that doesn’t work like other goods. In his second book on media, Democracy’s Detectives, he actually lays out the economic case for investigative journalism as being an overall net good benefit for American society, with the cost of the journalism delivering tremendous returns on investment even if thinking about economic framings such as efficient use of government resources. I would be remiss not to mention work by Phil Napoli and James Webster, who have focused more on the costs of the attention marketplace for news and information, but I’m thinking: who (and what) gives us the best evidence and arguments for the claim that the decline in journalism is undermining American democracy and uses something besides voting as the basis for determining this cost?
What other big areas of interest are we missing that economics/economic modeling could help provide insight on but are unlikely to do so?
Willingness to Pay: We use the phrase business model generally to mean revenue stream, but more so, it’s shocking to me just how few articles exist on the willingness to pay for news that make the most of the rich survey data that Pew and other organizations generate consistently, year-over-year, about how Americans view and use news and their communication behaviors and choices. For a good overview of the state of the literature in Journalism Studies, specifically, see this 2020 piece by O’Brien, Wellbrock, and Kleer.
As far as I know, there is not a single multi-nominal modeling of WTP from our field that looks at the subscription funnel (I am going to do this, using the paper/model that made the case for San Francisco’s BART as a framework). I believe this work is happening inside newsrooms like the NYT, which has Columbia and NYU academics on speed dial and sometime on payroll. But other than surveys, we don’t have anything to PREDICT or model future outcomes.
Assessing potential/current interventions in media ecologies
Funders really want to know whether a particular donation or investment or nonprofit outlet has made some sort of impact. Impact is hard to measure - is it articles? Is it awards? Is it awareness? Is it some sort of discernible policy change or policy debate? How do we know if what we did worked?
Josh Stearns lays out the potential for how journalists can be more transparent about they way they work. This is a research agenda that we do not know many empirical answers- really any… Do we know if increasing transparency makes a difference to news consumers? If so, what kind of difference? Trust, in particular, seems to be a tricky measure. (We have a few studies on this, especially from Talia Stroud’s lab, but not enough, and not with nearly enough external validity - and from a pure play academic standpoint, my anecdotal observation is that this isn’t getting the attention in the top-tier field wide journals that it should).
How much journalism do we actually need? How many journalists? Did we have too much journalism to begin with? Not enough? What is the break point for communities dissolving into puddles of infighting and limited knowledge about civically and culturally important community events/concerns? When we think about boots on the ground for a project like Report for America, how many more boots tips the scale? We are not so much bowling alone as we are voting against the bowling institutions, and it’s really in the end unclear how much information has to do with this mix - how important is information (and more specifically journalism produced by regularly paid journalists working at news organizations) in this mix? Maybe inequality, resentment, poverty, white supremacy, religion, demographic change, historical amnesia, etc. have far more to do with the fraying of democratic life in the US and really, our best attention could be spent helping to support analyses that surface these starting points from within our disciplinary expertise.
As a corollary: Is there some magic equilibrium that we reach optimum democracy thanks to the perfect balance of news supply and news demand? If we were to take a per capita approach to thinking about journalism that would be ideal to inform a democracy, how much is enough? How do we model supply in this case? What about demand? Do we have ideal numbers of police, housing, or so forth that can help us? What is the universal basic income for civic news and information?
What’s the outcome variable for journalism that really matters? Assuming for the moment that we can show some sort of direct effects or causal affects of journalism on a positively beneficial social outcome, what really is the variable that matters most? Voting actually may not be the best measure - more anti-democratic people are participating in American democracy than they have in the past fifty-odd years.
Does funding for journalism make a difference? What is that difference? Is it keeping up government accountability? (again my forthcoming paper offers a little insight). And would public funding ever, ever meaningfully do anything to mitigate perceptions of media bias?
Is developing trust in journalism really meaningful when political institutions are becoming increasingly anti-democratic at the structural level? How do we assess trust in journalism and determine whether greater trust actually matters to this larger goal of democratic, tolerant, pluralistic (and for me, at least, market-based) public life?
Can history (through trend analysis) help us understand what a better future can look like?
What does this have to do with economics? Well, this is more thinking about econometrics as a way to systematically assess data at scale, going a bit beyond a deep historical narrative. As a history major and current history autodidact/semi-obsessive, my unprofessional take is that there are really interesting ways to think about moments of strength and weakness in American public life, historically, and then compare and contrast how we might model for a better future. This approach also helps us think about whether the good times were good times to begin with, and provides fresh insight to consider what metrics matter when thinking about a democratic life where tolerance and mutual respect for laws and justice motivates civic and cultural life.
My favorite media historian, Heidi Tworek, provides good evidence that checks media plurality and availability as an antidote to the rise of fascism (Nazism actually). My recent stay in Cataluyna, which has prompted a deep deep dive into the Spanish Civil War, is a reminder that all the party activity and all the news and political engagement can mean very little in the face of dictators with superior arms, international inaction, and food shortages.
How can we model a best possible future? Or plan for the worst case scenario? One of the biggest takeaways for me has been that when you don’t have the data you want, you don’t actually have to go get this data to begin to think about what you would hope the data might tell you. Much like we might reverse engineer an algorithm, we might engineer the world we want to see - where we can see what our best hopes for various levels of civic participation, social tolerance, equity, etc. might need to look like (the universal basic income for communication, maybe?).
In other words, economists run lots and lots of simulations on invented data (really) - in order to make the case that either their math works out as they say it does (this seems to me to be the foundation of econometrics today) or because they’re literally trying to model a possible future outcome and the model itself is…. in the future. Scenario planning appears in Communication scholarship at the fringes, mostly out of organizational communication and more management-focused scholarship, especially in strategic communication (the only reason I know about scenario scholarship is from Patti Riley at USC, and these remain fundamentally qualitative affairs, often of worst and best outcomes - so what this looks like as quantitative imagined models I don’t know!).
OK, that’s enough from me today on this, as I’m still thinking. What ideas do you have for questions we aren’t answering well that perhaps rethinking in terms of quantifiable outcomes would make a difference to our conversation about the threats to democratic life in the US? Tell me it all. Please. Dímelo!