Under-informative News
in American Media

Andrew Trexler

University of Wisconsin-Madison
2024 November 19

The media matters for democracy

Actual media coverage looks different

Classic perspective

Jeff Zucker (CNN): “The idea that politics is sport is undeniable, and we understood that and approached it that way.”
James Hamilton (Stanford): “The chase for additional consumers means that content will often reflect the preferences of those least interested in hard news.”
Matt Baum (Harvard): “Public affairs-oriented ‘hard news’ is often unappealing to politically inattentive individuals.”

My argument

Typical approaches to coverage do not broaden the audience
Instead, they double down on the most attentive: political hobbyists
Makes it more challenging to learn critical information
That is, typical approaches to covering politics are under-informative

Today’s talk

New media ecologies
- Computational analysis of news headlines (in progress)
Who is demanding the types of news we see?
- Conjoint experiment on news demand
Is it really under-informative?
- Vignette experiment on learning from news
Future directions
- Field experiment (in progress)
Broader research agenda

1. New Media Ecologies

New media ecologies

Fragmentation & specialization
Regular politics consumers are weird
Robust tools for tracking weird demand

Superficial news engagement
Mobile consumption
News “snacking”
Headlines have outsized importance

Shaping information value

Lots of decisions shape information value
- Content itself
- Placement, distribution
- Style
  - Today’s focus

Public interest

Centers normative expectations in the approach to coverage

Typical styles of coverage

Partisan Conflict

Insider Jargon

Prediction-as-News

Clickbait

A big imbalance

Data: ~2M headlines published by major US outlets (2016-2020)
Train RAs to classify headlines
Build a training dataset
Train, fine-tune a semi-supervised machine learning model
DSL regression to estimate overall proportions

2. Evidence on Consumer Demand

Study design

Preregistered conjoint experiment
Choose between two news stories to read
Completed 10 decision tasks
Respondents told they would be asked to read one of their selections

Example decision task

Repackaging in any style

Coverage Style	Example Headline
Public Interest	Congress approves new military aid package for Ukraine in bipartisan votes
Partisan Conflict	Congress approves new Ukraine funding, delivering Biden victory over GOP objections
Insider Jargon	Johnson pushes through Ukraine aid bill despite objections from Freedom Caucus
Prediction-as-news	The House Speaker’s push to approve new Ukraine funding might cost him his job
Clickbait	Here’s how the House Speaker got around far-right opposition to secure Ukraine aid

Overall AMCE of style

Overall AMCE of style

News junkies want under-informative news

Appendix: PE Measures

Typical styles are more entertaining

3. Evidence on Learning

From content to style

Prior work focuses on content
- Strategy story < issue story
News articles have many elements
- Which to prioritize?
Matters for information access, especially for mobile users and news snackers

Study design

Preregistered vignette experiment
Presented 3 news articles (in random order)
DV: post-exposure recall of key factual information
IV: manipulated coverage styles, but held info constant
- Treatment: conflict, jargon, prediction, or clickbait
- Control: public interest

Informational equivalency

(Nearly) all text is identical across treatments
4 styled body paragraphs
Primary manipulation is to alter paragraph order
Some framing manipulation in headline and lede

Example vignette

Study design

News consumers rarely read thoroughly
Embedded limits on exposure duration to prompt skimming
- Unlimited (median 85s)
- Slightly constrained (60s)
- Severely constrained (30s)

Learning penalty

Appendix: Items

Learning penalty

Appendix: Items

Learning penalty by baseline engagement

Appendix: PE Measures

Core contributions

Usual approach does not broaden the audience
Instead, it deepens engagement from a few consumers
People learn less from today’s coverage
For the media to best serve democracy, it must…
- Ignore demand signals from sustainers
- Prioritize preferences of less-engaged audiences

4. Future Directions

Field experiment (in progress)

Partnered with Ground News
Recruited probability sample
Enrolled in email newsletter
- 3x/week for 8 weeks
- 5 headlines in each
Randomized headline styles

Track demand with digital trace data
Track learning with quiz surveys
Evaluate tradeoffs for prioritizing engaged vs. disengaged audiences

Explore news engagement and learning in social media and visual formats
Descriptive analyses of how style interacts with content and prioritization
Survey of journalists/editors, with follow-up semi-structured interviews

5. Broader Research Agenda

Local news & politics

Political Behavior (2024)

R&R at Journal of Politics

In Progress

Political Behavior & Public Opinion

Political Behavior (2024)

Public Opinion Quarterly (2024)

R&R at Policy Studies Journal

Methods

Under Review

In Progress

In Progress

Summary of Research Agenda

How do the media shape the information environment?
How do people access and use information in politics?
What are the implications for democracy?
Can we improve methods for exploring these questions?

Thank you!

andrew.trexler@duke.edu
atrexler.com

Appendix

Placement Choices

Placement Choices

Prioritizing Content

Shift to Mobile

Who seeks out political news?

Going After HVTs

Subscription revenue increasingly important for digital media
Intensive margin matters above extensive margin

Samples

Learning experiment
- Nonprobability sample (Prolific)
- Analysis sample n = 2,233
- Fielded in 2023 September 21-22
Demand experiment
- Nonprobability sample (Prolific)
- Analysis sample n = 2,101
- Fielded in 2024 April 26
Field experiment
- Probability sample (registered voters)
- Convenience sample (Ground News subscribers)
- Analysis sample TBD, n ~ 1,500
- Fielded 2024 July to October

Conjoint Sample

Vignette Sample

Measuring Political Engagement

“Subjective” measures
- Attention to politics (0.125)
- Interest in campaigns (0.125)
- Days/week consuming political news (0.25)

“Objective” measures
- Political knowledge items (0.5)

Back to: Conjoint

Back to: Vignette

Sample Distributions of Political Engagement

Conjoint Attributes

Attribute	Levels
Headline Style	Public Interest, Partisan Conflict, Insider Jargon, Prediction-as-news, Clickbait
News Topic/Story	Economy (x4), Environment (x4), Foreign Policy (x4), Immigration (x4), Public Health (x4)
Outlet	CNN, Fox News, New York Times, Politico, Wall Street Journal, Washington Post
Reading Time	1 minute read, 2 minute read, 3 minute read, 4 minute read

Predicted probabilities of selection

Source Outlet

Policy Issue Area

Reading Time

Apolitical Headlines

Nonlinear Interaction

Headline Evaluations

Learning Items

Texas bill
- Allows state to remove local elected officials
  - \(\alpha = 3.54\), \(\beta = -0.38\)
- Critics say undermines self-government
  - \(\alpha = 2.15\), \(\beta = -0.01\)
- Affects Houston
  - \(\alpha = 1.63\), \(\beta = -0.69\)
- Empowers appointee of governor
  - \(\alpha = 2.07\), \(\beta = -0.15\)

New York gerrymander
- Governor supports redrawing
  - \(\alpha = 1.43\), \(\beta = -1.31\)
- Current districts drawn for competitiveness
  - \(\alpha = 3.78\), \(\beta = 1.59\)
- Current districts drawn by ind. court appointee
  - \(\alpha = 3.34\), \(\beta = 1.17\)
- NY constitution bans gerrymandering
  - \(\alpha = 2.97\), \(\beta = -0.38\)

SCOTUS ethics bill
- Strengthens gift reporting requirements
  - \(\alpha = 3.50\), \(\beta = -0.55\)
- Scrutiny due to recent unreported gifts
  - \(\alpha = 7.89\), \(\beta = -0.85\)
- Concern because gifts are very large
  - \(\alpha = 2.24\), \(\beta = -0.99\)
- Bill applies to all justices
  - \(\alpha = 1.50\), \(\beta = 0.58\)

Back to Results

Replication

Variation by Style

Mechanical vs. Psychological Effects

Perceived Informativeness

Media Credibility

Support for Norm-breaking