Open menu

De Nederlandstalige versie van dit artikel vind je op de site van het Privacy & Identity Lab.

In short:

  • We used a short political quiz for measuring the socio-economic bias of Chilean news outlets.
  • The political orientation of the media landscape is subject to change.
  • To mitigate the effects of the ‘filter bubble’, it is not sufficient to only address personalization algorithms; one should also analyze differences in orientation within the media landscape.

Back in 2011 already, Eli Pariser taught us in his TED Talk “Beware online filter bubbles” that our online lives largely take place within a filter bubble. Facebook automatically selects the items that will reach your news feed based on your click behavior, and Google search results are personalized based on, among others, your current location and your search history. As a result, we mainly encounter information and opinions that match our own life philosophy.

In a similar fashion, traditional newspapers and other news outlets make a selection of the news items to be included. It is common knowledge that the New York Times has a liberal bias and Fox News a conservative bias, and that people usually choose for a newspaper that matches their own orientation and interests. By contrast, little is known about political bias in smaller, regional newspapers or in the still growing number of newsportals, among which the Huffington Post, Yahoo News, CBS, but also the Breitbart News Network.

We carried out a study to identify political bias within the media in Chile and obtained some surprising results that are relevant for the media landscape in general and for our personal, personalized news consumption.

The World’s Smallest Political Quiz

We played an automatized version of the  World’s Smallest Political Quiz (which we will abbreviate as PolQuiz) with 269 Chilean news outlets. The PolQuiz consists of ten questions, based upon which players are positioned in a socio-economic spectrum: left-wing or right-wing, libertarian or statist or somewhere in the middle.

The news outlets automatically ‘answered’ the questions by means of a collection two million tweets that were sent between October 2015 and June 2016. Matches between a PolQuiz question and relevant tweets were made based on relevant (Spanish) keywords. For example, for the statement “Government should not censor speech, press, media, or internet”, we used the keywords ‘censura’, ‘libertad’, ‘prensa’, ‘discurso’, ‘expresion’. The set of keywords was extended with the most occurring keywords in the tweets containing these keywords. For each PolQuiz question, the relevant tweets were manually and automatically classified as ‘Agree’, ‘Maybe’ or ‘Disagree’.

Absolute and relative bias of news outlets

As illustrated in the picture below, most Chilean news outlets are situated top-left, in the left-libertarian end of the spectrum. Note that the news outlets are a bit more pronounced in terms of economic issues than in terms of personal issues. The left-libertarian focus of the media is in line with the political orientation of the Chilean government between 2014 and 2018, led by Michelle Bachelet of the socialist party, with the Christian Democratic party as an important player within the coalition.

medialandschap chili 2015 2016The Chilean media landscape in 2015-2016

Interestingly, this left-libertarian bias does not exist in the perceived bias of these news outlets, as stated in Wikipedia and/or the official websites of these outlets. The perceived bias appears to be more differentiated than the actual bias that we measured. However, once we spread the measured bias over the whole spectrum, this relative bias was in line with the public perception.

To put it differently: the overall orientation within the media landscape in 2015-2016 was left-libertarian, but this general bias was not perceived by the general public. However, the perception of relative differences between the individual differences were in line with the differences that we measured. These results are stable, even if we would remove a small part of the tweets or would replace one PolQuiz question with another question.

The influence of government orientation on the media landscape

As described earlier, the orientation of the Chilean media landscape as well as the government in 2015-2016 was left-libertarian. By contrast, in the preceding period, Chile was governed by a right-wing conservative coalitation led by president Sebastian Piñera (who, as a matter of fact, was reelected in 2018). In order to compare the media landscapes of these two periods, we crawled another collection of 16.000 tweets from 186 Chilean news outlets, and again played the PolQuiz with these news outlets. As can be seen in the figure below, the results are quite different from the first analysis, with the media landscape shifted more towards the government of that period.

medialandschap chili 2010 2011

The Chilean media landscape in 2010-2011

An notable exception is El Mercurio, one of the larger media groups in Chile. We assume that due to its size, this outlet is the most stable one in terms of their editorial policy behavior.

We further investigated the nature of the bias by taking the Chilean abortion bill as an example, a bill that polarized the society and that was accepted in 2017. In short, we found that the left-libertarian news outlets mentioned politicians who voted in favor of the bill to a larger extent, and reported on politicians who voted against the bill with a more negative sentiment. The opposite effect was found for right-wing conservative news outlets. A survey among 54 Chileans confirmed that this bias was clearly visible for the general audience.

What are the implications of our study?

With the PolQuiz, we instrumentalized a simple, objective method for placing news outlets in a socio-economic spectrum. Our results largely align with the public perception, but we also observed a shift of the whole media landscape along with the shift in government from right-conservative to left-libertarian. It should be stressed that this shift should not be interpreted as financial or political dependencies between the media and the government, but as a correlation that most likely is caused by changes in the political agenda and the resulting topics that are discussed (such as the abortion bill in Chile).

The number of online news outlets is still increasing, with both official and serious media as well as non-serious blogs and sites that try to influence the public opinion with biased news coverage – possibly including propaganda and fake news. As argued in the beginning of this article, news aggregators (among which the newsfeed on Facebook) mainly contain news items that align with the user's values, the so-called filter bubble.

Particularly at times like this, in which words like manipulation, propaganda and fake news are used on a regular basis, it is important that users understand bias in their own, personalized media landscape and how they can fight unwanted effects. Much attention is being paid to the influence of algorithms used by Facebook, Google and the like.

Our research shows that, in addition, one should also pay serious attention to the overall orientation of the media landscape, which is – as we have seen – subject to change. Further, tools such as the PolQuiz are needed to make the political orientation of news outlets transparent, particularly those that are not mainstream and that have a particular political agenda.

Erik Elejalde, Leo Ferres, Eelco Herder (2018) On the nature of real and perceived bias in the mainstream media. PLoS ONE 13(3): e0193765.

eelcoherder 256px

Privacy Engineering, User Modeling, Personalization, Recommendation, Web Usage Mining, Data Analysis and Visualization, Usability, Evaluation

Dr. Ir. Eelco Herder
Associate Professor
Human-Centered Computing Group
Information and Computing Sciences
Utrecht University

Buys Ballotgebouw
Princetonplein 5
3584 CC Utrecht
The Netherlands