Quoting data visually

22 min read

There is a sense in which a visualization can quote the data it represents, and not just summarize it. I find the analogy with quoting text useful. In this post, I explain how to spot such visual quotes, how it works, and when to use it in your own visualizations. To do so, I will contrast traditional visualization, that I call reductionist, with the visualizations that quote, that I call extensionist.

Why we quote (in text)

Let us forget about visualization for a moment, and think of quotes in text. Quoting divorces a statement from its author and makes it available as object of discourse (Olson & Oatley, 2014). Quoting is reusing, and therefore, reinterpreting.

It may serve multiple purposes. One quotes a figure of authority to get legitimacy. A researcher quotes their colleague to pay their intellectual debt. In those examples, one could summarize or paraphrase instead of quoting. But I am interested in the situations where we quote precisely instead of paraphrasing or summarizing.

“The fundamental meaning of quote marks is conventionally delineated as based on a distinction between ‘direct’ and ‘indirect’ speech. To surround words by quote signs signifies that we have someone’s exact written or spoken words and excludes the possibility that it might merely be a paraphrase, surmise or reinterpretation. Inside the signs are the direct words of some other voice, marked as of a different status from a second-hand processing or reshaping of them.” (Finnegan, 2011)

The difference between direct and indirect speech is not just about minimizing the distortion inherent to the reinterpretation, it is about providing the reader with a way to double-check that reinterpretation.

Here is an example. During the 2008 US presidential campaign, circulated the claim that vice-presidential candidate Sarah Palin said she could see Russia from her house. The implication was, for Americans eager to see her ridiculed, that she used the absurd statement as a geopolitical insight. The claim originates in an interview on ABC News. However, Palin’s actual quote is quite different: “you can actually see Russia from land here in Alaska.” An objectively true fact, that Palin did not frame as a motive for any geopolitical insights (Mikkelson, 2011). In this example, the actual quotes gives the reader a chance to assess the claim by themselves, critically.

Remark that the difference made by the quoting is not about the faithfulness of the retranscription, but trust:

  • If I write Palin said she could see Russia from her house, the reader may think that I have misunderstood her or that I am ill-intended.
  • If I write Palin said you can actually see Russia from land in Alaska, the reader may also think that I have misunderstood or bent her words. The reader has no way to know that my statement is more faithful.
  • If I write Palin said: “you can actually see Russia from land here in Alaska”, the reader only needs to trust that I did not manipulate the quote.
  • And if I write Palin said: “I can see Russia from my house”, the reader also just need to trust that I quoted appropriately; but I will have a much higher price to pay for a fake quote than for a stretchy interpretation (litigation, credibility loss…), therefore the reader’s trust has reasons to be higher.

Quoting engages the author to a higher degree than paraphrasing. It can be efficiently combined with summarizing, as the reader can skip the quote if they do not want to double-check the faithfulness of the summary. Quoting in visualization works the same.

Quoting data in a visualization: example

Here I showcase an example from my own work where quoting is explicit. It consists of a poster showing a network map of the words used in academic papers about AI and algorithms. I have described the method in a previous post, but you don’t need to read it as I will describe the poster from a reader’s perspective.

I you walk towards the poster, you first see its general shape: red lumps and tendrils, unequally distributed across space, with a big hole more or less in the middle.

General view of the map

If you come closer, you see labels. Big labels for the biggest chunks of the map, in purple and mostly around. Medium labels in red, pointing at clusters or along the tendrils.

The zoomed-in section above is about social science (on top) and economics (bottom-left). It contains vocabulary about different fields, but always within paper mentioning algorithms, AI or machine learning (as this is how the corpus was delineated). In that sense, it is a semantic map. The clusters consist of words (or multi-word expressions) that appear in the same papers. The red labels form a manual coding of the different clusters: we (Anders Munk, Matilde Ficozzi and I) read samples of the abstracts to summarize what the related papers are about (process in this post).

The purple layer consists of very general annotations. The red layer consists of more precise annotations (manual coding) but is still, in some sense, a summarization. If the visualization only included these layers, the reader would have to trust our ability to represent the underlying data (expressions connected by co-occurrence) as a picture. They would see the image below and wonder: why did they draw the shapes like this?

Annotation layers only

This is why we also featured the underlying data itself. That layer consists of dots representing the expressions, and they have been placed by a network layout algorithm. The labels are very small, and even so, we could not display all of them. A light shadow is highlighting the areas where the dots are densely packed (explanations there). That layer just by itself looks like this:

Underlying data layer only

The underlying data layer is not very readable, but it is useful to the reader who wants to double-check our annotations. This is why we combined the two layers into the final image. The underlying data layer is present in cyan, and you can see where the dots are and read the expressions by getting very close to the poster. The cyan appears black when superimposed with the red. The reader can see that the red shapes follow the contours of the groups of cyan dots, but only imperfectly.

Annotations (red) are superposed with underlying data (cyan)

By design, the annotation layer is easy to see and the underlying data layer requires an effort. The red layer is a summarization, while the cyan layer is a quote. There are different ways to articulate these two layers. In this instance, we used an anaglyphic split of the color spectrum to allow seeing one layer or the other through a cyan or red filter.

As a result, one can navigate through the layers by wearing anaglyphic 3D glasses and blinking from one eye to the other.

If you spend time with this visualization and you compare our annotations with the underlying data, you realize that the clusters do not represent the underlying data equally well everywhere. We did our best, but we had to compromise sometimes.

In some cases, the clusters were well delineated, and you can check that there are (almost) no cyan dots around the red clusters. In these situations, the underlying structure could be appropriately visualized as a single thing, a single cluster.

In other cases, the clusters detected (by clique percolation) were partially entangled, overlapping. This situation represents the inherent ambiguity of a semantic space, the continuity of meaning between topics. Our annotations provided distinctions, but the underlying data shows that the distinct topics are interfering with each other: there are no clear gaps in the continuum of dots.

In yet other situations, we could define a cluster but with fuzzy borders. Instead of a tightly packed set of expressions that always come together, we had a nebula of loosely co-occurring terms orbiting around an identifiable center. This is visible in the visualization as clouds of dots floating around the red cluster. We then tried to draw fuzzy borders to our clusters. The reader can look at the surrounding expressions and ponder whether they belong to the cluster.

Finally, some of the structural clusters we detected were stretched by the layout algorithm, making them look like bridges. Those clusters can be called bridges, but it is worth stressing that from a structural standpoint, they are no less clustery than the others. We tried to capture that feeling by drawing the bridge on top of the appropriate dots, which you can check by yourself.

If you have tried navigating these images, you should have a sense of how the quoted data layer contextualizes the more readable summarized layer. This way of quoting data is admittedly sophisticated, but a similar effect lies at the heart of what I have called “big data visualization” in a previous blog post and in my PhD thesis; I do not like the name though, so today I will go with extensionist visualization.

Extensionist visualizations do quote

I intend the term extensionist as a counterpoint to reductionist. A reductionist visualization summarizes. There is nothing wrong to reductionism. It is at the heart of science, statistical analysis, and traditional data visualization. Here is a reductionist chart:

Number of recorded deaths of migrants in the Mediterranean Sea from 2014 to 2022 (source)

What makes it reductionist is the act of reducing the phenomenon (migrants drowning in the Mediterranean Sea while trying to reach Europe) to one of its features (evolution over time). As Latour (1999) theorized, reduction loses locality, particularity, materiality… but also amplifies compatibility, computability, universality… Reduction is a productive tradeoff. Basically, it summarizes. We are very used to it, and most visualizations are like this, so it does not stand out. But here is a non-reductionist visualization, an extensionist one (click to enlarge).

The Missing Migrants Map by Valerio Pellegrini and Michele Mauri

This piece by Valerio Pellegrini and Michele Mauri has been produced for the Italian newspaper Il Corriere Della Sera and won the Kantar Information Is Beautiful Award in 2016. It is composed of multiple smaller reductionist visualizations, but the central part is extensionist:

Zoom on the central part

The visualization dedicates space to each and every data point in the source corpus; and by data point, I mean the record of a dead or missing migrant. Those are not aggregated into a statistics, but visually spread out so that we get a sense of where it happened, and how much it represents. This lack of aggregation is the defining feature of an extensionist visualization.

To circle back to my argument about quoting in text, it is worth noting that the extensionist visualization works in tandem with the reductionist ones, the same way quotes and summarizations get along well. If you just want to know how much migrants are missing, you can just read the text (1,700 died and 2,200 were missing in 2015); if you want to know when it happened, you can look at the top-right chart. The map provides additional context to various ways of summarizing the same information, it efficiently combines with them. And of course, it is less abstract. It makes the missing migrants more real to the reader.

Seeing each dot as a missing migrant makes the visualization striking. This is made possible by quoting the underlying data literally. In this case the quoting is not a supplement to the main message, it is the main message.

Extensionist visualizations are made possible by big data, because they need many data points and relatable numeric dimensions (in this case, the geographic position). A visualization is extensionist when it visually quotes the data points in a way that displays recognizable patterns, and lets the reader engage with them without reducing them to a specific message. In that sense, extensionist visualizations differ from traditional visual communication because they offer the reader the possibility to explore.

In other words, extensionist visualizations are the heirs of cartography. Most cartographies are extensionist insofar as they provide knowledge without conveying a message, and let you recognize the patterns relevant to you. But extensionist visualizations do not necessarily depict geographical spaces. Typically, network maps are extensionist visualizations depicting non-geographical spaces. Also note that extensionist visualizations do not necessarily consist of dots. In the wind map below, the data points are essentially lines.

The Wind Map project by Fernanda Bertini Viégas and Martin Wattenberg (featured in the MoMA)

Reductionist visualizations are not extensionist because the aggregation they involve summarizes instead of quoting, so the visual does not refer to the data in a literal way. The point of the reduction being to obtain new insights, the visual patterns are, by design, co-produced by the method. We can summarize a series as an average or as a median, both being equally valid yet different, each giving us a different insight and interpretation. The reduction method is baked into the pattern. So although we always see patterns in a visualization, those are not always from the data. A bar chart has bars, which is a visual pattern, but it comes from the method, not from the data; conversely, the swirly pattern in the wind map above comes from the data.

Note that extensionism is a type of visualization, not a firmly delineated category. A visualization can be somewhere between extensionism and reductionism. This happens when the data points are not manifested as literally as they could, which is the case most of the time, because visualization is, in essence, translation. For instance, in a network map, the layout algorithm mediates the structure, and therefore the visual patterns depend on it, in addition to the data. The quote is only as literal as the algorithm is transparent to you, which depends on your expertise. One easily realizes that visual patterns (like clusters) only arise when the network has certain properties (a community structure), but unpacking how that translation works remains hard. The reader may find themselves in the relatively common situation where they trust that the visual pattern comes from the data, and yet they cannot explain how, so they cannot fully interpret the visualization. The edge case is when the reader is uncertain about whether patterns come from the data or something else, like the method, the algorithm employed, or a manual intervention. In that case, one could say that the visualization is partially extensionist.

The noema of extensionist visualization

As the defining feature of extensionist visualization is to offer visual patterns to explore, its noema is: the data have patterns (genesis of this idea in this post).

Noema basically means essence. I borrow the term from French semiotician Roland Barthes who theorized that the noema of photography was that-has-been. The noema refers to the process through which we attribute meaning to a piece of media. For Barthes, we attribute a meaning to a camera picture on the basis that a mechanical chain of reproduction has taken place between the photographed subject and who watches the picture. As such, even though what is represented may trick us in various ways, and not be “real”, and even though we know it, we still assume that what we see has, in some ways, been. Roland Barthes did not have Stable Diffusion.

Similarly, we make sense of extensionist visualization by assuming that the representation of the data points is sufficiently literal to grant that the patterns we see originate in the data. The visual patterns bear meaning precisely because we make sense of these visualizations as quotes of the data. But importantly, we might not have access to that meaning. We may only recognize that there are patterns in the data, not necessarily what those patterns are and how to interpret them. Look at the wind map shown before, and its massive spiral near Dallas: can you tell whether it is a hurricane or a normal wind pattern? I cannot, and yet I see the pattern, and since the picture quotes the data points, I can trust that the recorded winds are that swirly. The pattern is in the data; yet I lack the knowledge necessary to tell if this is common or remarkable. I cannot interpret it.

The purpose of the noema is to explain the appeal of extensionist visualizations. My motivation for conceptualizing it came from reading the excellent paper The Politics of Method: Taming the New, Making Data Official (Ruppert & Scheel, 2019). The authors analyze the showcasing of a dynamic visualization of the Estonian population to Estonian officials (screenshot reproduced below). They write: “The moving red dots become not only a vehicle for the data, but first and foremost for its claimed self‐evidence. … Through this ‘realist trick’ (Law 2012) mobility is enacted as a reality that exists independently of the methods that are used to describe it. There appears to be a seamless correspondence between the visualization (the moving dots) and the reality (commuting patterns in Estonia) it represents and renders ‘the phenomenal world (as if it) were self‐evident and the apprehension of it a mere mechanical task’ (Drucker 2011)” (emphasis mine). I agree with Ruppert and Scheel that extensionist visualizations are problematic for their self-evidence, that hides the existence of a mediation, and ultimately tricks the reader into believing that Big Data (in this case) provides an unfiltered access to reality itself. But I disagree that self-evidence is merely “claimed” and the “correspondence” between patterns and data just an illusion. The purpose of the noema, as a concept, is to help articulate that extensionist visualizations are dangerous because they are powerful, yes, but that this potential can also be used in legit (non misleading) ways, which can be very useful.

Screenshot of a dynamic visualization of the Estonian population over time, obtained from mobile data, whose use in the public sector was analyzed by Evelyn Ruppert and Stephan Scheel in The Politics of Method: Taming the New, Making Data Official (Ruppert & Scheel, 2019)

Deconstructing extensionist visualizations

Extensionist visualizations are worth deconstructing because they are powerful. It is most important to realize that our recognition of visual patterns (if any) must be taken seriously. The visual patterns are as real as it gets to the reader who perceives them; but the patterns are co-produced by the visualization method and may not come from the data, or not entirely.

As extensionist visualizations rely on a form of quoting, the correspondence with data points is generally stated in the legend or somewhere else. The reader has reasons the trust (or not) the authenticity of the quoting, which is external to the visualization itself. Trust is built outside the visualization, but also within it. Indeed, visible patterns reinforce trust in the authenticity of the quote, for at least two reasons. First, the presence of visual patterns suggests that quoting the data points is a design decision aiming at displaying them (the patterns). It provides a justification for the extensionist design. Second, the (assumed) imperfections in the patterns suggest that the quoting was transparent about the method’s limitations. The reader is given the autonomy to double-check the author’s interpretation, which helps build trust. Those two points do not imply that the visual patterns exist in the underlying data, but they have value on their own. Doctoring an extensionist visualization is actually harder than a reductionist one, because it offers many more opportunities to detect the fraud.

This ability to build trust is the source of the outstanding convincing power of extensionist visualizations. The accountability is real, it is not an illusion, but it does not fully cover the correspondance between the visual and the data. Indeed, visual patterns are co-produced by the data and the method, so that visual patterns point at the existence of a pattern in the data, but that pattern might be quite different if the method has interfered with it. The reader who perceives patterns is justified in believing that the data have patterns (the noema of extensionist visualization), but they cannot know what the patterns are in the data and how to interpret them without specific knowledge about the method employed. The danger of extensionist visualization lies in the reader’s excessive trust that what they see is an unfiltered representation of the data, or worse, of reality. Despite their trust, the reader should not forget that the visualization is a partial view on the data, and that data only partially capture empirical reality. To mitigate the invisibilization of the mediations involved, it is important to provide adequate context, especially to extensionist visualizations.

How to spot visualization that quote data

When a visualization quotes data points, it has a surabondance of graphical elements. This is the main tell. Then you should check whether you are provided with the autonomy to make your own interpretation, or challenge the author’s interpretation. Finally, you should check how the graphical elements relate to the data points, which should be explained. The simplest case is when each data point is represented as a dot, but there are many other possibilities.

What to do when facing a visual quote?

A visualization quotes the data points for a reason. Check why, because you may not be the audience. Don’t waste your time if you cannot understand the patterns. Exploring takes time, so you probably need a good motivation to invest your energy in it.

If you want to engage, I suggest this approach. Mind the patterns you recognize, and ponder whether they come from the data, the method, or a combination of both. Be methodical about the mediations. What was the process leading to shaping the graphical elements the way they are? What does it tell about the origin of the visual pattern? What can we take for granted about patterns in the data, and what can we not? Mind that you may be inclined to prematurely conflate the visual elements with the data points. The conflation is plausible and simpler to navigate for your cognition, but it will make you miss crucial elements about the validity of the visualization and its context.

More generally, when it comes to exploring data visually, don’t be docile. An intuitive and comfortable visualization is not your friend. When facing an extensionist visualization, you will see the expected patterns first. Your first finding will often be a confirmation of your beliefs, which will inflate trust towards the visualization too early. Slow down, suspend your preference for agreement, and engage with the visualization further. Become active and indocile: look for flaws and failures. You have seen things that are present where they are expected to. What are other expectations that are not met? And what are unexpected things, things that you expected to be absent but are nevertheless present? What are unexpected patterns? Build your trust on the possibility that you may disagree with the patterns, not on the possibility that you may agree; because you will always agree to some extent. You will always find a mix of agreement and disagreement, and the agreement will always come first. More about this in Thinking through the Databody, a chapter I coauthored (Munk et al, 2019).

Can you quote data in your visualizations?

Here is my best insight: quoting data points only makes sense with relatable spatial dimensions. Not all dimensions give you the opportunity to go the extensionist route, and you may have to be reductionist. You don’t get to decide every time.

I see four situations where you can afford an extensionist design.

1. Your data points are geolocated

Then you can make a map. This is the easiest case, because extensionist visualizations inherit from the cartographic tradition. Example: a dot map representing the US census. The wind map also belongs here.

2. Your data points can be placed in a relatable space

In this case the space is abstract, but can be understood by the audience. The main example is a scatterplot, but there are other possibilities, like the ternary plot below from this paper (Wilson et al, 2018). Obviously, “relatable” depends on the audience (the vertices of the triangle don’t mean anything to me).

3. Relatable spatial relations can be derived

This is the realm of network maps, but also plots from t-SNE and UMAP. Here it is not the space itself that is relatable, but the distance between the dots. The presence of a layout algorithm thickens the mediation, which is a problem. But in principle, proximity can be interpreted and we can still explore the image. Below, an example from this paper (Mokashi et al, 2021) where each dot is a cell and two cell are placed closer if they have a similar “expression pattern”.

4. You don’t need spatial dimensions

You may not need your data points to have spatial dimensions, in which case you may just arrange their corresponding graphical elements to get enough clarity. The reader will still be able to explore the visual variations between those elements, which may represent different dimensions. This kind of chart is sometimes called a pictorial chart but that is not a well-defined category. We nevertheless find many good examples, like the piece below by Nadieh Bremer for the Scientific American.

Note that you can combine a pictorial graph with other kinds of charts, like a bar chart. Find a good example by Andrew Van Dam and Renee Lightner for the Wall Street Journal below.

Should you quote data in your visualizations?

First of all, offering exploration does not engage your users more (Boy et al, 2015). It actually goes the other way around: exploration requires a strong engagement. And engagement is scarce. If your audience is not particularly engaged, then you should not quote the data points, or you should quote them in a non-intrusive way for the few interested, but keep the summarized layer clear for most of your audience.

I do not have a list of situations where quoting makes sense, but one stands out to me: when I produce a visualization for myself. Once in a while, for my own research, I have the time to engage with a large dataset. Extensionist visualization is a fantastic tool for exploration. Note that doing so does not prevent me from being reductionist in parallel: it works in tandem.

Besides exploration, the other strong reason I see to quote the data points is to allow the reader to double-check and challenge your interpretation. This is typically the case when you are exploring a dataset for which you lack expertise. When you meet experts to get more knowledge about the data, it is useful to have a visual support with your own temporary conclusions but allowing for other interpretations. You would then mix annotations and a straightforward representation of the data points. More generally, quoting the data visually is useful in collaboration settings, within science and beyond.

Finally, quoting the data can be useful to improve the legitimacy of the visualization, for instance as a data journalist. It allows the reader to be more critical, which is risky but can be rewarding. In that situation, mind that most people will not engage much, so provide a strong summarization, for instance as annotation, for the readers who just want the key insight.

References

BOY, Jeremy, DETIENNE, Francoise, et FEKETE, Jean-Daniel. Storytelling in information visualizations: Does it engage users to explore data?. In : Proceedings of the 33rd annual ACM conference on human factors in computing systems. 2015. p. 1449-1458.

DRUCKER, Johanna. Humanities approaches to graphical display. Digital Humanities Quarterly, 2011, vol. 5, no 1.

FINNEGAN, Ruth. Why do we quote?: the culture and history of quotation. Open Book Publishers, 2011.

LATOUR, Bruno. Pandora’s hope: Essays on the reality of science studies. Harvard university press, 1999.

LAW, John, 2012, Collateral realities. In : The politics of knowledge. New York : Routledge. p. 156–178.

MIKKELSON, David. Did Sarah Palin Say: ‘I Can See Russia from My House’? Snopes, 2011. https://www.snopes.com/fact-check/sarah-palin-russia-house/ (accessed 2023-07-12)

MOKASHI, Sneha S., SHANKAR, Vijay, MACPHERSON, Rebecca A., et al. Developmental alcohol exposure in Drosophila: effects on adult phenotypes and gene expression in the brain. Frontiers in Psychiatry, 2021, vol. 12, p. 699033.

MUNK, Anders Kristian, MADSEN, Anders Koed, et JACOMY, Mathieu. Thinking through the Databody. Designs for experimentation and inquiry: Approaching learning and knowing in digital transformation, 2019, p. 110.

OLSON, David R. et OATLEY, Keith. The quotation theory of writing. Written Communication, 2014, vol. 31, no 1, p. 4-26.

RUPPERT, Evelyn et SCHEEL, Stephan. The politics of method: Taming the new, making data official. International Political Sociology, 2019, vol. 13, no 3, p. 233-252.

WILSON, S., COLLINS, F., et LAVERY, R. Using ternary plots for interpretation of ground gas monitoring results. Ground Gas, 2018.



Cite this blog post
Mathieu Jacomy (2023, July 13). Quoting data visually. Reticular. Retrieved April 24, 2024, from https://reticular.hypotheses.org/7776

Leave a Reply

Your email address will not be published. Required fields are marked *

This site uses Akismet to reduce spam. Learn how your comment data is processed.

Search OpenEdition Search

You will be redirected to OpenEdition Search