Lada Adamic and Nathalie Glance published one of the first papers analyzing an empirical network of websites, Divided they Blog. It features a network of political blogs, harvested before the 2004 US presidential election. For good or for bad, its first figure may have had more influence than its findings. In this figure we see two clusters, one for the democrats, one for the republicans, and colored accordingly.
The paper was published in 2005 and has been hugely cited (2551 times in 2019 according to Google Scholar). I have two stories about this image, a tale and a horror story, which together form a bigger story. Let us start with the tale.
I. The Journey of Figure 1
I was super happy to discover a paper from Brooke Foucault Welles and Isabel Meirelles, Visualizing Computational Social Science: The Multiple Lives of a Complex Image, published just ten years after Divided they Blog. They take a close look at the journey of the figure 1 in academic literature and beyond.
As they observe, the image was not supporting any argument in its original paper. The conclusions were grounded on network metrics, independent on visualization. The role of the figure was only to illustrate methodology. It was not convoked as a piece of evidence. However, once it circulated, it became a piece of evidence to other authors. Foucault Welles and Meirelles cite for example “a popular-press book about network science”, Connected:
What immediately stands out is the extreme separation between liberals and conservatives. … Just like the real-world political networks …, the online social network appears to be strongly homophilous and polarized. This suggests that political information is used more to reinforce preexisting opinions than to exchange differing points of view.Connected, Christakis & Fowler, 2009, p. 206 (ellipses my own)
As Foucault Welles and Meirelles put it, “[if] readers simplify the image by ignoring the links connecting the parties, then the full blogosphere visualization more easily communicates the desired message of political polarization.” They even provide their own version of the figure, blurred in order to emphasize the effect it produces on an untrained eye:
Their paper makes a number of very interesting points, but I am not summarizing them here. You can read it by yourself, it is short enough. The part of the tale that I want to retrace here is how much these two clusters seem to mean to so many people. Few visualizations have traveled this far, or in such complicated conditions. Foucault Welles and Meirelles conclude this way:
Finally, and most critically, we call on computational social scientists, especially network scientists, to interrogate their own visualization practices. As discussed above, constructing network graphs remains as much an art as a science, with few conventions regarding the “right” way to represent node-link data.Visualizing Computational Social Science, Foucault Welles and Meirelles
I certainly agree, though in my perspective, there is a rigorous method behind network analysis. Good visual network analysis should be teachable. I take the occasion to quickly unfold how to read visual cluster, which will be necessary to our second story.
How to interpret visual clusters?
The key fact that the audience easily misses when looking at a network visualization, is that the layout ignores the node attributes. The position is independent of the color. The structure does not reproduce the content. The hyperlinks do not follow political affiliation. Until they do, and then it’s remarkable. That is why figure 1 tells something new. That is why there is a fact, a finding. The fact that each cluster has its own color is not a given, but an obtained empirical observation. A correlation between the content and the structure.
More generally, the backbone of such image-based rationale is as follows:
- The nodes are colored according to attribute X.
- The layout algorithm places the nodes according to their links, regardless of anything else. It optimizes the distances between nodes so that closer nodes have more chances to be connected (directly or indirectly).
- We observe areas of higher node density. According to the layout algorithm, those are “clusters” where nodes are more connected than on average in this network.
- We also observe that each cluster is mostly populated by nodes of a distinct modality of attribute X.
- Crossing these two observations, we conclude of homophily: in this network, nodes of the same modality of X have more chances to be connected.
This argumentation is important to the researcher during exploration. In a paper you need a stronger argument, and like Adamic and Glance, you should favor a metric, such as the densities of the groups formed by the modalities. This way you can ground your evidence on the data without the the mediation of the layout. But the final paper is not the only place where you need evidence. Exploration also requires it, albeit on a weaker standard. Although visual interpretation is not the best evidence, it guarantees that your exploration leads to findings. And that, is something.
II. A Horror Story
I was horrified by reading a paper published in the Journal of Machine Learning Research, Community Detection and Stochastic Block Models: Recent Developments. It is about the stochastic block model (SBM), a famous model used for community detection, ie. finding clusters. The paper is sound overall, and its core points seem a valuable contribution. But the figures are the worst misuse of network visualization I have ever seen published.
The relevant part of the paper explains what community detection aims at – finding clusters. On the surface, the figures seem to do the job. Take a look at this one, and I will explain its caption:
As in the case of Adamic and Glance, the image is not key to the argument, merely serving as a way to help the reader understand. But if you are familiar with network visualization, you ask yourself one of the following questions:
- Did the SBM change the structure of the graph? That’s weird because clustering algorithm are not supposed to. Or maybe…
- It is the same graph? But in that case, does it mean the SBM changed the nodes placement? Isn’t that the job of a layout algorithm?
It turns out that here, the network is the same “before” and “after”. The clustering classes are only known “after”, but the nodes and edges are the same in both cases. Which raises this question: why are there two different layouts? Remember: the layout algorithm only acknowledges the links, not the attributes (the clustering classes represented as color). There is no reason why the layout should be different.
The author seems to believe that a node placement algorithm ignores the structure and only looks at the attributes. Which is the opposite of what they really do. Or more precisely, what force-driven placement algorithms do, the kind used by Adamic and Glance, and known to manifest clusters. Interestingly, the image featured in the paper mimics the general layout resulting of such algorithm. What could work the opposite way of a force-driven placement algorithm, while producing similar shapes? A fake.
Both “before” and “after” images are faked. A random placement algorithm has been tweaked to produce this result – the paper calls it a “random arrangement”. This otherwise useless algorithm ignores links to place nodes randomly in a disk, possibly more often at the center. The image on the right is of the same kind, except that each cluster has been rendered separately and then placed apart. These images do not translate the network’s topology, while simultaneously referring to a practice where this translation is central.
The motivation is simple. Layout algorithms already visualize the clusters, but the author wants to illustrate how SBM finds the clusters. So I presume he designs or repurposes a process inapt at visualizing clusters, so that the lead role can be played by SBM. Note that I do not mean that layouts are preferable to community detection (both are super valuable for different uses). I just mean that this argumentation is dishonest.
I also consider the possibility that the author believes visualization is just artistic pictures aiming at a supplement of clarity, so that anyone can do whatever they want without consequences on the scientific level. This is equally problematic.
I am horrified that an academic audience could be tricked by these figures. I find it threatening because it shows that a complete misunderstanding of network visualization might go unnoticed under peer-review. In my worst nightmares, I am the only one to see the problem and everyone thinks I am splitting hairs. I hope you, reader, see where these figures are flawed. But I realize that it is not so easy to unfold.
Like if debating with flat-earthers, trying to expose the flaws of this rationale I find myself entangled in a web of nonsense. Without the backing of commonsense, I realize that establishing the obvious requires considerable effort and I start doubting my ability to show how dangerous it is. So instead, I will push the flawed logic further to expose its absurdity. And it will be about the figure 1 of Divided they Blog, because of course, the author also features it. Take a look at this glorious image, and its baffling caption:
In the fictional world where this rationale makes sense, Lada and Nathalie found themselves deeply puzzled by their data. No matter which algorithm they used to visualize the blogosphere, it always appeared unified as a pure center-periphery structure, an obscure hairball, all blogs orbiting randomly regardless of their political affiliation. What an incredible finding! The expected homophily was completely absent from the blogosphere. No metric could find a correlation. As they hesitantly started drafting a publication for Science, a computer science colleague suggested the stochastic block model. And as they took a try at it, an even more surprising event happened. Not only did the liberal/conservative divide appear, coloring the network in red and blue, but the very structure of the blogosphere suddenly unveiled. The blogosphere hatched before their very eyes, unfolding its two hidden clusters like the wings of a magnificent chicken. The initial network was an egg that only the stochastic block model could crack open, into an explosion of colors that would make them famous. This would be remembered as a moment of pure magic, the unveiling of realities hidden to the simple human mind, the grand opening of a portal to Knowledge whose only key has to be an algorithm.