Digital resources in the Social Sciences and Humanities OpenEdition Our platforms OpenEdition Books OpenEdition Journals Hypotheses Calenda Libraries OpenEdition Freemium Follow us

How to render a network map, part 3: color

25 min read

In this series of posts, I roam and gauge the design space of network maps. In the two previous posts, I explained how to render a network in black and white and in grayscale. It is now time to explore what color can do for network maps.

One of my goals is to find the best rendering approaches for network tools. This series exposes my rationale so that it can be discussed and criticized, but I will extract do’s and don’ts and takeaways along the way, so that you can also get reusable, practical insights about visualizing networks.

The criteria I will use to assess and discuss each design are the following:

  1. Do the nodes contrast with the background?
  2. Do the edges contrast with the background?
  3. Do the labels contrast with the background?
  4. Do the nodes contrast with many underlying edges?
  5. Do the labels contrast with many underlying edges?
  6. Do the labels contrast with many underlying nodes?
  7. Do we see which edges are connected to a given node?
  8. Does the heterogeneity of the node distribution jump at our face?
  9. Can we identify the edge bridges between clusters?
  10. Context: how many labels can we read?
  11. Colored communities: can we see distinct groups of colored nodes? (typically in a situation of community detection)

Visualizing in color

Color is a vast topic that I won’t try to cover exhaustively. Levels of gray imposed such constraints that we had pretty limited options to produce readable designs. With color, there are so many possibilities that one cannot inventory the good design strategies. In short, with a clever use of color, one can almost always “fix” the “bad” designs we have seen earlier. Therefore my goal here is just to provide safe choices for a common use case.

I will contain the scope of this post to using color to visualize a categorical node attribute. In this particular case, I will visualize the airport country. There are too many countries to use that many perceptively distinct colors. For strategic reasons, I will use only 7 distinct colors with a luminance of about 55% and a chroma of at least 50% in the HCL color space. Only the 7 countries with the most nodes (i.e., airports) are therefore colored (USA, Canada, China, Brazil, Australia, Russia and India) while the other countries (including all of Europe and Africa) are colored in gray (also with about 55% luminance). These numbers are in part arbitrary, but my choices have a specific purpose that will appear progressively.

Color science is complicated. To discuss what is at stake in the choice of colors, we need to draw on a body of knowledge and know-how from graphic design. As I would like this piece to be accessible to non graphic designers, I will not discuss my use of color in details. I am afraid that arguments about color get too technical too quickly, contrary to what our intuition suggest (the feeling that color is just empirical information put in our brain through the eyes, which is not at all how it works). I will instead prioritize the overall rationale, and I will make visual arguments instead of theoretical ones.

In short, I selected colors that have the same perceived lightness, so that I could use grayscale visualizations as a baseline. If you pick completely random colors, some will be dark and others will be light, and unsolvable contrast issues will arise.

Colored dots

We have seen that black dots already give a pretty good representation of the layout, which is the most important thing. Let us naively start from the same point, and just color the dots: each node’s color visualizes the country of the airport it represents.

Compared to just black dots, two problems arise. First, the lesser contrast makes the layout harder to read. Second, the color is not easy to perceive. Indeed, the color requires enough space to be perceived properly. Contained in separated small nodes, it is barely readable.

DO: Give room for color to breathe.
DON'T: Colorize just the nodes, as they do not occupy enough space.

Colored edges

Let me discuss upfront the practice of displaying edges and coloring them, because that is what I have seen many people do (and it’s Gephi’s default behavior). In the image below, the edges are colored as their source nodes. In other words, the airlines are colored by the take-off airport.

Assessment: The contrast of nodes and edges with the background is good, but not that of nodes with edges, as they have the exact same color. For the same reason, the layout is not prominent enough. The bridging edges are visible enough, though. Finally, the communities can be identified, but in an imprecise way, as it is the edges and not the nodes that carry the color.

There is a non-obvious issue with drawing colored edges. The edge drawing order matters. Indeed, the edges drawn last end up on top, and their color dominates. It matters more often than you might think. Here is a real-world example. It comes from Lada Adamic and Natalie Glance’s famous paper Divided they Blog, but is also featured in other papers, notably Lazer et al.’s Computational Social Science. We have two clusters, one of which is blue (nodes are liberal blogs) and the other, red (nodes are conservative blogs). The edges are hyperlinks. Within each cluster, the edges are colored like the nodes. But between the clusters, it looks like the edges are yellow or orange, but that is incorrect. The authors have colored the 781 edges from liberal to conservative in orange, and the 902 edges from conservative to liberal in purple. Those edges are the bridges between the clusters, and occupy the same space. If a color had to dominate, it should be purple, as it is more represented. But for contingent reasons, the orange edges ended up drawn last (on top) more often, hiding most of the purple ones. As a result, it looks like there are more hyperlinks from liberal to conservative blogs than the other way around, while the opposite is true.

Image from Divided they Blog.
Adamic, L. A., & Glance, N. (2005, August). The political blogosphere and the 2004 US election: divided they blog. In Proceedings of the 3rd international workshop on Link discovery (pp. 36-43).
DO: Randomize the order of the edges if your visualization process depends on it while the order is expected to be arbitrary.

Edges give more space to color, and it makes the color more visible, which is a good thing. The effect is dramatic. Coloring the edges makes the communities (i.e., clusters) more visible, even though it also makes the layout less readable. To make it work, we have to address two issues: the fact that the edge drawing order matters, and the fact that edge color is really just a (quite poor) proxy for node color.

Before we look at ways to preserve the readability of the layout, let us address an arbitrary choice: coloring the edges by source node. Here is, for instance, the same visualization but with edges colored like the target node.

The image is not significantly different, but that depends on the network. The difference can sometimes be striking. Here is a compromise: edges colored using the mix between the source and target node colors.

Depending on the network, one of these three choices may be the right thing to do. For instance, for a network of web pages connected by hyperlinks, like Divided they Blog, coloring by source node manifests the fact that hyperlinks are materially situated in the source page. With airlines, we do not have such a solid rationale. We also have a problem with the overlap of mutual edges: if there is an airline from A to B, there is one from B to A (people need to come back!). We could use strategies to avoid this overlap (ex: curved edges), but that is beyond the scope of this post. Let us just mention that a mixed color solves the issue in our case. Using a color mix for edges is the safest choice for us.

DO: If you color edges using a node attribute, think of what makes most sense in your situation: the color of the source node, of the target node, or a mix of both.

From there, with this design, the most important thing is to notice the conflict between the necessity to give room to colors, which requires loud edges, and the necessity to manifest the layout, which requires muted edges. The best solution is to avoid relying on edges to give room to color, to use a different solution; but that requires a sophisticated process. Before we get there, let us see how we can make the visualization work better by just coloring the usual graphical elements (nodes, edges, and if available, hulls). My starting point will be the grayscale visualizations from the previous post.

Flat style with hulls, colored nodes, white edges

For the sake of the argument, let us reuse the grayscale visualization with gray nodes, but coloring them instead. I use the same luminance. In that version, the edges were white, which prevents us from coloring them.

Assessment: The contrast between the different elements is good (except for labels with nodes, which is mediocre), and the layout is visible enough. However, as we have seen before when edges are not colored, the color lacks room and is hard to read. The communities are hard to see, except for the highlighted one (but hulls do not scale unless you use small multiples).

DO: Color the edges to give room for colors to breathe (unless you have access to a better technique, like color spreads).
DON'T: Color only the nodes, as they do not provide enough space for color to allow identifying the communities easily.

Flat style with hulls, colored nodes and edges

If we use colored edges, we obtain this result (also reusing the corresponding settings from visualization in grayscale):

Assessment: The contrast of the different elements with each other and the background is good, bridges are well visible but the emphasis on the layout is mediocre, one cannot afford that many labels, and the communities are visible but not with a good precision.

Important remark: we have modified the colors to control precisely their luminance (perceived lightness). This is why nodes and edges do not have the exact same color. That color theory trick can be done automatically, with libraries like D3 for instance. The visualization works well enough thanks to this. However, it comes with a drawback. As the original colors are modified, it is possible that two distinct colors get changed into the same resulting color (or two very similar colors). Typically, a dark green and a light green may result in the same green. The differences in lightness will get erased while the differences in hue will remain. So this strategy works only if you pick colors the right way (i.e., with distinct hues).

DO: As a tool maker, if you use advanced color tricks, pick the colors algorithmically to prevent color conflicts.
DON'T: Let the user pick colors freely if it may lead to conflicts between the modified colors of the visualization.

Simple style with colors

We can also draw the network with color in even simpler ways, while still retaining the idea of controling color lightness. In the example below, nodes, edges, and labels are colored with curated levels of lightness. No hulls, just a straightforward rendering with controlled colors.

Assessment: The contrast between the elements is decent to good, the layout is decently emphasized even though the bridges are not that visible, we cannot afford too many labels because they compete visually with the layout, and the communities are visible but imprecise (since mediated by the edges).

The main drawbacks are the same as before: colors can only be differentiated by their hue, which requires picking them carefully; the drawing order of edges matters for arbitrary reasons; and the edges mediate an information that actually refers to the nodes, which is imprecise. However, I find the result more readable than with the hulls. The layout is easier to read, even though the bridges are highlighted less.

Takeaway: a simple rendering is relatively efficient and can be implemented easily, but it still requires controlling the colors to ensure appropriate contrasts. Mind randomizing the order of the edges to remove an arbitrary dependency to how the data was produced.

Color changes how we think of contrast

Comparing the last two rendering helps understanding how, when it comes to contrast, color is beneficial and detrimental at the same time. It is beneficial because it can bring some contrast. Typically, a colored node will be pop more over a gray background than its gray equivalent of same lightness. But using color also forces us to use a narrower lightness range, as neither black nor white can be colored. And color can also be distracting, and can quicky draw attention away from the layout.

One cannot simply take grayscale designs and color them; it gives some results but they require adapting. Indeed, new tradeoffs appear. In particular, we have less room for differentiating the layers by their lightness, but we can compensate this constraint by gaining contrast when color is over gray. And we have to emphasize the layout even more, because color draws attention away from it.

This last point deserves unpacking. One may think that color makes the layout more readable, as it emphasizes groups or other structures, depending on which node attribute it maps. True enough; but even so, it acts like a shortcut to the network structure. And like any shortcut, it makes you miss part of the journey. The structures emphasized by color are more readable, yes, but then all the other ones become less visible. It interferes with the balance of the graphics, shifts the reader’s attention, and impacts their interpretation. In the examples above for instance, we differentiate well between airports from the USA (green) and Canada (purple, both on the right), even though they basically belong to the same North-American cluster. But on the contrary, and that is the problem, it draws attention away from the fact that this cluster has subclusters for the West coast (on top of the cluster) and the East coast (bottom-left of the cluster). The color steals attention from the distribution of nodes. It provides ready-made groups, but it makes density-based groups (topological clusters) less visible; and those are the reason why we make a network map in the first place.

Takeaways

I retain three. First, simpler designs are more efficient, because color tends to complicate everything. Second, do not repurpose colored network maps as grayscale by removing color. If a journal requires a colorless illustration, instead of removing color in a picture editor, render a new grayscale visualization. Design for grayscale if you need to. Third, ensuring that the layout pops out is even more difficult when using color, so you need to care even more about preserving the topological affordances.

Color spreads

I believe that the simple colored design just above is a good compromise between ease of implementation and readability, despite a few flaws. For a really high quality result, we need something sensibly more sophisticated. In this section I explain a technique that I call “color spreads”, for a lack of a better term.

Color spreads: proof of concept

Here is a proof of concept (POC). In short, it is not the job of edges to spread the color of nodes, so we should use a dedicated technique to do that.

This POC has been assembled manually in Photoshop, using images already generated. It is also far from perfect, which hints at the fact that doing this well is not straightforward. But it gives a practical idea of my goal. I aim at compromising between the following features:

  • Colored clusters are visible at a large scale, because color spreads around the nodes (color needs space, as we have seen).
  • But the color spreads do not compete with the details at a smaller scale. They should be as unintrusive as possible.
  • The color spreads should be as true to the nodes as possible (the POC is bad at that).

My strategy is to draw a very smooth colored area around each node, and composite it with the rest of the image. It is not as simple as it looks.

Color spreads: naïve implementation

The most naïve strategy is to draw the nodes and then apply some massive blur. As a starting point, let us draw color spreads as bigger, lighter nodes. We will omit the blurring for now.

An obvious issue arises: the drawing order of the nodes matters. The nodes drawn last (on top) spread their color too much, around nodes that are not of the same color. Blurring the image would not solve that issue. The natural solution to this problem is to use a Voronoï partition of space, which would guarantee that each pixel gets colored like the closest node.

Color spread: Voronoï

Each node is guaranteed to bathe in its own color. Two things are distracting, though: the borders between cells, and the circular outer border with the background. In both cases, the contrast creates lines that draw attention to them. We will address each issue one after the other.

Let us address the outer border first by using a gradient.

Color spread: Voronoï + gradient

The gradient here is simply the distance to the closest node (the node that gives the cell its color). It is similar to what we have seen previously with node halos.

The gradient still “falls” quite abruptly into the background, which can be improved. Indeed, the gradient is currently linear. We can use a smoother curve, so that the transition feels more natural:

We used the linear version so far, but we will now use the smooth version for the gradient (see below)

The curve now makes the cells look smaller, which we can compensate by augmenting the radius of the Voronoï cells. We can also add a touch of blur to address the issue of sharp borders between the cells. We obtain a color spread that gives room to the color of nodes, remains (quite) true to the color of the closest node, but without adding distracting details, as it remains smooth and blurry.

If we simply composite this color spread over a detail-rich rendering from before, we see that it does not compete with the details. The result is also better than our proof-of-concept tinkered in Photoshop.

Assessment: The contrast of the different elements (nodes, edges, labels) with each other and the background is good. We can see which edges connect to which nodes (to some extent). The layout is well emphasized. The bridges are visible. We can read many labels. Communities are readable via colors, even though we could do it even better.

This result shows that this strategy works. However, as we have seen, the presence of color requires rethinking the question of contrast. For instance, we want to color the dots too. We can do better than just repurposing the grayscale rendering. Now that we have a color spread that works, we can rethink how the image compositing for color, using all the techniques at our disposal.

Rethinking the compositing for color

As the raison d’être of the network map is the layout, I want to compare my result to a baseline where the layout readability is maximal. As a starting point, one can render a simple visualization with just three layers: hillshading (in gray), color spread (composited as “multiply”), and nodes (in color, with a controlled lightness, that is pretty dark).

Compositing the color spreads as “multiply” is the simplest thing to do, as it basically works as if white were transparency. However, it also means that the color spreads make the background darker around the nodes (as it adds their darkness). It also means that dark areas make the color less saturated, more dull. The effect is subtle in this map because it is very light and does not have much details, but it would add up as we stack other elements. For that reason, I think we should do better. We can afford a better baseline.

We can have more control over the homogeneity of lightness and preserve saturation by using another mode of composition: “color”. That mode applies the hue and the saturation but keeps the lightness unchanged, offering a better control over contrast. This is what we obtain by just compositing with “color” instead of “multiply”:

The contrast is better controlled, but a new problem arises. As white areas cannot be colored (white and black have no hue), the color spread is inconsistent. We see white (and therefore colorless) areas on the north face of the hills (metaphorically). The solution is simple: we can add a very light gray to the background, so that the color can spread everywhere.

This result is smoother and we can then control lightness as much as we want. The result may not look too different for now, but it will matter increasingly as we add details.

This visualization is our baseline. The layout is well emphasized, the color has enough room to be breathe, and we have full control over the contrast. Our goal from there is to add details while preserving the readability of the layout. We want to allow following edges and reading labels from close, while keeping a good legibility of the topological structure from a distance.

Once again, we will be compositing layers one after the other. To make it clear how the compositing works, I will present each major layer one by one, and show how they stack. I write “major” because most of those layers are also composited from sub-layers, as you will shortly see.

Background layer

Our starting point is just a gray background with hillshading, with twist for the nodes. First, we paint the background in light gray, keeping the space for nodes white.

We then composite the hillshading on top (mode “multiply”).

We paint the nodes in white over the background to be able to composite them in “multiply” later on without loosing some lightness due to the non-white background. However, we want to keep the hillshading on top of those white spaces because else, the final result feels unnatural. Indeed, the hillshading acts like a shadow that we intuitively expect to apply to the colored nodes too.

Edges layer

The second major layer we compose is dedicated to the edges. It is itself composed of sub-layers. In short, we draw the edges in a relatively light gray, but with some space for the labels and the nodes in white. The goal is to composite this layer using “multiply”, so the level of darkness matters (white acts as transparency).

We composite this layer on top of the background layer:

And we then composite the color spreads in “color” mode, to obtain a background where edges are not too prominent:

As you can see, the hillshading emphasizes the layout well despite the edges.

Nodes layer

The nodes layer will be composited in “multiply”, as we once again want to use white for transparency. We draw the nodes using their color but controlling lightness so that it is dark enough (so that the layout is well emphasized). On top of the nodes, we paint labels in the background color (that will matter later on). The labels must not be painted outside of the nodes, which can be achieved by painting the non-node space in white afterwards.

Once composited in “multiply” mode on the precedent image, we obtain this result:

The clear dots we had left in the background layer ensure that we can control the nodes’ darkness independently of the background shade, with the exception of the hillshading, that applies in addition to that.

Labels layer

The labels are drawn on a white background for “multiply” compositing. To avoid distracting from the layout, we control both the lightness and the saturation of the labels. Each label has the color of its node, but with a limited saturation and a lightness barely above that of edges.

The expected result is to have the labels be seen as a gray texture from a distance, similarly to the edges, so that the layout remains clearly readable. The labels are therefore not expected to be as clear as they could. Yet from close, thanks to the white border cut from the edges layer, they remain readable.

Fully composited image

With the labels added, the image is fully composited.

Assessment: The contrast of elements with each other and the background is decent to good. We can track which edges connect to which nodes to some extent, the layout is well emphasized, bridges are visible, and we can read many labels. We can easily read the communities from colors.

Here is a side-by-side comparison with the baseline:

And here is a side-by-side comparison of the POC (Photoshop mockup), the simple compositing with a grayscale rendering, and our final result:

Takeaways

Color spreads keeps a distribution of color that is more true to the nodes, as it does not depend on the edges. An appropriate compositing allows controlling color (hue) independently of lightness, in order to find an optimal compromise between the readability of each element and the highlighting of communities with color.

If your rendering environment does not allow you this level of sophistication, a simple rendering style with colored nodes and edges works well enough, at the condition to control the lightness to support appropriate contrasts.

Either ways, the constraints are too strong to allow picking colors freely. Because controlling contrast forces us to rely on the sole hue to differentiate colors, this basically gives us about 6 to 8 different colors maximum, if we want to be able to differentiate them reliably.

DO: Limit the number of different colors to 10 or less, preferably 6 or less. One cannot distinguish more in this context.
DON'T: Try to colorize everything at the same time, because it is not readable anyway, so there is no point.

How to render a network map, part 2: grayscale

35 min read

In this series of posts, I roam and gauge the design space of network maps. In the previous post, I explained how to render a network in black and white. It was essentially an occasion to explore the contrast issues inherent to visualizing networks as dots and lines. I will now move closer to a realistic use case: rendering a network map without colors, but with levels of gray (aka grayscale).

One of my goals is to find the best rendering approaches for network tools. This series exposes my rationale so that it can be discussed and criticized, but I will extract do’s and don’ts and takeaways along the way, so that you can also get reusable, practical insights about visualizing networks.

The criteria I will use to assess and discuss each design are the following:

  1. Do the nodes contrast with the background?
  2. Do the edges contrast with the background?
  3. Do the labels contrast with the background?
  4. Do the nodes contrast with many underlying edges?
  5. Do the labels contrast with many underlying edges?
  6. Do the labels contrast with many underlying nodes?
  7. Do we see which edges are connected to a given node?
  8. Does the heterogeneity of the node distribution jump at our face?
  9. Can we identify the edge bridges between clusters?
  10. Context: how many labels can we read?
  11. Colored communities: can we see distinct groups of colored nodes? (typically in a situation of community detection)

Classic stack of layers

We can think of background, edges, nodes and labels as four different layers stacked on each other, in this order. This is how most network maps are built. We can leverage contrast to overlay each one over the precedent using a darker color, and it will remain separated. This is what it looks like.

Assessment: The contrast of each element with others (nodes, edges, labels, background) is good, except for the labels over the nodes, and the edges over the background, that have room for improvement. The layout is visible enough, and we have sufficient context (labels). The bridging edges are not too visible though, and we still cannot read which edges connect to which nodes.

Note: here I reused the settings from the last post (black-and-white rendering) and just added levels of gray. In particular, I used the strategy I had recommended for the labels: sparse enough, with controlled font weight.

No need for node borders, no need for sophisticated techniques beyond selecting which labels to display. The nodes contrast with edges, and their distribution is visible. The edges are visible but not distracting. The labels are readable. It basically does fine on every aspect except reading which edges connect to which nodes. This is the baseline for my exploration of improved designs.

DO: Stack layers with levels of gray staged from white to black (or vice-versa), especially if your rendering tools don't have complex options, as it can be surprisingly efficient and reliable.

Let me mention three minor flaws, though. First, the levels of gray have to be staged carefully. The edges can be very light, and the nodes have to be dark enough that their distribution pops out. Second, the labels may still compete with the rest if they are too dense, so it is still important to limit their number. Third, as we use the whole spectrum of grays, the picture gets easily damaged in high-contrast situations, such as poor quality printing (dark gray conflated with black) or video projection (light gray conflated with white).

DO: Pick the edges as light as possible (but still visible), and the nodes as dark as possible (but with labels still readable on top). Mind the target medium (screen or print). Keep labels as thin as possible (light font) and as sparse as possible.
DON'T: Set the edges so dark that the nodes are hard to spot in dense areas.

Classic stack of layers with emphasis on nodes

The baseline works, but it makes an assumption worth challenging. It assumes that labels prime over nodes. Yet as we have mentioned, they should not. Our top priority is to keep the spatial distribution of nodes emphasized; readable labels are less important. What are our options?

Option 1: nodes over labels

This is for the sake of the argument, as illegible labels are particularly frustrating. The reader will probably conceive this design as a mistake. Yet the result is surprising and interesting.

Assessment: Compared to the baseline (classic stack of layers), we lose some contrast of labels with nodes and, from there, some context (labels are less readable overall), but we sensibly improve the emphasis of the layout.

I find it counterintuitive: the distribution of nodes is emphasized better, and the labels remain largely readable. From a purely functional perspective, it is arguably better than the baseline, except for a major flaw: in situations where nodes are densely packed, the labels are not readable at all (cf. example below).

Because of this flaw, this solution is problematic in general, but may works surprisingly well in situations where the nodes are not too big and not too closely packed.

Option 2: labels on top but lighter

This solution is not the best but it is worth noting why. The distribution of nodes retains some emphasis, the labels are more readable than before, but they are still too hard to read, and some amount of occlusion still occurs.

Assessment: Compared to the option 1 (nodes over labels and darker), this option is essentially the same with a minor improvement of the readability of the labels on top the nodes.

DO: consider displaying the nodes darker than the labels, and possibly over the labels, in particular if the nodes are sparse and/or small.
DON'T: do that if the nodes are very dense, as the labels would not be readable.

Finding the right value for the labels is a Catch-22 situation. They cannot be too dark to avoid competing with nodes (it would obfuscate the layout) but they cannot be too light or they will not be visible over the edges. Ultimately, no matter what value you pick, they will tend to compete with nodes over the background and the edges, and yet occlude the nodes when they are on top of them. The settings in the example above work a bit better than having the nodes on top of the labels, because we can at least decipher them, but they remain too hard to read and we cannot really get anything better. The labels are basically as thin and light as we can afford. And if we make them darker or thicker, they will compete with the nodes (see example below).

In this rendering, the labels downplay the layout too much (and they are not even that much more readable).

Option 3: labels on top with opacity

This solution aims at keeping the labels darker when they are on top of the nodes, but not too dark when they are on top of the edges or background. This can be achieved if nodes are not fully dark, by leveraging the opacity (alpha channel) of the labels layer.

Assessment: Compared to options 1 and 2, we have improved the contrast of labels with edges, but we have damaged the contrast of labels with the edges and the background, and of edges with the background.

Some labels are hard to read but it can be done, and we see the node placement well. The gain over option 2 is not indisputable, and it requires a precise tuning. In the example above, the image looks too pale. The strategy (barely) works but remains fragile, as many things could easily go wrong.

We have not completely solved the Catch-22 situation of option 2. Pushing this strategy forward would require controlling two parameters independently. First, when over the edges and background, the labels must be as light as possible to be readable but without competing with the nodes. Second, when over the nodes, the labels must contrast with the nodes, using border, a darker color, or both. This cannot be done with opacity alone.

DO: carefully stage the lightness/darkness of the different elements to provide appropriate contrasts.
DON'T: use opacity, as it does not provide sufficient control over the contrast of the different elements, and will often create multiple unwanted side effects.

Takeaways about stacking layers

Stacking layers with staged levels of gray works relatively well for everything except reading which edges are attached to which nodes. The classic approach lacks emphasis on the node placement, and fixing it requires compromising with the readability of labels. I still recommend it: render the labels on top of the nodes, but feel free to set the nodes darker than the labels so that the labels compete less with the layout.

As a tool maker, I find the classic option of having labels in dark over nodes dangerous, because the legitimate desire to display more labels (more context) easily jeopardizes the clarity of the node placement. For that reason, I favor strategies that keep the placement readable no matter what. I acknowledge, however, that those strategies are not great. This is why I think it is worth pushing the sophistication further.

Compositing layers

From this point I will use a more advanced strategy. It will be familiar to graphic designers who use applications like Photoshop or drawing systems like HTML Canvas. Canvas calls it “composite operations” and the idea is to have different ways to stack layers. In a given mode, white on black may produce white, and in another one, black, and so on. We can use different operations to compose the layers; you may think of it as choosing whether you want to add, subtract, multiply, etc. the layers. I will use the Canvas names for reference.

This technology is available in many environments, and from the standpoint of a tool maker, implementing it is not too complicated. But for the user, mastering compositing requires graphic design competences, which cannot be taken for granted. I think that a tool maker using these strategies would have to curate the settings exposed to the user, and/or keep it as presets.

Split rendering of labels

Here we split the rendering of the labels in two distinct graphic situations: (1) over the edges or the background; and (2), over the nodes.

First, we ensure that the labels are readable over the edges and the background, but not too visible. I suggest picking a gray that is just slightly darker than the edges, and using a white border. As the label border ensures visual separability from the edges, we can make the edges darker than previously.

Note: to make composition easier, we render the nodes in white but on top of everything else (i.e., at the end).

DO: for the labels, pick a the level of gray such that once merged with the background color used for the label borders (when we look at the map from a distance) results in about the same gray as the edges.

Second, and separately, we draw just the nodes and labels such that the labels are well separated over the nodes, but without the nodes getting too light, so that they keep contrasting with the background. The label borders should be gray while the nodes and the labels should both be black for maximal contrast.

Then we crop this image around the nodes. In practice, here is what we do: we keep a copy of the image with just the white background and the black dots, and we composite it on top of the image above using the composite operation “screen”, which boils down to adding just the white. We obtain this:

Now, we simply draw this last image (nodes) on top of the previous image (labels) using the composite operation “multiply”, which boils down to just adding the blacks. We obtain the final composited image.

And as the labels do not compete with nodes anymore, we can crank their density up! Here is where we land:

Assessment: This rendering almost nails it! The contrast of each element with each other is good, the prominence of the layout is good, bridges are visible enough, the only issue is that we cannot read which edges are connected to which nodes.

With this strategy, the labels are mostly visible up close (zoomed in) while the structure of the network remains clear from a distance (zoomed out). Compositing is demanding, but it basically allows fine-tuning multiple aspects of the picture simultaneously and independently. This version essentially addresses all of our needs but one: reading which edges are attached to which nodes.

DO: composite your rendering if you can, so that the labels are visible up close but do not obfuscate the layout from a distance.
DON'T: obfuscate the layout with the labels, because we do not need to read most labels from a distance (but we need to read the layout).

Hillshading

To emphasize the distribution of edges even more, we can borrow the hillshading technique from traditional geography. In a nutshell, we infer an elevation of the visualization space from the density of nodes. You can think of it as a heat map or a kernel density estimation, which boils down to the same thing. This elevation can then be rendered as shadows, also known as hillshading. You can find an implementation in JavaScript in this notebook.

The hillshading technique emphasizes the node distribution and is a valuable visual help in situations where many graphical elements compete with the nodes. I will not provide technical details here (find them in the notebook mentionned above), but highlight how it can be composited.

Here is the hillshading alone:

In this case, we can simply composite hillshading over the background layer (everything outside of nodes) in “multiply” mode.

Assessment: Compared to the version without hillshading, it slightly improves the readability of the layout.

The hillshading is so smooth that it does not compete much with the nodes even if it makes the overall image darker. At a distance, it emphasizes areas with densely packed nodes without being too intrusive. It helps guide the reader’s attention toward the most important thing, the spatial distribution of nodes.

Node halos

Let us tackle the last issue: how to see which edges attach to which nodes? The technique I propose makes room around each node where we can see only the edges that attach to it. I will describe how it can be done, then show the result, then discuss it.

The idea of the first step is basically to compute a Voronoï diagram. For each pixel or point of an arbitrary grid, we compute which is the closest node and what is the distance to it. We will tweak that in two ways, but let us keep it simple for now. If you just plot the distance to the closest node as a gradient, you get this:

It looks like little balls glued together, and for good reason. Two important remarks. First, we see very well the delimitation between areas that “touch” each other (nodes that are close to each other). Not all the cells (the “balls”) are round, as when they get densely packed, they get flattened by the neighboring cells. This information is key to drawing node halos properly for densely packed nodes. Second, the size of the balls is a parameter that has to be set. It will ultimately determine the size of the node halos. It has to be set by the user, but in a nutshell, it should have the order of magnitude of the smallest nodes’ size, which should also be that of the spacing between nodes.

In practice, we will tune this Voronoï in three ways. First, we will take into account the node sizes, so that the distance is computed to the border of the node instead of its center.

Second remark, we should tune the gradient so that the white is more visible. The white represents the halo, and the lighter it is, the more the edges belonging to that specific node are distinct from the other edges. That is an optional refinement that makes the result more natural in the end. In short, we need to see the edges connecting to each node, which requires very white halos.

Finally, we set the size (spreading) of the Voronoï cells so that for about every node except the most packed ones, the gradient of the cell drop is visible. The cells (white areas) will become the node halos. If they are too big, the pattern will become distracting. If they are too small, it does not serve its purpose (making it clear which edges attach to each node).

The next and final step is to draw the edges using the Voronoï data computed above. The strategy is the following. Each edge is converted into a path consisting of a series of points. If the edge is a line, we must not compute it all at once but bit by bit. The points must be close enough to roughly match the grid size used to compute the Voronoï. For each point of the edge, we compute the opacity from the Voronoï data, using the closest grid point or something more clever like an interpolation of the 4 closest ones. In short, the point should be opaque in black areas and transparent in white areas (in the Voronoï plot above) except when the closest node is the source or the target of that edge (that is why we must also store which is the closest node). Here is the result:

The technique is compatible with simple approaches as well as the most sophisticated ones. Here it is integrated with our baseline (classic stack of layers):

Assessment: Compared to the classic stack of layers without node halos, we sensibly improve the clarity of which edges are connected to which nodes, even though it is not perfect. This also helps following the bridging edges. There are no downsides.

And here is what it looks like with compositing and hillshading:

Assessment: Similarly, node halos make it much easier to understand where edges connect to nodes, with no downsides. Recap: the contrast of nodes, edges, and labels, with each other and the background, is good. The layout pops out very well. We can read many labels. We can see and follow the bridges well, and we can decently understand which edges connect to which nodes. It essentially checks all the boxes (aside from colored groups).

Note: In this case I have displayed many labels, and as they have borders, the node halos are quite often hidden behind the labels. We have added so many details that the visual competition becomes a concern again. This is why I assessed the ability to see which edges connect to which nodes as only decent, but we could get play this tradeoff differently by displaying less labels.

Discussion about node halos

Node halos are not intrusive. If you don’t understand what they are, you can basically ignore them. They will not cause the image to be significantly busier. But I think that they are in fact pretty intuitive, especially in areas where there are not too many nodes and not too many edges.

Hypothesis: node halos just feel natural in sparse areas.

However, as soon as you want to read node halos, the relevant space of a node is more than just the dot (and label), but also its surroundings (the halo). As a consequence, and even if the image does not get much busier, there are now more parts of the image that one does not want occluded. As a consequence, the competition for space (if not for attention) increases between the graphical elements.

DO: Set the layout algorithm so that close nodes have a constant margin between them, and set the range of the node halos so that it matches about the half of that distance.
DON'T: Let nodes overlap, as node halos will not help you there.

Even when they are not occluded, the node halos do not provide a perfect information. Indeed, it remains hard or even impossible to track any given edge. The key insight they provide, however, is to clarify the “reason” why each node has been placed where it has been. Indeed, some nodes are under the influence of many neighbors, which means that their position is determined by the general structure of the network. Those nodes appear as dots with many lines connected to them, and their halo is barely noticeable. On the contrary, some nodes are under the influence of just a few neighbors, and their position is determined at a local level. Such nodes may be placed in the middle of the picture, but only because they are neighbors to nodes that are structurally central. Their situation is that of a small satellite of a big planet. Those nodes will have a clear halo around them, because they do not have many edges, and we will be follow their edges easily. More generally, the halos make the local structures visible, and help make the distinction between large-scale patterns and small-scale patterns.

The big node’s placement is determined by the global structure, while the three small nodes’ placement is fully determined by their connection to the big node (local structure).

Note: node halos sometimes create a 3D-like antipattern with big, dense clusters. This comes from the fact that the nodes on the side of the cluster have most of their edges directed towards the inside of the cluster. As a result, node halos get more of the background color turned towards the outside of the cluster. This creates an unwanted illusion of depth. One may find it aesthetically pleasing, but it remains undesirable because it warps the perception of distances. The layout algorithm was applied in two dimensions, not three.

Node halo sometimes create a 3D-like antipattern with clusters. Example of a network map that I have rendered for the Tantlab a few years back (Cultural life on the Danish Facebook).

Grayscale, flat design, with hulls

I will now explore a different design direction, and for simplicity I will keep the design flat: no gradients (hillshading), and no advanced compositing. Just layers. So far we have used 4 layers: background, edges, nodes, and labels. We have seen that their order matters, and their contrast also matters. I will consider two new things: using different ways to organize the layers by contrast, and adding new layers to highlight the shape of the network and/or clusters (set of nodes).

Network hull

What I call here a network hull is a shape surrounding all (or most of) the nodes of the network. There are different ways to compute such hulls, and I will just stick to this one here: using a threshold on a heat map of the nodes (the same heat map as for hillshading, which you can also think of as a kernel-density estimation).

Hulls provide a form of highlighting. They are mostly relevant for clusters. They do not work very well if the highlighted nodes are disseminated everywhere. They work best when the highlighting nodes are packed together in one or more places.

The hull of the entire network can be used to highlight its structure with more or less subtlety:

Assessment: Compared to classic layers with black nodes but without the network hull, we just gain a bit of emphasis on the bridges, because the edges are now more visible between the clusters.

With the settings above (based on classic layers with black nodes), I retain three effects on the image.

  1. The network hull draws attention on the presence of nodes, not edges. Which is a good thing, as edges are baked into node positions.
  2. It lowers the contrast of edges above the hull (within the hull). As those edges tend to be the short edges, and those are accurately represented by the layout, it is not a big problem. On the contrary, it makes a less busy background to nodes and labels.
  3. By comparison, it highlights the edges outside of the hull, which are the long ones, and the less accurately represented by the layout. Those edges are bridges, and they have a special role in the structure. Highlighting them is beneficial.

Used this way, the network hull does not improve the visualization by a lot, but it can be useful. It also depends on how the contrast is articulated between the different layers, which we will explore.

As the network hull is much less busy than the full visualization while retaining information about its structure, it can be used by itself or with just the nodes to provide a simpler view on the structure. This can be used for mini-maps or small multiples (an example will follow).

Cluster hulls

Hulls can also be used to highlight groups of nodes (generally, a cluster). Here I will pick a single subset of the nodes to highlight with a hull: the airports situated in the USA (reminder: this is a network of airports connected by airlines). I picked settings that make the cluster hull fit nicely within the entire network’s hull. But of course, depending on the situation, you may want to use hulls differently. You may want to display multiple hulls at once. I will not explore this here as it depends too strongly on the shape and distribution of the clusters. I will stick to this single hull:

Possible misinterpretation: this looks like a heat map. One may understand that the highlighted cluster is somehow denser or higher on some unspecified scale, which is not the case.

It is worth noting that the hull shapes can be rendered as lines for a different compromise. Lines are more distracting than flat surfaces, but they obfuscate less.

I recommend to keep the network hull as a surface and the cluster hulls as contours, to emphasize the fact that highlighting a cluster is essentially an annotation. In this case, one draws the network hull at the bottom and the cluster hulls on top. Here is an example as a small multiple, in inverted colors for the sake of variety:

DO: Favor a filled hull for the network and contoured hulls for additional annotations (cluster highlight).
DON'T: Stack cluster hulls if it makes the map confusing or cluttered (use small multiples instead).

The design choices worth exploring from here are: in which order to draw the layers, how choose the right darkness to obtain appropriate contrasts, and when to use lines or surfaces for the hulls? There are more possibilities then immediately meets the eye.

Light edges

The priority is still the readability of the layout, therefore the graphical elements that need contrast the most are the dots representing the nodes. Those should be either black or white. I arbitrarily pick black here, but the rationale will also work for inverted images (negatives).

The dots need to be visible above their background, which may now consist of the edges, the network hull, or cluster hulls. As a result, none of those should be too dark.

The background color does not matter anymore in this case, as nodes are never above it. It could even be completely black. Let us put aside labels and edge readability: the layout remains readable even if everything else is in gray, even a relatively dark gray. Here is the proof of concept:

Assessment: The contrast of edges with the background (including the network hull) is very bad, as we do not see the edges over the hull. And when it comes to see which edges connect to which nodes, it is as bad as if we did not even have the edges. Yet we have three good points: a good contrast of the nodes with the edges, with the background, and the layout is decently highlighted. We also have a glimpse of where the bridging edges are. Despite all the flaws and the intentionally too-dark gray, the most key criteria are covered acceptably.

Building on top of this proof of concept, we can make observations that help us organize possible designs:

  • As long as the network hull and the edges have not-too-different colors, the layout remains readable.
  • Although distinguishing edges over the network hull is useful, the most important is to have contrast outside the hull (where some edges bridge across clusters)
  • As the network hull and the edges could get quite dark, we could then go for lighter labels and cluster hulls.

The key factor is the lightness of the edges. The most obvious choice (it will be our baseline) is to keep the edges pretty light. Then the network hull should also be pretty light. The background color should be contrasted enough, which leads us to a darker gray. Labels should be even darker so that we could read them well above edges and the network hull but keep them readable over the background. Here is an example:

Assessment: The contrast of the different elements is decent to good, except that of the labels with the nodes which remains poor. The layout is well emphasized, the bridging edges are readable, and we have a number of labels, although we cannot afford too many. And without node halos, we do not see which edges connect to which edges in general.

Remark: we can afford to draw the edges quite thicker, as in this context we care more about the long edges and the general structure than the small edges and the local structure. In a nutshell, for dense areas, we rely on the layout to convey the key information: within that cluster every node is more or less connected to every other one.

In the same spirit as this visualization, we could have the edges completely white. In that case the network hull should be very light and the background even less light.

Assessment: Like the previous rendering, but with an improved contrast of edges with the background.

This version works better because the lesser contrast between the network hull and the background is less distracting.

Remark: this strategy boils down to the Grandjean trick, but inverted (see previous post). The hulls help warrant that the background color around the nodes is consistent. Like for the black and white version, network borders are beneficial. Here is an inverted version of the same image:

Dark edges

Conversely, one could keep the edges quite dark with black nodes. The network hull should then be quite dark too. Light tones can then be used for labels and cluster hulls.

Assessment: The contrast of all the elements with each other and the background is good, except that of the edges over the background, which is disappointing. The layout and the bridges are well emphasized. But we have to keep the number of labels low, and we do not see which edges connect to which nodes.

We lose something on the way: the dots and the labels are no more of a similar color, so we lose the sense of association between them; labels might be misunderstood as annotations. But we also gain contrast between the label and the node, which makes labels more readable.

This works insofar as we can keep a fragile balance between the white (labels), the black (nodes) and the gray (the rest): the layout is manifested through black and gray while the white makes a more homogeneous layer. The reader must somehow accept to forget about the white layer when it comes to reading node densities. The highlight of the structure provided by the network hulk does help in this context.

DON'T: Use a very different color for the nodes and the labels if it makes it look like the labels don't belong to the nodes (it notably depends on your other annotations).

Gray nodes

In anticipation to adding color to the visualization (the next post in this series), it is worth considering gray nodes. Indeed, the lightness of any saturated color can neither be null (black) nor maximal (white). I will then use mid gray (50% lightness), and I will also arbitrarily set labels to black, on the motive that white labels are equally valid but can be derived by just inverting the resulting image.

The main constraint remains to keep the layout readable while the edges are gray. Their background has to provide contrast, therefore the network hull and the edges should be as light as possible. Since edges create a busy background behind the nodes, we should get more contrast in that case. As labels are black, edges have to be white and the network hull light gray, and then the background should be a less light gray.

Assessment: The contrast of all the elements with each other and the background is good, except for the labels with the nodes, which is poor. The layout and the bridges are decently emphasized. But we have to keep the number of labels low, and we do not see which edges connect to which nodes.

The result is quite similar to dark nodes with white edges, but the layout is less readable because the contrast of the nodes is not as good. It is not the best choice for actual grayscale network maps, but provides a baseline for color maps.

Gray nodes and edges

For reasons that I will develop in the next post of this series, we may also want to use color on edges, which means that they cannot be white. With this constraint, nodes have to be dark gray and edges light gray. We can then use white for labels, as long as we avoid very light tones for edges, the network hulls, and to a lesser extent, the background. Here is where we land.

Assessment: The contrast of the different elements with each other is decent to good. The layout is decently emphasized, and the bridges are visible enough. We cannot afford too many labels, and we still do not see which edges connect to which nodes.

Once again, the purpose of this exercise is to calibrate the contrast in anticipation of using color, not to produce an optimal grayscale image.

Takeaways about grayscale network maps

Your expectation should depend on the sophistication of your rendering system. If your environment is simple, then you will not have access to hillshading, node halos, and possibly hulls. You cannot expect to solve certain issues, but you still have multiple options to craft a readable network map. If your system is sophisticated, then you can afford to find the tradeoff that works best for you and craft a network map that conveys more context and a more accurate information.

If your system is simple, you cannot clarify which edges connect to which nodes. You should then aim for a well-staged contrast between the different elements, which is quite easy to obtain in different ways. I still recommend to prioritize the emphasis of the layout over the labels. You could have a stack of white background, light gray edges, black nodes, and dark gray labels; or the inverted version. But you could also stack a gray background, white edges, black nodes and dark gray labels; or the inverted version. If you do not need to display too many labels, you could even stack a light gray background, dark gray edges, black nodes and white labels; or the inverted version.

If you have access to node halos, you should use it as it improves the visualization by a lot with minimal downsides. The part of your audience that is expert at reading networks can get a lot of important information from the details of which edges connect to which nodes, even if it is not perfect.

If you have access to compositing options, I recommend that you tune your visualization so that the labels are visible from up close without drawing attention from a distance, which corresponds to what I see as the typical use case. However, this does not apply to situations where zooming is impossible (ex: small printed image).

Hulls have multiple applications, essentially as different ways to annotate the network. If you do not have access to color (some newspapers and academic articles), making small multiples with hulls is a good replacement to representing clusters with colors.

Finally, the hillshading is not the ultimate solution to emphasizing the layout, but it can help in many situations because it is not too intrusive.

>> Part 3: color

How to render a network map, part 1: black and white

25 min read

In this series of posts, I roam and gauge the design space of network maps. The series presents and discusses common solutions to the main readability problems that arise when visualizing large networks (1000+ nodes) as dot-line charts, typically based on a force-directed layout or equivalent. The series is aimed at helping multiple publics: beginners who have to choose the settings of a network map (in Gephi or another tool), tool-makers whose tools have to generate network maps, and graphic designers who visualize networks.

One of my goals is to find the best rendering approaches for Gephi and Gephi Lite. This series exposes my rationale so that it can be discussed and criticized. But I will extract do’s and don’ts and takeaways along the way, so that you can also get reusable, practical insights about visualizing networks.

My choices are opinionated. In particular, I will prioritize what I consider the raison d’être of network maps: the layout. I will always aim to have the layout jump at the face of the reader. My criteria for readability are partial. Let me explain in the next section, or jump ahead to the illustrated examples.

Criteria used to assess the readability of a network map

For the sake of the discussion, I will assess the network maps according to a non-exhaustive list of 11 criteria. I selected those because they capture the main practical problems of visualizing a network map. The problems we meet are of two kinds: contrast (legibility of the elements) and mediation (validity of the message conveyed).

My criteria

  1. Do the nodes contrast with the background?
  2. Do the edges contrast with the background?
  3. Do the labels contrast with the background?
  4. Do the nodes contrast with many underlying edges?
  5. Do the labels contrast with many underlying edges?
  6. Do the labels contrast with many underlying nodes?
  7. Do we see which edges are connected to a given node?
  8. Does the heterogeneity of the node distribution jump at our face?
  9. Can we identify the edge bridges between clusters?
  10. Context: how many labels can we read?
  11. Colored communities: can we see distinct groups of colored nodes? (typically in a situation of community detection)

Contrast issues (criteria 1-6)

Network maps generally have:

  • A background
  • Nodes
  • Edges
  • Labels

I will focus on those four elements, and I will also consider what happens if edges are not displayed. But I will omit other things that may also have to fit, such as annotations.

Obviously, nodes, edges, and labels have to contrast with the background. But there is more to it. New problems arise in specific situations, for instance if there are so many edges that they act like a new background. We must also consider the contrast of nodes and labels over edges, and of labels over nodes.

Mediation problems (criteria 7-11)

Reading the map of a small network is about following the paths. The reader wants to know who is linked to whom. A problem arises when a line intersect a node, because there is an ambiguity about whether or not the node is connected or just happens to be over an edge. This leads to path-reading mistakes.

Conversely, reading the map of a large network is generally not about following the paths, because there are too many edges anyway. Here, I assume that the visualization mediates the topology via the layout, rendering communities as visually dense node groups (clusters). The edges are implicitly encoded into the clusters, i.e. the spatial distribution of the nodes, also called the embedding. The main threat to appropriately understanding a network map is the obfuscation of its node distribution. The layout must be prominent and readable, as it is the main feature used to interpret the topological structure of the network.

It is therefore worth remarking that there is a tension between the readability goals of, on the one hand, the nodes, and, on the other hand, the edges and the labels. The node clusters are better visible when there the visual space between the clusters is empty. This negative space makes the structural holes visible. Yet on the other hand, edges precisely occupy the space between the nodes, filling this empty space. Worse! As labels cannot overlap, they tend to occupy space homogeneously, obfuscating the heterogeneity of node distribution. The typographic gray of the labels may hide the layout, which we want to avoid at all costs.

The labels obfuscate the distribution of nodes, which we want to avoid at all costs.

Then, there is a secondary problem. Even for large networks where displaying edges is less useful, it may be important locally. The typical situation is the presence of a bridge between two or more clusters. If the edges are not displayed, analyzing the topological situation of that node is difficult. Compromises are needed.

Finally, I want to mention a problem that I will not address here. Since the principle of the layout is to bring connected nodes closer, edges should be as short as possible. In some sense, they are, but it’s impossible in practice to make all edges short, so some edges have to be “sacrificed”: they will be allowed to be long so that even more edges can be short. To assess the success of the layout, one should look at the short edges. Unfortunately, short edges are the less visible, because they occupy less graphic space. It is the long edges that attract our attention, but those are not necessary the most important ones. It depends on your questions: long edges are those we want to see if we look for bridges, but not if we look whether where the layout manifested topological clusters.

The case: Airlines

In this series, I will visualize a single network: airports connected by airlines in 2021. It comprises 3290 nodes and 37411 edges. The layout, computed with Force Atlas 2 in LinLog mode, is always the same. The edges are weighted and directed, but I will ignore it entirely in this series. I do not deny that this aspect is very important, but I left this question aside precisely because there is already a large discussion of this aspect of network visualization.

I will most often offer three views of the visualization, at three different levels of zoom, and on different parts of the structure. They will always be the same so that you can compare. Click on them to check the full resolution.

The full map is rendered for a poster of 1 meter by 1 meter (the same surface as a A0 paper sheet) printed in 300 dpi, which also corresponds to a reasonably zoomable on-screen image (11811 x 11811 pixels). The details are rendered to be visible from very close or zoomed-in (smallest font size: 8 pts) while some patterns should be visible at a distance or zoomed-out (largest font size: 24 pts).

Black and white network maps

Here I look at the simplest case: black, white, and no levels of gray or colors. I will generally assume that the background is white, but a black background is just the same thing in negative. My points about contrast implicitly apply to the inverted images.

The priority is always to have nodes visible over the background, so nodes have to be, somehow, black. They might have a black border and a white fill, but I will start by considering them simple black dots.

Just black dots

It may surprise you, but displaying just black dots captures the essence of a large network visualization (the layout), although it does not work well for a small network (because we do not see the paths).

Assessment: nodes contrast well with the background and the layout is quite prominent, but we don’t see where the bridges are, and without the labels, we lack context.

DO: place and size the nodes so that we can separate them visually.
DON'T: make the nodes so big that they overlap and get undistinguishable.

When nodes are too big, dense areas get confusing:

Note: in this case, the bigger nodes make the layout pop out even more, which is good. But there are other ways to do so that do not have the drawback of amalgamating the nodes.

DO: place and size the nodes so that we see where they are.
DON'T: have some nodes so small that non-empty areas look empty.

When nodes are too small, the reader has to focus in order to find where the nodes are:

Black dots + lines

To see edges over a white background, we have to draw them black. The most obvious choice is to draw the nodes in black. The straightforward implementation works for small networks, but not for large ones, because the nodes become invisible over many edges, and because one does not see exactly which edges connect to which nodes. Edges also downplay the layout. Check it out:

Assessment: nodes do not contrast well with edges, we do not see which edges are connected to which nodes, and the layout is not that clear although we now see the bridging edges. Let us explore different ways to solve these issues.

Solution 1: very thin edges

The simplest solution is to have the edges so thin that they never compete with the blackness of the dots. This options is not always available, either because the printing process is not precise enough, or because there are just too many edges.

Assessment: nodes now contrast fine with edges, and the layout is more visible.

This solution is only as good as the gray texture resulting from the overlapping of edges remains light. The example above is acceptable but not great. In practice, the visual separation of black dots over the lines can range from the worst to the best. This solution is valid but not universal.

Solution 2: white edge borders

This solution is always available, and guarantees that edges never paint the background entirely black.

Assessment: it improves the contrast of nodes with edges, the readability of the layout, and slightly helps to see which edges connect to which nodes.

I like that this trick is not intrusive for sparse networks; it may even help! However, it has flaws. First, the texture obtained gets distractingly busy in dense areas. Second, the edge drawing order now matters. I recommend drawing shorter edges on top (last), because long edges have less chances to get entirely obfuscated. Third, it gets harder to track edges through dense areas (because the borders ultimately hide a lot of information). Besides those flaws, it does not improve much reading which edges connect to which nodes, because edges connected to a given node can easily get obfuscated.

Solution 3: white node borders

It specifically fixes the contrast issue of nodes over many edges. The node borders must be thick enough to ensure that they pop out even at a reasonable reading distance.

Assessment: it only slightly improves the contrast of nodes with edges.

This solution has multiple flaws. First, it becomes less obvious that edges connect to nodes, as it seems that they pass below nodes. The visual pattern is ambiguous. Second, the nodes over moderately dense areas where the edges make busy visual patterns, the nodes are not visible enough. The border only helps when the edges are so dense that it gets completely black. Third, and as a consequence of the previous point, we do not improve the sense of the spatial distribution of nodes. That is the main problem, and it does not fix it. For those reasons, I recommend against this approach.

Large borders improve the readability of the nodes and their distribution, but you may run into problems as soon as the borders get as large as the gaps between the nodes, as nodes will now overlap. Check it out:

The proper implementation is to draw all the node borders, then all the nodes without border, on top, as two separate layers.

Solution 4: very large node borders

With this solution, the node borders can become arbitrarily large. Functionally, they become something else. From there, I will call those large borders “halos”. For reasons that will appear later, I will even precise that those are “flat halos”. In short, halos can have a gradient which greatly helps with readability, but that is not available in the pure black-and-white case that we are considering for now.

Assessment: The contrast of nodes with edges gets good, and the layout gets more readable, but reading which edges connect to which nodes gets from bad to worse.

These flat halos can get arbitrarily large, which means that we can preserve the readability of the layout as much as we want, at the detriment of the edge readability. Indeed, not only do we lose clarity on which edges connect to which nodes (which was already problematic), but we also lose the general feeling that edges connect to nodes in general. The nodes seem to float over the edges.

That being said, this solution is compatible with the solutions 1 and 2. It is notably consistent with edge borders, but combining them is not that productive. Let us see what it looks like.

Solution 5: edge borders and very large node borders (flat halos)

It makes sense to combine node borders and edge borders, but since they have a major common flaw, reading which edges connect to which nodes, which does not get fixed.

Perspective on the contrast problem

At this point, if your intuition is like mine, you probably think: we should just extend the edges that connect to a given node through its halo! This way, we would conversely improve the readability of which edges connect to which nodes. And you would be right. In black and white, it looks like this:

Assessment: the contrast of nodes with edges gets good for most nodes (but not the most connected) and it becomes suddenly possible to get a pretty good sense of which edges connect to which nodes.

This technique is very promising, but I will not explore this direction for now. Indeed, to put it simply, a good implementation is complicated to design due to non-obvious issues. We will that technique anyway, and it makes more sense if one can use levels of gray. In an actual use case, for instance a figure in a black-and-white academic paper, it would make much sense to reduce a high-contrast but gray image to a black-and-white version through techniques like dithering. So for now, I am just acknowledging that the technique is a promising direction to explore later on.

White dots + black lines

Let me introduce a non-obvious approach known to me as the “Grandjean trick”, because I learned it from Martin Grandjean. On a white background, we have to draw the edges in black. But we can draw the nodes in white, provided that they have a black border:

Assessment: Compared to black dots and lines, we get an improved contrast of nodes with edges, although we lose some contrast of (disconnected) nodes with the background. The layout is also more readable.

In empirical networks like this one, we often have a few nodes that have many edges, and many nodes that have a few edges (aka a power-law distribution of the degree). It is also a common practice to set node sizes so that nodes with many edges are bigger. When that is the case, the Grandjean trick has multiple upsides. The large nodes are generally over a dense background of edges, precisely because they are the most connected. They are visible because they appear as big white dots over a quasi-black background (made of edges). The small nodes, on the other hand, are on the normal white background because they are poorly connected. But being small, their border matters more than their surface, and their border is black, so they remain visible. The border-to-surface can be set so that it compensate the kind of background where nodes lie, depending on their size.

That being said, there are some flaws to this technique. Mid-sized nodes are not very visible. The distribution of edges is more visible than the distribution of nodes, which is misleading (as it depends on it). And the need to read which edge connects to which node remains unaddressed. Yet the trick excels at making cluttered networks readable, which is invaluable.

The Grandjean trick is so easy to implement that basically every graph drawing system allows it. The only two points of vigilance are that it works only when node size is at least roughly aligned with the degree, and that the border size must be approximately the same as the size of the smallest nodes.

White dots + black lines with borders

The point of the Grandjean trick is to have the big (hyperconnected) nodes emphasized in white against the blackness of the underlying edges; but the point of edge borders is to mitigate that blackness. It is therefore an unlucky combination. Here is an example with relatively thin edge borders and edge thickness consistent with node borders. The idea is to emphasize the connection between nodes and edges. It still works poorly, mainly because the distribution of nodes remains relatively hard to see.

DON'T: use white nodes over edges with borders.

Takeaways about edges

Before we explore how to render labels, let us recap what we have seen about edges in black and white.

Drawing edges as thin lines is the best solution if available. The goal is to draw edges thick enough to see every edge but thin enough that the accumulation of edges does not become too dark, as it must not hide the distribution of black dots. It is not always possible, either because the network is too dense at places, or because the medium prevents drawing lines thin enough (low quality print).

If thin edges are not available, the next best thing to do is to use the Grandjean trick. Drawing edges with a border is never as good, as the pattern it creates is so distracting. The Grandjean trick is resilient, and works best with complex networks (they have very dense areas). It is the safest bet when thin edges don’t work.

The problems with labels

Labels come with two important problems: labels occlude each other, and they occlude the nodes. Those cause two different issues that we will address separately.

The second issue is the most critical, as hiding the layout makes the network map pointless. But solving the first issue will help with that. So for now, let us focus on the first issue, labels obfuscating each other.

In general, one cannot display all labels and have them readable at the same time. Indeed, the smallest nodes tend to be closely packed, and the labels would have to be very small to not overlap. I will acknowledge that it can still be done for vector rendering (PDF, SVG…) opened on a computer, because we can zoom in as much as we want; but vector graphics break when there are too many edges, among other problems.

In this piece I focus on a generic use case, where the output image can be printed or visualized on screen. I calibrated the rendering to a regular poster size, where the smallest labels can only be read from pretty close (font size 8 pts) while the biggest labels can be seen at a distance (font size 24 pts).

Black dots + all labels

Let us start with a naïve rendering to address the issue.

As you can see, the overlaps are absolutely massive. The labels would have to be much, much smaller to avoid overlaps; so small that one could not read them even on a very large poster.

DO: if you need to, pick the most important labels and display only those.
DON'T: display all labels, as they would generally overlap massively.

Precisions: following a common practice, I set the labels bigger for the bigger nodes; I drew the large labels on top, because the purpose of making some elements bigger is to emphasize them; I also used a white border around labels to separate them visually, which is strickly better than no borders. I see those choices as non controversial.

Depending on your tools and methodology, you may find a way to only display the labels of certain nodes. They may be manually picked, or filtered down following a criterion (ex: top 100 nodes by degree). This does not guarantee the absence of overlap, but may mitigate the problem sufficiently for all practical purposes.

Yet the apparatus should be helping. From the standpoint of the tool maker, we should be alleviating this burden if possible. Indeed, there are algorithmic ways to address the problem of label collision. I will address that aspect here.

Black dots + some labels

From there on, I will make use of a slightly advanced technique. I will detect collision between labels, and simply omit the smaller label when it collides with a bigger one. This ensures that the nodes that require emphasis get their labels displayed in priority. I described the technique and proposed an implementation in this Observable notebook.

The labels do not obfuscate each other anymore, and the dots are now visible. Which leads us to the second important problem: labels obfuscate the dots.

The problem gets worse as labels get packed more densely, as in this picture:

Even though the labels do not overlap, we do not see the dots very much. The issue is, in fact, twofold. First, we do not see some of the dots because they are covered by labels; but the labels still tell us that there is something there. Second, the labels compete visually with the dots. In short, it looks like every label is hiding some nodes, even when it does not.

The effect is not obvious because the general shape of the network remains visible. But densely packed labels create a texture that is gray on average and hides an important information about the density of nodes. We do not distinguish anymore between moderately dense and very dense areas. All areas with nodes look similar.

The effect can be made visible by blurring the image, calibrating its contrast, and posterizing it into a fixed number of levels (here, 8). Functionally, this boils down to a kernel density estimation of the black pixels rendered as a stepped heat map. In other words, the blacker an area, the denser its surroundings. If we apply this process to just the black dots, we see that some large areas remain gray: they are less dense than the black areas. If we apply the same to the image with the dense labels, we see that every area gets about the same level of blackness. The nuance about density has been lost. We also see that the clusters have bled left and right; naturally that is because labels expand horizontally (in this case).

In black and white, we can mitigate this effect by lowering the label density. In practice, it can be done by offsetting the hit boxes of the labels during collision detection. We have to significantly lower the number of displayed labels if we want the nodes distribution to be effortlessly visible. And despite this, the labels will keep competing with the nodes visually to some extent.

The takeaway is simple: as soon as labels visually compete with dots, the spatial distribution of nodes, which is the most important thing, gets less readable.

DO: keep enough space between labels so that the layout of the underlying nodes is clearly perceptible.
DON'T: cram so many labels that they obfuscate the layout.

About font weight

To improve the situation, I like to use an opinionated trick. It is worth sharing here, as it helps mitigate a little more the occlusion of nodes by labels, even though it sensibly complicates the implementation of label rendering. I generally like to compensate font weight across sizes so that every label seems to be drawn with the same pen. In other words, large labels get a lighter font and small labels get a bolder font. As large labels take up more space and often lie above dense areas, reducing how much they hide has some impact. It also ensure a better graphical consistency. However, it also makes the large labels pop out less, which might be an issue.

Assessment: The contrast of nodes with the background, of labels with nodes and with the background, are good. The layout is visible enough, and we can read enough labels to get some context, although the amount of text information has to remain limited.

I favor this approach because in a practical situation, I often prefer keeping the labels dedicated to close reading and have other layers dedicated to distant reading, like very large annotations of the main clusters.

It is also worth mentioning that the technique is far from obvious to implement, if only for the reason that it is not font-agnostic: different fonts stage their actual thickness differently across weights. Variable fonts solve this, but that technology is not available everywhere. So unless you engage with advanced scripting, you will probably have to overlook this trick.

Black dots, thin edges, labels

We will now combine edges with labels. As we have seen, we should consider two situations: do edges accumulate to the point they become a black mass, or do they not?

Let us assume first that we can draw thin edges that never get so dark that we do not distinguish the nodes. Following the same logic, we want to minimize the occlusion caused by labels, and keep the spatial distribution of nodes readable.

Assessment: The contrast of nodes, edges, and labels with the background is perfect. That of labels with nodes is good, and that of nodes with edges is good enough. However, the contrast of labels with edges is disappointing, as their textures are somewhat similar. The layout’s clarity is also disappointing, as so many elements compete with it. At least the bridges are visible, and we have some context from the labels, which still makes it our best visualization so far.

In our case, the result is acceptable but not great. In particular, the white border of the labels now makes them pop over the dark texture of edges, which creates an unwanted distracting pattern. This approach would work better on a slightly sparser network.

White dots, edges, white labels

We now assume that there are too many edges, and they will agglomerate into a black mass no matter what. We pull the Grandjean trick of having white dots circled in black. For the same reason (readability over a black background) we also invert the labels. They are now white with a black border.

Note: I had to make the labels a bit thicker to have them properly readable.

Assessment: Compared to the previous strategy (black dots, thin edges, labels) we slightly improve the contrast of labels with edges, but we lose contrast of the nodes with the background, and the layout get more difficult to read.

The strategy does not work that well with labels and in black and white, but it remains a valid option for very dense networks.

Black and white: takeaways

  • Use thin lines if the nodes remain visible, else go for the Grandjean trick (white nodes with a black border).
  • If you go for the Grandjean trick, apply it to labels as well.
  • Either way, do not display too many labels because they tend to hide the spatial distribution of nodes.

That being said, black-and-white pictures can also be derived from grayscale pictures through dithering, which will provide more options. My reason for starting with black and white was to highlight the inherent contrast problems of network visualization. If we check my criteria assessing the visualizations, we find that some of them are never satisfied (seeing which edges connect to which nodes), while others always work (the contrast of nodes with the background). This tells us which challenges we face, and where we can intervene to improve the our visualization.

  • Unproblematic: the contrast of nodes, edges and labels with the background.
  • Increasingly problematic: keeping the distribution of nodes (the layout) prominent, identifying the bridges.
  • Always problematic: reading which edges connect to which nodes.

We see a tension inherent to cartography: we can improve the visualization by adding graphical elements, but those elements simultaneously damage the general readability by competing with each other. That is why we have to pick each additional layer carefully, and find ways to avoid competition between layers if we can. Let me call this the curse of contextualization.

In my view, since the most important information comes from the spatial distribution of nodes, the challenge of network visualization is almost entirely dominated by the curse of contextualization. Indeed, as we have seen, just representing the nodes as black dots is already great at conveying the most important information. But as soon as we add other necessary contextual elements, we damage the most important thing (the clarity of the layout). Hence what I see as our top priority: preserving the layout’s ability to jump at the reader’s face.

Our second priority is to improve an aspect that we failed to address so far: clarifying which edges connect to which nodes. This will lead us to designing node halos in the next post of the series.

In order to do that, we will compromise with the aspects that have been unproblematic so far: the separation of nodes, edges and labels over the background. It would be productive to weaken it a little bit if it helps us reaching our two top priorities.

>> Part 2: grayscale

Gephisto

3 min read

The Gephisto paper is out! Anders Munk and I defend the idea that since people get to understand tools by using them, those should cultivate critical thinking through use, and even more, through ease of use.

The paper on Convergence (paywalled):
https://journals.sagepub.com/doi/abs/10.1177/13548565221129053

The preprint (full access):
https://reticular.hypotheses.org/files/2022/12/Gephisto-Paper-Preprint-1.pdf

Easier said than done, though. Gephisto is an attempt at doing this in practice. You can see it as a one-click Gephi. Give it a GEXF or GraphML, and it makes a network map. Your data is not uploaded, it all happens on your own computer.
https://jacomyma.github.io/gephisto/

Also check Verónica Espinoza’s post presenting the tool (thanks to her!).
https://medium.com/@vespinozag/gephisto-gephi-in-one-click-66915adea6f9

Gephisto is primarily intended for teaching, although you may find it useful as a network map maker. But if you find it frustrating, understand that this is integral to its design. One-click magic comes at a price.

We argue that for the user, making informed decisions is like climbing a ladder: becoming aware that a choice exists, knowing its options, what they entail, and how to assess them. Climbing the ladder requires efforts.


In practice, users must choose between climbing the ladder (learning) and meeting their needs (utilitarian use of the tool). They have limited resources, so they must compromise between learning and meeting their operational goals.

The problem is not to use the tool in utilitarian ways (unreflexive). Even seasoned experts go for shortcuts in time-constrained situations. The problem is to never climb the ladder. That’s where ease of use can be a trap.

Some people say that science tools should be hard to use, so that untrained users, supposedly uncritical and unreflexive, cannot access them. Such tools would then only be usable to the worthy users (critical, reflexive), and it would preserve good science.

But this argument fails to acknowledge that most beginners take shortcuts because they have to meet operational goals in time-constrained situations, not because they are uncritical. Climbing the ladder is just costly.

Hard-to-use tools are like Thor’s mythical hammer, Mjølnir, that can only be lifted by the worthy. It requires people to be worthy, but it does not make them worthy. So users will turn around the problem to meet their needs, leading to even worse misuses.

Contrary to Mjølnir, Gephisto is not hostile to the user. It delivers the shortcut (a visualization in one click). However, the more you use it, the more you get to understand the thickness of the decisions it makes for you.

It does that by making the decisions half-randomly. One can make good and bad choices, and Gephisto always tries to make the best ones. But even so, a number of decisions remain. For ex: black or white background? It picks at random, differently each time.

This inconsistency is the price of ease of use. It should become frustrating as you realize that you actually want to make certain decisions. Ex: you prefer the black background. You are climbing the ladder. You need to move to Gephi.

I know how to use Gephi, yet I sometimes use Gephisto. I have climbed the ladder, yet sometimes want a super quick result. It does not make me uncritical. Ease of use does not have to be the nemesis of reflexiveness.

We want to make tools that elevate the user, not tools that lock them out until they have found other ways to get “worthy”. It’s hard though. Gephisto works but it is a prototype, a statement. We hope it paves the way forward to better science tools.

Annotating a large network map on AI in science

15 min read.

This is a show and tell, a photo reporting, and open notes about this work.

In short, we have harvested ~1.5M academic abstracts about AI, machine learning and algorithms from Scopus, we have extracted expressions, and made a co-occurrence network. We have rendered this network as a big map, and we are annotating it.

Johan Søltoft harvested the data, Snorre Ralund lead the computations, and Anders Munk, Matilde Ficozzi and I did the qualitative work. This is part of the ADD project.

This is how we work. We have the map on a big table with a big screen, and we navigate between the original network in Gephi, where we can filter in various ways to look more closely into the topology, a spreadsheet where we have a series of clusters to annotate (obtained from clique percolation), and an ElasticSearch engine where we have stored all the abstracts, and where we can read them for a given cluster. We draw on the map with various markers.

Where we annotate the map. Done!

The network is mostly made of clusters, but not only. The non-cluster parts are the harder to work with, but also the most interesting. They are the reason why we do this qualitative work (annotating is a form of coding). Our very last piece was a bridge, for instance.

The last bridge to annotate. Finally! It was long. Weeks of full-time work. Matilde did the biggest part: summarizing the underlying literature of most of the clusters.

The base map consisted of 7.5K nodes and 85K edges. I had rendered it in a neutral way but with a bit of hillshading, so that we could read the clusters quickly while being able to annotate on top. Some of the node labels (expressions such as “telescope” or “orthopedics”) were displayed to help. We added about 250 landmarks (in red), 50 bridges (in orange), a dozen of areas (in blue) and a few borders (in green).

Overview of the annotated labels

It is nice that with a printed map, you can just get closer to read the labels. You zoom in and out with your own body. This is what it looks like from close. And this is just a tiny tiny part.

Zoom on a tiny part of the map: about the brain, and AI/algorithms, like everything else in this network.

A big part of our time has been dedicated to finding the labels of the landmarks. We invented the method while doing it, and it deserves a separate discussion, so I will not develop that here. But in short, we do read a sample of the corresponding abstracts, and we wonder how algorithms and AI are involved. Each landmark has a longer description in our spreadsheet, but we only have room for a quite short label. So the landmarks are not just about the words, we look into the articles to understand how AI/algorithms are involved.

My focus in this post is the manual annotation, and why we do it that way. Indeed, there is a simpler way to bring context to the network: we could apply community detection and code the clusters obtained. See below our network colored by clusters from Infomap (a clustering algorithm) in Gephi. We could document what each color is. In pink, on the right, we find health and medicine, for instance. That is a valid option. But we have a good reason to put the extra effort.

The same network in Gephi. Colors: clusters obtained from Infomap.

Our ultimate goal is twofold. We want to account for how AI, algorithms and machine learning are involved in science, and we want to make it accessible and understandable to a wide audience. This audience consisting of academics and non-academics, for instance journalists. We want this network to be part of an atlas readable by people not trained in network analysis. The output is not just a paper. We aim at something visually explicit. Therefore, we optimize our method for visual affordances. We want to get the most out of the visual medium.

The visual medium is at its best when a place (an area, a locality) in the image tells something clear about the represented thing (the network). This is why most of our annotations consist of landmarks: a red cross that implicitly applies to the space around it.

We use landmarks instead of hulls or other delimiting shapes because the clusters do not always have clear boundaries. See the cluster about cryptography below: it has distinct subclusters that we detected with an algorithm and annotated separately, but there is also a real blending between these topics in the literature, which is why they are merged together (topologically as well as visually). Using landmarks allows us to account for this continuity. The ambiguity is an empirical reality that we need to account for. We think that it is more exact to not add arbitrary boundaries when there are none in reality. In this case, we can leverage the visual medium in a productive way.

Cluster about cryptography.

However, the visual medium does not have much affinity with nonlocal structures, and that is a problem when it comes to account for this network. Nonlocality is not an intuitive notion and is hard to explain, because it works differently than our everyday experience of the world, but let me take a shot at it quickly. It is like the fourth dimension, but worse, because even high-dimensional spaces are Euclidean, and nonlocal structures are not. The simplest nonlocal structure is a bridge between two clusters. We call a cluster “local” because being close in the map means being connected in the topology (and vice-versa). It does not have to be an absolute rule, a good-enough correspondence between the topological structure of the network and the 2-dimensional space of the map suffices. That is why we can read it. We can infer the structure from the map space: what is drawn, and where it is drawn. From a visual cluster, we can infer a topological cluster. But bridges are topological clusters that cannot be represented as visual clusters because they are nonlocal. They connect two local structures (topological clusters) that are only local separately, but not together. The bridge is at the same time in different “localities” (because the clusters are distinct) and in the same “locality” (because the bridge is connected). The notion of “local” does not help us understand such a structure. And unfortunately visualizing on a 2D space requires locality, because either you draw things in the same place, or you do not. You have to commit to one or the other.

A simple example of a bridge, with planted partitions. The bridge is somehow in two distinct places at the same time; it is nonlocal in that sense.

Bridges are the reason why it is useful to display the edges in addition to the nodes. The long edges show nonlocal structures: connected nodes that have been split apart by the layout algorithm to preserve locality where it is possible. Do you see what I mean? If not, that is not really a problem. The takeaway is that reducing the topology to a set of clusters only accounts for a part of the structure, because of the bridges and other nonlocal structures. And at the same time, we can compensate for this loss of information by using a number of tricks, for instance drawing the edges, and emphasizing the bridges. The more general question is then: how to assess the situations where clusters are not enough to capture the structure, and how to annotate to account for it?

We annotate manually because we want to look at the edge cases straight into the eyes. We face the problems of representing this network as a map, and we look for solutions. So for the rest of this post, I will showcase the weird parts of the map, explain what makes them problematic, and how we dealt with them.

Simple bridges

Let me start with a simple bridge. The bridge in the image below connects the social science cluster (it has a lot of landmarks such as “text classification” and “optical character recognition”) with a small cluster about audio technologies, with two landmarks: “music genre recognition” and “application to cochlear implants”. We labelled the bridge itself “language”. We had to look into Gephi to understand it. It turns out that when machine learning is used to model language, it is sometimes at the same time about the written language and the spoken language (audio), which is why we find a number of papers that mix vocabularies that tend to be in distinct papers the rest of the time, such as “sentences” and “speaker”.

Focus on the “language” bridge.

The bridge is a real structure, it is a cluster from a topological standpoint. But we cannot represent it as a locality in the map without breaking the fact that the two other clusters are distinct localities. And by the way, for feasibility reasons, we decided to commit to the layout at this stage, so we would not change the node positions anyway. So far we decided to represent the bridge as a thick highlight with a label. We picked orange because it is close to the red of the landmarks: the bridges are, in fact, quite the same thing as a landmark, except that we cannot represent them as a place.

Artifacts

Some clusters were artifacts of our method. The algorithm we used picked up on expressions that were noise in the abstracts. We had to annotate them anyway, because they are visible in the map. We keep them, but annotate them accordingly. I just wanted to mention that.

Those clusters are artifacts of our method.

Artifact bridges

We ruled out a number of bridges as artifacts. The pattern generalizes very well, so let me give a typical example. Below, the cluster about signal processing (left) and the cluster about electricity (right) seem connected through the expression “modulation”. We can verify it by selecting it in Gephi.

The two cluster are connected through a single expression, “modulation”.

But as we can check as well in Gephi, the two clusters have exactly zero co-occurrence. Basically, there are no papers using some typical words of the signal processing field as well as of typical words of the electricity field. There is no cluster overlapping the two that could constitute a bridge.

The left cluster is highlighted, and we see that its neighbors are not in the right cluster: there are no real bridges.

In short, the explanation is that the word “modulation” is used in both fields, but possibly in different ways. The word has multiple uses, and possibly multiple meanings. This phenomenon happens all over the place in this network. That is just how language works. We ruled those as “artifact bridges”.

The artifact bridges are visible in the map, as the edges are saillant. So not every set of convergent edges is necessarily a “real” bridge. We always double-check, and our criterion is that the clusters have direct links aside from the bridging node. There are very few edge cases.

Bridges with nodes in the middle

For sure, bridges are always made of nodes. Because even edges are made of nodes. But those nodes are not always in the clusters at the two ends of the bridge. Sometimes, there are nodes in the middle. Often, in addition to the direct edges from one cluster to the other. When that happens, we try to have the bridge go over those nodes. Here is an example.

We included nodes in the path of the bridge “metamaterials applications”.

Another example below. In this case, we tried to fix the bridge by adding some blue (the background color, more or less) so that it keeps looking like a path. Keeping bridges path-shaped can be a challenge, as we will see.

Another bridge where we included nodes. We decided it after the fact, and we had to paint out part of it with a blue pen.

Does it sound like a good idea to you? I ask this because it comes with problems. If we decide that bridges must include the nodes they pass on, then we now have to avoid the nodes that do not belong to the bridge. The bridge is a topological cluster, as you know, so we can determine which nodes belong to it, or not.

We decided to go for this strategy nevertheless, and try to avoid passing over nodes that are not in the bridge when we can. It strongly shaped how we annotate bridges, but we think that it is still better.

Bent bridges

It happens quite often that an entire cluster is on the direct path of a bridge. When that happens, and insofar as we have enough space to do so, we bend the bridge to avoid confusion about where it starts and ends. The area below is very dense in bridges, and we had to bend many of them. I find it quite appealing, as an unexpected bonus.

This area is rich in bent bridges (health and medical science).

Keeping bridges in check

We sometimes had to annotate that a bridge does not own certain nodes. Finding a path for a bridge can be tricky sometimes, and we had to compromise. We used the blue pen in the examples below. We had to clarify some bridge intersections. The first bridges are easy to draw, and the last ones complicated, because the space gets crowded at places.

Breadcrumbs

Some of the bridges we found had so many nodes in the middle that we informally called them “breadcrumbs”. The pattern is very visible in the map, even without the edges displayed.

This bridge, “renewable energy storage”, consists of this whole trail of nodes. We called this pattern “breadcrumbs” because you can easily follow the bridge from it.

This pattern forced me to realize that bridges always consist of nodes, while I was tempted to think of them as edges. Below is another example of a breadcrumb we annotated first as a bridge before we found out it was a topological cluster found by our clique percolation algorithm, which incentivized us to also annotate it as a landmark.

Breadcrumbs may be small clusters. In this case, we used a landmark.

Bridges into nothingness

It happens sometimes that one of our actual bridges ends up on an artifact bridge. You can think of it this way: a small cluster is stretched between another cluster and a single node that connects to a distant cluster because it is polysemic. The small cluster will look like breadcrumbs because it is so stretched, so we cannot put a landmark, but we can annotate it as a bridge. Yet, there is seemingly nothing on the end of the bridge because it ends, in fact, by an artifact bridge. I struggle to explain! Anyway, find two examples below.

Some bridges look like they lead to nowhere. There are two in this map. This is because they end on artifact bridges.

Thick bridges / stretched clusters

In some cases there are so many nodes in the middle of the bridge that it may look almost like a cluster. When this happens, we have no other choice than to make the bridge as thick as necessary. Sometimes, it can be very thick, and weird, as below.

A thick bridge. Aqueous solutions analysis and operations. It is less dense as a cluster.

You may think of this bridge as a stretched cluster. We certainly do so, as we do for every bridge. Bridges are clusters in the topological sense. We find them by clique percolation, for instance (in addition to seeing them in the map). What makes them a bridge and not a cluster is the fact that they are so stretched that there is no single place where to put a landmark. Because they spread over a large area, and/or because they are not dense enough. Below is another example where the bridge is super thick and slightly less dense than a cluster, but not very far from it. I did not have enough space to write my label!

This bridge is almost as dense as a cluster. It’s “hemodynamics” but I did not have much room for the label.

We already know that we will rework our annotations to account for this continuity between visual clusters and bridges. There is a whole spectrum of in-between cases, which suggests that having one flexible way to annotate them might be better than two distinct ones (the red landmark and the orange highlighting).

Branching bridges

As bridges are made of nodes, there is no guarantee that they behave well and follow a path. Sometimes the topological cluster branches out. We have a bunch of two-legged bridges, for instance.

A two-legged bridge

But it can get much more complicated. We followed the topology as far as we could, and tried to represent such branching bridges as faithfully as possible. A pretty intricate example below. There is a lot going on… That is just how it is!

An area with multiple branching bridges.

An even more bizarre example below. Honestly, this one was so hard to grasp that we were reaching the limits of our ability to annotate. We just did what we could, and it’s not great.

An very intricate area that we could barely annotate.

Impossible to annotate

I will end this post with this. There is an area in this map that we failed to annotate. It does not look very threatening, but in short, it is so nonlocal that we could not do anything with it. Here it is.

An area impossible to annotate

Let me explain. There are a bunch of nodes floating in the middle of this picture. Those are many things at once. First of all, they are not connected together, so in that sense, they are not a cluster, even though they occupy the same space. In fact, they belong to 3 different quasi-bridges, very sparse, each of which connects to a different pair of clusters, and sometimes no cluster at all. One cluster is almost an artifact, and consists of heavy metals. The corresponding papers do not have much in common besides the fact that a list of heavy metals are listed in the abstract. Another cluster is about pollutants and water solutions, but not heavy metals. The nodes do connect to those around, and they fit the loose theme of that area (chemistry and materials) but not in a single, identifiable way. Here is what it looks like in Gephi.

The problematic area in Gephi

Next steps

Our next steps will be to redo the annotations a cleaner way, possibly improving our visual language, and have experts of the different fields represented comment on the map. One of those experts might have the key to understand better parts of the map.