Into prompt engineering for A.I.: drawing maps with Disco Diffusion

45 min. read
(or just look at the 757 images)

In short, Disco Diffusion is a Google Colab notebook that you can reuse for free to generate images from text. You can see it as artificial intelligence, as machine learning, as an assemblage of algorithmic techniques. You can also see it as a notebook additively modified by a community, and shaped by their practices. Good entry points into this community are the Discord and the subreddit.

At the heart of the notebook is the prompt: a sentence that Disco Diffusion uses to generate an image. It is the main way to dialogue with the algorithm. To negotiate. Because Disco Diffusion is thick; sometimes surprisingly savvy, sometimes bafflingly stupid, and often you hesitate about which one, because that is what makes it so effective. So one cannot anticipate the result of a prompt. We need to build specific knowledge about it, by trying it.

Prompt engineering is just query design for Disco Diffusion: the way you learn to obtain a certain result by trial-and-error iterations. That is how the members of this community build knowledge about the algorithm. Not by looking into how it works, as this does not help much, but by assessing what it produces, from the outside. They build post-hoc interpretations of Disco Diffusion. See for instance so-called “AI art studies”: a library of how certain terms or artist names shape the result. But there is more to it, because those bits of text interact in unexpected ways when you mix them into a prompt.

I write this post to show what happens under the hood, to expose the practice. You will easily find impressive pictures, quite often with the prompts they come from, but you will rarely see the process through which those prompts are designed. It is harder than it looks. The choice of each word has surprising implications. You will easily find a success story where the A.I. works like black magic, but in practice that is far from the case. And in fact, prompt engineering is a lot about turning the weaknesses of the A.I. into something productive. So by exposing the process of prompt engineering, I also hope to make visible the weird ways Disco Diffusion fails, or rather misbehaves. Which this misbehavior is, I think, why many artists love it so much (compared to for example DALL-E).

I start with a simple, open question: could I repurpose Disco Diffusion for rendering network maps as maps? As a way to style, to make up a network map rendered beforehand by other means. It is a vague question, and it has to be, because this is pretty exploratory. I will share my train of thoughts and comment some interesting things along the way. I will also not cherry-pick: every time, I will only render 4 images and show you them all. Hopefully, this will lead me somewhere. As I am writing these lines, I have no idea yet.

Vintage map, trending on Artstation

• 

• 

• 

• 

In case it is not clear, the prompt is the title above. It might sound weird if you’re new with Disco Diffusion, but a classic way to (try to) ensure consistent and aesthetically pleasing results is to add “trending on Artstation” to your prompt. Artstation is a platform for artist portfolios that was incorporated into the LAION dataset used for training the algorithm. I also added “vintage” for flavor.

Problems already appear. First of all, those are not maps as seen from above, but DD (Disco Diffusion) added a tilted angle and a narrow field of view. Some of the pictures also have some elevation. And the third picture is not really a map. Or a weird, specific one. Where does this come from? Too much is going on. Let me start again, from a more minimal prompt, and we will proceed step by step.

A map

• 

• 

• 

• 

This is flat, which fixes my issues. But it comes with new problems. It’s not pleasing, there is not much detail, and the maps look like children’s drawings. Big, black roads. This is a pretty specific take on the concept of map. I will now try to see if DD gives me different maps with different simple terms. I will first explore this, and then try to build more elaborate prompts.

Cartography

• 

• 

• 

• 

More varied and colorful, with now roads in white or other colors. Also aesthetically agressive, which I assume I can fix by adding stuff to the prompt. More problematic, the field of view and elevation are present. Takeaway: that does not just come from “trending on Artstation” as I initially assumed.

Topographic map

• 

• 

• 

• 

I expected geodesics, and they are somehow present, but reinterpreted as stacked folds or ravines. Somehow, DD did not “understand” the specificity of the pattern, and is trying to render the geodesics as if they were a geographic feature. Of course, those concepts do not exist as such within DD. Let’s say that “map” is clashing here with “topographic”.

Ground plan

• 

• 

• 

• 

I tried yet another possible synonym of “map”. This prompt captured a different kind of map, and it’s pretty detailed too. Nice finding. Remarkable features: white walls, green spaces.

Building map

• 

• 

• 

• 

Close to “ground plan” but with problems (for what I want to do). First, blurriness, 3D and field-of-view. Second, as we see most clearly in image #4, DD understands the prompt at the same time as “a map of a building” and as “a building drawn as a map”. DD has (known) issues at compositing, which I see as more of a boon than a curse, to be honest, but it does not help us here.

This prompt makes me think: let’s try to nudge DD to draw maps of different scales, and see if it gives us different kinds of maps. Even for me, there is not just one kind of map. Let me just replace “building” by “city” and other things.

City map

• 

• 

• 

• 

Captures yet another flavor of map. Pretty varied, with writings. Problem: the resolution is degraded. Note that this has been produced by DD. Yes, DD tried to reproduce images in low resolution. These algorithms are just stochastic parrots, after all. They often regurgitate things that are undesirable in the most obvious ways (to us, not to them). Also note that it tried to write “city” (image #4). This is in part a problem we have already seen: “city map” is also understood as “the word city rendered as a map”.

World map

• 

• 

• 

• 

This is indeed very different kind of map, yay! It comes with a pretty weird look though, between child drawing and play dough. The roads have disappeared, oceans have appeared. We have writings, and it feels a bit like it is contained in a circle (except image #2). It is remarkably consistent.

I wonder two things. Can it regurgitate the actual shape of the world? And if so, which framing and projection would be picked? This side-tracks me but I add, to my stack of things to try, actual geographical entities, and projections.

The oceans are not super consistent though, and I wonder how good would DD be at managing the blue areas. I keep this point for later, as I want to try more “XXX map” prompts.

Network map

• 

• 

• 

• 

You did see that one coming, didn’t you? DD seems to understand the prompt as the map of a network, as in underground or railroad network. It tends to be photorealistic. Also worth remarking that the images #2 and #3 have geographic map elements.

Sky map

• 

• 

• 

• 

This result comes as a total surprise to me, as I fully expected a night sky map, a map of the constellations. This is totally how Google Images understands “sky map”, for instance. But no, DD understands “the sky rendered as a map”, where sky seems to mean clouds over a blue expanse. Genius? Dumb? Both? You decide. Certainly usable, though.

Treasure map

• 

• 

• 

• 

I expected a big red X mark and a dashed path to it. We’re not quite there yet, but the maps produced seem to be in that spirit. We have a mark in image #3 and a red splash in #4.

Dungeon map

• 

• 

• 

• 

I expected a black-and-white plan of rooms and corridors, and this is pretty much what DD gave us. Nice! And usable. This simple prompt can certainly be repurposed to generate actual role-playing maps. Let me push a bit further in that direction.

RPG map

• 

• 

• 

• 

Again, a surprise, as I expected it to be pretty close to “dungeon map”, but no. It is much more colorful and varied. I suspect that “RPG” is understood as a videogame thing, because those images like Zelda levels to me. Case in point, the presence of tiles (all images) and geographic features that look like they have been placed onto a grid (#2, #4).

Videogame map

• 

• 

• 

• 

This is too much of “videogame rendered as a map”. Which is a shame, because many non-RPG games have fantastic maps, notably FPS and RTS. But I am not pushing more into that direction (which is certainly doable, starting with stuff like “FPS map” and then trying actual game titles).

Now, I want to explore a bit beyond the “XXX map” template.

Floorplan

• 

• 

• 

• 

I thought of that one while trying “dungeon map” and for once it does give the results that I expected: similar in style to “dungeon map” but more contemporary, more modern. Image #1 is blurry but the other ones have thin details. The realistic style that creeps in can be managed by engineering the prompt, I assume. Nice prompt overall.

Map projection

• 

• 

• 

• 

Following an idea mentioned before, what if I just ask for an unnamed projection? My rationale was that the expression “map projection” is mentioned in specific contexts, and I expected to see the most abstract features of world maps, such as meridians and parallels, and cuts in maps like some projections have (e.g. the Waterman-butterfly). Instead, those maps look more like world maps, to me, than those from the actual “world map” prompt. This is a classic lesson of query design: the best way to track something is rarely its name.

All these images look torn apart and reassembled. Open question: is it related to the fact that some projections have cuts? This result is mysterious to me.

I will now try to fix the problem of “XXX map” being interpreted as “XXX rendered as a map” by reformulating it into “map of XXX“.

Map of a building

• 

• 

• 

• 

These results seem sensibly better than those from “building map”. They now look more like maps, and they are pretty varied. The fix seems to work :)

Map of a city

• 

• 

• 

• 

We have lost some of the features of “city map”, notably the writings. But like before, it fixed the ambiguity. We still have the streets and rivers and parks. However, new problems appeared, like that pesky field of view. And streets are so white!

About water in maps: we have seen that for DD, the world map is not bathing in oceans. Can it draw an island, though?

Map of an island

• 

• 

• 

• 

The answer is yes, and the results are pretty detailed. It seems to hesitate between different rendering style (e.g. photographed or drawn) and mix them at times (image #1 and #2). Pretty good, though. This pattern seems promising: can it also represent other geographic features?

Map of mountains

• 

• 

• 

• 

This prompt works but the mountains still seem seen from the side, at least partially. At the same time, it retained the way mountains are rendered from above in classic cartography, with hill shading etc. It compromised between the two perspectives, as if the cartographer was Picasso. Despite the prompt template, DD still draws “mountains rendered as a map of mountains”.

Map of a desert

• 

• 

• 

• 

For some reason, I did not expect water and vegetation in a desert map. But now that I think of it, DD might be the most savvy here. In a desert, what you want on your map is the closest oasis. The results are very consistent, although the photo style creeps in like for islands. Well done, nevertheless! Can we move to even bigger things?

Map of a continent

• 

• 

• 

• 

Many details, despite some blurriness and 3D elevation creeping in. Like “world map” and unlike “map of an island”, the land masses are not bathing in water (it does not fill the map up to the sides). It nevertheless retained something of the scale of a continent in the depiction of the geographical features. All maps have are colored in naturalistic ways, sometimes with the addition of abstract colors. I see it as continent maps featuring countries.

It seems hard for DD to bathe land masses in water. What if I ask for just the ocean?

Map of the ocean

• 

• 

• 

• 

Better but still difficult. DD seems lost, here, and features not related to maps start to creep in: realistic waves and marine creatures (I see an octopus in #1). Lost, DD confuses parts of the prompt. It’s one of the way its resistance shows up. That being said, I know that ocean maps exist and that some of them are in the dataset it was trained on. I just did not succeed in connecting to them (assuming that they found a specific place in its head, i.e. a given location in the feature space). Let me try another prompt.

Oceanographic map

• 

• 

• 

• 

We captured a different map flavor, but we are not there yet either: marine life features appearing everywhere. That would be a problem to fix (or an opportunity to explore). I’m trying again.

Topographic map of the oceans

• 

• 

• 

• 

It’s a fail: the problems of the prompts above (wave and animal features) combined with the problems of “topographic map” (geodesics rendered as geographic features) produce this mess. In a way that is typical of DD’s genial idiocy, these now make sense as patterns of sand on the bottom of the sea.

This is more understandable if you know how the diffusion process works. In short, the features are drawn from large to detailed, and the later steps ignore what the earlier steps “had in mind” (i.e., they had nothing in mind, they were just parroting their thing). So geodesics are drawn early on, but later on, the algorithm, not knowing what to do with it, gives it another meaning.

Takeaway: DD is not great at dealing with seas and oceans in maps. Lakes and rivers seem fine, though.

Moving on to the next question on my stack: can it recognize known geographical shapes, like coastlines and countries?

Map of France

• 

• 

• 

• 

This is not France, not even remotely. Image #2 has an air, but I am probably hallucinating. I find comical the pathetic attempt at shipping the French flag as a clue about what this is. That being said, it captures features of the map of a country. I want another attempt!

Cartography of France

• 

• 

• 

• 

That is worse :( Maybe France is hard to characterize?

Map of Denmark

• 

• 

• 

• 

Image #3 is remarkably close to the Netherlands, too bad it’s the wrong country. Joke aside, DD retained that Denmark is basically surrounded by water, but that is all.

Cartography of Denmark

• 

• 

• 

• 

Worse, and once again we see DD becoming erratic. Or is all this red another failed attempt at drawing a flag? OK, DD does not give us country borders. Continents, maybe?

Map of Africa

• 

• 

• 

• 

I wonder if it is not a bit better. The map style seems different to me (warmer colors?). Difficult to assess, but DD is known to be biased in many ways including the most obvious ones, like being centered on the Western world. I write this because the result of “map” matches “map of France” and “map of Denmark” but much less “map of Africa”.

Cartography of Africa

• 

• 

• 

• 

Also a different style, but now image #1 has a shape close to Africa! Coincidence? Possibly not. I now think that it might be possible to improve the retrieval of some of the best known geographical features, but this requires more tweaking than just the prompt, and I will not try this here.

Short parenthesis about two things at once: DD’s biases, and its poor ability to draw faces. What do you think we get if we ask for a portrait of a man?

Portrait of a man

• 

• 

• 

• 

Answer: monstrous white dudes. End of the parenthesis.

Map of Paris

• 

• 

• 

• 

Consistent with previous observations: for DD the main features of Paris are the Eiffel Tower(s) and the architecture, but not the Seine or other geographic things.

Cartography of Paris

• 

• 

• 

• 

The same in worse.

Take away: DD is just bad at returning the shape of a city, country or continent. It tries instead to ship other elements that it sees as characteristic of the entity mapped. But if you ask for a city, country or continent, it will retain something of the scale of what you asked for. So “map of France” has a different flavor from “map of Paris” even though neither is recognizable.

Enough with the basic prompts. I want to make a selection to start adding stuff to them. We can get maps of different scales (building, city, country…) and I do not want to choose, so I will combine the prompts that work best. Only then will I try different ways to tweak them.

Map of an island with mountains, rivers and cities

• 

• 

• 

• 

The rationale for this kind of prompt is that I want to have a variety of things into the same map. Yet all I can get for sure, it seems, are mountains and some water expanse. Island is not granted, nor are rivers and cities. The results are pretty varied, however, which I appreciate. DD has more material to work with. It might be good enough for me. All these images are pretty map-like to me.

Next I would like to try a country. As we have seen, it is worth using an actual country name since it gives the map some flavor without imposing the actual shape of the country (let’s turn this weakness into an opportunity). Let me start with a country that is also an island!

Map of Japan with mountains, deserts, forests, fields, rivers and cities

• 

• 

• 

• 

Pros: map-like, detailed, and with a lot of character. Cons: strong presence of the flag, some writing, can be blurry. Even when the flag is not directly there, its colors creep into the image due to the diffusion process.

I want to try a country that is not an island, and that has a flag with colors that blend in more easily. I pick Ukraine.

Map of Ukraine with mountains, deserts, forests, fields, rivers and cities

• 

• 

• 

• 

Those maps look less drawn and more satellite-view-like. It is less charming. The yellow and blue of the flag keep creeping in, but it is less problematic. A very different flavor than the precedent prompt. I try Rwanda next, because it’s yet another continent and also, the flag has map-like colors.

Map of Rwanda with mountains, deserts, forests, fields, rivers and cities

• 

• 

• 

• 

Satellite-like too. Like previously, I do not see rivers and cities. Let’s try the continent scale, now.

Map of Africa with mountains, deserts, forests, fields, rivers and cities

• 

• 

• 

• 

The shape of Africa, in fact, appears, confirming my previous hypothesis. Maps of Africa apparently “want” to feature animals and people (image #2). The four images are pretty different. Interesting.

Map of Europe with mountains, deserts, forests, fields, rivers and cities

• 

• 

• 

• 

Those images are also pretty different from one another. I do not see rivers and cities, though. They also have this plastic-like appearance that we have seen a number of times now, notably with “world map”.

Map of Asia with mountains, deserts, forests, fields, rivers and cities

• 

• 

• 

• 

Like with Japan, those maps have a distinctive character. They still do not have rivers and cities, but they remain less satellite-like.

Next scale: the world. Since “map projection” gives us world map, let me try this:

World map projection

• 

• 

• 

• 

It is a fail. For some reason, those look even more torn apart than “map projection”, to the point that images #2 might not even be understood as a map. I need another angle.

World map with mountains, rivers and cities

• 

• 

• 

• 

Weirdly, mountains are on top of those maps, seen from the side. In some ways, the scale seems smaller than for continents. DD is not doing great with the world scale.

Moving on. Ground plan and floorplan are both nice and pretty similar, shouldn’t I combine them?

Ground plan, floorplan

• 

• 

• 

• 

It works but it is not fantastic. I think I preferred “ground plan” by itself. It is both sparse and too square. “Dungeon map” might compensate for that, maybe? But wait a minute, what if I shortened all of that into…

Dungeon ground floorplan

• 

• 

• 

• 

My bad: it is now trying to make maps that could also be how it imagines a dungeon: stone bricks (image #1), windows (#2) etc. Let me rephrase.

Dungeon map, ground plan, floorplan

• 

• 

• 

• 

Now it works!

Network map with communities

• 

• 

• 

• 

I wanted to nudge network maps toward node-link diagrams. It kind of works, but it seems that “community” is also understood in the sense of “people” (images #1 and #4).

Gephi network map

• 

• 

• 

• 

DD correctly picks the Gephi vibe. But at the same time, this is not the direction I want to take. I am looking for something less dramatic.

Node-link network map

• 

• 

• 

• 

The image #1 is surprising, but the others are like before, in sparse. My favorite attempt remains “network map with communities”.

At the end of the day, I will keep these 6 base prompts:

1. Dungeon map, ground plan, floorplan
2. Map of a city
3. Map of an island with mountains, rivers and cities
4. Map of Asia with mountains, deserts, forests, fields, rivers and cities
5. RPG map
6. Network map with communities

The four first are actual ways to (hopefully) generate maps. The last two are more like a control group, to track the impact of different modifiers. I do not expect those to necessarily produce map-like images, but who knows!

I will test different additions to these prompts to improve them. My goal will simply be to generate nice maps: aesthetically pleasing, but also recognizable as maps, and importantly, flat (as seen from above).

(…), trending on Artstation

Each row is a different base prompt.

• 

• 

• 

• 

• 

• 

• 

• 

• 

• 

• 

• 

• 

• 

• 

• 

• 

• 

• 

• 

• 

• 

• 

• 

This modifier visibly improves the aesthetic qualities of those images: better graphical consistency, more details. It comes with problems, though. First, a lot of narrow field of view. Second, the images tend to be realistic. Third, some elements are not drawn as seen from above, but rather from the side. These elements make the images less like map and more like 3D dioramas. Those are maps from a movie screenshot, not from a printable file. The only case where it makes sense is the “RPG map” prompt, that is already in that direction.

Of those three problems, one is a battle I do not want to fight: things drawn from the side. I doubt I can fight against that, and it does not bother me much, as it is something we sometimes see in old maps. I want to get rid of the out-of-focus blur though, and I want a flatter rendering. I assume that the former may come with the latter, so I will start by seeking map-like flatness.

I will try the classic strategy (for DD) to ask explicitly for a “beautiful painting”. But before that, since most of our prompts are already “maps”, has DD a notion of “beautiful maps”?

A beautiful (…)

• 

• 

• 

• 

• 

• 

• 

• 

• 

• 

• 

• 

• 

• 

• 

• 

• 

• 

• 

• 

• 

• 

• 

• 

This looks promising to me. Even though the out-of-focus blur and 3D features are still present sometimes, the result is flatter and more map-like than “trending on Artstation”, while improving the aesthetic quality. I am especially impressed by the city map (row 2) and RPG map (row 5).

A beautiful painting of a (…)

• 

• 

• 

• 

• 

• 

• 

• 

• 

• 

• 

• 

• 

• 

• 

• 

• 

• 

• 

• 

• 

• 

• 

• 

Those images are nice and flat, which is great. Are they map-like? I find them like painted maps, which is one kind of map. But do they look like painted maps of something, or like paintings of that thing? It depends, I think. The island map (row 3) and Asia map (row 4) are better this way, but the city map (row 2) is more like a city and less like a map of a city. What if maps are drawings rather than paintings?

A beautiful drawing of a (…)

• 

• 

• 

• 

• 

• 

• 

• 

• 

• 

• 

• 

• 

• 

• 

• 

• 

• 

• 

• 

• 

• 

• 

• 

These drawings are not always that different to paintings, but we have flatness, we have details, and we do not have background blur. Once again, the interaction with the base prompt can be surprising. Dungeon maps (row 1) already looked like drawing, and they still do, but other prompts now also look the same, notably city maps (row 2) and RPG maps (row 5). Also surprising, while the “beautiful painting” modifier worked well with island maps and Asia maps (rows 3 and 4), this modifier worked better for island maps (row 3) than Asia maps (row 4). With the latter, it was inconsistent, producing sometimes detailed, sometimes sparse, childish drawings. It interacted badly with RPG maps. I really like the network map #3 (last row). Overall, it might be more map-like than with “beautiful painting”, and it does solve some of my issues, but I am not entirely sold, and I want to push further and try to get a more modern style.

A beautiful (…), flat design

• 

• 

• 

• 

• 

• 

• 

• 

• 

• 

• 

• 

• 

• 

• 

• 

• 

• 

• 

• 

• 

• 

• 

• 

It comes as a complete surprise to me that “flat design” may lead to miniature dioramas (rows 3 and 4). And I cannot explain why city maps, specifically, are transmogrified into the cuteness of row 2. I expected uniform fills, and on that level it delivers. It is also nice overall, very aesthetically pleasing, and pretty map-like, which is hard to get, as we have seen. But we have the out-of-focus issue again, and some amount of 3Dness. Yet I want to highlight how versatile this modifier is. It pushed each base prompt in a different direction. It is compatible with RPG maps (row 5) but apparently incompatible with network maps (row 6), which makes sense.

My takeaway: uniform fills and subtle gradients have a map-like flavor that I really like. Let me try an alternative to this modifier, it may do better.

A beautiful (…), vector graphics, illustrator

• 

• 

• 

• 

• 

• 

• 

• 

• 

• 

• 

• 

• 

• 

• 

• 

• 

• 

• 

• 

• 

• 

• 

• 

We also get the same flatness as before, but city map (row 2) does not react differently from the other rows. Is that good, though? Open question. We do get out-of-focus blur, but it remains less diaporama-like than the “flat design” modifier, and more map-like to my taste, which is a clear win. Now I want to explore modifiers that change the style in other ways, and see if it also gives map-like images.

A beautiful (…), watercolor painting

• 

• 

• 

• 

• 

• 

• 

• 

• 

• 

• 

• 

• 

• 

• 

• 

• 

• 

• 

• 

• 

• 

• 

• 

I like the hand-made touch, and the images are very flat (no 3D or blur). It works for dungeons (row 1), cities (2) and networks (6) but not much for island (3) and Asia (4) that look more like landscapes than maps.

A beautiful (…), colored pencil art

• 

• 

• 

• 

• 

• 

• 

• 

• 

• 

• 

• 

• 

• 

• 

• 

• 

• 

• 

• 

• 

• 

• 

• 

It is sometimes nice, notably for row 4, but DD confuses the prompt in two ways. First, it wants to draw pencils (row 2). Second, wool textures creep in the pencil texture (row 6). Not necessarily bad, but not map-like. It’s less flat than watercolor (some 3D and blur appear).

A beautiful vintage (…)

• 

• 

• 

• 

• 

• 

• 

• 

• 

• 

• 

• 

• 

• 

• 

• 

• 

• 

• 

• 

• 

• 

• 

• 

This modifier is more effective than I expected. Despite the usual out-of-focus blur, the result generally feels quite map-like to me. Like a collage of colorized photos. Interesting results for city maps (row 2) and network maps (row 6).

I will now try the classic DD strategy of adding a painter’s name to borrow their style. I start with artists whose style could feel map-like.

A beautiful (…), by Wassily Kandinsky

• 

• 

• 

• 

• 

• 

• 

• 

• 

• 

• 

• 

• 

• 

• 

• 

• 

• 

• 

• 

• 

• 

• 

• 

Beautiful, but it does not feel like maps at all (except row 4 #2,3) because the features are too big. Interesting networks, though.

A beautiful (…), by Paul Klee

• 

• 

• 

• 

• 

• 

• 

• 

• 

• 

• 

• 

• 

• 

• 

• 

• 

• 

• 

• 

• 

• 

• 

• 

Better than Kandinsky, but still with the same problem: it resembles abstract painting more than maps. Too bad, row 4 #2 is otherwise fantastic.

A beautiful (…), by Max Ernst

• 

• 

• 

• 

• 

• 

• 

• 

• 

• 

• 

• 

• 

• 

• 

• 

• 

• 

• 

• 

• 

• 

• 

• 

Still pretty painting-like, but one more step toward mapiness. City maps (row 2) and Asia map (row 4) might be usable? Anyway, I will now swap to artists that are more illustrators than fine artists, and whose marked style could be interesting even though they did not produce map-like illustrations at all.

A beautiful (…), by Gustave Doré

• 

• 

• 

• 

• 

• 

• 

• 

• 

• 

• 

• 

• 

• 

• 

• 

• 

• 

• 

• 

• 

• 

• 

• 

Gustave Doré is mostly known as a printmaker, and I expected a fully black and white rendering, but that is not totally the case (and not an issue). Many of the renderings look hand-drawn, and would pass for printed maps, set aside the 3D features and out-of-focus blur that creeps in occasionally. Certainly usable. Island and Asia maps (rows 3 and 4) are close to the “vintage” modifier. The network maps are fantastic in that “community” has been interpreted as crowds of people (take a close look!).

A beautiful (…), by Edmund Dulac

• 

• 

• 

• 

• 

• 

• 

• 

• 

• 

• 

• 

• 

• 

• 

• 

• 

• 

• 

• 

• 

• 

• 

• 

Incredible rendering by this artist who does not make maps at all, I am enthusiastic about this one! His flat but vintage style is a great fit for maps.

A beautiful (…), by Victo Ngai

• 

• 

• 

• 

• 

• 

• 

• 

• 

• 

• 

• 

• 

• 

• 

• 

• 

• 

• 

• 

• 

• 

• 

• 

Victo Ngai is another incredible fit for maps, with a more digital style than Edmund Dulac, close to flat design, and a very distinctive flavor. The creeping tilt-shift blur needs to be managed, though. Island and Asia maps are very map-like, but network and dungeon maps are very rich and clean as well. RPG maps are more like RPG illustrations, though (and beautiful, nevertheless).

A beautiful (…), by Mary Blair

• 

• 

• 

• 

• 

• 

• 

• 

• 

• 

• 

• 

• 

• 

• 

• 

• 

• 

• 

• 

• 

• 

• 

• 

Those are disappointing in comparison to the last two, half-way through between maps and paintings. I have better options.

I have tested enough artists for what I need, and with some relative success, so it is now time to tackle two problems we have seen again and again: the landscapes seen from the side instead of above, and the out-of-focus blurriness.

A beautiful (…), seen from above, satellite view, trending on Artstation

• 

• 

• 

• 

• 

• 

• 

• 

• 

• 

• 

• 

• 

• 

• 

• 

• 

• 

• 

• 

• 

• 

• 

• 

The actual modifier tested here is “seen from above, satellite view” and I used “trending on Artstation” because it produced a number of images seen from an angle. This works well-enough, but I assume that the “satellite view” part changes the character on some of the images, notably the dungeon maps (row 1) where trees now appear. I will have to remove that part in some cases.

I now tackle the blurriness of the field of view (out-of-focus background). I do this by disincentivizing it with a negative weight. As it turn out, you are not limited to one prompt, but you can use multiple ones and even weight them. The weight is the number after the colon (defaults to 1) and I separate the prompts with a pipe, even though in practice the syntax is a little bit different.

A beautiful (…), seen from above, satellite view, trending on Artstation: 1.2 | tilt-shift: -0.1 | blurred: -0.1

• 

• 

• 

• 

• 

• 

• 

• 

• 

• 

• 

• 

• 

• 

• 

• 

• 

• 

• 

• 

• 

• 

• 

• 

“Tilt-shift” produces exactly the kind of tilted angle with out-of-focus background that I want to avoid, so I give it a negative weight, and similarly with “blurred” as a backup plan. As those do not have a precise thing to draw (only something to not draw) they must not compete too much with the main prompt, so they have a lower weight. I kept the total weight to 1 like before.

Unfortunately, we still have some blur and tilt-shifting (row 2) so I will increase the negative weight of those modifiers.

A beautiful (…), seen from above, satellite view, trending on Artstation: 2 | tilt-shift: -0.5 | blurred: -0.5

• 

• 

• 

• 

• 

• 

• 

• 

• 

• 

• 

• 

• 

• 

• 

• 

• 

• 

• 

• 

• 

• 

• 

• 

Now we run into another problem: the images lose their character because the other prompts compete too much. In short, DD is happy with whatever is neither blurry nor tilt-shifted, and it does not try as hard to draw maps. It may also over-compensate. You can see that to the many spiraling patterns and noise on the sides. I need to tone down the negative weights.

A beautiful (…), seen from above, satellite view, trending on Artstation: 1.4 | tilt-shift: -0.3 | blurred: -0.1

• 

• 

• 

• 

• 

• 

• 

• 

• 

• 

• 

• 

• 

• 

• 

• 

• 

• 

• 

• 

• 

• 

• 

• 

This work well-enough for me! As we can see (row 2) it does not get rid of things seen from the side entirely, but we do not have absolute control on what DD does.

I am confident now that I have enough modifiers to assemble different kinds of maps and solve most of the problems I have met. I have also explored enough to understand that different base prompt “want” to go in different directions. This is what I will be trying to get:

1. A black and white, drawn map for the building size (dungeon). Basically, something that could work for role playing.
2. A colorful tourist map for the city scale. I will not pursue a road map style, but something more illustrated.
3. A hand-painted vintage map for the island scale.
4. Something as map-like as possible for the continent scale (Asia). It might not actually be looking like a map of that scale, but it must look like a map.
5. A colorful hand-painted map for the RPG style. As we have seen, it does not necessarily look like a video game, but it often brings an aesthetically pleasing composition that I want to preserve.
6. Whatever could evoke a network while still looking like a map.

A beautiful drawing of a dungeon map and ground plan and floorplan seen from above by Gustave Doré and Victo Ngai, trending on Artstation, black and white, vector graphics:1.4 | tilt-shift:-0.3 | blurred:-0.1

• 

• 

• 

• 

My rationale here is to go for a drawing, add the modifier “black and white”, ask for Gustave Doré of course but also Victo Ngai for the cleanliness of the style, to avoid something too messy, and similarly ask for “vector graphics”. I did put “seen from above” but not “satellite view” to avoid trees.

There is still some tilt-shifting creeping in, and that might come from “trending on Artstation”. I also find that there is too much little details, and some color creeping in, that I attribute to Ngai’s style. I will try to correct for that by (1) adding “vintage”, (2) replacing Victo Ngai by Edmund Dulac, (3) removing “trending on Artstation” and (4) pushing a bit the negative weight of tilt-shift.

A beautiful drawing of a dungeon map and ground plan and floorplan seen from above by Gustave Doré and Edmund Dulac, black and white, vector graphics:1.6 | tilt-shift:-0.5 | blurred:-0.1

• 

• 

• 

• 

Here we are! Surprisingly, some figurative drawing creeps into the picture, but in fact I like it a lot because it does not interfere too-much with the mapiness of the image. It reminds me of annotations and “here be dragons” stuff. Let’s call that a happy accident, but you can see where it comes from (using Edmund Dulac in the prompt). Not all the renderings are equally good (#3 produced jpeg artifacts), but it does not matter as I can cherry-pick in the end. I am happy enough with that one. Let’s move on to the second style.

A beautiful watercolor painting of a map of a city seen from above by Edmund Dulac and Victo Ngai, flat design, trending on Artstation:1.4 | tilt-shift:-0.3 | blurred:-0.1

• 

• 

• 

• 

I went for a combination that was strong in the illustrative side. I merged “A beautiful painting” and “watercolor painting” into “A beautiful watercolor painting”. I hesitated but finally kept the “seen from above” part to ensure that it is map-like enough, also because it does not completely prevent features seen from the side.

The result passes well for a map, and is in fact pretty satisfactory, but I aimed at something more illustrated, less vintage and more digital. I will adjust by (1) removing Edmund Dulac, (2) replace “watercolor” by “digital”, and (3) remove “seen from above”.

A beautiful digital painting of a map of a city by Victo Ngai, flat design, trending on Artstation:1.4 | tilt-shift:-0.3 | blurred:-0.1

• 

• 

• 

• 

That is closer to my goal but there are problems, and in particular the absence (sometimes) of roads or pathways makes it look less like a map and more like an illustration of the city. I will try to make it more like a tourist map by replacing “map” by “tourist map” and adding “with main roads and shops and restaurants”. I will also remove “trending on Artstation” to minimize the chances of background blur, and also because the rest of the prompt should ensure an aesthetically pleasing image by itself.

A beautiful digital painting of a tourist map of a city with main roads and shops and restaurants by Victo Ngai, flat design:1.4 | tilt-shift:-0.3 | blurred:-0.1

• 

• 

• 

• 

Almost there! But some of those failed at being maps, and the detail level is sometimes too much. I will put back “trending on Artstation”, raise the negative weight of “tilt-shift”, simplify “digital painting of a tourist map” into just “digital map”, and simplify “with main roads and shops and restaurants” into “with roads and shops”.

A beautiful digital map of a city with roads and shops by Victo Ngai, flat design, trending on Artstation:1.6 | tilt-shift:-0.5 | blurred:-0.1

• 

• 

• 

• 

That works well enough for me :) And I like the presence of ideogram-like signs.

Moving on to the third style: a hand-painted vintage map of an island.

A beautiful vintage drawing of a map of an island seen from above with mountains, rivers and cities, satellite view by Edmund Dulac and Gustave Doré, trending on Artstation:1.4 | tilt-shift:-0.3 |blurred:-0.1

• 

• 

• 

• 

I went for the styles that looked similar to the “vintage” modifier, which includes “drawing” over “painting”, even though that will probably not look like a drawing in the end.

Why did a sausage land in image #2? Anyways, it turns out that the style of Gustave Doré is making the whole thing a bit dirty to my taste, so I will try to tone it down by adding a third artist, Mary Blair, because her style is clean but feels like the 60’s, which fits. I will also move “satellite view” after “trending on Artstation” to keep the main part of the sentence more coherent. I will try a new modifier to reinforce the map character by adding “coastal” before “map”, and I will add “blue mood” to nudge the general color of the map.

A beautiful vintage drawing of a coastal map of an island seen from above with mountains and rivers and cities by Edmund Dulac, Mary Blair and Gustave Doré, trending on Artstation, satellite view, blue mood:1.4 | tilt-shift:-0.3 | blurred:-0.1

• 

• 

• 

• 

Unfortunately, if the details are more map-like, I feel like we have still lost on the mapiness in the composition. I assume that “Mary Blair” interferes too much. I will replace that part by a “vector graphics” modifier, to get just a bit of cleanliness without altering too much the drawing style.

A beautiful vintage drawing of a coastal map of an island seen from above with mountains and rivers and cities by Edmund Dulac and Gustave Doré, trending on Artstation, satellite view, vector graphics, blue mood:1.4 | tilt-shift:-0.3 | blurred:-0.1

• 

• 

• 

• 

Great! I will stop there. To the fourth style: something as map-like as possible using the base prompt of the Asia map.

A beautiful map of Asia with mountains, deserts, forests, fields, rivers and cities seen from above by Victo Ngai, satellite view, flat design, vector graphics, illustrator:1.3 | tilt-shift:-0.1 | blurred:-0.2

• 

• 

• 

• 

The style is as clean as I hoped for, but still a bit too close to illustration for actual maps. Part of it is due to the vibrant colors of Victo Ngai, but also to the pattern she uses, that are not typical of maps. For that reason, I want to draw Edmund Dulac into this, as a counterpoint, hoping that he will not break the cleanliness of the render. I will also add “desaturated colors”. Finally, I will also add “outlines” to nudge at that kind of graphic signal.

A beautiful map of Asia with mountains, deserts, forests, fields, rivers and cities seen from above by Victo Ngai and Edmund Dulac, satellite view, flat design, vector graphics, illustrator, outlines, desaturated colors:1.3 | tilt-shift:-0.1 | blurred:-0.2

• 

• 

• 

• 

Unfortunately, the colors keep being pretty vibrant, and some tilt-shift style is occasionally creeping in. To address both at once, I will redistribute the prompts. I will move “desaturated colors” to a secondary prompt to which I will add “satellite map” to keep nudging DD into the right direction, and I will raise a bit the negative weight of “tilt-shift”.

A beautiful map of Asia with mountains, deserts, forests, fields, rivers and cities seen from above by Victo Ngai and Edmund Dulac, satellite view, flat design, vector graphics, illustrator, outlines:1.3 | tilt-shift:-0.3 | blurred:-0.2 | satellite map with desaturated colors:0.2

• 

• 

• 

• 

This did not work, as apparently the style of Victo Ngai is too dominant. From there, I will essentially split the main prompt into two with different weights, one by Dulac and one by Ngai with a small weight.

A beautiful map of Asia with mountains, deserts, forests, fields, rivers and cities seen from above by Edmund Dulac, satellite view, flat design, vector graphics, illustrator:1.2 | A beautiful map of Asia seen from above by Victo Ngai, satellite view, flat design:0.1 | tilt-shift:-0.3 | blurred:-0.2 | satellite map with desaturated colors:0.2

• 

• 

• 

• 

It was effective, but I feel like we are too far away from the style of a modern map. To compensate for that I will add “trending on Artstation” to the main part of the prompt, and give a bit more weight to the Ngai part. I also see that the patterns are more illustrative than map-like but at this point, I will not be able to address it.

A beautiful map of Asia with mountains, deserts, forests, fields, rivers and cities seen from above by Edmund Dulac, trending on Artstation, satellite view, flat design, vector graphics, illustrator:1.0 | A beautiful map of Asia seen from above by Victo Ngai, satellite view, flat design:0.3 | tilt-shift:-0.3 | blurred:-0.2 | satellite map with desaturated colors:0.2

• 

• 

• 

• 

The balance is better, but there is still too much grain despite all the modifiers like “flat design”. I will be attempting one more push in that direction by adding “digital” before “map” in the main part, and adding “Mary Blair” as a second artist in that same part. Fingers crossed.

A beautiful digital map of Asia with mountains, deserts, forests, fields, rivers and cities seen from above by Edmund Dulac and Mary Blair, trending on Artstation, satellite view, flat design, vector graphics, illustrator:1.0 | A beautiful map of Asia seen from above by Victo Ngai, satellite view, flat design:0.3 | tilt-shift:-0.3 | blurred:-0.2 | satellite map with desaturated colors:0.2

• 

• 

• 

• 

Pfiou! Here we are, or at least I am satisfied enough to stop there, even though it is far from perfect. But at least, the clean appearance is back. This one was hard. As you can see, the map-like style is largely a reconstruction, a delicate assemblage that draws on the observed behavior of the algorithm. Part of it is surprising behavior that we can learn to understand, part of it is interaction between sub-prompts that remains pretty chaotic, part of it is just coincidence, luck, and a lot of iteration.

For the fifth style, the RPG map, I will be letting DD follow its own slope, as this is how it started, and try to emphasize it.

A beautiful RPG map, flat design, trending on Artstation:1.1 | blurred:-0.1

• 

• 

• 

• 

I will leave it at that. I am making an exception here to allow tilt-shift. This will look more illustrative, like imaginary screenshots of cute RPGs that do not exist. By contrast with the previous style I tried to achieve, it’s so much easier when you do not fight against the algorithm. In that sense, you just get more out of the algorithm if you let it go where it wants to, which is also why so many A.I. art pieces look the same. And as you can see, getting something different may take a lot of work. A work that is, in nature, artistic.

For my sixth and last style, the network map, I will just look for a mix of moderate mapiness, some but not too many links, and rich details.

A beautiful painting of a vintage network map with communities seen from above by Paul Klee, Gustave Doré, Edmund Dulac and Victo Ngai

• 

• 

• 

• 

Well that is weird, fun, and nice. “Community” gives little persons, and I am happy with that. The balance is good enough for me, so be it! I now have my 6 styles.

A large rendering of each of the 6 prompts

For those I asked for more pixels, more details, and more diffusion steps. Each one takes above 30 minutes to render.

• 

• 

• 

• 

• 

• 

Recap of the prompts used, in the DD format:

# 1. Building map (dungeon style)
['A beautiful drawing of a dungeon map and ground plan and floorplan seen from above by Gustave Doré and Edmund Dulac, black and white, vector graphics:1.6',
'tilt-shift:-0.5',
'blurred:-0.1']

# 2. City map (tourist style)
['A beautiful digital map of a city with roads and shops by Victo Ngai, flat design, trending on Artstation:1.6',
'tilt-shift:-0.5',
'blurred:-0.1']

# 3. Island map (vintage style)
['A beautiful vintage drawing of a coastal map of an island seen from above with mountains and rivers and cities by Edmund Dulac and Gustave Doré, trending on Artstation, satellite view, vector graphics, blue mood:1.4',
'tilt-shift:-0.3',
'blurred:-0.1']

# 4. Country map (modern style)
['A beautiful digital map of Asia with mountains, deserts, forests, fields, rivers and cities seen from above by Edmund Dulac and Mary Blair, trending on Artstation, satellite view, flat design, vector graphics, illustrator:1.0',
'A beautiful map of Asia seen from above by Victo Ngai, satellite view, flat design:0.3',
'tilt-shift:-0.3',
'blurred:-0.2',
'satellite map with desaturated colors:0.2']

# 5. RPG map (video game style)
['A beautiful RPG map, flat design, trending on Artstation:1.1',
'blurred:-0.1']

# 6. Network map (painting style)
['A beautiful painting of a vintage network map with communities seen from above by Paul Klee, Gustave Doré, Edmund Dulac and Victo Ngai']

Styling a network map

Finally, can that be useful to give a different appearance to a network map? Here is a base image given to DD for each of the prompts, and how it was subsequently modified.

• 

• 

• 

• 

• 

• 

• 

---

---