Mechanistic interpretability as generative art

A neural network that has learned the concept “ocean” has learned a location rather than a definition. A point, or more honestly a region, in a high-dimensional space where everything it associates with oceans clusters together. Interpretability research usually treats that as something to diagram. Probe the space, label the axes, write the paper. I got more interested in a different question. If the concept is a place, what does it look like to go there?

That turned into a project. It optimises generative outputs, so images, audio and video, towards target coordinates in LanguageBind’s 768-dimensional multimodal embedding space, doing concept algebra across modalities to surface the ideals the network has learned. It sits on the seam between research and aesthetics, and the seam turns out to be more interesting than either side on its own.

One space for everything

What makes LanguageBind useful here is that it embeds image, text, audio and video into one shared space anchored on language. A photo of a beach, the word “beach” and the sound of waves all land near each other, because the model was trained to line them up. The space doesn’t care which door a concept came in through. Ocean is a region whether you arrived by image, by word or by sound.

Anchoring on language buys one more thing. Any point in the space can be projected back towards the text vocabulary, so even a non-text coordinate has a running readout in words. You can ask what the current image embeds as and get back something like {storm, crackling, electric}, which turns the optimisation from a black box into something you can watch as it runs.

That shared geometry lets you do something that shouldn’t intuitively work: algebra across senses. Take the embedding of a sound, subtract the embedding of a word, add the embedding of an image, and you land somewhere new, at a coordinate no single input could have named. It’s the classic king − man + woman ≈ queen move from word vectors, except the operands can be a photograph, a field recording and a sentence, mixed freely.

Generation as a search for a place

The move that turns geometry into pictures goes like this. Pick a target coordinate, either a concept or a piece of concept algebra. Then run a generator, a diffusion model for images and the equivalents for audio and video, and optimise its output so that the output’s own embedding lands as close as it can to that target. Nobody says “draw an ocean”. The instruction is to produce something, anything, that this network would file where it files oceans, and let the network be the judge.

What comes out is the generator’s best attempt to occupy the coordinate rather than an illustration of the concept. Sometimes that’s a recognisable beach. More often it’s stranger and more useful: the features the network associates most strongly with that region, rendered without any obligation to look like a real photograph. You’re seeing the concept the way the model holds it rather than the way the world presents it.

Why this is interpretability

It would be easy to write this off as a stylised image generator. What makes it interpretability is that the output is diagnostic. When you steer towards a concept and the result surprises you, when “trust” renders as something you wouldn’t have predicted, or an audio target produces an image whose logic only makes sense once you’ve heard the sound, you’ve learned something concrete about how the network organises that part of its space. The art is the readout.

Concept algebra is where this gets sharpest. If summer − heat lands somewhere coherent, the model has separated those two ideas cleanly. If it lands in noise, it hasn’t, and the concepts are tangled together in a way the geometry won’t let you pull apart. The generated output makes that legible in a way a cosine-similarity table never does. You can see whether the model’s internal world is well-organised, and where it isn’t.

This is the same instinct behind the better interpretability work in the field, the feature-visualisation lineage and the “what is this neuron looking for” question, pointed at the multimodal case and pushed until the answer is an image.

The aesthetics do real work

The reflex is to treat the art half as a wrapper around the research. It’s the most faithful rendering available of an object, a learned concept, that has no native visual form.

A concept in embedding space really is 768-dimensional. Any honest depiction throws most of those dimensions away. A scatter plot throws away all but two and asks you to trust the projection. A generated image throws away fewer, because the generator is trying to satisfy the full target coordinate across every dimension at once. It’s still lossy compression, and it’s richer lossy compression than the chart, carrying information the chart structurally can’t. The aesthetics are doing epistemic work.

There’s also a claim hiding in the word “platonic”. If many different networks, trained on different data, converge on similar internal structure, and there’s a growing body of evidence that they do, then steering towards a concept and rendering it gets at the shape a concept takes whenever a system this size learns it from the world, rather than at one model’s quirks. That’s a stronger claim than “pretty pictures from a model”, and it’s the one the work is making.

The seam is the point

Most interpretability tells you that a model represents a concept, and roughly where. This tries to show you what it is like for the model to represent it. That question has a research answer and an aesthetic answer at the same time, and refusing to separate them is deliberate.

The code is on GitHub if you want to steer towards your own coordinates and see what the network thinks lives there.