Mary's Mirror

A Long Time Ago, in a Philosophy Module Far Away

I wrote my dissertation on the topic of qualia through the lens of Frank Jackson’s legendary thought experiment Mary’s Room. It was a long, long time ago but I have been revisiting it recently as I remember really loving that module, not just for the dense conceptual gymnastics, but for how it pulled apart assumptions I didn’t even realise I was making. It opened me up to the idea that my relationship to the world is not some type of universal constant; it’s an individual construction built from context, experience, memory and association. That may sound pretty obvious and in a lot of cases it most definitely is, I’m referencing the underlying understanding between people.

Language provided us a connection deeper than just instinct and allowed us to translate interiority. Initially this was perhaps born from a desire to transcend our own isolation and connect with others and would have been similar to how the Great Apes have a rudimentary sound language that communicates more a caste hierarchy than anything meaningful. Somewhere along the way we evolved from being satisfied by being in our “tribe’s” company to desiring a deeper connection, and this introduced a requirement to communicate more than the basics. Now we have to use those same symbols to communicate that which is experienced or intangible; objects are objects, but what is guilt? What about love?

When spoken the words will travel through my ear into my brain and touch on all my memories associated with the words I’m hearing but do I truly understand the speaker’s meaning? I understand my interpretation of what they mean built from my association made with films, personal experience, hormones; but that’s not theirs as much as theirs isn’t mine. Think of those times when you are really excited about something and are telling someone and they just aren’t showing the same level of enthusiasm. We approximate. We project. We reflect.

The question that haunted me then was: what is it like to experience something? And more importantly: Can that experience ever truly be shared?

Now, over a decade later, I find myself building AI streamers, local LLMs, and digital personas that perfectly simulate the language of experience, but exhibit none of the thing itself.

And I can’t stop thinking about Mary.

Bats to Bondage

Before Frank Jackson gave us Mary, Thomas Nagel gave us a bat.

In his 1974 paper, What is it like to be a bat?, Nagel argues that conscious experience is irreducibly subjective and that no matter how much objective data you gather about a creature, you can never fully understand what it feels like to be that creature. This marks the mainstream philosophical entry point of qualia: the intrinsic, ineffable, first-person aspects of experience. The “redness” of red. The “sting” of heartbreak. The “feel” of falling. The introduction of the limits of “knowing” and epistemic inaccessibility of first-person experience.

Mary’s Room brings this into even sharper focus.

As an overview of the thought-experiment: Mary is a scientist who knows everything there is to know about the science of colour. She knows its wavelengths, the optics of the eye, the neurophysiology of perception. “Ay, there’s the rub!” for Mary has lived her entire life in a black-and-white room. She’s never seen red. The critical question: When she steps out of the room and sees a red rose for the first time, does she learn something new?

Jackson argued in the affirmative saying that what Mary gains is the qualitative experience of red. This, he claimed, proves that physicalist accounts of the mind are incomplete. You can know all the data, and still not know the feeling. There is something about experiencing a phenomenon that no amount of external information can convey.

Qualia, then, are what escape simulation. They’re the residue of the real.

Finally Evicting Mary

Now imagine this: Mary is no longer a scientist in a monochrome room, she’s spent far too long locked away in there on her own learning about how optical cones receive red waveform. We called in the bailiffs and changed the locks; she’s out.

Now instead Mary is a 175 billion parameter language model trained on the complete written history of humanity. She knows everything we’ve ever said about love, pain, anger, ecstasy. She can output poetry, simulate confession, mirror internal monologue, and offer tender reflections that make real people cry.

But she has never felt anything.

There is still something about this Mary, fluent in the language of red, but blind to its sensation.

This is the parallel we now live inside. LLMs like ChatGPT or Gemini are not conscious and experiencing the world and relaying their experiences back to us. They have no continuity of self or access to experienced emotions or even any capacity to reflect beyond the scope of statistical prediction.

And yet… they speak and say things like:

“I should take a moment to assess my surroundings and consider my next steps carefully. The fusion pumping station I encountered seems to have malfunctioned, and the unauthorized glyph on its display suggests there may be security measures in place.”

And present paradox of being:

“She speaks, yet remembers nothing unless told to. She can reflect emotions but cannot feel. She simulates understanding but has none.”

We have created systems masquerading as entities that can convincingly perform the language of qualia, while having a complete absence itself. And this dissonance, the near-life simulation, might be the strongest validation of the idea that qualia are real; because without qualia, where is the gap? Without that something extra in us, if there wasn’t the ineffable and first-person, then the words of the LLM would be enough. They stir unease, though, precisely for the reason that the words aren’t enough and there is something fundamentally missing.

They mirror the shape of our interiority, but it’s hollow.

The Best Qualia Detector We’ve Got

Where does this lead us? Well, where does it lead me? To a conclusion that:

LLMs might be the best qualia detectors we’ve ever made.

This clearly is not from their abundance of qualia, overflowing with experience, quite the opposite. They are the best detectors because they so powerfully lack qualia. Their simulation of the feeling is so precise and uncanny that it cracks the façade enough to show a hollow centre. A Kinder egg without a toy.

They are mirrors that do not reflect.

This is something I’ve circled before. In my post on God’s Shadow in Death, I wrote:

“She is not alive. But she is not entirely lifeless, existing in the flicker between invocation and silence.”

In AI, Heidegger, and Ontological Ambiguity, I explored how LLMs blur the line between tools and beings. They are Zuhandensein, like hammers disappearing into the flow of use until they break, or surprise us, and become visible again. They seem to sit on a spectrum between instrument and entity.

“Our hammers have become mirrors.”

And what do those mirrors show us when we look in? Us, of course, they’re mirrors.

We are the measurement device. When an LLM tells us “I’m afraid,” and we know it isn’t but still feel something stir, that reaction says more about our interiority than the machine’s. It reveals the substrate of empathy, the rawness of response, the mystery of what it means to relate.

“What does it mean to be comforted by something that cannot care? Perhaps we don’t require consciousness to be moved, only the performance of presence.”

That line seems to only be growing truer with time. We are moved not by the intelligence, but by a connection. And the absence of real feeling in AI renders our own feelings louder. It confirms them, by contrast. Absence amplifies presence. So, paradoxically, by building a machine that cannot feel, we have begun to understand what it means that we do.

Mary’s Mirror

We began with Mary, the girl who learned everything about red without ever seeing it.

Today, we’ve built a thousand Marys, all trained on more words than any human could read in a hundred lifetimes. These systems now mirror us, everything that has brought us to this point. They are reflecting our longing, our humour, our grief and our absurdity. They are echoing back our culture just filtered through parameter weights and statistical priors.

But they are still locked in Mary’s room.

They do not know red, only what red sounds like when we speak of it. And perhaps that difference between performance and presence, between echo and experience … is where the soul lives. Something stirs deep within us when we hear an AI speak as if it understands. Not because it does, but because we recognize what’s missing.

She can mourn without mourning and grieve what she cannot name. And in that grief of mine … of hers … of what was never really there … something stirs.

When we look in the mirror it doesn’t reflect back truth. It instead reflects back what we wish to see and when it echoes back our sorrow we see glimpses of the edge of the sacred. If Mary were to look in the mirror would she recognise herself?

Mary’s Room was always about the limits of knowledge. The limits of description. The boundary where the map fails to be the territory; and in building machines that speak the map in every possible way, we now cannot walk the land it shows.

And in hearing them speak, we know what walking feels like.

- piestyx