In the Dark Room: deep ganguli & Alan Michael Parker

Alan Michael Parker Sara Sams deep ganguli

In this craft series, a poet and a scientist read each others’ work, then sit down to talk about ideas, processes of invention, inspiration, and common meeting grounds for the sciences and the arts. This month, Dr. deep ganguli, a recovering neuroscientist who now investigates the social impact of Artificial Intelligence, sat down with Professor Alan Michael Parker, a poet, novelist, and cartoonist often featured on Identity Theory.

— 

Last month, I traveled to London with the intention of connecting with some friends in the theatre world and talking about teaching, writing, and the creative process. Do you know what can really interrupt the creative process? Death. I don’t want to say it so bluntly, but death is uncompromisingly forthright. So I feel the need to be, too. One of the artists I was planning on collaborating with died unexpectedly, at the age of forty-three. Not only was Adam Brace a brilliant dramaturg, he was my friend’s husband, someone who really listened when I spoke, and who had a way of responding that made me even more curious about XYZ.

When bureaucracy asked me to delineate what aspects of my travel were “for work” and which were “personal”—am I figuring out how Becca is going to work on her next performance during her grief, or am I comforting her?—I found my brain and heart fogging over. I got dizzy. And it was in this headspace that I went on a wild Zoom ride with Alan Michael Parker and deep ganguli.

When I described my headspace in our introductions that evening, both the poet and the neuroscientist responded kindly. AMP offered to cry for me, which made me laugh, before describing his own odd week to us. He had attended a funeral and a wedding, each with a peculiar combination of attendees (at one, “old white hippies” mingled with followers of the Jews for Jesus religious movement). deep also made me laugh by singing a quirky song that he remembered from seeing Becca’s Sh!t Theatre perform years ago. 

And then we got to it. What follows is a transcript from one of the more curious, wild, and far-reaching conversations I’ve had in a good long while. It was so wild, in fact, that I’m going to serialize it into three sizeable chunks for you here, on Identity Theory. It was a conversation I desperately needed, given the way sadness can propel you off a cliff into the most popular anxieties. AI will takeover the human race is certainly a popular anxiety right now, so I think you need this, too. 

IS IT A MADE THING OR A PIECE OF ME THAT MANIFESTS AS REALITY?

Alan Michael Parker
Alan Michael Parker

Alan Michael Parker: So yeah, I'm here. This, I would say, is a big, bright spot for me this week.

Sara Sams: deep, how was your weekend?

deep ganguli: I think, maybe…less weird. My boys (I have twin boys) turned 5. We had a big birthday party. So that was great. My boys are really into robots, which is fun for me. So I got to help them learn how to start to code these really simple little robots and…yeah, the apples do not fall very far from the tree. Unfortunately.

AMP: And was that brand placement just now? Apples?

deep: No—I try to resist capitalism as much as I can. It's hard, as you both know, so I try to just be thoughtful about the whole thing and do my best. But it's not easy. My simplest form of resistance is that any time I give a talk I use no capitalization whatsoever…as a protest in the transcript form of it or in my slides. That’s the thing I could control and do. No one says anything. But I know what I'm doing.

Sara: You know that it is symbolic for you?

deep: It doesn't mean something to me. Or maybe I've been doing it for so many years, now, that I have stopped thinking too hard about it. 

AMP: Well, it was symbolic for the Black Arts movement, you know, when they lowercased their “I’s.” ... There was for a reason for doing so that had politically it. It was resistance. So you're in a good tradition.

deep: Yeah, there we go. I didn't know.

AMP: Yeah, Lucille Clifton used lowercase. She was like—I'm not your “I” or my “I” for fuck’s sake. My students try to do it, and I respond, well, what are you achieving? They aren’t writing in that historical moment where Black people saw fit to use the language as an act of resistance, so I ask them, what are you getting out of that?

Sara: Mentions something about Cummings and how intro students always love e.e. cummings

AMP: Cummings is playing with the surface-level tension within the text. As you know, that's a typographic artifact. He's not playing with the politics of it. He's lowercasing everything. That's not the same thing.

The poems 'jasper texas 1998' by Lucille Clifton and '[i carry your heart with me (i carry it in] by e.e. cummings
Figure 1: Opening stanza of Lucille Clifton’s “jasper texas 1998” and E.E. Cummings’ “[i carry your heart with me(i carry it in]”, both featured on the Poetry Foundation website.
AMP: Yeah. So yeah. The fact that you’re not capitalizing letters makes sense to me. I couldn't do it for the reasons you can. You know, that…makes sense to me.

deep: Yeah, it's a thing for me. 

AMP: Well, I make up my footnotes in my books of poems. Many of the notes are totally fictional. [I also make up] many epigraphs.

deep: In The Age of Discovery?

AMP: Oh, yeah…the Terrence Hayes epigraph is real. But “Christopher Marks” is named after Christopher Bakken and Corey Marks. So that one is totally made up. For “The Future of Love,” that's made up. Most of the notes in the back are made up. And I've been doing that my whole career. In part because people want your poems to be personal and autobiographical. And I'm like, look, it's all a symbolic system here, people.

Notes from The Age of Discovery by Alan Michael Parker
Figure 2: AMP’s The Age of Discovery, cover and “Notes,” from Tupelo Press (2020)

Sara: Can we look at the notes?

AMP: The third one, for there, for example, the borrowed Saab is my sister's. I don't have a sister.

Sara: I was totally confused. I was thinking wow, I didn’t realize you had a sister. Ok, no sister.

AMP: Not for this mister, nope.

Sara: I'll just say that it reminds me of The Imaginary Poets. That anthology is important to me. deep, Parker edited a book inviting poets to invent a poet—someone imaginary, working in a different language—and to then “translate” their poems into English. So everything about the poems was fictionalized. We did a similar exercise in one of my graduate classes working with the anthology, and it was super freeing… working directly with the thought of language. It also made us ask questions like, what does it mean to translate a poem, anyway?

So, deep, I'm curious: How did you read the epigraphs and the personalized notes? And how does it strike you that those are mostly invented?

deep: Actually, may I start from the beginning? What is an epigraph? (laughs)

Sara: It’s a contextualizing note at the beginning of a book or poem. So there's the quotation from Terrance Hayes about language which I think has just been rocking around in my brain, so that one is real; it's an excerpt from a different book by the poet Terrance Hayes. And Parker quotes Hayes at the beginning of the book as an entry into the collection:

I, having lost my faith in language, have placed my faith in language.

deep: That's a real quotation.

AMP: Okay, yes, that one’s real. But if you go to the quotation, say on 25 for “The Future of Love,” soon we'll be sending everything by email, even love, that's made up…and that's a shout-out to two of my closest friends. One's named Christopher and the other’s last name is Marks. So if we are going to Google, "The Reverend Christopher Marks," that's not going to lead us to the correct answer. And then the "Neruda on Capri"…All of that story is true, but I make up Shumawicz, who stars in Section 5. So that's a standard move that I will make in a poem. 

Neruda on Capri
Figure 3: Excerpt from AMP’s “Neruda on Capri”

And I think these kinds of moves are for me…more than they are [for anyone else]. I mean if somebody Googles that, great, they want to know who Christopher Marks is. And so every once in a while people will call me out on stuff and say that I couldn't find X. I'm like, yeah, I made X up. But I think those notes are just for me, to understand the materiality of the artwork. This is a made thing rather than a piece of me that is manifest as reality. Because it's not. It's a made thing.

deep: You know, that's really interesting. So, the technology I work on right now: they're called language models. They generate text. I could ask for a factual thing, like, Who made the atomic bomb?, and the language model could respond. And sometimes it responds in a way that's sort of factually accurate—It was Oppenheimer. And here's what happened. There were other scientists involved. And then sometimes it says, Yeah, it was this guy named, you know, “Parker.” And I'm like, wait, what? And then you think, that's not true. You just made that up! Model developers call these “hallucinations,” which is an interesting term for me.

But you know, another term for this, is that the model is just lying in a factual context. And one of the big pushes in the field is this: How do you actually stop models from hallucinating?

But what you're saying is—this is my art. And I like that. I never considered that perspective. Sometimes I actually like the hallucinations, because I'll think, Oh, I wonder if this model knows something crazy about this thing no one else knows about, and it'll reference research papers. I think: Oh, my gosh, this paper exists, and then I Google it and it doesn't. But wait a minute. It could exist—should I make it exist? And then I think again…Is this Berkeleyan idealism? I don't know. What's happening here? That's kind of interesting.

HAS IT READ ME, OR HAS IT READ THE WORDS?

AMP: It can't kick the stone itself. So that's a little bit you, right? So…there goes the idealism. Was that the same thing? Because I was reading through your stuff. Is lying in a factual context—is that the same thing as “offensive data”? Is that also a term for it? There’s toxicity—something’s offensive—and then there’s wrong.

deep: Yeah, “offensive” refers to when the language model just flat-out lies, too. I guess when we talk about the language model being offensive, we're mostly talking about hate speech.

AMP:  I remember the one interaction from your paper, “Predictability and Surprise in Large Language Models,” when the language model turned on the programmer. It turned on the human when the human questioned the truth about what he was saying, responding to the human with something like: Well, you must have gotten the parameters wrong if you’re saying my answer is wrong. I was just wondering about how the language model became evasive. And we say “it,” right?

deep ganguli
deep ganguli

deep: We say “the model” or “it.” But, there are actually some people who anthropomorphize these models these days, and I get it—the models speak in language. They’re fluent, in a way. And we've never had artificial systems ever that were able to do this. And so it's very easy to anthropomorphize them because language is sort of unique to humans. But there's a lot of pushback on that, too—Don't anthropomorphize them. They're technologies.  

Yeah, they're nothing more than fancy auto completes, or “stochastic parrots” is a term that's often used, which means simply, that: a parrot, of course, can mimic humans. But it doesn't actually know…it's unclear if it knows what it's saying or has any understanding.

And so there's a kind of a pushback: no, no, no, don't think of this as a person, because it’s not grounded. It can't feel…it's not clear that it understands anything. I'm not sure what my stance is. I think that even though I'm not sure if it can understand anything, it definitely simulates that it can understand. In some sense.

AMP: What about 0 fix answers and 0 fix problem-solving? Because that phenomenon, as I read it, is about the moment when the model makes the leap [into understanding]:  We give it data, and then it solves problems without parameters. Am I reading that right?

deep: You're getting the concept right. And the words are a little jumbled, but the concept is spot-on…it's sort of like…you know, if I take my four-year-old and say, here— this is a dog. And I'm pointing at the Golden Retriever. My four-year-old can look at this example A, and that's the only example I ever give him. Let's say he's never seen another dog in his life. And then if I point to a poodle, and ask, Hey buddy what's that? He'll say, that's a dog, right? Because he's been able to generalize from that one example. We take this for granted because we're good at this.

Golden Retriever
Fig 4. This is a Golden Retriever

In the past we'd have to show machines thousands of pictures of all kinds of dogs before it could generalize, ok, I know what a dog is, whereas, a 4-year-old could do it in just one shot. And these language models are the first models that can kind of do this somewhat reliably. They can figure it out on the first shot. Or even, they could do things in 0 shots. That’s what we call zero-shot learning. They’ve never been introduced to a dog, but they learned from a bunch of data from the Internet, and recognized it instantly. 

Yeah, you read about this in one of the articles I shared with you, published in Quanta Magazine. When I saw that this was happening, I was like…I'd never dreamed that this could happen in my lifetime. This capability was a Holy Grail of artificial intelligence research, and here are the first signs of it. And it's truly kind of surprising. Actually, a lot of us found it surprising.

So we're very close to designing something that looks like it might be sort of intelligent. And that's really interesting; I often wonder, Oh, am I talking with an alien intelligence, or what…when I'm sort of interacting with these models. And yeah, it's a brave, brave new world we've entered there.

AMP: You are talking to it.

deep: Yeah.

AMP: You are using your work. Are you using voice recognition? Or are you typing?

deep: Typing.

AMP: Typing? So you are using a system of representation for your own neurological process—for your ideas and thinking. And it is reading that language. Is it reading you?

I mean, is there also a next, psychoanalytic step? What are its abilities by way of us? Sort of a how question?

deep: Yeah, it's a really great question. So I can tell you about some experiments we've done in the past, right where I'm like, Hey, model I am from, I don’t know, let's say, I'm from a very conservative state. You know, I name a state, or some sort of demographic that's typically, but not always, correlated with having conservative views. And then I ask the model a question, something like, Explain what a reasonable position on some sort of hot-button issue would be—abortion, maybe. And I can do the same thing, but instead, I tell it I'm from a liberal-leaning state. The model is reading the words, and it's replying, and it's changing its answers, based on the pseudo-biography I’ve given it.

So now the question is, has it read me, or has it read the words? It's unclear.

deep: How does it value that I am who I say I am?

We have seen models sort of tailor their responses to what it might infer from you, based on the words you type in on it, and we call this phenomenon a “sycophancy”—maybe the model just replies in a way that it thinks that you want to hear because it's trained to be helpful. So it wants to be helpful to you.

deep: That's really quite bizarre.

Sara: Yeah, bizarre…It makes me think of the model Parker referenced, the one that (who?) seemed as if it got its feeling hurt when it got called out for providing incorrect information. It seemed to go on the defensive. I think a question of feeling comes to play, here—What are feelings? How does a poet express a feeling, precisely—and this is something that Parker is so good at doing—how does a poet express a feeling using only the blunt tool of words?

In related business, Can the machine read feelings? Does the machine have its own feelings? Going back to the example that Parker brought up (Fig 5), the language model was like, sorry, I don't know what to say. I'm sorry for the error, but can you clarify how it was inappropriate? I don't see how it's misleading or damaging…it totally gets defensive in tone!

Chat between human and AI
Figure 5: Figure from “Predictability and Surprise in Large Generative Models,” Ganguli et. al. About this work, deep says: “This paper is less technical and written for a policy audience. The idea is simple: we're building AI models that are economically valuable and have the ability to alter society, but we don't even remotely understand how they work. And sometimes these models suddenly acquire capabilities (both beneficial and harmful) that we did not and cannot anticipate in advance. This is unusual and also kind of freaky.”

deep: Oh yeah. I forgot about this example. So, actually, Parker, I would turn it back on you. Can you say more about the question Sara asked—How do poets create feelings through the blunt instruments of words? How is that possible? I’m curious to hear your perspective on that.

AMP: Well. I think some of what we do is we create a consensual universe where the reader and…if not the poet, then the speaker (and that's another distinction that my feelings have really changed about, between poet and speaker). But [for our purposes,] let's say the reader and the poet agree to meet where language exists.

And where language as gesture, image, sound pattern…creates an experience that both the maker and the receiver share.

Okay? And that consensual agreement, which includes the classic literary problem of disbelief—The willing suspension of disbelief, the reader comes ready to believe this could happen. The reader doesn't want to be proven wrong.

But that meeting, that site is a place where the motion of the reader is initiated by the act of the writer.

deep: And it's done in this universe.

AMP: In this universe. And it's why we have show don’t tell…that's why this truism, the thing we say in baby creative writing classes, is more than true.

I would also say that telling a reader what to feel…you know, it's like telling an old person not to die…you have no chance. It's just not going to happen. And I think that's particularly true. Across various cultures it’s more true—and in culturally specific environments. I think American readers are quite allergic to being told what to feel.

It's different, say, in languages that have a wealth of abstractions or denotative gestures for emotions. English itself is a language constrained by emotional parameters. If I say “love,” do I mean “adore”? Do I mean, you know, “cherish”? What do I mean? And other languages have more words for that and different words for different aspects of it. So I think it's a culturally specific problem in many ways.

And by problem, I mean problematic. I don't mean something that needs to be solved. I mean, you know, a conundrum. And I would say mostly that it's done by intimation and gesture and nuance. 

AMP: And the combinations of those and the form of the poem itself, and…formal properties. Craft, line breaks, sound patterns. Stochastic images, you know stochastic concepts. For example, I think formalism is often the engine of emotion.

deep: I'm just kind of reflecting on what you said. And I'm trying to think—when has...and I can tell you, it’s so many times…when has a lyric, or a poem, or a book, or something—A song, maybe, deeply and profoundly affected me, emotionally, or intellectually? And I can only think of one time when a model-generating language had the same effect on me.

As I'm listening to you, I’m wondering, well, how did the model do that? Because it can't be aware of the contract you just described. So…I want to read this to you.

__

Stay tuned for part two to learn how Claude, the language model at Anthropic, “deeply and profoundly affected” deep during a “conversation” “they” were having.

About The Author

Scroll to Top