Tuesday, January 30, 2007

Chinese Panic Room

I seem to be writing what is largely a philosophy blog these days. This is problematic insofar as I have read nearly no philosophy in my life. The list pretty much begins and ends with Plato. As such, I tend to run roughshod over concepts and thought experiments that other people have already written, published, and forgotten about; I instead remain blissfully ignorant of the intellectual plagarism I may or may not be committing. I plan to solve this problem by majoring in Philosophy in my next life, though that will of course create several other problems, such how I'm going to feed myself in that life (That was not a dig at Sam. I was taking the piss out of Socrates there. Ha ha Socrates. You've totally been punked). Fortunately, I am friends with people who have read Philosophy, and they, intentionally or un-, draw my attention to books, ideas, or thought experiments that I have never otherwise heard of. Here is the most recent example; Periapse's comment (What? You haven't read it yet? Sadly, we can now no longer be friends) led me to Searle's thought experiment The Chinese Room, about which I will blog, starting now.

The Chinese Room is an argument that humans are not merely computational machines; it posits that a computer can act human (pass the Turing Test, in AI-speak) without either being conscious or having an understanding of meaning in a human-like way. It goes like this: there's a room with a computer terminal on the outside. The terminal is capable of carrying on a conversation in Chinese, such that a Chinese speaker can walk up to it, start typing, and the terminal will be able to carry on a conversation in Chinese with that person. The Chinese speaker thinks, "Aha! An artificially intelligent computer that speaks Chinese! How amazing!" Unbeknownst to the speaker, however, is that inside the room is Searle, sitting at another terminal with an elaborate rulebook in his hand. When someone comes up to the terminal and starts typing, he consults his rulebook, and types back whatever it tells him. He doesn't speak Chinese or understand what he's typing, nor does the rulebook define anything for him--he just types back the symbols that correspond to the symbols he receives, thereby carrying on a perfect conversation without ever understanding what he's saying.

It so happens that after college I won a Parshull-Dimm scholarship and went to the remote island chain of Chai-Neesrüm to study an isolated tribe of Pacific Islanders. The Chais and the Neesrüms are unique in that they communicate with very subtle changes in their facial expressions, the same type you make unconsciously while speaking--raising the brow, flexing the jaw muscles, softening the eyes, and so on. In the process of doing this, they make sounds with their vocal chords, but this is completely superfluous to their communication with each other, and they're only dimly aware that they do it--it's just a strange byproduct of their facial movements, and for some reason they can't seem to stop doing it. What I discovered as soon as I arrived in Chai-Neesrüm is that the sounds they make perfectly resemble spoken English. I stepped off the plane and the matriarch of the village came up to me, and while subtly lowering her eyes and shifting her chin to the left, voiced a series of sounds which sounded identical to the English phrase, "Dude! You rock like Slayer!" I said, "Uh...Thanks!" but of course without realizing it raised my eyebrow and turned the corners of my mouth upwards. This, of course, meant something totally different to her than it did to me, and...let's just say that I spent the next three months thinking that we were discussing Star Trek, The Next Generation, while they thought I was giving them an incredibly detailed, narrative-driven recipe for Gazpacho. Needless to say, the Parshull-Dimm committee absolutely ate up my report and the subsequent fame in anthropological circles has sustained me to this day.

Being that, to review, I've read only the Philosophical equivalent of Fun With Dick And Jane, I'm sure somebody has made the critique of The Chinese Room that I'm about to make--just because it's not on the Wiki doesn't mean somebody hasn't written, published, and forgotten about it already. But here it is anyway: Searle, in this case the person sitting inside the Chinese Room doing the translation, is a conscious, Strongly (un-)Artificially Intelligent entity. The Chinese Room doesn't prove that he isn't, it proves that this intelligence doesn't derive from its ability (or lack of ability) to understand written Chinese. Humans, as it happens, do all manner of tasks to which we don't attach, or don't agree on, special imbued meaning. We respond to light, pain, pressure, sound and/or temperature in extremely elaborate and varied ways. We respond to food by breaking it down and digesting it. We respond to air by taking it into our bodies, absorbing the oxygen in it, replacing it with carbon dioxide, and expelling it again. The shared meaning or lack thereof of these acts do not define the strong AI we happen to also possess. I wrote the above example mostly to amuse people who otherwise have absolutely no idea what I'm talking about at this point, but also to hopefully make this idea a little more clear. Each side in the Chai-Neesrüm dialog could reasonably conclude that the other was a fluent speaker of his or her dialect, but the entire time neither party understood the meaning the other was taking from the "conversation." Each side instead interpreted the byproducts of the other's communication--byproducts which we could reasonably conclude were entirely mechanical, entirely unconscious--as having meaning. You can argue all day about whether "meaning" is really "exchanged" in this conversation, but that's not the interesting point. The interesting point is both parties are still Strong AIs, and "meaning" and "understanding" have no bearing on this point.

This seems to me to have all sorts of interesting ramifications not just for AI but for our own non-artificial intelligence, especially as to where that intelligence "resides," so to speak. You're already completely confused, though, so I'll leave that until the next post.

Next: Confusion is nothing new!


Sam said...

No, no, NO: I'm not even one little bit confused. You have positively nailed it here.

It's not just that Searle's totally missing the point, though he is (and why chinese? what's up with the unnecessary latent orientalism?). You see, as far as I'm concerned, your little Island example doesn't just disprove Searle. The Island example serves as a perfect model for how we always communicate .

Understanding and meaning are just these crazy, lucky byproducts of the massive misunderstanding and different-rule-following that goes on all the time.

Transient Gadfly said...

What I found interesting was that the criticism of the argument that I found (and mind you, this is just in five minutes of searching, I'm sure there's better criticism out there) fell into the same trap: they tried to argue understanding or consciousness back into the experiment, e.g. the system of Searle and the rulebook understood Chinese, even if Searle didn't, or that we needed a new definition of "understand" or "meaning" or something like that.

That, of course, and the illustration given by my example, is an enormously long and involved sinkhole of a topic. So of course I'll probably blog about it next time. Mmm...sinkhole.

Periapse said...

Wow. I see this post as much more than a novel critique of the Chinese Room. More even than an observation on the reality behind communication (which I agree with Sam it is). If I understood it correctly it's a direct challenge to the Turing Test itself.

If understanding communication can be so problematic, then what value is there in an Intelligence test based on it? We should probably be thankful that any kind of meaningful communication (between human intelligences) is possible at all and just leave it at that.

This, to me, leads in two directions. One is to further explore how our internal Theory of Mind (that which allows us to model mental states of others and predict behavior) relates to communication potential and our ability to ascribe Intelligence to other beings based on it.

The other is to follow up on the implications for communication itself. Some people have actually embraced the idea that communication, particularly text, is a problematic endeavor. I believe it was Yale critic Harrold Bloom who first advanced the idea that all strong readings of text are actually mis-readings (in "Anxiety of Influence", if my memory serves). It may be that the "narrative-driven recipe for Gazpacho" (btw -- one of my favorite lines from the post) is actually more useful and interesting than the intended discourse on STNG.

So much to think about. So little mental bandwidth. I need a meta-cortex.

alicia said...

When i read this post i felt like i did when i had to read being and time in college. either you are so smart that your observations are beyond my pee-brain comprehension or you just don't make any sense.

a third possibility is that i only sleep 4 hours a night so i am now crazy from sleep deprivation and unable to comprehend anything.