chatGPT R Us or I, Chatbot

first take: 

I am a materialist, by which I mean I believe that in the case of human beings in particular, there is no substance other than the physical body to account for our having a soul or mind or spirit – there is no “ghost in the machine.” Somehow the functioning of this brain and body in the context of a world and universe, which themselves contain no substance other than what we are accustomed to characterizing as physical, produces in us the impression of our having a soul or mind etc. It then becomes a matter of choice or utility whether we continue to posit the existence of souls and minds etc., albeit understood as not composed of any distinct substance from the body, or to banish them from our ontology, except as illusions or hallucinations or delusions.

            Now let us consider the remarkable new AI phenom chatGPT. Apparently the fundamental operation that makes this app tick is as follows: Compile a humongous body of digitized text – for example, Wikipedia – and then design a program or algorithm that, when fed a digital verbal string (the question), will generate text (the answer) based on the probability of what the next word will be.[1] For example, if you typed in “and so,” it might spill out “on” if that is the most likely word to follow “and so.” If you prompted it with “and so on,” it might then produce “and so forth” as the most likely string to follow “and so on.” And so on and so forth!

            That chatGPT is, based on this utterly simple operation, able to produce the texts we have seen, is remarkable beyond ordinary belief. It would seem a hoax, like the mechanical chess player (“Turk”) that really had a person inside it, if not for the authoritative institutions that stand behind it. No, it is not yet as articulate or idiosyncratic as an actual human being is capable of being; but it is nevertheless an amazing achievement. Indeed, its very imperfections can mimic the typical responses of human beings.

            What prompts my writing this essay (and, to give away the punch line, does so in essentially the same way that a question prompts an answer from chatGPT) is the thought that what makes chatGPT seem so remarkable is that it appears to understand any question I pose to it. For example, here is an exchange I had with it:

“Do you just quote sources like Wikipedia, or do you compose the wording of your answers from scratch?”

To which chatGPT responded:

“As an AI language model, my responses are generated by algorithms that draw from a vast database of language patterns and knowledge. While I may use sources like Wikipedia or other online resources to provide accurate and relevant information, I do not simply quote them verbatim. Instead, I use my language generation capabilities to compose my responses in my own words, tailored to the specific question and context.”

            As impressive as that answer is, to me what is most noteworthy is that it feels irresistible to say and feel that chatGPT must have understood my question in order to produce that answer.[2] It is beyond my intuitive mind to cognize chatGPT’s answer as the result of only predicting what word or words follow some other word or words. That it is the result of that “dumb” process – that is what is astonishing.

            However, my mind is (presumably) capable of grasping how this trick is accomplished … provided I were to become immersed in computer stuff. Or at a minimum I can say that I do not seriously doubt that programming experts can pull off this trick, although I really would have doubted it had I not “seen it with my own eyes.”

            And this is not at all an unusual phenomenon. In fact just about every modern-day convenience is just as marvelous. Do I have the slightest idea how a performance of Beethoven’s Ninth Symphony can be conveyed to me from an orchestra playing on the other side of the Earth? Yes and no. It seems magical … when I am not simply taking it for granted. Clearly today’s children will take chatbots far more extraordinary than today’s chatGPT thoroughly for granted (although, if they have not been genetically engineered otherwise, they, like me, in a moment of reflection, will still be susceptible to wondering at it).

            Furthermore even natural phenomena partake of the same wonder. Hume pointed out that cause and effect can obtain between the most oddly paired things, provided only that we become used to it. But is it not wonderful that when you let go of something, it drops to the ground? Is this not a miracle, looked at without (per impossibile) preconceptions? But Darwin has shown us that even the intuitively irresistible perception of all of nature being fitted to every other part of nature, such that it must have been intentionally made so (“designed”), can itself be deconstructed into a simple fundamental principle that works without intention (survival of the fittest).

            But the amazingness of chatGPT’s seeming understanding is still not my punch line. The truly astounding revelation is that the “illusion” of chatGPT’s understanding is equivalent to our attributing understanding to human beings … even to oneself.[3] For – at least on the materialist view – we ourselves are physical beings functioning entirely according to natural laws, and hence are, in this sense, mechanisms exactly like chatGPT.[4] And as a kind of evidence of this, ask yourself: Do you have any more idea what will come out of your mouth before you say it, or even into your mind before you think it, than chatGPT does of what it will write in answer to your question?

            Granted, chatGPT was programmed by intelligent beings, and so is “artificial,” whereas nobody programmed us, so we are “natural.” So the laws that govern our behavior are the laws of nature, whereas the laws that govern chatGPT’s behavior (or, more precisely, the “behavior” of any material object that is running it) are both natural and intentionally imposed. But, it seems to me, in the most fundamental way we are the same, in that we both are entirely physical in composition and (ultimately) governed entirely by laws (if only statistical at the quantum level).

            So here again we have a choice: We can (to suit our purposes) maintain either that we do not understand things -- no more than does chatGPT (nor even a more mature chatGPT that truly will pass any Turing test) – or that we do understand things, but so does chatGPT. I see this as exactly analogous to choosing between asserting that we have emotions and so do other animals, or that nothing has emotions. Some animal experimenters are notorious for claiming or assuming (unless “proven” otherwise) that we have emotions but other animals do not … and no doubt some cognitive scientists and philosophers would claim that we have intelligence and understanding, whereas robots do not and could not. But my main point in this essay is that chatGPT demonstrates in a particularly striking way that this dichotomy is purely tendentious, and that it’s really all or nothing: Either everything that appears to understand does understand, or nothing does. In other words, I am arguing that if it looks like a duck and sounds like a duck, then it is a duck. 

second take: 

By now you have probably succumbed to your curiosity and given chatGPT a try. And you have been as amazed as anyone. In no time flat it begins to type out an answer to whatever query you have posed, and that answer appears wholly intelligent. You can even carry on a conversation by following up your initial question with a second one in response to chatGPT’s answer, which it will then answer in turn, and so on indefinitely. It seems as if chatGPT or a successor chatbot stands a good chance of passing the Turing Test, which is to say, it may soon be indistinguishable from a human interlocutor. After all, even now would you be able to tell if this very article is being written by chatGPT? Indeed, and ironically, my own opinion (assuming you believe I am the human being who appears to be authoring this article) is that the most likely reason a chatbot would fail the Turing Test is that it will have a better command of “proper” English (or whichever language) and sound logic and the knowledge base than the average human speaker or writer or thinker.

            Nevertheless, most of the experts who have written about this remarkable new product of AI have assured us that chatGPT is not, and could never be, intelligent. The reason is that this chatbot functions differently from a human brain, and in fact by the “mindless” following of an algorithm or set of rules. It is nothing but a program running inside a piece of hardware somewhere, and hence its appearance of intelligence is just that – an appearance. No matter how closely a simulacrum matches the real thing, it ain’t -- no more than a perfect replica of the Declaration of Independence is the actual Declaration of Independence that John Hancock signed.

            That argument is persuasive. However, I do not consider it definitive. Let me present a very simple counter-argument. Return to your original moment of amazement, as chatGPT begins to type out its answer to your first question. Now pause to reflect. When you typed your question, were you doing anything less amazing? Most likely you did not mentally rehearse your question before typing it on your keyboard. Your fingers began to tap, and right before your very eyes your question appeared on the screen in front of you in exactly the same way chatGPT’s answer did. Both you and chatGPT drew upon some unknown source to produce your respective question and answer. Even if you had mentally rehearsed your question beforehand, the same is true: There was no pre-rehearsal rehearsal, for that would lead ultimately to an infinite regress.

            We could also say that both of you relied on the functioning of a physical object: in the case of chatGPT, a computer somewhere in the Cloud, and in your case, your brain. I still call these sources “unknown” because you yourself (unless you are a computer engineer or a brain physiologist) have very little conception of what produced the words on the screen. And in fact, the illusion of an intelligence typing the answer is matched by an illusion of you typing the question -- that is, if you conceive of yourself as an intelligent mind or soul or self, and not just a(n admittedly remarkable) mass of brain tissue.

            We attribute all sorts of magical qualities to our “self,” such as intelligence and free will. But, if you are a materialist anyway, you believe that it’s really the brain that is running the show, and the brain works according to deterministic laws like all the rest of the physical universe. (Note: At the quantum level the laws may be statistical, but lawful they remain.)    Granted, the particular way in which the brain functions is different from the way chatGPT functions. But does a difference at the micro level always signify a difference at the macro level? A fly’s eyes function differently from a human’s eyes, but do we hesitate to call both eyes, and say that both flies and humans see? No, we do not. So are the experts really justified in denying intelligence to chatGPT (or its successor) just because its behavior is produced by a different mechanism from our own? My argument is that it is a matter of convenience or utility whether we designate chatGPT or human beings as intelligent. This position is sometimes called instrumentalism: We attribute qualities to thing based on the usefulness of doing so.

            Here is another way to think about this. The philosopher Ludwig Wittgenstein famously argued that the meaning of a word derives from its use. This hit home for me the other day when, once again (like now), I was typing away on my keyboard – this time replying to an email from a colleague – and found myself typing the word “nominal” in a sentence. For years I have been hazy on the meaning of that word, but have come across it now and again. My colleague, who is an engineer, had used it some time previously in a technical discussion we were having, and I had taken the opportunity to ask him what it means. He gave me an explicit definition. But I must admit it still left me uncomprehending.

            Lo and behold, there I was spontaneously using the word myself – its coming, as if “ex nihilo,” just like all the other words I wrote. And I was also confident in my usage. How had this happened? I surmise that I (or my brain), after years of exposure to the word being used by others, must finally have crossed some threshold and grokked the meaning of the word. I had picked it up by a kind of osmosis. And is this not precisely analogous to how chatGPT comes up with its sentences, through exposure to a mass of data input? And now I can push the point further and argue that it suggests not only that intelligence and the self are (if we find it useful to think of them this way) illusory, but so is meaning. Or, like Wittgenstein, we can choose to “retain” the notion of meaning as something real, but analyze it as nothing but how we use a word. There is nothing “deeper” here. So why insist there must be for chatGPT? Once again: It is a matter of choice or convenience or utility how we decide what trait to assign to something. Reality itself is mind-dependent in this way.

            Still, the experts will object that surely chatGPT is not conscious (and surely we are). And I agree. But my reason is the same for nevertheless ascribing intelligence to chatGPT. I, and I suspect you, find it intuitively irresistible to imagine we are conversing with an intelligent being when we use chatGPT. But we balk at attributing consciousness to a computer. Now, that may be a mere prejudice; and if chatGPT were hooked up to a metal or plastic body with a fully human face, my intuition about that could change too. Alternatively (again, it is a matter of choice) I am very sympathetic to philosopher Daniel Dennett’s notion of consciousness as itself an instrumental illusion. But even if we deem it something real, now that I understand verbal meaning to be a matter of how we use words, what is so important about being conscious of that meaning? Does my consciousness add anything to my ability to type the very words I am typing now? I doubt it. So how could the denial of consciousness to chatGPT prove detrimental to the attribution of intelligence to it?

            I conclude that chatGPT, despite its obvious differences from us, might nevertheless be intelligent.

---

[1] “ChatGPT generates realistic responses by making guesses about which fragments of text should follow other sequences, based on a statistical model that has ingested billions of examples of text pulled from all over the internet.” (Benjamin Weiser, “Here’s What Happens When Your Lawyer Uses ChatGPT,” The New York Times, May 28, 2023)

[2] Not that I am not also astounded by some AI responses. For example, when as a test I asked Google AI to give an account of my ethical theory of desirism, it provided five good extended examples of the theory’s application, none of which I myself had ever written about. On another occasion chatGPT suggested the addition of a sentence in the summation of a draft essay, which indeed saved the day, since otherwise my main point could easily have been missed by the reader.

[3] I am, in effect, offering a counterargument to Searle’s famous Chinese Room.

[4] In other words, there is no homunculus, no mechanical Turk inside us either.