The Chinese Room and Chinese Robot as Distinct Thought Experiments

(This is a third draft of a paper which I am working on for my Philosophy of Consciousness seminar. It is also a paper which I may eventually use for my writing sample, so thoughtful criticisms are greatly appreciated.)

The Chinese Room and the Chinese Robot

as Distinct Thought Experiments

In his seminal paper, “Minds, Brains and Programs,”[1] John Searle noted with seeming amusement that his opponents could not agree on what the proper reply to his Chinese Room thought experiment should be.[2] The objective of this paper will not be to add to the confusion which surrounds Searle’s argument by defending a reply which has already been offered, or, worse still, by providing yet another reply to his argument.  Instead, my primary focus will be to describe where the debate between Searle and his interlocutors has gone astray as well as point out where Searle’s argument is in critical need of clarification.  More to the point, I will argue that both sides of the debate have failed to appreciate, if not recognize that there are two highly distinct thought experiments which are at play throughout Searle’s paper, each aimed at illustrating a very different point.  Once these separate thought experiments, along with their independent conclusions are properly isolated from each other, one can then clearly identify the points of difference between Searle and his opponents as well as the exact point at which each side in the debate begins to speak past the other.

Furthermore, it will be shown that the conclusion which Searle draws from the Chinese Room thought experiment is strongly motivated, but seemingly irrelevant to the question at hand from the computationalist’s perspective namely, this question being whether the instantiation of the proper program is sufficient for the creation of a mind.  On the other hand, the conclusion which Searle takes the Chinese Robot thought experiment, as I will call it, to illustrate does cut to very heart of the computationalist program.  It is unclear, however, whether this second conclusion is at all motivated by the Chinese Room or anything else.

The Schank Program and the Chinese Room

Searle’s Chinese Room argument, which I will equate with the Chinese Room thought experiment for the purposes of this paper, should be read as a response to various claims which were supposedly made by computationalists regarding a computer program designed by Roger Schank.  Both the nature of Schank’s program and the claims made about it, we will see, are at the very heart of the Chinese Room argument and therefore merit our close attention.  What Schank did was program a machine such that it could pass a Turing Test of limited scope.  It was designed to receive some story as input after which it would answer questions which were then put to it about facts which are not explicitly stated in the story.  In order to do this, the program would extract implicit information from the story by comparing it to a databank of information, with which the machine had been pre-programmed, according to a set of rules, with which it had also been pre-programmed.  Having thus extracted the relevant information from the story by implementing such a program, the machine would then proceed to give answers to the questions that had been put to it.

The following two claims which many computationalists made with respect to Schank’s program are the specific targets of Searle’s Chinese Room argument:

(1)               The machine implementing the Schank program can literally understand the stories which are presented to it.

(2)               What the Schank program does explains the ability of humans to understand stories which are presented to them.

For the purposes of this paper, we will limit our attention almost exclusively to the first of these two claims, for a detailed treatment of how the Schank program relates, if at all, to human understanding would take us far beyond the present scope.  For the sake of clarity, we will break down (1) into two distinct claims:

(1a)      The machine implementing the Schank program has a mind of some kind.

(1b)      The mind produced by the machine implementing the Schank program understands the stories which are presented to it.

It is important that we clearly understand how these assertions regarding the Schank program relate to the claim that the Turing Test is a reliable indicator of understanding within a system.  The Turing Test, which the Schank program was designed to pass, is claimed to indirectly establish (1a) by directly establishing (1b) according to the following reasoning: if something can behave as if it understands a story, then it must actually understand the story.  Since (1a) is taken to be logically entailed by (1b), so the reasoning goes, the ability of some system to pass the Turing Test is a reliable indication that it must have a mind.  One of the conclusions which Searle draws from the Chinese Room argument is that the Schank program is able to pass the Turing Test without actually understanding anything of the stories fed to it or even having a mind at all.  Consequently, the Turing Test is not a reliable indicator of understanding within a system.

Let us now consider Searle’s Chinese Room argument in greater detail.  Suppose that John Searle, who knows no Chinese whatsoever, is locked inside of a room.  (For the purposes of clarity I will refer to the man in the Chinese room as “John” while reserving the name “Searle” for the author of the Chinese Room thought experiment.)  Inside this room, John is equipped with a huge book filled with nothing but Chinese characters and another book filled with rules written in English, books which respectively correspond to the databank and the rules with which the Schank machine was pre-programmed.  John is then fed a second large batch of Chinese characters which, unbeknownst to him, contain some story analogous to that fed into the Schank machine.  John is then fed a third batch of Chinese characters which, still unbeknownst to him, contain questions about the story contained in the second batch.  John then proceeds to follow the rules provided in his English rulebook by comparing the second batch to the first batch in order to respond to the third batch with yet a fourth batch of Chinese characters which, unbeknownst to John, contain answers to the questions received in the third batch.  The conclusion which is drawn from this thought experiment is that, despite the fact that John is passing a Chinese Turing Test of limited scope by shuffling all these batches of Chinese characters in the proper manner, he understands nothing of the relevant stories, questions or answers.  Additionally, no matter which English rules John is supplied with and no matter how big and detailed the databank of Chinese information is, he will never come to understand the Chinese characters as anything more than ‘squiggle squiggle’ or ‘squoggle squoggle.’  Consequently, since the Schank program has nothing which John does not have in the Chinese Room, (1b) must be false.  It also seems extremely doubtful that we can seriously endorse (1a) in the case of the Schank machine either.

The primary conclusion which Searle draws from the case of the Chinese Room is that syntax is not sufficient for semantics, or, to put it another way, symbol manipulation does not guarantee the presence of meaning.  At this point, however, it behooves us to be clear about what this claim actually amounts to.  What the Chinese Room illustrates is that one cannot understand what Chinese characters mean solely in virtue of their formal relations to other Chinese characters, regardless of what formal relations these turn out to be.  The Chinese characters which are being passed to, from and within the Chinese Room are meaningless to John since, from his perspective, they reference nothing but more Chinese characters.  The Chinese character for ‘hamburger’ does not refer to actual hamburgers, for there are no hamburgers in the Chinese databank or English rules which John is supplied with.  Rather, the Chinese character for ‘hamburger’ refers to the Chinese characters for ‘food’, ‘beef’, ‘bun’, etc. which in turn refer to nothing other than still more Chinese characters.  “Syntax is not sufficient for semantics” in the case of the Chinese Room should be read as follows:

What The Chinese Room Shows:

The meaning of linguistic symbols cannot be understood by a mind solely in virtue of their formal relations to other linguistic symbols.

At this point we must be clear about what claims have and have not been refuted.  As noted, at least (1b) and perhaps (1a) seem to have been thoroughly refuted by the Chinese Room argument.  Accordingly, the claim that the Turing Test is a reliable indicator of understanding in a system has also been called into serious question since the Schank program passes the Turing Test without understanding anything.  It must, however, be recognized that computationalism does not logically entail any of the above claims.  The computationalist program has, strictly speaking, relatively little invested in the claims (1a), (1b) or that regarding the Turing Test.  The computationalist can certainly grant that it is possible for a machine to pass the Turing Test without implementing the right program, and accordingly hold that the Schank program is just such a case.  Searle has certainly undermined the claims made by some computationalists regarding the Schank program and the reliability of the Turing Test, but he has to this point done very little to undermine the claim which lies at the heart of computationalism, namely that implementing the right program in some physical system is sufficient for creating understanding in that system.  From the perspective of the computationalist, the specific conclusion which should be drawn from the Chinese Room is a (rather trivial) claim about language rather than the relation which may or may not hold between computation and the mind.  In the Chinese Room, as Searle has thus far described it, consciousness is a rather peripheral issue at best and as a result the computationalist can simply react with disinterested agreement.  At the very least, Searle owes the computationalist a more detailed explanation for why the latter has anything invested in the Chinese Room argument as it has been described thus far.

It is significant that Searle never provides such an explanation, nor does he spell out in much detail what conclusions can and cannot be drawn from the Chinese Room argument.  Rather, he leaves what conclusions he does draw in a less specific, and therefore more general form: “syntax is not sufficient for semantics”, “formal programs cannot produce understanding”, “symbol manipulation does not guarantee meaning”, etc.  At least some readings of these claims have certainly been established by the Chinese Room argument, but Searle has left it unclear what these readings are and whether such readings are at all relevant to the computationalist program. 

It is at this crucial point in his paper that Searle transitions from the relatively modest, yet ambiguously phrased conclusions reached in the Chinese Room thought experiment to a pointed criticism which clearly strikes at the very heart of the computationalist program:

“One of the claims made by the supporters of strong AI is that when I understand a story in English, what I am doing is exactly the same – or perhaps more of the same – as what I was doing in manipulating the Chinese symbols.  It is simply more formal symbol manipulation that distinguishes the case in English, where I do understand, from the case in Chinese, where I don’t.  I have not demonstrated that this claim is false, but it would certainly appear an incredible claim in the example.”[3]

It is with this last claim that Searle completely shifts the burden of proof onto the computationalist as well as the premises from which the latter must argue.  In other words, at this point in his essay Searle ceases to criticize computationalism and instead embarks on the task of defending the common sense conclusions reached in the Chinese Room argument from computationalist claims.  Since the computationalist supposedly thinks that our understanding of English is “exactly the same” as what happens in the case of the Chinese Room, and since the Chinese Room obviously does not understand anything, computationalism cannot be an accurate account of our understanding of English.  It is with presumption (apparently) on his side that Searle can make such claims as the following:

 “No reason whatsoever has been offered to suppose that [formal principles] are necessary or even contributory [to understanding], since no reason has been given to suppose that when I understand English I am operating with any formal program at all.”[4]

The strategy which Searle adopts throughout the remainder of his paper can thus be summarized as follows: The Chinese Room shows that syntax is not sufficient for semantics.  Computationalism is committed to the claim that syntax is sufficient for semantics.  Unless the computationalist can provide us strong reason to think otherwise, we can safely assume that his position is false.

The Robot Reply and the Chinese Robot

As noted earlier, the clearest and most compelling conclusion drawn from the Chinese Room seems to have little relevance to the computationalist program.  It is with deep suspicion, then, that we should read Searle’s claim that what the Chinese Room does and what the competent English speaker does are, according to computationalism, exactly the same.  In order to make the contrast clearer, let us juxtapose the Chinese Room and the competent Chinese speaker.  The most obvious difference between the two is that while the former never encounters any hamburgers, beef or buns with which to match up with the Chinese characters for ‘hamburger’, ‘beef’ and ‘bun’, the latter does.  The computationalist’s Robot reply, to which we now turn our attention, should be interpreted as an attempt to correct this important difference between the Chinese Room and the competent Chinese speaker, the same difference which exists between the Schank program and the competent English speaker.  If we had a machine which was exposed not only to Chinese characters, but the objects to which such Chinese characters referred as well, then we would have no reason to think that such a machine, when properly programmed, did not understand Chinese. 

Let us pause at this point to be more explicit about the exact nature of such a machine.  In his paper, Searle suggests that the Robot reply tacitly concedes that something in addition to proper computation (environmental interaction to be specific) is necessary for a machine to understand.  Such, however, is not the case.  The Chinese Robot does not need to literally interact with its physical environment; it only needs to be programmed for such interaction.  The program for environmental interaction is what is essential to the machine understanding language, not physical interaction with the environment, and such a program could be created by way of mere virtual interaction or false memories of environmental interaction for example.  The point is that when the computationalist refers to an interactive robot he need not be speaking of a robot which is literally mobile.  Indeed, throughout the remainder of this paper it will often to be more helpful to assume that the robot does not physically interact with its environment in any way beyond the mere exchanging of stories, questions and answers as in the case of the Chinese Room.

In response to the Robot reply Searle provides what I have been referring to as the Chinese Robot thought experiment.  Suppose we give the Chinese Room arms, legs, eyes, etc. such that it is now able to physically interact with its environment (even though we have just seen such things to be unnecessary, strictly speaking).  This, Searle suggests, does nothing to change things from the perspective of John who is still inside the Chinese Robot.  All these interactions (be they literal or merely virtual) only amount to more Chinese characters which, unbeknownst to John, correspond to the robot sitting down in a restaurant and ordering a hamburger.  The robot’s environmental interactions, from John’s perspective within the robot, are just so much more syntax and we have already seen that syntax is not sufficient for semantics.  If the Chinese Room does not produce the relevant understanding, then, according to Searle, neither does the Chinese Robot, for the latter is just a modified version of the former.

The main thesis of this paper, however, is that the Chinese Robot is not just a modified version of the Chinese Room, but is actually a completely different thought experiment altogether.  There are two significant differences between the two which are worth pointing out.  First, there is a difference between the two accounts as to what, if anything, the Chinese characters which John manipulates refer to.  In the case of the Chinese Room the Chinese characters were actually stories, questions and answers which John was reading but not understanding.  The case of the Chinese Robot is radically different, for in it the Chinese characters which John is manipulating no longer contain any stories, questions or answers but instead correspond to the robot reading stories, questions and answers.  In the former case, the Chinese characters are identical with the input/output of the system while in the latter the Chinese characters correspond to, but are not identical with the input/output of the system.  In the case of the Chinese Room, the characters are linguistic symbols which have linguistic meaning, meaning which John fails to understand as he reads them.  In the case of the Chinese Robot, however, the characters are no longer linguistic in nature and accordingly have no linguistic meaning which John fails to understand. 

This last point is in need of further elaboration and justification.  It can be illustrated, however, by how it is possible to modify the Chinese Robot thought experiment in ways which the Chinese Room does not tolerate.  Suppose we were to replace all the Chinese symbols which John is busy manipulating with 1’s and 0’s.  The Chinese Robot tolerates such a modification while the Chinese Room does not, primarily because the latter depends upon there being linguistic meaning in the Chinese characters which John fails to understand.  In the case of the modified Chinese Robot, however, the 1’s and 0’s refer to nothing other than more 1’s and 0’s and there is no reason to think that they need to refer to anything beyond this.  The same can be said for the Chinese characters which John manipulates within the Chinese Robot; while such characters certainly do not merely refer to more characters within their normal linguistic context, John’s environment within the Chinese Robot is not such a context.  Let us approach this point from a different angle by modifying the original Chinese Robot scenario in a different manner.  Suppose that we replace John with a competent Chinese speaker who takes over the manipulation of the Chinese characters within the robot.  While such a speaker would certainly understand what each Chinese character usually refers to, it would be abundantly clear to him that the symbols which he is manipulating refer to no such things, for the strings of characters are nothing but (Chinese) character-salad.  While it might be possible for Searle to hold out at this point by suggesting that the strings of Chinese characters actually describe the robot’s environmental interactions, it is not at all clear that the computationalist is committed to such a scenario.  Furthermore, if it actually were the case that the Chinese Robot translated the actions of the robot into the meaningful Chinese sentences which referred to such actions, then we would almost surely be correct in attributing a significant amount of understanding to the non-human system, contra Searle.  In summary, whereas the Chinese Room rests on the strong intuition that John fails to understand some meaning in the Chinese characters which is there to be understood, there is no such intuition in the case of the Chinese Robot because the Chinese characters in the that case have no meaning for a mind to understand.

The second important difference between the Chinese Room and the Chinese Robot will be important to our later discussion of the Systems and Combination replies which Searle engages in his paper.  In the case of the Chinese Room, the Chinese speaking system is or at least can be a subsystem of John while in the case of the Chinese Robot John is necessarily a subsystem within the Chinese speaking robot.  (This is to assume, of course, that the Chinese Robot actually reads and speaks Chinese, although Searle does not explicitly say such a thing.  Let us again modify the scenario a bit by positing that the Chinese Robot has come to learn to speak and read English.  Within such a scenario John would be totally unaware of such a change in the robot’s abilities, because any English communications are, from John’s perspective, just more Chinese characters being manipulated.)  Whereas in the case of the Chinese Room, the walls separating John from the world contribute nothing to the system, in the case of the Chinese Robot the walls (or skin) separating John from the world contribute a great deal to the system.  The fact that the Chinese Room is a subsystem within John accomplishes two things:  First, it makes it possible for John to internalize everything which is happening in that thought experiment.  Second, it motivates the intuition that the room cannot understand the stories without John also understanding them.  On the other hand, the fact that John is a subsystem of the Chinese Robot accomplishes the very opposite of the two things accomplished in the case of the Chinese Room: First, it prevents John from being able to internalize everything which is happening in the thought experiment.  Second, it motivates the intuition that the robot can understand the stories without John also understanding them.

Let us now clarify what the computationalist is and is not committed to in the case of the Chinese Robot.  According to computationalism, if John implements the correct program in the Chinese Robot, then the Chinese Robot will have a mind which understands Chinese.  On the other hand, computationalism does not seem committed to the claim that John must understand everything the robot does while he is inside it.  The two differences we just saw between the Chinese Robot and the Chinese Robot seem to support this last intuition at first glance.  In the case of the robot learning to speak and read English, whether John can do such things himself as a subsystem of the robot is seemingly irrelevant, for he can already do such things.  Furthermore, when the robot is reading stories written in Chinese we want to know whether the robot understands the stories, not John, for John is being presented with nothing more than character-salad, not meaningful stories which he fails to understand.  What is important to the computationalist is that the robot can order a hamburger when it wants one, not whether John as a subsystem of the robot can order a hamburger or can even recognize that the robot is ordering a hamburger.  Such intuitions, however, can hardly be said to constitute a compelling argument.

Not only is the Chinese Robot an entirely different thought experiment from the Chinese Room, but Searle takes it to illustrate an entirely different conclusion from that which the computationalist was willing to grant in the latter case.  Whereas the Chinese Room illustrates how linguistic symbols cannot be understood by a mind solely in virtue of their formal relations to other linguistic symbols, Searle wants to say that in the case of the Chinese Robot, “the robot has no intentional states at all.”[5]  We can thus describe the conclusion which Searle attempts to draw from the Chinese Robot thought experiment as follows:

What the Chinese Robot to Supposed to Show:

A mind which understands cannot be produced solely by the formal manipulation of symbols.

The Chinese Room draws a rather compelling conclusion about language and the conditions under which a mind can and cannot understand the meaning of linguistic symbols.  The Chinese Robot, on the other hand, is supposed to illustrate a largely unsupported conclusion about the mental and the conditions under which a mind which can understand is created by the manipulation of non-linguistic symbols.  Whereas consciousness was a peripheral issue at best in the case of the Chinese Room, it is at the very heart of the Chinese Robot thought experiment.  While both conclusions can roughly be paraphrased as “syntax is not sufficient for semantics”, such a gloss fails to do justice to the numerous and important differences which exist between the two thought experiments and their respective conclusions.  The most important difference which exists between the two is that the computationalist can happily accept the conclusion drawn from the Chinese Room but is, however, committed to rejecting the conclusion which Searle attempts to draw in the case of the Chinese Robot.  At minimum, Searle owes the computationalist a more detailed account of how the two claims are relevant to each other.

The Systems Reply and a Return to the Chinese Room

The debate surrounding the nature of the Systems reply and Searle’s counter-reply to it are laden with confusion.  What has largely gone unappreciated by the computationalist who offers the Systems reply is that their criticism is not properly aimed at the Chinese Room at all.  Searle, in turn, has been all too willing to follow the computationalist’s lead in his false assumption.  To repeat, the computationalist accepts the conclusion drawn from the Chinese Room and does not need to challenge it at all.  What the computationalist must challenge, however, is the conclusion which Searle attempts to draw from the Chinese Robot.  By assuming with the computationalist that the systems reply is aimed at the Chinese Room, Searle has been able to easily sidestep what is otherwise a legitimate reply to the Chinese Robot.

The systems reply basically amount to a denial of the conclusion which Searle attempts to draw in the case of the Chinese Robot.  The computationalist asserts that whether John understands Chinese (or anything else for that matter) is completely irrelevant to the question at hand, for computationalism does not claim that the manipulation of formal symbols creates understanding is some pre-existent mind.  Rather, the manipulation of formal symbols creates an altogether separate mind which is itself capable of understanding the linguistic meaning of Chinese characters.  Even if John does not understand Chinese within the robot, so long as he actually is implementing the right program (unlike the Schank machine), the system as a whole does have the relevant understanding.

Searle’s counter to the Systems reply comes in two parts and each part clearly assumes the computationalist to be challenging the Chinese Room rather than the Chinese Robot.  First, Searle suggests that we allow John to internalize everything which is happening in the case of the Chinese Room by memorizing all rules, databanks, etc., thereby allowing John to approximate the entire physical system.  This modification, however, is not by itself enough to undermine the Systems reply, for it is still possible that in the case of the Chinese Room “there are really two subsystems … one understands English, the other Chinese, and ‘it’s just that the two systems have little to do with each other.’”  Note carefully how this all presupposes the Chinese Room thought experiment for, as we have seen, it is impossible for John to internalize the entire system in the case of the Chinese Robot.  Indeed, we will soon see that whatever success Searle can claim from the Chinese Robot depends upon John not being able to internalize the entire system.  Let us first return to the remainder of Searle’s counter to the Systems reply, which is really quite compelling so long as one reads the Systems reply as being aimed at the Chinese Room rather than the Chinese Robot:

 “Not only do [the two subsystems] have little to do with each other, they are not even remotely alike.  The subsystem that understands English (assuming we allow ourselves to talk in this jargon of ‘subsystems’ for a moment) knows that the stories are about restaurants and eating hamburgers, he know that he is being asked questions about restaurants and that he is answering questions as best he can by making various inference from the content of the story, and so on.  But the Chinese system knows none of this.  Whereas the English subsystem know that ‘hamburgers’ refer to hamburgers, the Chinese subsystem knows only that ‘squiggle squiggle’ is followed by ‘squoggle squoggle.’ All he knows is that various formal symbols are being introduced at one end and manipulated according to rules written in English, and other symbols are going out at the other end.  The whole point of the original example was to argue that such symbol manipulation by itself couldn’t be sufficient for understanding Chinese in any literal sense because the man could write ‘squoggle squoggle’ after ‘squiggle squiggle’ without understanding anything in Chinese.”[6]

Note well the fact about the Chinese Room which Searle takes to motivate his conclusion: The English subsystem understands the meaning which is there to be had while the Chinese subsystem does not.  Searle thus concludes that since the Chinese subsystem is merely a program and cannot possibly have the relevant understanding, in order for the English subsystem to have the relevant understanding that it clearly has it must be more than a mere program.  From Searle’s perspective, the Systems reply misses the entire point of the Chinese Room thought experiment.

To the computationalist, however, Searle’s counter-reply misses the entire point of their objection.  Both sides are now speaking past one another due primarily to the failure of both sides to recognize that the Chinese Robot is an entirely different scenario from that of the Chinese Room and it is the former which is the proper target of the Systems reply, not the latter.  Of course John fails to understand the meaning which is there to be had in the case of the original Chinese Room, the computationalist has already granted this.  In the case of the Chinese Robot, however, there is no such failure on John’s part, for the Chinese characters which he is manipulating do not have any meaning for John to understand. 

Before moving on to consider Searle’s attempt at addressing both the Robot reply and the Systems reply at the same time, let us first pause to review the nature of the debate surrounding the Systems reply.  Searle takes the objection to be aimed at the Chinese Room and easily refutes it accordingly.  When one recognizes, however, that the Systems reply is properly aimed at the Chinese Robot, Searle’s counter-reply simply begs the question, his assertion that the relevant intentional states are not being created in the case of the robot is supported by little, if any argumentation.  The only argument which he seems to offer in support of this conclusion is that of the Chinese Room which does not have any clear relevance to the case of the Chinese Robot from the computationalist’s perspective.  Until Searle can offer some explanation as to why the Chinese Room supports his conclusions regarding the Chinese Robot, the Systems reply will have gone largely unchallenged.

The Combination Reply

To summarize, the computationalist claims that in the case of the Chinese Room, if John is equipped with a rulebook and databank which contains both Chinese characters as well as their referents, he would understand Chinese in the relevant sense.  To this Searle replies that John would still not understand Chinese in the case of the Chinese Robot, to which the computationalist points out that he never claimed otherwise, for he was talking about the Chinese Room, not the Chinese Robot.  However, the computationalist continues, in the case of the Chinese Robot, if John is implementing the right program (a program which would be completely different from that implemented in the Chinese Room), the entire system would understand Chinese.  To this Searle replies that the Chinese Room could not understand Chinese unless John did as well and, in the Chinese Robot, John does not understand Chinese, to which the computationalist points out that he was talking about the Chinese Robot, not the Chinese Room.  It is up to Searle to provide an argument for why the Chinese Robot cannot understand Chinese unless John does as well.  It thus seems that Searle has been playing a shell game of sorts with the computationalist.  The former refuses to acknowledge the powerful replies which the latter has offered to each thought experiment by pointing out that neither reply is powerful enough to address both thought experiments at the same time. 

The computationalist can, however, turn this shell game against Searle.  As the latter understands it, the aspect of the Chinese Room that motivates his conclusion that syntax is not sufficient for semantics is the fact that John fails to grasp the meaning which is there to be had in the stories, questions and answers.  To preserve this failure within any thought experiment two things must, therefore, be present: meaning and ignorance of this meaning.  In the case of the Chinese Room the computationalist grants that meaning is present, but sees no reason for why ignorance must remain once we allow John to connect Chinese characters to their referents.  In the case of John while he is inside the Chinese Robot, on the other hand, the computationalist grants that there is indeed ignorance, but sees no reason to believe that there is any meaning which John fails to understand.  Finally, in the case of the Chinese Robot itself, the computationalist grants that meaning is present but sees no reason to believe that there is not any understanding on the part of the robot.  Thus, it seems difficult to see how Searle can consistently maintain the necessary meaning and ignorance within a single thought experiment in order to motivate the conclusion he wishes to draw.  Until Searle can provide such an account, computationalism will not appear to be threatened by his argument.

This tension can also be expressed in terms of the subsystem dilemma.  In the case of the internalized Chinese Room, ignorance is maintained by making the room a subsystem of John: John is never able to link up the memorized Chinese characters with their referents, which he also has memories of, because the system never references these latter memories.  When the computationalist points out that this isolation of Chinese characters from their referents is not necessary, Searle formulates the Chinese Robot where ignorance is maintained by making John a subsystem of the robot: John is never able to link up the relevant Chinese characters with their referents because he is never exposed to the latter.  Thus we have Searle affirming both of the following:

                     i.     The Chinese speaking system fails to understand because it is a subsystem of John. (The Chinese Room)

                   ii.     John fails to understand because he is a subsystem of the Chinese speaking system. (The Chinese Robot)

The computationalist thus finds himself is a rather comfortable position for, unless Searle can provide an argument to the contrary, the latter cannot affirm both (i) and (ii) without contradicting himself.  If Searle affirms (i), then the computationalist comes back with the Robot reply.  If Searle then attempts to counter-reply with an appeal to (ii), he contradicts himself.  If, on the other hand, Searle affirms (ii), then the computationalist responds with the Systems reply.  In this case, if Searle attempts to counter-reply with an appeal to (i), he contradicts himself.

It is with this conundrum in mind that we turn our attention to the Combination reply which is, from Searle’s perspective, an attempt on the part of the computationalist to run both the Robot and Systems replies at the same time.  In the language that we have been using in this paper the Combination reply should be seen as an attempt to apply the Systems reply to its proper target, the Chinese Robot.  If the Chinese Room and Chinese Robot thought experiments really are in conflict with each other as I have argued we should expect any attempt on Searle’s part at running both arguments at the same time to result in confusion, contradiction and/or question begging.

Searle’s response is therefore striking:

“I entirely agree that in such a case we would find it rational and indeed irresistible to accept the hypothesis that the robot had intentionality, as long as we knew nothing more about it…

“But as soon as we knew that the behavior was the result of a formal program, and that the actual causal properties of the physical substance were irrelevant we would abandon the assumption of intentionality…

“The only real locus of intentionality is the man, and he doesn’t know any of the relevant intentional states; he doesn’t, for example, see what comes into the robot’s eyes, he doesn’t intend to move the robot’s arm, and he doesn’t understand any of the remarks made to or by the robot.  Nor, for the reasons stated earlier, does the system of which the man and robot are a part.”[7]

There are two aspects of Searle’s counter-reply which deserve our attention.  The first is that this is a clear case of begging the question.  When the computationalist suggests that a particular instantiation of their program works Searle rejects such a proposition precisely because it is an instantiation of the computationalist program.  A less cynical interpretation of Searle’s reply would be to see it as a failure on Searle’s part to recognize the limited nature of the conclusions which can be drawn from the original Chinese Room thought experiment.  It is because Searle draws such an ambiguous conclusion from the Chinese Room that he has assumed presumption on the matter and consequently asserts with confidence that John is the only locus of intentionality in the system.  From the perspective of the computationalist, however, such presumption on Searle’s part is unwarranted, for he has provided no compelling reason to think that the original Chinese Room has any relevance to the question at hand.  The most charitable reading of Searle’s response, however, is that Searle has not yet clearly articulated the connection which he sees between the point illustrated by the Chinese Room and that which the Chinese Robot is supposed to illustrate.

The second point to notice about Searle’s counter to the Combination reply is how the final paragraph in the passage quoted completely fails to appreciate the inconsistencies we have just discussed.  It is true that within the Chinese Robot John does not understand anything of relevance, but this is because there does not seem to be anything of relevance to be understood; John is manipulating character-salad, not meaningful stories, questions and answers.  While the computationalist certainly grants that John sees nothing which comes into the robot’s eyes, the robot, however, does see such things and Searle has provided no clear argument for why the robot cannot come to understand in virtue of this fact.  The only reasons which Searle provides are those “given earlier.” These “reasons given earlier,” however, were aimed at the Schank’s program and the original Chinese Room argument, and Searle has failed to show how such reasons are at all relevant to the question which the Combination reply addresses. 

Conclusion

Searle’s (in)famous Chinese Room thought experiment purports to demonstrate that machines cannot come to have minds capable of understanding solely in virtue of implementing the right program.  It is not at all clear, however, that the Chinese Room actually supports such a strong conclusion.  In order to save his conclusion from the Robot reply which the computationalist brings against the Chinese Room, Searle changes the subject at hand by invoking an entirely different thought experiment in the Chinese Robot.  The Chinese Robot and the conclusion which Searle takes it to support, however, simply beg the question from the perspective of the computationalist who immediately returns with the Systems reply.  Finally, in order to diffuse the Systems reply which the computationalist brings against the Chinese Robot, Searle again changes the subject at hand by returning to the original Chinese Room argument.  Searle has thus been able to elude the criticisms of the computationalist by trying to force them to refute two entirely different thought experiments with a single reply.  Searle is almost surely right in thinking that such a thing cannot be done, but it remains for him to demonstrate why the computationalist need do such a thing.

Let us conclude by reflecting upon the force of Searle’s arguments as they apply to the Chinese Robot.  It should be kept in mind that in this paper no efforts have been made at directly refuting the point which Searle takes the Chinese Robot to illustrate.  Rather, this paper should be read an attempt at clarifying the debate surrounding Searle’s argument as well as a request for Searle to further clarify his own claims in this debate.  Four points are in special need of clarification:  First, what, relevance does the Chinese Room, where our intuitions are so clear and compelling, have to the Chinese Robot, where our intuitions are so confused and conflicted?  Second, must John understand the stories, questions and answers put to the Chinese Robot if the latter does?  Third, does computationalism allow for the Chinese Robot to understand the stories without John also understanding them?  Finally, does John understand the relevant stories when he is implementing the right program within the Chinese Robot?

- Jeffrey Giliam

---

[1] Searle, John.  “Minds, Brains, and Programs,” The Behavioral and Brain Sciences, 3, 1980.  Reprinted in Cummins, Robert and Cummins, Denise Dellarosa eds. Minds, Brains, and Computers: The Foundations of Cognitive Science An Anthology.  2000.

[2] Ibid. 142.

[3] Ibid. 142.

[4] Ibid. 142.

[5] Ibid. 145.

[6] Ibid. 143-144.

[7] Ibid. 146-147.