Can a Mechanical Chess-Player Outplay its Designer?
Date Published
W. Ross Ashby1
Published in the British Journal for the Philosophy of Science, 3 (1952), 44–57. Reprinted in Mechanisms of Intelligence: Ashby's Writings on Cybernetics, ed. R. C. Conant (Intersystems, 1981), pp. 281–294. Scan of the reprint on ashby.info.
1 Introduction¶
Mr Adkins' letter1 is welcome, for it insists on the discussion of a question often heard in conversation but not yet, so far as I am aware, adequately treated in print : to what extent is the machine restricted by the limitations of its designer ? The best form of the question is uncertain, but we can usually recognise it in its different forms. Thus, Descartes declared that there must be at least as much reality and perfection in the cause as in the effect. Kant2 asked, 'How can work full of design build itself up without a design and without a builder?' Another form has been used as title for this paper.
The question is not only of philosophic interest but is fast approaching practical importance. Some of us are studying the physiology of the brain by building models that will mimic its characteristic properties. Our working hypothesis is that we can eventually build a 'real' brain. But for it to be a 'real' brain it must produce cleverness of its own and not just give us back the ingenuity we have put into it. Here the truth or falsity of Descartes' dictum is crucial. If it is true, we are wasting our time; only if it is false can we succeed. So we must try to disprove it. If we can prove it false, not only may the way be shown open to a 'real' brain but, by scrutinising the proof, we may be able to get a hint about how to proceed, for the proof may show what the real difficulty is, and how it can be overcome.
2 Measuring 'Design'¶
Examining Descartes' dictum, we notice first that it is essentially quantitative. It says, in effect : if the quantity of design supplied by the designer is D, and if the quantity of design shown by the machine is M, then necessarily
Descartes, however, gave no hint as to how these quantities were to be measured, so we must develop a method. Fortunately we need not enquire exhaustively into all that is implied by 'design,' for the measurement of a quantity can properly precede the understanding of what it is that is being measured—engineers were measuring the 'electric fluid' and were lighting towns with it several decades before its real nature was understood.
How are we to obtain an objective and consistent measure of the 'amount of design' put into, or shown by, a machine? Abstractly, 'designing' a machine means giving selected numerical values to the available parameters. How long shall the lever be? where shall its fulcrum be placed? how many teeth shall the cog have? what value shall be given to the electrical resistance? what composition shall the alloy have? and so on. Clearly, the amount of design must be related in some way to the number of decisions made and also to the fineness of the discrimination made in the selection. I suggest that the measure appropriate to our purpose is one already developed in Shannon's theory of information.3 Though this theory was developed by communication engineers for technical purposes, it will, I believe, be found to have applications, and to be of philosophic importance, over a much wider range. Its use enables us to treat the question quantitatively, and it provides the discussion with a secure basis of experimental and practical experience.
Shannon's measure is defined fundamentally as follows. Suppose certain events E₁, E₂, . . . , Eₙ have probabilities p₁, p₂, . . . , pₙ, respectively, of occurring, with2
then the occurrence of the actual event is associated with the quantity
(If the probabilities are all equal, and each equal to 1/n, then the quantity becomes simply log n.) .As3 an example, suppose that we are going to draw and inspect one card from a shuffled pack and that we are interested only in the distinction between the three events
E₁ : the drawing of the King of Clubs,
E₂ : the drawing of any Spade,
E₃ : the drawing of any other card;
then p₁ = 1/52, p₂ = 1/4, p₃ = 19/26; and the quantity associated with the drawing of the card, and the discovery of which event has actually occurred, is
which, if the logarithms are to the base 2 (see below), has the value 0·94. This number measures the amount of information, relative to these E's, that is to be obtained by drawing a card.
The number 0·94, it should be noticed, does not belong to any particular card, for it can be calculated before the drawing takes place. It does not answer the question 'I have drawn the Three of Hearts: how much information have I received?'; rather it answers the question 'when I draw and look at the card, how much uncertainty will be removed?' It measures, primarily, the uncertainty existing before the drawing is made. This uncertainty will be dispelled by the actual drawing and inspection of the card, and may be used as a measure of the information to be expected in the drawn card if we regard 'information' as 'that which dispels uncertainty.' At first impression this way of measuring information may seem unnatural, almost the reverse of what one would expect; but its usefulness in communication engineering has put its practical value and the soundness of its method beyond doubt.
The amount can be measured in various units, which depend on the base used for the logarithms. The most convenient here is the 'bit'4—a contraction of BInary digiT—which is the amount of information given, or uncertainty removed, when a decision is made between two equally probable alternatives: a decision made by the spin of a coin, for instance, or that made after asking 'to be or not to be?' To work in this unit, the logarithms are taken to the base 2. So the drawing of a card in the example above gives a little less information than is obtained by seeing whether a penny has fallen heads or tails.
It is an important property of Shannon's function (2) that if two or more events become indistinguishable, so that some of the E's and p's have to be combined, then the numerical value of the function always diminishes. Thus suppose, in the example above, that we no longer distinguish between E₁ and E₂. p₁ and p₂ then become the single probability 7/26, i.e. 1/52 + 1/4, and the quantity becomes
which is 0·84, i.e. 0·10 less than the previous 0·94. Conversely, if we can make a distinction where previously none was made, then the quantity will be increased.
Any quantity calculated by the formula (2) will be referred to throughout this paper as the 'amount of information,' but I use the phrase purely for convenience, and wish to imply only what is either contained in the definition or deducible from it. I shall attempt to show that this quantity, in this context, is equivalent, in its properties, to the amount of design. This is not to show that information and design are intrinsically the same, but to show that as the object of our study is changed in ways that unquestionably alter the amount of design in it, so does the quantity given by Shannon's function always change similarly. The latter can then conveniently be used as an indicator for the former, just as the height of the mercury in a thermometer-tube can be used as an indicator of the hotness of its surroundings, though 'height' and 'hotness' are by no means identical.
To apply the measure to a designed machine, we regard the machine as something specified by a designer and produced, as output, from a workshop. We must therefore consider not only the particular machine but the ensemble of machines from which the final model has been selected. Let me give some examples. They will illustrate the method in detail and will show that the measure agrees satisfactorily with our intuitive sense of 'amount of design.'
First suppose an electrical engineer is going to design a network of resistances. Suppose he can take for granted that there will be three resistances radiating from a point, that their values will lie somewhere between 10 and 100 ohms, and that he need not specify closer than 20 per cent. All other variations have zero probability. What he has to do is to allot to each resistance one of the values 10, 15, 22, 33, 47, 67, or 100 ohms. If the a priori probabilities are all equal to 1/7, the amount of information is, for each resistance, log₂ 7 bits, and, for the whole network, 3 log₂ 7, i.e. 8·42 bits. This number gives an exact value to the symbol D in (1).
Had a finer discrimination been necessary, the amount of information would have been larger. Thus, had each resistance to be specified to the nearest ohm, each resistance would have to be selected in value from 10, 11, 12, . . . , 99, 100 ohms, i.e. from 91 possibilities. So the amount of information would be 3 log₂ 91, i.e. 19·52 bits. The extra 11·10 bits (the increase from 8·42 to 19·52) represents, and measures, the extra discrimination that has been used; for it is an important property of Shannon's function that if selections are made in stages, the components combine linearly.
We can use the same function to measure the amount of information (or design) shown by the network (M of (1)). Here again we consider the possible networks, as they differ in their particular values for the resistances; we associate a probability with each distinct network, and we calculate the amount of information in the usual way. It proves to be 8·42 bits; so in this example M equals D.
This equality does not occur necessarily, and in other networks the two quantities may be unequal. Suppose, for instance, that another network, instead of being arranged as three radii, had been arranged as three resistances in parallel. Two variations that differed only by a permutation of their resistances would not be distinguishable electrically; so the designer could make variations that did not make any effective difference to the network. In such a case, M would be less than D.
On the other hand, no possible rearrangement can make M greater than D, so this first example exemplifies Descartes' dictum.
As a second example, consider the case of the designer who, to provide himself with materials, has bought a toy engineering set that contains a definite number of parts capable of being joined in a certain variety of ways. If he constructs his machine from these parts we could, with some labour, calculate the exact amount of information in his final model. Again, the variety of machines constructible cannot exceed the variety available to the designer; and it may be less if some of the forms are indistinguishable. So this second example also exemplifies the dictum. In fact, if this way of measuring 'amount of information' (or design) be accepted, the whole of information theory and all the practical experience of communication engineering becomes available to support the thesis that, in cases such as these, no possible rearrangement of parts can make the amount of information in the machine greater than that used by the designer.
3 Dynamic Systems¶
Exactly the same conclusion is reached if we ignore the form or construction of the machine and consider only its behaviour.
Every determinate machine can be specified in its behaviour by a set of ordinary simultaneous differential equations of the first order
where the f's are single valued, and where the right-hand side contains no function of t (the time) other than those whose fluxions occur on the left.4 (For a machine is 'determinate' in its behaviour if the occurrence of a particular state determines what it will do. The equations say this in mathematical form, for if the f's are given, by being appropriate to some particular machine, the occurrence of a particular set of values of the x's is sufficient to determine the changes, the dx's, that will occur during the next instant dt.) If the machine, with variables x₁, . . . , xₙ, is brought, at time t = t₀, to an arbitrary state x₁⁰, . . . , xₙ⁰ and then released, the equations, through their integrals, determine how each x will change in value with time.
When a machine is 'designed,' the designer starts with more possibilities available :
where the α's are parameters under his arbitrary control. He then selects the numerical values that they are to have, fixes them permanently, and so restricts the forms φ, regarded as functions of the x's, to some particular form, such as (3).
It is well known that the equations (3) define, from each point (x₁, . . . , xₙ), a single trajectory. It follows that if the system (4) has k possible combinations of the α's, the machine cannot show more than k trajectories from any point, though it may show fewer. Measured in this way, the amount of information in its trajectories, or behaviours, cannot exceed the amount primarily introduced by the designer when he decides the values to be given to the α's.
The examples given so far are thus wholly in accord with Descartes' dictum.
4 Evolution and Design¶
The question might seem settled, were it not for the fact, known to every biologist, that Descartes' dictum was proved false over ninety years ago by Darwin. He showed that quite a simple rule, acting over a great length of time, could produce design and adaptation far more complex than the rule that had generated it. The status of his proof was uncertain for some time, but the work of the last thirty years, especially that of the geneticists, has shown beyond all reasonable doubt the sufficiency of natural selection. We face therefore something of a paradox.
There can be no escape by denying the great complexity of living organisms. Neither Descartes nor Kant would have attempted this, for they appealed to just this richness of design as evidence for their arguments. Information theory, too, confirms the richness. Thus, suppose we try to measure the amount of design involved in the construction of a bird that can fly a hundred miles without resting. As a machine, it must have a very large number of parameters adjusted. How many cannot be stated accurately, but it is of the same order as the number of all the facts of avian anatomy, histology, and biochemistry. Unquestionably, therefore, evolution by natural selection produces great richness of design.
Whence comes the richness? Can it be that the rule of natural selection—'the dead shall not breed'—together with its ancillary rules, is really more complex than it seems, and is at least as complex as the design it produces? This seems most unlikely, but even if it were true the paradox would not really be resolved, for the complexity that can be generated by evolution is independent of the complexity in these rules. To make the argument clear and quantitative, let me give an example. The essence of the evolutionary process is selection, a single selective operator acting over and over again. Consider, then, a very simple selective machine: a magnet that selects any iron ball from a stream of balls that rolls beneath it. Let us compare the quantity of design that went to its construction with the quantity of selection that it can make, using Shannon's measure for both. When it was designed, the designer had to select its components from those that were available to him: cogs, magnets, wires, batteries, etc.—what was available in his workshop and what could be bought with his resources. If there were A components, with equal a priori probabilities of being selected, the selection of a magnet involved log₂ A bits. (Theoretically, the accuracy of this measurement is limited only by our ability to give accurate definition to 'workshop,' etc.) The decision to allow balls to roll underneath involves, say, log₂ B bits; we need not enquire its exact value here. So the total amount of information put into the machine's design is log₂ A + log₂ B bits. Now consider what the machine can achieve as a selector: if there is one iron ball in C the machine will achieve a selection of log₂ C bits. It follows that log₂ C may well be greater than log₂ A + log₂ B, for A, B and C have no necessary relationship. Similarly, however complex the specification of 'natural selection,' evolution can in time produce a greater complexity.
Information theory, however, makes clear whence comes the extra information. The law that information cannot be created is not violated by evolution, for the evolving system receives an endless stream of information in the form of mutations. Whatever their origin, whether in cosmic rays or thermal noise, the fact that each gene may, during each second, change unpredictably to some other form makes each gene a typical information source. The information received each second by the whole gene-pattern, or by the species, is then simply the sum of the separate contributions. The evolving system has thus two sources of information, that implied in the specification of the rules of natural selection and that implied by the inpouring stream of mutations.
It is now clear that the paradox arose simply because the words 'cause' or 'designer,' in relation to a system, can be used in two senses. If they are used comprehensively, to mean 'everything that contributes to the determination of the system,' then Shannon and Descartes can agree that 'a noiseless transducer or determinate machine can emit only such information as is supplied to it.' This formulation will include the process of evolution if the 'cause' is understood to include not only the rules of natural selection but also the mutations, specified in every detail. If, on the other hand, by 'cause' or 'designer' we mean something more restricted—a human designer, say—so that the designer is only a part of the total determination, then the dictum is no longer true.
This gives us what we need. The evolutionary process transcends, in a sense, the bounds set by the dictum. What we have to do now is to develop a machine that shall, in some way, use an evolution-like process in its working.
5 Darwinian Machinery5¶
To develop the machine, first observe that we now have two ways of giving the instructions necessary for the production of, say, a bird. One way is to specify the details in all their complexity and individuality, filling perhaps several volumes in the process. The other way is just to write down the instructions: 'Take a planet with some carbon and oxygen; irradiate it with sunshine and cosmic rays; and leave it alone for a few hundred million years.' We may feel that this second way is somewhat irregular, but we cannot deny that it is effective.
To see what is involved, let us consider an even simpler example. Suppose we offer a prize to any inventor who, by using only 1000 bits6 in the specification of his machine, can build one that will itself produce more than 1000 bits. Suppose that one inventor proceeds as follows. He makes a little machine, using 100 bits; he makes a Geiger counter, using 500 bits; and he then uses 1 bit to decide that the counter shall, rather than shall not, feed into the machine. He then comes to us and claims that he has built a machine that (a) was designed, so far as he was concerned, with 601 bits, and that (b) can emit far more than this quantity. Can we deny his claim?
He has shown us how a human being, limited by his finite intelligence so that he cannot contribute more than 1000 bits to the design, can none the less produce a machine that will emit more than this quantity. The point is that the decision to use, or not to use, a source of information—or to select one source from several—requires only a bit or two, while the source itself then contributes a quantity which may be much greater.
He has, in fact, devised an 'information-amplifier.' For what is an 'amplifier'? A power-amplifier is a device that, if given a small amount of power, will emit a larger amount. A sound-amplifier, subjected to a small sound, will emit a larger sound. A 'money-amplifier' would obviously be a device that, if given a small amount of money, would emit a larger amount of money. In all cases the output is of the same quality as the input, but the input is not used to provide part of the output: it is consumed in controlling the flow of 'material' from a copious source, or large reservoir, of the same 'material'; and it is the reservoir that provides the output. Exactly the same method was used by the inventor: his 601 bits were used, not to provide part of the output but to control the flow of fresh information from a much larger source.
At this point the critic may well object that such information, being unorganised and chaotic, is useless. To this I would reply that chaotic information is by no means useless, but is, in fact, perfectly usable (as evolution has shown by its use of mutations) provided that the machine has been designed to make the necessary selection.
The main lines of the machine that is to go past the limit set by Descartes' dictum is now clear. The designer should make it selective, like the 'magnet-machine' considered earlier, and he should couple it to an abundant source of information such as a Geiger counter or a device using thermal noise. The amount of design needed to specify the machine's construction, and the amount of information that it can emit as output, will then be dissociated, and the amount that can be emitted is no longer limited to being equal to, or less than, the amount used in the design. If P bits went to its construction, and if it selects something as rare as 1 in Q, then if Q is greater than 2ᴾ the designer can claim that the machine is showing more design than he has put into it.
6 The Homeostat¶
The argument may be made clearer by now turning to the homeostat, which was built precisely to admit and to use unorganised information.5
First, what is the homeostat? Here we need its abstract functional, rather than its practical, details.
In a sense it is a machine within a machine, so its description is best taken in two stages. The primary machine consists of four Units, all of which act on each other, and each of which bears on its top a needle whose deviation from the central position is the focus of interest. The four deviations (x₁, x₂, x₃, x₄) depend on sixteen parameters aᵢⱼ in accordance with the equations (of the same form as (3) above)
Its only resting state is at the origin, where it can be in either stable or unstable equilibrium.
If the latent roots of the matrix [aᵢⱼ] have real parts that are all negative, the system will be stable. Its needles will then move to the central position and, if displaced, the Units will so interact that a co-ordinated movement brings the needles back to the centre. If one or more of the real parts is positive, the system will be unstable, 'vicious circles' of action will develop, and the needles will move away from the centre. To this extent the machine is simply an analogue computor, for if the values of the parameters aᵢⱼ are given, its behaviour provides integrals of the equations (5).
It has, however, further components which influence this first activity. If any one of the x's goes outside certain bounds (a deviation of more than about ± 45°) the extra components act and change the a's in the corresponding row to a new set of values that are taken at random from a distribution that lies evenly between − 9 and + 9. The actual values came originally from a table of random numbers.6 So long as the needle stays outside the bounds, so long do the a's keep changing, at about one-second intervals. As soon as the needle returns to within the bounds, the a's stop changing and retain their latest set of values.
When set working it behaves in the following way. If stable, the needles come to the centre; it will then demonstrate, if disturbed, the co-ordinated response typical of a stable system. If, however, it is unstable, the needles diverge, and the a's undergo substitution after substitution. The first stable combination of a's to arrive is retained. Whether a particular combination will be stable depends partly on what other conditions (the 'problem') have been imposed on the mechanical and other parts of the system (for commutators can reverse the normal polarities of connection, a needle or two can be locked, needles can be tied together, and so on, making the equations (5) more complicated than is shown). Its behaviour can therefore be described by saying that it hunts for, and retains, a matrix that is stable in the imposed conditions. If the conditions are changed, so that the matrix is no longer stable in these new conditions, it will at once hunt for, and eventually retain, a new matrix that gives stability in the new conditions.
Its process is clearly similar to that occurring in evolution. There the rules are: test the organism against the environment; if the organism is unfit remove it; replace it by a new organism that differs from it in some random way. In the homeostat the rules are: test the matrix for stability in the imposed conditions; if it is unstable remove it; replace it by a new matrix with random elements. In both, the new material varies merely randomly from the old.
The homeostat was built before Shannon's theory of information had been published. It is instructive now to apply the theory to the homeostat's design. Let us consider the original model. There are four uniselectors (stepping-switches) and each can take, independently, one of twenty-five positions. The amount of information in its output is thus 4 log₂ 25, i.e. 18·6 bits. The amount that went to its design can be very roughly estimated at 200–800 bits. It seems, therefore, that though the homeostat, in principle, can claim to produce more design than was put into it by the designer, the original model cannot so claim—its reservoir of new information is too quickly exhausted. Its amplification-factor is almost ludicrously below unity.
This deduction is somewhat sobering, but from it come two reflections. We can see how powerful is Shannon's theory in its quantitative grasp of these questions, and we can also notice, what the evolutionary process itself suggests, that a very great deal of selection may be necessary if Descartes' limit is actually to be passed.
7 Conclusion¶
It seems now to be clear that while Descartes' dictum is true if by 'designer' we mean 'every detail that contributes to the machine's performance,' it is not true if by 'designer' we mean 'the man who specifies its construction.' To defeat the limit implied by the dictum, the designer must include, as one of his specifications, 'admit other information.' Stated thus the method may seem to be mere trickery, on a par with that of the boy who, given three wishes, made one of them a request for three more wishes. Trickery or not, however, it offers the designer a practical method for overcoming the limitations of his own powers of design.
Once the method has been identified, we can see that it is not unreasonable. We can see, for instance, that the more minutely we design a machine to play chess just as we want it to—admitting no other information—the more certain it is to play just our sort of chess, with all our faults and wrongly-conceived strategies. We are, in fact, in exactly the same position as the father, a keen but mediocre chess-player, who wants his son to become world champion. It is true that the father should teach the child, but he must not teach his son every reply in detail lest he limit the son's play to being the merest replica of the father's. If the father really wants the son to beat him he must sooner or later stop telling the son what to do and must send him out into the world to be subjected to all sorts of un-selected experiences. The understanding father will not try to teach his son all chess but will try to teach him how to profit by future experiences.
Designing a machine has much in common with teaching a child, for in each case the almost infinite possibilities have to be reduced to a selection. Were Descartes' dictum to be stated in the form 'no child can know more than he has been taught' we would at once see the equivocation, for the teacher can not only teach the child facts but can also teach him how to use the 'free' information in the world, and thus how to surpass his teacher. If we wish to build a machine that can beat us at chess, or to build a 'real' brain, we must follow the same method; we must aim in design, not at a machine that will play chess, but at a machine that can make trials, and select. The homeostat was intended to be a first step in this direction.
8 Summary¶
The quantitative aspects of the question 'how much of a machine's behaviour is attributable to the designer?' can be treated accurately and meaningfully by the methods of information theory. Its application leads to the deductions:
It is quite possible for a mechanical chess-player to outplay the man who designed it.
For this to be possible, the designer must construct the machine so that it can receive and use information not provided by him in detail.
The extra information need not be organised or intelligible—that given by thermal noise may be sufficient.
(An examination of the homeostat illustrates the method in detail.)
Department of Research
Barnwood House
Gloucester
Bibliography¶
- This Journal, 1951, 2, 248
- General History of Nature, 1755
- C. E. Shannon, 'A mathematical theory of communication,' Bell System tech. J., 1948, 27, 379–423, 623–656; Norbert Wiener, Extrapolation, Interpolation and Smoothing of Stationary Time Series, New York, 1949
- W. Ross Ashby, 'The physical basis of adaptation by trial and error,' J. gen. Psychol., 1945, 32, 13–25 (so in the print; the paper's title is 'The physical origin of adaptation by trial and error'); 'The nervous system as physical machine,' Mind, 1947, 56, 1–16; L. von Bertalanffy, 'An outline of general system theory,' this Journal, 1950, 1, 134–165
- W. Ross Ashby, in Electronic Engineering, 1948, 20, 379–383. (A full account of the machine and of its principles in relation to the living brain, especially with regard to the origin of adaptive behaviour, is given in my Design for a Brain, to be published shortly by Chapman and Hall, London.)
- R. A. Fisher and F. Yates, Statistical Tables, Edinburgh, 1938
Footnotes
- ↩
Received 11. xii. 51
- ↩
sic — the print sets a comma after p₂ in the sum (p. 282); preserved.
- ↩
sic — a stray point before "As" in the print (p. 282); preserved.
- ↩
The word is inelegant, but it is already firmly established in use.
- ↩
The author is indebted to Professor Wiener for this apt adjective.
- ↩
Units of information, not parts.