The first AI textbook, on the control problem

The earliest introductory AI textbook I know about — excluding mere “paper collections” like Computers and Thought (1963) — is Jackson’s Introduction to Artificial Intelligence (1974).

It discusses AGI and the control problem starting on page 394:

If [AI] research is unsuccessful at producing a general artificial intelligence, over a period of more than a hundred years, then its failure may raise some serious doubt among many scientists as to the finite describability of man and his universe. However, the evidence presented in this book makes it seem likely that artificial intelligence research will be successful, that a technology will be developed which is capable of producing machines that can demonstrate most, if not all, of the mental abilities of human beings. Let us therefore assume that this will happen, and imagine two worlds that might result.

[First,] …It is not difficult to envision actualities in which an artificial intelligence would exert control over human beings, yet be out of their control.

Given that intelligent machines are to be used, the question of their control and noncontrol must be answered. If a machine is programmed to seek certain ends, how are we to insure that the means it chooses to employ are agreeable to people? A preliminary solution to the problem is given by the fact that we can specify state-space problems to require that their solution paths shall not pass through certain states (see Chapter 3). However, the task of giving machines more sophisticated value systems, and especially of making them ‘ethical,’ has not yet been investigated by AI researchers…

The question of control should be coupled with the ‘lack of understanding’ question; that is, the possibility exists that intelligent machines might be too complicated for us to understand in situations that require real-time analyses (see the discussion of evolutionary programs in Chapter 8). We could conceivably always demand that a machine give a complete output of its reasoning on a problem; nevertheless that reasoning might not be effectively understandable to us if the problem itself were to determine a time limit for producing a solution. In such a case, if we were to act rationally, we might have to follow the machine’s advice without understanding its ‘motives’…

It has been suggested that an intelligent machine might arise accidentally, without our knowledge, through some fortuitous interconnection of smaller machines (see Heinlein, 1966). If the smaller machines each helped to control some aspect of our economy or defense, the accidental intelligent might well act as a dictator… It seems highly unlikely that this will happen, especially if we devote sufficient time to studying the non-accidental systems we implement.

A more significant danger is that artificial intelligence might be used to further the interests of human dictators. A limited supply of intelligent machines in the hands of a human dictator might greatly increase his power over other human beings, perhaps to the extent of giving him complete censorship and supervision of the public…

Let us now paint another, more positive picture of the world that might result from artificial intelligence research… It is a world in which man and his machines have reached a state of symbiosis…

The benefits humanity might gain from achieving such a symbiosis are enormous. As mentioned [earlier], it may be possible for artificial intelligence to greatly reduce the amount of human labor necessary to operate the economy of the world… Computers and AI research may play an important part in helping to overcome the food, population, housing, and other crises that currently grip the earth… Artificial intelligence may eventually be used to… partially automated the development of science itself… Perhaps artificial intelligence will someday be used in automatic teachers… and perhaps mechanical translators will someday be developed which will fluently translate human languages. And (very perhaps) the day may eventually come when the ‘household robot’ and the ‘robot chauffeur’ will be a reality…

In some ways it is reassuring that the progress in artificial intelligence research is proceeding at a relatively slow but regular pace. It should be at least a decade before any of these possibilities becomes an actuality, which will give us some time to consider in more detail the issues involved.

Leave a Reply

Your email address will not be published. Required fields are marked *