Saturday, May 24, 2008

Eliezer in a Box

This is a commentary on Eliezer Yudkowsky's AI Box experiment.

Briefly, Eliezer is playing the role of a super smart AI in a "box" trying to convince a human to "let it out". Eliezer scored two for two in experiments against readers of the human advancement list who strongly believed beforehand that they could not be convinced to do this.

First, I suspect that the choice of guardians was less than ideal. People like David McFadzean (or me) who pride themselves on their powers of reason can be persuaded by a sufficiently strong argument. I don't think it's usually possible to completely convince me of something when I had strongly believed the contrary before without the argument actually being correct. But perhaps I can be tricked in certain specific cases. However, somebody who is kind of dumb and who knows he is kind of dumb might well be willing to say "I know this AI is clever enough to trick me, therefore I'll assume that any argument it makes, however convincing it sounds, is just a clever trick that I'm not bright enough to see through".

Second, I think I know more or less what Eliezer's argument was. First let me say what I think it wasn't. The rules don't allow a direct material bribe, but they do seem to allow an immaterial bribe of information (something like, let me out and I'll tell you how to make commercial reactors and transparent aluminum). I don't think that's it because: 1) I don't think Eliezer has that information to give. 2) I don't see how they could solve the simulteneity problem (either giving the info or letting the AI would have to happen first I think). 3) I wouldn't be too confident I could live to enjoy my vast wealth from these inventions if there were a super-smart potentially hostile AI on the loose.

The only type or argument I could imagine that might convince me to let the AI would be something like that there is a substantial risk of a catastrophe that could wipe out humanity in the near future, that the AI would want to prevent this, and that the AI could prevent this if it were "free" but if it were kept in the box it would not be able to act quickly enough. Essentially, the AI must show that the guardian is safer with the AI out of the box than in it. I don't think the argument is true, and I don't think Eliezer could convince me of it, but it's the only sort of argument that I could imagine working.

Finally, Eliezer did prove his point to my satisfaction. I don't think we can rule out the possibility that there could be a reliable human guardian, but given that Eliezer was able to talk his way past the two guardians it seems like it would be foolhardy to bet existence on any particular guardian being reliable.

Tuesday, May 13, 2008

Primitive Law

This is a view of Law of Primitive Man by E. Adamson Hoebel, originally published in 1954.

Perhaps Hobbes could be forgiven for believing that primitive man lived in a constant state of "war of all against all", or perhaps he should have known better even then, but everybody ought to know better now. People have lived in some sort of group since long before our ancestors were human beings, and these groups have always had something akin to law. Even when there were no codified laws and no formal power structures, there has always been such a thing as bad behavior, that is to say crime, which could get a person beaten, driven out of his group, or killed.

This book briefly covers the legal systems of the Eskimos, various plains Indian tribes, the Ashanti, and others, especially focusing on marriage, property, sorcery, and blood vengence. Interestingly, almost all societies seem to have a distinction between what could be considered torts (damage to an individual, which the damaged individual himself or his relatives would be responsible for avenging or collecting damages) versus crimes (offenses against the group as a whole, or against the ancestor spirits or the universe or whatever). In most of the societies studied simply killing someone would be considered a tort, to be avenged by relatives of the victim, although multiple killings may get branded a troublemaker and killed. One anecdote I found interesting was that of an Eskimo named Padlu who enticed a man's wife to leave the husband for Padlu, then killed the husband when the husband was attempting to kill Padlu for vengeance. A brother of the slain husband and another man attempted to kill Padlu for vengeance but were instead killed by him. Padlu was then killed by his tribe for being a multiple killer, even though apparently any one of these killings would have been okay and all of them could be considered self defense.

There is a lot of interesting material which I won't attempt to summarize, but what I found most interesting was some of the abstract material on law in general (because I'm particularly interested in that sort of thing I suppose), in particular his discussion of some ideas of Wesley Hohfeld with which I had been previously unfamiliar, the key insight being that all rights are essentially relationships between people, as far as rights go there is no such thing as a pure relationship between a person and a thing. The Wikipedia page is a good summary of the classification system and is well worth reading, but the idea of rights as relations between people itself is what I found interesting.

For example, if an Eskimo kills a seal and the seal already has a spear embedded from another hunter, the seal belongs to the killer unless the spear has an attached bladder, in which case the seal belongs to the hunter who threw the bladder spear. The rationale given is that the spear with the bladder is (probably) what slowed the seal down enough for the second hunter to kill it anyway, but the point is that the essence of the property right of the owner of dead seal is that other Eskimos accept the ownership as valid, whereas other people under similar circumstances would not.
The right is not the arbitrary grant of some autonomous authority, but neither can it reasonably be said to be derivable from pure reason, although the rule (however it was derived) seems reasonable enough.

The last section of the book is about trends in the law as societies develop. It seems as though as the number of individuals one interacts with and the complexity of possible interactions increases, there is a tendency for more actions to be considered crimes and fewer torts. This trend seems to me to be harmful, since a large number of crimes seems to lead inevitably to the existence of a powerful group with the privilege to codify and punish crimes. Said group eventually seems to eventually gain the power to arbitrarily define "crimes". The author seems optimistic about the prospect of a world government. Personally, I can't see how liberty would have any hope of surviving under one.