https://www.lesswrong.com/posts/Gv7be4R57a38m5xrE/defining-the-normal-computer-control-problem
There has been focus on controlling super intelligent artificial intelligence, however we currently can't even control our un-agenty computers without having to resort to formatting and other large scale interventions.
Solving the normal computer control problem might help us solve the super intelligence control problem or allow us to work towards safe intelligence augmentation.
We cannot currently keep our computers doing what we want easily. They can get infected with malware, compromised or they get updates that may be buggy. If you have sufficient expertise you can go in and fix the problem or wipe the system, but this is not ideal.
We do not have control our computers, without resorting to out of band manipulation.
n this language the normal computer control problem can be defined as.
What type of automated system can we implement to stop a normal general purpose computer system misbehaving (and carry on with its good behaviour ) if it has a malign program in it.
To make the differences explicit:
The normal control problem assumes a system with multiple programs some good, some bad
The normal control problem assumes that there is no specific agency in the programs (especially not super-intelligent agency)
The normal control problem allows minor misbehaviour, but that it should not persist over time
These make the problem more amenable to study. These sorts of systems can be seen in animals. They will stop pursuing behaviours that are malign to themselves. If a horse unknowingly walks into an electric fence whilst it was trying to get to an apple, they will stop trying to walk in that direction. This is operant conditioning, but it has not been applied to a whole computer system with arbitrary programs in.
Imagine being able to remove malware from a normal computer system by training that system. That is what I am looking to produce.
Comments
Post a Comment