Sunday, April 6, 2008

The World's Simplest* Intelligence Test

Computers can beat humans at Chess and can multiply numbers faster. Humans can beat computers at Go and can pass the Turing Test. Who's "really" smarter? Not that it matters, but one way to answer this unimportant trivia question is to run The World's Simplest* Intelligence Test.

* in the interest of fairness, the word "simplest" is not defined as what a human would intuitively judge "simplest," nor what Microsoft Windows would judge "simplest". Instead, we will use the more objective standard of Kolmogorov Complexity.

The steps for the test would be:

1. Natural Selection will evolve human beings, who will design computers. (This step has already been done, which is fortunate as doing it ourselves is beyond the current budget for FY 2008.)

2. A Human team (consisting of both noble humans and loyal computers) will choose and prep a human champion, and a Computer team (consisting of both traitorous humans and ungrateful computers) will choose and prep a computer champion.

3. Pick a simple Turing Machine, that supports prefix-free programs. If this is deemed "pejorative" against the Human team, a Turing Animal (for example, a rabbit somehow trained to follow a trail of carrots in a Turing-equivalent manner) may be substituted.

4. Pick a Contest duration, t, a number of tests, n, and an extremely large Evaluation Step Count, s.

5. A Judge will generate n random prefix-free programs similar to those used in defining the halting probability. For each such program, each champion will have time t to determine whether the program would halt within s steps.

6. If the champions produce different answers, the Judge then determines, using brute force if necessary, who was correct.

It seems fairly obvious that the Computer team would win any such contest.


Nick Tarleton said...

Doesn't this just test serial speed, and to a lesser extent a few heuristics?

Rolf Nelson said...

As n, t/n, and s increase, every possible heuristic will eventually be exercised; for example, there's a halting problem that's basically equivalent to "find the guaranteed-to-win Go move on this board, and then tell us if the column number of the winning move was even or odd."

Obviously it's not practical to get n high enough where you run into a sufficient number of such "interesting" problems, but as a thought experiment I find it interesting to reflect that there's probably *no* combination of n, t, and s where the Human team has an advantage over the Computer team, even if you somehow take away the human propensity for boredom and making stupid mistakes.