Wire Disagreement Dilemma

You are locked in a room with two other people and a time bomb. To disarm the bomb, you must choose correctly between cutting the red wire or the blue wire on the bomb; cutting the wrong wire, or failing to cut either of the wires in time, will trigger the bomb. Any one of the three of you can choose to lunge forward and cut one of the wires at any time.

Each of you puzzles over the circuit-wiring schematic. You find an airtight, 100% certain proof that the red wire is the wire that needs to be cut. But simultaneously, your two allies report that they have come up with airtight, 100% certain proofs the blue wire needs to be cut! You cannot come to a consensus, either because you do not have time, or because you simply cannot understand each others' proofs.

Your choices are:

1. Lunge forward and cut the red wire.

2. Allow your allies to cut the blue wire.

How do you make your decision? Call this the Wire Disagreement Dilemma.

Notes:

1. According to the most straightforward application of classical logic, you should lunge forward and cut the red wire.

2. Philosophical Majoritarianism doesn't tell you exactly what to do. PM seems to be a heuristic that you use alongside other, sometimes conflicting, heuristics. As I've seen it outlined, it doesn't seem to tell you much about when the heuristic should be used and when it shouldn't.

3. There's a sense in which you never have an actual proof when you make a decision, you only have a memory that you had a proof.

4. Consider two people, Alice and Bob. Alice should not automatically give her own beliefs "magical precedence" over Bob's beliefs. However, there are many circumstances where Alice should give her own beliefs precedence over Bob's; there are also circumstances where Alice should defer to Bob.

5. This type of thinking is so rare, that (to my knowledge) we don't even have a short word to describe the difference between "I believe X because I reasoned it out myself" and "I believe X because someone smarter or more experienced than me told me X, even though, on my own, I would have believed Y."

In normal conversation, you have to use cumbersome phrases and idioms: for example, "it seems to me like X" in the former case and "my understanding is that X" in the latter case.

Experience vs. Hearing: As technical terms, I'd propose that in the former case we say "I Experience X" or "my Experience is X." In the latter case we can say "I Hear that X" or "my Hearing is X."

6. One asymmetry, when Alice is evaluating reality, is that she generally knows her own beliefs but doesn't necessarily know Bob's beliefs. Bob may be unavailable; Bob may be unable to correctly articulate his beliefs; Alice may misunderstand Bob's beliefs; there may not be time to ask Bob his beliefs; or Bob may deliberately deceive Alice about his beliefs.

Occam's Meta-Razor

Let me define the Occam's Meta-Razor Problem as follows: What is the smallest and simplest set of basic philosophical postulates that a rational agent needs in order to act in a way that is intuitively satisfactory? The goal is that the behavior should satisfice, even if it's not necessarily optimal. Call this the Occam's Meta-Razor problem.

Intuitively, I think we want three items:

1. A simple way to analyze probabilities. Something like Solomonoff Induction might satisfice, if the Pascal's Mugging problem were solved.

2. A utility function. An initial start might be, “Maximize the expected amount of X in the Universe,” where X is some weighted combination of happiness, freedom from pain, autonomy, etc. A satisfactory but simple description for X would be difficult to unambiguously specify, especially in the case where the agent wields super-human intelligence. Two of many possible pitfalls:

• For almost all X, the current set of humans who are alive (and humanity in general) are going to be sub-optimal, from the point-of-view of the agent. However, we want the agent to decide against wiping out humanity and replacing it with species that are “more worthy” according to its utility function.
• We would want some portion of X to include the concept of “autonomy” and preserve our abilities to make informed, uncoerced decisions. But, a sufficiently smart agent could peacefully convince (trick?) me into making any number of ludicrous decisions. It's not clear how to unambiguously define “coercion” in the case of a super-intelligent agent.

3. A simple decision theory, such as Evidential Decision Theory (which I believe subsumes Hofstadter superrationality), or Causal Decision Theory (which is the standard in mainstream Game Theory.) Either should satisfice, though I regard Evidential Decision Theory as much simpler.

Being philosophical principles, obviously these can't be directly used to create a real, resource-limited AGI; for example, Solomonoff Induction is too slow for practical use.

But, as a set of normative philosophical principals for a human being to use, these seem like a reasonable starting point.

[edit -- decided to call it "Occam's Meta-Razor" rather than "Meta-Occam's Razor"]

Superrationality and the placebo effect

Let me introduce a conjecture that I will call the Strong Biological Placebo Effect. The Strong Biological Placebo Effect states: if you believe a course of action can improve your health, then the mere belief invariably triggers biological changes that improve your health.

If the Strong Biological Placebo Effect is true, then it creates a situation where superrationality applies to human beings. You can consistently, and rationally, choose to believe that setting your alarm clock to prime numbers will increase your health; alternatively, you can consistently, and rationally, choose to believe that setting your alarm clock to prime numbers will not increase your health. If you are superrational, you will choose the latter option, and you will be healthier because of the superrationality.

(Caveat: the Strong Biological Placebo Effect is probably not even remotely true, so don't whip out your magnetic bracelets quite yet.)

Wild Guess: Singularity in 2024

Suppose there is no World War III; suppose that there's no single disaster sufficient to wipe out more than, say, 10% of mankind in a single year. When will the Singularity arrive?

Kurzweil's scenario gives us affordable human-level hardware around 2024, according to my interpretation of his graph. I find his "accelerating exponential growth" model of pre-Singularity computer hardware to be more reasonable than straight Moore's Law, especially factoring in possible nanotech and biotech improvements. Note that Kurzweil states that his models gave him "10^14 to 10^16 cps for creating a functional recreation of all regions of the human brain, so (he) used 10^16 cps as a conservative estimate." I'm interested in "most likely" rather than "conservative", so I used 10^15, but that doesn't make a huge difference. I also picked a "most likely" spot on the gray error-bar rather than a conservative extreme, which does shift things significantly.

Kurzweil believes that the Singularity would arrive decades after we have cheap human-level hardware, but I think it's more likely to arrive a little bit ahead or a little bit behind. So, my wild guess is 2024: meaning that while it's unlikely to arrive in exactly that year, I give it a 50/50 odds of being before or after. Of course, it could end up being "never". It could end up being next year.