Groucho Marxism

The American physicist Richard Feynman once famously remarked: “If you think you understand quantum mechanics, you don’t understand quantum mechanics.” The point Feynman was making is that quantum mechanics is strange, weird, counter-intuitive, and that some questions about it cannot or should not be asked. However, this strangeness disappears if we are willing to accept that one of the assumptions that is usually made about quantum mechanics is not realized in nature. This assumption is known as ‘statistical independence’. Loosely speaking, statistical independence means that spatially separated systems can be considered uncorrelated, so removing this assumption means that they will be correlated, even in absence of a common past cause.

Statistical independence is an assumption that we make about nature which works well to describe our observations on the macroscopic ‘classical’ level; however, that doesn’t necessarily means it holds on the microscopic ‘quantum’ level. In quantum mechanics, statistical independence can be expressed as a lack of correlation between the ‘hidden variables’ of the theory and the detector settings of an experiment. The hidden variables represent all the information that is required to predict the outcome of an experiment for the detector settings. These variables are referred to as ‘hidden’ because they do not explicitly appear in the equations of quantum mechanics. Removing the statistical independence assumption means that the outcome of an experiment will depend on the detector settings.

The theory that emerges on the removal of the statistical dependence assumption is known as ‘superdeterminism’, as it suggests the evolution of the entire universe, including our measurement choices, is completely predetermined and correlated. It implies that everything in the universe is connected with everything else. Superdeterminism is often considered a ‘conspiracy’ in which the detector settings of an experiment are influenced by hidden variables in just the right way to give the results of quantum mechanics. But this objection is based on the schoolboy error of confusing correlation with causation. All superdeterminism assumes is that the hidden variables and detector settings are correlated; it doesn’t assumes anything about the direction of causality.

Another objection people make about superdeterminism is that it implies we have no free will, as it suggests that the choices experimenters make about what to measure are not truly independent but are predetermined and correlated with the particles being measured. But as I outlined in a previous blog post, free will is an illusion; and not only that, it is completely incoherent as a concept. I therefore take the lack of free will under superdeterminism as a reason to support rather than reject the theory. However, the main argument in support of superdeterminism is that it neatly sidesteps a well-known problem in the interpretation of quantum mechanics. This problem is encapsulated by what is known as ‘Bell’s theorem’, named after the Irish physicist John Stewart Bell.

Bell’s theorem says that under the assumption of statistical independence, either realism or locality must be violated. Which is to say that either particles don’t have definite properties before measurement (realism is violated) or influences can travel faster than light (locality is violated), or both. The theorem has its roots in the famous Einstein-Podolsky-Rosen thought experiment proposed by physicists Albert Einstein, Boris Podolsky, and Nathan Rosen in a 1935 paper. Their paper suggested that quantum mechanics is incomplete because it allows for ‘spooky action at a distance’: instantaneous correlation between entangled particles, seemingly violating locality. They argued that to avoid this violation of locality, there must be some hidden variables that aren’t accounted for by the theory.

Bell showed in 1964 that under the assumption of statistical independence, introducing hidden variables will not solve the problem of non-locality. In the words of Bell: “If [a hidden-variable theory] is local it will not agree with quantum mechanics, and if it agrees with quantum mechanics it will not be local.” But Bell’s theorem is predicated on the assumption of statistical independence; remove that assumption and the theorem no longer holds. In other words, removing the statistical independence assumption means that there might be hidden-variable theories that do not violate locality and agree with quantum mechanics. These hidden variables encapsulate definite properties of a particle, so both realism and locality are saved.

I am far from an expert in quantum mechanics so my opinion on this carries little weight. For what it’s worth, though, I think superdeterminism has a lot going for it. As already noted, it reinforces the truism that we lack free will. It also suggests that everything in the universe is connected, which nicely counters the bogus Western philosophical idea that human beings are atomic decision-making agents interacting with an external world. In this sense it has a lot more in common with Eastern philosophy. But the main reason I like superdeterminism is that it starts with what we know from experimental data must true (quantum mechanics, realism, and locality) and then asks which of our assumptions must be incorrect in order to allow these things to be true simultaneously.

Unfortunately, all too often we human beings do things the other way around. There seems to be something inherent to human nature that when faced with evidence that contradicts our prior assumptions, rather than re-examine our assumptions, our first reaction is to reject the evidence! Even quantum physicists, arguably the most intelligent members of our species, are guilty of this it seems. Academics are perhaps even more guilty of this than the general population, having often staked their careers on their prior assumptions being correct. This explains why so many academic disciplines end up going down blind alleys (mainstream economics being the most egregious example). If we are to have any hope of understanding the world around us, we have to start by looking at the evidence.