The Probability Paradox

[RArvay]

.
If quantum theory is substantially correct, then randomness is a fundamental basis upon which the universe exhibits phenomena. Probability theory, however, may contain a fundamental flaw. Understanding that flaw requires overcoming some semantic barriers.

A few years ago, I posted a brief comment to an online discussion board which resulted in acerbic responses from readers who accused me of everything from idiocy to trolling. I expect that my doing so here will initially elicit a similar impulse, except that the readership here is more willing to respond with reasoned arguments than with ad hominems, even if ad hominem attacks are justified.

The confusion (and it may be mine alone) arises from the way in which probability theory is usually worded, with the key words being “may,” and “must.” Given an infinite number of random coin flips, any single coin flip has an equal chance of landing heads or tails. But on the larger scale, whether half the coin flips “may” result in heads, or “must” result in heads, is (at least semantically) in dispute.

Some assert that probability theory— as defined— holds that half the flips “must” result in heads. The phrase, “as defined,” is critical to sorting out the argument. The question is not whether the coin flips may indeed have an equal distribution, but instead, whether probability theory predicts that they must. These are semantic eggshells upon which I am treading as carefully as possible.

Others assert that the theory states merely that the likelihood of the coin flips having an equal distribution is so large as to be a virtual certainty, but that any distribution is possible, according to theory.

Here is how I worded a thought experiment, basing it upon the probability theory that expresses a “must.” If half the coin flips “must” result in heads, then imagine a tiny universe, short-lived, in which there are only two events, two coin flips. I posited that, if the theory is worded correctly, and if the first coin flip is heads, then the second coin flip must of necessity be tails.

Here, the key word is “if,” but that got lost in the initial discussion, as did other vital parameters of the thought experiment.

The paradox is that if the second outcome is a certainty, then there is no probability. On the other hand, if probability theory can only be stated as a “maybe,” then it is no theory at all, but simply a circular conjecture.

Granted, the semantics here pose a barrier to understanding the core of what I am saying, but that core points to a fundamental principle of physics.

A key source of the semantic confusion is the concept of infinity. The question of whether an infinite quantity of anything can physically exist was debated as far back as the time of the ancient Greek philosophers. Probability theory is often said to be predicated on the idea that half of an infinite number of coin flips must land tails, which then translates into the statistical expectation that, given a very large number of coin flips, about half will land heads. The approximation takes into account that there are a finite number of coin flips at issue.

However, the concept of infinity refutes that. We normally think that an infinite distribution of coin flips is half heads and half tails, but it quickly becomes apparent that an infinite number of heads, with a corresponding infinite number of tails, might not have a one-to-one ratio. The ratio might be ten to one, since one tenth of infinity is still infinity.

Probability inherently involves ambiguity, but theories themselves should not. It is one thing for a theory to be uncertainly true or false, but the statement of a theory— the statement, mind you— should propose fact, not likelihood.

I believe it was Alan Guth who stated that in an infinite universe, everything that can happen, must happen, and indeed must happen an infinite number of times.

His use here of the word, “must,” defines probability as being non-probabilistic, a self-contradiction. While any single event can still be defined as probabilistic, on the infinite scale, there is only certainty. And in an infinite universe, what restrictions are there on what “can” happen?

If I choose both, then I have not chosen.
.
.
.

[Ben6993]

Not sure at all what you are getting at here. Not sure how much statistics you know.

Forget infinity, just concentrate on large N.
N can tend to infinity and you may get a finite limit for a calculation depending on the N, but N is only tending to infinity and not reaching infinity. So just think of large N,

You have a population of N objects and you sample n of them. There is no 'must' about the sample outcomes. Any outcome is possible, but some outcomes are more likely than others. This is for a calculation based on the n objects drawn at random from the N objects. You can roll two normal die and get outcome is sum=2, even though the outcome sum=7 is more likely.

I can't see what you are getting at for the case n=2 where you require one head and one tail? Why should you require that?

[minkwe]

If quantum theory is substantially correct, then randomness is a fundamental basis upon which the universe exhibits phenomena.

I don't agree with that statement. Quantum theory could be substantially correct (I believe it is) without requiring randomness to be a fundamental basis of any phenomena.

Probability theory, however, may contain a fundamental flaw. Understanding that flaw requires overcoming some semantic barriers.

While I agree with you that overcoming semantic barriers is key to many of these issues, there is no fundamental flaw in probability theory, if understood properly. You have to start from the beginning and define your terms consistently.

What is probability theory?
What is a probability?

I would recommend ET Jaynes book: Probability Theory: The logic of science. (a snippet of which is available freely online or his papers about Probability in Quantum mechanics which you can find http://bayes.wustl.edu/etj/articles/prob.in.qm.pdf, http://bayes.wustl.edu/etj/articles/cmystery.pdf)

For example, he asks the question:

Is probability theory a "physical" theory of phenomena governed by "chance" or "randomness"; or is it an extension of logic, showing how to reason in situations of incomplete information? For two generations the former view has dominated science almost completely.

I think the former view is the one underlying your post. The confusion and paradoxes are completely absent in the latter view, and Jaynes goes into more detail later in the paper especially as concerns QM.

Jaynes also provides a consistent definition of "Probability" which goes to the heart of what your post is about and "clears up the mysteries" if you get the pun:

In our system, a probability is a theoretical construct, on the epistemological level, which we assign in order to represent a state of knowledge, or that we calculate from other probabilities according to the rules of probability theory. A frequency is a property of the real world, on the ontological level, that we measure or estimate. So for us, probability theory is not an Oracle telling how the world must be; it is a mathematical tool for organizing, and ensuring the consistency of, our own reasoning. But it is from this organized reasoning that we learn whether our state of knowledge is adequate to describe the real world.

[RArvay]

.
In answer to the question of, what am I getting at, I propose that nature is best explained neither by determinism nor by chance, nor by any combination of these. If I may, I propose something called, “volition.” Simply stated, I am of the opinion that I freely choose my own actions, influenced by, but not enslaved by, natural law. Since that tends toward forbidden topics on science discussion boards, I restrict my treatment of this proposal to matters which can be subjected to scientific thinking. In particular, I am focusing on the role of probability in nature.

In my view, the theory of strict deterministic causation needs no discussion, since if it is an accurate model of the universe, everything is predetermined, and we are but cosmic robots, not free thinkers.

A purely random universe would be absurd as well, since an infinite array of possibilities without nonrandom parameters would be utterly unpredictable.

Natural materialist explanations therefore seem to have settled on some hybrid of determinism and random chance, and it is the topic of random chance that concerns me here.

Randomness cannot operate except within nonrandom parameters. A die roll may seem random, but its outcome is restricted by the number of sides on the die. There may be four, or more, sides to any die. The die may be balanced or unbalanced. If the universe makes dice at random, with any number of sides, then an infinitely large universe has an unrestricted array of possibilities, in which case randomness has no meaning.

Another challenge to my thinking on this is the concept of infinity. There are in fact two ways of conceptualizing infinity. One way is endless recursion, the n = n + 1 algorithm, which never ends, and never reaches infinity. According to this way of thinking, there is an infinite number of finite integers, which semantically is a self-contradiction. Infinite means endless, but does it mean endlessly expanding?

The other way is to calculate the number of geometric points on a finite line segment. If each point has dimensions of zero, then one cannot sequentially number the points on the line, because one never gets past the first point (0 times two equals zero, etc). In this case, we say that there are an infinite number of points on any finite line segment. We have reached infinity all at once, not in finite increments.

As to whether the term infinity has any meaning in actual physics is debatable.

One unanswered question in cosmology is whether the universe is of finite size or infinite. If the universe is endlessly expanding, but always of finite size, then we use the recursive algorithm. It may be, however, that the universe (or multi-verse) has already reached a size of infinity, analogous to the number of points on a finite line segment.

All this is somewhat of a diversion, however, as the subject is randomness. The side discussion of infinity is important only as it applies to randomness.

So let us ask this question: What is the definition of a fifty percent chance? Here the semantics become very important, and it is frightfully easy to impose one’s bias into the question.

If one defines something, he must speak in terms of certainty, or at least in terms of exclusion. A tree is not a rock. If we say we cannot define what a tree is, that is one thing. In mathematics, however, we cannot say that one plus one is approximately two. It is two. Period.

The definition of a fifty percent chance— not our perception, but the definition itself— is that there is an equal likelihood of outcome A or B, where A and B are mutually exclusive, and one of them or the other must occur.

By extension, what this means is that of an infinite set of occurrences, half must be A, half must be B. Again, it is crucial to note that regardless of the outcomes, it is the definition that we are concerned with. If the definition is “maybe,” then it is no definition at all, even if the actual reality is maybe.

Otherwise, we are saying that the definition of a fifty percent chance is the definition of some other percent chance. That would be a paradox.

This is why I simplified everything down to a thought experiment, a tiny universe in which only two events occur, each at random, with each event being defined as either A or B, and each of which has a fifty percent likelihood. If (and it is a very big IF) the definition of a fifty percent chance is that outcome A must occur half the time, then once the first event has occurred, there is no more chance. The second outcome is necessarily the opposite of the first. That is a self-contradiction, and illustrates the probability paradox.

The probability paradox, in my opinion, demonstrates that the universe cannot be inherently random. Utility tells me it cannot be deterministic, since no useful conclusion can arise from declaring that we are all helpless slaves, unable to think for ourselves.

That leaves volition as a basis of physical reality.
.
.
.

[Matthew]

You defined equal chances as "equal likelihoods" and said that therefore it follows that in an infinite set of trials, there will be equally many occurrences of the two outcomes. But you did not define "equal likelihood". It's true that Laplace defined probability by reduction to a prior concept of equally likely, whereby he was possibly thinking that physical reasons of symmetry might justify "equally likely".

So for Laplace, P(A) was defined as the ratio of the number of ways the event could happen, divided by the total number of possible outcomes, when all those possible outcomes are "equally likely".

We can go on to *prove* the law of large numbers and show that if all 2^n particular sequences of outcomes of n coin tosses are equally likely, and if n is rather large, then all except for an exponentially tiny fraction of sequences have close to equal numbers of "H" and "T".

Physical symmetry could justify the "equally likely" character of the 2^n sequences. Then if n is very large indeed, we will just have to use a further heuristic principle, namely that if P(A) is larger than 1 - 10 to the minus some very large number, then A is effectively certain.

In a toy universe where there is just one occasion when "nature makes a choice" (and the choice is binary) then the notion of probability just doesn't arise. This doesn't mean that Laplace's definition plus the heuristic principle of effective certainty is wrong. It just means that it does not apply to anything interesting in that toy universe. But so what?

[RArvay]

Actually, these are not MY definitions. They are the definitions inherent in the terms used to define probability.
Colloquially, we can say that a coin flip has one chance in two of landing heads. Finesse that as you will, but
the point remains the same-- which is [very obviously] that the definition of a fifty percent chance
is not the same as the definition of a chance that is NOT fifty percent.

If the chance of outcome A is fifty percent, it is not any other percent.

If the law of probability defines a chance as fifty percent, then the expectation -- as contrasted to the actual outcome --
is that half the outcomes WILL be A, and the other half WILL be B, given enough instances.
Not, close to half, but exactly half, by definition. That is the statistical expectation, regardless of the outcome.

"Given enough instances" requires an infinite number of instances to insure sufficiency, and this clouds the issue with definitions of infinity,
but once again, if we are careful not to stray from the central question, then we must focus on the DEFINITION
of fifty percent likelihood, regardless of the actual outcome.

We are discussing the pure theory of probability, which requires precise and exact definitions.

The relevance to physics, and in particular to the proposal of volition,
is that if pure chance is not an inherent property of the quantum universe, then the universe
is inherently deterministic, which rules out any possibility of free, independent inquiry in science,
but instead makes of all of us, cosmic puppets, predestined in our every thought, word and deed.

If the universe, on the other hand, is inherently random, then there must be non-random parameters
within which randomness operates.
Non-random, non-deterministic factors lead us to the concept of autonomous volition.

[Ben6993]

RAvray wrote:
If the law of probability defines a chance as fifty percent, then the expectation -- as contrasted to the actual outcome --
is that half the outcomes WILL be A, and the other half WILL be B, given enough instances.
Not, close to half, but exactly half, by definition. That is the statistical expectation, regardless of the outcome.

If you have a population of say 5 heads and 5 tails (N=10) then the exact expectation for an even sample (n) is that there will be an equal number of heads and tails in the sample. You need to calculate the standard deviation to find out how much deviation from expectation is likely to occur in the sample. And even the sample standard deviation is not immune from having its own error bar. But, as you have a known population of ten outcomes you do know that there are exactly five heads and five tails in the population. The N and n are finite, and there is no paradox here.

You seem to be taking the case when N and n are infinite. So look at the case when N and n are very large, and the population (N) has an equal number of Hs and Ts. There is still an exact expectation for an even sample that there will be an equal number of heads and tails in the sample. The standard deviation for the sample (standard error of the mean) is very much smaller now, but still not zero. So the expectation still has a small chance of not being met in the large sample. If you make N and n infinite, then the expectation has to be met. But how do you make N and n infinite? Your 'tiny universe' has n=2 and 2 does not have the same difficulties associated with using infinity, so I do not think your 'tiny universe' is helpful. So far, I do not see a paradox about probability, only a difficulty in dealing with infinity.

A problem with using infinite values of N and n is that you are looking at a calculation based on an infinite number of units eg the average number of heads in the infinite sample. In one sense you need to step forward to the end of the universe in order to perform the calculation. You could of course try to use Zeno's paradoxes to do the infinite calculation in a finite time, but that may not work in physics because of the uncertainty principle. And it won't work in my mind's eye version of this sampling which involves the population of Hs and Ts being in a very large bag and drawing out the sample by hand one at a time. A very long job. Unless you can think of ways to perform the infinite sampling sum in finite time? And even if you did, it would still be the case that for every unit of the infinite sample, I drew either an H or a T out of the bag with equal chance of that happening.

[RArvay]

Either it will, or it will not.
If it will not, then the statement of ratio is flat wrong.
If it will, then there is no probability involved.

[minkwe]

Either it will, or it will not.
If it will not, then the statement of ratio is flat wrong.
If it will, then there is no probability involved.

Unfortunately you are confusing "frequencies" with "probability" and getting yourself mixed up.

In our system, a probability is a theoretical construct, on the epistemological level, which we assign in order to represent a state of knowledge, or that we calculate from other probabilities according to the rules of probability theory. A frequency is a property of the real world, on the ontological level, that we measure or estimate. So for us, probability theory is not an Oracle telling how the world must be; it is a mathematical tool for organizing, and ensuring the consistency of, our own reasoning. But it is from this organized reasoning that we learn whether our state of knowledge is adequate to describe the real world.

A probability is a number you assign based on information you have about possibilities. A relative frequency is something you calculate from the results of repeated measurements. Probabilities do not tell you what happened or what will happen. They tell you what will "likely" or "probably" happen based on the information you used to assign them.

I really suggest you read the first few chapters of Jaynes' book or the articles I suggested earlier. It will be worth your while. Just because there is wide-spread misunderstanding of "probability" does not mean your criticizm of "Probability Theory" is legitimate. Please read those articles and see how your problems with probability simply disappear.

[RArvay]

Thank you. I will do that.
However, I am not asking regarding any after-the-fact observations.
I am asking about the definition.
What is the definition of a fifty-percent chance?
Is it exactly fifty percent expectation, or not?
I know that the actual expectation is uncertain.
The definition should not be.
.

[minkwe]

Thank you. I will do that.
However, I am not asking regarding any after-the-fact observations.
I am asking about the definition.
What is the definition of a fifty-percent chance?
Is it exactly fifty percent expectation, or not?
I know that the actual expectation is uncertain.
The definition should not be.
.

A "fifty-percent chance" means based on what I know about the situation, the "probability" that the event will happen in the future is just as likely as it not it not happening. In other words, I have no information to enable me to say either way whether the event will happen or not. A "75 percent chance" means based on what I know, the event is more likely to happen than not happen and that information allows me to assign a degree to the belief of .75 likely to happen 0.25 likely to not happen. Probabilities are simply logical mechanisms for us to organize our thoughts and reasoning based on incomplete information.

A relative frequency of 50% means the event happened 50% of the time in the past. There is nothing which prevents you from assigning a correct probability of 0.75 to a situation before the fact, and later calculate an actually measured relative frequency of 0.5. Probability theory simply provides consistent rules for manipulating and assigning probabilities based on prior information. It is not an oracle which decides what the world must be, nor does it tell you that you have all the relevant information. If all the information available to you implies you must assign a probability of .75, then that is the correct answer.

All these things are clearly explained in the references I provided.

[Schmelzer]

.
If quantum theory is substantially correct, then randomness is a fundamental basis upon which the universe exhibits phenomena.

Incorrect, there exists at least one deterministic theory which makes statistical predictions equivalent to QM, namely de Broglie-Bohm theory.

Probability theory, however, may contain a fundamental flaw. Understanding that flaw requires overcoming some semantic barriers.

Improbable. If one follows the approach of Jaynes, an objective variant of the Bayesian interpretation, probability theory is essentially the logic of plausible reasoning. This has, of course, nothing to do with the frequency interpretation which you seem to criticize.