Wednesday, January 8, 2014

The Truth About "Proof"

How many times have you heard an advertised muscle building or fat loss add espouse something to this effect:
"X product/program/diet is scientifically proven to make you stronger/leaner/a fat burning machine!"
We hear these sorts of marking ploys all the time, but what exactly is meant by the phrase "scientifically proven" (or any other such phrase)?

In short, this phrase simply means that the person/company/doctor/whoever else who was doing the advertising has no real understanding of how science works.

Logical Proofs vs. Scientific Probability


Many lay people, and even some scientists, make the common mistake that science is a tool which allows its practitioner to prove things.  In reality, science cannot be used to prove anything.

Without getting too technical, suffice it to say that "proofs" are a component of logic and mathematics, not science.  Logic and mathematics deal with formulas and abstract concepts that are considered to fall within the realm of universal truths.  For instance, 2+2=4 is a mathematical concept that is true, independent of the actual existence of 2 as a real, physical object.  In the case of logic, take this very rudimentary logical formula:  
All A are B.
X is an A.
Therefore, X is a B.
This logical formula is abstractly provable, despite the fact that our terms A, B, and X may not refer to anything in actual existence.  These terms are provable because, as we are dealing with abstract terms and not real things, we essentially are dealing with a closed system wherein the only variables with which we have to contend are those provided in our logical phrase.

In the world of science, however, things become much, much, much more complicated.  A phrase such as "all A are B," if it is going to apply to actual things in existence, necessitates that we know it to be true that all things "A" really are "B."  For example, the phrase "all swans are white" necessitates 1) that all the swans we have observed are white and 2) that all the swans we have observed are all the swans there are in existence for us to observe.

I don't think I have to go into too much more detail to demonstrate the very real epistemological (that having to do with the nature of knowledge, its presuppositions and foundations, and its extent and validity) problems which arise from this very simple phrase.  How do we know all swans are white?  How can we be sure we've observed all the swans that ever were, are, or will be in existence?  Are we just referring to all the swans in existence now?  I could raise one question after another ad nauseam, but I think you get the picture.

This issue of epistemology is the very reason why science cannot deal in proofs.  It can only deal in probabilities.  Moreover, these probabilities can only be inferred from a very small subset or fraction of the whole system or scientific theory in question.  In order for my theory to stand as highly probable, that all swans are white, I must demonstrate after repeated observation that all the swans I have observed thus far have not been anything other than white.  The fact that all the swans I have observed have been white does not prove anything.  Thus, my goal in conducting continual observation is to falsify my theory, not prove it.  If I find just one black swan, my theory is falsified.  Grant it, one might say "well, can't we thus prove a theory is false?"  Perhaps, if we wanted to get cutesy with our terms, but let's not digress.

Reality as a Deck of Cards:  A Very Big Deck of Cards


If there's anything that we could most liken scientific experimentation to, it could be a deck of cards.  On top of that, make it an infinite deck of cards.

Say I start drawing cards from the deck, and I notice that all the cards I've drawn are aces of spades.  From this observation I make a hypothesis that all the cards in this infinite deck are aces of spades.  So, in order to gain further support for this hypothesis, I make repeated observations by continuing to draw cards from the deck.  Each time I draw another card and it turns out to be an ace of spades, my hypothesis gains credibility.  Eventually, my hypothesis becomes a theory (which is a well tested and as yet un-falsified hypothesis).

Now, because this deck is infinitely large, I can never know or prove that all the cards in the deck are aces of spades.  I can only say it seems probable that all the cards in the deck could be aces of spades. 

Let's shift gears and imagine the whole of the human race as a deck of cards.  Say I have a hypothesis that X supplement will yield Y result in whoever takes Z milligrams of it daily.  In order to demonstrate the probability that this hypothesis is probable, I must make a number of repeated observations of humans who take Z milligrams of X supplement daily to see if they experience Y.  I thus set out "draw" several people out of the deck of humanity.  We could even add some complexity here and say there are multiple human decks, each presenting a different demographic.  There could be a deck for the elderly, the young, the healthy, the obese, athletes, recreational exercisers, sedentary people, etc.  If I only draw from one deck, not only can I not make any inferences about the other decks, but I cannot say with certainty that all the people in the deck tested will experience Y if they take X.

To further confound the situation, these decks that I'm drawing from are continually changing.  After all, demographics change in size and scope all the time.  People die, they are born, they could start out healthy then become obese, or start out obese and become healthy.  So, what of these people who move from one demographic to another?  Should we just move them to one of the other existing decks, or, because their situation is not entirely like that of the people who have been long time members of a given deck, must we create entirely new decks for these people who change their status?

This situation is even more complex in the real world (as if it wasn't already complex enough!).  In the real world, not even everyone in a given experiment or observation will see exactly the same results with a given supplement, diet, or training regimen.  Thus, when scientists examine data from their experiments, they have to base their conclusions on the average or mean result gotten when crunching the numbers of the wide range of results obtained from experimentation.  Some people may not respond well to supplement X, others may have benefited greatly.  When scientists look at their results, they have to make very tentative suggestions based on their findings (one common suggestion researchers make is that more testing and research should be conducted), because, not only may supplement X not have universally worked in all subjects tested (say the product was effective for 70% of subjects tested), we still have no idea whether supplement X will benefit the same percentage of people in the entirety of the demographic the scientists chose to observe.

People with a descent awareness of these issues (not necessarily and understanding of them) will be open and honest about the limits thereof afforded to the conclusions made from scientific experimentation.  Now you too can be one those people who are aware that, if anyone ever says such and such as been scientifically proven to do this, that, and the other, said anyone has no idea what kind of fallacious statement they're actually making.

The Importance of Self-Experimentation and Knowing Your Own Body


While I hope I've not entirely destroyed your faith in science (though I hope I have somewhat diminished any romanticized views of science you may have previously had), be aware that we should not throw the baby out with the bath water.  If anything, these issues faced by scientists' attempts to extrapolate the various uses of supplements, exercises, and diets ought to free us somewhat to make our own conclusions about how said supplements, exercises, and diets can work for us.

It's incredibly difficult to make a universal claim about A being B.  It's by far easier to assess whether a particular a exists as a particular b.  Though we can certainly use universal probabilities as a starting point, ultimately we must be willing to look out our own bodies and personal experiences to assess whether a universal probability is our particular reality.

We should never feel obligated to dismiss our anecdotal experiences, or those of others, just because some dimwit looking to make a buck sensationalized a scientific experiment for the purpose of pedaling some supplement, workout, or diet as a "proved" reality.  

Keep these ideas in mind as you peruse the web and are attacked by popups and advertisements espousing proven methods or supplements for weight loss, muscle gain, or life extension.  Nothing in science is proven.  

Now that you know this, you can confidently disregard 99% of all online, radio, and television marketing ploys.  

No comments:

Post a Comment