What we know about critical thinking
Here, I want to cover some of the important particulars of critical thinking. Keep in mind the word: critical. It is no accident this word is related to the word “criticism” - for that is the very technique required in critical thinking. One must criticise. What does this mean? Criticise means what it does to almost all people all of the time with one caveat: it does not (or should not) carry the emotional liabilities and baggage people tend to associate with it. Criticism is a wonderful thing - when applied to ideas. It is the means by which ideas are improved. Criticism is about pointing out what is, or may, be wrong with an idea. It is about highlighting, pointing out, making obvious - the weaknesses, flaws and false assumptions of an idea.
Criticism is a broad topic. In science criticism might, most obviously, consist of an experimental refutation. If the experimental results disagree with the hypothesis - one of three things must hold: the hypothesis is false or the experiment was flawed or both. Something that a scientist is thinking about what is going on there - is false. But this is why we are (or at least should be) very careful in science. As careful as we can be. Truly carefully controlled, well performed, accurate and precise experiments can be called “crucial” experiments and can decide between scientific theories. This is a topic of my previous two podcasts. In a situation where two theories make incompatible predictions about the outcome of an experiment we have a problem - which one is correct if, all else being equal they have hitherto made the same predictions about the world. Well we may be in the fortunate position to design and then perform this crucial experiment - this crown in the jewel of the scientific enterprise.
So, the experiment is performed and just one of the theories disagrees with this results. So we say it is, tentatively as always, shown to be false. And thus progress is made. One of those theories is successfully criticised (and typically mortally wounded) in the process. This method of criticism has a simple aim: to show as false an idea. I have written previously about the specifics of how this can work in physics and for that look up my article titled: general relativity and the role of evidence.
This idea of experimental refutation - also called falsification is essential in science. It’s not just a convenient add on (be critical of what you hear from some actual scientists - always physicists - on this topic. They are making an error about what science is).
But experimental testing - although essential it is not the whole story. It is indispensable - but this is only to say it is necessary - but not sufficient. Far from sufficient. Indeed many theories need never actually be tested by an experiment to be refuted scientifically.
My readers will be well aware of David Deutsch’s brilliant examples in his excellent book “The Fabric Of Reality” where he asks us to consider the theory that eating a kilogram of grass is a cure for the common cold. That is a testable theory. But do we need to do the experiment to refute it?
No.
And the reason is because we can be critical in another way. Another mode of criticism if you like - less about the scientific methodology of the situation and more about the epistemological fundamentals of what is being claimed with the grass-eating theory. Or rather “grass eating prediction” - after all that so-called theory or hypothesis is nothing but a claim of the sort “If you eat the grass then your cold will be cured” - it is a prediction but it contains no explanation. Now this idea that some claim is predictive but not explanatory is eminently scientific way and wonderfully philosophical. (We can use both words here deliberately to illustrate that here, at the boundary, the techniques really do bridge the permeable wall that separates our rational investigations into the world). Explanations run the entire gamut of our knowledge of the world but strictly, predictions are the domain of science, properly construed. And even then good scientific predictions are always subject to what else we might learn in the future that could change what we said was going to happen (eg: a prediction that the world will warm by 3 degrees Celsius in the next 200 years is not an accurate prediction at all if it does not come true and it might not come true if people do things to alter the course of events such that it never comes to pass that the warming does not happen. For example by sucking carbon dioxide out of the atmosphere using some method to be invented in the next 50 years). Whatever the case - if someone comes to you with a claim that eating grass for example will cure your cold does it need testing? No - because it is explanation less.
In my discussion with Naval on the Naval podcast - look for that it’s called “The Beginning of Infinity” podcast - we spoke about actually doing this experiment as David Deutsch does in The Fabric of Reality. If you do the experiment we woulds need to specify the quantity of grass and if we specified it was exactly 1kg of grass and then we find that eating precisely that amount does not cure the cold, this would not show that 1.1kg of grass would not cure the cold. Or 0.9kg. Or any of an infinite number of variations. Although the hypothesis is falsifiable - it is also infinitely variable and experiment cannot possibly refute all variations of it (we can imagine some medical crank saying “well maybe it’s 1.01 kg. Or 1.001 kg and so on).
Experiment really is not the appropriate critical tool here even though it is one of the critical tools in our critical arsenal. So what else do we have?
The appropriate critical tool here is the explanation criteria. What is the explanation criteria? It is simple the distinction between whether a claim comes with an explanation or not. It’s that simple. Most especially if the claim is scientific, but not necessarily (in sport there are often reasons a skilled practitioner does a certain thing but a fully formed good explanation might be elusive. This gets into the weeds a bit but basically there are lots of places where the knowledge is inexplicit. We’ll ignore that wrinkle for now. Let’s stick to science, which we can generalise to many other areas).
So let’s apply the explanation criteria to “grass cures the cold”. What is the explanation of how grass cures the cold? ....Um....what explanation?
Right! Do we even have an explanation of how grass might cure the cold? No! If we did we would be able to criticise that (for example by using an experiment). But absent an explanation that is the criticism.
No explanation? That’s a flaw. That’s a problem. Indeed in science: that’s an error.
For someone who comes along and says “Grass cures the common cold” or indeed comes along and makes any claim ask: how? If there is no (reasonable/satisfactory/good - call it what you like - etc) if there is no explanation - it can be rejected.
Without an accompanying mechanism of action we can reject it without ever doing any kind of experiment (let alone on ourselves). This is true of almost all claims made in science and elsewhere: we don’t even bother with the experiment. What if it did work? Then we would seek to find out why and until we had a criticism of the theory “eating grass cures colds” that is our best theory that it would be possible to improve (for example, we might find particular amounts work better than others and chemists might isolate the active ingredient. A story much like this explains the theory that chewing on the bark of a willow tree relieves headaches...and that eventually led to pills of aspirin as a really effective headache treatment).
In medicine once upon a time it was the case that some treatment was found to “work” but it was not known why. And by work we mean: better than no intervention at all. But these days in medicine that is a rare exception to the general rule: there are entire fields of science from pharmacology to physiology, neurology and all kinds of carefully practised and error corrected surgical interventions and even where some medicine is used “off label” as we say there is often a good explanation known by at least some members of the medical community even if it has not been published. One reason for this is that, as it turns out, doctors are very busy people and getting around to going through the relatively recent phenomenon that is the “peer review process” is rather more difficult than word of mouth or these days a more informal review process. Peer review of double blind placebo controlled trials is indeed a gold standard of a kind - but it is also the most lengthy and difficult way of showing the efficacy of what might already be known to be a good explanation of efficacy. To be “known” in this sense has multiple meanings: someone might already know of a treatment and a good explanation for it, before the rest of the scientific community does. Take for example the famous case of - let me plug Australia here - the Australian doctor Barry Marshall who had an idea that stomach ulcers were caused by a bacteria and therefore could be cured by antibiotics. Until Barry Marshall many people thought stomach ulcers (indeed today many people still think they) are caused by stress, anxiety or something like that.
Anyway so the story goes - true story - Barry Marshall swallowed a nasty mix of ulcer causing bacteria. A good Aussie risk taker. The crocodile hunter of medicine. Anyways - yes - he got stomach ulcers. Now at that point - in pain and no doubt writhing in agony in bed waiting for the penicillin or whatnot to kick in and cure him - he knew. And his colleagues knew. They really knew the good explanation. But it was not yet double blind placebo controlled and reported in peer reviewed journals. So the community did not know. But he knew. And he would have been remiss had he seen a patient the following week and used antibiotics off label for anyone who came to see him and who he diagnosed as having stomach ulcers. All he could say was “I know this works. I did it to myself. Stomach ulcers are caused by bacteria and I know how and why regardless of what the literature does or does not say.”
So much is true of any scientific discovery yet to be published. Of course errors can be made and corrected - but here is the point - until such time as an error is found that IS the best explanation going. An astrophysicist who detects a wobble in a star 200 light years away over the course of 3 years of their PhD and does the appropriate calculations knows long before the publication of their thesis much less the publication in a journal somewhere much less before any other scientist even bothers (if they ever do) to check the results…that they have found a planet. That is the other thing about some scientific journal articles - some are barely ever read. And if they are only a small portion of those are ever checked carefully - which is to say the experimental results where they exist are checked - at least in a timely fashion. So some individuals or small groups of people can indeed have good explanations when others do not. The criticism has happened not in the way we are supposed to believe it can only happen in science - peer review- but rather validly and reliably in the mind of a single person. Or a small group of people.
And criticism in science is indeed the way we make progress. It is the means by which we reject false theories. And bad explanations. And no-explanations. This criticism - this critical attitude - is cached out through experiments. But not only experiments as I explain above: it can be knock-down, full stop, over-and-out refutation even without an experiment.
And this is true in all subjects, in all spheres of knowledge, always. So how does it work elsewhere?
Part 3
Here, I want to cover some of the important particulars of critical thinking. Keep in mind the word: critical. It is no accident this word is related to the word “criticism” - for that is the very technique required in critical thinking. One must criticise. What does this mean? Criticise means what it does to almost all people all of the time with one caveat: it does not (or should not) carry the emotional liabilities and baggage people tend to associate with it. Criticism is a wonderful thing - when applied to ideas. It is the means by which ideas are improved. Criticism is about pointing out what is, or may, be wrong with an idea. It is about highlighting, pointing out, making obvious - the weaknesses, flaws and false assumptions of an idea.
Criticism is a broad topic. In science criticism might, most obviously, consist of an experimental refutation. If the experimental results disagree with the hypothesis - one of three things must hold: the hypothesis is false or the experiment was flawed or both. Something that a scientist is thinking about what is going on there - is false. But this is why we are (or at least should be) very careful in science. As careful as we can be. Truly carefully controlled, well performed, accurate and precise experiments can be called “crucial” experiments and can decide between scientific theories. This is a topic of my previous two podcasts. In a situation where two theories make incompatible predictions about the outcome of an experiment we have a problem - which one is correct if, all else being equal they have hitherto made the same predictions about the world. Well we may be in the fortunate position to design and then perform this crucial experiment - this crown in the jewel of the scientific enterprise.
So, the experiment is performed and just one of the theories disagrees with this results. So we say it is, tentatively as always, shown to be false. And thus progress is made. One of those theories is successfully criticised (and typically mortally wounded) in the process. This method of criticism has a simple aim: to show as false an idea. I have written previously about the specifics of how this can work in physics and for that look up my article titled: general relativity and the role of evidence.
This idea of experimental refutation - also called falsification is essential in science. It’s not just a convenient add on (be critical of what you hear from some actual scientists - always physicists - on this topic. They are making an error about what science is).
But experimental testing - although essential it is not the whole story. It is indispensable - but this is only to say it is necessary - but not sufficient. Far from sufficient. Indeed many theories need never actually be tested by an experiment to be refuted scientifically.
My readers will be well aware of David Deutsch’s brilliant examples in his excellent book “The Fabric Of Reality” where he asks us to consider the theory that eating a kilogram of grass is a cure for the common cold. That is a testable theory. But do we need to do the experiment to refute it?
No.
And the reason is because we can be critical in another way. Another mode of criticism if you like - less about the scientific methodology of the situation and more about the epistemological fundamentals of what is being claimed with the grass-eating theory. Or rather “grass eating prediction” - after all that so-called theory or hypothesis is nothing but a claim of the sort “If you eat the grass then your cold will be cured” - it is a prediction but it contains no explanation. Now this idea that some claim is predictive but not explanatory is eminently scientific way and wonderfully philosophical. (We can use both words here deliberately to illustrate that here, at the boundary, the techniques really do bridge the permeable wall that separates our rational investigations into the world). Explanations run the entire gamut of our knowledge of the world but strictly, predictions are the domain of science, properly construed. And even then good scientific predictions are always subject to what else we might learn in the future that could change what we said was going to happen (eg: a prediction that the world will warm by 3 degrees Celsius in the next 200 years is not an accurate prediction at all if it does not come true and it might not come true if people do things to alter the course of events such that it never comes to pass that the warming does not happen. For example by sucking carbon dioxide out of the atmosphere using some method to be invented in the next 50 years). Whatever the case - if someone comes to you with a claim that eating grass for example will cure your cold does it need testing? No - because it is explanation less.
In my discussion with Naval on the Naval podcast - look for that it’s called “The Beginning of Infinity” podcast - we spoke about actually doing this experiment as David Deutsch does in The Fabric of Reality. If you do the experiment we woulds need to specify the quantity of grass and if we specified it was exactly 1kg of grass and then we find that eating precisely that amount does not cure the cold, this would not show that 1.1kg of grass would not cure the cold. Or 0.9kg. Or any of an infinite number of variations. Although the hypothesis is falsifiable - it is also infinitely variable and experiment cannot possibly refute all variations of it (we can imagine some medical crank saying “well maybe it’s 1.01 kg. Or 1.001 kg and so on).
Experiment really is not the appropriate critical tool here even though it is one of the critical tools in our critical arsenal. So what else do we have?
The appropriate critical tool here is the explanation criteria. What is the explanation criteria? It is simple the distinction between whether a claim comes with an explanation or not. It’s that simple. Most especially if the claim is scientific, but not necessarily (in sport there are often reasons a skilled practitioner does a certain thing but a fully formed good explanation might be elusive. This gets into the weeds a bit but basically there are lots of places where the knowledge is inexplicit. We’ll ignore that wrinkle for now. Let’s stick to science, which we can generalise to many other areas).
So let’s apply the explanation criteria to “grass cures the cold”. What is the explanation of how grass cures the cold? ....Um....what explanation?
Right! Do we even have an explanation of how grass might cure the cold? No! If we did we would be able to criticise that (for example by using an experiment). But absent an explanation that is the criticism.
No explanation? That’s a flaw. That’s a problem. Indeed in science: that’s an error.
For someone who comes along and says “Grass cures the common cold” or indeed comes along and makes any claim ask: how? If there is no (reasonable/satisfactory/good - call it what you like - etc) if there is no explanation - it can be rejected.
Without an accompanying mechanism of action we can reject it without ever doing any kind of experiment (let alone on ourselves). This is true of almost all claims made in science and elsewhere: we don’t even bother with the experiment. What if it did work? Then we would seek to find out why and until we had a criticism of the theory “eating grass cures colds” that is our best theory that it would be possible to improve (for example, we might find particular amounts work better than others and chemists might isolate the active ingredient. A story much like this explains the theory that chewing on the bark of a willow tree relieves headaches...and that eventually led to pills of aspirin as a really effective headache treatment).
In medicine once upon a time it was the case that some treatment was found to “work” but it was not known why. And by work we mean: better than no intervention at all. But these days in medicine that is a rare exception to the general rule: there are entire fields of science from pharmacology to physiology, neurology and all kinds of carefully practised and error corrected surgical interventions and even where some medicine is used “off label” as we say there is often a good explanation known by at least some members of the medical community even if it has not been published. One reason for this is that, as it turns out, doctors are very busy people and getting around to going through the relatively recent phenomenon that is the “peer review process” is rather more difficult than word of mouth or these days a more informal review process. Peer review of double blind placebo controlled trials is indeed a gold standard of a kind - but it is also the most lengthy and difficult way of showing the efficacy of what might already be known to be a good explanation of efficacy. To be “known” in this sense has multiple meanings: someone might already know of a treatment and a good explanation for it, before the rest of the scientific community does. Take for example the famous case of - let me plug Australia here - the Australian doctor Barry Marshall who had an idea that stomach ulcers were caused by a bacteria and therefore could be cured by antibiotics. Until Barry Marshall many people thought stomach ulcers (indeed today many people still think they) are caused by stress, anxiety or something like that.
Anyway so the story goes - true story - Barry Marshall swallowed a nasty mix of ulcer causing bacteria. A good Aussie risk taker. The crocodile hunter of medicine. Anyways - yes - he got stomach ulcers. Now at that point - in pain and no doubt writhing in agony in bed waiting for the penicillin or whatnot to kick in and cure him - he knew. And his colleagues knew. They really knew the good explanation. But it was not yet double blind placebo controlled and reported in peer reviewed journals. So the community did not know. But he knew. And he would have been remiss had he seen a patient the following week and used antibiotics off label for anyone who came to see him and who he diagnosed as having stomach ulcers. All he could say was “I know this works. I did it to myself. Stomach ulcers are caused by bacteria and I know how and why regardless of what the literature does or does not say.”
So much is true of any scientific discovery yet to be published. Of course errors can be made and corrected - but here is the point - until such time as an error is found that IS the best explanation going. An astrophysicist who detects a wobble in a star 200 light years away over the course of 3 years of their PhD and does the appropriate calculations knows long before the publication of their thesis much less the publication in a journal somewhere much less before any other scientist even bothers (if they ever do) to check the results…that they have found a planet. That is the other thing about some scientific journal articles - some are barely ever read. And if they are only a small portion of those are ever checked carefully - which is to say the experimental results where they exist are checked - at least in a timely fashion. So some individuals or small groups of people can indeed have good explanations when others do not. The criticism has happened not in the way we are supposed to believe it can only happen in science - peer review- but rather validly and reliably in the mind of a single person. Or a small group of people.
And criticism in science is indeed the way we make progress. It is the means by which we reject false theories. And bad explanations. And no-explanations. This criticism - this critical attitude - is cached out through experiments. But not only experiments as I explain above: it can be knock-down, full stop, over-and-out refutation even without an experiment.
And this is true in all subjects, in all spheres of knowledge, always. So how does it work elsewhere?
Part 3