Let me start out by saying that I deeply value and respect rational thinking. I think that rational thought is a valuable foundation for decision-making and I value the sensibility that it embodies. However, I am not entirely enamored by claims of "rational thought" and here are a few reasons why.
Rational thinking is only half the story. I have found that if a person has a strong emotional stake in an outcome, he or she usually constructs arguments to support that outcome. In the scientific literature, this is called "motivated reasoning" and a brain imaging study has shown that it activates very different brain regions from "cold reasoning". In many matters of life, motivated reasoning masquerades as cold reasoning and "rational" thought, when in fact, it is reason that is based on an emotional response. Even if people say something with a straight face and blinking eyes, this form of rational utterance often has an emotional basis. When people have an emotional stake in an outcome, I almost never consider their arguments to be "purely rational".
The brain imaging study that showed different brain activation for motivated reasoning as opposed to regions previously linked to cold reasoning, also showed that implicit and not explicit emotional centers contributed to this motivated reasoning. That is, we are not usually aware of how emotions shape our decisions, but they do. On the surface, we may be convinced that we have removed all emotion from a decision, but we cannot exclude the possibility that our unconscious emotional stake has influenced our decision. I do not believe that this is anything to be ashamed of either. Our emotions carry important information that is often critical for decision-making. Unconscious fear, for example, may help us avoid dangerous situations.
Another reason that I think that "surface rationality" is questionable is that we often make decisions based on how options are presented to us. This has been called the "framing effect". In fact, the brain can try to overcome this effect, but overcoming this often requires more time than we have. Often, past experiences strongly influence how we respond "rationally" and may also influence how we take information in, even if we develop a rational framework to explain our thought processes. Essentially, emotions and world-views powerfully affect the way in which we construct arguments. Ask a Democrat to come up with reasons for why taxes should not be cut for small business owners, or ask a homophobe to provide arguments for why gay marriage should be allowed, and you will see how difficult it is for people to think outside of their identified allegiances.
Rational thinking also often rests on "believing" what the brain sees, but there are countless examples of how our brains can trick us into thinking things. We see mirages in deserts even when there is no water. If we bring two horizontal lines in the same plane close enough together, our brains will see them and report them to us as one. Amputees can feel pain in a limb that is not present. We cannot hear dog whistles. Our "convictions" rely heavily on our senses, but our senses do not always tell us the truth. Building a rational argument based on what can be seen or heard or touched has its limitations.
Another reason that I think that "rational" thinking is not what it might appear to be is that our brains are limited in the amount of information they can take in at any one point in time. Thus, while we may feel as though we are being thorough in the way we think, we usually filter out many things so that the brain is not overwhelmed. This filtering sometimes leaves out critical aspects of rational thinking and we are far less comprehensive than we might think we are.
I once saw a man in my practice because he thought that he was being followed by the "Hells Angels". He was otherwise a high functioning person who ran a successful business. His "evidence" for this was that he had been to a "Grateful Dead" show and that people looked at him strangely. He also noticed bikers on the highways on several occasions. And when he walked into a hardware store, a man in a leather biker jacket bumped into him. All of these points of evidence were probably true, but his conclusion turned out to be false. It took him a long time to let go of these associations. While his delusions are extreme, our brains are making associations like this all the time. We remember things incorrectly or forget things often as well. Our brains may make up stories that join time points to create a sense of continuity, much like how they can make two horizontal lines look like one when they are close enough.
Rational thinking may therefore not be as "rational" as it seems. Perhaps we need to learn to accept and be more open about how our emotions influence the ways in which we think, since that is the reality anyway?