How "Mental Disorders" are Diagnosed

For decades, many researchers have been trying to find biological markers that could be linked to particular mental disorder diagnoses, hoping that such discoveries could lead to better drug treatments. So we often hear of new evidence about biochemical imbalances of serotonin or dopamine, genetic roots of mental disorders, brain scans revealing types of schizophrenia, blood analyses for detecting depression, and various other biological tests for mental disorders. Though often trumpeted in the media, none of these experimental findings has ever been scientifically validated. There are currently no genetic, biological, chemical or other physical tests of any kind that can determine the presence or absence of any mental disorders.

For many of us this can be difficult to accept or believe because we have heard otherwise so frequently. One factor that sows confusion is the way in which evidence of physical effects is often misrepresented or misconstrued as evidence of disease pathology and causes of pathology. It’s important to understand that, for example, while it’s certainly possible to detect physical effects of anxious feelings such as accelerated breathing, rising heart rate, or increased blood flow in parts of a person’s brain, those observations do not tell us anything about what is causing the person’s anxiety. Furthermore, being able to observe those physical changes does not prove that the person “has” a “disease” called “Generalized Anxiety Disorder”.

Another cause of public misunderstanding is the fact that the treatment of mental disorders as if they were well-understood, brain-based, biochemical illnesses has become a large and lucrative industry that spends, conservatively, tens of billions of dollars annually in promotional, “educational” and public relations activities. (For more information, please read ICI’s “How Psychiatric Drugs are Researched and Marketed”.) As part of these promotional efforts, leading proponents and experts for this industry are often reluctant to publicly clarify that there continue to be no known biological causes of mental disorders or biological ways of detecting mental disorders. It was, for example, only amid a groundswell of high-profile criticism and questioning in 2013 surrounding the release of the fifth edition of the Diagnostic and Statistical Manual of Mental Disorders that the American Psychiatric Association issued a news release clarifying for news reporters and the general public that, indeed, there are no known biological markers for any mental disorders.

Short of reading extensively in the scientific literature oneself, then, the next best evidence for the fact that there are no known biological markers for any mental disorders is to observe how mental disorders are actually diagnosed in everyday clinical practice. As we’ll explore in more detail below, mental disorders are not diagnosed with any laboratory tests, but are diagnosed primarily through simply listening to what a person says and observing the person’s behaviors.

Since there are no biological tests to establish the presence or absence of any mental disorders, a mental disorder diagnosis is more appropriately understood as a label than as a diagnosis.

It’s understandable that many of us have become confused about this, because it is common for medically-trained psychiatrists and medical doctors to assign mental disorder labels, refer to them as “diagnoses,” and suggest “treatments” for them. But a good example of the difference is to imagine that you go to a doctor complaining about intense and persistent headaches. If biological tests identify that you have one of the known possible physical causes of severe headaches, then that will be your diagnosis: brain tumor, sinus pressure, or bacterial infection, for example. However, if biological tests fail to find any known cause, then the doctor might say that you have migraines. The word migraine literally means “intense and persistent headaches”. In that sense, migraines is not a biological diagnosis of the cause of your headaches, it is simply a commonly used label for describing those types of headaches.

It is exactly the same with psychiatric diagnoses and treatments. If for example you complain to your doctor of intense and persistent feelings of depression, then if no physical cause is found – such as for example a nutritional deficiency or cardiovascular illness – then your doctor may say that you are depressed or have Major Depressive Disorder. The term “Major Depressive Disorder” is a label for what you’re apparently experiencing, not a biological, medical diagnosis of the causes of your experiences. No biological tests will be done to confirm this diagnosis or to clarify the best treatments, because the doctors and lab technicians would not know what to specifically look for in your brain or body.

The American Psychiatric Association’s Diagnostic and Statistical Manual of Mental Disorders (DSM) is the primary guide for diagnosing mental disorders in the United States. Governments and the insurance industry use the DSM as a common standard for medical insurance and billing purposes, and this is part of what makes use of the DSM in the U.S. so widespread. Please read ICI’s “An Overview of Psychiatry’s Diagnostic Manual (the DSM)” for more information about the history, nature, development and uses of the DSM. But briefly, in the DSM, the labels for several hundred mental disorders are listed together with the experiences, feelings, thoughts or behaviors that the American Psychiatric Association’s DSM Task Force members decided to list as “symptoms” of those disorders. For example, in the DSM, the symptoms of Major Depressive Disorder include experiences such as fatigue, restlessness, insomnia, and despair.

The DSM does not include any attempts to explain or account for why some people might have particular experiences; it is simply a system for grouping and labeling various types of experiences. If a person exhibits or reports having thoughts, feelings, emotions or behaviors that appear to match a certain number of symptoms that are listed under the name of a particular mental disorder in the DSM, then that person can be labeled by a mental health practitioner as having that particular disorder. For instance, to be diagnosed as having Attention Deficit Hyperactivity Disorder (ADHD), the most recent edition of the DSM states that a child must exhibit or report at least 6 of 9 listed behaviors, such as often talking excessively, often not engaging in play quietly, often having difficulty waiting in line, or often interrupting others. Generally, for a diagnostic label to be applied, a person must also report experiencing – or be judged by the practitioner to be experiencing – some level or type of impairment of his or her daily functioning, such as being unable to work as efficiently as normally.

Throughout the DSM, many of the key terms like “often”, “excessive” and “impairment” are not clearly defined. Consequently, the assigning of diagnostic labels is always strongly influenced by a practitioner’s subjective opinion. A practitioner’s interpretations of a patient’s experiences and behaviors – and the diagnostic label the practitioner chooses to give to the patient – is often influenced by biases like the practitioner’s own age, gender, race, ethnicity, sexuality, religious or anti-religious beliefs, cultural background, past experiences, or financial circumstances.

Many people’s first experiences with the mental health diagnostic labeling process involve screening tests. Mental health screening tests typically include about 5-20 questions with multiple-choice answers. They are available widely online and often administered during mental health awareness-raising programs and at schools, drug stores, doctors’ offices, and in general and psychiatric hospitals. They are also often used in clinical research, and in telephone surveys to try to determine the prevalence of mental disorders in the general population, such as is done by the Substance Abuse and Mental Health Services Administration and U.S. National Survey on Drug Use.

Mental health screening tests are typically developed by individuals or small groups of mental health researcher-practitioners, often while being funded by pharmaceutical companies. One of the most common screening tests is the Patient Health Questionnaire-9 (PHQ-9), which is actually copywrited by a drug company. Other common screening tests include the Generalized Anxiety Disorder-7 (GAD-7), and World Mental Health Composite International Diagnostic Interview (WMH-CIDI).

Usually, people complete the questions by giving responses about themselves, but some screening tests are designed to be answered by a parent, family member, school teacher, doctor, or mental health practitioner about someone else. The questionnaires ask about emotions, thoughts, feelings, moods, beliefs and behaviors. The questions are tailored to help match people’s answers to the symptoms of mental disorders that are listed in the Diagnostic and Statistical Manual of Mental Disorders. Numbered scores are attributed to each multiple-choice answer in the questionnaire, and the total score is used to label a person as being at low, medium or high risk of suffering from one or more mental disorders. Usually, the higher the score on the test, the higher the person’s risk is alleged to be. Some tests try to screen for a variety of different mental disorders, and other times a single questionnaire focuses on experiences related to a single diagnostic category such as anxiety, depression or schizophrenia.

Generally, the results of mental health questionnaires are not considered to be diagnoses. Instead, it is more commonly claimed that higher scores on such questionnaires are “warnings” that people should see a mental health practitioner for further assessment. Nevertheless, most practitioners use these same screening tests and diagnostic questionnaires – or ask patients similar questions based on them – to arrive at their diagnoses. And, as we will explore throughout this article, it is unclear what these kinds of questions are actually measuring.

One primary problem is that many key terms in these questionnaires tend to be conversational, loosely defined and unclear, and can be interpreted to mean widely different things. Consequently, it is uncertain what the tests are actually measuring or what the scores truly mean for any particular person at any particular time. For example, typical questions ask whether in the previous 2-4 weeks you never, occasionally, often, very often, or always “feel tired out for no good reason”, “worry too much about different things”, or “find it difficult to get a hold of your thoughts”. All of these kinds of questions require the people completing the tests to make their own personal, ad-hoc assessments of the meanings of key words and ideas, including pivotal issues relating to intensity, frequency, duration and context. For example:

• How intensely must you be experiencing a particular thought, feeling or mood before you answer on the test that you are experiencing it at levels that are “too much”? (intensity)
• How frequently must you experience a particular thought, feeling or mood to answer on the test that you feel it “sometimes” vs. “often” vs. “very often”? (frequency)
• For how many consecutive seconds, minutes or hours out of a day must you experience a particular thought, feeling or mood before you answer on the test that you experience the thought, feeling or mood for “one day a week” or “nearly every day”? (duration)
• When trying to answer, how do you definitively determine if you are feeling something “for no good reason”, or for a reason that’s good, or for a reason that you simply don’t know? Or how do you decide what “a hold” on your thoughts really looks like in practice? (context)

The questionnaires do not come with instructions about how to make any of these kinds of decisions before you answer. People completing the questionnaires must answer the questions based on their own beliefs and opinions about what the words are intended to mean. Consequently, in the end, people who complete mental health screening tests or diagnostic questionnaires are usually essentially rendering a personal opinion about their own mental, cognitive, physical and emotional experiences. If you answer the questions in a way that indicates that you yourself believe that you feel too depressed, too anxious, or too out of control of unusual thoughts too much of the time, the tests will typically score you as being at high risk of having depressive, anxiety or psychotic disorders respectively. In short, when completing most mental health questionnaires, you are diagnostically labeling yourself.

Many proponents claim that mental health screening tests and diagnostic questionnaires have excellent reliability. This depends on what one interprets the word “excellent” to mean.

In medicine, the accuracy of screening tests and diagnostic questionnaires is usually measured by comparing their success rate to an established diagnostic tool that has nearly perfect accuracy. For example, the measures of how reliable mammography screening tests are for detecting cancerous tumors in breasts are calculated by comparing mammography findings to corresponding cancer biopsy diagnostic findings. For mental disorders, though, there is no diagnostic tool like a biopsy to use as a near-perfect standard. In fact, as discussed above, there are no objective, biological tests of any kind at all to determine the presence or absence of any mental disorder. This means that there is ultimately no way to mathematically measure the accuracy of any mental health screening test or diagnostic tool in the normal way that this is done in medical science.

So when mental health practitioners nevertheless claim that certain mental health screening questionnaires or diagnostic tools have “excellent” reliability, and provide numbered ratings to demonstrate it, what do they mean? Ratings for all mental health screening and diagnostic questionnaires are derived simply from comparing how well the results of a particular questionnaire match the results of other mental health screening tests or diagnostic questionnaires. In the psychiatric literature, they refer to this as “calibrating” one screening test or diagnostic tool with another, as opposed to objectively “validating” the accuracy of any of them. Often the ratings for tests and questionnaires are determined by measuring their results against the results of a Structured Clinical Interview for DSM Disorders (SCID); however, the SCID questionnaires are themselves essentially just much lengthier and more detailed versions of mental health screening tests.

This is explained succinctly in an essay by the developers of the World Health Organization’s international standard World Mental Health Composite International Diagnostic Interview (WMH-CIDI): “As no clinical gold standard assessment is available,” the authors wrote, “we adopted the goal of calibration rather than validation; that is, we asked whether WMH-CIDI diagnoses are 'consistent' with diagnoses based on a state-of-the-art clinical research diagnostic interview [the SCID], rather than whether they are 'correct'.”

A helpful way of understanding calibration versus validation is this: We could cut two pieces of wood out of a plank and use eyeball, saw and sandpaper to compare the two pieces to each other, and gradually calibrate them to being close to the same length. We could then use them to confidently and consistently determine whether a specific object was longer or shorter than each of our pieces of wood. But without reliable measuring tape to aid us at any point along the way, we still wouldn’t be able to use our pieces of wood to validate exactly how long any particular object actually was in inches or millimeters.  

This is similar to the situation with mental disorder screening tests and diagnostic tools: We can calibrate two different tests or diagnostic approaches so that they tend to assign the same diagnostic labels to the same people; however, these results still do not validly determine whether any of the labeled people do or do not in fact have any kind of mental disorder.

And as we’ll soon see, even the highest-rated mental health questionnaires do not agree with each other with very high levels of consistency, in any case.

Related reading:

Kessler, Ronald C., Jamie Abelson, Olga Demler, Javier I. Escobar, Miriam Gibbon, Margaret E. Guyer, Mary J. Howes, et al. “Clinical Calibration of DSM-IV Diagnoses in the World Mental Health (WMH) Version of the World Health Organization (WHO) Composite International Diagnostic Interview (WMHCIDI).” International Journal of Methods in Psychiatric Research 13, no. 2 (2004): 122–39.

As we discussed above, many people claim that mental health screening tests and diagnostic questionnaires have excellent reliability, but this depends on how we interpret the word “excellent”. In fact, mental health screening tests, diagnostic questionnaires, and psychiatric experts generally achieve very low levels of agreement with each other by normal medical standards. To illustrate this, it’s helpful to understand a little better exactly how these tests, tools and methods are rated.

Some of the highest-rated mental health screening tools are purported to have a sensitivity of 90% and specificity of 80%. In a mental health context, this means that the highest-rated tools tend to concur with each other about 90% of the time on which people should be labeled with a mental disorder, and concur about 80% of the time on which people should not be labeled with a mental disorder. Many people – including many improperly informed practitioners – casually assume that 90% sensitivity and 80% specificity mean that such a test will be wrong only about 10-20% of the time. But there’s a very important third factor in rating screening tests: The prevalence in the general population of what the test is screening for.

A helpful way to think about screening test effectiveness is to imagine searching for a needle in a haystack. Suppose you have a machine that can quickly distinguish needles from strands of hay with about 80-90% accuracy, and you know that there are just a few needles that you’re trying to find in a large haystack. How many stalks of hay will your machine wrongly identify as needles before you finally find all of the needles? The answer is: A lot of hay.

This online calculator for screening tools demonstrates how it works. Let’s say temporarily for the sake of discussion that 10% of Americans should be labeled as having a mental disorder. Out of 1,000 people, then, we know that 100 of them should be labeled with a mental disorder. Using a mental health diagnostic test with 90% sensitivity, we will correctly identify 90 of those 100 people. Meanwhile, the 80% specificity of our test means that, of the 900 people who should not be labeled with a mental disorder, we will wrongly label 180 of them as having a mental disorder. In total, then, our test will identify 270 people, or 27% of the population as having a mental disorder, even though we believed at the start that only 10% of the population should be labeled with a mental disorder. Using our own standards and assumptions, our test has wrongly identified as mentally disordered nearly twice as many people as it identified correctly.

This is what actually happens with real-world mental health screening tests and clinical diagnostic tools, and it’s why critics say that these tools inappropriately, unnecessarily or even potentially dangerously push large numbers of people towards mental health practitioners, diagnoses and treatments. Some people counter that this isn’t a problem, because “expert” mental health clinicians will be able to reliably sort out who truly should be labeled with a mental disorder and who shouldn’t. But in fact, as we’ll discuss below, even expert clinicians do not reliably agree with each other on diagnoses.

It’s common for many of us – including physicians and mental health practitioners – to feel confident that we simply “know”, by looking and listening, who is “crazy” and who is not, and what some people’s most obvious “psychological problems” are. Our opinions may sometimes be legitimate, or they may not be; however, to be called medical science, our evaluations must be based in more than these kinds of personal opinions, feelings, beliefs and prejudices. Furthermore, to say that someone seems to be experiencing a relatively unusual intensity of mental or emotional confusion or distress or is engaging in relatively unusual behaviors is very different than saying that the person “clearly has a clinical mental disorder”.

And in the vast majority of cases, it’s actually not very obvious. For example, during official field trials of the DSM-5 (the most recent, fifth edition of the Diagnostic and Statistical Manual of Mental Disorders), the American Psychiatric Association tested the reliability of 23 mental disorder diagnoses. In the tests, different psychiatrists interviewed and diagnosed the same people, and then calculations were done to determine how often the same diagnoses were reached by the different practitioners, or how reliably they agreed with each other. After the tests, the APA stated that their findings showed that absolutely none of the 23 disorders that they tested achieved “excellent” reliability. For five of the diagnoses, they stated, psychiatrists agreed with “very good” reliability and for nine they agreed with “good” reliability. On the other nine diagnostic categories psychiatrists reportedly had “questionable” or “unacceptable” levels of disagreement.

These results were even weaker than they may sound, though. This is because, since 1980, the APA and its DSM Task Forces have taken the position that the commonly agreed-upon standards in medicine to determine expert agreement reliability values are too ambitious for psychiatry. So they shifted their interpretations of their own findings upwards. That is to say, the APA described levels of agreement between psychiatrists as “good” and “very good” when by more common medical standards most of those levels of agreement would have been described as being “poor” or “moderate”. And it’s important to note that these field trials were conducted under highly biased and controlled conditions with a pre-selected patient population, where the psychiatrists knew they were being tested on their ability to agree with each other on a very small number of possible diagnoses. In real-world conditions, we would expect dramatically less agreement to occur. And indeed, it is not uncommon for people who have been involved in the mental health system for years with different practitioners to be given five or more different diagnoses and a dozen or more different treatments – even when practitioners are aware from patient medical records of each other’s diagnoses.

Related reading:

Kraemer, Helena Chmura, David J. Kupfer, Diana E. Clarke, William E. Narrow, and Darrel A. Regier. “DSM-5: How Reliable Is Reliable Enough?” American Journal of Psychiatry 169, no. 1 (January 1, 2012): 13–15. doi:10.1176/appi.ajp.2011.11010050.

Regier, D. A., Narrow, W. E., Clarke, D. E., Kraemer, H. C., Kuramoto, S. J., Kuhl, E. A., & Kupfer, D. J. (2013). "DSM-5 field trials in the United States and Canada, Part II: Test-retest reliability of selected categorical diagnoses." American Journal of Psychiatry, 170, 59-70.

There are differences in training between the various practitioners who can render a legally-recognized mental health diagnosis. Physicians and psychiatrists both receive a standard medical education, and psychiatrists receive some additional training specific to psychiatry. Licensed counselors, social workers, psychologists, and psychotherapists are not physicians and usually have no medical education, but they also often have legal authority to diagnose someone as having a mental disorder. 

In order to be reimbursed for their time, practitioners are expected by law, for insurance billing purposes, to label a person with one of the mental disorder labels listed in the Diagnostic and Statistical Manual of Mental Disorders. There are, however, no established legal or scientific standards for how practitioners can arrive at assigning these labels; the DSM simply provides voluntary guidelines for selecting them. Therefore, in common hospital, clinic and private psychiatric settings, physicians or mental health practitioners can in practice diagnose people and assign mental disorder labels through any methods and based on any observations that they deem to be relevant or appropriate.

Some practitioners may ask you to complete one or two psychological questionnaires or mental health screening tests, while others may simply talk with you for 5-10 minutes or longer before assigning a diagnostic label to you. Other practitioners may have weekly discussions with you over a period of time. Or they may order you to stay in a psychiatric hospital under general observation for a number of days and then reach their diagnosis after talking with you, hospital staff, and people who know you.

When a practitioner labels a person as having a mental disorder, this creates significant risks. Being told by a socially respected authority that your brain is “diseased” can be an insulting, humiliating or frightening experience – especially for a child or youth. The label can lead to a person becoming a victim of social discrimination and bigotry, and to having his or her basic civil rights more easily taken away under mental health laws.

In addition, in our culture today, people who receive a mental disorder label from their doctor or psychiatrist become much more likely to believe that their problems are rooted in genetic or biochemical causes beyond their control that will last for their entire lives – and that the best they can ever hope for is to be able learn to “manage” this “illness”. Such beliefs can create anxiety, despair, and hopelessness. These beliefs also make people more likely to feel that they have no choice but to turn to invasive biological interventions such as psychiatric drugs and electroconvulsive therapy (ECT),  despite the physical dangers.

Therefore, it is important to always be honest and forthright about what mental disorder labels actually are and are not. A mental disorder label is only ever a somewhat rough, subjective label for certain experiences and behaviors, and not a reliable, accurate, objective medical diagnosis of any known biological condition.