Representativeness of Participants in a Clinical Trial for Attention-Deficit/Hyperactivity Disorder? Comparison With Adults From a Large Observational Study
Craig B. H. Surman, MD; Michael C. Monuteaux, ScD; Carter R. Petty, MS; Stephen V. Faraone, PhD; Thomas J. Spencer, MD; Nicole F. Chu, BA; and Joseph Biederman, MD
Background: Clinical trials have demonstrated that pharmacotherapies can safely treat attention-deficit/hyperactivity disorder (ADHD) in adulthood. Eligibility criteria in these trials may significantly limit their external validity by excluding a significant portion of adults with ADHD in the general population. In particular, exclusion criteria may frequently exclude individuals with comorbid mental health conditions, which are common in the adult ADHD population.
Method: We addressed the representativeness of clinical trials by comparing 146 adult clinical trial participants with DSM-IV ADHD and a community sample composed of 124 adults with DSM-IV ADHD and 123 non-ADHD controls. Subjects were compared on socioeconomic status, Hollingshead occupational code, cognitive measures, lifetime psychopathology, and Global Assessment of Functioning (GAF) scale ratings.
Results: Adults with ADHD in the community sample had higher rates of lifetime psychiatric comorbidity, lower GAF scores, and lower occupational codes than those in the clinical trial. The clinical trial eligibility criteria would have excluded 61% of community sample adults with ADHD. This excluded portion of the community sample had higher rates of lifetime psychiatric comorbidity and lower GAF scores than clinical trial participants.
Conclusions: Adults with ADHD participating in the clinical trial had less evidence of functional impairment and endorsed less psychiatric comorbidity than the majority of community sample subjects with ADHD. This suggests that findings from clinical trials may have limited external validity for adults with ADHD in the general population, particularly for those adults with ADHD with the greatest burden of comorbid psychopathology.
J Clin Psychiatry 2010;71(12):1612-1616
© Copyright 2010 Physicians Postgraduate Press, Inc.
Submitted: May 08, 2009; accepted July 14, 2009.
Online ahead of print: August 10, 2010 (doi:10.4088/JCP.09m05344pur).
Corresponding author: Craig B. H. Surman, MD, Clinical and Research Programs in Pediatric Psychopharmacology and Adult ADHD, Massachusetts General Hospital, Ste 2000, 185 Alewife Brook Pkwy, Cambridge, MA 02138 ([email protected]).
In recent years, several clinical trials for adults with attention-deficit/hyperactivity disorder (ADHD) have contributed to the advancement of safe and effective treatment for this disorder. However, because clinical trials have inclusion and exclusion criteria, it is possible that subjects participating in these clinical trials are not representative of the larger population of adults with ADHD in the community. Of particular concern is the routine exclusion of subjects with comorbid psychiatric disorders from clinical trials of treatments for ADHD in adults, which are common in adults with ADHD in the general population.
Whether adults with ADHD participating in clinical trials are representative of the greater population of individuals with the disorder has implications for the external validity of these studies. If subjects with ADHD participating in clinical trials differ from subjects with ADHD in the general population, evidence from clinical trials may not provide accurate guidance for practitioners treating adults with ADHD that would have been excluded from clinical trials.
The main aim of the present study was to investigate whether adults with ADHD participating in clinical trials are representative of adults with ADHD in the community. To this end, we systematically identified and compared clinical correlates of adults with ADHD participating in a clinical trial and subjects with ADHD identified in a large community observational study. We hypothesized that subjects ascertained from the community will manifest higher levels of psychosocial impairments and psychiatric comorbidity than those participating in a clinical trial.
METHOD
The clinical trial participants consisted of 146 adults with ADHD between 19 and 60 years of age recruited for participation in a double-blind, randomized, placebo-controlled, 6-week trial of methylphenidate treatment ascertained from clinical referrals and advertisements in the greater Boston area.1 Subjects had to satisfy full diagnostic criteria for Diagnostic and Statistical Manual of Mental Disorders, Fourth Edition (DSM-IV) ADHD on the basis of clinical assessment and confirmed by structured diagnostic interview.2 We excluded potential subjects if they had clinically significant chronic medical conditions; abnormal baseline laboratory values; an Intelligence Quotient (IQ) score less than 80; delirium, dementia, or amnestic disorders; a current impairing psychiatric condition other than ADHD (eg, current major depression, panic disorder, or generalized anxiety disorder); history of a clinically unstable psychiatric condition (eg, history of bipolar disorder or psychosis, suicidality); drug or alcohol abuse or dependence within the 6 months preceding the study; or a previous adequate trial of stimulant treatment (> 0.5 mg/kg/d of methylphenidate or equivalent). Current use of psychotropics, other than stable treatment with selective serotonin reuptake inhibitors, was also exclusionary. We also excluded pregnant or nursing women from participation.
The community sample consisted of 124 subjects ascertained as part of a naturalistic study of ADHD in adults between the ages of 18 and 55 years.3 These subjects were ascertained from referrals to the Massachusetts General Hospital and advertisements in the greater Boston area. All subjects met full DSM-IV criteria for ADHD on the structured diagnostic interview,2 with onset of symptoms by age 7 years. Subjects were excluded if they had major sensorimotor handicaps (eg, deafness, blindness) or psychosis. Non-ADHD controls were recruited through advertisements in the greater Boston area.
Both studies excluded subjects with an inadequate command of the English language or an IQ score less than 80, as measured by an IQ estimate from the Block Design and Vocabulary subtests of the Wechsler Adult Intelligence Scale-Revised.4 No ethnic or racial group was excluded in these studies. These studies were approved by the institutional review board, and all subjects completed a written informed consent process before inclusion in the study.
Assessment Measures
Subjects in both studies were assessed with the Structured Clinical Interview for DSM-IV Axis I Disorders5 supplemented with modules from the Kiddie Schedule for Affective Disorders and Schizophrenia for School-Age Children-Epidemiologic Version6 to assess ADHD and other childhood disorders. We conducted direct interviews with subjects. We considered a disorder positive if DSM-IV diagnostic criteria were unequivocally met. Although standardized algorithms were used to determine each diagnosis, interviewers conducting the assessment needed a mechanism to determine the clinical relevance of symptoms when subjects were only able to provide unclear or imprecise information. Thus, a committee of board-certified child and adult psychiatrists, who were blind to the subject’s ADHD status, referral source, and all other data, resolved diagnostic uncertainties. Diagnoses presented for review were considered positive only when the committee determined that diagnostic criteria were met to a clinically meaningful degree. We estimated the reliability of the diagnostic review process by computing κ coefficients of agreement for clinician reviewers. For these diagnoses, the median reliability between individual clinician- and the review committee-assigned diagnoses was 0.87. The κ coefficients for individual diagnoses included the following: ADHD (1.0), conduct disorder (1.0), major depression (1.0), bipolar disorder (0.78), separation anxiety (0.89), agoraphobia (0.80), panic disorder (0.77), substance use disorder (1.0), and tics/Tourette’s disorder (0.68).
The interviewers were blind to the subject’s ascertainment group and all prior assessments. The interviewers had undergraduate degrees in psychology and were extensively trained. First, they underwent several weeks of classroom-style training, learning interview mechanics, diagnostic criteria, and coding algorithms. Then, they observed interviews by experienced raters and clinicians. They subsequently conducted at least 6 practice (nonstudy) interviews and at least 3 study interviews while being observed by senior interviewers. Trainees were not permitted to conduct interviews independently until they executed at least 3 interviews that achieved perfect diagnostic agreement with an observing senior interviewer. J.B. supervised the interviewers throughout the study. We computed κ coefficients of agreement by having experienced, board-certified child and adult psychiatrists and licensed clinical psychologists diagnose subjects from audiotaped interviews. Based on 500 assessments from interviews of children and adults, the median κ coefficient was 0.98. The κ coefficients for individual diagnoses included the following: ADHD (0.88), conduct disorder (1.0), major depression (1.0), mania (0.95), separation anxiety (1.0), agoraphobia (1.0), panic disorder (0.95), substance use disorder (1.0), and tics/Tourette’s disorder (0.89).
Interviewers assessed the degree of impairment on daily functioning associated with each disorder that subjects endorsed on a 3-level ordinal scale: minimal (ie, little to no impairment), moderate (ie, difficulties in daily life tasks), or severe (ie, unable to perform essential daily tasks). We analyzed major depression only if the depressive episode was associated with severe impairment, in order to avoid false-positive diagnoses. This is consistent with prior research methods.7,8
Socioeconomic status (SES) was measured by the Hollingshead measures, in which occupational and educational statuses are coded between 1 and 7, with higher numbers reflecting lower social status, and these items are weighted to produce a total SES score, with low values indicating high SES.9 As a measure of overall functioning, we used current and past lowest Global Assessment of Functioning (GAF) scale scores.
Statistical Analysis
Comparisons were made between subjects with ADHD from the clinical trial (clinical trial ADHD sample, n = 146), subjects with ADHD from the naturalistic study (community sample ADHD, n = 124), and non-ADHD controls from the naturalistic study (community sample non-ADHD, n = 123). Age and sex were compared between the groups to allow significant differences to be controlled for in all subsequent analyses. Comparisons were also made between the clinical trial ADHD group, subjects with ADHD from the naturalistic study who would have met inclusion criteria for the clinical trial (screened in, n = 48), and subjects with ADHD from the naturalistic study who would have been excluded from the clinical trial (screened out, n = 76). Linear regression was used to analyze continuous variables (eg, full-scale IQ), logistic regression was used for binary variables (eg, individual diagnoses), negative binomial regression was used for count variables (eg, number of comorbidities), and ordered logistic regression was used for ordinal variables (eg, SES).
We also compared clinical trial subjects with community sample subjects who were likely to have met the inclusion and exclusion criteria for the clinical trial. We categorized individuals from the community sample as "screened out" if they met criteria for current subthreshold or full psychosis, severely impairing major depression, generalized anxiety, panic disorder, agoraphobia, anorexia, bulimia, alcohol or drug abuse or dependence, or bipolar disorder. Individuals were also excluded if they met criteria for any alcohol or drug abuse or dependence in the past 2 years.
RESULTS
Sociodemographic Characteristics of the Sample
Clinical trial subjects and community sample ADHD subjects were both significantly more likely to be older than the community sample non-ADHD group (Tables 1 and 2). Thus, all subsequent analyses controlled for these variables.
Comparison of Naturalistic Study Participants and Subjects From the Clinical Trial
As shown in Table 2, with the exception of nicotine dependence, rates of all other lifetime psychiatric comorbid disorders assessed were significantly higher in the community sample ADHD group than the clinical trial group. Individuals from the community sample also had a significantly higher mean number of comorbidities than individuals with ADHD from the clinical trial sample.
In addition, relative to the community sample non-ADHD group, the community sample ADHD group had significantly higher rates of all studied comorbidities, whereas the clinical trial sample only had significantly higher rates of bipolar disorder, alcohol dependence, nicotine dependence, and antisocial personality disorder.
Current and past GAF scores were significantly more impaired in individuals with ADHD from the community sample than in those from the clinical trial sample. Individuals from the community sample had significantly lower occupation (lower Hollingshead occupation code) and lower socioeconomic status (higher Hollingshead total SES score) than clinical trial subjects. There were no significant differences identified between the community sample and clinical trial ADHD groups on IQ, Wide Range Achievement Test (WRAT) arithmetic, or WRAT reading scores.
Subjects with ADHD from both groups had significantly more impaired WRAT arithmetic, past GAF, and current GAF scores, as well as lower educational attainment (lower Hollingshead education code) than community sample non-ADHD individuals. Although occupational and socioeconomic status was lower for subjects with ADHD from the community sample than in the non-ADHD community sample (as reflected in higher Hollingshead total SES score and lower Hollingshead occupation code), there was no significant difference in the Hollingshead socioeconomic or occupation codes between the clinical trial group and the non-ADHD community sample.
Comparison of Community Participants With ADHD Who Would and Would Not Have Been Eligible for the Clinical Trial
As shown in Table 3, 61% of the ADHD community sample subjects fell into the screened-out category because they were unlikely to have met inclusion and exclusion criteria for the clinical trial at their time of assessment. These screened-out subjects had significantly higher rates of all assessed lifetime psychopathologies than both the screened-in subjects who would have been eligible for the clinical trial and the clinical trial participants. Major depression was the only lifetime psychopathology that was significantly more common among clinical trial participants than subjects who screened in from the community. As detailed in Table 3, screened-out subjects from the community sample had significantly more impaired mean current and past GAF scores than the screened-in subjects and clinical trial subjects. In contrast, there was no significant difference between the mean current and past GAF scores of the community sample screened-in group and that of the clinical trial sample. Hollingshead socioeconomic status was significantly higher in both the clinical trial and screened-in samples than the screened-out sample (as reflected in lower Hollingshead total SES score). The clinical trial sample had lower average occupational attainment than in the screened-out community samples (as reflected in higher Hollingshead occupation code).
DISCUSSION
This study evaluated whether adults with ADHD participating in clinical trials are representative of other adults with ADHD in the community. Measures of psychopathology and psychosocial functioning revealed that adults with ADHD in a clinical trial differed significantly from adults with ADHD in a community sample. Furthermore, a majority of the adults from our community sample would not have been eligible for the clinical trial. These findings suggest that results from clinical trials for adults with ADHD may not generalize to subjects with ADHD in the community.
Our analysis revealed that adults with ADHD who were recruited for a clinical trial had a significantly lower burden of all studied lifetime psychiatric comorbidities, except nicotine dependence, compared to other adults with ADHD in the community. They also had significantly better current and lifetime GAF scores and a significantly higher occupation level. Differences between the clinical trial and community sample ADHD subjects are further emphasized by our finding that community ADHD subjects, but not clinical trial subjects, had higher rates of major depression, multiple anxiety disorders, and substance dependence and lower occupational level and socioeconomic status than non-ADHD community sample subjects.
Our analysis also showed that 61% of ADHD individuals in the community sample would not have been eligible to participate in the clinical trial. Compared to eligible subjects, ineligible individuals had a higher burden of all psychopathologies studied, more impaired current and past global functioning, and lower socioeconomic status. Clinical trial subjects did not differ from the trial-eligible community subjects on any of the clinical or functional domains studied except on lifetime presence of major depression, which was lower in the clinical trial subjects. Relative to community ADHD subjects ineligible for the trial, clinical trial subjects had significantly lower rates of all lifetime psychopathologies, had higher current and past global functioning mean scores, and had higher occupational and socioeconomic statuses.
The pattern of psychopathological and functional impairments in adults with ADHD from our community sample is representative of impairments demonstrated in both epidemiologic and clinical studies of adults with ADHD. For example, the National Comorbidity Survey Replication, an epidemiologic assessment of United States adults between the ages of 18 and 44 years, found that adults with ADHD are at higher risk than non-ADHD subjects to have experienced another mental health condition in the past year with odds ratios of 2.7-7.5 for mood disorders, 1.5-5.5 for anxiety disorders, and 1.5-7.9 for substance use disorders.10 Studies of comorbidity in clinical samples of adults with ADHD have similarly found higher rates of psychopathology when compared to samples of adults without ADHD.11-13 Studies have also clearly demonstrated a significant impact of ADHD on occupational, educational, and relationship roles.2,10,14,15 Taken together, this evidence supports our conclusion that clinical trials have limited representativeness of community subjects with ADHD and that the findings reported in clinical trials may not generalize to community samples. It is possible that different estimates of efficacy and tolerability would be found if all subjects with ADHD were to be included in clinical trials. Expansion of eligibility for clinical trials in adult ADHD may extend evidence-based interventions for this population and may provide more externally valid information on how to best treat this disorder in the community.
Our findings need to be viewed in light of some methodological limitations. We did not screen our community sample for clinically significant medical comorbidity, which excluded participants from the clinical trial. Our assessment of representativeness was limited to the dimension of mental health. More comprehensive assessment of representativeness of clinical trial participants relative to community subjects would have identified medical conditions and treatments that are often exclusionary for trial participation. This would typically include conditions for which sympathomimetics are contraindicated, such as hypertension and hyperthyroidism, and medications that impair cognitive function, such as sedatives. Our report also does not assess the extent to which the clinical trial would have excluded clinically significant presentations that do not meet criteria as Axis I conditions, such as suicidality or severe personality disorders. In addition, since the vast majority of participants in our studies were white, our findings may not generalize to other racial groups. Despite these considerations, we have demonstrated that a typical clinical trial in ADHD adults is limited in its validity for the larger community of adults with ADHD.
Our investigation suggests that typical eligibility criteria in clinical trials for adults with ADHD actually exclude the large subpopulation of adults with ADHD that carries the greatest comorbidity burden. The higher rate of psychopathology in these excluded individuals also implies that clinical trials for adults with ADHD are not including adults with ADHD who have the greatest need for treatment. Intervention trials with less restrictive inclusion criteria could provide more externally valid information and could extend the ability of practitioners to effectively manage ADHD in the general population. This could be accomplished through clinical trials that included individuals with the most common psychiatric and medical comorbidities complicating ADHD, such as mood and substance use disorders, and common stabilized conditions that may be exacerbated by sympathomimetic ADHD agents, such as hypertension.
Author affiliations: Clinical and Research Program in Pediatric Psychopharmacology and Adult ADHD, Massachusetts General Hospital, Yawkey Center for Outpatient Care, Boston (Drs Surman, Monuteaux, Spencer, and Biederman; Mr Petty; and Ms Chu); and Department of Psychiatry and Department of Neuroscience and Physiology, SUNY Upstate Medical University, Syracuse, New York (Dr Faraone).
Potential conflicts of interests: Dr Surman has received research support from Abbott, Alza, Cephalon, Eli Lilly, ElMindA, the Hilda and Preston Davis Foundation, McNeil, Merck, New River, National Institutes of Health (NIH), Organon, Pfizer, Shire, and Takeda; has been a speaker for Janssen-Ortho, McNeil, Novartis, Shire, and Massachusetts General Hospital Academy/Reed Medical Education (which receives funding from multiple pharmaceutical companies); and has been a consultant/advisor for McNeil, Shire, and Takeda. Dr Faraone has, in the past year, received consulting fees and has been on advisory boards for Eli Lilly, Ortho-McNeil, and Shire; has received research support from Shire and the NIH; has, in previous years, received consulting fees from, has been on advisory boards for, or has been a speaker for Shire, McNeil, Janssen, Novartis, Pfizer, Ortho-McNeil, and Eli Lilly; and has, in previous years, received research support from Eli Lilly, Shire, Pfizer, and the NIH. Dr Spencer has received research support from, has been a speaker for, has been a member of the speaker’s bureau for, has been an advisor for, or has been a member of the advisory board for Shire, Eli Lilly, GlaxoSmithKline, Janssen, McNeil, Novartis, Cephalon, Pfizer, and the National Institute of Mental Health (NIMH). Dr Biederman is currently receiving research support from Alza, AstraZeneca, Bristol-Myers Squibb, Eli Lilly, Janssen, McNeil, Merck, Organon, Otsuka, Shire, NIMH, and National Institute of Child Health and Human Development; received, in 2009, a speaker’s fee from Fundacion Areces, Medice Pharmaceuticals, and the Spanish Child Psychiatry Association; and has, in previous years, received research support, consultation fees, or speaker’s fees for/from Abbott, AstraZeneca, Celltech, Cephalon, Eli Lilly, Esai, Forest, GlaxoSmithKline, Gliatech, Janssen, McNeil, National Alliance for Research on Schizophrenia and Depression, National Institute on Drug Abuse, New River, Novartis, Noven, Neurosearch, Pfizer, Pharmacia, the Prechter Foundation, Shire, the Stanley Foundation, UCB Pharma, and Wyeth. Dr Monuteaux, Mr Petty, and Ms Chu report no conflicting interests.
Funding/support: Data analyzed in this work come from studies supported by grants from the NIH to Dr Faraone (R01MH57934) and to Dr Spencer (R29MH57511). Novartis supported a portion of the cost of active medication in the latter study. There were no other sponsors of this study.
Previous presentation: Presented as a poster at the 162nd Annual Meeting of the American Psychiatric Association; May 16-21, 2009; San Francisco, California.
REFERENCES
1. Spencer T, Biederman J, Wilens T, et al. A large, double-blind, randomized clinical trial of methylphenidate in the treatment of adults with attention-deficit/hyperactivity disorder. Biol Psychiatry. 2005;57(5):456-463. PubMed doi:10.1016/j.biopsych.2004.11.043
2. Biederman J, Faraone SV, Spencer T, et al. Patterns of psychiatric comorbidity, cognition, and psychosocial functioning in adults with attention deficit hyperactivity disorder. Am J Psychiatry. 1993;150(12):1792-1798. PubMed
3. Faraone SV, Biederman J, Spencer T, et al. Diagnosing adult attention deficit hyperactivity disorder: are late onset and subthreshold diagnoses valid? Am J Psychiatry. 2006;163(10):1720-1729, quiz 1859. PubMed doi:10.1176/appi.ajp.163.10.1720
4. Wechsler D. Manual for the Wechsler Adult Intelligence Scale—Revised. Wechsler Adult Intelligence Scale III. San Antonio, TX: The Psychological Corporation; 1981.
5. First M, Spitzer R, Gibbon M, et al. Structured Clinical Interview for DSM-IV Axis I Disorders. Washington, DC: American Psychiatric Press; 1997.
6. Orvaschel H. Schedule for Affective Disorders and Schizophrenia for School-Age Children Epidemiologic Version. Ft Lauderdale, FL: Nova Southeastern University, Center for Psychological Studies; 1994.
7. Gershon ES, Hamovit J, Guroff JJ, et al. A family study of schizoaffective, bipolar I, bipolar II, unipolar, and normal control probands. Arch Gen Psychiatry. 1982;39(10):1157-1167. PubMed
8. Leckman JF, Sholomskas D, Thompson WD, et al. Best estimate of lifetime psychiatric diagnosis: a methodological study. Arch Gen Psychiatry. 1982;39(8):879-883. PubMed
9. Hollingshead AB. Four Factor Index of Social Status. New Haven, CT: Yale University Press; 1975.
10. Kessler RC, Adler L, Barkley R, et al. The prevalence and correlates of adult ADHD in the United States: results from the National Comorbidity Survey Replication. Am J Psychiatry. 2006;163(4):716-723. PubMed doi:10.1176/appi.ajp.163.4.716
11. Biederman J, Faraone SV, Spencer T, et al. Gender differences in a sample of adults with attention deficit hyperactivity disorder. Psychiatry Res. 1994;53(1):13-29. PubMed doi:10.1016/0165-1781(94)90092-2
12. Biederman J, Faraone SV, Monuteaux MC, et al. Gender effects on attention-deficit/hyperactivity disorder in adults, revisited. Biol Psychiatry. 2004;55(7):692-700. PubMed doi:10.1016/j.biopsych.2003.12.003
13. McGough JJ, Smalley SL, McCracken JT, et al. Psychiatric comorbidity in adult attention deficit hyperactivity disorder: findings from multiplex families. Am J Psychiatry. 2005;162(9):1621-1627. PubMed doi:10.1176/appi.ajp.162.9.1621
14. Mannuzza S, Klein RG, Bessler A, et al. Adult outcome of hyperactive boys: educational achievement, occupational rank, and psychiatric status. Arch Gen Psychiatry. 1993;50(7):565-576. PubMed
15. Murphy K, Barkley RA. Attention deficit hyperactivity disorder adults: comorbidities and adaptive impairments. Compr Psychiatry. 1996;37(6):393-401. PubMed doi:10.1016/S0010-440X(96)90022-X
