Reliability and Validity of the Mini International Neuropsychiatric Interview for Children and Adolescents (MINI-KID)
Objective: To investigate the concurrent validity and reliability of the Mini International Neuropsychiatric Interview for Children and Adolescents (MINI-KID), a short structured diagnostic interview for DSM-IV and ICD-10 psychiatric disorders in children and adolescents.
Method: Participants were 226 children and adolescents (190 outpatients and 36 controls) aged 6 to 17 years. To assess the concurrent validity of the MINI-KID, participants were administered the MINI-KID and the Schedule for Affective Disorders and Schizophrenia for School Aged Children-Present and Lifetime Version (K-SADS-PL) by blinded interviewers in a counterbalanced order on the same day. Participants also completed a self-rated measure of disability. In addition, interrater (n = 57) and test-retest (n = 83) reliability data (retest interval, 1-5 days) were collected, and agreement between the parent version of the MINI-KID and the standard MINI-KID (n = 140) was assessed. Data were collected between March 2004 and January 2008.
Results: Substantial to excellent MINI-KID to K-SADS-PL concordance was found for syndromal diagnoses of any mood disorder, any anxiety disorder, any substance use disorder, any ADHD or behavioral disorder, and any eating disorder (area under curve [AUC] = 0.81-0.96, κ = 0.56-0.87). Results were more variable for psychotic disorder (AUC = 0.94, κ = 0.41). Sensitivity was substantial (0.61-1.00) for 15/20 individual DSM-IV disorders. Specificity was excellent (0.81-1.00) for 18 disorders and substantial (> 0.73) for the remaining 2. The MINI-KID identified a median of 3 disorders per subject compared to 2 on the K-SADS-PL and took two-thirds less time to administer (34 vs 103 minutes). Interrater and test-retest kappas were substantial to almost perfect (0.64-1.00) for all individual MINI-KID disorders except dysthymia. Concordance of the parent version (MINI-KID-P) with the standard MINI-KID was good.
Conclusions: The MINI-KID generates reliable and valid psychiatric diagnoses for children and adolescents and does so in a third of the time as the K-SADS-PL.
Trial Registration: clinicaltrials.gov Identifier: NCT00579267
J Clin Psychiatry 2010;71(3):313-326
© Copyright 2010 Physicians Postgraduate Press, Inc.
Submitted: April 19, 2009; accepted July 23, 2009 (doi:10.4088/JCP.09m05305whi).
Corresponding author: David V. Sheehan, MD, MBA, Depression & Anxiety Disorders Research Institute, University of South Florida College of Medicine, 3515 East Fletcher Ave, Tampa, FL 33613-4706 ([email protected]).
Pediatric treatment and epidemiology studies require accurate, reliable, and reproducible information on psychiatric diagnoses. The increasing use of standardized structured and semistructured diagnostic interviews, specifically designed for children and adolescents, has reduced the risk of inadequate assessment in these populations.1-3 However, many of these diagnostic instruments are lengthy and time-consuming. For example, the Schedule for Affective Disorders and Schizophrenia for School Aged Children,4 the Child and Adolescent Psychiatric Assessment,5 and the Diagnostic Interview for Children and Adolescents6 each require up to 2 to 3 hours to administer. Many of the existing interviews also require complicated cross-checking and synthesis to score and are difficult to navigate, and most require considerable training. These considerations have led to increasing calls for a brief but accurate and reliable diagnostic instrument that is easy to administer in this population.7 The Mini International Neuropsychiatric Interview for Children and Adolescents (MINI-KID) was developed to meet this need.8
STUDY OBJECTIVES
This study investigates the reliability of the MINI-KID and its validity in relation to the Schedule for Affective Disorders and Schizophrenia for School Aged Children-Present and Lifetime Version (K-SADS-PL), treated here as the criterion or reference standard. The primary objectives were to evaluate (1) the concordance between MINI-KID and K-SADS-PL diagnoses and (2) the interrater and test-retest reliability of the MINI-KID. Secondary objectives included assessing (1) the comparative administration times of the MINI-KID and K-SADS-PL and (2) the concordance between the parent-rated version of the MINI-KID (MINI-KID-P) (with only the parent present) and the standard MINI-KID (with the child and a parent both present).
We also report the results for the suicidality disorder section of the MINI-KID, and we report the results of preliminary tests to measure the concurrent validity of the Sheehan Disability Scale as a measure of impairment to administer with the MINI-KID.
METHOD
Sample
Two hundred thirty-one children and adolescents (195 outpatients and 36 community controls) participated. Five children (four 7-year-olds and one 11-year-old, all outpatients) had to be excluded from the analysis of results because they were unable to complete both interviews. This left a final sample of 226. The mean age of the sample was 12.8 ± 3.5 years, and the age range was 6 to 17 years. Thirty-seven percent (n = 83) were children aged 6 to 12 years, and 63% (n = 143) were adolescents aged 13 to 17 years. Fifty-eight percent (n = 132) were male, and 42% (n = 94) were female. Eighty-six percent (n = 195) reported their race/ethnicity as white, 8% (n = 18) as black or African American, 3% (n = 6) as Hispanic, and 3% (n = 4) as multiracial or other. Gender and age distributions were similar for community controls and identified patients.
Recruitment
Child and adolescent patient volunteers, aged 6 to 17 years 11 months, were recruited from the child and adolescent outpatient program of the University of South Florida Psychiatry Center and its affiliated programs including 4 community substance use disorder programs for adolescents and a local community hospital for eating disorder patients. Child and adolescent controls were recruited from a local community church organization. Children and adolescents with mental retardation, significant developmental disorders, congenital abnormalities, delirium, dementia, a language problem, a serious medical illness, or brain damage were excluded. To assess for exclusionary criteria, parents and referring physicians/staff were contacted by one of the MD or PhD investigators once a referral was made. At that time, subjects meeting exclusion criteria such as organic brain damage were excluded. Almost all children were appropriately referred and were included. Participation was voluntary. The study was approved by the institutional review board of the University of South Florida, and all subjects and their parents received an explanation of the study and gave informed consent before the study interviews took place.
Interviewers, Site, and Training
Interviewers included 3 board-certified faculty psychiatrists, 6 psychiatry residents or child psychiatry fellows, 4 doctoral-level faculty psychologists, 1 RN/MS, and 5 BA and MS research coordinators at the University of South Florida College of Medicine or one of its affiliated institutions. Fifty-eight percent (n = 131) of the interviews were conducted at the University of South Florida College of Medicine Psychiatry Center. An additional 21% (n = 48) were conducted at affiliated alcohol and substance treatment centers, 5% (n = 11) at a specialty eating disorder hospital, and 16% (all controls) (n = 36) in the community, mostly at churches serving the local population. All of the raters received training on the MINI-KID and the K-SADS-PL prior to conducting interviews. For the MINI-KID, rater training took approximately 2 hours. For the K-SADS-PL, because of the substantial training time required, only raters who were previously trained were used.
Instruments
Mini International Neuropsychiatric Interview for Children and Adolescents. The MINI-KID is a structured clinical diagnostic interview designed to assess the presence of current DSM-IV and ICD-10 psychiatric disorders in children and adolescents aged 6 to 17 years in a way that is comprehensive and concise. The interview is administered to the child/adolescent together with the parent(s), although it can be administered to adolescents without a parent present. The MINI-KID follows the structure and format of the adult version of the interview (MINI), which has been validated against the Structured Clinical Interview for DSM-III-R8 and against the World Health Organization-designed Composite International Diagnostic Interview.9 Like its adult counterpart, the MINI-KID is organized in diagnostic sections or modules. Using branching tree logic, the instrument asks 2 to 4 screening questions for each disorder. Additional symptom questions within each disorder section are asked only if the screen questions are positively endorsed. All questions are in binary “yes/no” format. Discrepancies between parent and child reports are resolved at the item (individual question) level by asking for further input from the child and the parent and using clinical judgment to break ties. Diagnostic criteria are summarized and documented within each disorder section and on a summary sheet. The instrument screens for 24 DSM-IV and ICD-10 psychiatric disorders and suicidality (Table 1) and takes approximately a half an hour to administer. A parent-rated version of the MINI-KID (MINI-KID-P), designed only for parents, is also available.8
Click figure to enlarge
Schedule for Affective Disorders and Schizophrenia for School Aged Children-Present and Lifetime Version. The K-SADS-PL is a semistructured diagnostic interview designed to assess current and past DSM-III-R and DSM-IV psychiatric disorders in children and adolescents aged 6 to 17 years. The interview requires prior training and is administered by a clinician to the parent(s) and separately to the child or adolescent. The instrument has a 10- to 15-minute unstructured introductory clinical and demographic interview followed by an 82-item screen interview for current and most severe past episodes of psychopathology. Up to 5 additional diagnostic supplements (for affective, psychotic, anxiety, behavioral, substance abuse, and other disorders), each with several pages of additional items, are completed for problematic areas. For each individual item on the screen interview and supplements, the interviewer enters separate parent and child scores, ranging from 0 to 3, for 2 time frames, current episode and most severe pastepisode. After the 2 interviews are completed, the clinician cross-checks screen and supplement information, then meets with the parent and child together to resolve discrepancies and enters an additional 2 summary scores for a total of 6 scores for each item. Summary diagnostic checklists indicating the presence of definite or probable diagnoses are completed only after all summary scores are cross-checked against DSM-III-R or DSM-IV criteria placed, as a rule, at the end of the appropriate supplements. The K-SADS-PL identifies up to 30 current and past DSM-III-R and DSM-IV disorders and takes 2½ to 3 hours (75-90 minutes each for the parent and child) to administer to child and adolescent psychiatric patients.4,10
Children’s Global Assessment Scale. The Children’s Global Assessment Scale (C-GAS), a required component of the K-SADS-PL, is a clinician-rated measure of global functioning for children and adolescents. The scale is rated from 0 to 100, where 1 represents the most impaired functioning and 100 represents superior functioning. Adapted from the Adult Global Assessment Scale, the C-GAS is considered to be a valid and reliable tool for rating a child’s general level of functioning on a health-illness continuum.11
Sheehan Disability Scale. The Sheehan Disability Scale (SDS) is a self-rated composite of 3 self-rated items designed to measure functional disability across 3 domains: work or school, social life and home/family life. Using a 10-point discretized visual analog (Discan) scale that contains verbal descriptors and numeric scores, the patient rates the extent to which his or her symptoms have disrupted or interfered with his or her (1) work or school, (2) social life, and (3) home or family life on a visual analog continuum. The SDS is widely used in clinical psychopharmacology trials and has been validated in adult populations.12-14 It is appropriate for children and adolescents because of its simplicity and its inclusion of the school domain as an alternative to the work domain.
Procedures
MINI-KID/K-SADS-PL comparison. Each subject (N = 226) was administered the MINI-KID and the K-SADS-PL in a predetermined, randomly assigned, counterbalanced order such that half received the MINI-KID before the K-SADS-PL and half received it after the other interview. All interviews were conducted on the same day by 2 different interviewers blinded to the results of the other interview. For 50 subjects, all adolescents in substance abuse treatment, the MINI-KID and K-SADS-PL were administered without a parent or parents present. For all other subjects (n = 176), a parent or parents were interviewed along with the child or adolescent. After MINI-KID instructions were given, the MINI-KID was administered to the child or adolescent and parent(s) together (in a joint interview). Following standard K-SADS procedure, the K-SADS-PL was administered in separate interviews to the parent(s) and child or adolescent. As in previous studies, for children, the K-SADS-PL was administered to the parent first, and for adolescents, the order was reversed.4
Disability assessment. Each subject (N = 226) completed the patient-rated SDS, a measure of impairment in school, social, and family life, so that it could be compared to the C-GAS, a clinician-rated measure of functioning that is administered as part of the K-SADS-PL. The SDS was administered before the MINI-KID.
MINI-KID-P/MINI-KID comparison. The parent(s) of a subset of children (n = 140) were administered the parent version of the MINI-KID (MINI-KID-P) in addition to the standard MINI-KID. These interviews were conducted in a randomly assigned counterbalanced order, by different interviewers, such that half received MINI-KID-P before the standard MINI-KID and half received it afterward. The child was not present for these parent interviews, and data from these interviews were not made available to subsequent interviewers.
Interrater and test-retest reliability tests. For a subset of the subjects (n = 57), the MINI-KID was rated by a second live rater to assess its interrater reliability. For another subset (n = 83), the MINI-KID was readministered by a third rater, blinded to the results of the earlier interviews, 1 to 5 days after the initial rating, to assess its retest reliability. This method of examining the stability of the instrument is a stringent one in that different examiners were used to conduct the test and retest interviews. With the addition of a potential source of error variance, the test-retest reliability analysis would be expected to produce a very conservative estimate of stability.
All of the data were collected between March 2004 and January 2008.
Statistical Analysis
We assessed agreement between the MINI-KID and the K-SADS-PL both for current syndromal diagnoses (any anxiety disorder, any mood disorder, any conduct disorder, any alcohol or substance use disorder, any psychotic disorder, any eating disorder) and for current individual disorder diagnoses.
We used 2 approaches to evaluate agreement: Cohen’s κ15 and the area under the receiver operating characteristic curve (AUC).16 Cohen’s κ is a chance-corrected measure of agreement that ranges from 0 to 1. Landis and Koch17 have suggested the following guidelines for interpreting kappas: < 0 (no agreement), 0.0-0.20 (slight), 0.21-0.40 (fair), 0.41-0.60 (moderate), 0.61-0.80 (substantial), and 0.81-1.00 (almost perfect). Others, eg, Shrout et al,18 after Fleiss,19 suggest that κ values greater than approximately 0.75 indicate excellent agreement beyond chance, values below approximately 0.40 represent poor agreement beyond chance, and values in between indicate fair to good agreement beyond chance. Although κ is commonly used as a measure of agreement in validity studies of psychiatric disorders, it is dependent on prevalence and can be low even when there is high concordance on low-prevalence conditions.20 Moreover, κ can vary across populations when there are differences in prevalence rates, even when those populations have the same sensitivity (SN) and specificity (SP). The AUC, interpreted as the probability that a randomly selected clinical case will score higher on the test than a noncase, has been proposed to correct this problem.20 Originally developed to analyze the association between a continuous predictor and a dichotomous outcome, the AUC can be used in a situation in which the predictor is a dichotomy. In this case, the AUC equals (SN + SP) / 2.21 Following Agresti,22 we considered the AUC to be excellent evidence of concordance if ≥ 0.90, good evidence of concordance if between 0.80 and 0.90, acceptable although only average if between 0.70 and 0.80, and poor if below 0.70.
For the MINI-KID/K-SADS-PL and MINI-KID-P/MINI-KID comparisons, we report the following additional information: prevalence of syndromal and individual disorder diagnoses by each instrument; the absolute numbers of participants in each cell of the cross-tabulation by test and criterion status: true-positive, false-negative, false-positive, and true-negative; and the sensitivity, specificity, positive predictive value (PPV), negative predictive value (NPV), and efficiency of the test. For the MINI-KID/K-SADS-PL comparisons, sensitivity is defined as the probability of the MINI-KID correctly diagnosing a subject with a particular disorder (using the K-SADS-PL results as the criterion). Specificity is defined as the probability of the MINI-KID correctly concluding that a subject does not have the disorder (using the K-SADS-PL results as the criterion). The PPV is defined as the probability that a subject who is identified as having the disorder on MINI-KID actually has the disorder using the K-SADS-PL as the criterion. The NPV is the probability that a subject identified as not having the disorder actually does not have the disorder based on the results for the criterion measure. Efficiency is a measure of the overall accuracy of the MINI-KID, ie, the number of cases correctly classified by the MINI-KID divided by the sample size.
Before analyzing the data for individual disorders for the MINI-KID/K-SADS-PL comparison, we combined the following disorders: all diagnoses of bipolar disorder (I, II, not otherwise specified, etc) with mania and hypomania for the K-SADS-PL and mania with hypomania for the MINI-KID. We also combined alcohol dependence with alcohol abuse, substance dependence with substance abuse, the 3 subcategories of attention-deficit/hyperactivity disorder (ADHD) (inattentive, hyperactive, and combined) and subcategories of tic disorders (vocal tics, motor tics, and Tourette’s disorder), and all subtypes of psychotic disorders on both instruments. In addition, we combined mania and hypomania in comparisons of the standard version of the MINI-KID and the parent version and in interrater and test-retest analyses.
Comparisons for pervasive developmental disorder were not made, since none of the subjects met criteria for this disorder. Comparisons for enuresis and encopresis were not made, since these disorders are not present on the MINI-KID. The suicide module of the MINI-KID was similarly excluded in the comparison with the K-SADS-PL, since the K-SADS-PL does not have a suicide module. Adjustment disorders were excluded, since the rates were too low to compare. K-SADS-PL diagnoses of avoidant and overanxious disorder were included in the syndromal counts of anxiety disorder diagnoses for the K-SADS-PL, but could not be compared at an individual disorder level since the MINI-KID does not include these disorders.
For MINI-KID/K-SADS-PL comparisons of interview length by primary diagnosis, we used the primary syndromal diagnosis on the K-SADS-PL. For reporting of the MINI-KID suicidality results by diagnosis, we also used the primary syndromal diagnosis on the K-SADS-PL. However, we separated out behavioral disorders with and without concurrent mood and/or anxiety disorders for these comparisons.
Paired t tests were used to evaluate mean differences in interview administration time for all subjects and for subgroups of subjects by age, gender, and primary diagnosis. Mean differences between subjects meeting criteria for a MINI-KID or K-SADS-PL diagnosis were evaluated using independent t tests. T tests were also used to test the validity of the SDS, by comparing disability scores for those meeting and not meeting diagnostic criteria on the MINI-KID and K-SADS-PL, respectively, and correlations were used to examine agreement between the SDS and the C-GAS. The Wilcoxon 2-sample test was employed to compare rates of disorders for adolescents who were interviewed alone compared to those who were interviewed with a parent present and providing input.
RESULTS
Number of Diagnoses
Within the identified patient group (n = 190), 93% (n = 176) met criteria for at least 1 current DSM-IV diagnosis on the MINI-KID and 78% (n = 149) met criteria for at least 1 current diagnosis on the K-SADS-PL. Seventy-five percent (n = 142) met criteria for multiple current diagnoses on the MINI-KID, and 56% (n = 127) met criteria for multiple current diagnoses on the K-SADS-PL. Only a few patients met criteria for only 1 diagnostic syndrome or had what could be called a “pure” diagnostic profile on the MINI-KID. Of these, 6 met criteria for an anxiety disorder alone, 3 for a mood disorder alone, 23 for ADHD and/or a behavioral disorder alone, and 13 for a substance use disorder alone.
Overall, the MINI-KID identified significantly more current DSM-IV disorders (3.4 ± 2.5; range, 0-14) compared to the K-SADS-PL (2.1 ± 1.9; range, 0-9; paired t1 = 8.3, P < .0001).
Among children aged 6 to 12 years (n = 83), the MINI-KID and K-SADS-PL identified similar rates of major depressive episode (10% and 7%), ADHD (61% and 60%), and oppositional defiant disorder (20% and 17%). However, compared to the K-SADS-PL, the MINI-KID identified a higher percentage of cases of anxiety (35% vs 22%), mania or hypomania (32% vs 18%), conduct disorder (32% vs 18%), and psychosis or mood disorder with psychotic features (12% vs 3%).
Among adolescents aged 13 to 17 years (n = 143), the two instruments identified similar rates of alcohol or substance abuse or dependence (44% and 38%) and oppositional defiant disorder (13% and 14%). However, compared to the K-SADS-PL, the MINI-KID identified a higher percentage of cases of anxiety (44% vs 30%), major depressive episode (22% vs 15%), mania/hypomania (37% vs 15%), ADHD/tic disorders (38% vs 29%), conduct disorder (40% vs 18%), and psychosis or mood disorder with psychotic features (20% vs 6%).
Agreement Between MINI-KID and K-SADS-PL
Table 2 shows the results for agreement between the MINI-KID and the K-SADS-PL at the level of syndromal diagnoses, and Table 3 shows the results for agreement at the level of individual disorders.
Click figure to enlarge
Click figure to enlarge
At the syndromal level (any mood disorder, any anxiety disorder, etc), AUC values were good to excellent (0.81-0.96) and the uncorrected or raw concordance (efficiency) of the MINI-KID with the K-SADS-PL was high (79%-96%). Kappa scores, which are influenced by base rates, were more variable, ranging from fair (0.41) for any psychotic disorder to excellent (0.87) for any alcohol or substance use disorder. Sensitivity was high (0.81-1.00) for all of the syndromal diagnoses, with the exception of any eating disorder (0.71), and specificity was good to excellent (0.74-0.99) at this level. Positive predictive values (the proportion of MINI-KID syndromal diagnoses confirmed on the K-SADS) were good to excellent (0.60-0.84) for all syndromal diagnoses with the exception of any psychotic disorder (0.30). This low PPV was a function of both low base rates and the higher identification of psychotic disorder and mood disorder with psychotic features on the MINI-KID. Negative predictive values were uniformly high (≥ 0.91).
At the level of individual disorders (Table 3), AUC values were good to excellent (≥ 0.81) for 13 disorders and average (0.71-0.80) for 5 disorders. AUC concordance values for major depressive episode, dysthymia, generalized anxiety disorder (GAD), oppositional defiant disorder, and lifetime mood disorder with psychotic features were all below average. Raw concordance (efficiency) was excellent (81%-100%) for all of the individual disorder diagnoses except mania at 78%. Kappa agreement was more variable. Kappa values were slight for current mood disorder with psychotic features (0.10), dysthymia (0.16), and social phobia (0.18). Kappa values were only fair (0.21-0.40) for major depressive episode, most of the anxiety disorders (agoraphobia, separation anxiety, specific phobia, GAD), oppositional defiant disorder, and current psychotic disorder. Kappa values were in the moderate range (0.41-0.60) for mania/hypomania, panic disorder, obsessive-compulsive disorder, conduct disorder, and lifetime psychotic disorder. Kappa agreement was substantial (0.61-0.80) for ADHD, posttraumatic stress disorder (PTSD), tic disorders, and anorexia and excellent (> 0.81) for alcohol dependence or abuse, drug dependence or abuse, and bulimia.
Overall, the operating characteristics of the MINI-KID for individual disorder diagnoses were very good. Sensitivity was high (≥ 0.81) for 9 disorders, substantial (0.64-0.80) for 7 disorders, and acceptable (0.43-0.60) for the remaining 7 disorders. Specificity was high (≥ 0.81) for 21 disorders and substantial (0.73-0.79) for the remaining 2 disorders. Positive predictive values were less even. Positive predictive values were substantial to high (0.72-0.86) for 6 disorders (bulimia, alcohol and drug abuse or dependence, PTSD, Tourette’s disorder, and ADHD). They were in the moderate range (0.41-0.60) for 5 disorders (mania/hypomania, panic disorder, conduct disorder, oppositional defiant disorder, and anorexia). Positive predictive values were fair or poor (≥ 0.40) for the remaining disorders. These low PPV values, clustered in the anxiety, psychotic, and mood disorders, are a function of low base rates and a trend for the MINI-KID to identify more positive cases than the K-SADS-PL. Negative predictive values were uniformly high (0.90-1.00) for all of the individual disorder diagnoses.
Length of Interviews
Some sections of the K-SADS-PL (the introductory clinical and demographic section and the impairment module) are not explored in the MINI-KID. To facilitate comparison of the administration time of the two interviews, we excluded the time it took to administer these sections from the overall calculation of the duration of the K-SADS-PL, thus artificially shortening the usual length of time it takes to administer the full K-SADS-PL.
Click figure to enlarge
Nevertheless, the mean time it took to administer the MINI-KID was only about a third of the time it took to administer the K-SADS-PL (33.5 ± 14.5 vs 103.4 ± 41.3 minutes) (Figure 1). This difference was statistically significant (paired t216 = 27.9, P < .0001). The approximately 3-fold lower duration of the MINI-KID held for community controls (18.6 ± 8.8 vs 64.7 ± 34.5 minutes, paired t35 = 10.3, P < .0001) as well as for identified patients (36.4 ± 13.6 vs 111.2 ± 38.8 minutes, paired t180 = 27.3, P < .0001), for the interviews conducted with a parent present (33.3 ± 15.2 vs 104.1 ± 40.6 minutes, paired t165 = 25.9, P < .0001) and without a parent present at the interview (34.1 ± 11.8 vs 101.8 ± 43.7 minutes, paired t50 = 11.4, P < .0001), for males (34.3 ± 14.7 vs 102.7 ± 40.5 minutes, paired t121 = 19.8, P < .0001) and females (32.3 ± 14.3 vs 104.9 ± 42.2 minutes, paired t92 = 19.7, P < .0001), and for children aged 6 to 12 years (35.6 ± 14.7 vs 102.3 ± 35.4 minutes, paired t77 = 18, P < .0001) and adolescents aged 12 to 17 years (32.3 ± 14.3 vs 104.2 ± 44.5 minutes, paired t138 = 15.5, P < .0001). As shown in Table 4 and graphically depicted in Figure 2, the MINI-KID provided a reduction in the median administration time over the K-SADS-PL of 64% or more for patients with primary diagnoses of major depressive episode or mania, anxiety, psychosis, Tourette’s disorder, and behavioral disorder and 70% or more for community controls and for patients with primary diagnoses of substance dependence or abuse or an eating disorder.
Click figure to enlarge
Click figure to enlarge
Adolescents Interviewed Alone Versus With a Parent
For 51 of the 143 adolescents in the study, the MINI-KID and the K-SADS were both administered without parental presence or input (although all parents gave prior informed consent). Surprisingly, there were no significant differences in the mean lengths of either interview for adolescents interviewed alone compared to those interviewed with a parent present (34.1 vs 33.3 minutes for the MINI-KID, F1 = 0.82, P = .36 and 102.5 vs 105.0 minutes for the K-SADS-PL, F1 = 0.10, P = .74).
On the MINI-KID, adolescents interviewed without a parent present had similar rates of the following disorders compared to adolescents interviewed with a parent present: any mood disorder (53% vs 43%; z = 1.18, P = .23), any anxiety disorder (49% vs 42%; z = 0.85, P = .39), any ADHD/tic disorder (39% vs 38%; z = 0.05, P = .95), and any psychotic disorder (z = −0.7, P = .09). Rates for any conduct disorder or oppositional defiant disorder were higher for adolescents interviewed alone (53% vs 38%), but differences were not statistically significant (z = 1.6, P = .09). The adolescents interviewed alone did have higher rates of alcohol and substance use disorders (z = 6.9, P < .0001) and eating disorders (16% vs 6%; z = 1.9, P = .06), but these differences were to be expected since a higher percentage of adolescents interviewed alone were interviewed in a community substance use treatment center (69% vs 15%) or an eating disorder treatment hospital (12% vs 7%).
Concordance between the MINI-KID and K-SADS-PL was also surprisingly similar across adolescents interviewed alone and those interviewed with a parent for most of the syndromal diagnoses. For any anxiety disorder, the AUC was identical (0.82). For any psychotic disorder, the AUC was also identical (0.93). For any mood disorder, it was similar (0.80 vs 0.77). This was also the case for any ADHD/tic disorder (0.85 vs 0.88). On the other hand, MINI-KID/K-SADS-PL concordance for adolescents interviewed alone versus those interviewed with a parent was somewhat higher for conduct or oppositional defiant disorder (AUC = 0.87 vs 0.77) and somewhat lower although still good for any alcohol or substance use disorder (0.83 vs 0.93).
Agreement Between MINI-KID-P and MINI-KID
For 140 subjects, including 104 subjects identified as patients and 36 community controls, the parent version of the MINI-KID (MINI-KID-P) was administered separately to the parent or parents. In these MINI-KID-P interviews, the child was not present. The gender and racial distribution of this subsample was similar to that of the total sample. However, the mean ± SD age of the subjects in this subsample was slightly lower (11.5 ± 3.5 vs 12.8 ± 3.5 years).
Table 5 shows the agreement for the comparison between the parent version of the MINI-KID (with only a parent present) and the standard MINI-KID (with the child and a parent both present) as the criterion or reference. Raw concordance (efficiency) was 89% or higher for all disorders with the exceptions of oppositional defiant disorder and the inattentive subtype of ADHD, which was 87%. AUC values were good to excellent (≥ 0.81) for 14 of 22 individual diagnoses including suicidality and acceptable, if only average (≥ 0.70), for the remaining 8 diagnoses. Kappa agreement was high (≥ 0.81) for 5 of 22 individual disorders, substantial (0.61-0.80) for 9 disorders, and moderate (0.46-0.60) for the remaining 7 disorders. Overall, sensitivity was higher for disorders with external manifestations (mania, substance abuse, tic disorders, ADHD, and conduct disorder) and lower for disorders that might not have external manifestations (anxiety and phobias). Although the parent version showed good sensitivity for suicidality, there was some evidence of parental overestimation of suicidality. This may be because parents, when interviewed alone, are more likely to identify and report suicidality, or it could be because children, when interviewed jointly with a parent, are less likely to admit to suicidal thoughts or behaviors. Conversely, parents tended to underestimate softer symptoms such as those associated with mood disorder with psychotic symptoms.
Click figure to enlarge
Interrater and Test-Retest Reliability
For 57 subjects, all identified patients, the MINI-KID was rated by a second rater at the same time as the original rating, and for 83 subjects, including 35 controls and 48 identified patients, the MINI-KID was readministered 1 to 5 days after the first interview by a third blind rater. As shown in Tables 6 and 7, AUC values for interrater reliability were excellent (≥ 0.94) for all 7 syndromal diagnoses and high (≥ 0.89) for all individual disorder diagnoses with the exceptions of hypomania and the hyperactive subtype of ADHD, for which the numbers of cases were very low. When mania and hypomania were combined, the κ and AUC values were perfect (1.0). These statistics indicate very good interrater reliability.
Click figure to enlarge
In general, reliability coefficients for test-retest interviews tend to be lower than those for interrater interviews.23 In this case, the test-retest reliability results were very good to excellent (0.87-1.00) for all 7 syndromal diagnoses and acceptable to excellent (0.75-1.00) for all of the individual disorder diagnoses with the exceptions of dysthymia and panic disorder.
Suicidality
Suicidality, defined as the presence of suicidal ideation and/or suicidal behaviors, including suicide attempts and self-injurious acts, is measured in a separate MINI-KID module. This module contains 14 categorical yes/no questions addressing current and lifetime suicidal ideation (active and passive), suicidal intent and plans, suicidal acts, suicide attempts, and self-injurious behavior. Three additional questions elicit frequency and intensity of current suicidal ideation and type of current suicidal intent. The module can be used to produce a dichotomous suicide risk rating (present or not present), and it also produces a numeric suicide risk score with identified anchors for “low,” “moderate,” and “high” suicide risk.
Overall, on the MINI-KID, suicide risk, defined as any nonzero suicidal ideation, plan, or attempt or nonsuicidal self-damaging act, was significantly more frequent among patients compared to controls (44% vs 6%; χ21 = 18.8, P < .0001). As shown in Figure 3, suicide risk was highest among subjects with a primary diagnosis of psychotic disorder (67%). However, more than half (56%) of those with a primary diagnosis of a mood disorder (depression or mania) or a combination of a behavioral and a mood disorder (60%) met criteria for lifetime suicidality.
Click figure to enlarge
Click figure to enlarge
As shown in Table 7, MINI-KID interrater and retest reliabilities for current and lifetime suicidality were good to excellent (AUC = 0.89-0.99, κ = 0.81-0.96). The concordance of the parent-rated version (MINI-KID-P) with the standard version of the MINI-KID for current and lifetime suicidality was lower but still good (AUC = 0.83-0.84, κ = 0.63−0.64).
The concordance of the suicide risk ratings of the MINI-KID could not be tested against the K-SADS-PL, since the latter does not provide similar ratings or a suicide module. The K-SADS-PL, however, does collect data on current and past suicidal ideation, suicidal behavior, and nonsuicidal self-damaging acts in 5 items in the depression module of the K-SADS-PL screen interview. Comparisons on individual items showed the MINI-KID reporting a slightly higher lifetime suicidal ideation rate (35% vs 30%) compared to the K-SADS-PL when K-SADS-PL subthreshold and threshold scores were included and a much higher lifetime ideation rate (35% vs 13%) when only threshold scores on the K-SADS-PL were used. Raw agreement for these comparisons was 81% and 74%, with AUC values of 0.81 and 0.78, respectively. The two interviews identified similar rates of lifetime suicide attempts (7% vs 8%; AUC = 0.78) and nonsuicidal self-damaging acts (6% vs 7%), although the correspondence for the latter comparison was only average (AUC = 0.70).
Functional Impairment/Disability
We examined the validity of the SDS as a measure of impairment by comparing SDS scores of those who did and did not meet criteria for a current DSM-IV disorder using t tests. As another test of the concurrent validity of the MINI-KID, we used simple linear regression with the Pearson correlation to test the validity of the SDS against the C-GAS, a measure of global functioning routinely administered with the K-SADS-PL. In addition, we conducted a discriminant analysis of the SDS to establish a meaningful cutoff score for the scale for children and adolescents.
The mean SDS total score for the full sample (N = 226) was 13.3 ± 9.1 (range, 0-30). Mean ± SD item scores were 4.7 ± 3.6 for the school item, 3.8 ± 3.2 for the social item, and 4.7 ± 3.4 for the family item. Total SDS scores were highest for subjects with an eating disorder, followed by those with a behavioral disorder plus a concurrent mood or anxiety disorder (Figure 4).
Click figure to enlarge
Total SDS scores were significantly higher for children and adolescents meeting criteria for at least 1 MINI-KID disorder compared to those not meeting criteria for any MINI-KID disorder (15.7 ± 8.0 vs 3.8 ± 6.5; t1 = 8.7, P < .0001). Total SDS scores were also significantly higher for children and adolescents meeting criteria for at least 1 K-SADS-PL disorder compared to those not meeting criteria for a K-SADS-PL disorder (16.3 ± 9.0 vs 7.5 ± 7.5; t1 = 7.6, P < .0001). These SDS scores are similar to those found in clinical trials and primary care for adults meeting criteria for DSM-IV psychiatric disorders.14,24
Since the C-GAS measures global functioning, with low scores indicating more impairment, and the SDS measures disability, with low scores indicating less impairment, we expected an inverse relationship between the two measures. Simple linear regression confirmed this expectation and showed that the SDS predicted C-GAS scores with reasonable accuracy (B = −0.58, F1 = 97, P = .0001). The correlation between the 2 measures, however, was only moderate (r = −0.58), and the regression explained only 34% of the variance in functioning. This result may have occurred because, unlike the clinician-rated C-GAS, the SDS is self-rated. In addition, the SDS may tap into different sources of impairment, since it summarizes disability in 3 specific domains (school, social, and family life), whereas the C-GAS provides a global score.
Using a discriminant analysis, we found that an elevated total SDS score (≥ 5) was associated with an increased risk of meeting criteria for at least 1 psychiatric disorder on the MINI-KID and yielded respectable sensitivity (0.87), specificity (0.72), and positive and negative predictive values (PPV = 0.92, NPV = 0.61).
DISCUSSION
The MINI-KID assesses the presence of current and some lifetime DSM-IV and ICD-10 psychiatric disorders in children and adolescents. This study is the first examination of the interrater and test-retest reliability of the MINI-KID and the first to compare it with a validated interview (the K-SADS-PL). It is also the first to report on the validity of SDS as a measure of impairment in this population.
The study had 2 important strengths: (1) assessments were all made by raters who were blind to the results of the other interviews (mitigating potential rater bias) and (2) interviews were randomly sequenced (mitigating potential order effects). The study also had at least 3 limitations: (1) a few subjects became frustrated or disruptive and were unable to complete both interviews, leading to a sample loss of 5 subjects; (2) for 51 subjects, all adolescents in substance use treatment, corroborative information could not be collected from parents since these subjects were interviewed without a parent present; and (3) the number of cases for some disorders (eg, PTSD, panic disorder, anorexia) was low.
Overall, the data suggest that the MINI-KID succeeds in reliably and validly eliciting symptom criteria used in making DSM-IV diagnoses in children and adolescents and does so in a third of the time required for the K-SADS-PL. The interrater reliabilities were consistently high. Test-retest reliabilities were also high and comparable to those reported for other structured and semistructured diagnostic interviews.4,5,25 Concordance with the K-SADS-PL was excellent at the syndromal level and good to excellent for most of the individual disorder diagnoses. Most of the differences between the MINI-KID and the longer K-SADS-PL were in the direction of the MINI-KID identifying more cases of disorders, ie, being more inclusive in “making the diagnosis.” For a more than 66% reduction in the administration time over the K-SADS-PL, sensitivity and specificity were very good. There was no evidence of inflation in false-positives in the community control population.
The high specificity of the MINI-KID suggests that it is a desirable diagnostic tool for confirmatory diagnoses. Its high sensitivity suggests that it is also good at ruling out disorders.
Although the MINI-KID had a high rate of “false-positives” for some diagnoses, it is arguable that many of these “false-positives” were not actually false. Relative to the MINI-KID, the K-SADS-PL is long and difficult to administer. Previous work has shown that the likelihood of children (and their parents) responding affirmatively to structured interview questions decreases when an interview is long, possibly because they know that a “yes” answer could generate additional questions and lengthen the interview still further.26 The length of the K-SADS-PL may have made it harder to “make the diagnosis,” particularly if the child or adolescent was tired initially, became fatigued, or had an attention deficit. The use of multiple criteria (DSM-III-R + DSM-IV) by the K-SADS-PL may have also contributed to its lower rate of positive diagnoses. The K-SADS-PL was initially designed with DSM-III-R criteria in mind. As a result, several of the screen and some of the supplement questions (eg, for panic disorder) are asked using DSM-III-R terminology and time frames. This means that the interviewer must keep a mental or actual note of DSM-IV terminology and time frames and recheck all item responses with DSM-IV in mind to make the diagnosis.
Differences in diagnostic exclusion rules and algorithms could also have contributed to a lower identification of cases on the K-SADS. For example, for major depressive episode, both instruments require endorsement of 1 of 2 DSM-IV screen criteria and 5 of 7 additional criteria. The K-SADS-PL, however, precludes parental input on 1 of the 2 screen criteria (depressed mood) if the parent is thought to be concurrently depressed. The MINI-KID does not make this exclusion, and it is not required by DSM-IV. In addition, for major depressive episode, the K-SADS precludes 2 of the 7 additional criteria (concentration problems and psychomotor disturbance) from being counted toward the requisite 5 if the child has ADHD unless “there was a worsening in [the symptom] that corresponded with the onset of depressed mood.” The impact of these exclusions/qualifications on the K-SADS-PL is difficult to assess, but it is likely that they made it more difficult to make the diagnosis of major depressive episode as well as mood disorder with psychotic features (which requires the presence of a mood disorder) on the K-SADS.
In addition, we tried to make the MINI-KID as “kid friendly” as possible. However, the wording of questions could have had an impact on case identification and the number of apparent “false-positives.” On the social phobia module, for example, it is possible that the use of a descriptor such as “embarrassed” in addition to “shy” led to a higher rate of “yes” answers on MINI-KID screen questions, with the result that more subjects completed the remaining questions and were found to have the disorder.
The choice of specific screen questions could also have influenced the results. We designed the MINI-KID to be especially sensitive to mania and hypomania, conditions that along with depression may contribute to a diagnosis of 1 or more subtypes of bipolar disorder and that were underidentified in children and adolescents in the past. On the MINI-KID, the full mania module must be completed if the child or adolescent endorses either expansive mood or irritability. On the K-SADS PL, endorsement of elation, expansive mood, racing thoughts, increases in activity, or a decrease in the need for sleep requires the full mania supplement. However, irritability is not included in the screen. The almost 2-fold higher rate of identification of mania/hypomania on the MINI-KID may have been at least partially a function of more subjects completing the full MINI-KID mania/hypomania module after endorsing irritability.
There was no evidence that the MINI-KID’s higher rate of identification of mania/hypomania was a function of greater overlap with ADHD. The frequency of co-occurring ADHD for subjects meeting criteria for mania/hypomania on the MINI-KID was similar to that on the K-SADS-PL (73% vs 66%).
The parent version of the MINI-KID, administered without the child present, was closest to the standard version (with the child present) for disorders with clear outward manifestations (Tourette’s disorder, alcohol or substance abuse/dependence, panic disorder, and dysthymia). The parent version was moderately close to the standard version for psychotic disorder but less close for disorders that required more assessment of interior states, including most of the anxiety disorders (agoraphobia, separation anxiety, specific phobia, social phobia, PTSD) and suicidality. These findings support those of Foley et al,27 who reported that agreement between parents and children on the presence of anxiety tends to be low possibly because of differences in the way parents and children conceptualize anxiety.
Preliminary results for the analysis of the SDS suggest that it is a valid measure of impairment in children and adolescents and can be used in conjunction with the MINI-KID to identify mental health-related functional impairment in this population.
Recent Developments, Implications, and Future Directions
On the basis of the results of this study, the investigators have since strengthened several questions on the instrument to enhance its sensitivity and specificity. Also, for studies on schizophrenia, schizoaffective disorders, and mood disorder with psychotic features, a more detailed MINI-KID psychotic disorder module now exists. As is the case with the adult MINI, this alternative module can be used as a stand-alone or as a substitute for the standard MINI-KID psychotic disorder module.
The MINI-KID has been translated into several languages including Spanish, French, Russian, Hungarian, German, and Turkish.
Over the long run, an important question, beyond the concordance of the MINI-KID with other structured and semistructured diagnostic interviews and its reliability, is the more complicated question of true validity. How well does the MINI-KID, or for that matter other instruments like the K-SADS-PL, actually identify “true” cases in the child and adolescent population? While it is possible that the MINI-KID overestimates some conditions (eg, psychotic disorder), it is also possible that the K-SADS-PL underestimates several of the anxiety disorders (eg, panic disorder, social phobia, agoraphobia). One implication of this study is that further work is needed for all structured diagnostic interviews to reduce misclassification. Brugha et al,28 following Jobe and Mingay,29 have suggested that cognitive interviewing of respondents to check their understanding of the meaning of structured questions can improve accuracy. Also, some criteria (eg, for psychosis) require considerable clinical experience to accurately rate. We now recommend that only experienced clinicians rate this module. However, following the recommendations of Brugha et al,28 in the absence of highly experienced clinicians, it may be possible to incorporate options such as verbatim descriptions for later rating by an experienced assessor.
Potential Applications
Research. Although the MINI-KID provides less disorder subtyping than the K-SADS-PL, it covers a broad range of diagnoses applicable to children and adolescents and takes much less time to administer. The MINI-KID, as well as individual diagnostic modules of the instrument, can be used by researchers for rapid screening of homogenous samples for treatment studies and clinical trials. It has potential applications for epidemiology studies. It can also be used by researchers in tandem with symptom severity rating scales to assess degree of illness severity, and we encourage researchers to use severity rating scales of their choosing to supplement the diagnostic results of the MINI-KID.
Pediatric primary care. The MINI-KID can be used as a screening tool in pediatric primary care settings when psychiatric specialists are not available. In these settings, we advise that the child be consulted and treated as the primary respondent, with the parent providing collateral input. We also advise that administration of the psychotic disorders module may not be appropriate since it requires advanced training and experience to administer.
Clinical practice. The MINI-KID is a valuable adjunct to diagnostic screening in inpatient and outpatient child and adolescent mental health settings.
Health care systems, health maintenance organizations, and managed care. The MINI-KID has potential applications as a first step in outcome tracking and continuous quality improvement. One of its advantages is that it can be used by properly trained health information technicians or physician extenders who are not physicians or doctoral-level psychologists. In addition, individual sections of the MINI-KID (eg, for major depressive episode or ADHD) can be administered as stand-alone modules for specific purposes, and its brevity makes inclusion in office practice feasible.
CONCLUSIONS
The MINI-KID is a reliable and valid measure of current child and adolescent psychopathology that can be administered in almost a third of the time required for the K-SADS-PL. The MINI-KID has the advantage of identifying psychiatric disorders that may be missed on the longer interview.
Author affiliations: Depression and Anxiety Disorders Research Institute (Drs D. V. Sheehan, K. H. Sheehan, and Janavs) and Department of Psychiatry and Behavioral Medicine (Drs Shytle, Milo, Stock, and Wilkinson and Ms Bannon), University of South Florida College of Medicine, Tampa; and Department of Psychiatry and Human Behavior, Brown University, Providence, Rhode Island (Dr Rogers).
Possible conflicts of interest: Dr D. V. Sheehan has received grant funding support, been affiliated with, or received honoraria and travel expenses related to lectures/presentations or consultant activities from the following organizations (1 = consultant, 2 = grant/research support, 3 = lectures/presentations, 4 = stockholder): Abbott,1,2,3 Ad Hoc Committee—Treatment Drug & Assessment Research Review,1 Alexa,1 Alza,1 American Medical Association,2 American Psychiatric Association Task Force on Treatments of Psychotic Disorders,1 American Psychiatric Association Working Group to Revise DSM-III Anxiety Disorders Section,1 Anclote Foundation,2 Anxiety Disorders Resource Center,1 Anxiety Drug Efficacy Case—US Food and Drug Administration,1 Applied Health Outcomes/XCENDA,1 AstraZeneca,1,2,3 Avera,1,2 Boehringer Ingelheim,3 Boots,3 Bristol-Myers Squibb,1,2,3 Burroughs Wellcome,2,3 Cephalon,1 Charter Hospitals,3 Ciba Geigy,3 Committee (RRC) of NIMH on Anxiety and Phobic Disorder Projects,1 Connecticut & Ohio Academies of Family Physicians,1 Cortex Pharmaceuticals,1 Council on Anxiety Disorders,1 CPC Coliseum Medical Center,1 Cypress Bioscience,1 Dista Products Company,3 Division of Drugs and Technology—American Medical Association,1 EISAI,1,2 Eli Lilly,2,3 Excerpta Medica Asia,3 Faxmed,1 Forest,1,2 Glaxo,3 GlaxoSmithKline,1,2,3 Glaxo Wellcome,2 Hospital Corporation of America,3 Humana,3 ICI,3 INC Research,1 International Clinical Research (ICR),2 International Society for CNS Drug Development (ISCDD),1 Janssen,1,2,3 Jazz,1,2 Kali-Duphar,2,3 Labopharm,1 Layton Bioscience,1 Lilly Research Laboratories,1 Lundbeck Denmark,1 Marion Merrill Dow,3 McNeil,3 Mead Johnson,2,3 Medical Outcome Systems,4 MediciNova,1,2 Merck Sharp & Dohme,2,3 National Anxiety Awareness Program,1 National Anxiety Foundation,1 National Depressive and Manic Depressive Association,1 National Institute on Drug Abuse,2 National Institute of Health,2 Novartis,2 Novo Nordisk,3 Organon,1,3 Orion,1 Parexel,1 Parke-Davis,2,3 Pfizer,1,2,3 Pharmacia,1 Pharmacia & Upjohn,1,3 Philadelphia College of Pharmacy & Science,1 Pierre Fabre France,1 Quintiles,2 Rhone Laboratories,3 Rhone-Poulenc Rorer Pharmaceuticals,3 Roche,1 Roerig,3 Sandoz,2,3 Sanofi-Aventis,1,2,3 Sanofi-Synthelabo Recherche,1,2 Schering,3 Sepracor,1 Shire,1 SmithKline Beecham,1,2,3 Solvay,1,3 Takeda,1 Tampa General Hospital,1 University of South Florida Psychiatry Center2—University of South Florida College of Medicine, TAP,2,3 Targacept,1 TGH-University Psychiatry Center,3 Tikvah,1 Titan,1 United Bioscience,2 The Upjohn Company,1,2,3 US Congress—House of Representatives Committee,1 USF Friends of Research in Psychiatry—Board of Trustees,1 Warner Chilcott,2,3 World Health Organization,1 Worldwide Clinical Trials,2 Wyeth-Ayerst,1,2,3 ZARS,1 and Zeneca.2 Dr K. H. Sheehan has received independent consultant fees from Janssen, Layton Bioscience, Merck, Solvay, and Upjohn; has received grant/research support as subinvestigator, investigator, and/or statistical consultant on studies with AstraZeneca, Bristol-Myers Squibb, GlaxoSmithKline, Kali-Duphar, Labopharm, Layton Bioscience, Medicinova, Pfizer, Pharmacia, Sanofi Aventis, SmithKline Beecham, and Wyeth Ayerst; and is Dr D. V. Sheehan’s spouse. Dr Shytle has been a consultant for Pfizer, Yaupon Therapeutics, AstraZeneca, and Natura Therapeutics. Dr Janavs has received grant/research support from Abbott, AstraZeneca, Avera, Bristol-Myers Squibb, Cephalon, Dainippon Sumitomo, Eisai, Eli Lilly, Glaxo, Glaxo-Wellcome, GlaxoSmithKline, Indevus, Janssen, Jazz, Labopharm, Medicinova, Merck Sharp and Dohme Research Labs, National Institute of Health, Novartis, Otsuka, Pfizer, Repligen, Roerig-Pfizer, Sandoz, Sanofi Synthelabo, SmithKline Beecham, Stanley Medical Research Institute, Synthelabo, Takeda, UCB Pharma, Upjohn, Wyeth-Ayerst, and Zeneca and has been a consultant or lecturer or given presentations or symposia for Allergan, Biovail/Ingenix, Boehringer-Ingelheim, Cephalon, Forest, Synosia, GlaxoSmithKline, Lundbeck, Merck, NIH/NIAAA, Pfizer, Roche, Sanofi-Aventis, Sanofi-Synthelabo Therapeutics, Synthelabo, TAP, and Wyeth-Ayerst. Dr Stock has received grant/research support from GlaxoSmithKline and Bristol-Myers Squibb and has served on the speakers/advisory boards for Janssen, Ortho-McNeil, and Bristol-Myers Squibb. Drs. Rogers, Milo, and Wilkinson and Ms Bannon report no additional financial or other relationship relevant to the subject of this article.
Funding/support: This study was funded by GlaxoSmithKline, Philadelphia, Pennsylvania.
Previous presentation: The data were presented at the 49th Annual Meeting of NCDEU (poster session II, poster 23), July 1, 2009, Hollywood, Florida.
Acknowledgment: The authors are grateful to the following people for their valuable contributions to the study: P. Powers, MD; M. F. Sheehan, MD; B. Curtright, PhD; K. Weiner, PhD; H. Perry Mills, MD; L. Hilko, MS; S. Bream, BA; A. Corlett, BA; T. Leveille, BA; S. Iwanicki, BA; A. Jenkins, MBA; D. Beamon, BA; A. Sheehan; I. Fouad, BA; and A. Cole, BA. The authors also thank the administrators and staff of Phoenix House (Tampa, Florida) and Operation PAR (Largo, Florida).
Supplementary material: Available at PSYCHIATRIST.COM.
REFERENCES
1. Hughes CW, Rintelmann J, Mayes T, et al. Structured interview and uniform assessment improves diagnostic reliability. J Child Adolesc Psychopharmacol. 2000;10(2):119-131. PubMed doi:10.1089/cap.2000.10.119
2. McClellan JM, Werry JS. Introduction: research psychiatric diagnostic interviews for children and adolescents. J Am Acad Child Adolesc Psychiatry. 2000;39(1):19-27. PubMed doi:10.1097/00004583-200001000-00013
3. McClellan JM, Werry JS. Evidence-based treatments in child and adolescent psychiatry: an inventory. J Am Acad Child Adolesc Psychiatry. 2003;42(12):1388-1400. PubMed doi:10.1097/00004583-200312000-00005
4. Kaufman J, Birmaher B, Brent D, et al. Schedule for Affective Disorders and Schizophrenia for School-Age Children-Present and Lifetime Version (K-SADS-PL): initial reliability and validity data. J Am Acad Child Adolesc Psychiatry. 1997;36(7):980-988. PubMed doi:10.1097/00004583-199707000-00021
5. Angold A, Costello EJ. The Child and Adolescent Psychiatric Assessment (CAPA). J Am Acad Child Adolesc Psychiatry. 2000;39(1):39-48. PubMed doi:10.1097/00004583-200001000-00015
6. Reich W. Diagnostic Interview for Children and Adolescents (DICA). J Am Acad Child Adolesc Psychiatry. 2000;39(1):59-66. PubMed doi:10.1097/00004583-200001000-00017
7. Renou S, Hergueta T, Flament M, et al. Diagnostic structured interviews in child and adolescent’s psychiatry [in French]. Encephale. 2004;30(2):122-134. PubMed doi:10.1016/S0013-7006(04)95422-X
8. Sheehan DV, Lecrubier Y, Sheehan KH, et al. The Mini-International Neuropsychiatric Interview (M.I.N.I.): the development and validation of a structured diagnostic psychiatric interview for DSM-IV and ICD-10. J Clin Psychiatry. 1998;59(suppl 20):21-32.
9. Lecrubier Y, Sheehan DV, Weiller E, et al. The Mini International Neuropsychiatric Interview (MINI): a short diagnostic structured interview: reliability and validity according to the CIDI. Eur Psychiatry. 1997;12(5):224-231. doi:10.1016/S0924-9338(97)83296-8
10. Ambrosini PJ. Historical development and present status of the Schedule for Affective Disorders and Schizophrenia for School-Age Children (K-SADS). J Am Acad Child Adolesc Psychiatry. 2000;39(1):49-58. PubMed doi:10.1097/00004583-200001000-00016
11. Shaffer D, Gould MS, Brasic J, et al. A children’s global assessment scale (CGAS). Arch Gen Psychiatry. 1983;40(11):1228-1231. PubMed
12. Leon AC, Shear MK, Portera L, et al. Assessing impairment in patients with panic disorder: the Sheehan Disability Scale. Soc Psychiatry Psychiatr Epidemiol. 1992;27(2):78-82. PubMed doi:10.1007/BF00788510
13. Leon AC, Olfson M, Portera L, et al. Assessing psychiatric impairment in primary care with the Sheehan Disability Scale. Int J Psychiatry Med. 1997;27:93-105. PubMed
14. Sheehan KH, Sheehan DV. Assessing treatment effects in clinical trials with the discan metric of the Sheehan Disability Scale. Int Clin Psychopharmacol. 2008;23(2):70-83. PubMed doi:10.1097/YIC.0b013e3282f2b4d6
15. Cohen J. A coefficient of agreement for nominal scales. Educ Psychol Meas. 1960;20(1):37-46. doi:10.1177/001316446002000104
16. Hanley JA, Lippman-Hand A. If nothing goes wrong, is everything all right? interpreting zero numerators. JAMA. 1983;249(13):1743-1745. PubMed doi:10.1001/jama.249.13.1743
17. Landis JR, Koch GG. The measurement of observer agreement for categorical data. Biometrics. 1977;33(1):159-174. PubMed doi:10.2307/2529310
18. Shrout PE, Spitzer RL, Fleiss JL. Quantification of agreement in psychiatric diagnosis revisited. Arch Gen Psychiatry. 1987;44(2):172-177. PubMed
19. Fleiss HJL. Statistical Methods for Rates and Proportions. 2nd ed. New York, NY: John Wiley & Sons; 1981.
20. Byrt T, Bishop J, Carlin JB. Bias, prevalence and kappa. J Clin Epidemiol. 1993;46(5):423-429. PubMed doi:10.1016/0895-4356(93)90018-V
21. Kessler RC, Akiskal HS, Angst J, et al. Validity of the assessment of bipolar spectrum disorders in the WHO CIDI 3.0. J Affect Disord. 2006;96(3):259-269. PubMed doi:10.1016/j.jad.2006.08.018
22. Agresti A. Categorical Data Analysis. 2nd ed. Hoboken, NJ: John Wiley & Sons, Inc; 2002.
23. Rush AJ, Bose A. Escitalopram in clinical practice: results of an open-label trial in a naturalistic setting. Depress Anxiety. 2005;21(1):26-32. PubMed doi:10.1002/da.20051
24. Zimmerman M. Diagnosing personality disorders. a review of issues and research methods. Arch Gen Psychiatry. 1994;51(3):225-245. PubMed
25. Welner Z, Reich W, Herjanic B, et al. Reliability, validity, and parent-child agreement studies of the Diagnostic Interview for Children and Adolescents (DICA). J Am Acad Child Adolesc Psychiatry. 1987;26(5):649-653 PubMed doi:10.1097/00004583-198709000-00007
26. Lucas CP, Fisher P, Piacentini J, et al. Features of interviews questions associated with attenuation of symptom reports. J Abnorm Child Psychol. 1999;27(6):429-437. PubMed doi:10.1023/A:1021975824957
27. Foley DL, Pickles A, Maes HM, et al. Course and short-term outcomes of separation anxiety disorder in a community sample of twins. J Am Acad Child Adolesc Psychiatry. 2004;43(9):1107-1114. PubMed doi:10.1097/01.chi.0000131138.16734.f4
28. Brugha TS, Bebbington PE, Jenkins R, et al. Cross validation of a general population survey diagnostic interview: a comparison of CIS-R with SCAN ICD-10 diagnostic categories. Psychol Med. 1999;29(5):1029-1042. PubMed doi:10.1017/S0033291799008892
29. Jobe JB, Mingay DJ. Cognitive laboratory approach to designing questionnaires for surveys of the elderly. Public Health Rep. 1990;105(5):518-524. PubMed
