This article is freely available to all

Article Abstract

Objective: To characterize the feasibility and safety of a wireless networked system incorporating physiologic assessments and direct confirmation of digital tablet ingestions in ambulatory patients with schizophrenia or bipolar disorder.

Method: In this 4-week observational study conducted between May 2010 and May 2011 at 2 US academic clinical study sites, 12 adults with bipolar disorder and 16 adults with schizophrenia (all diagnosed according to DSM-IV criteria) utilized a digital health feedback system (DHFS). All subjects were on a stable regimen of oral medication. The DHFS utilized a digital tablet, consisting of an ingestion sensor that was embedded in a tablet containing nonpharmacologic excipients, which subjects coingested with their regularly prescribed medication. The formulation of this digital tablet allowed ingestion sensor separation and activation by stomach fluids after ingestion, followed by communication of a unique identifying signal from the ingestion sensor to an adhesive sensor worn on the torso, which automatically logged the date and time of each digital tablet ingestion. The wearable sensor also collected physiologic measures including activity and heart rate. The primary study objective was to compare the accuracy of DHFS in confirming digital tablet ingestion versus a method of directly observed ingestion; secondary aims included characterization of adherence and physiologic measures longitudinally in these cohorts.

Results: 27 of 28 subjects (96%) completed the study. The mean adherence rate was 74% (95% CI, 64%-86%), and 67% (95% CI, 55%-79%) of doses were taken within 2 hours of the prescribed dosing time. Activity consisted of 847 to 15,930 steps daily, and sleep duration ranged from 3.2 to 15.2 hours daily. For individual subjects, mean sleep disruption, defined as the amount of brief arousals and postural changes during sleep events (eg, subject sitting up during the night), was as low as 5% and as high as 43% for the entire study period. The most common adverse event was minor skin irritation that occurred at the site of the wearable sensor in 5 subjects (18%), which did not lead to early discontinuation. No adverse events occurred due to the ingestion sensor. No subjects developed worsening of psychosis attributable to use of the DHFS. Of the 27 subjects who completed the study, 19 (70%) found the DHFS concept easy to understand, 21 (78%) said they would like to receive reminders on their cell phone if they forgot to take their medications, and 24 (89%) thought the DHFS could be useful to them.

Conclusions: The DHFS provided a novel means of confirming medication ingestion and tracking selected physiologic parameters, and it was generally well tolerated by patients.

Trial Registration: ClinicalTrials.gov identifier: NCT01804257

J Clin Psychiatry 2013;74(6):e533-e540

Submitted: October 9, 2012; accepted March 25, 2013 (doi:10.4088/JCP.12m08222).

Corresponding author: Lorenzo A. DiCarlo, MD, 2600 Bridge Pkwy, Ste 101, Redwood City, CA 94065 ([email protected]).

First Experience With a Wireless System Incorporating Physiologic Assessments and Direct Confirmation of Digital Tablet Ingestions in Ambulatory Patients With Schizophrenia or Bipolar Disorder

Vertical divider

ABSTRACT

Objective: To characterize the feasibility and safety of a wireless networked system incorporating physiologic assessments and direct confirmation of digital tablet ingestions in ambulatory patients with schizophrenia or bipolar disorder.

Method: In this 4-week observational study conducted between May 2010 and May 2011 at 2 US academic clinical study sites, 12 adults with bipolar disorder and 16 adults with schizophrenia (all diagnosed according to DSM-IV criteria) utilized a digital health feedback system (DHFS). All subjects were on a stable regimen of oral medication. The DHFS utilized a digital tablet, consisting of an ingestion sensor that was embedded in a tablet containing nonpharmacologic excipients, which subjects coingested with their regularly prescribed medication. The formulation of this digital tablet allowed ingestion sensor separation and activation by stomach fluids after ingestion, followed by communication of a unique identifying signal from the ingestion sensor to an adhesive sensor worn on the torso, which automatically logged the date and time of each digital tablet ingestion. The wearable sensor also collected physiologic measures including activity and heart rate. The primary study objective was to compare the accuracy of DHFS in confirming digital tablet ingestion versus a method of directly observed ingestion; secondary aims included characterization of adherence and physiologic measures longitudinally in these cohorts.

Results: 27 of 28 subjects (96%) completed the study. The mean adherence rate was 74% (95% CI, 64%-86%), and 67% (95% CI, 55%-79%) of doses were taken within 2 hours of the prescribed dosing time. Activity consisted of 847 to 15,930 steps daily, and sleep duration ranged from 3.2 to 15.2 hours daily. For individual subjects, mean sleep disruption, defined as the amount of brief arousals and postural changes during sleep events (eg, subject sitting up during the night), was as low as 5% and as high as 43% for the entire study period. The most common adverse event was minor skin irritation that occurred at the site of the wearable sensor in 5 subjects (18%), which did not lead to early discontinuation. No adverse events occurred due to the ingestion sensor. No subjects developed worsening of psychosis attributable to use of the DHFS. Of the 27 subjects who completed the study, 19 (70%) found the DHFS concept easy to understand, 21 (78%) said they would like to receive reminders on their cell phone if they forgot to take their medications, and 24 (89%) thought the DHFS could be useful to them.

Conclusions: The DHFS provided a novel means of confirming medication ingestion and tracking selected physiologic parameters, and it was generally well tolerated by patients.

Trial Registration: ClinicalTrials.gov identifier: NCT01804257

J Clin Psychiatry 2013;74(6):e533-e540

Submitted: October 9, 2012; accepted March 25, 2013 (doi:10.4088/JCP.12m08222).

Corresponding author: Lorenzo A. DiCarlo, MD, 2600 Bridge Pkwy, Ste 101, Redwood City, CA 94065 ([email protected]).

Numerous studies across all areas of medicine identify the actual taking of medication as prescribed as one of the major challenges in promoting public health. Although medication adherence in the setting of acute illness is often higher, the management of many chronic diseases suffers from problems in continued medication adherence, which in turn contributes to an enormous proportion of avoidable emergency department visits and hospital days, as well as poor overall outcomes. Osterberg and Blaschke,1 in their comprehensive review of the topic, suggested that, of all medication-related hospital admissions in the United States, 33%-69% are due to poor medication adherence, with a resulting cost of approximately $100 billion per year.

The ability of health care providers and caregivers to identify and quantify nonadherence has significant limitations. Existing methods that are simplest (direct questioning or self-report tools) tend to be inaccurate, whereas those that are more accurate tend to be cumbersome, costly, and not easily scalable. Prescription refill data in a closed health care system can be informative, and electronic monitors capable of tracking the time of opening bottles have been in use for many years; however, these methods are still indirect and do not provide data on whether the patient actually took the medication.

Since a large proportion of nonadherence is not willful, conscious refusal to take medication,1 tools that can assist and empower patients and caregivers to play a more informed role in their own health care offer an opportunity to improve adherence. Accurate data about adherence and factors that may influence it are key ingredients in that process. In addition, pairing data on medication adherence with other passive measures of patient behavior, such as sleep and physical activity, may allow for more specific and targeted interventions. More generally, such measures can provide an important perspective on the patient’s health status and specific medication effects so as to improve disease management.

This study was intended to test the feasibility of utilizing a networked system to electronically confirm ingestion of oral medications using ingestion sensors. The goal of this pilot study was to test the feasibility and acceptability of this approach for patients with schizophrenia or bipolar disorder.

METHOD

Study Design

A pilot, observational, small-scale, 2-site study was conducted over 28 days. The study was registered on ClinicalTrials.gov (identifier: NCT01804257). The 2 US clinical study sites were The Zucker Hillside Hospital, Glen Oaks, New York, and Massachusetts General Hospital, Boston. This study was reviewed and approved by the North Shore-Long Island Jewish Health System Institutional Review Board and the Massachusetts General Hospital Institutional Review Board; the study was routinely monitored by the sponsor according to Good Clinical Practice guidelines.

Materials

The investigational product is a networked digital health feedback system (DHFS) (Proteus Digital Health, Inc; Redwood City, California) that electronically confirms ingestion of oral medications by use of an ingestion sensor, acquires physiologic metrics, and presents this information in an integrated manner (Figure 1).

The DHFS consists of an ingestion sensor sized smaller than a sesame seed (1 mm ×— 1 mm) and made of materials in the food chain. For this initial study in a mental health population, the ingestion sensor was embedded in a placebo tablet. This digital tablet was formulated to allow ingestion-sensor separation after ingestion and activation by gastric fluid and electrolytes.2 Activation does not depend upon stomach pH. After activation, the sensor communicates a unique identifying signal to an adhesive sensor that is worn on the torso and logs the date and time of the ingestion event. Activation of the ingestion sensor lasts approximately 5-7 minutes and communicates to the wearable sensor in a manner similar to an electrocardiogram; communication of the ingestion sensor to the wearable sensor cannot be detected beyond the patient. After the ingestion sensor’s activation and communication, the remainder of the ingestion sensor passes through the digestive system and is removed from the body by fecal elimination.

The wearable sensor also captures physiologic metrics, including heart rate, body position, physical activity, and sleep characteristics. Utilizing encrypted wireless communication, the wearable sensor relays this information to a mobile device, which stores the de-identified data and periodically transfers it to a password-protected server using secure encryptions that are used by the banking industry.

At the server level, data are collected, analyzed, and distributed to appropriate parties through different interfaces that are also secure. In this study, subjects were provided with a mobile phone-based user interface to enter self-reported supplemental information that helped to contextualize the data, and health care providers were provided a secure web-based Provider Panel for viewing data across subjects and per subject.

Figure 1

Click figure to enlarge

The Provider Panel is protected by login and password; once the login is authenticated, an overview of cohort data is presented (Figure 2). Providers are able to see all subjects currently using the DHFS and their corresponding data. The information shown on the overview screen is customizable; all data or a subset of data stored on the secured server can be displayed. Additionally, providers can view the data on a per-subject level and take a detailed look at a subject’s adherence, physiologic, and self-reported data and their trends over time. In this study, subjects were provided a personalized weekly report during their clinic visits. There was no prescribed schedule for use of the Provider Panel.

Figure 2

Click figure to enlarge

clinical points

  • Management of many chronic diseases suffers from problems with continued medication adherence, which in turn contributes to an enormous proportion of avoidable emergency department visits and hospital days, as well as poor overall outcomes.
  • In appropriately selected patients, the use of a digital health feedback system (DHFS) provides a means of acquiring, summarizing, and communicating data on medication use, health status, and activities of daily living.
  • Acquiring such information through a DHFS system may facilitate earlier and more targeted interventions for patients at risk of disease progression or relapse.

Study Population

Candidates were recruited through referrals and advertisements at 2 academic centers. Subjects with the diagnosis of schizophrenia were recruited at The Zucker Hillside Hospital of the North Shore-Long Island Jewish Health System, Glen Oaks, New York, and subjects with the diagnosis of bipolar disorder were recruited at Massachusetts General Hospital, Boston. Subjects who were identified for inclusion were willing to adhere to study procedures and provide informed consent; met DSM-IV3 criteria for bipolar disorder or schizophrenia; scored 3 or below on the Clinical Global Impression-Schizophrenia scale4; and were on a stable regimen of oral mood stabilizers or antipsychotic medication for at least 14 days, with no anticipation to change or titrate the regimen throughout the course of the study.

Study candidates were excluded from the study if they were considered a serious homicide or suicide risk, as assessed by the evaluating clinician; had a score of 3 or higher on the suspiciousness/paranoia item of the Brief Psychiatric Rating Scale (BPRS)5; had unstable medical illnesses, including cardiovascular, hepatic, renal, respiratory, endocrine, neurologic, or hematologic disease; had DSM-IV diagnoses or symptoms of substance use disorders, active within the prior 60 days; had a history of significant gastrointestinal disease or gastrointestinal surgery, with clinical instability that, in the investigator’s opinion, could preclude safe participation; had known allergies that could preclude safe participation; were currently utilizing an electronically active implanted medical device; were currently participating or had participated in another drug or device study within the last 30 days; demonstrated the inability to provide consent; or were female and pregnant or of childbearing potential but not employing a medically accepted means of contraception.

As part of safety monitoring, the BPRS was administered to study subjects at pretreatment and at the time of each scheduled study visit during treatment. Subjects were withdrawn for a score of 3 or higher on the suspiciousness/paranoia item of the BPRS or if there was clinical evidence of symptomatic worsening of any kind that, in the investigator’s opinion, could preclude safe participation in the study. Additional assessments of subjects with bipolar disorder at baseline and during the study included the Young Mania Rating Scale (YMRS)6 and the Montgomery-Asberg Depression Rating Scale (MADRS).7

Study subjects were enrolled after providing written informed consent per the International Conference on Harmonisation Guideline for Good Clinical Practice.8

Study Objectives

Due to the small sample size in this observational study, the investigation was largely exploratory. The primary objective was to compare the detection accuracy of the ingested digital tablets using the DHFS to that of a directly observed method. Secondary objectives aimed at (1) characterizing medication-taking behavior in a bipolar or schizophrenia patient population utilizing co-ingestion of digital tablets; (2) characterizing physiologic metrics, ie, activity and sleep; (3) characterizing the system safety; (4) obtaining user satisfaction data with the system; and (5) characterizing the use of the system and its components by subjects, caregivers, and health providers.

Study Procedures

To achieve the main objectives described above, the following procedures were implemented. For each subject, there was an enrollment and baseline visit to determine eligibility for participation and DHFS training, there were up to 4 weekly visits to receive the next week’s supplies and for safety evaluations, and there was 1 safety follow-up visit approximately 30-40 days after the last day of system exposure. Excluding the safety follow-up, subjects were instructed to ingest 2 digital tablets under directly observed ingestions at each clinic visit. Each digital tablet had 2 ingestion sensors affixed to it; thus, each subject ingested 4 ingestion sensors per directly observed ingestion session, totaling 32 ingested sensors per subject.

Subjects were instructed to wear their wearable sensors continuously for the duration of the study. In addition, subjects were asked to co-ingest 2 digital tablets with their regular medications daily. Each of the digital tablets that were provided for off-site use had only 1 ingestion sensor affixed to it. Subjects were also asked to carry the mobile phone as much as possible or leave it in a dedicated location at home, where they could have frequent and easy access to it. Subjects were asked to manually input into the mobile phone their daily "sleep quality" score, which consisted of a 7-point subjective rating of the quality of sleep for the previous night (Table 1).

Table 1

Click figure to enlarge

Outcome Measures

To characterize the performance of the DHFS, several key outcome measures were defined and assessed; this information is summarized in Table 1.

Statistical Plan

Positive detection accuracy (PDA). PDA was defined as the number of sensor ingestions detected by the DHFS, divided by the number of directly observed sensor ingestions. A 95% CI for the PDA across all subjects was calculated using a mixed model for repeated measures, with terms for sensor ingestion day as a fixed effect and subject as a random effect. The PDA results were used to compare the detection accuracy of the DHFS to that of a direct observation of sensor ingestion during the clinic visits that were scheduled for the study.

Adherence metrics. Descriptive statistics (n, mean, median, standard deviation, minimum, and maximum) across all subjects at a given site are presented along with 95% CIs. Overall descriptive data (combining the 2 sites) are also presented. Success measure for these metrics was based on the ability to quantify adherence consistently and reliably using the DHFS.

Safety

The proportion of subjects with an adverse event was summarized by event, relationship to the device, relationship to the procedure, and severity.

RESULTS

The study period was from May 2010 to May 2011. Enrollment of a maximum of 40 subjects was planned. The study was discontinued after 28 subjects were enrolled, as continuing DHFS development was underway; this fact would have made it impractical to assimilate data from additional subjects. Of the 28 enrolled subjects, 27 (96%) completed the 4-week study. A total of 761 subject-days and 440 directly observed ingestion data points from 28 subjects were available for analysis. Demographic data are presented in Table 2.

Table 2

Click figure to enlarge

Study Withdrawal

One subject was withdrawn from the study per protocol on the basis of meeting prespecified BPRS criteria for withdrawal. The subject was known to have a history of paranoia but did not have paranoid ideation at the time of baseline assessment. At the time of withdrawal, the subject’s paranoia had worsened, but he did not express any concern regarding the DHFS or study staff. The investigator determined the withdrawal to be unrelated to system use or participation in the study.

Medication Adherence

The PDA of the DHFS as compared to directly observed ingestion was 94% (95% CI, 85%-100%), assuming a repeated-measures model. Of the 25 nondetections that occurred, failure analysis demonstrated that 10 were due to inadvertent administration of digital tablets that had identical ingestion sensor identification numbers, 4 were due to failure to place the wearable sensor on the subject’s torso before digital tablet administration, 8 were due to digital tablet administration before completion of the wearable sensor firmware activation cycle, and 3 were due to indeterminate reasons.

The mean ± SD adherence rate was 74% ± 25% (95% CI, 64%-86%). Timing adherence rate was 67% ± 31% (95% CI, 55%-79%) for doses that were taken within ± 2 hours of the prescribed dosing time. This latter percentage represents a subset of the doses that were taken at all (mean adherence).

Physiologic Metrics

Other physiologic metrics were also analyzed and quantified. Activity consisted of 847 to 15,930 steps daily, and sleep duration ranged from 3.2 to 15.2 hours daily. Across subjects per day, mean ± SD activity was 6,909 ± 3,218 steps (95% CI, 5,661-8,157 steps); sleep duration was 9.3 ± 2.5 hours (95% CI, 8.3-10.3 hours); and sleep disruption, defined as the amount of brief arousals and postural changes during sleep events (eg, subject sitting up during the night), was as low as 5% and as high as 43% for the entire study period. No differences in these metrics were observed when patients with bipolar disorder were compared to patients with schizophrenia (P > .05).

The subjects were also asked for their self-assessment of sleep quality; the mean ± SD score on a 7-point scale was 5.0 ± 1.0 (95% CI, 4.5-5.4).

Additional correlation between system-derived and subject-assessed sleep metrics was explored using the Spearman rank correlation coefficients. These coefficients were low, 0.3 and 0.2, for sleep duration and sleep disruption, respectively, ie, there was no meaningful correlation of system-derived sleep duration or sleep disruption with subjective assessment of these sleep metrics by study subjects.

Physiologic metrics, medication-taking adherence, and scheduling adherence were plotted as a function of study day (Figure 3). There were no significant correlations among these metrics.

Figure 3

Click figure to enlarge

Safety

No serious adverse device effects or unanticipated adverse device effects occurred. No subjects developed new onset of paranoid ideation while using the DHFS or experienced recurrence of paranoia related to DHFS use.

One serious adverse event was reported; the subject had a history of schizoaffective disorder and completed the planned 4 weeks of system use per protocol. During the safety follow-up period, the subject was voluntarily admitted for a transient exacerbation. The investigator determined this event to be unrelated to the study device or study participation. The exacerbation resolved, and the patient was discharged uneventfully.

Nine nonserious adverse events were reported by 6 subjects, and 6 of these 9 events were characterized as device-related and occurring in 5 subjects (18%). All of these device-related, nonserious adverse events were characterized as skin irritation at the site of the wearable sensor placement. The skin-related adverse events were mild in nature and self-limited.

There was no significant change from baseline in the mean scores for the BPRS, the YMRS psychosis item, the MADRS, or the YMRS for the study population.

Digital Health Feedback System Usability

Of the 27 subjects who completed the study, 19 (70%) found the DHFS concept easy to understand, 21 (78%) said they would like to receive reminders on their cell phone if they forgot to take their medications, and 24 (89%) thought the DHFS could be useful to them.

DISCUSSION

Wirelessly observed therapy has been studied in more than 250 human subjects and has included more than 14,000 ingestion sensor ingestions. Subjects have included healthy volunteers and individuals with diabetes, heart failure, hypertension, renal transplantation, and tuberculosis. Clinical studies have included women as well as men, with more than 5,000 subject-days of system use.2,9 The ingestion sensor and wearable sensor have been cleared by the Food and Drug Administration for use in the United States and have received the Commonwealth of Europe (CE) mark for use in Europe. In this 2-site study of 28 individuals with bipolar disorder or schizophrenia, the DHFS yielded high detection accuracy when compared with directly observed ingestion and was well tolerated by and acceptable to patients. Taken together, these results indicate the potential utility of electronic assessment of patient treatment and status in psychiatry.

Poor treatment adherence represents a major modifiable contributor to poor patient outcomes and increased health care costs.1 For example, poor adherence has been described even in bipolar patients engaged in long-term treatment programs and was associated with poorer outcomes10,11; there have been similar observations among schizophrenia patients.12 For this reason, a multitude of strategies have been developed to enhance adherence. These range from individual or group cognitive-behavioral intervention to Medication Event Monitoring System (MEMS) caps to Short Message Service (SMS) or e-mail reminders; each intervention has its benefits as well as its limitations. Our results suggest that, by providing a reliable and not overly intrusive means of assessing medication-taking and patient status in real time, the DHFS may complement existing strategies. Of note, no subjects developed new onset of paranoid ideation while using the DHFS or experienced recurrence of paranoia related to the study device and procedures, including the close monitoring of subjects’ activities.

Our results suggest that it appears to be feasible to use electronic systems to monitor changes in sleep characteristics in outpatients. In individuals with bipolar disorder or schizophrenia, worsening or disruption of sleep is often an early indicator of impending relapse or recurrence. The ability to characterize sleep on a continuing basis may, in high-risk patients, provide a potential means of detecting these changes early enough to allow intervention. For other patients, worsening depression may be suggested by decreases in activity levels before a patient or family member is actually aware that such changes are occurring.13 The lack of correlation between the system-derived and subject-assessed sleep metrics in this study is not surprising: discrepancies are known to exist between objective and subjective measures of sleep in patients with depression or schizophrenia.14,15 At the same time, depressed patients demonstrate objective changes in sleep continuity and architecture,16 and sleep disturbance has been reported in 30%-80% of patients with schizophrenia depending on the degree of psychotic symptomatology.17 Longer-term studies will be required to establish the optimal role of electronic assessment of psychiatric patients. The present results suggest that such questions merit further study.

One limitation of the present investigation arises from the study population, that is, individuals who were motivated to participate in the study and to use such a device may also have been more likely to adhere to medication and more willing to tolerate minor adverse effects. This possibility may have biased our results to be more favorable than those that might occur in a broader psychiatric population.

Patients with more severe depression, manic symptoms, or severe psychosis were excluded per protocol, even though this group would be likely to exhibit more extreme symptoms that could correspond more directly to changes in physiologic measures and gaps in medication-taking. For example, severe psychomotor agitation could possibly be detected by actigraphy; the emergence of mania might be associated with a reduction in sleep duration, an increase in sleep fractionation, or an increase in physical activity. Further study may help to determine whether electronic assessment might have a future role in detecting state changes.

CONCLUSION

This pilot investigation demonstrates the feasibility and potential utility of integrating electronic assessment using DHFS in outpatient practice for patients with mood and psychotic disorders. With further study, systems such as DHFS may be usefully applied to characterize not only medication use but also, potentially, other measures of patient status, thus allowing integration of such data to facilitate earlier and more targeted interventions.

Author affiliations: The Zucker Hillside Hospital, Glen Oaks, New York (Drs Kane and Petrides); Department of Psychiatry, Massachusetts General Hospital, Boston (Dr Perlis); and Proteus Digital Health, Inc, Redwood City, California (Drs DiCarlo and Au-Yeung and Ms Duong).

Potential conflicts of interest: Dr Kane has served as a consultant for, has received honoraria from, or has been a member of the speakers or advisory boards for Otsuka and Proteus. Dr Perlis has served as a consultant for Proteus, Pamlab, and Genomind; has received grant/research support from Healthrageous, RID Ventures, and Proteus; has been a member of the speakers or advisory boards for Healthrageous and Genomind; and has received other financial or material support from Concordant Rater Systems. Drs DiCarlo and Au-Yeung and Ms Duong are employees of Proteus Digital Health, Inc. Dr Petrides has no conflicts of interest to disclose relative to this article.

Funding/support: This study was funded by Proteus Digital Health, Inc, Redwood City, California.

Previous presentation: Presented in part at the 51st Annual Meeting of the National Institute of Mental Health New Clinical Drug Evaluation Unit (NCDEU), sponsored by the American Society of Clinical Psychopharmacology; June 13-16, 2011; Boca Raton, Florida.

REFERENCES

1. Osterberg L, Blaschke T. Adherence to medication. N Engl J Med. 2005;353(5):487-497. PubMed doi:10.1056/NEJMra050100

2. Au-Yeung KY, Moon GD, Robertson TL, et al. Early clinical experience with networked system for promoting patient self-management. Am J Manag Care. 2011;17(7):e277-e287. PubMed

3. American Psychiatric Association. Diagnostic and Statistical Manual of Mental Disorders, Fourth Edition. Washington, DC: American Psychiatric Association; 1994.

4. Haro JM, Kamath SA, Ochoa S, et al; SOHO Study Group. The Clinical Global Impression-Schizophrenia scale: a simple instrument to measure the diversity of symptoms present in schizophrenia. Acta Psychiatr Scand. 2003;107(suppl 416):16-23. PubMed doi:10.1034/j.1600-0447.107.s416.5.x

5. Overall JE, Gorham DR. The Brief Psychiatric Rating Scale. Psychol Rep. 1962;10(3):799-812. doi:10.2466/pr0.1962.10.3.799

6. Young RC, Biggs JT, Ziegler VE, et al. A rating scale for mania: reliability, validity and sensitivity. Br J Psychiatry. 1978;133(5):429-435. PubMed doi:10.1192/bjp.133.5.429

7. Williams JBW, Kobak KA. Development and reliability of a structured interview guide for the Montgomery-Asberg Depression Rating Scale (SIGMA). Br J Psychiatry. 2008;192(1):52-58. PubMed doi:10.1192/bjp.bp.106.032532

8. US Department of Health and Human Services, Food and Drug Administration, Center for Drug Evaluation and Research (CDER), and Center for Biologics Evaluation and Research (CBER). Guidance for Industry, E6 Good Clinical Practice: Consolidated Guidance. http://www.fda.gov/downloads/Drugs/Guidances/ucm073122.pdf. Updated April 1996. Accessed March 27, 2013.

9. Wuthrich RP, Eisenberger U, Bock A, et al. Towards a gold standard for adherence assessment in transplantation: high accuracy of the Proteus Raisin System (PRS) combined with enteric-coated mycophenolate sodium (ECMPS) in stable kidney transplant recipients. Presented at the 24th International Congress of the Transplantation Society; July 15-19, 2012; Berlin, Germany.

10. Perlis RH, Ostacher MJ, Miklowitz DJ, et al. Clinical features associated with poor pharmacologic adherence in bipolar disorder: results from the STEP-BD study. J Clin Psychiatry. 2010;71(3):296-303. PubMed doi:10.4088/JCP.09m05514yel

11. Crowe M, Wilson L, Inder M. Patients’ reports of the factors influencing medication adherence in bipolar disorder – an integrative review of the literature. Int J Nurs Stud. 2011;48(7):894-903. PubMed doi:10.1016/j.ijnurstu.2011.03.008

12. Kane JM. Improving treatment adherence in patients with schizophrenia. J Clin Psychiatry. 2011;72(9):e28. PubMed doi:10.4088/JCP.9101tx2c

13. Paffenbarger RS Jr, Lee IM, Leung R. Physical activity and personal characteristics associated with depression and suicide in American college men. Acta Psychiatr Scand. 1994;89(suppl 377):16-22. PubMed doi:10.1111/j.1600-0447.1994.tb05796.x

14. Rotenberg VS, Indursky P, Kayumov L, et al. The relationship between subjective sleep estimation and objective sleep variables in depressed patients. Int J Psychophysiol. 2000;37(3):291-297. PubMed doi:10.1016/S0167-8760(00)00110-0

15. Haimov I, Reshef A, Bloch B, et al. Sleep in schizophrenic and PTSD patients: discrepancy between self-reported versus recorded sleep [abstract]. Presented at the 20th Congress of the European Sleep Research Society; September 15, 2010; Lisbon, Portugal. http://registration.akm.ch/einsicht.php?XNABSTRACT_ID=111885&XNSPRACHE_ID=2&XNKONGRESS_ID=118&XNMASKEN_ID=900. Accessed May 7, 2013.

16. Erman MK. Insomnia: comorbidities and consequences. Prim Psychiatry. 2007;14(6):31-35.

17. Cohrs S. Sleep disturbances in patients with schizophrenia. CNS Drugs. 2008;22(11):939-962. PubMed doi:10.2165/00023210-200822110-00004