Paper: CareMedia: Automated Video and Sensor Analysis for Geriatric Care Bharucha, A.J., Poster presenting the feasibility pilot data at the 29th Annual Meeting of the American Medical Directors Association (AMDA), Dallas, Texas, March 17, 2006 Abstract:   The design and implementation of specific mental health interventions for long-term care (LTC) residents with dementia complicated by neuropsychiatric symptoms has been hampered by the lack of sound understanding of the biopsychosocial and environmental context of these disturbances. CareMedia: Automated Video and Sensor Analysis for Geriatric Care (NSF 0205219) is an National Science Foundation project that hopes to develop computer vision and machine learning technologies that capture in real-time, continuously an audiovisual and sensor record of activities, behaviors and social interactions of LTC residents while simultaneously developing tools for automated data reduction and extraction, and safeguarding privacy. Beginning with raw video, audio and sensor data captured via digital cameras, microphones, and sensors, human coders specify events of interest, such as falls, punching or kicking. Based on the human coding, computer algorithms are developed to automatically detect the event and its contextual characteristics. In this manner, the computer is "trained" to detect salient events, eventually reducing the need for human data coding to only those circumstances that are phenomenologically and semantically challenging even for human eyes. Furthermore, irrelevant and redundant data can be reduced using techniques developed over the past decade so as to permit analysis of large volumes of raw data that previously would have been prohibitive. Thus, this technology has the potential to record, analyze and document quality-of-care and quality-of-life in LTC by developing tools that: (a) augment qualitative observations with quantitative dimensions, thus transforming largely subjective assessments into more measured, objective ones, (b) augment discrete cross-sectional human observations with a machine-captured, continuous longitudinal record, (c) detect and annotate the antecedents and consequences of salient events such as physical aggression, (d) detect and trace the longitudinal trajectory of subtle change in resident functioning, and (e) refine and expand existing methodologies for coding affect, behavior and social interactions. A naturalistic, longitudinal study is underway at two representative community-based dementia units (total 66 beds) to capture the activities, behaviors and social interaction of these residents in the shared spaces of the dementia units (hallways, activity and dining rooms) 24/7 for 4 weeks. In addition to baseline collection of demographic data, medication list, the Physical Self Maintenance Scale and the Cumulative Illness Rating Scale – Geriatrics, a clinical research associate will complete the Cohen-Mansfield Agitation Inventory, Neuropsychiatric Inventory – Nursing Home version, and the Cornell Scale for Depression in Dementia weekly by interviewing the primary caregiver. Moreover, primary caregivers will complete the Ryden Aggression Scale 2 twice a day to capture aggressive behaviors that occur in the private spaces. The list of medical problems and medications will be updated weekly. The digitally captured data will be correlated with that captured by the dementia unit staff. We hope to demonstrate that the application of machine intelligence technologies as data collection, analysis and outcomes assessment tool in LTC is feasible, and will permit truly longitudinal, ecological studies of chronic care populations with continuous capture of objective measurements of their patterns of activities, behaviors and social interactions. AMDA is an ideal setting to present our emerging findings. Close