Clinical Evidence

Much research has validated the approach to outcomes measurement used by PAC-Metrix. Here is a sample…

Diane U. Jette, Mary Stilphen, Vinoth K. Ranganathan, Sandra D. Passek,Frederick S. Frost, and Alan M. Jette  Validity of the AM-PAC “6-Clicks” Inpatient Daily Activity and Basic Mobility Short Forms PHYS THER March 2014 94:379-391; published ahead of print November 14, 2013, doi:10.2522/ptj.20130199

The “6-Clicks” scores differed across patients’ age, preadmission living situation, and number of therapy visits. The areas under receiver operating characteristic curves derived using “6-Clicks” scores at the first visit to predict patients receiving more than one visit were 0.703 and 0.652 using basic mobility and daily activity scores, respectively. The “6-Clicks” scores at the final visit were correlated with scores on subscales of the FIM completed on admission to inpatient rehabilitation facilities (r=.65 and .69). Standardized response means were 1.06 and 0.95 and minimal detectable changes with 90% confidence level (MDC90) were 4.72 and 5.49 for basic mobility and daily activity scores, respectively. Internal consistency reliability of basic mobility and daily activity scores was .96 and .91, respectively. [Link]

Andres PL, Haley SM, Ni P. Is patient-reported function reliable for monitoring post-acute outcomes? Am J Phys Med Rehabil. 2003 Aug; 82(8):614-21.

A major challenge in the development of a comprehensive measurement system to evaluate effectiveness across a broad range of post-acute care settings is the stability and consistency of outcomes measures across respondents and settings. The objective of this study was to investigate the test-retest and subject-proxy reliability of activity scores for use in a new post-acute care outcome instrument using an interview format across different care settings.

Sandel, ME, Jette, A, Appelman, J, Terdiman J, Liu L, Cheng, P, TeSelle, M, Delmonico, R, Wang, H, Camicia, M, Rasch, E, Brandt, D, Chan, L.. Designing and Implementing a System for Tracking Functional Status after Stroke: a Feasibility Study. Arch Phys Med Rehabil. In Press, 2013.

Patient and proxy ratings on the AM-PAC achieved adequate agreement for use in stroke research when using proxy respondents could reduce sample selection bias. The AM-PAC data can be implemented across institutional as well as community care settings while achieving precision and reducing respondent burden. [link]

Chan, L, et al Does Post-Acute Care Site Matter? A longitudinal study assessing functional recovery after a stroke. Arch Phys Med Rehabil. In Press, 2013.

Of the 222 patients analyzed, 36% went home with no treatment, 22% received home health/outpatient care, 30% included an inpatient rehabilitation facility (IRF) in their care trajectory, and 13% included a skilled nursing facility (but not IRF) in their care trajectory. At 6 months, after controlling for important variables such as age, functional status at acute care discharge, and total hours of rehabilitation, patients who went to an IRF had functional scores that were at least 8 points higher (twice the minimally detectable change for the AM-PAC) than those who went to a skilled nursing facility in all 3 domains and in 2 of 3 functional domains compared with those who received home health/outpatient care. [Link]

Jette A, Ni P, Rasch E, et al: Evaluation of patient and proxy responses on the Activity Measure for Post Acute Care. STROKE, Mar;43(3):824-9,2012.

Patient and proxy ratings on the AM-PAC achieved adequate agreement for use in stroke research when using proxy respondents could reduce sample selection bias. The AM-PAC data can be implemented across institutional as well as community care settings while achieving precision and reducing respondent burden. [Link]

Latham N, Mehta V, Nguyen A, Jette AM, et al: Performance-based of self-report measures of physical function: Which should be used in clinical trials of older adults. Archives of Phys Med & Rehabil. 89(11): 2146-2155, 2009.

Findings reveal that the validity, sensitivity, and responsiveness of self-report measures of physical function are comparable to performance-based measures in a sample of patients followed after fracturing a hip. From a psychometric perspective, either type of functional measure would be suitable for use in clinical trials where improvement in function is an endpoint of interest. The selection of the most appropriate type of functional measure as the primary endpoint for a clinical trial will depend on other factors, such as the measure’s feasibility or the strength of the association between the hypothesized mechanism of action of the study intervention and a functional outcome measure. [Link]

Haley S, Ni P, Lai J et al: Linking the AM-PAC and the NeurQOL. Archives of Physical Medicine & Rehabilitation, 92(10), Suppl 1, S37-S43, 2011.

An IRT-based linking method placed AM-PAC and Neuro-QOL mobility and ADL scores on a common metric. This linking allowed estimation of AM-PAC mobility and ADL subscale scores based on Neuro-QOL mobility and ADL subscale scores and vice versa. The accuracy of these results should be validated in a future sample in which participants respond to both instruments. [Link]

Coster W, Haley S, Jette A, Tao W, Siebens H.Predictors of basic and instrumental activity of daily living performance in persons receiving rehabilitation services. Arch Phys Med.2007; 88(7):928-935.

The objective of this study was to examine the relations among cognitive and emotional function and other patient impairment and demographic variables and the performance of daily activities.

Coster W, Haley S, Jette A. Measuring patient-reported outcomes after discharge from inpatient rehabilitation settings. J Rehab Med. 2006; 38:237-242.

The objective of this study was to examine the sensitivity of the Short Form Activity Measure for Post-Acute Care (AM-PAC) in comparison to the Functional Independence Measure (FIM™) across a 12-month period after discharge from rehabilitation hospital.

Coster W, Haley SM, Jette A, Tao W, Siebens H. Predictors of basic and instrumental activities of daily living performance in persons receiving rehabilitation services. Arch Phys Med Rehabil. 2007; 88(7):928-35.

The objective of this study was to examine the relations among cognitive and emotional function and other patient impairment and demographic variables and the performance of daily activities.

Coster WJ, Haley, SM Andres PL, Ludlow LH, Bond T. Refining the conceptual basis for rehabilitation outcome measurement: personal care and instrumental activities domain. Medical Care. 2004; 42(1 Suppl):I62-I72.

Rehabilitation outcome measures routinely include content on performance of daily activities; however, the conceptual basis for item selection is rarely specified. These instruments differ significantly in format, number, and specificity of daily activity items and in the measurement dimensions and type of scale used to specify levels of performance. We propose that a requirement for upper limb and hand skills underlies many activities of daily living (ADL) and instrumental activities of daily living (IADL) items in current instruments, and that items selected based on this definition can be placed along a single functional continuum. The objective of this study was to examine the dimensional structure and content coverage of a Personal Care and Instrumental Activities item set and to examine the comparability of items from existing instruments and a set of new items as measures of this domain.

Coster, WJ, Haley, SM, Ludlow LH, Andres PL, Ni PS. Development of an Applied Cognition Scale to Measure Rehabilitation Outcomes.  Arch Phys Med. 2004; 85:2030-2035.

The objective of this study was to examine the structure and content coverage of an item pool of new items based on the Activity categories from the International Classification of Functioning, Disability and Health and items from existing instruments to measure the applied cognition dimension of function.

Gandek B, Sinclair J, Jette A, Ware, J. Development and initial psychometric evaluation of the Participation Measure for Post-Acute Care. Am J Phys Med Rehabil. 2007; 86(1):1-15.

The objective of this study was to evaluate a patient-reported participation measure constructed within the framework of the International Classification of Functioning, Disability, and Health.

Haley S, Andres P, Coster W, Kosinski M, Ni P, Jette A. Short-Form Activity Measures for Post-acute Care (AM-PAC). Arch Phys Med. 2004; 85:649-660.

The objective of this study was to develop a comprehensive set of short forms using item response theory (IRT) and item pooling procedures for the purpose of monitoring post-acute care functional recovery.

Haley S, Gandek B, Siebens H, Black-Schaeffer R, Sinclair J, Tao W, Coster W, Ni P, Jette A. Computer adaptive testing follow-up after discharge from inpatient rehabilitation.  II. Participation Outcomes. Am J Phys Med Rehabil. 2008; 89(2):275-83.

The objective of this study was to measure participation outcomes with a computerized adaptive test (CAT) and compare CAT and traditional fixed-length surveys in terms of score agreement, respondent burden, discriminant validity, and responsiveness.

Haley S, Ni P, Hambleton R, Slavin M, Jette A. Computer Adaptive Testing Improved Accuracy and Precision of Scores over Random Item Selection in a Physical Functioning Item Bank. J Clin Epidemiol. 2006; 59:1174-1182.

Measuring physical functioning (PF) within and across post-acute settings is critical for monitoring outcomes of rehabilitation; however, most current instruments lack sufficient breadth and feasibility for widespread use. Computer adaptive testing (CAT), in which item selection is tailored to the individual patient, holds promise for reducing response burden, yet maintaining measurement precision. We calibrated a PF item bank via item response theory (IRT), administered items with a post hoc CAT design, and determined whether CAT would improve accuracy and precision of score estimates over random item selection.

Haley SM, Coster WJ, Andres PL, Kosinski M, Ni P.  Score comparability of short-forms and computerized adaptive testing:  an illustration with the Activity Measure for Post-Acute Care (AM-PAC). Arch Phys Med. 2004; 85:661-666.

The objective of this study was to compare simulated short-form and computerized adaptive testing (CAT) scores to scores obtained from complete item sets for each of the 3 domains of the Activity Measure for Post-Acute Care (AM-PAC).

Haley SM, Coster WJ, Andres PL, Ludlow LH, Bond T, Sinclair SJ, Jette AM. Activity Outcome Measurement for Post-acute Care. Medical Care. 2004; 42(1 Suppl):I49-I69.

This study presents results from an initial exploratory factor analysis of AM-PAC items. Three distinct, interpretable factors were identified and accounted for 72% of the variance: Applied Cognition (44%), Personal Care & Instrumental Activities (19%), and Physical & Movement Activities (9%); these 3 activity factors were verified by a confirmatory factor analysis. Scaling assumptions were met for each factor in the total sample and across diagnostic groups. Internal consistency reliability was high for the total sample (Cronbach alpha = 0.92 to 0.94), and for specific diagnostic groups (Cronbach alpha = 0.90 to 0.95). Rasch scaling, residual factor, differential item functioning, and modified parallel analyses supported the unidimensionality and goodness of fit of each unique activity domain.

Haley SM, Langmuir L. How do current post-acute functional assessments compare with the activity dimension of the International Classification of Functioning and Disability (ICIDH-2)? J Rehabil Outcomes Meas. 2000; 4:51-56.

(No abstract available.)

Haley SM, Ni P, Coster WJ, Black-Schaffer R, Siebens H, Tao W. Agreement in functional assessment: graphic approaches to displaying respondent effects. Am J Phys Med Rehabil. 2006; 85(9):747-55.

The objective of this study was to examine the agreement between respondents of summary scores from items representing three functional content areas (physical and mobility, personal care and instrumental, applied cognition) within the Activity Measure for Post-Acute Care (AM-PAC). We compare proxy vs. patient report in both hospital and community settings as represented by intraclass correlation coefficients and two graphic approaches.

Haley SM, Ni P, Hambleton RK, Slavin MD, Jette AM.  Computer adaptive testing improves accuracy and precision of scores over random item selection in a physical functioning item bank.  J Clin Epidemiol. 2006; 59: 1174-1182.

Measuring physical functioning (PF) within and across post-acute settings is critical for monitoring outcomes of rehabilitation; however, most current instruments lack sufficient breadth and feasibility for widespread use. Computer adaptive testing (CAT), in which item selection is tailored to the individual patient, holds promise for reducing response burden, yet maintaining measurement precision. We calibrated a PF item bank via item response theory (IRT), administered items with a post hoc CAT design, and determined whether CAT would improve accuracy and precision of score estimates over random item selection.

Haley SM, Siebens H, Coster WJ, Tao W, Black-Schaffer RM, Gandek B, Sinclair SJ, Ni P. Computerized adaptive testing for follow-up after discharge from inpatient rehabilitation: I. Activity outcomes. Arch Phys Med. 2006; 87:1033-1042.

To examine score agreement, precision, validity, efficiency, and responsiveness of a computerized adaptive testing (CAT) version of the Activity Measure for Post-Acute Care (AM-PAC-CAT) in a prospective, 3-month follow-up sample of inpatient rehabilitation patients recently discharged home.

Halm EA, Magaziner J, Hannan EL, Wang JJ, Silberzweig SB, Boockvar K, Orosz GM, McLaughlin MA, Koval KJ, Siu AL. Frequency and impact of active clinical issues and new impairments on hospital discharge in patients with hip fracture. Arch Intern Med. 2003; 163(1):108-13.

Hip fracture is associated with significant mortality and disability. Patients who are discharged from the hospital with active clinical problems may have worse outcomes than those patients without active clinical problems. The objective of this study was to assess the frequency and impact of clinical problems at discharge on clinical and functional hip fracture outcomes.

Hart DL. On “Prospective Evaluation of the AM-PAC-CAT…” Jette et al. Phys Ther. 2007;87:385-398. Phys Ther. 2007 May;87(5):609-11; author reply 611.

(No abstract available.)

Jette A, Haley S, Ni P: A Comparison of Functional Status Tools used in Post-Acute Care.  Health Care Fin Review. 2003; 24(3).

There is a growing health policy mandate for comprehensive monitoring of functional outcomes across post-acute care (PAC) settings.  This article presents an empirical comparison of four functional outcome instruments used in PAC with respect to their content, breadth of coverage, and measurement precision.  Results illustrate limitations in the range of content, breadth of coverage, and measurement precision in each outcome instrument.  None appears well equipped to meet the challenge of monitoring quality and functional outcomes across settings where PAC is provided. Limitations in existing assessment methodology has stimulated the development of more comprehensive outcome assessment systems specifically for monitoring the quality of services provided to PAC patients.

Jette A, Haley S, Tao W, Ni P, Meyers D, Zurek M. Prospective evaluation of the AM-PAC-CAT in outpatient rehabilitation settings.  Phys Ther. 2007; 87(4):385-398.

The purpose of this study was to prospectively evaluate the practical and psychometric adequacy of the Activity Measure for Post-Acute Care (AM-PAC) “item bank” and computerized adaptive testing (CAT) assessment platform (AM-PAC-CAT) when applied within orthopedic outpatient physical therapy settings.

Jette A, Haley S. Contemporary measurement techniques for rehabilitation outcome assessment. J Rehab Med. 2005; 37(6):339-345.

In this article, we review the limitations of traditional rehabilitation functional outcome instruments currently in use within the rehabilitation field to assess Activity and Participation domains as defined by the International Classification of Function, Disability, and Health. These include a narrow scope of functional outcomes, data incompatibility across instruments, and the precision vs feasibility dilemma. Following this, we illustrate how contemporary measurement techniques, such as item response theory methods combined with computer adaptive testing methodology, can be applied in rehabilitation to design functional outcome instruments that are comprehensive in scope, accurate, allow for compatibility across instruments, and are sensitive to clinically important change without sacrificing their feasibility. Finally, we present some of the pressing challenges that need to be overcome to provide effective dissemination and training assistance to ensure that current and future generations of rehabilitation professionals are familiar with and skilled in the application of contemporary outcomes measurement.

Jette A, Tao W, Haley S. Blending activity and participation sub-domains of the ICF. Disabil Rehabil. 2007; 29(22):1742-1750.

The objective of the study was to test the supposition that distinct Activity and Participation sub-domains of the International Classification of Functioning, Disability, and Health (ICF) could be identified using physical function questionnaire items drawn from the Activity and Participation Measures for Post Acute Care.

Jette A, Tao W, Norweg A, Haley S. Interpreting rehabilitation outcome measurements. J Rehabil Med. 2007; 39(8):585-90.

With the increased use of standardized outcome instruments in rehabilitation, questions frequently arise as to how to interpret the scores that are derived from these standardized outcome instruments. This article uses examples drawn from the Activity Measure for Post Acute Care to illustrate 4 different data analysis and presentation strategies that can be used to yield meaningful outcome data for use in rehabilitation research and practice.

Jette AM, Haley SM, Ni P, Moed R. Toward functional outcome assessment across post-acute care settings. Am J Phys Med Rehabil. In press, 2008.

(Abstract not available.)

Jette AM, Haley SM, Ni P, Moed R. Adaptive short-forms for outpatient rehabilitation outcome assessment. Am J Phys Med Rehabil. 2008; 87(10):842-52.

To develop outpatient Adaptive Short Forms for the Activity Measure for Post-Acute Care item bank for use in outpatient therapy settings.

Jette AM, Haley SM. Achieving Uniformity in Functional Status Monitoring across PAC Settings.  Centers for Medicare and Medicaid. 2006.

(Abstract not available.)

Jette AM, Haley SM.  Longitudinal outcome monitoring across Post-acute care (PAC) settings.  In: Uniform Patient Assessment for Post-acute Care.  Final Report.  Division of Health Care Policy and Research, University of Colorado, 2006, p 100-120.

The purpose of this chapter is to introduce the reader to contemporary methods for developing and evaluating outcomes measures which, we believe, provide the CMS with a methodology that can be implemented to track outcomes and quality of care provided across entire episodes of PAC.  In the first section of this chapter we will summarize the major limitations in traditional methodology for assessing health outcomes.  We will then briefly introduce contemporary measurement concepts of Item Response Theory (IRT) and Computer Adaptive Testing (CAT) methods as alternatives to the traditional approaches used to assess and monitor PAC outcomes.  Finally, we will explain how contemporary techniques for measuring outcomes can be used as a strategy to achieve episode-wide assessment in PAC and discuss some recommendations to CMS for implementing episode-wide outcomes assessment and monitoring in PAC.

Jette AM, Keysor J, Coster W, Ni P, Haley S. Beyond function: predicting participation in a rehabilitation cohort. Arch Phys Med Rehabil. 2005; 86(11):2087-94.

The objective of this study was to monitor participation in a rehabilitation cohort and to identify determinants of change during a 12-month period posthospitalization following the onset of one of several major disabling conditions.

Jette AM, Norweg A, Haley S. Achieving meaningful measurements of ICF concepts. Disabil Rehabil. 2008; 30(12-13):963-9.

This paper reviews the strengths and weaknesses of two different approaches to assessing ICF concepts: coding versus quantitative scales. It illustrates the advantages of an alternative, integrative approach, called functional staging.

Jette AM. Disablement models: Toward a common language for function, disability, and health. Phys Ther. 2006; 86(5):726-734.

Within physical therapy, the disablement model has proven useful as a language to delineate the consequences of disease and injury. This perspective provides an update on the changing language of disablement, reviews selected contemporary disablement models, and discusses some challenges that need to be addressed to achieve a universal disablement language that can be used to discuss physical therapy research and clinical interventions. The World Health Organization’s International Classification of Functioning, Disability and Health (ICF) framework has the potential of becoming a standard for disablement language that looks beyond mortality and disease to focus on how people live with their conditions. If widely adopted, the ICF framework could provide the rehabilitation field with a common, international language with the potential to facilitate communication and scholarly discourse across disciplines and national boundaries, to stimulate interdisciplinary research, to improve clinical care, and ultimately to better inform health policy and management.

Jette, A, Haley S, Kooyoomjian J. Are the ICF Activity and Participation Dimensions Distinct? J Rehabil Med. 2003; 35:145-149.

The objective of this study was to test the hypothesis that distinct Activity and Participation dimensions of the International Classification of Functioning, Disability, and Health could be identified using physical functioning items drawn from the Late Life Function and Disability Instrument.

Labarere J, Bosson JL, Sevestre MA, Sellier E, Richaud C, Legagneux A. Intervention targeted at nurses to improve venous thromboprophylaxis. Int J Qual Health Care. 2007; 19(5):301-8.

The objective of this study was to assess the effectiveness of an intervention targeting both physicians and nurses vs. physicians only in improving venous thromboprophylaxis for older patients.

Larivière N. Analysis of the concept of social participation: definitions, illustration, dimensions of activity and indicators. Can J Occup Ther. 2008; 75(2):114-27.

French. Social participation is an integral part of human life. This concept has appeared in recent health literature and is considered one of the main goals of rehabilitation. The purpose of this article is to clarify the concept of social participation and its applicability in mental health. The concept of social participation is distinguished from other related concepts and measurement tools assessing social participation are examined.

Latham N, Haley SM.  Measuring functional outcomes across post-acute care: Current challenges and future directions.  Phys Med State of the Art Reviews. 2003; 15:83-98.

This article reviews the challenges of developing outcome systems that are designed to assess function across post-acute care settings and describes state-of-the-art research that is helping produce the next generation of functional instruments. There is growing recognition of the importance of measuring function across post-acute settings; however, current limitations of existing instruments include (1) the lack of a clear theoretical framework, (2) a narrow scope of content, (3) the fragmentation of instruments across settings, and (4) a lack of feasibility. The authors highlight the key methods, such as item response theory, and introduce some preliminary findings in computerized adaptive testing applications that will guide and stimulate future research in this area.

Latham NK, Mehta V, Nguyen AM, Jette AM, Olarsch S, Papanicolaou D, Chandler J. Performance-based or self-report measures of physical function: which should be used in clinical trials of hip fracture patients? Arch Phys Med Rehabil. 2008; 89(11):2146-55.

The objective of this study was to assess the validity, sensitivity to change, and responsiveness of 3 self-report and 4 performance-based measures of physical function: activity measure for post-acute care (AM-PAC) Physical Mobility and Personal Care scales, the Medical Outcomes Study 36-Item Short Form Health Survey Physical Function scale (SF-36 PF), the Physical Functional Performance test (PFP-10), the Short Physical Performance Battery (SPPB), a 4-meter gait speed, and the six-minute walk test (6MWT).

Motin M, Keren O, Ring H. Relationship between type and amount of treatment and functional improvement in first-stroke during in-patients rehabilitation. Harefuah. 2001; 140(12):1127-33, 1232.

Hebrew. The effectiveness of ongoing rehabilitation services for post-acute stroke patients is poorly documented. The aim of the present study was describe the relationships between functional status at discharge and intensity of therapies, including occupational therapy, physical therapy, speech therapy and nursing care, during inpatient medical rehabilitation.

Prvu Bettger JA, Coster WJ, Latham NK, Keysor JJ. Analyzing change in recovery patterns in the year after acute hospitalization. Arch Phys Med Rehabil. 2008; 89(7):1267-75.

The objective of this study was to examine trajectories of recovery and change in patterns of personal care and instrumental functional activity performance to determine whether different assessment interval designs within a 12-month period yield different estimates of improvement and decline after acute hospitalization and inpatient rehabilitation.

Siebens H, Andres PL, Pengsheng N, Coster WJ, Haley SM. Measuring physical function in patients with complex medical and postsurgical conditions: a computer adaptive approach. Am J Phys Med Rehabil. 2005; 84(10):741-8.

The objective of this study was to examine whether the range of disability in the medically complex and postsurgical populations receiving rehabilitation is adequately sampled by the new Activity Measure-Post-Acute Care (AM-PAC), and to assess whether computer adaptive testing (CAT) can derive valid patient scores using fewer questions.

Tao W, Haley S, Coster W, Ni P, Jette A. An Exploratory Analysis of Functional Staging Using an Item Response Theory Approach. Arch Phys Med Rehabil. 2008; 89(6); 1046-1053.

To develop and explore the feasibility of a functional staging system (defined as the
process of assigning subjects, according to predetermined standards, into a set of hierarchic levels with regard to their functioning performance in mobility, daily activities, and cognitive skills) based on item response theory (IRT) methods using short forms of the Activity Measure for Post-Acute Care (AM-PAC) and to compare the criterion validity and sensitivity of the IRT-based staging system to a non–IRT-based staging system developed for the FIM instrument.