Using
the time and motion method to study clinical work processes and
workflow:methodological inconsistencies and a call for standardized research
Author:
Kai Zheng
Objective
To identify ways for improving the consistency of design, conduct, and results reporting
of time and motion (T&M) research in health informatics.
Materials
and methods We analyzed the commonalities and
divergences of empirical studiespublished 1990e2010 that have applied the
T&M approach to examine the impact of health IT implementation on clinical
work processes and workflow. The analysis led to the development of a suggested ‘checklist’
intended to help future T&M research produce compatible and comparable
results. We call this checklist STAMP (Suggested Time And Motion Procedures).
Results
STAMP outlines a minimum set of 29 data/ information elements organized into
eight key areas, plus three supplemental elements contained in an ‘Ancillary Data’
area, that researchers may consider collecting and reporting in their future
T&M endeavors.
Discussion
T&M is generally regarded as the most reliable approach for assessing the
impact of health IT implementation on clinical work. However, there exist considerable
inconsistencies in how previous T&M studies were conducted and/or how their
results were reported, many of which do not seem necessary yet can have a
significant impact on quality of research and generalisability of results.
Therefore, we deem it is time to call for standards that can help improve the consistency
of T&M research in health informatics. This study represents an initial
attempt.
INTRODUCTION
While there has been a widely acknowledged consensus
that health IT implementation often introduces radical changes to clinical work
processes and workflow, 1e4 it remains unclear what these changes are and how
they impact actual clinical efficiency, team coordination, and ultimately
quality of care and patient safety. Developing such an understanding requires
rigorously conducted research that can generate compatible and comparable
results to inform effective technology designs and implementation approaches.
Methods for studying
changes to work processes and workflow vary widely depending on research contexts
and research objectives. In this paper, we focus on quantitative approaches for
measuring the impacts of such changes by means of quantifying clinicians’ time
utilization and delineating how their time is allocated to different types of clinical
and non-clinical activities. Among several approaches commonly used to date,
time and motion (T&M), which involves continuous and independent observation
of clinicians’ work, is generally regarded as the most reliable method compared
to alternative methods such as work sampling and time efficiency questionnaires.
In recent years, informatics research utilizing T&M has grown substantially
thanks to a series opioneering paper7 8 that led to a widely-used T&M data
acquisition tool made available through the Agency for Healthcare Research and
Quality.9.
METHOD
As the focus of this research is on informatics
studies conducted in health and healthcare domains, we limited the literature
search to PubMed/MEDLINE only. The search was also restricted to empirical studies
published in English within the last 20 years (1990e2010). Further, we applied
two salient inclusion/exclusion criteria in study selection, namely: (1) the quantitative
method employed by the study must be T&M, and the T&M method used must
conform to the following definition: ‘independent and continuous observation of
clinicians work to record the time required to perform a series of clinical or
non-clinical activities.’ 5 6 With this definition, we excluded work sampling
research, ‘time’ studies for collecting efficiency measures on isolated events
(eg, medication turnaround time),
DISCUSSION
In addition to the issues mentioned above, our
review of theliterature also raised two other concerns. First, that reports
common reasons for paper exclusion, the term‘time and motion’ may have been
overused as it appears frequently in studies that did not actually use the
T&M methodaccording to the prevalent ; examples include work sampling
studies, studies on turnaround time, and questionnaire surveys. This issue
could create chaos in knowledge accumulation and cause unnecessary difficulties
for researchers conducting systematic reviews and meta-analyses. Second, there does
not seem to be a standard way to train T&M observers or to calibrate
inter-observer disagreements (if applicable). This fact may exacerbate
potential observer biases and undermine the validity of cross-study research
synthesis. Developing a standard observer training set, possibly in the form of
video recordings of typical clinical sessions annotated by T&M experts,
would therefore be highly valuable.
CONCLUSION
We reviewed existing empirical studies that have
applied the T&M approach to evaluate the impact of health IT implementation
on clinical work processes and workflow. Based on the results, we developed a
suggested ‘checklist’ intended to help standardize the design, conduct, and
results reporting of future T&M studies. We believe that the resulting
checklist, STAMP (Standard Time And Motion Procedures), may contribute to improving
the methodological and results reporting consistency of T&M research, so
that meaningful insights can be derived from across-study synthesis and health
informatics, as a field, will be able to accumulate knowledge from these
studies.
Mamta Sahare 44
