Full text of DOI:10.1111/j.1553-2712.2001.tb01141.x

1200
HAND-HELD COMPUTER
Bird et al. • RESIDENT DOCUMENTATION
hiring, privileging, and promotion
processes. The Residency Review
Committee (RRC) for Emergency
Medicine requires that residency
programs document residents’ expe-
rience with ‘‘invasive procedures’’
and ‘‘major resuscitations of all age
BRIEF REPORTS
Emergency Medicine Resident Patient Care
Documentation Using a Hand-held
Computerized Device
1
groups.’’
S
TEVEN B. BIRD, MD, ROBERT S. ZARUM, MD,
Recent technologic advances
have made it possible to build small,
hand-held computers (commonly re-
ferred to as personal digital assis-
tants—or PDAs) that are reliable
and durable. Data entered into the
PDA are easily downloaded, or
FRANCIS P. RENZI, MD
Abstract. Objective: To determine whether emergency medicine
EM) resident documentation of procedures, patient encounters, and
(
patient follow-ups improved after implementation of a personal digital
assistant (PDA) hand-held recording system. Methods: All first-year
EM residents were provided a PalmV (Palm, Inc., Santa Clara, CA)
PDA. A customized patient procedure and encounter program was con-
structed using Pendragon Forms (Pendragon Software Corporation,
Libertyville, IL) and loaded into each PDA. Residents were instructed
to enter information on patients who had any of 21 procedures per-
formed or were considered to be clinically unstable. These data were
downloaded to the residency coordinator’s desktop computer. The
mean number of procedures, encounters, and follow-ups performed per
resident were then compared with those of a group of 36 historical
controls from the three previous first-year resident classes who re-
corded the same information using a handwritten card system. Data
from the historical controls were combined and the means of each
group were compared by Student’s t-test. Results: Mean documenta-
tion of three procedures was significantly increased in the PDA group
versus the index card system: conscious sedation 5.8 vs. 0.03 (p <
‘
‘synced,’’ into a desktop computer.
Some residency programs have be-
gun using PDAs to store procedural
and other data, replacing traditional
handwritten index cards and log-
books. A common problem with
any method of data tracking, how-
2
,3
4
ever, is noncompliance by residents.
The objective of this study was to
examine first-year EM resident doc-
umentation of 21 procedures, four
patient resuscitation categories, and
patient follow-ups after implemen-
tation of a PDA system, and compare
the data with those recorded by a
historical control of residents using
a traditional handwritten procedure
log.
0.000005), thoracentesis 2.2 vs. 0.0 (p = 0.002), ultrasound 6.3 vs. 0.0
(
p = 0.002). The mean numbers of pericardiocenteses and unstable
pediatric surgical patient evaluations were significantly decreased in
the hand-held group [from 1.2 to 0.4 (p = 0.03) and from 9.1 to 2.2 (p
=
0.02), respectively]. Patient follow-up documentations were not sta-
tistically different between the two groups. Conclusions: Use of a
hand-held PDA was associated with an increase in first-year EM res-
ident documentation in three of 20 procedures and a decrease in one
procedure and the number of unstable surgical pediatric patient re-
suscitations. The overall time savings in constructing a resident pro-
cedure database, as well as the other uses of the PDAs, may make
transition to a hand-held computer-based procedure log an attractive
option for EM residencies. Key words: emergency medicine; procedure;
resident; documentation; computer. ACADEMIC EMERGENCY MED-
ICINE 2001; 8:1200–1203
Emergency medicine (EM) proce- cations, and deaths, it may be ex-
dural documentation is of increasing pected that perceived objective mea-
importance. With current national sures of a physician’s skills will
attention on medical errors, compli- become important in the licensing,
From the Department of Emergency dragon Forms software, and manuscript
Medicine, University of Massachusetts development was provided by the De-
Medical School (SBB, RSZ, FPR), Wor- partment of Emergency Medicine, Uni-
cester, MA.
Received May 15, 2001; revision received Address for correspondence and reprints:
July 17, 2001; accepted August 2, 2001. Steven B. Bird, MD, Department of
Presented in part at the SAEM annual Emergency Medicine, 55 Lake Avenue
meeting, Atlanta, GA, May 2001.
Support for the PalmV computers, Pen- 856-6902; e-mail: birds@ummhc.org
versity of Massachusetts Medical School.
Figure 1. The customizable data col-
North, Worcester, MA 01655. Fax: 508- lection application installed into the
hand-held personal digital assistant.

ACADEMIC EMERGENCY MEDICINE • December 2001, Volume 8, Number 12
1201
T
ABLE 1. Mean Numbers of Procedures, Unstable Patient Evaluations, and
METHODS
Patient Follow-ups per Resident during the PGY1 Year
Study Design. This was a pro-
spective study involving a retrospec-
tive cohort that analyzes documen-
Study Group
Mean (Range)
5% Confidence
Interval
Cohort
Mean (Range)
95% Confidence
Interval
9
Procedure
Arthrocentesis
p-value
tation
of
procedures,
patient
resuscitations, and patient follow-
ups by EM residents during their
1.5 (0–4)
0.8, 2.2
1.0 (0–10)
0.4, 1.7
0.36
0.09
first year. This study was considered Cardioversion/defibrillation
exempt from review by the Institu-
tional Review Board of the Univer-
3.3 (0–14)
0.8, 5.8
30.6 (8–58)
6.2 (0–23)
4.3, 8.1
28.9 (1–102)
22.7, 35.1
0.0 (0–1)
Central catheter
0.76
2
1.8, 39.4
sity of Massachusetts Medical
Conscious sedation
School.
5.8 (3–9)
<0.0000005
1.0
4
.5, 7.1
Cricothyroidotomy
0.3 (0–2)
0, 0.7
0.3 (0–2)
Ϫ0.1, 0.7
5.2 (0–9)
0.3 (0–3)
0.1, 0.5
0.1 (0–1)
Ϫ0.1, 0.1
4.2 (0–25)
2.7, 5.7
47.1 (3–103)
40.5, 53.7
2.5 (0–12)
1.5, 3.5
14.1 (0–53)
9.6, 18.6
1.5 (0–7)
0.8, 2.2
73.8 (2–218)
58.2, 89.4
11.2 (2–28)
8.9, 13.5
3.7 (0–28)
1.9, 5.5
Study Setting and Population.
This study was conducted at the
Diagnostic peritoneal lavage
0.25
University of Massachusetts Medi-
cal Center, in Worcester, Massachu-
Dislocation reduction
0.42
3
.7, 6.7
setts. The EM residency is an RRC-
accredited three-year program with
Endotracheal intubation
54.8 (35–101)
0.22
4
4.8, 64.8
1
2 residents per year. The Univer-
External cardiac pacing
2.3 (0–8)
0.86
sity of Massachusetts Medical Cen-
0
.7, 3.9
ter is a level I trauma center with Fracture reduction/splinting
an annual emergency department
14.5 (1–45)
7.4, 21.6
1.4 (0–4)
0.92
(
ED) census of 75,000 visits. The sec- Internal cardiac pacing
0.92
0
.7, 2.1
68.2 (30–154)
7.6, 88.8
10.4 (2–19)
.7, 13.1
1.9 (0–4)
.0, 2.8
ondary training site in Worcester
has a volume of 55,000 emergency
visits per year. Residents were in-
structed to enter pertinent informa-
tion as soon as possible following the
procedure or resuscitation. Docu-
mentation from the 12 monthly ro-
tations was collected. Residents
Laceration repair
Lumbar puncture
Nasal packing
0.67
4
0.67
7
0.09
1
Pericardiocentesis
0.4 (0–1)
0.1, 0.7
1.2 (0–11)
0.5, 1.9
0.03
were not explicitly told to update Thoracentesis
their PDAs before the study was
2.2 (0–6)
1.2, 3.2
0.2 (0–1)
0.0 (0)
0.002
0.44
Thoracotomy
conducted.
0.3 (0–6)
Ϫ0.1, 0.7
4.4 (0–12)
3.4, 5.4
0
, 0.4
3.6 (0–5)
.8, 4.4
Tube thoracostomy
0.22
Study Protocol. Upon beginning
2
his or her residency in 1999, each Ultrasound
6.3 (0–16)
3.4, 9.2
29.8 (0–49)
0.0 (0)
0.002
0.84
resident was provided
(
a PalmV
Palm, Inc., Santa Clara, CA) PDA. Vaginal delivery
28.9 (0–103)
23.4, 34.4
309.6 (0–1,116)
214.7, 404.5
64.9 (0–421)
37.0, 92.8
25.8 (0–172)
13.4, 38.2
9.1 (0–58)
4.1, 14.1
2
2.4, 37.2
A customizable data collection appli-
cation was constructed by one of the
authors (RZ) using Pendragon
Forms 3.0 (Pendragon Software Cor-
poration, Libertyville, IL), and in-
stalled into every PDA. The Pen-
dragon Forms input key was
designed to fit on a single PDA
screen (Fig. 1).
Unstable adult medical
Unstable adult surgical
Unstable pediatric medical
Unstable pediatric surgical
Follow-ups
200.6 (66–501)
36.5, 264.7
30.8 (0–128)
0.07
1
0.07
8
.7, 52.9
16.0 (0–58)
.5, 26.5
0.24
5
2.2 (0–10)
0.4, 4.0
25.4 (0–201)
0.02
9.1 (0–78)
3.1, 15.1
0.35
Data from the three previous
first-year resident classes—a total of
Ϫ6.8, 57.6
3
6 residents—were reviewed: this
constituted the retrospective cohort. into a computer database by our res- Then, from pull-down menus the fol-
Previously, residents in this EM pro- idency coordinator.
gram were required to track 20 pro-
cedures, four types of resuscitations, Measurements.
lowing data were entered: follow-up
call (‘‘yes’’ or ‘‘no’’); patient status
the (‘‘stable,’’ ‘‘unstable adult medical,’’
Using
and patient follow-ups using a tra- PalmV’s handwriting recognition ca- ‘‘unstable adult surgical,’’ ‘‘unstable
ditional handwritten index card sys- pability, the residents were required pediatric medical,’’ or ‘‘unstable pe-
tem (the retrospective cohort did not to enter the specific rotation, date, diatric surgical’’); and procedures
track nasotracheal intubations). patient’s initials, six-digit medical performed, up to a maximum of four.
These data were then transcribed record number, age, and diagnosis.
The 20 procedures tracked are

1202
HAND-HELD COMPUTER
Bird et al. • RESIDENT DOCUMENTATION
shown in Table 1. For the study were not statistically different be- creased fashion: conscious sedation,
group, the nasotracheal and endo- tween the two groups.
tracheal intubations were combined,
thoracentesis, and ultrasound. It
may be possible to explain these
findings in part based on recent dy-
namic changes in ED care. Thora-
centesis, formerly performed for di-
agnostic and therapeutic purposes
on an inpatient basis, is now rou-
tinely done in the ED because of the
prolonged time that patients wait
for an inpatient hospital bed. The in-
creased acceptance of ultrasound
and conscious sedation routinely
performed by emergency physicians
likely explains the increase in the
documentation of those procedures
within the study group.
and are listed as endotracheal intu-
bations.
The residents were instructed to
download (or ‘‘synch’’) the data from
their PalmV to our residency coor-
dinator’s desktop computer database
during our weekly EM conference.
This synchronization process takes
approximately 30 seconds.
D
ISCUSSION
While several authors have reported
computerized tracking of resident
2–6
procedures, this study represents
the first compilation of EM resident
procedure tracking using hand-held
PDAs. Documentation via these de-
vices offers several advantages over
traditional procedural logbooks.
First, the small size of the device
makes it convenient to carry. The
ease of use of the software program
Data Analysis. Data from the
study group and retrospective cohort
were entered into a Microsoft Excel
(
Microsoft Corporation, Redmond,
(
as written by one author) also al-
WA) worksheet. Data from the ret-
rospective cohort were combined,
with means, standard errors, and
lows procedures to be entered
quickly into the PDA. Yet another
advantage of a PDA, one that is still
in evolution, is the addition of other
software programs (such as drug in-
formation, medical calculators, etc.)
for use in the ED. Finally, the ease
with which the data are synchro-
nized to our residency coordinator’s
computer database saves approxi-
mately five hours of secretarial time
per month and eliminates transcrip-
tion errors.
L
IMITATIONS AND
FUTURE
QUESTIONS
95% confidence intervals calculated
for all measurements. Means are
representative of the number of pro-
cedures, unstable patients, and fol-
low-ups per resident for the entire
year. These cohort and study group
means were compared using a two-
tailed Student’s t-test. Given the 25
total comparisons between the study
and cohort groups, a Bonferroni cor-
rection for multiple comparisons
was performed. The upper limit of
statistical significance maintaining
an alpha error of 0.05 is therefore p
While this study demonstrated a sig-
nificant increase in the incidence of
documentation of three out of 20
procedures in the PDA group, it is
not possible to determine with cer-
tainty whether the observed in-
crease in the study group was due to
more fastidious documentation on
the part of the residents, or simply
to a greater number of procedures
performed by them.
As with any study involving doc-
umentation compliance, some data
may be incomplete. Twenty-one of
the 25 comparisons from the retro-
spective cohort and 18 of 25 from the
study group involve a lower range of
zero, while none of the study groups’
means, and three of the cohorts’
means, were zero. Given the capri-
ciousness of resident documentation,
these numbers are not too surpris-
ing. The ease with which any partic-
Hand-held computers, however,
are not without limitations. The in-
itial cost of implementing the PDAs
into the EM program approached
=
0.002, rather than 0.05.
$
5,000 (approximately $350 per
As the RRC-EM and the Council
PalmV, and several hundred dollars
for the Pendragon Forms multiuser
license). Additionally, there are oc-
casional malfunctions in the PDAs,
which can lead to loss of data. Fre-
quent synchronization with the
desktop computer, however, de-
creases the risk of data loss. Perhaps
the greatest limitation is resident
compliance with documentation.
of Residency Directors for Emer-
gency Medicine have not set explicit
standards for the number of inva-
sive procedures required to attain
competence, no assumption of what
would constitute a clinically mean-
ingful change in procedure perfor-
mance was made.
RESULTS
Previous studies have shown proce- ular resident’s procedure log may be
queried by the residency director us-
ing the PDA database may allow
dural and resuscitation documenta-
The average numbers of procedures, tion capture rates of 27–84%.
6
–8
unstable patient evaluations, and While this study was not intended to more timely feedback to the resi-
patient follow-ups per resident from address the issue of procedural cap- dent, thus allowing correction of doc-
the study and cohort groups are pre- ture, it has been our experience that umentation shortfalls.
sented in Table 1. Mean documen- those residents who are fastidious
Another limitation to this inves-
tation of three procedures increased with manual documentation are tigation is the small study group
significantly in the study group ver- generally better with computerized size. The limited number of subjects
sus the cohort: conscious sedation documentation as well. Those resi- in the study group, in conjunction
5
.8 vs. 0.03, thoracentesis 2.2 vs. 0.0, dents who are poorly compliant with with the Bonferroni correction for
and ultrasound 6.3 vs. 0.0. The documentation will continue to be multiple comparisons, makes a type
mean number of pericardiocenteses so, regardless of the method offered II error a distinct possibility.
was significantly decreased in the to them.
Perhaps the most important is-
hand-held group (from 1.2 to 0.4), as
Of the 20 procedures tracked in sue regarding performance of pro-
were unstable pediatric surgical pa- both the study and retrospective co- cedures, which could not be ad-
tient encounters (from 9.1 to 2.2). hort groups, three were documented dressed in this study, is what
Patient follow-up documentations in
a statistically significant in- constitutes procedural competence?

ACADEMIC EMERGENCY MEDICINE • December 2001, Volume 8, Number 12
1203
5
. Langdorf, MI, Strange G. Computer-
Several authors have sought to an attractive option to EM residen-
quantify competence for isolated cies.
ized tracking of emergency medicine res-
ident clinical experience. Ann Emerg
Med. 1990; 19:764–73.
9
,10
skills,
but no consensus exists on
how to fully determine an EM resi- The authors thank Carol Bloom, resi-
6. Langdorf MI, Bontague BJ, Bearie B,
dent’s clinical or procedural compe- dency coordinator for the Department of Sobel CS. Quantification of procedures
tence. Further additions to the PDA Emergency Medicine at the University of
and resuscitations in an emergency med-
Massachusetts, for her help with the icine residency. J Emerg Med. 1998; 16:
software program to allow recording
1
21–7.
handwritten and computerized proce-
dure logs.
of procedural complications and at-
tending physician attestations re-
garding the proficiency with which a
procedure or skill was completed
might allow the question of compe-
tency to be more fully addressed.
7
. Ray VG, Garrison HG. Clinical pro-
cedures performed by emergency medi-
cine resident physicians: a computer-
based model for documentation. J Emerg
Med. 1991; 9:157–9.
References
8
. Baldor RA, Broadhurst J. Document-
1
. Graduate Medical Education Direc-
ing resident procedure and diagnostic ex-
perience: simplifying the process. Fam
tory. Program Requirements. Chicago,
IL: American Medical Association, 2001.
CONCLUSIONS
2. Garvin R, Otto F, McRae D. Using Med. 1997; 29:629–33.
9
. Chapman DM, Cavanaugh SH. Using
handheld computers to document family
practice resident procedure experience.
Fam Med. 2000; 32:115–8.
receiver operating characteristic (ROC)
analysis to establish the previous expe-
rience threshold for critical-procedure
competency. Acad Med. 1996; 71(10
Use of a hand-held computer device
as an EM resident procedure log
compared favorably with traditional
3
. Rosenthal M, Wolford RW. Resident
procedure and resuscitation tracking us-
handwritten logbooks. The time sav- ing a Palm computer [abstract]. Acad suppl):S7–S9.
Emerg Med. 2000; 7:1171.
. Dire DJ, Kietzman LI. A prospective ML, Verhulst SJ. Using a receiver oper-
10. Colliver JA, Barnhart AJ, Marcy
ings in construction of a resident
procedure database, as well as the
numerous other medical uses of a
Palm device, may make the transi-
4
survey of procedures performed by emer- ating characteristic (ROC) analysis to set
gency medicine residents during a 36- passing standards for a standardized-pa-
month residency. J Emerg Med. 1995; 13: tient examination of clinical competence.
tion to a PDA-based procedural log 831–7.
Acad Med. 1994; 69:S37–S39.
ⅷ