Full text of DOI:10.1093/gerona/56.7.M405

Journal of Gerontology: MEDICAL SCIENCES
Copyright 2001 by The Gerontological Society of America
2001, Vol. 56A, No. 7, M405–M411
Accuracy and Bias of Licensed Practical Nurse and
Nursing Assistant Ratings of Nursing Home
Residents’ Pain
Veronica F. Engle, Marshall J. Graney, and Anna Chan
University of Tennessee Health Science Center, College of Nursing and College of Medicine, Memphis.
Background. This study evaluated the accuracy of licensed practical nurses’ (LPN) and nursing assistants’ (NA)
Minimum Data Set (MDS) pain ratings of nursing home residents and evaluated the bias in pain ratings associated with
residents’ race, gender, mental status, function, depression, or disruptive behavior.
Methods. Data were obtained on the same day directly from residents, LPNs, and NAs by trained interviewers in two
safety-net nursing homes. A total of 252 residents were included in this study: 79% were Black, and 60% were men.
MDS items J2a and J2b evaluated pain frequency and pain intensity during the last 7 days (weekly pain frequency and
weekly pain intensity). A parallel question evaluated pain intensity on the day of the interview (daily pain intensity).
MDS data were obtained for the MDS Cognition Scale, the MDS Activities of Daily Living-Long Form Scale, the MDS
Depression Rating Scale, and the MDS Disruptive Behavior Scale.
Results. Kappa coefficients documented fair to good resident–LPN (K ꢀ .70, .56, and .50) and resident–NA (K ꢀ
.72, .58, and .60) agreement for weekly pain frequency, weekly pain intensity, and daily pain intensity ratings. LPNs and
NAs underestimated residents’ weekly pain frequency (p ꢁ .001 for LPNs, and p ꢁ .001 for NAs), weekly pain inten-
sity (p ꢁ .001 for LPNs, and p ꢁ .001 for NAs), and daily pain intensity (p ꢁ .001 for LPNs, and p ꢀ .002 for NAs).
LPNs underestimated weekly and daily pain intensity more than NAs did (p ꢀ .016 for weekly pain intensity, and p ꢀ
.035 for daily pain intensity). LPN and NA pain ratings were not biased by resident race, gender, mental status, function,
depression, or disruptive behavior.
Conclusions. Results documented that (i) LPNs and NAs underestimated residents’ pain frequency and pain inten-
sity, (ii) NAs were more accurate than LPNs for pain intensity, and (iii) resident characteristics did not bias LPN or NA
pain ratings.
LTHOUGH approximately 80% of nursing home resi-
dents can provide meaningful information about their
idents’ pain were not related to their administration of pain
medication (14). Physicians likewise failed to detect the
presence of residents’ pain, particularly the pain of residents
with neurological disorders (15). Nursing staff and residents
may also discount chronic pain (15).
Undertreatment of nursing home residents’ pain may be
due, in part, to the nursing home system of care. Most care
is provided by nursing assistants (NAs) (16) who may have
limited ability to assess residents’ pain and to communicate
their ratings to the licensed practical nurse (LPN) unit
“charge nurse” or to the registered nurse (RN) supervisor.
NAs are rarely involved in formal care planning and rarely
have knowledge of professional goals for resident care (17),
yet NAs are able to accurately assess ADLs (18) and symp-
toms such as edema and shortness of breath (19). Compared
with NAs, RNs are more often involved with administrative
and documentation activities (17).
Results are equivocal regarding bias in nursing home
staff pain ratings (not using the MDS) compared with resi-
dents’ self-reports (10,13,15,20). Little attention has been
given to the focal role of the NA as a key informant with
knowledge and experience vital to MDS pain rating. De-
spite differences between licensed nursing staff (RN and
LPN) and NAs in education and in the amount of direct con-
tact with residents, many studies do not differentiate be-
A
pain (1), pain is frequently undertreated in nursing homes.
Undertreatment of nursing home residents’ pain has been
recognized as a national problem (2), affecting the quality
of life of 49% to 83% of residents (3). Inadequate pain treat-
ment is not unique to nursing homes but is pervasive in all
health care settings (4), even hospice (5), where pain man-
agement is a high priority.
Two studies of nursing home residents’ pain using the
Minimum Data Set (MDS) for pain rating, with data col-
lected by nursing home staff in their usual manner, identi-
fied multivariate predictors for the presence of daily pain:
female gender, black race, depression, poorer physical
function, being bedridden, or terminal prognosis (2,6). Pre-
dictors of receiving no analgesia in spite of daily pain were
male gender, black race, older age, poorer physical func-
tion, and cognitive impairment. Studies not using the MDS
found that residents’ pain was associated with disruptive be-
haviors (7,8), but results were equivocal regarding the ef-
fects of pain on the ability to perform activities of daily liv-
ing (ADLs) (9,10), depression (9,11,12), and cognitive
status (9,10).
Nursing staff ratings of resident pain location did not
agree with residents’ reports (13), and nurses’ ratings of res-
M405

M406
ENGLE ET AL.
tween types of nursing staff when evaluating nursing home
care. RNs, LPNs, and NAs may be studied together as
“nursing staff” (13,21) rather than separately in recognition
of their differences. Furthermore, researchers frequently
combine pain ratings of RNs, LPNs, and NAs under the ru-
bric of “nursing staff,” making it impossible to estimate and
compare the accuracy of pain ratings of nursing staff at any
given level.
The accuracy of the MDS pain rating provides a founda-
tion for standardized care planning, quality indicators, and
reimbursement for the treatment and evaluation of resi-
dents’ pain (22). Although directions for using the MDS
specifically instruct the RN to ask the resident directly
about pain, the RN is also instructed to ask NAs and thera-
pists if the resident has had complaints or indicators of pain.
In practice, even if the resident is able to respond verbally,
the RN may often ask the LPN about the resident’s pain be-
cause of the RN’s limited contact with the resident. It is not
feasible, due to time constraints and the limited numbers of
RNs in nursing homes, for the RN to assess the resident
daily for the 7 days needed to complete the MDS for pain
frequency during the last 7 days (MDS item J2a) (22). The
recommended MDS pain-rating process is in contrast to the
gold standard for pain rating, which is an individual’s state-
ment about his or her own pain (23).
Thus, the purposes of this study were to (i) estimate and
compare the accuracy of LPNs’ and NAs’ resident pain rat-
ings using the MDS and (ii) to evaluate the bias effects of
resident race, gender, mental status, function, depression,
and disruptive behavior on nursing staff pain ratings. This
study extends previous research on the pain of nursing
home residents assessed with the MDS by obtaining concur-
rent MDS pain-rating data from LPNs, NAs, and residents,
and by using MDS data obtained by trained interviewers
rather than relying solely on MDS chart data. We extend
previous research on pain ratings by evaluating the bias ef-
fects of resident characteristics on LPN and NA pain rat-
ings.
estimates of race effects. Use of these sites also controls for
the effects of staff turnover on resident rating.
Participants
This study was approved by the University’s Institutional
Review Board. Residents (N ꢀ 380) were enrolled sequen-
tially as admitted to the two nursing homes. They met the
criteria of not declining to participate in the larger study and
remaining in the nursing home for at least 2 weeks. Of the
380 residents admitted, 73% (n ꢀ 277) were able to provide
pain data verbally or nonverbally. Nursing home staff data
were unavailable for 15 otherwise qualified residents, so the
study sample numbered 252.
Measurements
Pain.—Resident pain was evaluated using two MDS pain
items and one additional pain question. MDS item J2a eval-
uated pain frequency during the last 7 days (weekly pain
frequency) using a three-point scale: 0 (no pain), 1 (pain
less than daily), or 2 (daily pain). MDS item J2b evaluated
pain intensity during the last 7 days (weekly pain intensity)
using a three-point scale: 1 (mild pain), 2 (moderate pain),
or 3 (horrible pain). A parallel question evaluated pain in-
tensity on the day of the interview (daily pain intensity) and
was also scored on a three-point scale: 1 (mild pain), 2
(moderate pain), or 3 (horrible pain). Data were obtained
concurrently the same day by trained interviewers of the
resident, the LPN charge nurse, and the NA who cared for
the resident.
Mental Status.—Mental status was evaluated by the
MDS Cognition Scale (MDS-COGS) (27). MDS items
(MDS B2a, B2b, B3b, B3d, B3e, B4, C4, G1Ag) were re-
coded and summed for a score ranging from 0 to 10 (27).
Higher scores indicate greater cognitive impairment. The
MDS-COGS was developed on 200 residents, race not spec-
ified, with sensitivity, specificity, chance-corrected agree-
ment (Kappa), and area under the receiver operating charac-
teristic curve all ꢂ0.80 according to data collected by
trained research staff (27). When comparing the MDS-
COGS with the Cognitive Performance Scale (CPS), which
is also composed of MDS items, the MDS-COGS was re-
ported to be more strongly correlated with the Global Dete-
rioration Scale (r ꢀ .77) and Mini-Mental State Exam (r ꢀ
ꢃ.75) than the CPS in a sample of 290 residents, race not
specified (28). Data were obtained by interviewing and ob-
serving the resident and by questioning the NA who cared
for the resident.
METHODS
Design
As part of a larger study, data were obtained on the same
day directly from residents, LPNs, and NAs by trained in-
terviewers during the first 2 weeks following admission.
Settings
Participants were recruited from two county-financed
safety-net nursing homes that provide indigent care and his-
torically admit both black and white residents in a large city
in the midsouth. The homes are dually licensed for interme-
diate and skilled care and have 250 and 300 beds. The nurs-
ing staff is represented by collective bargaining and re-
ceives the highest pay in the metropolitan area, ensuring
low staff turnover. The majority of the LPNs and NAs were
women, approximately 50% of the LPNs were black, and
all of the NAs were black. Use of these two sites minimizes
potentially confounding effects of gross differences in so-
cioeconomic status (24), limited access to nursing homes by
black older adults (25), and region of the country (26) on
Function.—Resident function was evaluated by the
MDS ADL-Long Form Scale (29). MDS items (MDS
G1Aa, G1Ab, G1Ae, G1Ag, G1Ah, G1Ai, G1Aj) were each
scored on a five-point scale: 0 (independent), 1 (supervi-
sion), 2 (limited assistance), 3 (extensive assistance), and 4
(total dependence). Items were summed, with scores rang-
ing from 0 to 28 (29). Higher scores indicated greater func-
tional impairment. The MDS ADL-Long Form Scale norms
were established using 175,000 MDS ratings in a seven-
state area, with a KR20 ꢀ 0.94, a flat overall scale distribu-

ACCURACY AND BIAS
M407
tion, and an overall scale mean of 15.24 (SD 9.25). Trained
interviewers obtained data from the NA who cared for the
resident to control for source of ADL data (30). NAs are
able to assess accurately residents’ ADLs (18), and one ob-
servation of MDS ADLs during a 7-day period can usually
accurately represent ADLs during the previous 7 days (31).
Data Analysis
All statistical data analysis procedures were accom-
plished using SPSS (33) desktop computer programs. Two-
tailed probability of Type I error of .05 or less was the crite-
rion of statistical significance. The Kappa coefficient (K)
(34) was used to evaluate the extent to which resident pain
ratings agreed with LPN or NA pain ratings. Qualitative rat-
ing of Kappa’s used the criteria of Fleiss (34): K ꢂ .75 ꢀ
excellent, .40 ꢅ K ꢅ .75 ꢀ fair to good, and K ꢁ .40 ꢀ
poor agreement beyond chance. Paired data t tests were
used to evaluate within-subject differences in raters’ pain
ratings. Independent samples t tests evaluated the possible
bias of nursing staff pain ratings associated with resident
race or gender. Correlation coefficients were used to evalu-
ate possible associations between mental status, function,
depression, or disruptive behavior, and LPN or NA underes-
timation of resident pain.
Depression.—Resident depression was evaluated by the
MDS Depression Rating Scale (MDS DRS) (32). MDS
items (MDS E1a, E1d, E1f, E1h, E1i, E1l, E1m) were each
scored on a three-point scale: 0 (not exhibited in last 30
days), 1 (exhibited up to 5 days a week), 2 (exhibited 6 to 7
days a week). Items were summed for a scale score ranging
from 0 to 14. Higher scores indicated more depression. The
MDS DRS was developed from 16 MDS mood and behav-
ior items, the Hamilton Depression Rating Scale, and the
Cornell Scale for Depression in Dementia using a sample of
108 residents, race unknown. There were five MDS factors:
disturbed mood, anxiety, fear, loss of meaning, and affect.
The MDS DRS was more sensitive and more specific than
the Geriatric Depression Scale (29). The interviewer ob-
tained the data by observing the resident and questioning
the NA who cared for the resident.
Variable Specification
For all pain ratings, regardless of source, larger values in
raw data represented more frequent or more intense pain, in
accordance with the MDS. The resident–LPN difference
data and resident–NA difference data analyzed were the in-
dividual resident’s pain ratings minus the corresponding
LPN or NA pain rating, as appropriate. These variables rep-
resented the amount that the LPN or NA underestimated
resident pain, because positive values indicated that the res-
ident rated pain worse than the LPN or NA did, zero indi-
cated no difference, and negative values indicated that the
LPN or NA rated pain as worse than the resident did. Find-
ings presented in Tables 2, 4, 6, 7, and 8 are sample aver-
ages of these within-subject resident–LPN difference data
and resident–NA difference data. Findings presented in Ta-
bles 3 and 5 are the difference between resident–LPN and
resident–NA data.
Disruptive Behavior.—Resident disruptive behavior
was evaluated by summing selected MDS items from sec-
tion E, Mood and Behavior Patterns (MDS E4Ab, E4Ac,
E4Ad, E4Ae). Each item was scored on a four-point scale: 0
(not exhibited in the last 7 days), 1 (occurred 1 to 3 days in
the last 7 days), 2 (occurred 4 to 6 days but less than daily),
3 (occurred daily). Items were summed for MDS Disruptive
Behavior Scale (MDS DBS) scores ranging from 0 to 12,
with higher scores indicating more disruptive behavior.
These items were chosen to measure verbally abusive be-
havior, physically abusive behavior, socially inappropriate
behavior, and resisting care. Interviewers obtained the data
by observing the resident and interviewing the NA who
cared for the resident.
RESULTS
Sample
Of the 252 study residents, 79% were Black, and 60%
were men. Their mean age was 64.3 years (SD 18.3). The
mean level of education was 9.6 years (SD 3.6). Marital sta-
tus included 30.8% never married, 12.3% married, 32.8%
widowed, 8.3% separated, and 15.8% divorced. The mean
MDS-COGS score was 3.7 (SD 3.1), the mean MDS ADL-
Long Form Scale score was 13.9 (SD 9.0), the mean MDS
DRS score was 0.9 (SD 1.7), and the mean MDS DBS score
was 0.6 (SD 1.4).
Demographics.—Resident demographic information in-
cluded gender (MDS AA2); race (MDS AA4); marital sta-
tus (MDS A5); age in years, computed from the resident’s
birthday and date of interview; and education in years.
Procedure
After consent was obtained, each resident’s pain was as-
sessed one time during the first 2 weeks after admission by
trained interviewers who interviewed the resident directly.
To assess each resident’s pain, interviewers concurrently in-
terviewed the LPN who was the charge nurse on the unit in
which the resident resided and the NA who cared for the
resident that day. Mental status, function, depression, dis-
ruptive behavior, and demographic data were obtained by
interviewing and observing the resident and by questioning
the NA who cared for the resident on the day of the inter-
view. Residents were interviewed primarily during the sec-
ond week after admission to ensure staff familiarity with the
resident. Three trained interviewers had an inter-rater reli-
ability ꢄ0.85 for this data-collection procedure.
Pain Variables
Descriptive data for the pain variables by assessor (resi-
dent, LPN, or NA) are presented in Table 1.
Accuracy of LPN and NA Pain Ratings
Kappa coefficients (K) were calculated to evaluate the
agreement between resident and LPN and between resident
and NA pain ratings for weekly pain frequency, weekly pain
intensity, and daily pain intensity. Table 2 documents K val-
ues, their 95% confidence intervals (CI), and p values test-
ing a null hypothesis of K ꢀ 0 or no within-subject associa-

M408
ENGLE ET AL.
Table 1. Description of Pain Variables by Assessor
Table 3. Difference in LPN vs NA Agreement With Resident
Pain Rating
Pain Variable
Resident
LPN
NA
Pain Variable
Kappa Difference*
p Value†
Weekly frequency
Weekly intensity
Daily intensity
0.63 ꢆ 0.83
0.82 ꢆ 1.12
0.52 ꢆ 0.89
0.48 ꢆ 0.77
0.50 ꢆ 0.84
0.35 ꢆ 0.71
0.52 ꢆ 0.79
0.57 ꢆ 0.91
0.41 ꢆ 0.78
Weekly frequency
Weekly intensity
Daily intensity
.02
.02
.10
.574
.300
.004
Notes: Values are mean ꢆ SD. LPN ꢀ licensed practical nurse; NA ꢀ nurs-
ing assistant.
Note: LPN ꢀ licensed practical nurse; NA ꢀ nursing assistant.
*Kappa difference ꢀ resident–LPN Kappa minus resident–NA Kappa.
†Test for equal Kappa’s from nonindependent samples.
tion. All K values were highly significant (p ꢁ .001) in their
differences from the null value of zero. The K values for the
resident–LPN ratings ranged from .50 to .70, indicating fair
to good agreement according to the criteria of Fleiss (34).
The K values for the resident–NA ratings ranged from .58 to
.72, again highly significant (p ꢁ .001) and indicating fair
to good agreement according to the criteria of Fleiss (34).
The K values for resident–LPN and resident–NA were
compared for difference in agreement with resident pain rat-
ings. As shown in Table 3, there were no statistically signif-
icant within-subject differences between resident–LPN ver-
sus resident–NA K values for weekly pain frequency or
weekly pain intensity. However, in comparing their K val-
ues for daily pain intensity, NA agreement with the resident
was significantly better than LPN agreement with the resi-
dent (p ꢀ .004).
Paired data t tests were used to evaluate possible system-
atic bias in either LPN or NA pain ratings. Table 4 docu-
ments the average within-subject differences between resi-
dent and LPN and between resident and NA pain ratings,
their 95% CI, and the p values for within-subject paired data
t tests comparing resident pain ratings with LPN and NA
pain ratings. There were statistically significant differences
between resident and LPN pain ratings and between resi-
dent and NA pain ratings: all t tests were significant at the
.002 level or beyond. On average, both LPN and NA signif-
icantly underestimated residents’ weekly pain frequency,
weekly pain intensity, and daily pain intensity.
mated residents’ weekly pain intensity and daily pain inten-
sity less than LPNs underestimated these ratings. The
average within-subject difference in weekly pain frequency
approached, but did not attain, statistical significance for the
comparison of the resident–LPN difference with the resi-
dent–NA difference. Thus, there was no significant differ-
ence between LPNs’ and NAs’ underestimations of resi-
dents’ weekly pain frequency.
Bias of LPN and NA Pain Ratings by
Resident Demographics
Independent samples t tests evaluated if LPN or NA rat-
ings of resident pain were affected or biased by resident
race (black or white) or resident gender. The pain data ana-
lyzed were resident–LPN difference and resident–NA dif-
ference scores. There was no statistically significant finding
of resident race or resident gender bias in LPN and NA
weekly pain frequency, weekly pain intensity, or daily pain
intensity ratings. Table 6 shows a summary of LPN find-
ings, and Table 7 shows a summary of NA findings.
Correlation coefficients determined the bias effects of
resident behavioral or psychological characteristics on LPN
or NA ratings of resident pain. Resident mental status, func-
tion, depression, and disruptive behavior were the continu-
ous variables included in this analysis. Pain data analyzed
were the resident–LPN and the resident–NA difference
scores. There were no statistically significant findings of
mental status, function, depression, or disruptive behavior
bias of LPN and NA ratings for weekly pain intensity,
weekly pain frequency, or daily pain intensity. Table 8
shows summaries of the LPN and NA findings.
Paired data t tests were also used to test for within-subject
difference in the resident–LPN difference compared with
the resident–NA difference in pain ratings. Table 5 docu-
ments the average difference score for the resident–LPN
difference minus the resident–NA difference, t test statis-
tics, and p values. Significant differences between the
amount of difference comparing the resident–LPN ratings
with the resident–NA ratings documented that, on average,
the NA was significantly closer to the resident self-assess-
ment of pain intensity than the LPN was. NAs underesti-
DISCUSSION
Residents in our study met the inclusion criteria of being
able to provide pain data verbally or nonverbally. Our sam-
ple, therefore, was typically mildly to moderately cogni-
tively impaired rather than profoundly cognitively impaired.
Study residents usually required limited to extensive assis-
tance with ADLs but were not depressed and exhibited few
disruptive behaviors. On average, these residents reported
mild daily pain and mild weekly pain intensity, with pain
occurring less than daily. Nursing home residents tend to
have more chronic and nociceptive pain from musculoskel-
etal disorders and neuropathies (23), with fluctuations of
chronic pain (35).
Table 2. Agreement of Resident–LPN and Resident–NA
Pain Ratings
Resident–LPN Agreement
Resident–NA Agreement
Pain Variable
Kappa* 95% CI p Value Kappa* 95% CI p Value
Weekly frequency
Weekly intensity
Daily intensity
.70
.56
.50
0.62–0.79 ꢁ.001
0.48–0.64 ꢁ.001
0.40–0.60 ꢁ.001
.72
.58
.60
0.65–0.80 ꢁ.001
0.50–0.66 ꢁ.001
0.51–0.69 ꢁ.001
Study results indicated that all resident–LPN and resi-
dent–NA pain ratings had fair to good Kappa coefficients of
agreement when using MDS pain items. Although both
LPNs and NAs significantly underestimated residents’
Note: LPN ꢀ licensed practical nurse; NA ꢀ nursing assistant; CI ꢀ confi-
dence interval.
*Kappa coefficient of agreement; all with fair to good agreement.

ACCURACY AND BIAS
M409
Table 4. Within-Subject Differences Between Resident–LPN Difference and Resident–NA Difference Pain Ratings
Resident–LPN Difference*
Resident–NA Difference†
Pain Variable
Mean ꢆ SD
95% CI
t Value‡
p Value
Mean ꢆ SD
95% CI
t Value‡
p Value
Weekly frequency
Weekly intensity
Daily intensity
0.15 ꢆ 0.50
0.32 ꢆ 0.74
0.17 ꢆ 0.65
0.08–0.21
0.23–0.41
0.09–0.25
4.63
6.78
4.09
ꢁ.001
ꢁ.001
ꢁ.001
0.11 ꢆ 0.47
0.25 ꢆ 0.71
0.11 ꢆ 0.55
0.05–0.17
0.17–0.34
0.04–0.18
3.77
5.70
3.09
ꢁ.001
ꢁ.001
ꢁ.002
Note: LPN ꢀ licensed practical nurse; NA ꢀ nursing assistant; CI ꢀ confidence interval.
*Resident–LPN difference ꢀ resident pain score minus LPN pain score; positive value indicates that LPN underestimates resident’s pain.
†Resident–NA difference ꢀ resident pain score minus NA pain score; positive value indicates that NA underestimates resident’s pain.
‡Paired data t test.
weekly pain frequency, weekly pain intensity, and daily
pain intensity, NAs were more accurate than LPNs when
underestimating residents’ weekly pain intensity and daily
pain intensity. LPN and NA pain ratings were not biased by
resident race, gender, mental status, function, depression, or
disruptive behavior.
All Kappa coefficients for the resident–LPN (range .50–
.70) and resident–NA (range .58–.72) pain rating agreement
were in the fair to good range, but none attained excellence.
Excellent, rather than merely fair to good, Kappa’s for resi-
dent–LPN and resident–NA agreement may be required for
optimal pain evaluation and treatment.
erate pain), and 3 (horrible pain). The three-point scale may
have attenuated the study results.
Other studies support the findings of underestimation of
resident pain by nursing home staff and greater accuracy of
pain ratings by people with greater resident contact. Weiner
and colleagues (13) found that nursing home nursing staff
(RNs, LPNs, and NAs, studied as one group) were more ac-
curate in their pain ratings than family caregivers who vis-
ited at least twice a month. Physicians also underestimate
resident pain (15), but physician pain ratings were inferred
from chart review for progress notes indicating pain or pain
medication orders.
LPNs and NAs underestimated weekly pain frequency,
weekly pain intensity, and daily pain intensity. This bias
was in the expected direction because nurses in other health
care settings also underestimated pain (36). The nursing
home nursing staff were educated primarily in acute-care
hospital settings and had acute-care experience. They evi-
dently carried their skills and biases in pain rating and treat-
ment from the hospital into the nursing home.
NAs underestimated both weekly and daily pain intensity
less than LPNs did. NAs may be more accurate in their pain
intensity ratings because of their prolonged contact with
residents when providing daily care compared with LPNs.
NAs provide about 60% of resident care, and LPNs provide
about 20% of resident care (37). For underestimation of
weekly pain frequency the comparison of LPN to NA data
approached, but did not attain, statistical significance. This
may be due to sampling error or scale scoring for pain inten-
sity. Weekly pain frequency was measured using a three-
point scale: 0 (no pain); 1 (pain less than daily); and 2 (daily
pain); however, weekly and daily pain intensity was scored
using a four-point scale: 0 (no pain), 1 (mild pain), 2 (mod-
The more accurate source of pain frequency data, LPN or
NA data, was not established by study data. Also, the rela-
tionship of a series of seven daily resident’s pain intensity
ratings to resident, LPN, and NA MDS weekly pain fre-
quency and MDS weekly pain intensity ratings is not
known. Residents may not be able to remember their pain
intensity and pain frequency during the previous week, so
this is a question for future research. Due to time con-
straints, it is not feasible for RNs in the nursing home to as-
sess residents’ pain daily to accurately complete the MDS.
Time constraints may also prevent the NA from reporting
resident pain to the LPN (8).
We recommend that if the pain intensity data cannot be
obtained directly from mild or moderately cognitively im-
paired nursing home residents by the RN, daily and weekly
pain intensity data should be obtained from the NA and
used to complete the MDS. The RN is responsible for ensur-
Table 6. Bias of LPN and NA Pain Ratings by Resident Race
Black
White
Rating
(Mean ꢆ SD) (Mean ꢆ SD) t Value* p Value
Resident–LPN Difference†
Weekly frequency
Weekly intensity
0.15 ꢆ 0.50
0.34 ꢆ 0.77
0.17 ꢆ 0.67
0.13 ꢆ 0.52
0.25 ꢆ 0.65
0.15 ꢆ 0.57
.22
.77
.18
.823
.440
.854
Table 5. Within-Subject Differences in LPN vs NA
Underestimation of Resident Pain Rating
Daily intensity
Resident-NA Difference‡
Weekly frequency
Weekly intensity
Difference*
0.10 ꢆ 0.46
0.27 ꢆ 0.72
0.11 ꢆ 0.58
0.15 ꢆ 0.50
0.21 ꢆ 0.70
0.09 ꢆ 0.45
.71
.51
.18
.477
.605
.858
Pain Variable
Mean ꢆ SD
t Value†
p Value
Daily intensity
Weekly frequency
Weekly intensity
Daily intensity
0.04 ꢆ 0.31
0.06 ꢆ 0.41
0.06 ꢆ 0.45
1.8
2.4
2.1
.072
.016
.035
Note: LPN ꢀ licensed practical nurse; NA ꢀ nursing assistant.
*Independent samples t test.
†Resident–LPN difference ꢀ resident pain score minus LPN pain score;
positive value indicates that LPN underestimates resident’s pain.
‡Resident–NA difference ꢀ resident pain score minus NA pain score; posi-
tive value indicates that NA underestimates resident’s pain.
Note: LPN ꢀ licensed practical nurse; NA ꢀ nursing assistant.
*Difference ꢀ resident–LPN difference minus resident–NA difference.
†Paired data t test.

M410
ENGLE ET AL.
Table 7. Bias of LPN and NA Pain Ratings by Resident Gender*
were older, black, cognitively impaired, more functionally
able, or less depressed were more likely to have daily pain (2).
Could the race and gender of the LPN and NA also be a
source of bias in their ratings of residents with different
characteristics? All study NAs were black, approximately
one half of the LPNs were black, and the majority of LPNs
and NAs were women. We have neither race variance for
NAs nor accurate race data for LPNs and are unable to an-
swer this question. However, because there was no resident
race or resident gender bias for either LPN or NA ratings,
the race and gender of the assessor does not appear to influ-
ence pain ratings.
Our results can be generalized to other safety-net nursing
homes that provide care to the underserved and indigent.
Our sample had more men (60%) and black (80%) and
younger (mean age 64 years) residents than the average
nursing home. Although the use of safety-net nursing homes
allowed us to study race and gender differences, additional
studies are needed to replicate our results in other regions of
the country and in different types of nursing homes.
Male
Female
Rating
(Mean ꢆ SD) (Mean ꢆ SD) t Value† p Value
Resident–LPN Difference‡
Weekly frequency
Weekly intensity
0.15 ꢆ 0.49
0.32 ꢆ 0.76
0.16 ꢆ 0.65
0.14 ꢆ 0.53
0.32 ꢆ 0.72
0.18 ꢆ 0.66
.20
ꢁ.01
.24
.838
ꢂ.999
.812
Daily intensity
Resident–NA Difference§
Weekly frequency
Weekly intensity
0.11 ꢆ 0.49
0.25 ꢆ 0.74
0.10 ꢆ 0.60
0.11 ꢆ 0.45
0.27 ꢆ 0.66
0.12 ꢆ 0.48
.06
.25
.28
.956
.800
.780
Daily intensity
Note: LPN ꢀ licensed practical nurse; NA ꢀ nursing assistant.
*LPN or NA underestimation of resident pain compared with resident pain
rating.
†Independent samples t test.
‡Resident–LPN difference ꢀ resident pain score minus LPN pain score;
positive value indicates that LPN underestimates resident’s pain.
§Resident–NA difference ꢀ resident pain score minus NA pain score; posi-
tive value indicates that NA underestimates resident’s pain.
ing the completeness of the multidisciplinary MDS rating that
assesses weekly pain frequency and weekly pain intensity
and often relies on the LPN for resident rating data rather
than on the NA, whose accuracy may be underrecognized.
For optimal pain management and treatment, additional
data on the quality of pain (e.g., burning or aching), the tem-
poral pattern of the pain (e.g., constant or intermittent), and
the location of pain (e.g., joint, head, or feet) may be needed
by the health care provider writing prescriptions for analge-
sic and adjuvant pain medications. However, the MDS does
not obtain information on the quality or pattern of pain; it
obtains information on the location of pain. Additional re-
search is needed to establish the accuracy of LPN and NA
ratings of pain quality, pain pattern, and pain location.
We found no literature evaluating the effect of resident
characteristics (mental status, function, depression, and dis-
ruptive behavior) on the accuracy of nursing home nursing
staff pain ratings. Studies have identified that residents who
Summary
Our results documented that (i) LPNs and NAs underes-
timated nursing home residents’ weekly pain frequency,
weekly pain intensity, and daily pain intensity; (ii) NAs un-
derestimated weekly and daily pain intensity less than LPNs
did; and (iii) LPN and NA pain ratings were not biased by
resident race, gender, mental status, function, depression, or
disruptive behavior. There was fair to good Kappa agree-
ment of LPN and NA pain ratings with resident ratings, but
consistently excellent Kappa coefficients may be required
for optimal pain rating and management.
Acknowledgments
This study was funded by the NIH National Institute of Nursing Re-
search and was presented at the annual meeting of the American Geriatrics
Society, 2000. We thank the residents, nursing staff, and administration of
Shelby County Health Care Center and Oakville Health Care Center for
their participation in and support of the study.
Address correspondence to Dr. Veronica Engle, 877 Madison Avenue,
Room 616, Memphis, TN 38163. E-mail: vengle@utmem.edu
Table 8. Biases of LPN and NA Pain Ratings by Resident Mental
Status, Function, Depression, and Disruptive Behavior
Mental
Status*
Disruptive
References
Function† Depression‡ Behavior§
1. Ferrell BA, Ferrell BR, Rivera L. Pain in cognitively impaired nursing
home patients. J Pain Symptom Manage. 1995;10:591–598.
2. Bernabei R, Gambassi G, Lapanne K, et al. Management of pain in el-
derly patients with cancer. JAMA. 1998;279:1877–1882.
p
p
p
p
Rating
r
Value
r
Value
r
Value
r
Value
Resident–LPN
3. Fox PL, Raina P, Jadad AR. Prevalence and treatment of pain in older
adults in nursing homes and other long-term care institutions: a sys-
tematic review. CMAJ. 1999;160:329–333.
4. Engle VF, Fox-Hill E, Graney MJ. The experience of living-dying in a
nursing home: self-reports of black and white older adults. J Am Geri-
atr Soc. 1998;46:1091–1096.
5. Morris JN, Mor V, Goldberg RJ, Sherwood S, Greer DS, Hiris J. The
effect of treatment setting and patient characteristics on pain in termi-
nal cancer patients: a report from a national hospice study. J Chronic
Dis. 1986;39:27–35.
6. Won A, Lapane K, Gambassi G, Bernabei R, Mor V, Lipsitz LA. Cor-
relates and management of nonmalignant pain in the nursing home.
SAGE Study Group. Systematic Assessment of Geriatric drug use via
Epidemiology. J Am Geriatr Soc. 1999;47:936–942.
Weekly frequency ꢃ.09 .215
Weekly intensity ꢃ.06 .368
.01 .906 .06 .375 ꢃ.11 .079
.01 .831 .10 .105 ꢃ.11 .089
ꢃ.01 .935 ꢃ.04 .563 .04 .547 ꢃ.74 .245
Daily intensity
Resident–NA
Weekly frequency ꢃ.14 .600 ꢃ.04 .493 .04 .510 ꢃ.08 .207
Weekly intensity ꢃ.05 .466 .02 .760 .04 .483 ꢃ.12 .056
Daily intensity ꢃ.01 .872 ꢃ.05 .427 .05 .452 ꢃ.04 .501
Note: LPN ꢀ licensed practical nurse; NA ꢀ nursing assistant; r ꢀ correla-
tion coefficient.
*MDS Cognition Scale.
†MDS ADL-Long Form Scale.
‡MDS Depression Rating Scale.
§MDS Disruptive Behavior Scale.
7. Cohen-Mansfield J, Billig N, Lipson S, Rosenthal AS, Pawlson LG.

ACCURACY AND BIAS
M411
Medical correlates of agitation in nursing home residents. Gerontol-
ogy. 1990;36:150–158.
8. Feldt KS, Warne MA, Ryden MB. Examining pain in aggressive cog-
nitively impaired older adults. J Gerontol Nurs. 1998;24:14–22.
9. Ferrell BA, Ferrell BR, Osterweil D. Pain in the nursing home. J Am
Geriatr Soc. 1990;38:409–414.
10. Parmalee PA, Smith B, Katz IR. Pain complaints and cognitive status
among elderly institutionalized residents. J Am Geriatr Soc. 1993;41:
517–522.
11. Cohen-Mansfield J, Marx MS. Pain and depression in the nursing
home: corroborating results. J Gerontol Psych Sci. 1993;48:P96–P97.
12. Parmalee PA, Katz IR, Lawton MP. The relation of pain to depression
among institutionalized aged. J Gerontol Psych Sci. 1991;46:P15–P21.
13. Weiner D, Peterson B, Keefe F. Chronic pain-associated behaviors in
the nursing home: resident versus caregiver perceptions. Pain. 1999;
80:577–588.
Blacks and Whites: recent trends in mortality and morbidity. In: Willis
DP, ed. Health Policies and Black Americans. New Brunswick: Trans-
action Publishers; 1989:129–199.
25. Wright R, Mindel CH. Economic, health, and service use policies: im-
plications for long-term care of ethnic elderly. In: Baressi CM, Stull
DE, eds. Ethnic Elderly and Long-Term Care. New York: Springer;
1993:247–263.
26. Taylor RJ, Chatters LM. Correlates of education, income, and poverty
among aged Blacks. Gerontologist. 1988;28:435–441.
27. Hartmaier SL, Sloane PD, Guess HA, Koch GG. The MDS Cognition
Scale: a valid instrument for identifying and staging nursing home res-
idents with dementia using the minimum data set. J Am Geriatr Soc.
1994;42:1173–1179.
28. Cohen-Mansfield J, Taylor L, McConnell D, Horton D. Estimating the
cognitive ability of nursing home residents from the Minimum Data
Set. Outcomes Manag Nurs Pract. 1999;3:43–46.
14. Kaasalainen S, Middleton J, Knezacek S, et al. Pain and cognitive sta-
tus in the institutionalized elderly: perceptions & interventions. J
Gerontol Nurs. 1998;24:24–31, 50–51.
15. Sengstaken EA, King SA. The problems of pain and its detection
among geriatric nursing home residents. J Am Geriatr Soc. 1993;41:
541–544.
16. Beck C, Ortigara A, Mercer S, Shue V. Enabling and empowering cer-
tified nursing assistants for quality dementia care. Int J Geriatr
Psychiatry. 1999;14:197–211; discussion 211–212.
17. Feldt LS, Ryden MS. Aggressive behavior: educating nursing assis-
tants. J Gerontol Nurs. 1992;18:3–12.
18. Hartig MT, Engle VF, Graney MJ. Accuracy of nurse aides’ functional
health assessments of nursing home residents. J Gerontol Med Sci.
1997;52:M142–M148.
19. Ranhoff AH. Reliability of nursing assistants’ observations of func-
tioning and clinical symptoms and signs. Aging. 1997;9:378–380.
20. Weiner DK, Ladd KE, Pieper CF, Keefe FJ. Pain in the nursing home:
resident versus staff perceptions. J Am Geriatr Soc. 1995;43:SA2.
21. Parke B. Gerontological nurses’ ways of knowing. J Gerontol Nurs.
1998;24:21–28.
22. Rantz MJ, Popejoy L, Zwygart-Stauffacher M, Wipke-Tevis D,
Grando VT. Minimum Data Set and Resident Assessment Instrument:
can using standardized assessment improve clinical practice and out-
comes of care? J Gerontol Nurs. 1999;25:35–43, 54–55.
23. Weissman DE, Matson S. Pain assessment and management in the
long-term care setting. Theor Med Bioeth. 1999;20:31–43.
24. Manton KG, Patrick CH, Johnson KW. Health differentials between
29. Morris JN, Fries BE, Morris SA. Scaling ADLs within the MDS. J
Gerontol Med Sci. 1999;54A:M546–M553.
30. Rubenstein LZ, Schairer C, Wieland GD, Kane R. Systematic biases in
functional status assessment of elderly adults: effects of different data
sources. J Gerontol. 1984;39:686–691.
31. Graney M, Engle V. Stability of performance of activities of daily liv-
ing using the MDS. Gerontologist. 2000;40:582–586.
32. Burrows AB, Morris JN, Dixon SE, Hirdes JP, Phillips C. Develop-
ment of a minimum data set-based depression rating scale for use in
nursing homes. Aging. 2000;29:165–172.
33. SPSS. SPSS Reference Guide. Chicago, IL: SPSS Inc; 1990.
34. Fleiss JL. Statistical Methods for Rates and Proportions. 2nd ed. New
York: Wiley; 1981;xviii, 321.
35. Ferrell BA. Pain management in elderly people. J Am Geriatr Soc.
1991;39:64–73.
36. Grossman SA, Sheidler VR, Swedeen K, Mucenski J, Piantadosi S.
Correlation of patient and caregiver ratings of cancer pain. J Pain
Symptom Manage. 1991;6:53–57.
37. Harrington C, Kovner C, Mezey M, et al. Experts recommend mini-
mum nurse staffing standards for nursing facilities in the United States.
Gerontologist. 2000;40:5–15.
Received October 26, 2000
Accepted October 30, 2000
Decision Editor: John E. Morley, MB, BCh