Full text of DOI:10.1111/j.1755-6686.2001.tb00134.x

Pre-training assessment tool (JPAT) –
A pilot study
J. Chow and L. Bennett
Renal Unit, Liverpool, Australia.
Summary
A tool for assessing the suitability of candidates for home dialysis (Jo-Pre-training Assessment Tool – JPAT) was devel-
oped. JPAT acts as a screening instrument to identify suitable candidates for the home dialysis programme, and there-
fore increases a patient’s chance of learning to manage the programme. JPAT is in the form of an interview question-
naire consisting of 38 assessment items in six domains: physical stability, nutritional status, communication, ability to
maintain self-care, psychological suitability and social support. A pilot study was conducted (n = 20, 1996-1997) using a
descriptive study design, with subjects randomly selected from an existing dialysis programme. Pearson correlation and
2
-tailed tests were employed to explore the relationship between the assessment outcome (i.e. the initial JPAT scores)
and the follow up data (i.e. data collected within the seven days following the initial JPAT assessment). Many of the
variables attained statistical significance (p < 0.05). The inter-rater reliability was calculated at an average Kappa value
of 0.909. Overall, results suggest that JPAT is sufficiently reliable to be used as a tool for assessing patients who suffer
from ESRD.
Key words
–
–
–
–
Assessment
Pre-dialysis
Selection criteria
Home dialysis
ND STAGE RENAL DISEASE (ESRD) is a complex dis- indicate that there is a need to develop such a tool, through
E
ease that has a relentless and wide-ranging effect on a per- which the candidates’ existing and potential health-related
son ’s health and lifestyle. ESRD is malfunction of the kidneys to problems can be identified (1). Candidates could then be
the point that renal replacement therapy is required. The per- referred to the appropriate professionals among the members of
son who has ESRD must be equipped to deal with the disease the multi-disciplinary team for assessment and care prior to
and its effects. In order to identify the factors and barriers that commencement of training. Such a practice has the potential to
affect a person ’s ability to learn to perform dialysis treatment, improve the success rate for the home dialysis-training pro-
the person should be assessed prior to the commencement of gramme.
the training. The person will then be in a position to make
informed decisions about his or her health and life.
Thomas (2) conducted a study to identify the influences,
which affect how well individuals with renal failure learn to be
A literature review indicates that there is no previous formal self-caring on CAPD. Findings suggested that individual ’s moti-
tool, which serves the purpose of assessing potential candidates vation and sense of control over the therapy has an impact on
prior to the commencement of home dialysis training. Further the success of self-care. The literature indicates that there are
investigation of pre-dialysis training assessment strategies relatively few studies, which specifically target pre-training
around the world (Singapore, Hong Kong, Italy & several States assessment for home dialysis. This lack of literature dealing
in the USA) has indicated that this assessment ranges from ad- with these issues may be a reflection of the unique practice in
hoc attempts to usually not being addressed at all. The results Australia, in promoting the home dialysis programme. Regard-
of the telephone survey in Australia and the literature review less of the outcome of this research project, home dialysis in
both haemodialysis and peritoneal dialysis is the preferred
Dr. Josephine Chow began working as a renal
nurse in 1991. Her experience in renal nursing
covers both Haemodialysis and Peritoneal
option of treatment for patients who suffer ESRD. Therefore, it
is essential for researchers to undertake an appropriate project,
which support the management of the ESRD population.
Dialysis. Prior to working as a Research
Manager, she worked as a Nurse Educator and
Nursing Course Co-ordinator in South Western
Sydney Area Health Service in Sydney, Australia.
Instrument design
JPAT aims at identifying the candidates’ existing and potential
health related conditions that would undesirably affect the suc-
cess of the home dialysis training. It was designed in the form
of an interview questionnaire, consisting of four sections:
A) Demographic characteristics of the candidate,
She completed her Doctor of Philosophy.
Dr. Lydia Bennett began working as a nurse in 1975. She is a registered
nurse, a registered psychologist and a Senior Lecturer in the Department
of Professional Nursing Studies, University of Sydney. She is also Director
of CANU, Clinical Academic Nursing Unit based at Sydney Hospital and
Sydney Eye Hospital and The University of Sydney – Faculty of Nursing.
EDTNA|ERCA JOURNAL 2001 XXVII 1
37

B) Past and the present clinical findings of the candidate,
C) Current assessment of the candidate,
D) Pre-home dialysis assessment suitability result.
The interview was further subdivided into six sections for
assessment criteria, they were:
i) Physical stability;
ii) Nutritional status;
iii) Communication;
iv) Ability to maintain self-care;
(5). Examples of questions used in the JPAT can be found in
Table 1.
Pilot study
In order to ensure optimal validity and reliability, a pilot study
was conducted, using the designed tool. The pilot study served
to refine the instrument before the systematic use for the ESRD
programme and allowed the investigator to address issues of
inter-rater reliability, tool clarity and the ease of use (3). This
study used a descriptive study design, with subjects randomly
selected from an existing dialysis programme. This study
v) Social support and,
vi) Psychological suitability.
The interviewer was required to observe patient activities such involved an interview with the subjects and used follow up data
as hand-eye co-ordination as part of the interviewing process. obtained from interviews with health professionals. The subject
The content of JPAT was derived from a consensus of a characteristics reflected a typical population of renal patients
focus group (expert opinion), literature review, telephone sur- similar to the population group JPAT was designed to assess.
vey, a national survey of 36 renal units conducted in 1996 and
a review of their pre-dialysis training assessment methods. The
Subject and setting
focus group, literature review and information from the tele- The sample consisted of 20 patients with ESRD. All patients
phone and national survey established key factors for JPAT. received some form of maintenance dialysis therapy at a major
These factors were cross-referenced with other pre-training teaching hospital in Sydney. Thirty-five percent of the patients
interview forms from other training centres, to determine the were 60 years old or above (Table 2). Thirty-five percent of the
actual format and questions for JPAT.
patients were female and 65 percent were married (Table 3). It
Refinement of the JPAT involved extensive discussion with was recognised that patients who are married or in primary
clinical experts utilising a modified version of the Delphi tech- cohabit relationships may have spouses or partners participat-
nique to obtain consensus. Literature review and mock inter- ing in conducting the dialysis at home. Overall, 55 percent of
views were conducted to confirm applicability of JPAT. This the patients had received formal education (i.e. completion of
resulted in JPAT Version 2.0, a 37-items interview questionnaire high school and/or above).
organised into six assessment categories: physical stability (14),
nutritional status (5), communication (2), ability to maintain
self-care (7), social support (4) and psychological suitability The subject selection process involved asking staff at the partic-
ipant hospital to identify 20 cases from their programme, using
Subject selection criteria
Section III:
a random selection method. These subjects were either receiv-
ing home dialysis or attending the preliminary training sessions
Communication (excluding the deaf and dumb)
(
a) How effective is the patient’s verbal
or dialysing at an institution (Satellite Dialysis).
The following criteria were used to recruit the appropriate
subjects for the pilot study:
i) The subject was over eighteen years of age.
ii) The subject received some form of renal replacement
therapy.
communication?
[
[
[
[
5] Cannot communicate
(
Cannot understand patient)
4] Very difficult communication
Most communication is unclear)
1] Slightly difficult communication
Communication is unclear at times)
0] No communication problem
(
(
Frequency
Percent
(%)
Valid Percent Cumulative
(%)
30
35
15
15
5
Percent (%)
30
Valid 18-40
41-60
61-70
71-80
> 80
6
7
30
35
15
15
5
Section IV:
65
Ability to maintain self-care
3
80
(
a) Please observe the patient’s psychomotor
3
95
coordination. Is the patient
1
100
Able to stand unassisted?
Able to walk unassisted?
Able to stand straight with
eyes open and then closed?
Able to flex and extend both
arms against resistance?
Able to pick up a 2 litre dianeal
bag and put on to hook?
[0] Yes
[0] Yes
[4] No
[4] No
Total
20
100
100
Table 2: Age distribution for the pilot study
Frequency
Percent
Valid Percent Cumulative
[0] Yes
[0] Yes
[0] Yes
[2] No
[1] No
[4] No
(
%)
(%)
25
Percent (%)
25
Valid Single
Married
Divorced
Total
5
13
2
25
65
10
65
90
10
100
20
100
100
Table 1: Examples of questions from the JPAT
Table 3: Marital status for subjects
38
EDTNA|ERCA JOURNAL 2001 XXVII 1

ability to maintain self-care and social support. These scores
were compared with the pre-home dialysis assessment (JPAT)
suitability result.
Besides comparing the results with the follow up data,
observations were obtained by the investigator throughout the
study on the following aspects:
i) The clarity of the tool;
ii) The appropriateness of instructions for conducting the
interview;
Good
%
Marginal
%
Inadequate
%
Physical stability
Nutritional status
Communication
19 95
20 100
20 100
1
0
0
5
0
0
0
0
0
0
0
0
Ability to maintain
self care
15 75
15 75
4
3
20
15
1
2
5
Social support
10
Psychological
suitability
19 95
1
5
0
0
Table 4: The pre-home dialysis assessment (JPAT) suitability result
iii) The response of the subjects to the questions;
iv) The organisation of the tool; and
Procedure
v) Any other recommendations for improving the predictive
validity of the tool.
Ethics Committee approval was obtained prior to the study.
Informed consent was obtained from all subjects prior to the
interviews. Data collection for the study took place between
Results
September 1996 and February 1997. All patient interviews The pre-home dialysis assessment (JPAT) suitability result of the
were conducted in a private room near the dialysis unit, gener- 20 subjects is shown in Table 4. Fifteen to 20 percent of the
ally prior to the appointment with either dialysis therapy, train- subjects fall into the marginal area for the two assessment areas,
ing session or clinic follows up visits.
“Ability to maintain self-care” and “Social support”. Five to 10
The investigator, who had no prior knowledge of the sub- percent fall into the inadequate column for both of these areas.
jects, interviewed all 20 subjects using JPAT. This served to These patients were dialysed at the Satellite Haemodialysis unit
avoid bias compared to interviews conducted with known sub- at the time the interviews were conducted. However, due to the
jects. The investigator attempted to identify current dialysis small sample size, caution has to be taken in interpreting the
patients who would have characteristics similar to those with data to avoid both type 1 (alpha) and type 2 (beta) error (4).
whom the instrument would be eventually used.
All data from the pre-home dialysis assessment (JPAT) suit-
Follow up data was obtained from various sources to iden- ability result and the follow up data were analysed using the
tify the predictive ability of JPAT. These sources included:
a) Nursing assessment/evaluation
b) Medical assessment/evaluation
c) Patients’ medical records
SPSS program (version 8.0). A series of linear regression equa-
tions were used to explain the comparison between the assess-
ment outcomes (i.e. the initial JPAT scores) and the follow up
data (i.e. data collected within seven days follow the initial JPAT
assessment). Many of the variables attained statistical signifi-
d) Current treatment
Each follow up data was measured on a five-point Likert scale cance (p < 0.05). The results obtained were reported in Table 5a
ranging from excellent (5) to poor (0). The total score was & b. However, further analysis with a larger sample in the main
added up for each assessment category: physical stability, nutri- study will employ an analysis of correlation.
tional status, psychological suitability, communication, and
The time spent on interviewing the patient using JPAT was
Physical
Current
Medical
NursingPatient
assessment
(Follow up
data)
assessment
treatment
(Follow up
data)
assessment
(Follow up
data)
record
(
JPAT)
(Follow up
data)
Physical
Pearson
assessment (JPAT)
Correlation
1.000
0.943
0.000
0.777
0.000
0.667
0.001
0.594
0.003
Sig. (2-tailed), p =
N/A
Current treatment
Pearson
(Follow up data)
Correlation
0.943
0.000
1.000
0.831
0.000
0.629
0.001
0.780
0.000
Sig. (2-tailed), p =
N/A
Medical assessment
Follow up data)
Pearson
(
Correlation
0.777
0.000
0.831
0.000
1.000
0.699
0.000
0.831
0.000
Sig. (2-tailed), p =
N/A
Nursing assessment
Follow up data)
Pearson
(
Correlation
0.667
0.001
0.629
0.001
0.699
0.000
1.000
0.629
0.001
Sig. (2-tailed), p =
N/A
Patient record
Follow up data)
Pearson
(
Correlation
0.594
0.003
0.780
0.000
0.831
0.000
0.629
0.001
1.000
Sig. (2-tailed), p =
N/A
N/A = Not applicable
Table 5a: Correlation between the assessment outcomes (i.e. the initial JPAT scores) and the follow up data (i.e. Data collect within seven days follow
the initial JPAT assessment) (Physical assessment)
EDTNA|ERCA JOURNAL 2001 XXVII 1
39

Social
Current
Medical
Nursing
Patient
record
assessment
treatment
(Follow up
data)
assessment
(Follow up
data)
assessment
(Follow up
data)
(
JPAT)
(Follow up
data)
Social
Pearson
assessment (JPAT)
Correlation
1.000
0.459
0.042
1.000
0.000
0.927
0.000
0.943
0.000
Sig. (2-tailed), p =
N/A
Current treatment
Pearson
(Follow up data)
Correlation
0.459
0.042
1.000
0.459
0.042
0.577
0.008
0364
Sig. (2-tailed), p =
N/A
0.114
Medical assessment
Follow up data)
Pearson
(
Correlation
1.000
0.000
0.459
0.042
1.000
0.927
0.000
0.943
0.000
Sig. (2-tailed), p =
N/A
Nursing assessment
Follow up data)
Pearson
(
Correlation
0.927
0.000
0.577
0.008
0.927
0.000
1.000
0.812
0.000
Sig. (2-tailed), p =
N/A
Patient record
Follow up data)
Pearson
(
Correlation
0.943
0.000
0.364
0.114
0.943
0.000
0.812
0.000
1.000
Sig. (2-tailed), p =
N/A
N/A = Not applicable
Table 5b: Correlation between the assessment outcomes and the follow up data (Social support)
JPAT Version 2.1 was thus developed as a 38-item interview
questionnaire organised into six assessment categories: physical
stability (14), nutritional status (5), communication (2), ability
to maintain self-care (8), social support (4) and psychological
suitability (5).
Mean Std. Deviation
N
20
Time for JPAT assessment
Time for follow up data
21.85
7.97
47.40
15.13
20
Table 6: Time comparison descriptive statistics
compared to the time for the follow up data collection (Table 6).
On average, there was more than 25 minutes (SD = 7.16) sav-
ing per patients by using JPAT.
Examining inter-rater (observer) reliability
“Reliability or consistency, is concerned with error in measure-
Modification of the tool was then undertaken. This includ- ment” (5). Inter-rater reliability is a crucial part of a tool devel-
ed modifying the format of the questions. Since JPAT is an inter- opment. “Inter-rater reliability is the consistency of observa-
view questionnaire, it would be more appropriate to ask the tions between two or more observers; often expressed as a
patient questions instead of using a third party format. For percentage of agreement between raters or observers or a coef-
example, “What is the religion of the patient?” was changed to ficient of agreement that takes into account the element of
“What is your religion?” The item on “Is the patient compliant chance. This is usually used with the direct observation
with the treatment?” was changed to two separate items, “Do method” (6). As JPAT depends on direct observation of behav-
you ever miss an appointment to the doctor or clinic?” and “Do iour that requires systematically recording, testing for inter-rater
you follow prescribed treatment?” Compliance is a stigmatised reliability was considered crucial.
concept and it is very difficult to measure.
To determine the inter-rater reliability, two different clinical
The subjects’ current employment status, whether full or staff were asked to separately complete the interview using JPAT
part time and their current treatment therapy was compared for the 20 subjects. In this case, the 20 subjects completed three
using the Pearson Correlation. Results showed that patients interviews conducted separately by the investigator (No. A),
who were currently working or houseworking were likely to be and the two clinical staff (No. B & C). The clinical staff received
home therapy candidates. The Pearson correlation was signifi- training on how to observe and administer JPAT in a standard
cant (0.734–1.000, p < 0.001) (Table 7).
and consistent manner. Analysis was conducted at the end to
assess the degree of agreement between the three raters.
Current
Employment
“For tables that have the same categories in the column as
treatment status
in the rows (for example, measuring agreement between two
raters), select Cohen ’s Kappa” (SPSS for Windows, Version 8.0).
A Kappa value of 0.4 or above is considered fair while a Kappa
value of 0.6 is considered good. The Kappa values for this study
were calculated and found to range between 0.444 and 1.000
with an average value of 0.909 (Table 8).
Current
Pearson
treatment
Correlation
Sig. (2-tailed), p =
N
1.000
20
0.734
0.000
20
Employment Pearson
status
Correlation
0.734
1.000
20
Sig. (2-tailed), p = 0.000
Discussion
N
20
Table 7: Employment status & current treatment
The JPAT was found to be useful in assessing patient conditions
40
EDTNA|ERCA JOURNAL 2001 XXVII 1

conceptual areas were reviewed and it was acknowledged that
slight variations between raters were to be expected and was not
sufficient to jeopardise reliability of the tool.
Item
Rater
Measurement
of agreement
(
Kappa)
1.000
1.000
1.000
1.000
1.000
1.000
1.000
1.000
1.000
0.643
1.000
0.444
1.000
1.000
1.000
0.643
0.643
1.000
Physical stability
No. A Versus No. B
No. B Versus No. C
No. A Versus No. C
Conclusion
The primary aim of this study was to validate a pre-training
assessment tool for home dialysis (JPAT). By using typical target
population for the pilot study, the generalisability of the result
was increased.
Nutritional status No. A Versus No. B
No. B Versus No. C
No. A Versus No. C
Communication
No. A Versus No. B
No. B Versus No. C
No. A Versus No. C
No. A Versus No. B
No. B Versus No. C
No. A Versus No. C
Overall, results suggest that JPAT is sufficiently reliable to be
used for patients with ESRD. While a higher reliability would
have been achieved by a further study with a larger sample, this
pilot study provides a realistic approximation to the reliability
of the tool in actual research and clinical applications.
There is currently no screening instrument for home dialy-
sis programmes in Australia, which adequately serves the pur-
pose of identifying patients’ health problems prior to training. If
the research recommendations are implemented, this assess-
ment tool will provide valuable data to the home dialysis com-
munity. This will further ensure efficient utilisation of health
resources and contribute to improving the quality of life for all
home dialysis patients. JPAT would also be useful as a tool for
Psychological
suitability
Ability to maintain No. A Versus No. B
self-care
No. B Versus No. C
No. A Versus No. C
No. A Versus No. B
No. B Versus No. C
No. A Versus No. C
Social support
Table 8: Agreement between raters (Inter-rater reliability) for JPAT
and needs prior to dialysis training. Correlation between vari- on going assessment of current dialysis patients as their physi-
ous components of the tool and follow-up data were consis- cal and mental status can change as times goes by.
tently high indicating the ability of the tool to assess patients in
A study involving a larger sample and an additional site was
keeping with other lengthier forms of assessment. Further, the recommended and conducted and the results will be discussed
JPAT has the advantage of being able to conduct pre-training in the future.
assessment for home dialysis in a significantly shorter period of
time.
Variations in the rater ’s measures of psychological suitabili-
ty and social support components may be due to the subjective
nature of a portion of this assessment. The items related to these
Received: May 2000
Revised: July 2000
Accepted: August 2000
Address for correspondence
Josephine Chow and Lydia Bennett
Renal Unit
Liverpool Health Service
Locked Bag 7103
Liverpool BC NSW 1871
Australia
References
1
. Chow J. A pre-training assessment tool for home dialysis. European Dialysis and Transplant Nurses’ Association/European Renal Care Association (EDTNA/ERCA)
1998; 24(4): 30-31.
2. Thomas NM. Who does best on CAPD? A study to identify self-care determinants. European Dialysis and Transplant Nurses’ Association/European Renal Care
Association (EDTNA/ERCA) 1997; 23(3): 17-21.
3. Green LW. and Lewis FM. Measurement and evaluation in health education and health promotion. Mayfield Publishing Company, California 1986.
4. Argyrous G. Statistics for Social Research. Macmillan Education Australia Pty. Ltd, Melbourne 1996.
5. McDowell I. and Newell C. Measuring Health. A guide to rating scales and questionnaires. Oxford University Press, New York 1996.
6. Beanland C. Schneider Z. LoBiondo-Wood G. and Haber J: Nursing Research – Methods, Critical Appraisal & Utilisation. Mosby Publishers Australia Pty. Ltd.,
Sydney 1999.
EDTNA|ERCA JOURNAL 2001 XXVII 1
41