Full text of DOI:10.1111/j.1478-4408.2003.tb00180.x

Evaluation of daylight simulators. Part 2:
Assessment of the quality of daylight
simulators using actual metameric pairs
Coloration
Technology
†
H Xu, M R Luo and B Rigg
Colour and Imaging Institute, University of Derby, Derby DE22 3HL, UK
Received: 7 April 2003; Accepted: 2 June 2003
Society of Dyers and Colourists
This paper discusses two methods for evaluating the quality of daylight simulators, namely the band-
value method specified in British standard BS 950 and the CIE metamerism index method. Six daylight
simulators of various types and manufacturers were used for the assessment and a psychophysical
experiment was conducted to evaluate the two methods. A range of 70 metameric pairs was assessed
by a panel of observers under each of the six simulators and the reliability of their visual results was
examined in terms of observer accuracy and repeatability. The visual results were also compared for
each pair of simulators to reveal how the results could be affected by using different simulators. The
effectiveness of four colour difference formulae was tested using the visual results. Finally, four methods
were developed using two statistical measures of performance factor, band-value deviation and CIE
metamerism index for evaluating the quality of daylight simulators.
Introduction
this. These include the CIE metamerism index [8]; the band
value method specified in BS 950 [2]; and the ‘goodness of
fit’ of SPD suggested by Wyszecki [12].
The aim of the present study was to investigate which
method gives the most accurate prediction of visual results
based upon the visual assessment of actual metameric pairs.
The study has concentrated on the band value method and
the CIE metamerism index method, as they are the ones
most widely used in industry.
This paper is Part 2 in the series ‘The impact of practical
daylight simulators on industrial colour control’. In Part 1, a
number of D65 and D50 simulators currently used in
industry were evaluated and their colorimetric and spectral
characteristics reported [1]. Large variations were found
between these simulators. In addition, a number of discrep-
ancies were noted between the various national and inter-
national standards specifying the characteristics of daylight
simulators. These included the British standard BS 950 [2];
the American standards ANSI PH 2.31, ASTM D1729-96
and AATCC EP9 [3–5]; the Japanese standard JIS Z 8716 [6];
the CIE standards 13.3, 51 and 51.2 [7–9]; and the ISO
standard 3664 [10]. The British, American and ISO
standards were established mainly for specifying standard
viewing conditions for industrial and commercial applic-
ations. As daylight illumination is one of the most important
viewing requirements in practice, CIE daylight illuminants
were chosen as sources for these standards.
BS 950 band value method
British Standard BS 950:1967 Specification for Artificial
Daylight for the Assessment of Colour was one of the earliest
standards for specifying a daylight simulator. The standard
is divided into two parts, Part 1: Illuminant for colour
matching and colour appraisal, and Part 2: Viewing
conditions for the graphic arts industry. The CIE D65 and
D50 illuminants were chosen as the standard sources for
Parts 1 and 2, respectively. For D65 simulators, the standard
x and y chromaticities were given, together with a 12-sided
polygon defining the acceptable range. For D50 simulators,
only the standard chromaticities were indicated. For each
daylight simulator, the difference between the SPD of the
simulator and that of the CIE illuminant was required to lie
within certain limits, the band value tolerance. The concept
of band value, also termed band luminance, had earlier been
proposed by Crawford in his study on colour rendering [13].
The essence of the method is to simplify the SPD of a light
source by dividing the spectrum into a number of bands. A
band value, which is an integration of the SPD of the light
source and the CIE standard photometric observer V(λ)
function, is defined for each band. This method was later
adopted in the British Standard for specifying the quality of
daylight simulators. In this standard, a set of standard band
values was specified for the two CIE illuminants D65 and
D50. Each set included values for six visible bands and two
The specifications for daylight simulators are the key
component of these standards. Colorimetric features, such
as chromaticity coordinates, correlated colour temperature
(CCT), colour rendering index and metamerism index, are
generally taken into account when describing the quality of
daylight simulators. According to the CIE recommendations
[7–8,11], the spectral power distribution (SPD) of a daylight
simulator determines its colorimetric performance. The
quality of an individual daylight simulator therefore depends
primarily on its SPD, and it is important to find an
appropriate method to relate the SPD to the corresponding
CIE illuminant. Some techniques have been developed to do
†
This publication is sponsored by the Society’s Colour
Measurement Committee; for part 1 of this paper, see reference 1
Web ref: 20030501
© Color. Technol., 119 (2003) 253

ultraviolet bands. When evaluating the quality of a daylight
simulator using the BS 950 method, the SPD of the simulator
is first multiplied by the V(λ) function and a luminous power
value generated. Subsequently, the luminous power value
for each wavelength is summed for each band, as defined
in Table 1. Note that for the wavelength bordering two bands,
the luminous power value is divided by two, and half used
in each band.
(CIELAB or CIELUV) were calculated for each metamer
under both the CIE illuminant and the test simulator using
the CIE 1964 Supplementary Colorimetric Observer. The
CIE metamerism index, which is the average colour
difference of the metamers, was defined for both visible and
ultraviolet ranges (denoted as MIvis and MI , respectively).
uv
The test simulator was rated from A to E, based on its
metamerism index value, as shown in Table 2.
Table 1 Band value and band value tolerance specified in BS 950
Table 2 CIE specification on metamerism index category
Band value for 100
lumen flux (mW
for two UV bands)
Colour difference
CIE metamerism
Band value
tolerance (%)
CIELAB
CIELUV
index category
Spectral
band
Wavelength
range (nm)
CIE D65/
CIE D50
CIE D65 CIE D50
<
>
0.25
0.32
A
B
C
D
E
0
0
1
.25–0.5
.5–1.0
.0–2.0
0.32–0.65
0.65–1.3
1.3–2.6
>2.6
UVA
UVB
300–340
340–400
400–455
455–510
510–540
540–590
590–620
620–760
11.2
43.2
0.79
4.70
22.40
0.57
±30
±30
±15
±15
±15
±15
±15
±15
2.0
Visible 1
Visible 2
Visible 3
Visible 4
Visible 5
Visible 6
11.2
23.1
43.7
14.4
6.8
9.60
21.80
44.20
15.80
8.01
The final metamerism index result was expressed as the
index for MIvis followed by that for MI , for example, AB.
uv
It was also specified in this method that the chromaticity
of a daylight simulator should lie within a radius of 0.015
u′ v′ units from the appropriate CIE D illuminant in the
CIE 1976 uniform chromaticity scale diagram. This degree
of chromaticity deviation corresponds to an average of
around 13 ∆E*ab units for a reference white.
A daylight simulator having MIvis rating A means that
the simulator gives almost the same performance as the CIE
illuminant in assessing metamerism in the visible range.
On the other hand, a simulator with MIvis rating E means
it is much poorer than the CIE illuminant in revealing
metamerism.
The metamerism index for the ultraviolet range (MIuv)
is important in the assessment of fluorescent samples. It is
generally desirable that a high quality daylight simulator
should have high metamerism index ratings for both visible
and ultraviolet ranges. In the ASTM standard D1729-96 [4]
and the Japanese standard JIS Z 8716 [6], it is stated that
the CIE metamerism index for a high quality daylight
simulator should be BC or better.
The band values are then normalised so that the sum of
the six visible bands corresponds to a luminous flux of
1
0
10
1
00 lm. Finally, the normalised value for each band is
compared with those listed in Table 1. For a simulator
under test, the deviation of its SPD with respect to that of
the CIE illuminant is quantified as the difference in the
band values. The band value tolerance is also specified in
the standard, and is given as ± 15% for the six visible bands
and ± 30% for the two ultraviolet bands. Table 1 lists the
band compositions and values for both D65 and D50
simulators. The band values specified in this standard
provide a clear indication for judging whether the SPD of
a daylight simulator is close to that of the CIE illuminant.
According to the standard, a daylight simulator having a
band value well inside the tolerance specified is regarded
as fully acceptable in quality.
CIE metamerism index method
This method was first recommended by the CIE in
Publication 51 in 1981 [8]. It uses a limited number of pairs
of virtual metamers, each a perfect match under the CIE
illuminant but potentially a mismatch under a test simulator
designed with SPD corresponding to that of the CIE
illuminant. The method was developed from earlier
research conducted by a number of workers, including
Berger and Strocka [14–15], Richter [16], Nayatani et al.
The CIE method for assessing the quality of daylight
simulators has been established for over 20 years. Some
doubts have yet to be resolved, such as whether the virtual
metamers specified are representative of actual metamers
encountered in practice. The present study was also
intended to clarify these doubts. Figures 1 and 2 show the
reflectance values of the five virtual metamers in the visible
range for testing a D65 simulator, and the location of these
metamers in the CIE a*b* diagram, respectively.
[17,18] and Ganz [19].
In its original version, the method used a range of
Figure 1 shows the two reflectance curves for each of
the five metamers. It can be seen that each metamer has
a different number of cross-overs at different wavelengths.
Figure 2 also shows that these metamers are located
almost only in the first and fourth quadrants of the CIE
a*b* diagram, implying insufficient coverage of the colour
gamut. To evaluate the effectiveness of these metamers,
McCamy proposed various perturbations associated with
metamers for evaluating daylight simulators corresponding
to D55, D65 and D75. For each type of illuminant a set of
eight metamers was provided as part of the CIE method, five
in the visible range (400–700 nm) and three in the ultraviolet
300–400 nm). An updated version was published in 1999
and included a new set of metamers for evaluating the
simulator corresponding to D50 [9]. Colour differences
(
2
54 © Color. Technol., 119 (2003)
Web ref: 20030501

1
.8
.6
.4
.2
0
60
40
(
a)
0
0
0
0
2
0
0
–
–
–
20
40
60
4
00
460
520
580
640
700
λ, nm
1
.8
.6
.4
.2
0
(
b)
–60
–40
–20
0
20
40
60
a*
0
0
0
0
Figure 2 Sample distributions for the five CIE metamers for
visible range
the SPDs of daylight simulators [20], imitating defects
known to occur in the manufacturing process. He
concluded that his study gave reasonable assurance that
the new metamers do in fact assign categories on the
internationally accepted scale of quality well within the
precision required. However, his approach was of a
theoretical nature and further investigation is still needed
for a thorough evaluation of the CIE method.
Visual assessment, employing a wide range of practical
metamers and simulators, is a straightforward approach to
achieve this objective. A recent study conducted by Lam
and Xin reported the evaluation of five daylight simulators
using metameric sample pairs [21]. Their results indicated
that better agreement between instrumental and visual
colour differences was obtained using the SPD of a D65
simulator than using the SPD of the D65 illuminant itself.
However, this finding is not surprising because the
simulator was used for the visual assessment. An earlier
study by Kuo and Luo reported a similar finding [22]. In
the present study, 70 actual metameric pairs were used to
evaluate six simulators of different types. Both the band-
value method and the CIE metamerism index method were
tested using the visual results.
4
00
460
520
580
640
700
λ
, nm
1
.8
.6
.4
.2
0
(
c)
0
0
0
0
4
00
460
520
580
640
700
λ, nm
1
.8
.6
.4
.2
0
(d)
0
0
0
0
Experimental
D65 simulators
Six D65 simulators from four manufacturers (General
Electric, GretagMacbeth, Toshiba and VeriVide) were used.
These simulators represent different types, and include one
filtered tungsten lamp, four broad-band fluorescent lamps
from different manufacturers, and one three-band
fluorescent lamp. The filtered tungsten lamp was fitted into
a commercial viewing cabinet, while the fluorescent lamps
were housed in a purpose-built viewing cabinet in order
to accommodate the slight differences in tube length. The
interior paint inside the two cabinets was different in
lightness, the former having a Munsell Value of N7 and the
latter N5. For simplicity, these lamps are described as
simulators A to F, in which A was the filtered tungsten
lamp, B to E the broad-band fluorescent lamps and F the
three-band fluorescent lamp.
4
00
460
520
580
640
700
λ, nm
1
.8
.6
.4
.2
0
(
e)
0
0
0
0
4
00
460
520
580
640
700
λ, nm
The SPD of each simulator was evaluated by measuring
a white plate made of diffusely reflecting polytetrafluoro-
Figure 1 Reflectance for the CIE metamers for visible range:
metamer (a) 1; (b) 2; (c) 3; (d) 4; (e) 5
Web ref: 20030501
© Color. Technol., 119 (2003) 255

ethylene, using a Minolta CS1000 tele-spectroradiometer.
The plate was placed in the centre of the cabinet floor and
was measured at approximately 0/45° geometry. The
measured spectral radiance was corrected using the known
reflectance function for the plate. The telespectroradi-
ometer employs a diode array detector and conformed to
the National Physical Laboratory standard. The accuracy
of measurement indicated in the instrument manual was
500
(
a)
4
00
300
00
2
100
0
±
4% for luminance and ± 0.0015 and ± 0.0010 for the x
3
80
480
480
480
480
480
480
580
680
680
680
680
680
680
780
780
780
780
780
780
and y chromaticities, respectively, for a transfer standard
of a tungsten source. The auto-measurement mode was
used, in which a sensor detects the target stimulus and
automatically sets a suitable integration time according to
the amount of light received. The bandwidth and
wavelength interval of the instrument is 5 nm, and the
wavelength ranges from 380 to 780 nm.
λ
, nm
500
(
b)
400
300
2
1
00
00
0
Each simulator had been in use for about 100 h before
measurement started and was assessed on three occasions
(
before, during and after the experiment) over a period of
3
80
580
three months. The stability of the simulators was examined
in terms of variation (difference between maximum and
minimum), which was 4.5% for the luminance level (cd
λ, nm
500
(
c)
4
00
300
00
–2
m ) and 0.001 and 0.002 for the chromaticity coordinates
x and y, respectively. Each simulator was allowed to warm
up for 5 min before starting the measurements, and the
ambient lighting was then turned off.
2
The SPD measurement for each simulator, except for the
three-band fluorescent lamp, was normalised according to
the CIE recommendation, in which the normalised power
at wavelength 560 nm is set at 100. The three-band
fluorescent lamp had relatively low output at 560 nm, so
the maximum power was scaled to 400 to normalise its SPD
at a level similar to the other simulators. The normalised
SPD for the six simulators is shown in Figure 3. The SPD of
all the simulators used in this study (except the three-band
lamp) were scaled to 100 at 560 nm. Simulator F was scaled
to 400 at the wavelength of maximum radiance.
100
0
3
80
580
λ, nm
500
(
d)
400
300
2
1
00
00
0
Table 3 summarises the colorimetric results for the
simulators studied, including the chromaticity coordinates
x, y and u′ , v′ ; the chromaticity deviation from the CIE
3
80
580
λ, nm
1
0
10
illuminant ∆u′ v′ ; CCT, the CIE colour rendering index
500
1
0
10
(
e)
(
(
Ra) [7]; and the CIE metamerism index for the visible range
MIvis). The CIE 1964 Supplementary Colorimetric
400
Observer was used, as specified in CIE 51.2 [9].
300
Table 4 lists the band value and band deviation results
as specified in BS 950 for the six simulators investigated.
Only the six visible bands were considered. The band
deviation results exceeding the BS 950 tolerance of ± 15%
are noted in bold italics. The results clearly indicate that
simulators C and F cannot be regarded as high quality
simulators. In addition, the summation of the absolute value
of the band deviation for the six visible bands, an index
proposed by the present authors to represent the total band
deviation for a simulator, is also included in the table.
The colorimetric results illustrated in Table 3 show
some variations among the six simulators investigated.
Regarding the luminance level, simulator F, the three-band
fluorescent lamp, had the highest value and A, the filtered
tungsten lamp, the lowest. Bearing in mind that all the
simulators using fluorescent tubes were measured at the
same wattage, it is evident that the three-band fluorescent
lamp exhibited high energy efficacy in comparison with
2
1
00
00
0
3
80
580
λ, nm
500
(
f)
4
00
300
00
2
100
0
3
80
580
λ, nm
Figure 3 SPDs of the six D65 simulators investigated in (a) to
(f), corresponding to Simulators A to F, respectively
2
56 © Color. Technol., 119 (2003)
Web ref: 20030501

Table 3 Colorimetric results for the six D65 simulators
Simulatora Lamp type
Band type
L (cd/m2)
x
y
u′10
v′10
∆u′10v′10 CCT (K) Ra
MIvis
A
B
C
D
E
Filtered tungsten
410.7
562.8
550.8
534.8
491.9
985.2
0.3127 0.3311 0.1978
0.3087 0.3259 0.1956
0.3158 0.3317 0.2000
0.3156 0.3272 0.2016
0.3198 0.3314 0.2012
0.3135 0.3212 0.2025
0.4701 0.0006
0.4689 0.0024
0.4720 0.0033
0.4694 0.0037
0.4738 0.0054
0.4654 0.0062
6486
6751
6296
6326
6090
6529
96 0.18 (A)
98 0.26 (B)
93 0.70 (C)
93 0.82 (C)
97 0.34 (B)
88 2.30 (E)
Fluorescent
Fluorescent
Fluorescent
Fluorescent
Fluorescent
Broad band
Broad band
Broad band
Broad band
Three-band
F
a The same lamps were analysed in an earlier paper [23], although the colorimetric results shown in this paper are slightly different from those in the earlier paper; the
reason for this is that in the current paper the average of three measurements were used in the calculations
Table 4 Band value and band deviation for the six D65 simulators
Band valuea
Deviation of band valuea (%)
Sum
Simulator Band 1 Band 2 Band 3 Band 4 Band 5 Band 6 Band 1 Band 2 Band 3 Band 4 Band 5 Band 6 (abs.)
A
B
C
D
E
0.8
0.8
0.8
0.9
0.7
1.0
10.8
11.6
9.5
9.6
11.2
8.7
21.9
21.3
22.0
21.1
20.2
10.9
46.3
45.4
47.0
47.4
45.6
57.1
13.7
13.8
13.5
14.5
15.0
18.7
6.6
7.0
7.2
6.6
7.2
3.7
1.6
4.3
4.0
7.6
–6.2
25.7
–3.8
3.4
–15.2
–14.7
0.0
–5.3
–7.7
–4.9
–8.6
–12.6
–52.8
5.8
4.0
7.5
8.5
4.4
–4.6
–4.1
–6.1
0.7
4.5
29.6
–3.6
2.9
5.6
–3.5
5.9
–45.9
24.9
26.4
43.3
43.6
33.7
206.9
F
–22.3
30.6
a Band 1, 400–455 nm; band 2, 455–510 nm; band 3, 510–540 nm; band 4, 540–590 nm; band 5, 590–620 nm; band 6, 620–760 nm
the broad-band types. The daylight simulator using a
filtered tungsten lamp was much less energy efficient than
the fluorescent simulators.
With regard to chromaticity, simulator F showed the
largest chromaticity deviation (0.0062) from the CIE D65
illuminant, and simulator A the least (0.0006). These
results all lie well within the CIE chromaticity tolerance
of 0.015 [9].
Sample preparation
The 70 metameric pairs prepared by Kuo and Luo [22]
were again used in this study. These samples were produced
by dyeing plain wool serge, and were mounted in two layers
on stiff card (3 × 3 in). A grey scale (referred to as ‘grade
G’ in this current study) with five gradations (grade 1 to 5)
was also prepared using matt paint, of similar size to the
textile samples. The darkest grade (grade 5) was duplicated
and taken as the standard. The CIE L*a*b* values for each
grade and the colour difference results between the
standard and each grade are listed in Table 5.
The summary in Table 5 shows that all grades were quite
close to neutral and had good colour constancy under the
six simulators investigated, plus the CIE D65 and A
illuminants. A GretagMacbeth Color-Eye 7000A spectro-
photometer was used for all the measurements. The
measurement wavelength ranged from 400 to 700 nm at
10 nm intervals. The measurement conditions were set at
large aperture, specular reflection and ultraviolet included.
The simulators were measured at 5 nm interval from 380
to 780 nm. Weightings which integrate the SPD of a light
source and the CIE 1964 Supplementary Colorimetric
Observer were calculated for each simulator at 5 nm
intervals. These weights were then abridged to 10 nm
intervals using third order Lagrange polynomials to calculate
the tristimulus values of each colour sample [24].
For the CIE general colour rendering index (R ),
a
simulator B gave the best result (98) and simulator F the
poorest (88). Simulator A had the best rating (A) for the
CIE metamerism index and simulator F gave the worst
rating (E). The other simulators had either B or C ratings.
The colour rendering index indicates the closeness of
match in colour appearance for given test samples between
a reference CIE daylight illuminant and a test simulator.
This is different from the metamerism index, which is an
assessment of the change in colour difference within pairs
of samples using a reference illuminant and a simulator.
The results show that simulators with high metamerism
index ratings, such as simulators A and B, also have good
colour rendering property, indicating some degree of
correlation between these two properties.
The band value results in Table 4 show that simulators
having power distribution appropriate for the six visible
bands specified in BS 950 (e.g. simulators A and B), gave
band deviation results well within the BS 950 tolerance of
The colour change for each sample during the whole
experimental period was investigated. All the samples were
measured three times, before, during and after the
experiment. For each sample, the colour difference (CIELAB)
was calculated between an individual measurement and the
mean of the three measurements. The mean/maximum
values for all the samples were 0.1/0.3 under CIE D65
±
15%. These simulators also displayed good overall quality,
with high metamerism index ratings and good colour
rendering indices. On the other hand simulator F had band
deviation close to or exceeding the BS 950 tolerance at one
or more of the six bands, and at the same time had a poor
metamerism index rating and colour rendering index.
Web ref: 20030501
© Color. Technol., 119 (2003) 257

Table 5 CIE L*a*b* and colour difference results for each grade under simulators A–F, plus CIE D65 and CIE A
illuminants
Simulator A
L* a*
53.89 –0.23 –0.49 15.19 53.89 –0.24 –0.51 15.19 53.89 –0.23 –0.52 15.19 53.89 –0.19 –0.50 15.18
Simulator B
Simulator C
Simulator D
Source
b*
∆E*ab L* a*
b*
∆E*ab L* a*
b*
∆E*ab L* a*
b*
∆E*ab
Grade 1
Grade 2
Grade 3
Grade 4
Grade 5
46.83 –0.11 –0.60 8.13 46.83 –0.10 –0.63
43.59 –0.79 –0.28 4.96 43.59 –0.79 –0.33
40.92 –0.38 –0.26 2.26 40.92 –0.38 –0.29
8.14 46.82 –0.11 –0.65
4.96 43.59 –0.81 –0.33
2.27 40.92 –0.39 –0.30
8.13 46.83 –0.07 –0.63
4.96 43.59 –0.77 –0.29
2.26 40.92 –0.36 –0.26
8.13
4.95
2.26
0.03
38.70
0.08 –0.22 0.03 38.70
0.09 –0.24
0.03 38.70
0.07 –0.25
0.03 38.71
0.09
0.23
Simulator E
L* a*
53.89 –0.26 –0.50 15.19 53.89 –0.12 –0.54 15.17 53.89 –0.25 –0.48 15.19
Simulator F
CIE D65 illuminant
CIE A illuminant
Source
b*
∆E*ab L* a*
b*
∆E*ab L* a* b*
∆E*ab L* a* b*
∆E*ab
Grade 1
Grade 2
Grade 3
Grade 4
Grade 5
53.82 –0.49 –0.60 15.14
46.82 –0.13 –0.61 8.13 46.83
0.01 –0.67
8.13 46.83 –0.13 –0.60
4.97 43.58 –0.82 –0.29
2.28 40.92 –0.40 –0.27
8.13
4.96
2.26
0.03
46.77 –0.42 –0.68
43.47 –1.06 –0.52
40.85 –0.60 –0.40
38.69 –0.14 –0.21
8.09
4.87
2.21
0.03
43.58 –0.83 –0.31 4.95 43.61 –0.71 –0.33
40.92 –0.40 –0.27 2.26 40.96 –0.30 –0.29
38.70
0.07 –0.22 0.03 38.71
0.15 –0.27
0.03 38.70
0.07 –0.22
illuminant. For the 70 metameric pairs, the mean/maximum
colour differences (CIELAB) were 1.9/4.9 under CIE D65
illuminant and 7.1/14.5 under CIE A illuminant. The sample
distributions under CIE D65 and CIE A illuminants are
shown in Figure 4.
8
6
4
2
0
0
0
0
0
(
a)
L* > 60
40 < L* < 60
L* < 40
It can be seen that the sample pairs cover a large colour
and lightness area. Each vector in Figure 4 represents a pair
of samples. Figure 4a shows that the majority of sample
pairs were a close match for one other; in other words, the
magnitudes of the vectors were quite small. This was to be
expected because these pairs were aimed to be a good
match under illuminant D65. Note that it was not possible
to produce an exact match under illuminant D65, even
though a number of corrections were made to the dye
formulation. The majority of the metamers were visually
assessed to give a reasonable match between the two
samples. The others were chosen because, although they
had a quite large perceived colour differences under D65,
they had much larger colour differences under the other
light source, for example illuminant A. However, the
magnitude of the vectors is much larger in Figure 4b
–
–
–
20
40
60
–
60
–40
–20
0
20
40
60
80
a*
80
(b)
(
illuminant A), indicating a high degree of metamerism in
6
4
2
0
0
0
0
these pairs. Figure 4b also shows that the majority of vectors
are almost always shifted in the red–green direction.
Psychophysical experiment
A panel of 10 observers, four male and six female, aged
between 23 and 38, took part in assessing the colour
differences of the 70 metameric pairs under each of the D65
simulators. These were either students or staff of the Institute
and all had previous experience of visual assessment. All
the observers were assessed using the Ishihara test and found
to have normal colour vision. The grey scale method, which
has been extensively used by many researchers [21–22,25,
–20
–
–
40
60
2
6], was used in the experiment. The whole experiment was
conducted using a viewing cabinet located in a dark room.
Before commencing the experiment, each observer was first
asked to allow his or her vision about 2 min to adapt to the
interior grey wall of the viewing cabinet, illuminated by the
–
60
–40
–20
0
20
40
60
80
a*
Figure 4 Sample distribution of the 70 metameric pairs under
(a) CIE D65 and (b) CIE A for a range of values of L*
2
58 © Color. Technol., 119 (2003)
Web ref: 20030501

simulator under test. The experimenter then chose a
metameric pair at random from the 70 on offer and placed
it in the viewing cabinet. Each observer was then asked to
judge the colour difference shown by this metameric pair
in terms of grade G. The same pair was assessed twice in
separate sessions by each observer. In this way a total of
If ∆E and ∆V follow a perfectly linear relationship, CV
i i
will be zero. The larger the CV value, the greater the degree
of independence between ∆E and ∆V .
i
i
The gamma factor (γ) is a measure of the proportional
relationship between two set of data, here ∆E and ∆V . The
i
i
logarithm of ∆E /∆V usually displays a normal distribution
i
i
2
0 observations were accumulated for each metameric pair
and the standard deviation, ln (∆E /∆V ), is used as a
i i
under one test simulator.
measure of fit (Eqn 4). Vab is derived as in Eqn 5.
Analysis of the visual results
2
N 
As the raw experimental data in terms of grade G is not
directly proportional to the visual colour difference ∆V, a
fourth-order polynomial was constructed to correlate these
two parameters, as shown in Eqn 1:





1
∆E
∆E


i 

i 

ln(γ) =
ln
−ln
∑




(4)
N

∆V 

∆V 

i
i
i=1 

4
3
2
∆
V = 0.14117G −1.97900G +10.28683G
N
2
(
1)
1
N
(∆E − F∆V )
i
i
−
26.18400G +32.92700
VAB =
∑
∆E F∆Vi
i
i=1
where the coefficients of the equation were derived by
relating the CIELAB colour difference ∆E*ab (given in Table
where
(
5)
5
) to the corresponding grade G.
N
N
∆
∆
Ei
Vi
∆Vi
The grey scale exhibited good reproducibility. The
F =
∑
∑
∆Ei
i=1
CIELAB colour difference for each grade under the test
simulators are almost identical to those under CIE D65
illuminant, as shown in Table 5. Therefore, the coefficients
in Eqn 1 were actually optimised using the colour
difference ∆E*ab under CIE D65 and the equation is equally
applicable for all the test simulators.
i=1
For perfect agreement, CV, Vab and PF/3 should all be
zero and γ equal to 1. A PF/3 value of 30 means the
disagreement is about 30% between the two sets of data.
The performance factor (PF/3) may be used to indicate
the agreement between two sets of data, for example
instrumental colour difference (∆E ) and visual colour
i
Results and Discussion
difference (∆V ). This measurement was originally
i
developed by Luo and Rigg [25], and has since been used
by many researchers [21–22,26]. The performance factor
is calculated by combining three different statistical
measures, the coefficient of variation (CV), the gamma
Visual reliability
The reliability of the visual results was evaluated in terms
of observer accuracy and repeatability in PF/3 units. For
each simulator, the mean of the ∆V values for each pair
was recorded. Its accuracy for each observer was then
calculated as the PF/3 measure between each observation
and mean visual results for each of the metameric pairs.
The two PF/3 values for each observer were then averaged
to represent his or her accuracy. The repeatability was
calculated similarly for the two results for each observer.
Table 6 gives the accuracy and repeatability results for
all observers.
The observer accuracy results in terms of the average
across the six simulators in Table 6 range from 27 to 42,
with an average PF/3 value of 33. The observer repeatability
results range from 20 to 32, with an average of 26. These
results are quite close to those reported by Kuo and Luo in
factor (γ) and (V ), with a suitable weighting for each
ab
according to Eqn 2. The smaller the PF/3 value, the better
the agreement between two data sets being compared. If
agreement is perfect, PF/3 is zero.
PF /3 = 100(γ −1+VAB +CV /100)/3
(2)
The coefficient of variation (CV) is a measure of the
deviation from a linear relationship between two sets of
data, in this case ∆E and ∆V . It is the root mean square
i
i
deviation of the two (suitably scaled) as a percentage of the
mean ∆E value, as illustrated in Eqn 3.
1
995 [22], in which a similar visual assessment was
N
conducted under a D65 simulator. The typical observer
accuracy in their study was 33, exactly the same as that in
the present study. The typical repeatability found by Kuo
and Luo was 39, 13 units worse than that in the current
study.
1
2
∑(
∆E − f∆V )
i
i
N
i=1
CV =
×100
∆E
where
The results did not reveal a significant difference in
either observer accuracy or repeatability under the six
simulators investigated. It is interesting to note that observer
4 gave the most repeatable results, but the least accurate
results against the panel results (Table 6). This implies that
these two measures do not have a direct relationship.
N
(
∆E ∆V )
(3)
∑
i
i
N
1
N
i=1
f =
, ∆E =
∆Ei
N
∑
∆Vi)2
i=1
∑(
i=1
Web ref: 20030501
© Color. Technol., 119 (2003) 259

Table 6 Observer accuracy and repeatability results (PF/3) for the various simulators A–F
PF/3 (Observer accuracy) PF/3 (Observer repeatability)
Observer
A
B
C
D
E
F
Av.
A
B
C
D
E
F
Av.
1
2
3
4
5
6
7
8
9
27
52
35
38
42
32
29
36
35
37
36
52
27
38
32
40
40
41
32
23
43
41
36
37
33
36
32
45
30
43
29
31
32
32
34
26
34
40
48
32
39
35
24
25
27
33
27
29
34
40
40
24
25
22
21
29
29
24
41
29
41
25
27
20
37
33
26
30
41
20
29
37
35
42
35
33
27
32
31
31
33
42
27
30
31
26
22
30
21
26
29
26
23
26
31
21
30
35
24
22
33
34
21
24
31
32
29
35
21
31
32
24
21
27
34
21
28
23
27
27
34
21
25
31
20
24
37
33
31
25
17
21
26
37
17
26 24
33 27
26 27
16 17
34 30
22 25
22 17
29 29
15 13
26 26
25 23
34 30
15 13
27
31
25
20
32
28
23
27
21
26
26
32
20
1
Av.
Max.
Min.
0
43 45 48 40
23 29 24 21
Performances of colour difference formulae
Table 7 Performances of different colour difference
formulae (PF/3) for the various simulators A–F
The experimental results were used to test four colour
difference formulae, CIELAB [11], CIE94 [27], CMC [28]
and CIEDE2000 [29]. CIE tristimulus values, calculated
from the appropriate energy distribution for each simulator,
were used to calculate the ∆E values. The PF/3 value was
calculated between the ∆E value predicted by the formula
for a specific simulator and the visual colour difference,
Formula
A
B
C
D
E
F
Av.
45
CIELAB
46
45
52
39
46
42
CIE94 (1:1:1)
CMC (1:1)
CIEDE2000 (1:1:1) 39 40 43 27
Av. 42 43 46 32
43 44 44 32
40 41 45 30
40 34 39
40 34 38
38 35 37
∆
V. These are given in Table 7, together with the mean value
41
36
for each formula and for each simulator.
The results show that for the six simulators studied,
CIEDE2000 performed the best, but no great difference was
found between CIE94, CMC and CIEDE2000. The last three
formulae all outperformed the CIELAB formula, however,
and the average of their PF/3 values was quite close to the
typical observer accuracy of 33. This also demonstrates that
each formula performed quite consistently; they all perform-
ed best with simulator D and worst under simulator C.
Table
8
Correlation of visual results
between every two simulators (PF/3)
Simulator
A
B
C
D
E
F
B
C
D
E
F
16
20
17
11
44
21
18
17
13
52
23
17
13
44
22
11
39
20
Comparing different simulators
39
17
The PF/3 measure was also calculated for the visual
difference ∆V between all possible combinations of two
simulators. The results are given in Table 8. It was found
that simulator E was the most representative simulator, as
its visual results were in closer agreement with those of
the other simulators. Simulator F showed the worst
agreement, which was not unexpected, since this is a three-
band fluorescent lamp, unlike the others which are of the
broad-band type. In general, the visual results obtained
from all simulators except simulator F were quite similar
and gave only small disagreements, about 21 PF/3 units on
average.
This implies that the variation in visual results caused
in practice by using different daylight simulators may be
less significant than observer accuracy and repeatability,
as shown above in Table 6.
The mean visual difference (∆V) for each pair of
metamers was also compared between every combination
of two simulators, using scatter diagrams as shown in Figure
Av.
44
perfect agreement, the data points in the scatter diagram
would all lie on the 45° line. Figures 5e, 5i, 5l, 5n and 5o
show that visual colour differences (∆V) under simulator F
displayed worse agreement than those under the other
simulators. This again illustrates that the three-band
fluorescent lamp is very different from the others in terms
of visual assessment. Figures 5d, 5h, 5k and 5m also
demonstrate that visual colour differences (∆V) under
simulator E were in better agreement than those under the
other simulators. This further confirms that this fluorescent
lamp is highly representative of the D65 simulators
investigated, except for the three-band lamp. The fact that
excellent agreement was obtained in these cases adds
considerable confidence to the mean visual results.
Assessment of the simulators
5
. The degree of scatter in the diagrams reflects the degree
The final analysis was carried out using four methods for
assessing the quality of each simulator investigated. The
of disagreement between each two sets of ∆V data. For
2
60 © Color. Technol., 119 (2003)
Web ref: 20030501

8
6
4
2
0
8
6
4
2
0
8
6
4
2
0
8
6
4
2
0
8
6
4
2
0
(
a)
(b)
(c)
(d)
(e)
0
2
4
V(A)
6
8
0
2
4
V(A)
6
8
0
2
4
6
8
0
2
4
6
8
0
2
4
6
8
∆
V(A)
∆V(A)
∆V(A)
∆
∆
8
6
4
2
0
8
8
6
4
2
0
8
6
4
2
0
8
6
4
2
0
(
f)
(g)
(h)
(i)
(j)
6
4
2
0
0
2
4
6
8
0
2
4
6
8
0
2
4
6
8
0
2
4
6
8
0
2
4
6
8
∆
V(B)
∆V(B)
∆V(B)
∆V(B)
∆V(C)
8
6
4
2
0
8
6
4
2
0
8
6
4
2
0
8
6
4
2
0
8
6
4
2
0
(
k)
(l)
(m)
(n)
(o)
0
2
4
6
8
0
2
4
6
8
0
2
4
6
8
0
2
4
6
8
0
2
4
6
8
∆
V(C)
∆V(C)
∆V(D)
∆V(D)
∆V(E)
Figure 5 Comparing visual colour differences ∆V between all possible combinations of two simulators
results are shown in Table 9. (Note that the results in Tables
and 9 were modified from those reported earlier by the
authors [23]. The reason for this is that the colour
measurement results were updated due to an instrument
drift previously undetected. However, these changes did not
alter the conclusion.)
Secondly, the PF/3 measure was computed between pairs
of colour differences calculated by the SPD of each simulator
and the CIE D65 illuminant. These are expressed under the
headings ‘PF/3 – ∆E*ab (simulator) vs ∆E*ab (CIE D65
illuminant)’ and ‘PF/3 - ∆E*00 (simulator) vs ∆E*00 (CIE D65
illuminant)’. This measure removes the observation errors
inherent in the visual results (∆V). In other words, it is similar
to the CIE metamerism index, but uses many more pairs of
real metamers.
7
Table 9 first lists the PF/3 measure between the visual
results (∆V) and the corresponding colour differences
(CIELAB and CIEDE2000) obtained from the SPD of the CIE
D65 illuminant, expressed as ‘PF/3 – ∆V (simulator) vs ∆E*ab
Thirdly, the CIE metamerism index for the visible range
(MIvis), which is the mean colour difference calculated
using virtual metamers, is listed. Note that the CIE method
calculates CIELAB colour differences, whereas the present
study has also adopted the CIEDE2000 formula in
calculating the metamerism index. Finally, the summation
(CIE D65 illuminant)’ and ‘PF/3 – ∆V (simulator) vs ∆E*00
(CIE D65 illuminant)’. This measure assumes that the visual
results from a high quality simulator should agree well with
the colour differences calculated using the SPD for the CIE
D65 illuminant.
Table 9 Comparing the performance of each investigated simulator using four different methods
Method
A
B
C
D
E
F
PF/3 – ∆V (sim.) vs ∆E*ab (CIE D65 illum.)
PF/3 – ∆V (sim.) vs ∆E*00 (CIE D65 illum.)
PF/3 – ∆E*ab (sim.) vs ∆E*ab (CIE D65 illum.)
PF/3 – ∆E*00 (sim.) vs ∆E*00 (CIE D65 illum.)
CIE MIvis (CIELAB)
43
38
10
12
0.18
0.14
25
46
42
23
23
0.26
0.17
26
53
51
52
59
0.70
0.61
43
51
47
43
37
0.82
0.50
44
47
43
25
25
0.33
0.23
34
66
67
101
103
2.30
1.69
207
CIE MIvis (CIEDE2000)
Sum of absolute band-value deviation (%)
Ranking – ∆V (sim.) vs ∆E*ab (CIE D65 illum.)
Ranking – ∆V (sim.) vs ∆E*00 (CIE D65 illum.)
Ranking – ∆E*ab (sim.) vs ∆E*ab (CIE D65 illum.)
Ranking – ∆E*00 (sim.) vs ∆E*00 (CIE D65 illum.)
Ranking – CIE MIvis (CIELAB), Category
1
1
1
1
2
2
2
2
5
5
5
5
4
4
4
4
3
3
3
3
6
6
6
6
1 (A)
2 (B)
4 (C)
5 (C)
3 (B)
6 (E)
Ranking – CIE MIvis (CIEDE2000)
Ranking – Sum of absolute band-value deviation (%)
1
1
2
2
5
4
4
5
3
3
6
6
Web ref: 20030501
© Color. Technol., 119 (2003) 261

of absolute band value deviation is reported as a percentage
the final column in Table 4). This index is proposed by
the present authors as a method of representing the total
band value deviation of a simulator. Using all these results,
the performance rankings and the CIE metamerism index
category for each simulator are also listed.
that the band value method reveals information about the
spectral deviation of a daylight simulator relative to the
corresponding CIE daylight illuminant. It provides
information for judging whether the band power is adequate
and would be valuable to lamp manufacturers by providing
the band value figures. However, the CIE metamerism index
has in the past been widely used in the lighting industry.
Furthermore, for a colourist who conducts day-to-day visual
colour assessments under daylight simulators, the
metamerism index results using the CIE method are more
informative.
(
In Table 9, the PF/3 value derived from the visual colour
differences is seen to vary much less than the PF/3 value
calculated from the colour differences. In addition, the two
colour difference formulae, CIELAB and CIEDE2000, do not
give a satisfactory result with all the simulators tested. This
is a result of the inherent observer variations included in
the visual data. However, the two types of PF/3 results
correspond to the same ranking results for all six simulators
investigated (see the lower part of Table 9). The two colour
difference formulae used in calculating the PF/3 measure-
ment both demonstrate similar value for the evaluation of
daylight simulators. The CIE metamerism index results
generally agree with the two types of PF/3 results, in fact
they give similar rankings for the six simulators. The only
disagreement between the PF/3 results and the CIE
metamerism index results are for simulators C and D, for
which the PF/3 results gave slightly better results for
simulator D, while the CIE metamerism index results show
the opposite.
However, both simulators C and D belong to the same
category (C) according to the CIE specifications [9]; this
disagreement disappears if the CIELAB colour difference
formula is replaced by CIEDE2000 in calculating the
metamerism index. This indicates that the CIE method is
suitable for evaluating the quality of daylight simulators.
Its performance may however be further improved for the
calculation of metamerism index by employing a more
advanced colour difference formula such as CIEDE2000.
The results also illustrate a problem with the CIE method.
simulator B had MIvis of 0.26, only 0.01 ∆E unit outside the
highest category (A). Such a small difference is unlikely to
be noticeable in practice. A similar finding was reported
by Lam and Xin [21]. They concluded that, according to the
statistical analysis results, there was no significant visual
difference in assessing metameric pairs using two D65
simulators, one with metamerism index category of A and
the other in category B. However, this is a problem common
to all methods where results from a continuous scale are
converted to a small number of categories. Simplicity is
achieved, but at the cost of one or two anomalies.
Conclusions
Six daylight simulators were studied. Psychophysical
experiments have been conducted to assess the colour
difference of 70 metameric pairs under each simulator. The
observer variation results showed reasonable accuracy and
repeatability, and good agreement with those from an
earlier study [22]. The visual results were used to test four
colour difference formulae, CIELAB, CIE94, CMC,
CIEDE2000. The last three formulae gave a similar degree
of accuracy in predicting visual data, and all outperformed
the CIELAB formula. In general, all the simulators studied
agreed well with each other in terms of visual results, part
from the one simulator having a three-band SPD. Finally,
four types of measurements were used to quantify the
performance of each investigated simulator, including two
kinds of PF/3 results, CIE metamerism index results and
the summation of absolute band value deviation, specified
in BS950. The results showed that the two new methods,
using a combined statistical measure PF/3, agreed well
with the CIE method and band value method, indicating
that both of the latter are equally reliable for evaluating
the quality of daylight simulators. The results also revealed
that a simulator having a band value deviation well below
the BS 950 tolerance corresponded to a high metamerism
index rating and was therefore of good quality.
Acknowledgements
The authors would like to thank the Society of Dyers and
Colourist for supporting this project. They are also grateful
to all the observers who kindly took part in the psycho-
physical experiments.
The summation of band value deviation gave exactly the
same ranking as the CIE metamerism index results for the
six simulators investigated, indicating that the band value
method is also an effective measure for evaluating the
quality of daylight simulators. Tables 3 and 4 reveal that a
good quality simulator with a CIE metamerism index rating
of A or B, for example simulators A and B, corresponds to
a band value deviation well within the BS 950 tolerance
of ± 15%. The quality of simulators such as simulators C
and F will be affected if their band value deviations exceed
the tolerance in one or more bands.
Although the band value method and the CIE metamerism
index method use quite different approaches for assessing
the quality of daylight simulators, the current study has
shown that both methods are in fact reliable. Each is
therefore useful for industrial application. It should be noted
References
1
2
. H Xu, M R Luo and B Rigg, Color. Technol., 119 (2003) 59.
. BS 950, Specification for Artificial Daylight for the Assessment
of Colour, Part 1: Illuminant for colour matching and colour
appraisal, Part 2: Viewing conditions for the graphic arts
industry (London: British Standards Institution, 1967).
. A Johnson and M Scott-Taggart, Guidelines for Choosing the
Correct Viewing Conditions for Colour Publishing
3
4
(
Leatherhead: PIRA International, 1998) 37.
. ASTM D1729-96, Standard Practice for Visual Evaluation
of Colors and Color Difference of Diffusely-Illuminated
Opaque Materials, in Annual Book of ASTM standards:
Paints, Related Coatings, and Aromatics (Pennsylvania:
ASTM, 1998).
5
. AATCC EP9, Visual Assessment of Color Difference of Textiles
(North Carolina: AATCC, 1999).
2
62 © Color. Technol., 119 (2003)
Web ref: 20030501

6
. JIS Z 8716, Fluorescent lamp as a simulator of CIE standard
illuminant D65 for a visual comparison of surface colours -
Type and characteristics (Tokyo: Japanese Standards Institute,
16. K Richter, Lichttechnik, 24 (1972) 370.
17. Y Nayatani and K Takahama, J. Opt. Soc. Am., 62 (1972)
140.
1
991).
18. Y Nayatani, H Sobagaski and K Takahama, Proc. AIC Color
77, New York, US (Bristol: Adam Hilger Ltd, 1978) 441.
19. E Ganz, Appl. Optics, 16 (1977) 806.
20. C S McCamy, Col. Res. Appl., 24 (1999) 322.
21. Y M Lam and J H Xin, Col. Res. Appl., 27 (2002) 243.
22. W G Kuo and M R Luo, J.S.D.C., 112 (1996) 312.
23. H Xu, M R Luo and B Rigg, 9th Session of the Association
Internationale de la Couleur (AIC Color 01), Rochester, USA
(2001) 697.
24. C J Li, M R Luo and H Huang, International Colour
Management Forum, Derby, UK (1999) 51.
25. M R Luo and B Rigg, J.S.D.C., 103 (1987) 86.
26. G Cui, M R Luo, B Rigg and W Li, Col. Res. Appl., 26 (2001)
394.
7
8
9
. CIE Publication 13.3, Method of measuring and specifying
colour rendering properties of light sources (Vienna: Central
Bureau of the CIE, 1995).
. CIE Publication 51, A method for assessing the quality of
daylight simulators for colorimetry (Vienna: Central Bureau
of the CIE, 1981).
. CIE Publication 51.2, A method for assessing the quality of
daylight simulators for colorimetry (Vienna: Central Bureau
of the CIE, 1995).
0. ISO 3664: 2000, Viewing conditions - Graphic technology
and photography (Geneva: ISO, 2000).
1
1
1. CIE Publication 15.2, Colorimetry, 2nd Edn (Vienna: Central
Bureau of the CIE, 1986).
1
1
1
1
2. G Wyszecki, Die Farbe, 19 (1/6) (1970) 43.
3. B H Crawford, Br. J. Appl. Phys., 14 (1963) 319.
4. A Berger and D Strocka, Appl. Optics, 12 (1973) 338.
5. A Berger and D Strocka, Appl. Optics, 14 (1975) 726.
27. R McDonald and K J Smith, J.S.D.C., 111 (1995) 376.
28. F J J Clarke, R McDonald and B Rigg, J.S.D.C., 100 (1984)
128.
29. M R Luo, G Cui and B Rigg, Col. Res. Appl., 26 (2001) 340.
Web ref: 20030501
© Color. Technol., 119 (2003) 263