DISCUSSION
being evaluated, is the fact that only journal article
evaluations can be performed using a synchronous impact
factor. Diachronous impact factors, on the other hand, can
also be calculated for one-off publications, such as books
containing contributions of different authors, or confer-
a database host that provides access to the citation data-
bases produced by ISI, e.g. Dialog as used by CIS in an
evaluation of nine Danish research centres carrying out
environmental research. In this study a range of different
indicators were calculated including baseline values that
make it possible to compare the performance of the nine
centres with the domain impact for the research carried
out in Denmark and in the world. These domain impact
baselines can be tailored to match the research profile of
each individual centre. A combination of data collection
methods was used: citations to the individual articles and
data for calculation of impact factors were collected
online, whereas data from National Science Indicators
(NSI) were used to establish the baselines for domain im-
pact. As all indicators are calculated using a diachronous
methodology, they are directly comparable. (See ref. [14]
for an account of the methodological aspects of the study).
Although the methodology for calculating diachro-
nous impact factors has been known for some time and
applied in a number of evaluation studies as indicated
above, diachronous impact factors have not yet gained
widespread recognition and use outside the bibliometric
community. The main reason for this is probably that im-
pact factors from JCR are readily available. Thus it is
more common that impact factors from JCR are used in
measurements of research performance. Yet, ill-consid-
ered use of these may lead to wrong conclusions.
One typically assigns impact factors of journals, as
obtained from the JCR, to each article published by the
scientists (or scientist) under evaluation. These impact
factors are then cumulated and maybe averaged. The re-
sulting number is seen as a measure of the total or average
impact. In this simple way the synchronous impact factors
from JCR are used directly as a measure of research per-
formance. The demand for resources is quite low com-
pared to actually collecting citations to each individual
article, because the same impact factor from JCR will be
assigned to all articles from the same journal. Although it
is a cheap and easy way to evaluate research there are a
number of problems associated with this method that af-
fect the validity of the results obtained.
[
10]
ence proceedings . Scientists often complain that con-
tributions to such publications are not taken into account
[
9]
during research evaluation exercises . In some fields the
most important results are made public during conferences.
The fields of computer sciences and of information tech-
nology are cases in point. If conferences are held on a
regular, e.g. yearly, basis then it is possible to extend the
pool of source items (often ISI’s database in practice) to
cover this series of conference proceedings, and use it as a
basis for diachronous impact factors. This has been done
for the first and second international conference on bib-
[
10]
liometrics, scientometrics and informetrics . Note that
the number of citations, and hence the impact factors of
journals must be recalculated accordingly (incorporating
citations in these proceedings). Finally, it is even possi-
ble (and perhaps useful if only for theoretical reasons) to
calculate diachronous impact factors for journal volumes,
issues or even subsections of an issue.
3
Some further comments on research evaluation
and the use of impact factors in general
Obviously, for a librarian the long-term impact (per-
haps 10-year) is of more considerable importance than the
short-term (2-year) impact of a journal. Using different
generalised impact factors, or different windows, allows a
comparison of the long-term versus the short-term journal
[
11]
impact. Garfield performed such an investigation. He
found that journals such as Cell, The New England Jour-
nal of Medicine, Proceedings of the National Academy of
Sciences, Nature, and Science always had a high impact,
whatever the period chosen (two, seven or fifteen years).
Other journals moved up or down significantly. Letter
journals in particular suffered considerable downward
changes in ranking.
In the evaluation studies mentioned above data for
impact factor calculations have often been extracted from
databases that are constructed especially for research
evaluation purposes. At the Centre for Informetric Studies
(
ⅰ) Journal impact is not equal to article impact.
Firstly, it is important to note that the impact factor of a
journal is a measure based on the average number of cita-
tions of all the articles in the journal, and that the impact
of an individual article may be considerably different from
the overall impact factor of the journal. Indeed it has been
shown that in a typical journal a few articles receive most
of the citations, and consequently contribute heavily to the
(CIS) in Copenhagen an alternative data collection strat-
egy has been developed in which data are collected di-
rectly from the online versions of the ISI (Institute for
[
12, 13]
Scientific Information) databases
. This makes it pos-
sible to use diachronous impact factors for research
evaluation purposes without having to construct a data-
base especially for this purpose. This method is also ac-
cessible to a broader group of people, e.g. research li-
brarians and other information professionals working in a
range of institutions. All that is needed is online access to
[
15]
impact factor . Differences between the few articles
receiving many citations and the large numbers of normal
[
16]
articles that receive few or no citations will be masked
.
Assigning JCR’s journal impact factor to each article
could well result in an unfair assessment of research im-
526
Chinese Science Bulletin Vol. 46 No. 6 March 2001