Full text of DOI:10.3152/147154302781781001

Science and Public Policy, volume 29, number 3, June 2002, pages 189–200, Beech Tree Publishing, 10 Watford Close, Guildford, Surrey GU1 2EP, England
Emerging technology
Measuring national ‘emerging technology’
capabilities
Alan L Porter, J David Roessner, Xiao-Yin Jin and
Nils C Newman
How can national capabilities to develop emerg-
ing technologies be measured? We use INSPEC
and EI Compendex class codes to examine 33
countries’ research and development activity. We
select candidate emerging technologies based on
the Rand Corporation’s categories. We screen
these to tally those that show strong recent, and
increasing, R&D publication rates. The resulting
measures show strong convergence; indeed, their
lack of divergence is unsettling. Our measures
suggest that China now stands forth as an
ECHNOLOGY INDICATORS aim to meas-
ure capabilities to allow tracking of changes
over time, to inform technology policy
T
(
1
(
Godin, 1996; Mani, 2000; Sirilli, 1997; van Raan,
988). Effective technological innovation indicators
Hansen, 1999) tap into capabilities that drive eco-
nomic development (Antonelli and de Liso, 1997;
Archibugi and Michie, 1998; WCY 1998–2000;
Mani, 2000; OECD, 1999; OECD, 2000). In turn,
these should help in policy formulation (Porter and
Stern, 1999).
In recent years, the world has become increas-
ingly interested in tracking national capabilities for
technological innovation and for technology-based
exports. National level technological indicator de-
velopments include creation of the European Inno-
vation Scoreboard [http://www.cordis.lu/itt/itt-en/
‘
emerging technology’ research power compara-
ble to Germany, the UK, and France. A number
of other nations evidence a striking lack of R&D
activity, posing questions about their longer-
range high-tech competitiveness.
0
1-5-spec/overview.htm], UNIDO’s (UN Industrial
Development Organization) forthcoming Industrial
Development Scoreboard, and the UN Commission
for Science and Technology for Development’s
forthcoming Indicators of Technology Development.
Technology is the key driver of economic com-
petitiveness (Archibugi et al, 1999; Clark and Guy,
Alan Porter is Co-Director of the Technology Policy and As-
sessment Center (TPAC), and Professor Emeritus of Industrial
and Systems Engineering (ISyE) and of Public Policy, ISyE, 765
Ferst Drive, Georgia Tech, Atlanta, GA 30332-0205, USA; Tel:
1
998). ‘Science-driven’ technologies are increasing
their industrial role relative to incremental techno-
logical gains (Lane and Makri, 2000; Grupp, 1995).
Emerging technologies are defined here as those that
could exert much enhanced economic influence in
the coming (roughly) 15-year horizon. The challenge
is to devise ways to measure national propensities in
this regard (Sirilli, 1997).
+
1 404 894 2330; Fax: +1 404 894 2301, E-mail: alan.
porter@isye.gatech.edu. At this time he is Visiting Professor of
Technology, Policy and Management at the Technical Univer-
sity of Delft. J David Roessner is Professor Emeritus of Public
Policy at Georgia Tech, and Associate Director, Science and
Technology Policy Program, SRI International; E-mail: david.
roessner@mindspring.com. Xiao-Yin Jin is Senior Researcher at
TPAC; E-mail: j.xiyiu@isye.gatech.edu. Nils Newman is Presi-
dent, IISC; E-mail: newman@iisco.com.
Since the mid-1980s, Georgia Tech’s Technology
Policy and Assessment Center has been generating
Science and Public Policy June 2002
0302-3427/02/030189-12 US$08.00 Ó Beech Tree Publishing 2002
189

Measuring national ‘emerging technology’ capabilities
competitiveness), socio-economic infrastructure,
technological infrastructure, and productive capa-
city. The output indicators are technological stand-
ing (competitive performance), technological
emphasis, and rate of technological change.
Alan L Porter’s major concentration is technology forecas t-
ing and assessment. He has led development of ‘technology
opportunities analysis’ — mining electronic, bibliographic
data sources to generate intelligence on emerging tech-
nologies. At Georgia Tech, he is Professor Emeritus of
Industrial and Systems Engineering, and of Public Policy.
He directed the Technology Policy and Assessment Center
from 1989–2001, and continues as co-director. He is pres-
ently Director of R&D for Search Technology, Inc, Norcross,
GA, pursuing application of VantagePoint software to ana-
lyze emerging technologies and profile research domains.
He co-founded the International Association for Impact
Assessment (IAIA) in 1980, serving as president (1995–96).
Considerable inertia resists change in the compo-
sition of the indicators, so as to enable comparisons
over time. However, we have recently been engaged
in a significant conceptual overhaul. With support
1
from the National Science Foundation, we are
addressing:
J David Roessner is Associate Director of the Science and
Technology Policy Program at SRI International, Professor
of Public Policy Emeritus at Georgia Institute of Technology,
and Research Associate in the Center for International Sci-
ence and Technology Policy at George Washington Univer-
sity. He is principal author of The Impact of Office
Automation on Clerical Employment, 1985–2000 (Quorum
Books, 1985), editor of Government Innovation Policy: De-
sign, Implementation, Evaluation (St Martin’s Press, 1988),
and editor/co-editor of special issues of Research Policy on
evaluation of government innovation programs (1989),
evaluation of industrial modernization programs (1996), and
the economics of technology policy (2000). His first degree
was in electrical engineering.
·
a review of additional statistical measures to en-
rich the indicators’ formulation;
· how to take into account ‘high-tech’ services as
well as products, since these increasingly cross
national borders;
· suitable patent measures;
·
·
various methodological refinements; and
measures of emerging technologies.
This paper shares our ideas on the last of these. We
want to engage others in considering how best to
measure national capabilities to take potentially
economy-transforming technologies to market.
We seek new component measures to include in
our technological infrastructure (TI) indicator (“the
institutions and resources that contribute to a
nation’s capacity to develop, produce, and market
new technology”). The TI indicator incorporates
measures of scientific and engineering manpower,
electronic data processing purchases, relationship of
R&D to industrial application, and ability to make
effective use of technical know le dge. It lacks meas-
ures of R&D output.
The HTI series is sufficiently consistent from
1990 that we can examine the predictive relation-
ships between the input indicators in 1990 and the
key output indicator, technological standing (TS), in
1999. TI for 1990 correlates 0.81 with TS-1999.
However, if we partition our 33 countries, we find
this apparent strength of relationship is misleading.
TI-1990 correlates 0.94 with TS-1999 for 12 highly
developed countries, but only 0.39 for 14 industrial-
izing countries (Porter et al, 2001). Improving the
predictive validity of TI motivates our examination
of ‘emerging technology’ and patent measures.
This paper examines national R&D publication
rates as a way to measure emerging technology capa-
bilities. As we set forth on this examination, we ask
you to think about which half-dozen countries you
would expectto be most actively researching emerg-
ing technologies. We will develop our answer and
returntothis question inthe “Observations”section.
Xiao-Yin Jin is senior researcher at the Technology Policy
and Assessment Center (TPAC), Georgia Institute of Tech-
nology, and guest professor of the Shanghai Institute for
Science of Sciences, China. He received a BS in chemical
engineering from Tianjin University in 1962, and attended
the one-year program for senior managers on Management
of Technology at MIT in 1984. Before joining TPAC, Mr Jin
served as research professor at Shanghai Institute of Ce-
ramics, Chinese Academy of Sciences, and deputy general
manager at Shanghai Industry Foundation.
Nils C Newman is the president and founder of IISC
(www.iisco.com). He founded IISC in 1995 on the belief that
the Information Revolution brings not only more information
but also new and better ways to process it. Mr. Newman’s
research interests focus on the development of analytical
tools for management of technology. He is also interested in
the assessment of development trends and the study of the
innovation process. Formerly with SRI International, he has
also worked as a research analyst at the Technology Policy
and Assessment Center of Georgia Tech. Mr Newman has
a Bachelor of Mechanical Engineering and an MS in Tech-
nology and Science Policy from Georgia Tech.
‘
high tech indicators’ (HTI) of national competitive-
ness. These are produced every three years to inform
the US National Science Foundation’s Science and
Engineering Indicators. Elsewhere, we discuss HTI
conceptual bases (Roessner et al, 1992), country
contrasts (Porter et al, 1996), and comparisons with
other national indicators of innovation and of com-
petitiveness (Roessner et al, 2002; see also Porter
and Stern, 1999). The 1999 HTI are available at our
website [http://tpac.gatech.edu], along with the de -
veloping 2002 HTI.
HTI now address 33 industrialized and indus-
tria lizing nations. Our four input and three output
indicators combine statistical and expert opinion
measures. The input indicators are national orien-
tation (to pursue technology-based economic
Challenges
We focus on R&D activity measures to tap emerging
technology readiness. Obviously, the presence of
research activity in frontier areas is not a sufficient
1
90
Science and Public Policy June 2002

Measuring national ‘emerging technology’ capabilities
condition to assure commercial innovation. How-
ever, the premise is that countries with such activity
should be better poised to proceed into emerging
technology product, process, and service commer-
cialization. R&D bibliometrics (counts of literary
activity) appear the most suitable candidate meas-
ures, with certain considerations:
Using publication counts we can:
tabulate national emerging technology
measures on a timely basis from
available data; measures can adapt to
changing emerging technologies over
time; and we can elicit topical
·
·
·
we want a broad, relatively simple measure;
the measure should remain available over time;
citations are really only accessible for the Science
Citation Index (SCI) and this does not appear
comparisons and overall composites
‘
technological’ enough for our purposes;
·
·
research expenditures (current) across our 33
nations would be to ugh to obtain for emerging
technologies;
patents, while of great interest, seem less apt to
anticipate path-breaking emerging technologies
out to a 15-year horizon (but possible patent
measures could also be very interesting).
seek something more specific than general R&D.
Which technologies are considered to have greatest
economic transformation potential? RAND Corpora-
tion analyses (Popper et al, 1998) offer an attractive
starting point, most notably from their survey of
corporate leaders. Their issues of most concern ad-
dress “how technology enhances functionality and
fits into a larger business process.” That said, their
priority current (as of 1998) emerging technologies
include:
Our current investigation suggests intriguing trade-
offs among candidate national patent measures:
1
. patent applications anywhere (home or abroad);
. foreign patent applications;
. patent applications in the country by nationals;
. patent applications in the country by foreigners.
2
3
4
· Software
· Microelectronic and telecommunications
technologies
·
Advanced manufacturing technologies
The advantage of patent applications over patents
issued is the considerably reduced lag time. Meas-
ures 2 and 3 offer attractive complementarities be-
tween a nation’s indigenous capabilities (3) and its
attractiveness as a market (4); disparities between
the two can be astounding (Canada: 4,192 patents
filed in 1997 by nationals vs 50,254 by foreigners —
World Intellectual Property Organisation (WIPO),
Industrial Property Statistics CD). Measure 2 is es-
pecially attractive as an indicator of indigenous
capabilities with external technological commer-
cialization intent. We briefly compare patent with
publication measures for our countries later.
Given these considerations, we focus on publica-
tion counts for national emerging technology meas-
ures. Publication counts offer several advantages: we
can tabulate these measures on a timely basis from
available electronic R&D abstract databases, for all
of our 33 countries; measures can adapt to changing
emerging technologies over time; and we can elicit
topical comparisons of interest in addition to overall
composites.
On the other hand, some disadvantages lurk in
counting R&D publications: we have to compile
these ourselves; determining categories of emerging
technologies requires judgment; categorizing publi-
cations into those categories is not unambiguous;
compilation of journal articles and conference pa -
pers in databases lags conduct of the research on the
order of a couple years; and the databases favor Eng-
lish language publication.
· Materials
· Sensor and imaging technologies.
The RAND group also addresses “Over the horizon:
technologies in evolution and revolution.” Major
emerging technology classes are:
· Software
· Computer hardware (data storage, displays)
· Manufacturing equipment used to make computer
components (lithography)
· Communications technologies
· Biotechnology (relating to medicine, agriculture,
the environment, communications)
· Materials (making old materials new ways; envi-
ronmentally friendly materials)
· Energy (considerably fewer mentions).
We start with the first six of these RAND “over the
horizon” areas, albeit recognizing the desirability of
an ongoing classification scheme (there is no assur-
ance that this RAND Science and Technology Policy
Institute activity will be regularly redone). We
choose not to include energy because it received
considerably fewer mentions from the business lead-
ers responding to the RAND inquiries.
Data resources for publication tabulations must be
determined. We favor use of established databases
as filtered and focused collections. Excellent R&D
databases are available. We propose to use two that
together cover ‘technology’ R&D publication very
2
3
Emerging technologies need to be specified. We
effectively — INSPEC and EI Compendex.
Science and Public Policy June 2002
191

Measuring national ‘emerging technology’ capabilities
Depending on th e elaboration of science-based and
biomedical technologies, we would consider exten-
sion to MEDLINE and SCI in the future.
exploration. These initial examinations used partial
thesauri for the INSPEC and EI Compendex class
codes, so they are not exhaustive; we will re-
examine before determination of final measures for
use in HTI for 2002. However, as will become clear,
this does not seem to matter much.
Given general targeting to the RAND categories,
we need to determine how to identify emerging
technology records in the two databases. We consid-
ered searching on explicit terms in keywords (sub-
ject index terms) and/or titles, but decided this was
too detailed and problematic. There are so many
specific communications technologies, biotechnol-
ogy tools and applications, advanced materials, and
so on. Instead, we use the database class codes to get
national counts on publications in approximate
target areas.
Time frame presents another set of choices. For
what time periods should we tally publication activ-
ity? We favor recent activity, but with reasonably
robust measures. Examination of counts, by country,
over time, confirms that annual tallies work satisfac-
torily for general emerging technology categories.
We further focus on ‘h ot’ R&D areas — those
that show both strong recent, and increasing,
research interest. The rationale is that technology,
especially emerging technology, is increasingly sci-
ence-driven (Lane and Makri, 2000). Hence, active
research and development efforts appear highly sali-
ent to developing emerging technology capabilities.
In these initial empirical analyses, we operational-
ize this by computing two measures. First, we in -
clude only technology class codes for which some
Table 1 presents the results. For this analysis,
‘manufacturing equipment used to make computer
components’ was combined with ‘advanced materi-
als’ to approximate ‘advanced materials pertaining
to computer/communications manufacturing.’ This
has some intuitive appeal, but, more critically, we
did not identify manufacturing technologies meeting
our ‘recent and growing’ R&D criteria. This reduces
our target emerging technologies from six to five.
The top few rows show the overall database
scope. These two databases capture a significant
portion of the world’s open literature (journal papers
and conference papers) concerning technology
R&D. Each year the two databases together capture
4
about 500,000 new contributions.
Table 1 conveys the sense that the dual criteria
work — a sizable percentage of the R&D on a tech-
nology having been published in the most recent
year (the current 10% value would need to be re-
examined as a database grows) and strong growth
(double the number of articles of three years back).
For present analyses, we override these in some
instances. Most notably, we include software, even
though the growth criterion is violated badly,
because this seems a vital emerging technology do-
main. Future refinement will be needed to handle
such anomalies. In general, the dual criteria are he lp -
ful in screening out mature from emerging elements
within these technology categories.
Examination of Table 1 yields interesting
observations. All the communications technologies
identified as ‘emerging’ (those flagged with a ‘y’ in
the last column) are optics-related. The
biotechnology set is appealing; it requires
elimination of certain EI Compendex classes that
appear different in nature — general health and
medical, and human engineering — even though
they meet the dual criteria.
1
0%, or more, of the total articles occur in the most
recent full year (of those published in journals or
presented at conferences since 1969 for INSPEC and
since 1970 for EI Compendex). Second, we calculate
the ratio of publications in a technology category in
the most recent full year (1999) to those three years
earlier (1996).
Scanning our emerging technology categories, we
find a ratio of at least two to be an effective screen.
We recognize that changes in class code terminology
can affect this determination, but such changes
should align roughly with technologies being per-
ceived as emerging and important.
We also experimented with limiting analyses to
class codes containing at least 10,000 records. In -
stead, we found that we could explore sub-classes of
the INSPEC and EI Compendex codes to select ‘hot’
ones, then recombine these to constitute reasonably
robust measures.
Our resulting emerging technologies, excepting
software, derive exclusively from EI Compendex.
We will revisit why INSPEC categories seem not to
meet our criteria, since INSPEC generally tends to
be more research oriented than EI Compendex
(
which exhibits more applied research and engineer-
ing development). This may reflect different rates of
classification code revision by the two databases.
If, in the future, we include both relevant INSPEC
and EI Compendex codes for certain emerging tech-
nologies, there could be significant duplication. We
often see these two databases giving 20% or so over-
lap on certain topics. We would expect duplication
rates to be generally comparable for countries, so
this should not bias either national or temporal com-
parisons. We need to beware that counts could over-
state activity where they combine results from both
Initial empirical results
We examined the INSPEC and EI Compendex class
codes to identify those relating to the first six ‘over
the horizon’ RAND categories noted. We then tal-
lied hits (records) for each category to see which met
the ‘hot’ technology tests mentioned — our thresh-
old levels for recent (10% in the most recent year)
and increasing (two times the publications of three
years ago) evolved through this empirical
1
92
Science and Public Policy June 2002

Measuring national ‘emerging technology’ capabilities
Table 1. Exploration of Emerging Technologies in INSPEC and EI Compendex
Over-the-horizon
technologies
Tallies on or about 20 Jan 2001
INSP
codes codes
ENGI All years
1999
1996
Percent
1999/total 99 to 96
Ratio Use?
total
ENGI database (1970–2000)
INSPEC database (1969–2000)
5,300,000 219128 239899
6,700,000 328994 322153
Software
software
info science & documentation
c61$
c72$
440974
80982
36414
7941
35246
4468
8.26
9.81
1.03
1.78
y
Computer hardware
data comm equip & techniques
comp peripheral equip
analog & digital computers & sys
printed circuit, thin & thick films, hy-
brid ICs
c56$
c5540
c5470
b22$
106027
15008
19659
52265
7308
88
1598
2398
6665
387
1382
6.89
0.59
8.13
4.59
1.1
0.23
1.16
semiconduct. devices & materials
telecommunications
electronic components & tubes
b25$
b62$
714
289028
290421
180000
117635
103034
19705
18363
1981
23233
20276
19565
17007
460
8141
7319
6.82
6.32
1.10
19.75
19.68
1.01
1.08
4.31
2.85
2.77
714.$
semiconductor devices & ICs
714.2
y
Comm technologies
waveguides
714.3
717
717.1
717.2
741
741.1
741.1.1
741.1.2
741.3
743
11095
40089
10796
4214
231873
109186
9082
15016
77733
8064
electro-optical communications
optical communications
optical equipment
light, optics & optical devices
light optics
non-linear optics
fiber optics
1692
1940
855
4401
22848
1788
3335
15770
377
589
707
397
1414
6586
424
1048
4006
4.22
17.97
20.29
1.90
20.93
19.69
22.21
20.29
4.68
2.87
2.74
2.15
3.11
3.47
4.22
3.18
3.94
y
y
y
y
y
y
optical devices & systems
holography
holographic applications, etc
semiconductor lasers
743.$
744.4.1
2642
11546
389
1966
14.72
17.03
850
2.31
y
Advanced materials for rare metals — sum of
543.$
549.3
712
712.1
712.2
<10,000
23982
99214
28064
1183
comp/comm tech
silicon, tellurium & zirconium
electronic & thermionic materials
semiconductor materials
thermionic materials
4835
26
6110
35
1378
18
1882
20.16
0.03
21.77
2.96
3.51
1.44
3.25
y
y
Biotech
bio materials, engineering
bioengineering
biomedical engineering
biological materials
biomechanics
human engineering
human rehabilitation engineering
medicine
health care
biotechnology
biology
biological equipment
biomedical equipment (general)
Biomaterials
461.$
461
86070
107605
26125
31100
7271
10609
3069
17109
194
5072
5904
1647
1466
732
4276
1519
1875
4350
5
6409
19.88
0.18
2.67
461.1
461.2
461.3
461.4
461.5
461.6
461.7
461.8
461.9
462
2120
2620
545
544
337
2217
617
778
850
7
19.40
19.00
22.70
13.80
23.90
21.30
21.70
18.60
26.90
0.00
2.39
2.25
3.02
2.69
2.17
1.93
2.46
2.41
5.12
0.71
1.77
1.28
y
y
y
20109
6994
10069
16154
38448
5745
y
y
462.1
462.5
886
768
501
598
15.40
13.90
5537
Note:
ENGI = EI Compendex
databases. (This is not a problem in the present tabu-
lations since each component is measured in only
one or the other database.)
contrasting stronger familiarity with English. Brazil
poses strong language concerns — the possibility
that Portuguese would be less likely to be indexed in
INSPEC and EI Compendex. Thailand is of interest
because the counts are so low.
Database biases
We listed the leading institutions and publication
outlets for each of the four countries. We then asked
knowledgeable persons from these countries: 1) how
much significant publication from their country
would they estimate occurs in journals or confer-
ences not included? and 2) do they know of impor-
tant R&D organizations likely to publish elsewhere
that we seem to be missing? Table 2 tabulates the
sense of the responses.
We wondered how badly database biases (especially
toward English) might distort country comparisons?
To address this issue we ran a four-country tally on
two emerging technologies — software and ad-
vanced computing/communication materials. We
prepared tables presenting the results for 1998, 1999,
and 2000 for India, China and Hong Kong, Thailand,
and Brazil. These four nations touch a range of pos-
sible concerns. China poses language and inclusion
concerns (Hong Kong separately searched and in -
cluded; Taiwan not considered here). India provides
We interpret these reviews as reassuring. Cer-
tainly we shall miss some important R&D because
of database coverage emphases, but it does not
Science and Public Policy June 2002
193

Measuring national ‘emerging technology’ capabilities
Table 2. Observations on national coverage
Country
How much publication appears to be missed (based on journals and conferences included)?
India
fair amount — but respondent points to theses (this is not a real national bias)
not very much — points to 1 international and 1 Indian technical series
likely that most are covered; all significant publications would be in English
only a little; all significant research in India published in English
China
20–40% seems not to be collected by INSPEC and EI Compendex
don’t know extent of loss of Chinese language publications; many researchers do publish in international journals
a little
Thailand
very little
very little
little
a lot (but we inferred considerable institutional loyalty coloring this person’s response)
Brazil
20–25% missed
Very little
several locally important contributions, but all those most important globally are included in these periodicals
Are we missing the work of significant R&D organizations?
India
technical reports from government, semi-government agencies, corporations (this is not a national bias)
no
not applicable
surely abstracted in INSPEC and EI Compendex
China
most important R&D organizations like to publish in places abstracted, especially into EI Compendex
maybe military R&D institutes don’t publish some
not applicable
not applicable
Thailand
Brazil
not applicable
all important R&D organizations seem to be included
notes three Thai annual journals and proceedings
lists ten organizations apt to publish in Portuguese, less likely to be abs tracted in INSPEC and EI Compendex
lists three organizations not appearing in our list
some good Portuguese journals probably not abstracted
Note:
Each line summarizes an observation of one reviewer from that country upon reviewing our tallies of emerging technology R&D
publications for these countries
appear severe. Furthermore, coverage should be re la -
tively comparable for a given country over time so
that within-country comparisons should be good.
Concerns about ‘classes’ of important R&D institu-
tions not represented well do not seem qualitatively
different than would be the case in countries such as
the USA.
An intriguing sidenote to this exploration was our
surprise at the relative software-related publication
from China and India. We would have hypothesized
that India would dominate because of its English
language usage and its extensive software develop-
ment activity. Not so; overall in INSPEC, we
identified 12,766 software-related papers from
China (and Hong Kong) versus 2,713 for India. The
disparity appeared even greater most recently as we
found 1,920 Chinese software publications in the
year 2000 versus 133 Indian.
So, at least in this instance, international literature
contributions do not seem heavily determined by
national usage of the English language. Researchers
in many countries seem to be encouraged to report
their work in international journals or conferences,
and those tend to favor English, but we have not sys-
tematically assessed the extent of this.
Emerging technology R&D activity by country
A mundane, but difficult, issue concerns how best to
identify country of authors. Both INSPEC and EI
Compendex only provide the institutional affiliation
and country of the first author. In some (few) cases
country is not indicated. These do not present a ma-
jor ‘indicators’ problem for us, but users of these
measures should recognize there is some loss of in -
formation — we won’t tally ‘all’ a country’s R&D.
Certain countries provide particular challenges.
The UK, for instance, may also be indicated as Eng-
land, Scotland, Northern Ireland, or Wales. How-
ever, just finding the string “Wales” in the affiliation
We were surprised at the relative
software-related publication from
China and India: at least in this
instance, international literature
contributions do not seem heavily
determined by national usage of the
English language
1
94
Science and Public Policy June 2002

Measuring national ‘emerging technology’ capabilities
field does not guarantee the UK — it may reflect
New South Wales, Australia. This requires devel-
opment of country thesauri for INSPEC and for EI
Compendex to capture most national records while
minimizing noise. We have developed initial such
thesauri. Changing country designations, such as
USSR, East Germany, could cloud historical track-
ing. In the case of Hong Kong, we extract its records
separately, then combine with China for most
purposes.
Results are interesting. Table 3 shows the counts
for our 33 countries. Countries are grouped as in
HTI to facilitate comparisons among industrialized
and industrializing nations and among regions as
follows:
· Asian Tigers (Singapore, South Korea, and
Taiwan)
· Asian Cubs (Malaysia, China, Thailand, Indone-
sia, Philippines, and India)
· Latin America (Mexico, Brazil, Argentina, and
Venezuela).
The first column shows each country’s total number
of articles and conference papers abstracted by EI
Compendex for 1999. The following five columns
break out each of our five emerging technologies.
The last two columns sum these, with or without
software included (since ‘software’ does not meet
our dual criteria of recent and growing R&D
activity).
Figure 1 depicts the results for the sum of the five
emerging technologies (next to last column of Table
3), leaving out the ‘research superpowers,’ USA and
Japan, to improve scaling. The most striking obser-
vation concerns China’s strong presence — it is one
of the research powerhouses for these emerging
technology areas, along with Germany, UK, and
France. Figure 1 suggests we could identify tiers in
research activity on emerging technologies:
·
·
·
·
The ‘Big Three’ — United States, Japan, and
Germany
Western Europe (UK, France, Netherlands, Italy,
Switzerland, Sweden, Spain, and Ireland)
English Heritage Nations + Israel (Canada, Aus-
tralia, South Africa, New Zealand, and Israel)
Eastern Europe (Russia, Poland, Hungary, and
Czech Republic)
Table 3. National ‘emerging technology’ R&D publication activity for 1999
Counts
ENGI
total
Optical
comm
(ENGI)
Comp
hdwr
(ENGI)
Semi
mtls
(ENGI)
Biotech
Software
Sum
Sum
‘4 ETs’
(ENGI)
(
no)
(ENGI)
(INSPEC)
(mix)
USA
Japan
Germany
UK
France
Netherlands
Italy
Switzerland
Sweden
Spain
57479
22686
11616
11349
8397
2886
5652
1959
2798
3553
222
8790
4527
2375
1720
1588
483
990
403
360
599
48
6591
2808
1216
839
817
261
550
268
269
261
37
2224
1870
852
489
548
120
304
108
151
206
10
4637
1042
634
710
411
261
300
130
233
177
21
9627
2820
2520
2681
1436
588
1122
393
397
616
96
31869
13067
7597
6439
4800
1713
3266
1302
1410
1859
212
22242
10247
5077
3758
3364
1125
2144
909
1013
1243
116
Ireland
Canada
Australia
South Africa
New Zealand
Russia
6469
3594
666
716
521
49
417
165
39
165
113
31
484
222
35
54
305
77
55
49
68
142
191
6
366
13
6
2
207
76
109
24
8
125
982
897
80
125
264
227
88
2764
1918
234
1782
1021
154
123
2382
810
209
278
738
2073
1596
46
3965
44
23
7
1327
583
656
147
52
560
50
8
11
248
5182
2115
665
1421
437
84
152
311
918
576
26
341
160
37
315
136
33
2646
1037
297
Poland
Hungary
Czech Repub
Singapore
South Korea
Taiwan
Malaysia
China
Thailand
Indonesia
Philippines
India
Mexico
Brazil
Argentina
Venezuela
Israel
671
42
35
93
371
1873
4975
4608
211
13890
217
62
35
4462
1257
2169
521
249
665
610
13
757
8
4
0
266
120
126
12
110
348
219
1
550
3
5
0
279
78
109
32
307
748
598
35
1656
50
2
3
229
95
339
41
1045
2821
2194
81
5621
94
25
10
1556
678
995
2292
20
8
5
575
309
312
79
188
71
791
196
1564
26
270
6
94
12
64
19
238
553
Notes: Detailed class codes consolidated into the counts shown appear in Table 1
Optical comm’ here corresponds to ‘Comm technologies’ in Table 1
Comp hdwr’ is computer hardware
Semi mtls is advanced materials for computing/communication technologies
Sum’ adds the five separate emerging technologies
Sum – 4 ETs’ excludes software
‘
‘
‘
‘
‘
Science and Public Policy June 2002
195

Measuring national ‘emerging technology’ capabilities
Not
USA
Japan – 13,067
shown:
– 31,869
Number of articles published in 1999 on
selected ‘emerging technologies’
by authors of these nationalities
8000
7000
6000
5000
4000
3000
2000
1000
0
Figure 1. Emerging technology publication activity by country
·
·
Superpowers: USA and Japan
research powerhouses: Germany, UK, China, and
France
the gap from Malaysia to the others becomes
pronounced.
·
strong players (those with over 2,000 annual pub-
lications): Italy, South Korea, Canada, Russia,
Taiwan
Normalizing the data
In constructing an indicator, we face choices as to
how best to normalize the data. Note the tremendous
range for our 33 countries in the sum of their annual
emerging technology R&D publication act iv ity —
from 10 papers for the Philippines to almost 32,000
for the USA — over three orders of magnitude. We
might consider per capita (or per scientist and engi-
neer) metrics, but HTI prefers total national activity
measures as more salient to export competitiveness.
Should readers wish to explore other re la tionships in
these data, Table 3 provides raw counts of items
·
solid presence: 11 countries with 670–2000
annual publications — Australia, Spain, the Neth-
erlands, India, Sweden, Switzerland, Singapore,
Poland, Brazil, Israel, Mexico
laggards (those with about 200–400 annual pub-
lications): the Czech Republic, Hungary, New
Zealand, South Africa, Ireland, Argentina
those lacking critical research mass (<100
publications): Thailand, Malaysia, Venezuela, In-
donesia, and the Philippines.
·
·
(
nearly all journal articles and technical conference
Keeping in mind that the 33 countries profiled
are chosen for their high-tech proficiency or prom-
ise, these results raise concerns about the ability of
those without a critical emerging technology re-
search enterprise to participate early and strongly in
attendant technology-based competition in the fu-
ture. (Note also that the 33 countries selectively
sample the heavily industrialized and industrializing
nations; they do not cover all such countries.) Possi-
bly most notable in this apparent research weakness
is Malaysia, which scores strongest among the Asian
Cubs on our High Tech Indicators, close on the heals
of the Tigers — Singapore, South Korea, and
Taiwan. However, were we to place stock in a
measure of emerging technology R&D activity,
papers) indexed by the database (either EI Com-
pendex or INSPEC).
HTI reports indicators as ‘S-scores’ — scaling the
33 countries on a relative basis with the leader on a
particular variable as ‘100.’ This is to make it easier
to quickly compare countries’ relative performance
(the early HTI indicators reported values as Z-scores
— how many standard deviations a country was
from the mean — that proved awkward for many
people to assimilate easily). In recent technological
indicators development, UN Comm is sion on Science
and Technology for Development (UNCSTD) and
UNIDO use a similar scaling approach.
For HTI expert opinion response variables scaled
from 1–5, for instance, a perfect score of all 5s
1
96
Science and Public Policy June 2002

Measuring national ‘emerging technology’ capabilities
would transform to an S-score of 100; whereas a
worst score of all 1s would transform to 0. However,
all the present data are numerical. So the highest
country score becomes 100 and the minimum is
taken as 0. In other words, S-scores on the present
data are the country’s fraction of the highest country
value, multiplied by 100 to scale from 0–100, that is,
S-score is the percentage of the highest country.
For instance, look at the USA and Japan emerging
technology ‘Sum’ (next to last column of Table 3).
Japan’s 13067 divided by the USA’s 31869 = 0.41
or 41%. Note Japan’s S-score for the sum is 45.99
research publication — ranks and logarithms. These
certainly reduce the skewness, but reinforce the
sense of high correlation across the five emerging
technology areas. For ranks, correlation of each
emerging technology with each other for 1999 pub-
lication, across countries, ranges from 0.90 (for
software with semiconductor materials) to 0.99 (op-
tical communication with semiconductor materials),
with a mean of 0.95. For logarithms, the mean corre-
lation between pairs of the five emerging technology
publication counts, across countries, is again 0.95,
with a range of 0.89 (software with semiconductor
materials) to 0.98 (optical communication with
computer hardware). Rank and log data for the 33
countries show strong correspondence. Raw counts
show a similar meancorrelation of 0.95, ranging from
(
last column of Table 4). That is because this is the
average of Japan’s S-score (percentage of the lead-
ing country) on each of the five emerging techno lo -
gies that contribute to the sum. In each case the USA
is the leader, so we simply compute Japan/USA and
average: (51.5 + 42.6 + 84.1 + 22.5 + 29.3)/5 = 46
5
0.90 (softwarewith biotech) to 0.98 (three pairs).
Moreover, these emerging technology emphases
are strikingly similar to overall activity level in the
EI Compendex database (Table 1). Correlations of
raw counts for EI Compendex with each of the five
emerging technologies average 0.97 (range of 0.93
to 0.99). Correlations of ranks with each of the
five also average 0.97 (range of 0.95 to 0.99). In
terms of indicator development, this lack of dis-
crimination among the emerging technologies and
with overall engineering R&D publication is some-
what discouraging.
(
corresponding to 45.99). Note here that Japanese
and American emphases are not identical across
these five emerging technologies.
S-scores for the sum of emerging technology
publications in 1999 reflect the huge disparity in
research activity levels. For instance, pegging the
USA as 100, the Philippines score 0.03 (last column
of Table 4); its contribution to the open research lit-
erature on these five technologies is minimal.
We consider two bolder alternatives to explore
Table 4. National ‘emerging technology’ R&D publication ranks and average S-score for 1999
Ranks
ENGI
Optical
comm
Comp
hdwr
(ENGI)
Semi
mtls
(ENGI)
Biotech
Software
Average
S-score
(5 ETs)
(ENGI)
(ENGI)
(INSPEC)
USA
Japan
Germany
UK
France
Netherlands
Italy
Switzerland
Sweden
Spain
1
2
4
5
6
1
2
3
5
6
15
8
17
18
11
28
10
14
27
26
7
16
24
23
20
9
1
2
3
4
5
15
9
13
12
15
25
10
18
24
29
11
19
25
23
17
7
1
2
3
6
5
1
2
4
3
6
1
2
4
3
6
13
7
15
14
11
23
8
100.00
45.99
24.73
19.49
15.78
5.32
10.28
4.08
4.82
6.05
0.60
8.51
5.52
0.83
0.73
8.96
3.51
1.02
1.19
3.38
9.40
7.20
0.21
17.48
0.26
0.11
0.03
5.99
2.29
3.25
0.69
0.26
2.51
15
8
19
16
14
28
7
13
24
26
9
18
25
23
20
10
11
30
3
29
32
33
12
22
17
27
31
21
16
10
17
15
13
27
11
19
26
28
7
14
24
23
20
9
12
30
4
31
32
33
8
10
9
17
11
15
28
5
12
26
24
8
20
23
25
22
16
14
31
7
29
31
33
13
21
19
27
30
18
Ireland
Canada
Australia
South Africa
New Zealand
Russia
9
27
22
18
21
26
25
17
10
12
30
5
28
33
32
20
24
16
29
31
19
Poland
Hungary
Czech Repub
Singapore
South Korea
Taiwan
Malaysia
China
Thailand
Indonesia
Philippines
India
Mexico
Brazil
Argentina
Venezuela
Israel
12
29
4
8
27
6
31
32
33
13
21
19
25
29
22
29
32
33
14
21
20
28
31
22
22
18
25
29
21
Science and Public Policy June 2002
197

Measuring national ‘emerging technology’ capabilities
Patent measures
· Russia ranges from 7th (optical communication)
to 11th on four of the five, but lags at 18th on one
(software).
· India surprisingly shows strongest at 8th (semi-
conductor materials) to weakest at 20th
(software).
· South Korea peaks at 7th (computer hardware),
ranging down to 16th (biotech).
· Taiwan shows as a steady 12th to 14th on four of
the five, showing notably higher on computer
hardware (8th).
As mentioned, we are also developing patent meas-
ures, and these can differ dramatically with each
other. We compared non-resident patent applications
for each nation as a whole (not by technology cat-
egories) for 1997 with our measures. Correlations of
the patent S-scores with our emerging technology
S-scores ranged from 0.42 for biotech to 0.51 for
software — much lower than the publication inter-
correlations just noted.
Patent S-scores correlated more highly with R&D
publication ranks (ranging from 0.68 with optical
communications to 0.76 for software). Ranking
moderates the extremes of the publication statistics.
In fact, we would not expect these measures to cor-
relate highly — this patent measure reflects market
attractiveness, while our publication measures get at
indigenous generation of emerging technology
developments.
On the other hand, patent applications by nationals
of one country anywhere (home or abroad) better
reflects indigenous development activity. This
measure correlates very highly with our five emerg-
ing technology publication rates — from 0.84 for
semiconductor materials to 0.98 for biotech. This
broadly distributed measure (ranging from 576 Ar-
gentinean patents in 1997 to 1.5 million American
ones) mirrors the extreme distribution of R&D pub-
lications across countries. (This patent measure cor-
relates much less with the toned down publication
rank measures, ranging from 0.50 to 0.54.)
· Canada seems surprisingly strong at 5th on b io -
tech and 8th on software, ahead of its placement
on the other three (10th or 11th).
· Australia shows relatively high variability, with
9th on software and 12th on biotech, but only 18th
and 19th, respectively, on computer hardware and
semiconductor materials.
Observations
Emerging technologies do not stay constant. One
advantage of the proposed approach is that the set
of emerging technologies would be continually
adapted to seek ‘frontier’ technology R&D. The dual
criteria of strong recent emphasis and growth in ac-
tivity provide good bases for this adaptation. In this
initial exploration, we augmented these with judg-
ment based on categorical intent (for instance, ex-
cluding human engineering from biotechnology, and
including software despite failure to meet the dual
criteria).
Overlap
The level of specificity for emerging technologies
could be set broader or finer (consider Table 1).
We think the current algorithm is at about the right
level for national comparisons as an indicator of
‘technological infrastructure’ that enables countries
to partake in the commercialization of potentially
economically potent new technologies. In prac-
tical terms, each additional class code adds about 50
searches (33 countries plus complicated country
designators, for instance, UK includes England, and
so on). So, for HTI we cannot pursue too much
detail.
A reviewer of this article wondered how much over-
lap there was among the articles in our emerging
technology categories. For the four categories drawn
from EI Compendex, we can provide that informa-
tion. The biotech category overlaps only about 1%
with the computer hardware and semiconductor
materials categories, but 14% with optical commu-
nications. Those three categories — computer hard-
ware, semiconductor materials, and optical
communications — overlap heavily with each other,
ranging from 23% to 32% common records. Those
are certainly not mutually exclusive, so it should not
be very surprising that there is high correlation given
this commonality.
Once search algorithms are finalized, we should
Emerging technologies do not stay
constant: in this approach the set of
emerging technologies is continually
adapted to seek ‘frontier’ technology
R&D; dual criteria of strong recent
emphasis and growth in activity are
good bases for this adaptation
National emphases
For rank data, the overall R&D publication pat-
terns reaffirm the similarity in national emphases
across the five emerging technology categories.
Table 4 shows that the USA ranks #1 for all; Japan,
#
2 for all (as well as for overall EI Compendex
publication in 1999). In general, R&D activity levels
across these five emerging technology categories are
very similar. Some interesting variability does
surface:
1
98
Science and Public Policy June 2002

Measuring national ‘emerging technology’ capabilities
be able to write a macro (script the steps) to perform
a large set of searches and the subsequent analyses
quickly and reproducibly. This could be used to
generate periodic updates (such as every year) to
alert to pronounced national initiatives in particular
emerging technologies. Differential activity meas-
ures (showing extent of change over time for each
nation) might prove indicative of shifting R&D
emphases.
While we therefore plan to pursue this emerging
technology measure as indicated, we hope that others
willbe stimulatedtoexplorealternatives.Inparticular,
finercategorizations (for instance,agriculturalbiotech
vs medical biotech) could provide usefulR&D policy
metrics. One might also want to pursue issues suchas
would become a component of our Technological
Infrastructure ‘High Tech’ Indicator for 2002. In
future years (for instance, 2005), the specific techno-
logical classes would be revised to reflect most
recent publication emphases.
The current tabulation incorporates some 16 spe-
cific technological classes of research publication
from EI Compendex and INSPEC relating to five
emerging technologies. Our 33 nations account for
80–90% of the research publications in the four
1999 EI Compendex samples. This is surprisingly
high, considering we do not include all the OECD
(Organization for Economic Cooperation and De-
velopment) countries.
For each of these four emerging technologies (not
including software), 1999 accounts for 20–21% of
the total records from 1970–2000. The ratio of hits
in 1999 to those in 1996 is 2.7 for biotech, 3.46 for
optical communications technology, 3.36 for semi-
conductor materials, and 2.77 for computer hard-
ware. In contrast, the software category shows a
ratio of 1.03. So, the amount of publication in these
‘over the horizon’ technology categories has in-
creased about three-fold from 1996 to 1999. That is
hot. The steady research level in software could war-
rant further examination of changes in topical em-
phases and approaches within this category. (While
class codes and total records abstracted change over
time, the changes over this time period were quite
moderate.)
Yet to what degree does lack of research publica-
tion activity portend lack of high-tech economic
competitiveness for nations? How does it fit in
national and global sy stems of innovation (Archi-
bugi et al, 1999)? These are open questions. To
illustrate the contrasts more specifically, consider
the biotecharea. For 1999, our tally for the class codes
“
what percentage of a country’s engineering re-
search is in these emerging technologies?” Figure 2
shows the profile for the 1999 emerging technology
publications (next to last column of Table 3) as a
percentage of the country’s overall engineering re-
search (first column of Table 3).
Given the categorization issues involved, it is
necessary to be cautious in interpretation. Further-
more, the low numbers of publications of countries
such as Ireland preclude making too much of this
measure. Nonetheless, it is interesting to see which
countries show greater relative emphasis on these
emerging technologies, and which show less.
We plan to average relative national standing for
these five emerging technologies (as shown in the
last column of Table 4) to report this as a measure of
national emerging technology capability. We con-
sidered the possibility of differential weighting for
the five component categories; however, the overall
similarity among the five for these countries obvi-
ated the need for special weighting. This measure of
research activity in five emerging technologies
1
00%
9
8
7
6
5
4
3
2
1
0 %
0 %
0 %
0 %
0 %
0 %
0 %
0 %
0 %
0
%
U K
USA
ITALY
JAPAN
SPAIN
RUSSI AI SRAEL
SWEDEN
INDIA
MEXICO
AUSTRALIA
POLANT DA IWANBRAZIL
HUNGARY THAILAND
CANADA
CHINA
IRELAND
FRANCE
MALAYSIA
GERMANY
SINGAPORE
SOUTH KOREA
CZECH REPUB
INDONESIAVENEZ AU RE GL AE NTINA
PHILIPPINES
SWITZERLAND
NETHERLANDS
NEW ZEALAND
SOUTH AFRICA
Figure 2. Emerging technology emphasis by country
Science and Public Policy June 2002
199

Measuring national ‘emerging technology’ capabilities
comprising this domain was 13,974 (EI Compendex
abstract records). Of those, our 33 countries account
for 80%, led by the USA (33% itself — 4637
publications). In contrast, four of our countries had
fewer than ten biotech publications each, using
this coding. That would seem to severely constrain
their potential to commercialize this emerging
technology.
The implications of this measure merit explora-
tion. Our overall ‘high tech indicators’ (HTI) point
toward a dramatic broadening of high-tech competi-
tiveness across these 33 nations. This ‘emerging
technology’ measure points to a markedly different
future in which relatively few of these nations domi-
nate technological competitiveness over a 15-year or
so horizon. In terms of relative (S) scores, six
nations score 15 or higher; 27 score under 10, and of
those, a dozen score about 1 or under (publishing
fewer than 400 papers per year in these areas).
This disparity challenges those who would set
national policy to foster technological competitive-
ness (Clark and Guy, 1998; Kim, 2000). Do the
industrializing nations need to bolster their R&D in
emerging technologies to enable them to compete
economically in these areas in the future?
References
Antonelli, G, and N de Liso (editors) (1997), Economics of
Structural and Technological Change (Routledge, London,
New York).
Archibugi, D, J Howells and J Michie (1999), “Innovation systems
in a global economy”, Technology Analysis and Strategic
Management, 11(4), pages 527–539.
Archibugi, D, and J Michie (editors) (1998), Trade, Growth and
Technical Change (Cambridge University Press, Cambridge).
Clark, J, and K Guy (1998), “Innovation and competitiveness: a
review”, Technology Analysis and Strategic Management,
10(3), pages 363–395.
Godin, B (INRS/CIRST) (1996), “The state of S&T indicators in
the OECD countries”, reference paper submitted to Statistics
Canada, August.
Grupp, H (1995), “Science, high technology and the competitive-
ness of EU countries”, Cambridge Journal of Economics, 19,
pages 209–223.
Hansen, J A (1999), “Tech innovation indicators — a survey of
historical development and current practice”, working paper,
State University of New York College at Fredonia.
Kim, S (2000), “Is government investment in R&D and market
environment needed for indigenous private R&D in less devel-
oped countries?: evidence from Korea”, Science and Public
Policy, 26(1), February, pages 13–22.
Lane, P, and M Makri (2000), “Responding to diminishing techno-
logical opportunities: a socio-cognitive model of science and
innovation”, working paper, Arizona State University College of
Business.
Mani, S (2000), “Exports of high technology products from devel-
oping countries: is it real or a statistical artifact?”, working
paper, UNU/INTECH, The Netherlands, March.
Recalling our earlier question, the leading
countries based on our measures are: USA; Japan;
Germany; UK; China; and France (compare, any of
Table 3 ‘sum,’ Table 4 ‘average S-score,’ Figure 1).
The surprise to us is China. In the past few years
China has moved up dramatically on Georgia Tech’s
HTI. Her strength in emerging technologies suggests
this nation may well power forward into a leadership
role in next generation technologies.
OECD (1999) Project Report: “OECD science, technology and
industry scoreboard, 1999 benc hmarking knowledge-based
economies”.
OECD (2000) Project Report: “A new economy: the changing role
of innovation and information technology in growth”, available
at <http://www.sourceoecd.org/content/html/index.htm>, last
accessed 16 July 2002.
Popper, S W, C S Wagner and E V Larson (1998), “New force at
work: industry views critical technologies”, The US Rand Cor-
porate Publication MR-1008-OSTP, available at <http://www.
rand.org/publications/MR/MR1008/#contents>, last accessed
1
6 July 2002.
Porter, A L, J D Roessner, N Newman and D A Cauffiel (1996),
Indicators of high tech competitiveness of 28 countries”,
“
International Journal of Technology Management, 12(1),
pages 1–32.
Porter, A L, J D Roessner, X-Y Jin and N C Newman (2001),
Notes
“
Changes in national technological competitiveness: 1990–93–
1
.
.
NSF project #9901310, “Indicators of Technology -based
Competitiveness,” J D Roessner, Principal Investigator.
INSPEC: our analyses emphasize abstract records from the
past five years (just over 1.5 million records). INSPEC is
produced by IEE (<http://www.iee.org.uk/publish/inspec/>). It
abstracts articles from over 4000 science and technology
journals, plus about 2000 conference proceedings, and other
technical sources. The database includes physics, electrical
engineering, communications, computing, and ni formation
technology.
96–99”, Technology Analysis and Strategic Management,
13(4), pages 477–496.
2
Porter, M E, and S Stern (1999), “The new challenge to America’s
prosperity: findings from the innovation index”, Council on
Competitiveness, Washington DC, available at <http://www.
compete.org/>, last accessed 16 July 2002.
Roessner, J D, A L Porter and H Xu (1992), “National capacity to
absorb and institutionalize external science and technology”,
Technology Analysis and Strategic Management, 4(2), pages
99–113.
3
.
.
EI Compendex (also called Engineering Index ) — is pro-
duced by Engineering Information (<http://www.ei.org>). It
abstracts articles from about 2600 journals, conference
proceedings, and technical sources — for 1995–1999, over
Roessner, J D, A L Porter, N C Newman and X-Y Jin (2002), “A
comparison of recent assessments of the high-tech competi-
tiveness of nations”, International Journal of Technology Man-
agement, 23(6), pages 536–557.
Sirilli, G (1997), “Science and technology indicators: the state of
the art and prospects for the future”, in Antonelli and de Liso
(1997).
van Raan, A F J (editor) (1998), Handbook of Quantitative Studies
of Science and Technology (North Holland, Amsterdam).
WCY (1998–2000), The World Competitiveness Yearbook 1998–
2000, edited by WCY Research Team (International Institute
For Management Development (IMD), Lausanne, Sw it zerland)
available at <http://www02.imd.ch/wcy>, last accessed 16 July
2002.
1
.1 million records. It covers all engineering disciplines.
As of January, 2001, when these tallies were made, the year
000 activity was not fully indexed. The EI Compendex tally
for the year 2000 was 176,022 (vs about 230,000 for 1998 or
4
2
1
1
999); the INSPEC tally was 230,009 (vs about 325,000 for
998 or 1999). That is why we are using 1999 as our most
recent full year in this analysis.
5.
S-scores give exactly the same correlations as raw scores.
Converting raw values to S-values preserves the relative or-
dering identically.
2
00
Science and Public Policy June 2002