Full text of DOI:10.1109/85.841136

Com p ut er Net w ork-Based Scient ific
Collab orat ion in t he Energ y Research
Com m unit y, 1973–1977: A Mem oir
Gerald Estrin
In 1973, four national laboratory groups and three university energy
research groups started an investigation of whether scientific
collaboration could be improved with computer networks. Four years
later, the seven groups were piggybacked onto the Arpanet, and
more than 60 case histories of applications had been documented.
This article reviews the early collaboration and tells how the energy
research community was introduced to modern computer
networking.
In 1973, som e energy research groups started an
th e Division of Ph ysical Research th rough its
investigation of whether scientific collaboration Math em atics an d Com puter Scien ce Research
could be im proved with com puter n etworks. Program directed by Milton E. Rose (deceased
This Energy Research and Developm ent Agency 22 August 1993). Th e report was com piled by
project lasted from 1973 to 1977 and culm inat- th e ERDA Workin g Group on Com puter
ed in a four-volum e report, General Purpose Networkin g, of wh ich I was Ch air. Th ere were
Computer Networks and Resource Sharing in 12 con tributin g auth ors, an d th is article draws
1
ERDA, issued in 1977, brin gin g togeth er th e liberally from th e report (see Figure 1).
fin din gs of in vestigators at Argon n e Nation al
The investigation was of heterogeneous net-
Laboratory, Brookh aven Nation al Laboratory, works in wh ich a variety of dissim ilar com put-
Lawren ce Berkeley Laboratory, Lawren ce ers an d operatin g system s were in tercon n ected
Liverm ore Laboratory, Massach usetts In stitute n ation wide. Th e in vestigation itself was an
of Technology’s Laboratory for Nuclear Science, exam ple of com putation al resource sh arin g
New York Un iversity’s Couran t In stitute, an d th rough a h eterogen eous n etwork. More th an
UCLA’s Com puter Science Departm ent. (See the 20 in dividuals, represen tin g seven ERDA com -
sidebar on software an d also see th e sidebar on
putin g sites, coordin ated, com piled, an d for-
organ ization s.) Th e project was supported by m atted th e data th at m ade up th e report.
Volum e 1 sum m arized th e en tire four-year
investigation into com putational resource shar-
ing through heterogeneous com puter networks
an d offered recom m en dation s for th e future.
Volum e 2 discussed tech n ical an d fin an cial
issues. Volum e 3 an alyzed th e ben efits an d bar-
riers en coun tered an d provided case h istories.
Volum e 4 provided the basic inform ation need-
ed to access an d use th e ERDA sites th at were
con n ected to th e Arpan et.
Th is story begin s with presen tation of th e
first volum e’s sum m ary and recom m endations:
Th e purpose an d scope of th e study is stated as:
Th e ERDA research com m un ity en com passes
th e n ation al research laboratories, un iversities,
Figure 1. Logo of Working Group on Computer Networking. (From
and private contractors undertaking energy relat-
ed research , developm en t, an d dem on stration
projects. ERDA’s m ission requires th e develop-
General Purpose Computer Networks and Resource Sharing in ERDA,
vol. 1)
4
2
IEEE An n als of t h e Hist ory of Com p ut in g
1058-6180/00/$10.00 © 2000 IEEE

Soft w are
BKY-Sesame: LBL Interactive System Monitor
FTP: file transfer protocol (used to transfer
files between computers)
Org an izat ion s
Ames-ISU: Iowa State University at Ames, Iowa had extensive ener-
gy research support.
ANL: Argonne National Laboratory
Linda: BNL modeling software
BNL: Brookhaven National Laboratory
MACSYMA: MIT symbolic differential equa-
tion solver
CCA: Computer Corporation of America, developer of the
Datacomputer (CCA had received government support in the
early 1970s to develop an experimental machine called the
Datacomputer that would be capable of storing and managing
a very large database. For example, it had the goal of manag-
ing a 10-year history of weather data that would be large
enough to swamp even the largest ERDA sites. This provided a
clear example of unique hardware and software that might be
shared if accessible through the Arpanet. Note that Figure 5
shows CCA as a site on the Arpanet.)
MINPACK: ANL minimization software
MULTICS: MULTiplexed Information and
Computing Service (a time-sharing oper-
ating system developed at MIT that served
with the Xerox Graphic Printer for group
generation, editing, and production of the
four-part report on General Purpose
Computer Networks and Resource Sharing
in ERDA—see sidebar, Network Experience
Report, UCLA-1).
ERDA HQ: The Energy Research and Development Agency
Headquarters staff participated in our networking effort by facil-
itating our work and keeping the Washington office aware of
our progress.
NOS-TELEX: NYU’s CDC 6600 time-sharing
software
Planet: Institute for the Future’s teleconfer-
encing system
RATS: LLL gateway software
ERDA MCS: The Energy Research and Development Agency’s
Mathematical and Computer Sciences office.
LASL: Los Alamos Scientific Laboratory
Runoff: text formatting program
SARA: UCLA System ARchitects Apprentice
design and simulation software
Scope-Intercom BNL: CDC 6600 time-shar-
ing software
LBL: Lawrence Berkeley Laboratory
LLL: Lawrence Livermore Laboratory
LNS: Laboratory for Nuclear Science, Massachusetts Institute of
Technology
MFE: Magnetic Fusion Energy research program.
MFENET: Magnetic Fusion Energy NETwork; this included a CDC
7600 at LLL and would include a gateway and a CDC 6600 and
other equipment.
NYU: Courant Institute, New York University
NASA-Ames: The Illiac IV was scheduled for delivery to NASA-Ames
and was to be another example of a unique hardware resource
that would benefit from widespread access through the
Arpanet.
SETL: NYU set logic software
Spice: LBL electronic chip design software
Telenet: an early commercial network service
TELNET: telecommunications network pro-
tocol
TSO: Time Shared Option (IBM)
WYLBUR: ANL time-sharing software
RHEL: Rutherford High Energy Laboratory, UK
SACNET: Secure Automatic Communications Network (available
in the early 1970s but was not suitable for the kind of open
resource sharing that we felt would provide great benefits to
energy research).
m en t of large scale m odelin g system s, an alysis
program s an d data bases. ERDA m ain tain s an
extrem ely large inventory of com puting facilities
wh ich supply powerful com putation al capabili-
ties for a wide spectrum of en ergy research an d
developm en t activities.
SLAC: Stanford Linear Accelerator Center
U of I: University of Illinois
Sin ce th e ERDA com m un ity is geograph ically
distributed, it h as been difficult to provide th e
com plete spectrum of com putation al an d data
m an agem en t facilities equally to all research ers.
Th e con sequen ce h as been duplication of effort
in som e cases, an d lack of com putation al facili-
ties in oth er cases. Gen eral purpose com puter
n etworks offer th e prospect of a m ore balan ced
distribution of com puter resources th rough out
th e ERDA com m un ity.
UCLA: University of California, Los Angeles (Computer Science
Department)
UCLA CCN: University of California, Los Angeles (Campus
Computer Network)
en cin g were m ade accessible to scien tists with in
an d with out ERDA th at n eed m igh t be fulfilled.
In th e 1960s an d 1970s, th e n ation al
weapon s laboratories (Los Alam os an d
Lawren ce Liverm ore) played a m ajor role in
gettin g com puter m an ufacturers an d un iversi-
ties to propose n ew com puter system s to solve
problem s th at took too lon g to solve on exist-
Sin ce en ergy problem s in volve research ers in
diverse fields residin g at differen t in stitution s,
m ore effective scien tific collaboration h as
becom e n ecessary. If m odern tech n iques such as
electron ic m ail an d com puter-based telecon fer-
Ap ril–Jun e 2000
43

Com p ut er Net w ork-Based Scien t ifi c Collab orat ion
WORKING GROUP ON COMPUTER
It is often forgotten that computer based
systems cannot function without the involvement
of human beings. Many people contributed to
this investigation.The current panel members
are listed on the inside front cover. On this page
we wish to acknowledge all others who made
significant contributions. If anyone has been left
out it was inadvertent.
NETWORKING
Mathematical and Computer Sciences (EE-03-04)
Office of Engineering,
Mathematical and Geo-sciences
Division of Basic Energy Sciences
ERDA
Harvey Baumel, LNS
Jerome Becker, ANL
Herbert Bernstein, BNL
Robert Braden, UCLA-CCN
Bertram Bussell, UCLA
Irving Cohen, UCLA
Robert Oaley, MIT
Y. Feinroth, NYU
Gerald Estrin, Chairman
IMPLEMENTATION PANEL
Ed Franceschini. Chmn, NYU-Courant
Lawrence Amiot, ANL
Graham H. Campbell, BNL
James E. Donnelley, LLL
Robert Fink, LBL
Anna Fisher, LNS
Lyman R. Hazelton, Jr., LNS-MIT
Michael Ford, LNS
Sheldon R. Furst, LBL
Robert L. Gardner, Jr., UCLA
Charles Gear, U of I
Robert Harvey, LBL
Robert Healy, LBL
Bradford Heckman, LBL
April Heiberger, ANL
S. Heller, BNL
INVESTIGATORS PANEL
Dennis E. Hall, Chmn, LBL
Ronald A. Bare, ANL
Graham H. Campbell, BNL
James E. Donnelley, LLL
Ed Franceschini, NYU-Courant
Terence E. Gray, UCLA
Lyman R. Hazelton, Jr., LNS-MIT
Muriel Kannel, LNS-MI1
Steve Kelem, UCLA
Roger B. Lazarus, LASL
P. Maclean, NYU
F. McGirt, LASL
OBJECTIVES PANEL
Gerald Estrin, Chmn, UCLA
Donald M. Austin, LBL
Charles N. Brashears, ERDA HQ
Elizabeth J. Campbell, LNS-MIT
Sidney Fernbach, LLL
Robert McMahon, ANL
Deane W. Merrill, LBL
A. Milonas, NYU
B. Miller, MFE
M. Nihalani, NYU
Doris Pahis, ANL
Beverly Robertson, UCLA
W. Russell, NYU
Kalman Saffran, LNS
Steven Scholbrock, ANL
M. H. Schwartz, ERDA
Marsha Joyce Shapiro, UCLA
Y. Shimamoto, BNL
L. Smith, SLAC
Max Goldstein, NYU
Glen Ingram, ERDA MFE
Edward C. Jacques, ERDA HQ
Clair G. Maple, AMES-ISU
Robert B. Marr, BNL
James C.T. Pool, ERDA MCS
Milton E. Rose, ERDA MCS
Richard J. Royston, ANL
T. Stuart, NYU
T.Tsun, NYU
Ronald Schwartz, ERDA HQ
B.Wells, LASL
Joseph Wells, SLAC
Steven Wolfe, UCLA-CCN
Jack Worlton, LASL
L.Yeh, BNL
Figure 2. Implementation Panel, Investigators
Panel, and Objectives Panel. (From Report on
General Purpose Computer Networks and
Resource Sharing I in ERDA)
Figure 3. Significant contributors to working group
effort. (From report on General Purpose Computer
Networks and Resource Sharing in ERDA)
in g system s. Th e laboratories com m itted to
purch ase th e first system s th at cam e off th e
production lin e of key com puter m an ufactur-
ers (e.g., IBM an d Con trol Data), even provid- dened with com plex security requirem ents. We
in g program m ers to develop operatin g system s could ignore security issues when we dealt with
and application program s. Following traditions th e feasibility of usin g th e in secure Arpan et.
establish ed at h igh -en ergy accelerator in stalla- When we organized the resource-sharing study,
tion s, research ers would go to th e com puta- we could draw on th e experien ce of th ose wh o
tional centers to do their experim ents and then
h ad worked with Arpan et. Furth erm ore, th ere
return to th eir h om e bases. Th ere were special- was in terest on th e part of th e Defen se
ized ERDA n etwork developm en ts to facilitate Advan ced Research Projects Agen cy to brin g
rem ote com m un ication , but th ey were bur- powerful ERDA resources on to th e n etwork,
4
4
IEEE An n als of t h e Hist ory of Com p ut in g

Figure 4. ERDA sites on the ARPA network. (From “Summary,
Conclusions and Recommendations,” General Purpose
Computer Networks and Resource Sharing in ERDA, vol. 1, p. 14)
Figure 5. Arpanet geographic map, August 1976 (From
“Summary, Conclusions and Recommendations,” General
Purpose Computer Networks and Resource Sharing in ERDA,
vol. 1, p. 14)
Figure 6. Accessing unique hardware: Reachable Through
Arpanet (From “Implementation of Arpanet Facilities,”
General Purpose Computer Networks and Resource
Sharing in ERDA, vol. 2, p. 5)
Figure 7. Accessing unique software: Reachable Through
Arpanet (From “Implementation of Arpanet Facilities,”
General Purpose Computer Networks and Resource Sharing in
ERDA, vol. 2, p. 7)
guaran teein g a cooperative attitude.
As of July 1977, seven ERDA research centers
were connected and had direct on-site access to
th e full ran ge of Arpan et facilities, in cludin g Figure 4 sh ows th e ERDA sites on th e Arpan et.
rem ote tim e-sh arin g system s, file tran sfer, an d Figure 5 gives th e com plete Arpan et geograph -
rem ote batch capabilities. Durin g th e previous ic m ap as of August 1976. Figure 6 sh ows som e
two years, m ore th an 60 n etwork-based appli- of th e un ique h ardware. Figure 7 depicts som e
cation s in volvin g on goin g research projects of th e un ique software m ade accessible by con-
h ad been in itiated an d an alyzed. Som e tech n i- n ection to th e Arpan et.
cal barriers to easy use of th e n etwork (such as
in adequate con sultin g support, differen ces in
(The software included the SARA design and
sim ulation software at UCLA, th e RATS gate-
com m an d an d program m in g lan guages, an d way software at LLL, th e Spice electron ic ch ip
saturation of h ost m ach in es) were discovered design software at LBL, th e MINPACK m in i-
an d h igh ligh ted. Som e im portan t n on tech n i- m ization software at ANL, the MACSYMA sym -
cal barriers to n etwork-based resource sh arin g bolic differen tial equation solver software at
were exposed; th e m ost sign ifican t was poten - MIT, th e Lin da m odelin g software at BNL, an d
tial loss of fun din g at local com putin g facilities th e SETL set logic software at NYU.)
because users could access lower-cost or m ore-
powerful rem ote facilities.
Figure 6 an d Figure 7 h igh ligh t th e basic
appeal of resource sharing through networks. If
Figure 2 iden tifies th e workin g group an d th e solution s of problem s at on e in stallation
th e study pan els. Figure 3 lists people wh o were lim ited by th e perform an ce of local com -
m ade sign ifican t con tribution s to th e project. puters an d if m ore-powerful com putation al
Ap ril–Jun e 2000
45

Com p ut er Net w ork-Based Scien t ifi c Collab orat ion
resources were accessible at a rem ote site, th en
Th e 60 or so n odes on th e Arpan et offer a variety
of h ardware capabilities wh ich h ave n ever been
collected in a sin gle site accessible to th e gen eral
scientific com m un ity.... Th ese include, for exam -
ple, th e Illiac IV parallel processor m ach in e at
NASA-Am es, th e CDC 7600s at LBL an d BNL, an
IBM 370/195 at ANL an d an IBM 370/195 at
RHEL for large scale com putation . Th ere are ter-
abit m ass storage devices available at CCA (th e
Datacom puter), at LBL (th e IBM Ph otodigital
Ch ipstore), an d at NASA-Am es (th e Un icon
device). Com puter output to m icrofilm devices
can be accessed at several sites ... providin g h igh
quality graph ics output for n etwork users.
pressure would obviously grow to gain access
to th e rem ote resources. If un ique software h ad
been developed over m an y years at a site, th en
users would m ove data th rough th e n etwork
an d determ in e th e applicability of th at un ique
software. If successful, th is rem ote use would
avoid th e costly an d som etim es im practicable
process of tran sportin g or redevelopin g soft-
ware; at th e very least, it would allow testin g
th e effectiven ess of th e rem ote software as a
prelude to developin g local production soft-
ware. Th e con clusion s of th e resource-sh arin g
study were set down as follows:
It is gen erally accepted th at stan dardization in
computing is slow in coming. Even a classical lan -
guage like Fortran , because it perm its m ach in e
depen den cies to exist covertly, can n ot always
claim portability for its program s. Furth erm ore,
really large scale software system s ten d to take
advantage of m achine architectures and thus cre-
ate portability problem s. If th ey can be used over
n etworks, th ere is n o n eed for tran sportin g.
It h as been dem on strated th at gen eral purpose
com puter n etworks can be used in th e ERDA
com m un ity to ach ieve two prin cipal ben efits of
rem ote resource sh arin g: better utilization of
resources an d im proved quality of research .
Th e Arpan et is a sm ooth ly fun ction in g, gen -
eral purpose com puter com m unications network
wh ich supports th e m ost versatile capability for
h eterogen eous, in depen den t com puter cen ters
in existen ce. In particular, th e large scale com -
putational facilities available at ERDA centers can
be adapted to support practical resource sh arin g
on an operation al basis.
Access to unique large data bases presents a nat-
ural application of distributed processin g. To
transport a large data base to m any com puter sys-
tem s is a difficult an d wasteful task. Wh ile few
people ever require th e en tire data base, th ey
often n eed to sam ple freely from it. If on e can
store the data efficiently on one system , and pro-
vide easy and efficient access to all users, real sav-
in gs in tim e an d effort m ay accrue. Th e
Datacom puter at CCA is a facility for im plem ent-
in g such a sch em e. It provides a system an d lan -
guages to perm it the creation, m aintenance, and
access for data bases on a terabit storage device.
Com puter n etworks offer th e possibility of
en h an cin g research collaboration by scien tists
from several in stitution s. Th e field of h igh en er-
gy ph ysics is particularly affected by th is require-
m en t, because th ere are on ly a few large particle
accelerators available. In order to do experim ents,
groups from several in stitution s join th eir efforts
in settin g up th e experim en t, takin g data, an d
an alyzin g th e data. In addition to facilitatin g th e
sharing of data and software, com puter networks
can , th rough facilities such as n etwork m ail an d
th e Plan et telecon feren cin g system , en h an ce
com m un ication s am on g th e research ers.
Un ique or special com putation al resources
will be effectively utilized in en ergy research an d
developm en t program s if th e resources are read-
ily available to research ers.
Electron ic m ail an d com puter telecon feren c-
in g in crease scien tific collaboration an d m an -
agem en t coordin ation .
Duplication of effort in software developm en t
can be reduced by providing network facilities for
accessing special program s at rem ote sites and for
tran sportin g software between cooperatin g sites.
Large scale com putation s can be off-loaded to
large centers, providing extra resources to a wider
ran ge of research ers.
In teraction with n on -ERDA research activities
in th e un iversity an d private sectors is greatly
en h an ced by com puter n etwork facilities.
Th e in itial costs for n etwork con n ection s are
m in im al com pared to th e lon g term ben efits
obtain ed.
Th e followin g section describes th e experi-
m en ts in im plem en tation of Arpan et facilities.
Table 1 sh ows th e im plem en tation m ile-
ston es th at were m et between 1973 an d 1977.
Im p lem en t at ion of Arp an et Facilit ies
An Im plem en tation Pan el, ch aired by E. Rem ot e Resource-Sh arin g Exp erien ce
Fran cesch in i at NYU-Couran t In stitute (see an d Fin d in g s
Figure 2), un dertook th e task of overseein g th e
In October 1975, th e tech n ical problem s
con n ection of all sites to th e Arpan et. Th e associated with con n ectin g th e participatin g
pan el n oted:
ERDA sites to the Arpanet were understood suf-
4
6
IEEE An n als of t h e Hist ory of Com p ut in g

Table 1. Implementation Milestones: 1973–1977
1
1
1
973 Q2 All facilities available at MIT-MULTICS and UCLA-CCN; IMP and HOST/IMP. Interface installed at LBL; Gateway to
MFENET.
Q3 Implementation goals defined at ANL, BNL, LBL, LNS, NYU, UCLA; Time sharing access to network from LLL
974 Q1 ARPA approval for ANL, BNL, and NYU to join Arpanet.
Q3 Varian V73’s installed at ANL.
Q4 PDP 11/40 installed at NYU.
975 Ql IMP installed at ANL; HOST/IMP. Interface installed at NYU.
Q2 Very distant host connection installed at BNL; IMP installed at NYU; Time sharing access to network from LBL; Time
sharing access to CDC-6600 (BKY-SESAME) at LBL.
Q3 Time sharing access to network from ANL and NYU; PDP 11 installed at BNL; File transfer and batch job submittal to
CDC 6600/7600.
1
1
976 Q2 Time sharing access to CDC-6600 (NOS-TELEX) at NYU.
Q3 Batch job submittal to IBM 370/195 at ANL; PDP 11/34 upgrade installed at BNL; File transfer and batch job
submittal to CDC 6600 at NYU.
Q4 Time sharing access to CDC-6600 (Scope-Intercom) at BNL.
977 Q1 Time sharing access to IBM 360/75 (TSO; WYLBUR) and file transfer at ANL.
Q2 Batch job submittal to CDC 6600/7600 and file transfer at BRL.
Source: “Implementation of Arpanet Facilities,” Gen eral Purpose Com puter Networks an d Resource Sh arin g, vol. 2, p. 12.
ficien tly to face th e critical m easures of success
for th e resource-sh arin g project. An
In vestigators Pan el was form ed, ch aired by
Den n is Hall of Lawren ce Berkeley Laboratory.
Th e pan el set out to gain firsth an d experien ce
in usin g th e n etwork for en ergy research .
Th e in dividual pan el m em bers explored use
of th e n etwork for particular projects. At each
ERDA site, n etwork prim ers were prepared to
ease access to rem ote ERDA resources. Th ese
prim ers were distributed to pan el m em bers
an d, with available con sultin g, m ade it reason-
able for en ergy research ers to use th e n etwork;
furth erm ore, guest accoun ts were set up to
rem ove an y adm in istrative fun din g barrier.
Fin ally, m ore th an 60 case h istories were docu-
m en ted at th e seven ERDA sites, all based on
active projects. Volum e 3 of th e ERDA report
presen ts all of th e case h istories.
in form ation , an d n etwork access facilities. In
short, these are all the operating system resources
wh ich are n ot selectable in th e program m in g
lan guage itself. Specification is don e in wh at h as
com e to be called a “com m and language” or “job
con trol lan guage,” an d m ust also be m ade
m ach in e readable....
Th e separation between program m in g lan -
guages an d com m an d lan guages is n ot a clean
on e … wh at m ay be specified in
a given
Program m in g lan guage at on e h ost m ay h ave to
be specified in th e com m an d lan guage at a dif-
feren t h ost. Th is is a con tin ual source of diffi-
culty in th e developm en t of exportable
software, an d is en coun tered repeatedly in n et-
work application s.
Th e effort required to learn en ough of a com -
m an d lan guage to in voke a given task is called
th e “startup barrier” for th at task.... th ere are n o
accepted com m an d lan guage stan dards.... In th e
Arpan et com m un ity alon e th ere are 18 m ajor
operatin g system com m an d lan guages. A sim i-
lar diversity is foun d in th e ERDA sites partici-
patin g in th is study.
Th e In vestigators Pan el explain ed som e
tech n ical problem s en coun tered in rem ote
resource sh arin g:
ERDA scien tists prescribe com putation al steps in
a program m in g lan guage such as Fortran , an d
th ese are m ade m ach in e readable eith er by key-
pun ch in g or by direct en try from an in teractive
term in al. For m ost application s it is also n eces-
sary to provide form atted data to th e program .
Most often th ese are supplied by autom atic col-
lection devices, or by very large online data bases.
To cause th e com putation to be perform ed,
th e scien tist m ust also specify th e operatin g sys-
tem services n eeded. Exam ples of th ese in clude
input/output and data handling services, job and
task con trol, security and protection, accoun ting
Im portant econom ic barriers, alluded to ear-
lier, were also h igh ligh ted:
On e im portan t use of th e n etwork is to offload
large scale com putin g tasks from sm aller
m ach in es to th e m ore powerful m ach in es at th e
larger sites ... these tasks represent a m ajor source
of operatin g fun ds.... In ligh t of th is, th e h igh
degree of cooperation am ong the ERDA laborato-
ries participating in this study is both rem arkable
and encouraging.
Ap ril–Jun e 2000
47

Com p ut er Net w ork-Based Scien t ifi c Collab orat ion
Figure 8. Using the network as a high-quality,
high-speed telephone connection. (From Remote
Resource Sharing Experience and Findings, vol. 3, p. 5)
Figure 9. Using the LLL gateway to connect the ARPA net to the MFE
net (From Remote Resource Sharing Experience and Findings, vol. 3, p. 5)
Th e ERDA In vestigators Pan el
ch aracterized th e case h istories in
Figure 8, Figure 9, Figure 10, Figure
1
1, an d Figure 12.
At each site, the first use of the net-
work was as a high-quality, high-speed
telephone connection as exhibited in
Figure 8. There was a negligible start-
up barrier, because the connection was
usually m ade to a fam iliar system ; for
exam ple, a ph ysicist wh o tran sferred
to LBL from the Rutherford Laborato-
ry in th e Un ited Kin gdom n eeded to
finish his Gam m a Detector Efficiency
Code th at h e h ad started at RHEL.
After a few m inutes of instruction, he
was able to access the 8,000-m ile-dis-
tan t Ruth erford com puter com plex
from LBL as if it were local. Sim ilarly,
scien tists were able to access th e
Magn etic Fusion En ergy n etwork
developed at Lawrence Livermore Lab-
oratory and thereby run program s on
a Control Data 7600 as shown in Fig-
ure 9. Figure 10 exem plifies th e kin d
of scientific collaboration that was car-
ried out usin g electron ic m ail an d
m ain tain in g a con tin uous tran script
of every in form ation exch an ge. A
m uch m ore dram atic exam ple of sci-
en tific collaboration arose in th e dis-
Figure 10. Using the network and electronic mail for scientific collaboration (From
Remote Resource Sharing Experience and Findings, vol. 3, p. 5)
tributed
gen eration
of
th e
Figure 11. Teleconferencing (From Remote Resource Sharing Experience and
Findings, vol. 3, p. 7)
four-volum e rem ote resource-sharing
report using the facilities illustrated in
Figure 11 and described in the Network
Experience Report UCLA-1 (Figure 12)
reproduced from volum e 3, Remote
Resource Sharing Experience and Find-
ings. A sim ilarly dram atic exam ple
illustrating the use of software devel-
oped at different sites to solve a prob-
lem an d avoid costly local software
developm ent is illustrated in the side-
bar on Usin g Com plem en tary Soft-
ware to Reduce Duplicate Effort an d
described in th e sidebar on Network
Experience Report LBL-7.
Figure 12. Network experience report UCLA-1 (From Remote Resource Sharing
Experience and Findings, pp. 129–131, Appendix H)
4
8
IEEE An n als of t h e Hist ory of Com p ut in g

Usin g Com p lem en t ary Soft w are t o Red uce Dup licat e Effort
(
From Remote Resource Sharing Experience and Findings, vol. work gives authors the freedom to choose one they like.
3
, p. 7)
Frequently, a remote host was accessed purely because
that host had a “better” text editor than the one available
on the local host. All participants were given access to the
files being accumulated at MULTICS so everyone could
review and comment, as well as see visible signs of
Network Experience Report UCLA-1
ERDA Program: BES–M&CS
Title: Distributed Report Generation
Cooperating Sites: ANL, BNL, LBL, LLL, LNS, NYU, progress. The Planet teleconferencing system at BBN was
UCLA
used extensively during the report writing period to facili-
tate dissemination of procedures, guidelines, status, etc.
UCLA Contact: G. Estrin
Resources Accessed: ANL, BBN.CCN, CCA, LBL, MIT- and as a message board for participants to solicit help in
Multics, LLL, BNL, NYU, MIT-MC, MITXGP
accomplishing a particular task, such as moving a file from
one host to another. Other remote resources were used in
preparing this report as well. A facility at MULTICS for stor-
Mot ivat ion
Producing this report required the participation of ing all of the user input and system output from an inter-
ERDA sites that were widely distributed geographically. active terminal session on a file was used in preparing some
Each site was involved in determining what information of the scenarios in Appendix F, which show how to use
should be in the report, who should be responsible for some of the resources at each of the participating labora-
which parts, and how they should be organized. Each tory sites. A procedure was established to aid in communi-
site was to prepare their own set of appendices, which cating suggested changes to a portion of the report. Once
would eventually be merged into one set. In addition, the author had placed the section draft online, he would
individuals were charged with coordinating certain sec- send a message to whomever he wanted to review the sec-
tions and writing portions of the main body. From the tion, stating the name and location of the file, as well as any
first discussion of the report, it was clear that computer necessary access control information. Then the reviewer(s)
teleconferencing could be an extremely valuable com- would print the section locally and annotate changes on
munication and coordination tool. Later it became even their own copy. Then they would make a copy of the file
more clear that a great deal of time could be saved dur- and, using a text editor, incorporate their suggested
ing the production phase of the report if network changes in this copy of the file. The convention was estab-
resources were fully utilized. In addition, the use of text lished that all changes would be made in uppercase, so that
formatting and editing programs, and a Xerox Graphic they would easily stand out when the author was perusing
Printer, promised the possibility of a better report than the modified file. Deletions were indicated by a series of Xs.
could have been produced using conventional manual After the suggested modifications were made, the author
methods.
and whoever else was involved in that section was given
—
—
Perhaps the most important motivation was the the name and location of the modified file. The modified
prospect of distributing portions of the report for review file was then printed locally by the interested parties for fur-
as soon as they were written and at electronic speeds. ther review. Once the initial versions of each section were
Distributing changes was equally important.
available at MULTICS, they were grouped into larger sec-
tions, and additional format control information was insert-
ed. The text was then processed by a text formatting
Proced ure
It was agreed that the report would be produced using program called Runoff and the output directed to a Xerox
text formatting programs at MIT-MULTICS and MIT’s Xerox Graphic Printer [XGP]. At this point, a meeting was held to
Graphic Printer. A new directory was established in the review the initial version of the report in its printed form,
MULTICS Storage System for files associated with this make additional revisions, and merge graphical informa-
report. In some cases, portions of the report were generat- tion. After all revisions had been made to the text, the
ed at an author’s local host and then transmitted, using a entire report was processed once again by the Runoff pro-
File Transfer Program, to MULTICS. In other cases, MULTICS gram and printed on the XGP. During the period when files
was accessed remotely using a network connection and were being accumulated at MULTICS, procedures were
text files created there using one of the MULTICS text edi- established for periodically making backup copies of all files
tors. Since MULTICS is connected to both Arpanet and and storing them on the CCA Datacomputer. Also, since it
Telenet, the report authors (and staff) could establish a con- was important for all of the participants to have access to
nection by dialing a local Telenet port, using an Arpanet TIP all the files, procedures were established to make sure that
or going through their local host and establishing a telnet one person making changes to a file did not wipe out the
connection. It should be mentioned that there are a wide
variety of text editing programs in existence, and the net-
changes made by someone else.
continued on p. 50
Ap ril–Jun e 2000
49

Com p ut er Net w ork-Based Scien t ifi c Collab orat ion
continued from p. 49
Ob served Ben efi t s
imized only through greater standardization of key net-
work resources (e.g., command languages) and better
A much higher quality report was produced than could educational tools, including consultants, self-teaching pro-
have been possible using conventional methods (i.e., type- grams, and improved documentation—both online and
writers and the Post Office). In addition, the work was conventional.
accomplished in a much shorter time than would have
Additional problem areas related to differences in writ-
been possible otherwise. Only three face-to-face meetings ing style and format of the several coauthors and the prob-
were required from inception to completion (a period of lems associated with controlling font changes on the XGP
about six months). This was largely due to the effective- printer. The former problem provided strong evidence that
ness of the Planet teleconferencing system as a coordina- time spent in establishing standards for style and format—
tion tool. Planet also minimized the number of lengthy before anyone starts writing—is more than worthwhile. In
phone calls between participants. From the time the deci- fact, the development of text editor macros that define a
sion was made to use network facilities to produce the standard format for, say, experimental results, is an excel-
report to the printing of the first version of the completed lent idea. The difficulties associated with font changes on
report was less than three weeks.
the XGP were due to limitations of the text formatting pro-
gram. Escape sequences were defined that would later be
translated to XGP font change commands. These escape
Ob served Barriers
Becoming familiar with the various network resources, sequences proved to be confusing and cumbersome.
such as text editors, formatters, file transfer programs, etc.,
required additional learning for many of the participants. Evaluat ion
While not an insurmountable barrier, this cost can be min-
Successful.
There was one event, characteristic of efforts
to use newly developing technologies, that arose
durin g th e UCLA-1 experien ce. We were figh t-
ing deadlines to put together the first draft of the
four-volum e report, and I recall one night when
Terry Gray an d I were workin g in to th e wee
h ours to en ter som e of our m aterial. Our data
entry person suddenly said that som ething was
wron g with th e system , because m aterial th at
she was sure she had entered was not there. Gray
began som e detective work. He discovered that
som eon e else on th e n etwork h ad becom e so
enam ored with a new spell-check program that
h e h ad collected several report section s, put
th em th rough th e spell-ch eck, an d th en put
th em back. Th e on ly problem was th at, m ean -
while, there had been new entries made, and the
correctly spelled m aterial overwrote th e m ore
recen tly edited work. We foun d it n ecessary to
awaken th e well-m ean in g culprit at h om e an d
m ake him undo the havoc that he had created.
Th is explain s wh y th e case h istory refers to
explicit protocols for group editin g. An oth er
dram atic scen e th at will stay forever in m y
memory occurred the last night of editing, when
participan ts at every ERDA site were reviewin g
th e report docum en t. A sm all group of us h ad
accepted the task of making final changes. It was
a ch aracteristic of th e ERDA support structure
th at everyon e’s fun din g cam e from th e sam e
source, and each task force member became par-
ticularly tun ed to m akin g sure th at th e report
said good things about his or her laboratory and,
in som e cases, revealed a relative weakness in a
Figure 13. Department of Energy letter. (Personal letter, 16 January 1978)
5
0
IEEE An n als of t h e Hist ory of Com p ut in g

Net w ork Exp erien ce Rep ort LBL-7 ERDA Prog ram : Hig h -En erg y Ph ysics (EC)
Title: PBAR-P Partial Wave Analysis
Cooperating Sites: LBL, RHEL
further eliminate unlikely solutions. The results of the con-
tinuity fit will then be transferred to RHEL (again by FTP),
LBL Contact: D. Hall (415) 843-2740 x6053 or FTS 451-6053 which will further eliminate unlikely solutions. The results
of the continuity fit will then be fed to the RHEL pole-extrac-
Resources Accessed:
tion program which will use them as weak constraints to
limit the number of possible solutions still further.
These results will then be fed to a graphics display code
(MUGWUMP) at RHEL for final checking and publication.
Experimental data—RHEL, CERN, BNL.
Partial wave generator code—RHEL
Electronic mail, MUGWUMP—RHEL
Cluster analysis code—LBL
Continuity fitting code—LBL
Pole extraction code—RHEL
Graphics display code—RHEL
Ob served Ben efi t s
Energy-related scientific research and development
Scientific collaboration and communication
Mot ivat ion
Reduction of duplicate software development
Physicists using the network as a mechanism for main-
taining communication discovered that complementary Ob served Barriers
software existed at their respective sites which would allow
a highly sophisticated processing chain to be constructed Insufficient Network Hardware Support
for analysis of proton antiproton annihilations.
File transfer speed between RHEL and the network is
Transporting either set of software to the other site would limited by the 1,200-baud telephone link between from
have been considerably more effort than merely transfer- the London TIP to RHEL. For this application, file transfers
ring output at each stage of the process.
typically require two hours to complete. It is significant
that the physicists are sufficiently interested in the results
to be willing to come in nights and weekends to perform
Proced ure
Data from previous experiments from many groups was the transfers when network traffic and the LBL file system
formatted and transmitted to RHEL (currently, CERN uses are relatively quiescent.
an RJE link and BNL uses air mail, but the potential for data
acquisition over the network is obvious). Next, the partial Evaluat ion
wave generation code at RHEL was used to produce many
Successful and continuing.
sets of partial wave coefficients as possible solutions. These
This impressive procedure was initiated by the physi-
coefficient sets were transmitted to LBL (using FTP) for fur- cists themselves, which is typical in high-payoff situations.
ther analysis. Familiarity with their local command languages and pro-
The cluster analysis program at LBL was invoked to sep- gramming languages meant that neither group had to
arate the possible solutions into several groups. This is an learn the other’s system, beyond what had already been
iterative process which requires judgment on the part of the learned in establishing initial contact. The fact that slow
physicist. The separation process is currently underway with file transfer speeds were deemed acceptable is further evi-
physicists from both sites communicating by electronic mail. dence of the importance attached to the application. In
When cluster analysis is completed, the results will be general, however, 1,200 baud is too slow for most
passed to a continuity fitting code also at LBL, which will resource-sharing applications.
com petin g laboratory. As th e deadlin e com puters dedicated to a special application
approach ed, th ere developed a stream of pro- dom ain) and through SACNET (which was con -
posed changes that flooded our lines. Som ehow cerned with secure com m unications). However,
th e clock ticked away th e rem ain in g tim e, an d piggybacking on the em erging Arpanet allowed
we were uniform ly happy with the result.
testin g of n ew m eth ods th at m igh t n ot h ave
occurred for years an d probably spurred th e
growth of Arpan et by in troducin g a well-recog-
Con clusion
Th e four-year project laun ch ed th e en ergy n ized an d supported application dom ain to
research com m un ity in to th e explodin g world dem on strate resource-sh arin g effectiven ess.
of com puter n etwork tech n ology. Th e Figure 13 sh ows th e Jan uary 1978 letter from
research ers were given lin ks to a m uch larger th e Departm en t of En ergy testifyin g to th e sig-
research com m un ity. Th ere h ad been earlier n ifican ce of th is project.
exploration of resource sh arin g th rough th e
Th ere are at least two oth er ways in wh ich
MFENET (wh ich attem pted to in tercon n ect th is early work h ad a m ajor im pact on com -
Ap ril–Jun e 2000
51

Com p ut er Net w ork-Based Scien t ifi c Collab orat ion
putin g. Th e en ergy research com m un ity con - at Defense Advanced Research Projects Agency-
sisted of profession als wh o faced ch allen ges at supported sites to share their facilities with out-
th e cuttin g edge of com putation al capabilities. siders. In both cases, I believe, the com bination
Defen se fun ds were available to buy th e m ost- of in cen tive fun ds an d th reat of loss of support
advan ced com puter system s as soon as th ey bough t th e in terest an d cooperation of cen ter
were produced; generally, funds perm itted buy- directors.
in g on e at LLL an d participatin g in develop-
m en t of th e software system s th at put th e
Referen ce
h ardware to work. Th ere was little collabora-
tion between th e n ation al laboratories; in fact,
there was a great deal of com petition to acquire
the next com puter, because that helped the lab-
oratory to attract fun ds an d research ers. Th e
practice am on g h igh -en ergy ph ysicists was for
th em to travel to a particle accelerator cen ter
an d take turn s usin g th e equipm en t. Th e com -
puter n etwork brough t th em in to a n ew era, in
wh ich th ey could do m uch m ore work rem ote-
ly, sh are resources, an d th en gath er face-to-face
as n eeded to discuss th eir m odels an d experi-
m en ts. Furth erm ore, th e collaboration dem on-
strated in th e plan n in g, execution , an d
reportin g on experim en ts in th ese reports left
a legacy for future n etwork-based research . Th e
resource sh arin g th at was docum en ted m ade it
m uch m ore likely th at an expen diture on n ew
technology would be utilized m ore fully within
th e relatively sh ort lifetim e of a com puter sys-
tem an d th at th ere m igh t be less waste in soft-
ware developm en t efforts.
1. General Purpose Computer Networks and
Resource Sharing in ERDA, 15 July 1977, Working
Group on Computer Networking, Mathematical
and Computer Sciences (EE-03-04), Office of
Engineering, Mathematical and Geo-sciences,
Division of Basic Energy Sciences, ERDA. Vol. 1,
Summary, Conclusions and Recommendations, 26
pp. Vol. 2, Implementation of Arpanet Facilities, 55
pp. Vol. 3, Remote Resource Sharing Experience
and Findings, 141 pp. Vol. 4, Arpanet Resource
Handbook for ERDA Sites, 247 pp.
Gerald Estrin received the PhD
in electrical en gin eerin g at th e
Un iversity
of
Wiscon sin ,
Madison in Jan uary 1951.
Estrin was a m em ber of Joh n
von Neum an n ’s Electron ic
Th is project was a clear exam ple of th in kin g
Com puter Group at th e
“
outside th e box.” Rose was h ead of th e
In stitute for Advan ced Study in Prin ceton , New
Jersey, from 1950 to 1956. After com pletion of th e
Math em atics an d Com puter Scien ce Research
program in ERDA in 1973. I was, th en , IAS m ach in e, h e took leave to direct th e buildin g of
Prin cipal In vestigator for a research program
ERDA supported at UCLA. I h ad also been
Prin cipal In vestigator of Defen se Advan ced
th e WEIZAC com puter at th e Weizm an n In stitute of
Scien ce in Reh ovot, Israel.
In 1956, Estrin becam e a faculty m em ber at th e
Research Projects Agen cy’s pre-Arpan et con - Un iversity of Californ ia, Los An geles, wh ere h e built
tract at UCLA; th erefore, several of th e gradu- th e academ ic program in com puter en gin eerin g an d
ate studen ts wh o worked with m e h ad been
led research on recon figurable com puter arch itecture
and continued to be deeply involved in the cre- an d m eth ods an d tools to support design of com put-
ation an d growth of th e Arpan et. Durin g a dis- er system s. Estrin was in volved in ARPA n etworkin g
cussion with Rose, I suggested th at com puter
scien tists an d applied m ath em atician s m igh t
projects startin g in 1965 an d sparked in terest of th e
en ergy research com m un ity in resource sh arin g
sign ifican tly in crease th eir im pact on th e en er- th rough n etworks. He served as Ch air of th e UCLA
gy research program . If h e would auth orize a
project to explore resource sh arin g th rough
Com puter Scien ce Departm en t from 1979 to 1982
an d from 1985 to 1988. Estrin h as been recogn ized
gen eral-purpose com puter n etworkin g, I was as AAAS Fellow, Guggen h eim Fellow, IEEE Fellow,
willing to lead the working group in evaluating an d IEEE Com puter Society Pion eer.
its poten tial. He stuck h is n eck out, an d I th in k
h e n ever regretted doin g so. With out h is offi-
cial support an d th e im pact of Rose’s sm all but
Direct question s an d com m en ts to Gerald
precious budget, th is project would n ot h ave Estrin , Professor Em eritus, Com puter Scien ce
succeeded. It suffered from th e sam e kin d of Departm en t, 4731C Boelter Hall, Un iversity of
problem th at caused great difficulty in in itial Californ ia,
Los
An geles,
CA 90095;
developm en t of th e Arpan et. Th ere was n o estrin @cs.ucla.edu.
in cen tive for com puter cen ter directors at
ERDA n ation al laboratories or cen ter directors
5
2
IEEE An n als of t h e Hist ory of Com p ut in g