5P41-RR00785-12 RXDX Project

If fees for using SUMEX resources were imposed, this would have a drastically limiting
effect on the value of the system to us. Even if we had a budget to purchase such
services, the inhibiting effect of having a meter running would cause us to make less
use of it that we should. We have been conscious of the costs of the system and feel
that we have not used it imprudently, even though we have not directly borne its costs.

201 E. H. Shortliffe
§P41-RRO00785-12 Stanford Knowledge Systems Laboratory

Appendix A
Stanford Knowledge Systems Laboratory

ARTIFICIAL INTELLIGENCE RESEARCH IN THE
KNOWLEDGE SYSTEMS LABORATORY
(Incorporating the Heuristic Programming Project)

Stanford University
Department of Computer Science/Department of Medicine
April 1985

The Knowledge Systems Laboratory (KSL) is an artificial intelligence research
laboratory of about 90 people -- faculty, staff, and students -- within the Departments
of Computer Science and Medicine at Stanford University. KSL is the new name for
the interdisciplinary AI research community that has evolved over the past two decades.
Begun as the DENDRAL Project in 1965 and known as the Heuristic Programming
Project from 1972 to 1984, the new organization reflects the increasing complexity and
diversity of the research now under way. The KSL is a modular laboratory, consisting
of five collaborating yet distinct groups with different research themes:

e The Heuristic Programming Project (HPP), Professor Edward A. Feigenbaum,
scientific director -- blackboard systems, concurrent system architectures for AI,
and the modeling of discovery processes. Executive director: Robert Engelmore.
Research scientists: Harold Brown, Byron Davies, Bruce Delagi, Peter Friedland,
Barbara Hayes-Roth, and H. Penny Nii. Consulting professor: Richard Gabriel.

e The HELIX Group, Professor Bruce G. Buchanan, scientific director -- machine
learning, transfer of expertise, and problem solving. Faculty: Paul
S. Rosenbloom (joint appointment, Computer Science and Psychology). Research
scientists: James Brinkley, William J. Clancey, Barbara Hayes-Roth.

e The Medical Computer Science (MCS) Group, Professor Edward H. Shortliffe,
scientific director (Department of Medicine with courtesy appointment in
Computer Science) -- research on and advanced application of AI to medical
problems; includes the Medical Information Sciences (MIS) program. Research
scientist: Lawrence M. Fagan.

e The Logic Group, Professor Michael R. Genesereth, scientific director -- formal
reasoning and introspective systems. Research scientist: Matthew L. Ginsberg.

e The Symbolic Systems Resources Group (SSRG), Thomas C. Rindfleisch,
scientific director (joint appointment, Computer Science and Medicine)
~~ research on and operation of computing resources for AI research, including
the SUMEX facility. Assistant director: William J. Yeager.

Tom Rindfleisch serves as KSL project director.

This brochure summarizes the goals and methodology of the KSL, its research and
academic programs, its achievements, and the research environment of the laboratory.

203 E. H. Shortliffe
Stanford Knowledge Systems Laboratory 5P41-RR00785-12

Basic Research Goals and Methodology

Throughout a 20-year history, the KSL and its predecessors, DENDRAL and HPP, have
concentrated on research in expert systems -- that is, systems using symbolic reasoning
and problem-solving processes that are based on extensive domain-specific knowledge.
The KSL's approach has been to focus on applications that are themselves significant
real-world problems, in domains such as science, medicine, engineering, and education,
and that also expose key, underlying AI research issues. For the KSL, AI is largely an
empirical science. Research problems are explored, not by examining strictly theoretical
questions, but by designing, building, and experimenting with programs that serve to test
underlying theories.

The basic research issues at the core of the KSL's interdisciplinary approach center on
the computer representation and use of large amounts of domain-specific knowledge,
both factual and heuristic (or judgmental). These questions have guided our work since
the 1960s and are now of central importance in all of AI research:

1. Knowledge representation. How can the knowledge necessary for complex
problem solving be represented for its most effective use in automatic inference
processes? Often, the knowledge obtained from experts is heuristic knowledge,
gained from many years of experience. How can this knowledge, with its
inherent vagueness and uncertainty, be represented and applied?

2. Knowledge acquisition. How is knowledge acquired most efficiently -- whether
from human experts, from observed data, from experience, or by discovery?
How can a program discover inconsistency and incompleteness in its knowledge
base? How can knowledge be added without perturbing the established
knowledge base?

3. Use of knowledge. By what inference methods can many sources of knowledge
of diverse types be made to contribute jointly and efficiently toward solutions?
How can knowledge be used intelligently, especially in systems with large
knowledge bases, so that it is applied in an appropriate manner at the
appropriate time?

4. Explanation and tutoring. How can the knowledge base and the line of
reasoning used in solving a particular problem be explained to users? What
constitutes a sufficient or an acceptable explanation for different classes of
users? How can problem-solving systems be combined with pedagogical and
user knowledge to implement intelligent tutoring systems?

5. System tools and architectures. What kinds of software tools and system
architectures can be constructed to make it easier to implement expert programs
with greater complexity and higher performance? What kinds of systems can
serve as vehicles for the cumulation of knowledge of the field for the
Tesearchers?

Research and Academic Programs
CURRENT RESEARCH PROJECTS

The following list of projects now under way within the five KSL research groups gives
a brief summary of the major goals of each project and lists the personnel (staff and
Ph.D. candidates) directly involved. More complete information on individual projects
can be obtained from the person indicated as the project contact. Inquiries should be

addressed in care of:

E. H. Shortliffe 204
5P41-RR00785-12 Stanford Knowledge Systems Laboratory

Knowledge Systems Laboratory
Department of Computer Science
Stanford University

701 Welch Road, Building C
Palo Alto, CA 94304
415-497-3444

The Heuristic Programming Project

e Advanced Architectures Project -- Design a new generation of computer
architectures to exploit concurrency in blackboard-based signal understanding
systems.

Personnel; Edward A. Feigenbaum (contact), Harold Brown, Byron Davies (TT),
Bruce Delagi (DEC), Richard Gabriel, Penny Nii, Sayuri Nishimura, Jim Rice,
Eric Schoen, Jerry Yan.

« Knowledge~Based VLSI Design Project -- Study the hierarchical design process
involved in the development of complex very large scale integrated circuits.
Personnel: Harold Brown (contact), Jerry Yan.

e Blackboard Architecture Project -- Integrate current knowledge about blackboard
framework problem-solving systems and develop a domain-independent model
that includes knowledge-based control processes.

Personnel: Barbara Hayes-Roth (contact).

« MOLGEN -- Study the processes of scientific theory formation and
modification, using recently developed models of genetic regulation as an
example.

Personnel: Peter Friedland (contact), Charles Yanofsky (Biological Science), Peter
Karp.

The HELIX Group

e PROTEAN -- Study complex symbolic constraint-satisfaction problems in the
blackboard framework with application to protein structure determination from
nuclear magnetic resonance data.

Personnel; Bruce Buchanan (contact), Oleg Jardetzky (Nuclear Magnetic
Laboratory), Jim Brinkley, Barbara Hayes-Roth, Russ Altman, Olivier Lichtarge.

¢ NEOMYCIN/GUIDON2 -- Develop knowledge representation and explanation
capabilities for the computer-aided teaching of diagnostic reasoning.
6a Bill Clancey (contact), Stephen Barnhouse, Diane Hasling, David
. Wilkins,

e SOAR -- Develop a_ general production-system-based problem-solving
architecture that integrates reasoning, domain expertise, learning, and planning
of problem-solving strategies.

Personnel: Paul Rosenbloom (contact), Andrew Golding, Amy Unruh.

* Knowledge Acquisition Studies -- Study the processes for transferring knowledge
into a computer program, including learning by induction, analogy, watching,
chunking, reading, and discovery.

Personnel: Bruce Buchanan (contact), Li-Min Fu, Russell Greiner, Ramsey
Haddad, David C. Wilkins.

The Medical Computer Science Group

e ONCOCIN -- Develop knowledge-based systems for the administration of
complex medical treatment protocols such as those encountered in cancer

205 E. H. Shortliffe
Stanford Knowledge Systems Laboratory $P41-RRO00785-12

chemotherapy. Personnel: Ted Shortliffe (contact), Charlotte Jacobs (Oncology),
Larry Fagan, David Combs, Gregory Cooper, Jay Ferguson, Christopher Lane,
Janice Rohn, Homer Chin, Holly Jimison, Curt Langlotz, Mark Musen, Glenn
Rennels.

e PATHFINDER -- Develop a knowledge-based system for diagnosis of lymph
node pathology.
Personnel: Ted Shortliffe, Bharat Nathwani (USC), Larry Fagan (contact), David
Heckerman, Eric Horvitz.

The Logic Group

¢ Metalevel Representation System (MRS) -- Study logic-based introspective
programs that can reason about and control their own problem-solving activities.
Personnel: Mike Genesereth (contact), Matt Ginsberg, Russ Greiner, Ben Grosof,
Yung-Jen Hsu, David E. Smith, Devika Subramanian, Richard Treitel.

¢ The DART/HELIOS Project -- Study an integrated design environment that
includes capabilities for design specification, refinement, and validation;
fabrication engineering; and failure diagnosis and testing.

gersonnel: Mike Genesereth (contact), Glenn Kramer (Fairchild), Narinder
ingh,

e Intelligent Agent Project -- Study planning and problem-solving activities for
an intelligent interface between human users and complex computing
environments.

Personnel: Mike Genesereth (contact), Matt Ginsberg, Jeff Finger, Jeff
Rosenschein, Jock Mackinlay, Vineet Singh.

¢ Intelligent Task Automation -- Build a program that can use the description of a
manufacturing task to develop a plan by which a robot can carry out the task.
Personnel: Mike Genesereth (contact), Matt Ginsberg, Jeff Finger, David
E. Smith, Richard Treitel.

The Symbolic Systems Resources Group (SSRG)

e SUMEX~-AIM Resource -- Develop and operate a national computing resource
for biomedical applications of artificial intelligence in medicine and for basic
Tesearch in AI at KSL.

Personnel: Tom Rindfleisch (contact), Bill Croft, Frank Gilmurray, Christopher
Schmidt, Andrew Sweer, Israel Torres, Bob Tucker, Nicholas Veizades, Bill
eager,

e Financial Resource Management -- Develop an expert system for financial
resource planning.
Personnel: Tom Rindfleisch (contact), Bruce Buchanan.

Other Projects

The KSL also has close ties to collaborative projects. These include PIXIE, developing
an intelligent tutoring system, under Derek Sleeman in the School of Education, and
RADIX, studying discovery of knowledge from databases, under Bob Blum in Computer
Science.

STUDENTS AND SPECIAL DEGREE PROGRAMS

Graduate students are an essential part of the research productivity of the KSL.
Currently 41 students are working with our projects centered in Computer Science and

E. H. Shortliffe 206
§P41-RR00785-12 Stanford Knowledge Systems Laboratory

another 12 students are working with the MCS/MIS programs in Medicine. Of the 41
working in Computer Science, 25 are working toward Ph.D. degrees, and 16 are working
toward M.S. degrees. A number of students are pursuing interdisciplinary programs and
come from the Departments of Engineering, Mathematics, Education, and Medicine.

Because of the highly interdisciplinary and experimental nature of KSL research, two
special degree programs have been established:

The Medical Information Sciences (MIS) program is an interdepartmental program
approved by Stanford University in 1982. It offers instruction and research
opportunities leading to the M.S. or Ph.D. degree in medical information sciences, with
an emphasis on either medical computer science or medical decision science. The
program, directed by Ted Shortliffe and co-directed by Larry Fagan, is formally
administered by the School of Medicine, but the curriculum and degree requirements are
coordinated with the Dean of Graduate Studies and the Graduate Studies Committee of
the University. The program reflects our local interest in the interconnections between
computer science, artificial intelligence, and medical problems. Emphasis is placed on
providing trainees with a broad conceptual overview of the field and with an ability to
create new theoretical and practical innovations of clinical relevance.

The Master of Science in Computer Science: Artificial Intelligence (MS:AI) program is a
terminal professional degree offered for students who wish to develop a competence in
the design of substantial knowledge-based AI applications but who do not intend to
obtain a Ph.D. degree. The MS:AI program is administered by the Committee for
Applied Artificial Intelligence, composed of faculty and research staff of the Computer
Science Department. Normally, students spend two years in the program with their
time divided equally between course work and research. In the first year, the emphasis
is on acquiring fundamental concepts and tools through course work and and project
involvement. During the second year, students implement and document a substantial
AI application project.

Academic and Research Achievements

The primary products of our research are scientific publications on the basic research
issues that motivate our work, computer software in the form of the expert systems and
AI architectures we develop, and the students we graduate who continue AI research in
other academic and industrial laboratories.

The KSL has averaged publishing more than 45 research papers per year in the AI
literature, including journal articles, theses, proceedings articles, and working papers. In
addition, many talks and invited lectures are given annually. In the past few years, 11
major books have been published by KSL faculty, staff, and former students, and
several more are in progress. Those recently published include:

e Heuristic Reasoning about Uncertainty: An Al Approach, Cohen, Pitman, 1985.

e Readings in Medical Artificial Intelligence: The First Decade, Clancey and
Shortliffe, Addison-Wesley, 1984.

e Rule~Based Expert Systems: The MYCIN Experiments of the Stanford
Heuristic Programming Project, Buchanan and Shortliffe, Addison-Wesley, 1984.

e The Fifth Generation: Artificial Intelligence and Japan's Computer Challenge to
the World, Feigenbaum and McCorduck, Addison-Wesley, 1983.

« Building Expert Systems, F. Hayes-Roth, Waterman, and Lenat, eds., Addison-
Wesley, 1983.

207 E. H. Shortliffe
Stanford Knowledge Systems Laboratory 5P41-RR00785-12

e System Aids in Constructing Consultation Programs: EMYCIN, van Melle, UMI
Research Press, 1982.

« Knowledge-Based Systems in Artificial Intelligence: AM and TEIRESIAS, Davis
and Lenat, McGraw-Hill, 1982.

° The Handbook of Artificial Intelligence, Volume I, Barr and Feigenbaum, eds.,
1981; Volume II, Barr and Feigenbaum, eds., 1982; Volume III, Cohen and
Feigenbaum, eds., 1982; Kaufmann.

e Applications of Artificial Intelligence for Organic Chemistry: The DENDRAL
Project, Lindsay, Buchanan, Feigenbaum, and Lederberg, McGraw-Hill, 1980.

Our laboratory has pioneered in the development and application of AI methods to
produce high-performance knowledge-based programs. Programs have been developed
in such diverse fields as analytical chemistry (DENDRAL), infectious disease diagnosis
(MYCIN), cancer chemotherapy management (ONCOCIN), pulmonary function
evaluation (PUFF), machine fault diagnosis (DART), VLSI design
(KBVLSI/PALLADIO), and molecular biology (MOLGEN). Some of these programs
Tival human experts in solving problems ‘in restricted domains. A number of projects
have developed generalized software tools for representing and using knowledge; of
these, EMYCIN, AGE, MRS, and BB] are available to outside research groups. Some of
our systems and tools (eg., DENDRAL, PUFF, UNITS, and EMYCIN) are now also
being adapted for commercial development and use in the burgeoning AI industry.

Following our lead in work on biomedical applications of AI and the development of
the SUMEX-AIM computing resource, a nationally recognized community of academic
projects on AI in medicine has grown up.

Central to all KSL research are our faculty, staff, and students. These people have been
recognized internationally for the quality of their work and for their continuing
contributions to the field. KSL members participate extensively in professional
organizations, government advisory committees, and journal editorial boards. They have
held major managerial posts and conference chairmanships in both the American
Association for Artificial Intelligence (AAAI) and the International Joint Conference
on Artificial Intelligence (IJCAI).

Several KSL faculty and former students have received significant honors. In 1976, Ted
Shortliffe received the Association of Computing Machinery Grace Murray Hopper
award. In 1977, Doug Lenat received the IJCAI Computers and Thought award, and in
1978, Ed Feigenbaum received the National Computer Conference Most Outstanding
Technical Contribution award. In 1981, Ted Shortliffe's book Computer-Based Medical
Consultation: MYCIN was identified as the most frequently cited work in the IJCAI-81
proceedings. In 1982, Doug Lenat won the Tioga prize for the best AAAI conference
paper while Mike Genesereth received honorable mention. In 1983, Ted Shortliffe was
named a Kaiser Foundation faculty scholar, and Tom Mitchell received the UCAI
Computers and Thought award. In 1984, Randy Davis and Doug Lenat were named
among the 100 most promising U.S. scientists under 40 by a prestigious scientific panel
assembled by Science Digest. Also in 1984, Ed Feigenbaum was elected a fellow of the
American Association for the Advancement of Science (AAAS), and he and Ted
Shortliffe were elected fellows of the American College of Medical Informatics.

E. H. Shortliffe 208
5P41-RRO00785-12 Stanford Knowledge Systems Laboratory

KSL Research Environment

Funding -- The KSL is supported solely by sponsored research and gift funds. We have
had funding from many sources, including DARPA, NIH/NLM, ONR, NSF, NASA, and
foundations and industry. Of these, DARPA and NIH have been the most substantial
and long-standing sources of support. All, however, have made complementary
contributions to establishing an effective overall research environment that fosters
interchanges at the intellectual and software levels and that provides the necessary
physical computing resources for our work.

Computing Resources -- Under the Symbolic Systems Resources Group, the KSL
develops and operates its own computing resources tailored to the needs of its
individual research projects. Current computing resources are a networked mixture of
mainframe host computers, Lisp workstations, and network utility servers, reflecting the
evolving hardware technology available for AI research. Our host machines include a
DEC 2060 and 2020 running TOPS-20 (these are the core of the national SUMEX
biomedical computing resource) and a VAX 11/780 running UNIX. Our growing
complement of Lisp machines includes more than 25 Xerox 1100's, a Xerox Dorado, a
Symbolics LM-2, eight Symbolics 3600's, and five Hewlett-Packard 9836's. Network
printing, file, gateway, and terminal interface services are provided by dedicated
machines ranging from VAX 11/750's to microprocessor systems. These facilities are
integrated with other computer science resources at Stanford through an extensive
Ethernet and to external resources through the ARPANET and Tymnet. Funding for
these resources comes principally from DARPA and NIH.

209 E. H. Shortliffe
5P41-RR00785-12 AIM Management Committee Membership

Appendix B
AIM Management Committee Membership

Following are the current membership lists of the various SUMEX-AIM management
committees:

AIM Executive Committee:

SHORTLIFFE, Edward H., M.D., Ph.D. (Chairman)
Principal Investigator - SUMEX
Medical Computer Science, TC135
Stanford University Medical Center
Stanford, California 94305
(415) 497-6970

FEIGENBAUM, Edward A., Ph.D.
Co-Principal Investigator - SUMEX
Heuristic Programming Project
Department of Computer Science
701 Welch Road, Building C
Stanford University
Stanford, California 94305
(415) 497-4879

KULIKOWSKI, Casimir, Ph.D.
Department of Computer Science
Rutgers University
New Brunswick, New Jersey 08903
(201) 932-2006

LEDERBERG, Joshua, Ph.D.
President
The Rockefeller University
1230 York Avenue
New York, New York 10021
(212) 570-8080, 570-8000

LINDBERG, Donald A.B., M.D. (Past Adv Grp Chrmn)
Director, National Library of Medicine
8600 Rockville Pike
Bethesda, Maryland 02114
(617) 726-8311

MYERS, Jack D., M.D.
School of Medicine
Scaife Hall, 1291
University of Pittsburgh
Pittsburgh, Pennsylvania 15261
(412) 624-2649

211 E. H. Shortliffe
AIM Management Committee Membership $P41-RR00785-12

AIM Advisory Group:

MYERS, Jack D., M.D. (Chairman)
School of Medicine
Scaife Hall, 1291
University of Pittsburgh
Pittsburgh, Pennsylvania 15261
(412) 624-2649

AMAREL, Saul, Ph.D.
Department of Computer Science
Rutgers University
New Brunswick, New Jersey 08903
(201) 932-3546

COULTER, Charles L., Ph.D. (Exec. Secretary)
Bidg 31, Room 5B41
Biomedical Research Technology Program
National Institutes of Health
Bethesda, Maryland 20205

FEIGENBAUM, Edward A., Ph.D. (Ex-officio)
Co-Principal Investigator - SUMEX
Heuristic Programming Project
Department of Computer Science
701 Welch Road, Building C
Stanford University
Palo Alto, California 94305
(415) 497-4879

KULIKOWSKI]I, Casimir, Ph.D.
Department of Computer Science
Rutgers University
New Brunswick, New Jersey 08903
(201) 932-2006

LEDERBERG, Joshua, Ph.D.
President
The Rockefeller University
1230 York Avenue
New York, New York 10021
(212) 570-8080, 570-8000

LINDBERG, Donald A.B., M.D.
Director, National Library of Medicine
8600 Rockville Pike
Bethesda, Maryland 02114
(617) 726-8311

MINSKY, Marvin, Ph.D.
Artificial Intelligence Laboratory
Massachusetts Institute of Technology
545 Technology Square
Cambridge, Massachusetts 02139
(617) 253-5864

E. H. Shortliffe 212
5P41-RR00785-12 AIM Management Committee Membership

MOHLER, William C., M.D.
Associate Director
Division of Computer Research and Technology
National Institutes of Health
Building 12A, Room 3033
9000 Rockville Pike
Bethesda, Maryland 20205
(301) 496-1168

PAUKER, Stephen G., M.D.
Department of Medicine - Cardiology
Tufts New England Medical Center Hospital
171 Harrison Avenue
Boston, Massachusetts 02111
(617) 956-5910

SHORTLIFFE, Edward H., M.D., Ph.D. (Ex-officio)
Principal Investigator - SUMEX
Medical Computer Science, TC135
Stanford University Medical Center
Stanford, California 94305
(415) 497-6970

SIMON, Herbert A., Ph.D.
Department of Psychology
Baker Hall, 339
Carnegie-Mellon University
Schenley Park
Pittsburgh, Pennsyivania 15213
(412) 578-2787, 578-2000

213 E. H. Shortliffe
AIM Management Committee Membership 5P41-RRO0785-12

Stanford Community Advisory Committee:

FEIGENBAUM, Edward A., Ph.D. (Chairman)
Heuristic Programming Project
Department of Computer Science
Margaret Jacks Hall
Stanford University
Stanford, California 94305
(415) 497-4879

LEVINTHAL, Elliott C., Ph.D.
Departments of Mechanical and Electrical Engineering
Building 530
Stanford University
Stanford, California 94305
(415) 497-9037

SHORTLIFFE, Edward H., M.D., Ph.D.
Principal Investigator - SUMEX
Medical Computer Science, TC135
Stanford University Medical Center
Stanford, California 94305
(415) 497-6970

E. H. Shortliffe 214
$P41-RRO0785-12 Scientific Subproject Abstracts

Appendix C
Scientific Subproject Abstracts

The following are brief abstracts of our collaborative research projects.

215 E. H. Shortliffe
Scientific Subproject Abstracts §P41-RR00785-12

Stanford Project: GUIDON/NEOMYCIN --
KNOWLEDGE ENGINEERING
FOR TEACHING MEDICAL DIAGNOSIS

Principal Investigators: William J. Clancey, Ph.D.
701 Welch Road
Department of Computer Science
Stanford University
Palo Alto, California 94304
(415) 497-1997 (CLANCEY@SUMEX-AIM)

Bruce G. Buchanan, Ph.D.

Computer Science Department

701 Weich Road

Stanford University

Palo Alto, California 94304

(415) 497-0935 (BUCHANAN@SUMEX-AIM)

SOFTWARE AVAILABLE ON SUMEX

GUIDON--A system developed for intelligent computer-aided instruction. Although it
was developed in the context of MYCIN's infectious disease knowledge base, the tutorial
Tules will operate upon any EMYCIN knowledge base.

NEOMYCIN--A consulation system derived from MYCIN, with the knowledge base
greatly extended and reconfigured for use in teaching. In contrast with MYCIN,
diagnostic procedures, common sense facts, and disease hierarchies are factored out of
the basic finding/disease associations. The diagnostic procedures are abstract (not
specific to any problem domain) and model human reasoning, unlike the exhaustive,
top-down approach implicit in MYCIN's medical rules. This knowledge base will be
used in the GUIDON2 family of instructional programs, being developed on D-
machines.

REFERENCES

Clancey, W.J.: Overview of GUIDON. In A. Barr and E.A. Feigenbaum (Eds.),

THE HANDBOOK OF ARTIFICIAL INTELLIGENCE, Vol. 2. William Kaufmann
Assoc., Los Altos, CA, 1982. (Also to appear in J. of Computer-based

Instruction)

Clancey, W.J.: Methodology for building an intelligent tutoring system.

In Kintsch, Polson, and Miller, (Eds.), METHODS AND TACTICS IN COGNITIVE
SCIENCE. 7 Erlbaum Assoc., Hillsdale, NJ. 1984. (Also STAN-CS-81-894,

HPP 81-18

Clancey, W.J.: Acquiring, representing, and evaluating a competence
model of diagnosis. In Chi, Glaser, and Farr (Eds.), THE NATURE
OF EXPERTISE. In preparation. HPP-84-2,

E. H. Shortliffe 216
5P41-RR00785-12 Scientific Subproject Abstracts

Stanford Project: MOLGEN -~ AN EXPERIMENT PLANNING SYSTEM
FOR MOLECULAR GENETICS

Principal Investigators: Edward A. Feigenbaum, Ph.D.
Department of Computer Science
Stanford University

Charles Yanofsky, Ph.D. (YANOFSK Y@SUMEX-AIM)
Department of Biology

Stanford University

Stanford, California 94305

(415) 497-2413

Contact: Dr. Peter FRIEDLAND@SUMEX-AIM
(415) 497-3728

The goal of the MOLGEN Project is to apply the techniques of artificial intelligence to
the domain of molecular biology with the aim of providing assistance to the
experimental scientist. Previous work has focused on the task of experiment design.
Two major approaches to this problem have been explored, one which instantiates
abstracted experimental strategies with specific laboratory tools, and one which creates
plans in toto, heavily influenced by the role played by interactions between plan steps.
As part of the effort to build an experiment design system, a knowledge representation
and acquisition package--the UNITS System, has been constructed. A large knowledge
base, containing information about nucleic acid structures, laboratory techniques, and
experiment-design strategies, has been developed using this tool. Smaller systems, such
as programs which analyze primary sequence data for homologies and symmetries, have
been built when needed.

New work has begun on scientific theory formation, modification, and testing. This
work will be done within the domain of regulatory genetics. We plan to explore
fundamental issues in machine learning and discovery, as well as construct systems that
will assist the laboratory scientist in accomplishing his intellectual goals.

SOFTWARE AVAILABLE ON SUMEX
SPEX system for experiment design.
UNITS system for knowledge representation and acquisition.
SEQ system for nucleotide sequence analysis.
REFERENCES

Friedland, P.E.: Knowledge-based experiment design in molecular genetics,
(Ph.D. thesis). Stanford Computer Science Report, STAN-CS-79-771.

Friedland, P.E.: Knowledge-based experiment design in molecular genetics,
Proc. Sixth IJCAI, Tokyo, August, 1979, pp. 285-287.

217 E. H. Shortliffe
Scientific Subproject Abstracts 5P41-RRO0785-12

Stefik, M.J.: An examination of a frame-structured representation system,
Proc. Sixth IJCAI, Tokyo, August, 1979, pp. 845-852.

Stefik, MJ.: Planning with constraints, (Ph.D. thesis).
Stanford Computer Science Report, STAN-CS-80-784, March, 1980.

E. H. Shortliffe 218
S5P41-RR00785-12 Scientific Subproject Abstracts

Stanford Project: ONCOCIN -- KNOWLEDGE ENGINEERING FOR
ONCOLOGY CHEMOTHERAPY CONSULTATION
Principal Investigator: Edward H. Shortliffe, M.D., Ph.D.

Departments of Medicine and Computer Science
Stanford University Medical Center - Room TC135
Stanford, California 94305

(415) 497-6979 (SHORTLIFFE@SUMEX-AIM)

Project Director: Dr. Lawrence M. Fagan

The ONCOCIN Project is overseen by a collaborative group of physicians and computer
scientists who are developing an intelligent system that uses the techniques of knowledge
engineering to advise oncologists in the management of patients Teceiving cancer
chemotherapy. The general research foci of the group members include knowledge
acquisition, inexact reasoning, explanation, and the representation of time and of expert
thinking patterns. Much of the work developed from research in the 1970's on the
MYCIN and EMYCIN programs, early efforts that helped define the group's research
directions for the coming decade. MYCIN and EMYCIN are still available on SUMEX
for demonstration purposes.

The prototype ONCOCIN system is in limited experimental use by oncologists in the
Stanford Oncology Clinic. Thus much of the emphasis of this research has been on
human engineering so that the physicians will accept the program as a useful adjunct to
their patient care activities. ONCOCIN has generally been well-accepted since its
introduction, and work is underway to transfer the program to professional workstations
(rather than the central SUMEX computer) so that it can be implemented and evaluated
at sites away from the University.

SOFTWARE AVAILABLE ON SUMEX

MYCIN-- A consultation system designed to assist physicians with the selection
of antimicrobial therapy for severe infections. It has achieved expert
level performance in formal evaluations of its ability to select
therapy for bacteremia and meningitis. Although MYCIN is no longer
the subject of an active research program, the system continues to be
available on SUMEX for demonstration purposes and as a testing
environment for other research projects.

EMYCIN-- The “essential MYCIN” system is a generalization of the MYCIN
knowledge representation and control structure. It is designed to
facilitate the development of new expert consultation systems for
both clinical and non-medical domains.

ONCOCIN-- This system is in clinical use but is designed for special high speed
terminals and therefore cannot be tested or demonstrated via network
connections. Much of the knowledge in the domain of cancer
chemotherapy is already well-specified in protocol documents, but
expert judgments also need to be understood and modeled.

219 E. H. Shortliffe
Scientific Subproject Abstracts SP41-RR00785-12

REFERENCES

Shortliffe, E.H., Scott, A.C., Bischoff, M.B., Campbell, A.B., van Melle,
W. and Jacobs, C.D.: ONCOCIN: An expert system for oncology protocol
management. Proc, Seventh IJCAI, pp. 876-881, Vancouver, B.C., August,
1981.

Duda, R.O. and Shortliffe, E.H.: Expert systems research. Science
220:261-268, 1983.

Langlotz, C.P. and Shortliffe, EH.: Adapting a consultation system to
critique user plans. Int. J. Man-Machine Studies 19:479-496, 1983.

Bischoff, M.B., Shortliffe, E.H., Scott, A.C., Carlson, R.W. and Jacobs,
C.D.: Integration of a computer-based consultant into the clinical

setting. Proceedings 7th Annual Symposium on Computer Applications in
Medical Care, pp. 149-152, Baltimore, Maryland, October 1983.

E. H. Shortliffe 220
5P41-RR00785-12 Scientific Subproject Abstracts

Stanford Project: PROTEAN Project

Principal Investigators: Oleg Jardetzky (JARDETZK Y@SUMEX-AIM)
Nuclear Magnetic Resonance Lab, School of Medicine
Stanford University Medical Center
Stanford, California 94305

Bruce Buchanan, Ph.D. (BUCHANAN@SUMEX-AIM)
Computer Science Department

Stanford University

Stanford, California 94305

The goals of this project are related both to biochemistry and artificial intelligence: (a)
use existing AI methods to aid in the determination of the 3-dimensional structure of
proteins in solution (not from x-ray crystallography proteins), and (b) use protein
structure determination as a test problem for experiments with the Al problem solving
structure known as the Blackboard Model. Empirical data from nuclear magnetic
resonance (NMR) and other sources may provide enough constraints on structural
descriptions to allow protein chemists to bypass the laborious methods of crystallizing a
protein and using X-ray crystallography to determine its structure. This problem
exhibits considerable complexity. Yet there is reason to believe that AI programs can
be written that reason much as experts do to resolve these difficulties

REFERENCES

1. Erman, L.D., Hayes-Roth, B., Lesser, V.R., Reddy, D.R:The HEARSAY-II
Speech Understanding System: Integrating Knowledge to Resolve
Uncertainty. ACM Computing Surveys 12(2):213-254, June, 1980.

2. Hayes-Roth, B.: The Blackboard Architecture: A General Framework for
Problem Solving? Report HPP-83-30, Department of Computer Science,
Stanford University, 1983.

3. Hayes-Roth, B: BBI: An Environment for Building Blackboard Systems
that Control, Explain, and Learn about their own Behavior. Report
HPP-84-16, Department of Computer Science, Stanford University, 1984.

4. Hayes-Roth, B.A Blackboard Architecture for Control. Artificial Intelligence
In Press, 1985.

5. Hayes-Roth, B. and Hewett, M.: Learning Control Heuristics in BBI. Report
HPP-85-2, Department of Computer Science, 1985.

6. Jardetzky, O.: A Method for the Definition of the Solution Structure of
Proteins from NMR and Other Physical Measurements: The LAC-~Repressor
Headpiece. Proceedings of the International Conference on the Frontiers of
oy and Molecular Biology, Alma Alta, June 17-24, 1984, October,

4,

221 E. H. Shortliffe
Scientific Subproject Abstracts 5P41-RR00785-12

Stanford Project: RADIX -- DERIVING KNOWLEDGE FROM
TIME-ORIENTED CLINICAL DATABASES

Principal Investigators: Robert L. Blum, M.D.
Departments of Medicine
and Computer Science
Stanford University
Stanford, California 94305
(415) 497-9421 (BLUM@SUMEX-AIM)

Gio C.M. Wiederhold, Ph.D.

Department of Computer Science

Stanford University

Stanford, California 94305

(415) 497-0685 (WIEDERHOLD@SUMEX-AIM)

The objective of clinical database (DB) systems is to derive medical knowledge from the
stored patient observations. However, the process of reliably deriving causal
relationships has proven to be quite difficult because of the complexity of disease states

and time relationships, strong sources of bias, and problems of missing and outlying
ata,

The goal of the RADIX Project is to explore the usefulness of knowledge-based
computational techniques in solving this problem of accurate knowledge inference from
non-randomized, non-protocol patient records. Central to RADIX is a knowledge base
(KB) of medicine and statistics, organized as a taxonomic tree consisting of frames with
attached data and procedures. The KB is used to retrieve time-intervals of interest
from the DB and to assist with the statistical analysis. Derived knowledge is
incorporated automatically into the KB. The American Rheumatism Association DB
containing records of 1700 patients is used.

SOFTWARE AVAILABLE ON SUMEX
RADIX--(excluding the knowledge base and clinical database) consists of approximately

400 INTERLISP functions. The following groups of functions may be of interest apart
from the RADIX environment:

SPSS Interface Package -- Functions which create SPSS source decks and read
SPSS listings from within INTERLISP.

Statistical Tests in INTERLISP -- Translations of the Piezer-Pratt
approximations for the T,F, and Chi-square tests into LISP.

Time-Oriented Data Base and Graphics Package -- Autonomous package for
maintaining a time-oriented database and displaying labelled time-intervals.

E. H. Shortliffe 222
§P41-RR00785-12 Scientific Subproject Abstracts

REFERENCES

Monograph

Blum, R.L.: Discovery and representation of causal relationships
from a large time-oriented clinical database: The RX project.

IN D.A.B. Lindberg and P.L. Reichertz (Eds.), LECTURE NOTES IN
MEDICAL INFORMATICS, Vol. 19, Springer-Verlag, New York, 1982.

Journal Articles

Bium, R.L.: Discovery, confirmation, and incorporation of causal
relationships from a large time-oriented clinical database:

The RX Project. Computers and Biomedical Research 15(2):164-187,
April, 1982.

Blum, R.L.: Displaying clinical data from a time-oriented database.
Computers in Biology and Medicine 11(4):197-210, 1981.

Conference Proceeding

Blum, R.L.: Modeling and encoding clinical causal relationships.
Proc. SCAMC83, IEEE, Baltimore, MD, October, 1983.

223 E. H. Shortliffe
Scientific Subproject Abstracts 5P41-RR00785-12

National AIM Project: CADUCEUS (formerly INTERNIST)

Principal Investigators: Jack D. Myers, M.D. (MYERS@SUMEX-AIM)
Harry E. Pople, Ph.D. (POPLE@SUMEX-AIM)
University of Pittsburgh
Pittsburgh, Pennsylvania 15261
Dr. Pople: (412) 624-3490
Dr. Myers: (412) 624-2649

The major goal of the CADUCEUS Project is to produce a reliable and adequately
complete diagnostic consultative program in the field of internal medicine. Although
this program is intended primarily to aid skilled internists in complicated medical
problems, the program may have spin-off as a diagnostic and triage aid to physicians’
assistants, rural health clinics, military medicine and space travel. In the design of
CADUCEUS and its predecessor INTERNIST I, we have attempted to model the
creative, problem-formulation aspect of the clinical reasoning process. The program
employs a novel heuristic procedure that composes differential diagnoses, dynamically,
on the basis of clinical evidence. During the course of a CADUCEUS or
INTERNIST-1 consultation, it is not uncommon for a number of such conjectured
problem foci to be proposed and investigated, with occasional major shifts taking place
in the program's conceptualization of the task at hand.

SOFTWARE AVAILABLE ON SUMEX
Versions of INTERNIST are available for experimental use, but the project continues to

be oriented primarily towards research and development: hence, a stable production
version of the system is not yet available for general use.

E. H. Shortliffe 224
5P41-RR00785-12 Scientific Subproject Abstracts

National AIM Project: CLIPR -- HIERARCHICAL MODELS
OF HUMAN COGNITION
Principal Investigators: Walter Kintsch, Ph.D. (KINTSCH@SUMEX-AIM)

Peter G. Polson, Ph.D. (POLSON@SUMEX-AIM)
Computer Laboratory for Instruction

in Psychological Research (CLIPR)

Department of Psychology

University of Colorado

Boulder, Colorado 80302

(303) 492-6991

Contact: Dr. Peter G. Polson (Polson@SUMEX-AIM)

The CLIPR Project is concerned with the modeling of complex psychological processes.
It is comprised of two research groups. The prose comprehension group has completed
a project that carries out the text analysis described by van Dijk & Kintsch (1983)
yielding predictions of the recall and readability of that text by human subjects. The
human-computer interaction group is developing a quantitative theory of that predicts
learning, transfer, and performance for a wide range of computer-tasks, e.g. text editing.

SOFTWARE AVAILABLE ON SUMEX

A set of programs has been developed to perform the microstructure text analysis
described in van Dijk & Kintsch (1983) and Kintsch & Greeno (1985). The program
accepts a propositionalized text as input, and produces indices that can be used to
estimate the text’s recall and readability.

REFERENCES

Fletcher, R. C. Understanding and solving word arithmetic
problems: A computer simulation. Technical Report NO. 135, Institute of
Cognitive Science, Colorado, 1984.

Kieras, D.E. and Polson, P.G.: The formal analysis of
user complexity. Int. J. Man-Machine Studies, In Press.

Kintsch, W. and van Dijk, T.A. Toward a model of text
comprehension and production. Psychological Rev. 85:363-394, 1978.

Kintsch, W. and Greeno, J.G.:Understanding and solving word
arithmetic problems. Psychological Review, 1985, 92, 109-129.

Polson, P.G. and Kieras, D.E.: A formal description of users’
knowledge of how to operate a device and user complexity.
6 oe Research Methods, Instrumentation, & Computers, 1984,
» 249-255.

Polson, P.G. and Kieras, D.E.: A quantitative model
of the learning and performance of text editing knowledge.
Proceedings of the CHI 1985 Conference on Human Factors in
Computing. San Francisco, April 1985.

225 E. H. Shortliffe