REFEREE Project 3P41-RR00785-13 REFEREE may therefore determine on its own whether the authors’ data support their conclusions. The interaction between general techniques and specific clinical trials poses the most significant obstacle to REFEREE's success. The automated consultant must capture the ill-defined knowledge that enables the expert biostatistician to reason from theoretical principles to specific applications and interpretations in the field of medicine. In addition, REFEREE must produce, in clear English prose, a comprehensive critique of every clinical trial submitted for analysis. Future versions of REFEREE must synthesize those analyses into rules for an automated medical decision maker. At present, REFEREE enables the clinician to read clinical trials more critically. As such, REFEREE represents only the first step in a larger research plan, the automation of knowledge acquisition. Current work in the restricted domain of clinical trials will, we hope, illustrate general principles in the design of decision makers that gather expertise from written text and multiple knowledge sources. D. Relevant Publications Haggerty, J: REFEREE and RULECRITIC: Two prototypes for assessing the quality of a medical paper. REPORT KSL-84-49. Master's Thesis, Stanford University, May 1984. E. Funding Support REFEREE currently has no funding. The personnel involved donate their time to this project. Il. INTERACTIONS WITH THE SUMEX-AIM RESOURCE A. Medical Collaborations Dr. Brown and Dr. Feldman of the Stanford University School of Medicine are actively involved in the REFEREE project and are the primary domain experts for this project. B. Interactions with other SUMEX-AIM projects C. Critique of Resource Management The SUMEX computer resource has been essential for the work to date, and the SUMEX staff has continued to be very cooperative with the REFEREE project. Il. RESEARCH PLANS A, Goals & Plans REFEREE will consist of three separate but closely related programs: 1. System I will assist in the evaluation of the quality of a single clinical trial. The user will be imagined to be the editor of a journal reviewing a manuscript for publication, but the program will be tested on a variety of readers, including. clinicians, medical scientists, medical and graduate students, and clerical help. 2. System II will assist in the evaluation of the effectiveness of the treatment or intervention examined in a single published clinical trial. The user will be imagined to be a clinician interested in judging the efficacy of the treatment being tested in the trial. 3. System III will assist in the evaluation of the effectiveness of a single treatment examined in a number of published clinical trials. We also intend to test the first two systems against unassisted evaluations by the various categories of readers. The testing will include a formal testing of the programs by comparing the speed and number of flaws found in using the program with similar E. H. Shortliffe 176 5P41-RR00785-13 REFEREE Project Measurements on unassisted reading. In addition there will be a more informal evaluation by questionnaire of the subjective impressions of users of the program, ascertaining the likelihood of routine use and the value of such a program to the user. B. Justification for continued SUMEX use The DEC 2060 of the SUMEX computing resource is currently the only system we have available for developing REFEREE in Interlisp and the EMYCIN framework. C. Need for other computing resources The REFEREE project needs a high performance LISP workstation to assist in the development and execution of the REFEREE programs. We believe that such a machine is important in order to explore the issues of making the programs easy to use through graphics and a good user interface, as well as to make the programs easily exportable. 177 E. H. Shortliffe Pilot AIM Projects 5P41-RR00785-13 IV.D. Pilot AIM Projects Following is a description of the informal pilot project currently using the AIM portion of the SUMEX-AIM resource, pending funding, full review, and authorization. In addition to the progress report presented here, an abstract is submitted on a separate Scientific Subproject Form. E. H. Shortliffe 178 $P41-RR00785-13 PATHFINDER Project IV.D.1. PATHFINDER Project PATHFINDER Project Bharat Nathwani, M.D. Department of Pathology University of Southern California Lawrence M. Fagan, M.D., Ph.D. Department of Medicine Stanford University I. SUMMARY OF RESEARCH PROGRAM A. Project Rationale Our project addresses difficulties in the diagnosis of lymph node pathology. Five studies from cooperative oncology groups have documented that, while experts show agreement with one another, the diagnosis made by practicing pathologists may have to be changed ‘by expert hematopathologists in as many as 50% of the cases. Precise diagnoses are crucial for the determination of optimal treatment. To make the knowledge and diagnostic reasoning capabilities of experts available to the practicing pathologist, we have developed a pilot computer-based diagnostic program called PATHFINDER. The project is a collaborative effort of the University of Southern California and the Stanford University Medical Computer Science Group. A pilot version of the program provides diagnostic advice on 72 common benign and malignant diseases of the lymph node based on 110 histologic features. Our research plans are to develop a full-scale version of the computer program by substantially increasing the quantity and quality of knowledge and to develop techniques for knowledge representation and manipulation appropriate to this application area. The design of the program has been strongly influenced by the INTERNIST/CADUCEUS program developed on the SUMEX resource. A group of expert pathologists from several centers in the U.S. have showed interest in the program and helped to provide the structure of the knowledge base for the PATHFINDER system. B. Medical Relevance and Collaboration One of the most difficult areas in surgical pathology is the microscopic interpretation of lymph node biopsies. Most pathologists have difficulty in accurately classifying lymphomas. Several cooperative oncology group studies have documented that while experts show agreement with one another, the diagnosis rendered by a “locai” pathologist may have to be changed by expert lymph node pathologists (expert hematopathologists) in as many as 50% of the cases. The National Cancer Institute recognized this problem in 1968 and created the Lymphoma Task Force which is now identified as the Repository Center and the Pathology Panel for Lymphoma Clinical Studies. The main function of this expert panel of pathologists is to confirm the diagnosis of the "local" pathologists and to ensure that the pathologic diagnosis is made uniform from one center to another so that the comparative results of clinical therapeutic trials on lymphoma patients are valid. An expert panel approach is only a partial answer to this problem. The panel is useful in only a small percentage (3%) of cases; the Pathology Panel annually reviews 179 E. H. Shortliffe PATHFINDER Project 5P41-RR00785-13 only 1,000 cases whereas more than 30,000 new cases of lymphomas are reported each year. A Panel approach to diagnosis is not practical and lymph node pathology cannot be routinely practiced in this manner. We believe that practicing pathologists do not see enough case material to maintain a high-level of diagnostic accuracy. The disparity between the experience of expert hematopathology teams and those in community hospitals is striking. An experienced hematopathology team may review thousands of cases per year. In contrast, in a community hospital, an average of only ten new cases of malignant lymphomas are diagnosed each year. Even in a university hospital, only approximately 100 new patients are diagnosed every year. Because of the limited numbers of cases seen, pathologists may not be conversant with the differential diagnoses consistent with each of the histologic features of the lymph node; they may lack familiarity with the complete spectrum of the histologic findings associated with a wide range of diseases. In addition, pathologists may be unable to fully comprehend the conflicting concepts and terminology of the different classifications of non-Hodgkin's lymphomas, and may not be cognizant of the significance of the immunologic, cell kinetic, cytogenetic, and immunogenetic data associated with each of the subtypes of the non-Hodgkin's lymphomas. In order to promote the accuracy of the knowledge base development we will have participants for multiple institutions collaborating on the project. Dr. Nathwani will be joined by experts from Stanford (Dr. Dorfman), St. Jude’s Children's Research Center -- Memphis (Dr. Berard) and City of Hope (Dr. Burke). C. Highlights of Research Progress C.l Previous Accomplishments Since the project's inception in September, 1983, we have constructed several versions of PATHFINDER. The first several versions of the program were rule-based systems like MYCIN and ONCOCIN which were developed earlier by the Stanford group. We soon discovered, however, that the large number of overlapping features in diseases of the lymph node would make a rule-based system cumbersome to implement. We next considered the construction of a Aybrid system, consisting of a rule-based algorithm that would pass control to an INTERNIST-like scoring algorithm if it could not confirm the existence of classical sets of features. We finally decided that a modified form of the INTERNIST program would be most appropriate. The original version of PATHFINDER is written in the computer language Maclisp and runs on the SUMEX DEC-20. This was transferred to Portable Standard Lisp (PSL) on the DEC-20, and later transferred to PSL on the HP 9836 workstations. Two graduate students, David Heckerman and Eric Horvitz, designed and implemented the program. The prototype knowledge base was constructed by Dr. Nathwani. During the early part of 1984, we organized two meetings of the entire team, including the pathology experts, to define the selection of diseases to be included in the system, and the choice of features to be used in the scoring process. During the last year, we have focused on methodologies for more accurately representing expert beliefs. In particular, we have used influence diagrams to represent dependencies among features in the PATHFINDER knowledge base. A great deal of effort has been devoted to assessing and representing the intricate relationships among features that exist in the domain. We believe that this process will help to overcome some of the limitations of medical diagnostic systems. C.l The PATHFINDER knowledge base The basic building block of the PATHFINDER knowledge base is the disease profile or E. H. Shortliffe 180 5P41-RR00785-13 PATHFINDER Project frame. Each disease frame consists of features useful for diagnosis of lymph node diseases. Currently these features include histopathologic findings seen in both low- and high-power magnifications. Each feature is associated with a list of exhaustive and mutually exclusive va/ues. For example, the feature pseudofollicularity can take on any one of the values absent, slight, moderate, or prominent. These lists of values give the program access to severity information. In addition, these lists eliminate obvious interdependencies among the values for a given feature. For example, if pseudofollicularity is moderate, it cannot also be absent. Qualitative dependencies among features for each disease are represented using the influence diagram methodology mentioned above. An influence diagram contains nodes and arcs. Nodes represent features and arcs represent dependencies among features. In particular, an arc is drawn from one feature to another when an expert believes that knowing one feature can change his beliefs that another feature will take on its possible values even when the diagnosis is known. Probabilities are used to quantitate the beliefs asserted by the expert. C.2 New Hewlett Packard Workstation Through the USC-affiliated Information Sciences Institute, Dr. Nathwani has obtained a Hewlett-Packard Workstation that is similar to the 9836. The Pathfinder program has been brought up on this machine. This means that the program now exists on three different machines, in three separate locations, using one standard language (Portable Standard Lisp). Thus, the need for support of networked machines and communications has increased during this last year. D. Publications Since January 1984 Horvitz, E.J., Heckerman, D.E., Nathwani, B.N. and Fagan, L.M.: Diagnostic Strategies in the Hypothesis-directed PATHFINDER System, Node Pathology. HPP Memo 84-13. Proceedings of the First Conference on Artificial Intelligence Applications, Denver, Colorado, Dec., 1984. Horvitz, E.J., Heckerman, D.E., Nathwani, B.N. and Fagan, L.M.: The Use of a Heuristic Problem-solving Hierarchy to Facilitate the Explanation of Hypothesis-directed Reasoning. KSL Memo 86-2. Proceedings of MedInfo, Washington D.C., October, 1986. E. Funding Support Research Grant submitted to National Institutes of Health Grant Title: "Computer-aided Diagnosis of Malignant Lymph Node Diseases” Principal Investigator: Bharat Nathwani Funding pending from the National Library of Medicine Professional Staff Association, Los Angeles County Hospital, $10,000. University of Southern California, Comprehensive Cancer Center, $30,000. Project Socrates, Univ. of Southern Calif., Gift from IBM of IBM PC/XT. Hi. INTERACTIONS WITH THE SUMEX-AIM RESOURCE A. Medical Collaborations and Program Dissemination via SUMEX Because our team of experts are in different parts of the country and the computer 181 E. H. Shortliffe PATHFINDER Project 5P41-RR00785-13 scientists are not located at the USC, we envision a tremendous use of SUMEX for communication, demonstration of programs, and remote modification of the knowledge base. The proposal mentioned above was developed using the communication facilities of SUMEX. B. Sharing and Interaction with Other SUMEX-AIM Projects Our project’ depends heavily on the techniques developed by _ the INTERNIST/CADUCEUS project. We have been in electronic contact and have met with members of the [NTERNIST/CADUCEUS project, as well as been able to utilize information and experience with the INTERNIST program gathered over the years through the AIM conferences and on-line interaction. Our experience with the extensive development of the pathology knowledge base utilizing multiple experts should provide for intense and helpful discussions between our two projects. The SUMEX pilot project, RXDX, designed to assist in the diagnosis of psychiatric disorders, is currently using a version of the PATHFINDER program on the DEC-20 for the development of early prototypes of future systems. C. Critique of Resource Management The SUMEX resource has provided an excellent basis for the development of a pilot project. The availability of a pre-existing facility with appropriate computer languages, communication facilities (especially the TYMNET network), and document preparation facilities allowed us to make good progress in a short period of time. The management has been very useful in assisting with our needs during the start of this project. Ill. RESEARCH PLANS A, Project Goals and Plans Collection and refinement of knowledge about lymph node pathology The knowledge base of the program is about to undergo revision by the experts, and then will be extensively tested. A logical next step would be to extend the program to clinical settings, as well as possible extensions of the knowledge base. Other possible extensions include: developing techniques for simplifying the acquisition and verification of knowledge from experts, creating mapping schemes that will facilitate the understanding of the many classifications of non-Hodgkin’s lymphomas. We will also attempt to represent knowledge about special diagnostic entities, such as multiple discordant histologies and atypical proliferations, which do not fit into the classification methods we have utilized. Representation Research We hope to enhance the INTERNIST-1 model by structuring features so that overlapping features are not incorrectly weighted in the decision making process, implementing new methods for scoring hypotheses, and creating appropriate explanation capabilities. B. Requirements for Continued SUMEX Use We are currently dependent on the SUMEX computer for the use of the program by remote users, and for project coordination. We have transferred the program over to Portable Standard Lisp which is used by several users on the SUMEX system. While the switch to workstations has lessened our requirements for computer time for the development of the algorithms, we will continue to need the SUMEX facility for the E. H. Shortliffe 182 5P41-RRO00785-13 PATHFINDER Project interaction with each of the research locations specified in our NIH proposal. The HP equipment is currently unable to allow remote access, and thus the program will have to be maintained on the 2060 for use by all non-Stanford users. C. Requirements for Additional Computing Resources Most of our computing resources will be met by the 2060 plus the use of the HP9836 workstation. We will need additional file space on the 2060 as we quadruple the size of our knowledge base. We will continue to require access to the 2060 for communication purposes, access to other programs, and for file storage and archiving. D. Recommendations for Future Community and Resource Development We encourage the continued exploration by SUMEX of the interconnection of workstations within the mainframe computer setting. We will need to be able to quickly move a program from workstation to workstation, or from workstation back and forth to the mainframe. Software tools that would help the transfer of programs from one type of workstation to another would also be quite useful. Until the type of workstations that we are using in this research becomes inexpensive ($5000 or less), we will continue to need a machine like SUMEX to provide others with a chance to experiment with our software. 183 E. H. Shorttiffe RXDX Project 5P41-RR00785-13 IV.D.2. RXDX Project RXDX Project Robert Lindsay, Ph.D.. Michael Feinberg, M.D., Ph.D. Manfred Kochen, Ph.D. University of Michigan Ann Arbor, Michigan I. SUMMARY OF RESEARCH PROGRAM A, Project Rationale We are developing a prototype expert system that could act as a consultant in the diagnosis and management of depression. Health professionals will interact with the program as they might with a human consultant, describing the patient, receiving advice, and asking the consultant about the rationale for each recommendation. The program uses a knowledge base constructed by encoding the clinical expertise of a skilled psychiatrist in a set of rules and other knowledge structures. It will use this knowledge base to decide on the most likely diagnosis (endogenous or nonendogenous depression), assess the need for hospitalization, and recommend specific somatic treatments when this is indicated (e.g., tricyclic antidepressants). The treatment recommendation will take into account the patient's diagnosis, age, concurrent illnesses, and concurrent treatments (drug interactions). B. Medical Relevance and Collaboration There is a documented shortage of psychiatrists in the US (GMENAC, 1980), and the estimates of the prevalence of psychiatric illness used to develop that report were lower than the figures in recent population surveys (Myers et al. 1984). Further, most prescriptions for antidepressants are written by non-psychiatrists (Johnson, 1974; Kline, 1974) and the great majority of depressed patients seen by a sample of primary care physicians were treated inappropriately (Weissman et al. 1981). These data highlight the need for improving the treatment provided to the majority of mentally ill patients. We believe that computers can act as consultants to non-psychiatrist clinicians, resulting in improved patient care.. The potential benefits to Psychiatry include: making relatively skilled psychiatric consultation widely available in underserved areas, including some public mental health facilities where patients are seen by non-psychiatrists and have relatively little direct patient-physician contact; providing mnon-psychiatrically trained physicians with additional information about psychiatric diagnosis and treatment; avoiding errors of oversight caused by inaccessible patient data; and increased productivity in patient care. Like any good consultant, the program will be able to teach the interested user, and can function as a teaching tool independent of direct clinical application. We examined two other SUMEX-based psychiatry projects, the BLUEBOX project of Mulsant and Servan-Schreiber (1984), and the HEADMED project of Heiser and Brooks (1978, 1980). Mulsant and Servan-Schreiber visited us at Michigan and discussed the rationale and progress of their project. Heiser also visited with us and agreed to collaborate with our project as a consultant. This relationship ended with the termination of our NIMH Smail Grant in Sept. 1985. We have discussed possible E. H. Shortliffe 184 5P41-RR00785-13 RXDX Project collaboration with William Swartout, who plans to expand his explanation system methodology into other medical domains as part of the process of making the method more general. C. Highlights of Research Progress During the past year, we have concentrated on two separate expert systems. The first is an expert on diagnosis, specifically in distinguishing endogenous (i.e, drug-responsive) depression from nonendogenous depression. We are also adding the knowledge needed to separate other subtypes of depression. We are using the Pathfinder program written by Fagan and his co-workers (Horvitz et al., 1984) with their gracious permission. The knowledge base for this program contains data on the prevalence of each illness in the population of interest and the frequency of occurrence of each symptom in each illness. These data are used in a modification of Bayes' rule to reach a diagnosis. The program performs very well (Feinberg and Lindsay, in press). We have been working on SUMEX and are now trying to move this program to a personal computer to make it more widely available. Our major project during the past year has been an expert system for the somatic treatment of (endogenous) depression, where somatic treatment includes antidepressant drugs, electroshock, and lithium. We are writing this system using KEE, an expert system shell generously donated by Intellicorp, running on a Xerox 1108 workstation. We have been able to incorporate the work we did earlier on SUMEX, either directly by transporting the rules or indirectly by using what we learned about building expert systems in general. The knowledge base includes information about the side effects of each of the drugs and about the physiological mechanisms of these side effects. This information allows us to predict drug interactions and the likelihood of occurrence of various side effects in a given patient, and to base explanations on knowledge of the underlying physiology. The knowledge base also includes specific information about drug regimens, about preventing and treating side effects, and about how to take all of this into account in selecting a drug and dosage regimen for the individual patient. D. List of Relevant Publications 1. We have one publication in press: (*) Feinberg, M. and Lindsay, R. K.: Expert systems. Psychopharmacol. Bull., in press. 2. We also presented our work at the World Congress of Biological Psychiatry, Philadelphia, Sept. 1985. : (*) Feinberg, M. and Lindsay, R. K.: Expert systems. 3. We have submitted an abstract to AAAI 1986 and to MedInfo and are awaiting a response. We will be presenting our work in a panel at MedInfo. E. Funding Support We have received support from the Vice-President for Research at the University of Michigan, and from the NIH "Small Grants” Program (Grant Number RO03MH40239-01; Total Direct Costs = $13,850). These funds have enabled us to gather the pilot data for grant applications submitted to NIH. II. INTERACTIONS WITH THE SUMEX-AIM RESOURCE A. Medical Collaboration and Program Dissemination via SUMEX We have established via SUMEX a community of researchers who are interested in Al applications in psychiatry. We also have used the message system to communicate with other Al scientists at SUMEX and elsewhere. As noted above, we have transferred the Pathfinder program from HP workstations to an IBM Model AT via SUMEX. 185 E. H. Shortliffe RXDX Project 5P41-RRO00785-13 B. Sharing and Collaboration with other SUMEX-AIM Projects Our use of EMYCIN and AGE has been of major importance. In addition, we have worked with Dr. Larry Fagan to learn about his Pathfinder program. We used that program, on SUMEX, to obtain some information for the RxDx project by applying it to data we previously collected on depression symptom frequencies. As described above, we are continuing to use the Pathfinder program, both on SUMEX and on a microcomputer. C. Critique of Resource Management We have been using EMYCIN, AGE, and Pathfinder in our work, and have found these programs very valuable, saving us many hours of programming in LISP. There are some problems with the first two, many of which center around discrepancies between the versions described in the manuals and the versions actually running on SUMEX. We would suggest that software be more strongly supported than is now the case, so that it and SUMEX can be even more useful to beginners in AI in Medicine. SUMEX itself has been invaluable. We don't have ready access to any other machine of equal computing power which also has a strongly supported LISP available. Specifically, the LISP compiler available on the Amdahl 5860 here differs from those used at major AI centers such as Stanford and MIT. We have also made good use of the ARPANET connections that SUMEX offers. Feinberg spent a month of his sabbatical working with Prof. Peter Szolovits at MIT, learning about AI in Medicine. This visit was arranged using computer mail through SUMEX. Lindsay and Feinberg were able to continue their collaborative work while the latter was in Cambridge, using the same medium. The alternative would have been days lost in the mails and many dollars spent on phone calls. Most of the limitations of SUMEX, and they are often severe, derive from the necessity to access it via TYMNET. Response time is often impossibly slow, and even at its best the delays are annoying and frustrating, even for editing and debugging. SUMEX has just been made available through UNINET, and this seems much better after about two weeks of use. HI. RESEARCH PLAN A. Project Goals and Plans Our immediate objective is to develop expert systems that can differentiate patients with the various subtypes of depressive disorder, and prescribe appropriate treatment. This system should perform at about the level of a board-certified psychiatrist, i.e. better than an average resident but not as well as a human expert in depression. Eventually, we plan to enlarge the knowledge base so that the expert system can diagnose and prescribe for a wider range of psychiatric patients, particularly those with illnesses that are likely to respond to psychopharmacological agents. We will design the system so that it could be used by non-medical clinicians or by non-psychiatrist MD's as an adjunct to consultation with a human expert. We plan also to focus on problems of the user interface and the integration of this system with other databases. B. Justification and Requirements for continued SUMEX use The access to SUMEX resources is essentially our sole means of maintaining contact with the community of researchers working on applications of AI in medicine. Although we are moving our system to local workstations, the communications capability of SUMEX will continue to be important. We anticipate that our requirements for computing time and file space will continue at about the same level for the next year. E. H. Shortliffe 186 5P41-RR00785-13 RXDX Project C. Needs and Plans for Other Computing Resources AS our project evolves and we run into the limitations of the time-shared SUMEX facility, we anticipate employing different expert systems software. We are now using KEE and Golden Common LISP, the former on a Xerox 1108; the latter on an IBM Model AT. Ultimately, when our consultant is made available for field trials and clinical use, it will need to be transported to a personal computer that is large enough to support the system yet inexpensive enough to be widely available. Both KEE and common LISP will now run on microcomputers, the former requiring a more powerful central machine as well. The cost of the hardware needed to run our systems has dropped by half or more in one year, and is continuing to drop. Our design strategy is to avoid limiting ourselves and our aspirations to that which is affordable today; instead we will attempt to project the growth of our project and the price-performance curve of computing such that they meet at some reasonable point in the future. D. Recommendations for Future Community and Resource Development Valuable as the present SUMEX facilities are to us, they are in many ways limited and awkward to use. The major limitation we feel is the difficulty and sometimes the impossibility of making contact with everyone who could be of value to us. We hope that greater emphasis will be put on internetwork gateways. It is important not only to establish more of these, but to develop consistent and convenient standards for electronic mail, electronic file transfers, graphic information transfer, national archives and data bases, and personal filing and retrieval (categorization) systems. The present state of the art feels quite limiting, now that the basic concepts of computer networking have become available and have proved their potential. We expect that the role of the SUMEX-AIM resource will continue to evolve in the direction of increased importance of communication, including graphical information, electronic dissemination of preprints, and database and program access. The need for computer cycles on a large mainframe will diminish. We hope to have continued access to the system for communication, but do not anticipate continued use of it as a LISP computation server beyond the next year or eighteen months. If fees for using SUMEX resources were imposed, this would have a drastically limiting effect on the value of the system to us. Even if we had a budget to purchase such services, the inhibiting effect of having a meter running would cause us to make less use of it that we should. We have been conscious of the costs of the system and feel that we have not used it imprudently, even though we have not directly borne its costs. 187 E. H. Shortliffe 5P41-RR00785-13 Following are the current membership lists of the various SUMEX-AIM management committees: Appendix A AIM Management Committee Membership AIM Executive Committee: SHORTLIFFE, Edward H., M.D., Ph.D. (Chairman) Principal Investigator - SUMEX Medical Computer Science, TC135 Stanford University Medical Center Stanford, California 94305 (415) 497-6970 FEIGENBAUM, Edward A., Ph.D. Co-Principal Investigator - SUMEX Heuristic Programming Project Department of Computer Science 701 Welch Road, Building C Stanford University Stanford, California 94305 (415) 497-4879 KULIKOWSKI, Casimir, Ph.D. Department of Computer Science Rutgers University New Brunswick, New Jersey 08903 (201) 932-2006 LEDERBERG, Joshua, Ph.D. President The Rockefeller University 1230 York Avenue New York, New York 10021 (212) 570-8080, 570-8000 LINDBERG, Donald A.B., M.D. (Past Adv Grp Chrmn) Director, National Library of Medicine 8600 Rockville Pike Bethesda, Maryland 02114 (617) 726-8311 MYERS, Jack D., M.D. School of Medicine Scaife Hall, 1291 University of Pittsburgh Pittsburgh, Pennsylvania 15261 (412) 624-2649 189 E. H. Shortliffe AIM Management Committee Membership AIM Management Committee Membership 5P41-RR00785-13 AIM Advisory Group: MYERS, Jack D., M.D. (Chairman) School of Medicine Scaife Hall, 1291 University of Pittsburgh Pittsburgh, Pennsylvania 15261 (412) 624-2649 AMAREL, Saul, Ph.D. Department of Computer Science Rutgers University New Brunswick, New Jersey 08903 (201) 932-3546 COULTER, Charles L., Ph.D. (Exec. Secretary) Bldg 31, Room 5B41 Biomedical Research Technology Program National Institutes of Health Bethesda, Maryland 20205 FEIGENBAUM, Edward A., Ph.D. (Ex-officio) Co-Principal Investigator - SUMEX Heuristic Programming Project Department of Computer Science 701 Welch Road, Building C Stanford University Palo Alto, California 94305 (415) 497-4879 KULIKOWSKI, Casimir, Ph.D. Department of Computer Science Rutgers University New Brunswick, New Jersey 08903 (201) 932-2006 LEDERBERG, Joshua, Ph.D. President The Rockefeller University 1230 York Avenue New York, New York 10021 (212) 570-8080, 570-8000 LINDBERG, Donald A.B., M.D. Director, National Library of Medicine 8600 Rockville Pike Bethesda, Maryland 02114 (617) 726-8311 MINSKY, Marvin, Ph.D. Artificial Intelligence Laboratory Massachusetts Institute of Technology 545 Technology Square Cambridge, Massachusetts 02139 (617) 253-5864 E. H. Shortliffe 190 5P41-RR00785-13 AIM Management Committee Membership MOHLER, William C., M.D. Associate Director Division of Computer Research and Technology National Institutes of Health Building 12A, Room 3033 9000 Rockville Pike Bethesda, Maryland 20205 (301) 496-1168 PAUKER, Stephen G., M.D. Department of Medicine - Cardiology Tufts New England Medical Center Hospital 171 Harrison Avenue Boston, Massachusetts 02111 (617) 956-5910 SHORTLIFFE, Edward H., M.D., Ph.D. (Ex-officio) Principal Investigator - SUMEX Medical Computer Science, TC135 Stanford University Medical Center Stanford, California 94305 (415) 497-6970 SIMON, Herbert A., Ph.D. Department of Psychology Baker Hall, 339 Carnegie-Mellon University Schenley Park Pittsburgh, Pennsylvania 15213 (412) 578-2787, 578-2000 191 E. H. Shortliffe AIM Management Committee Membership 5P41-RR00785-13 Stanford Community Advisory Committee: FEIGENBAUM, Edward A., Ph.D. (Chairman) Heuristic Programming Project Department of Computer Science Margaret Jacks Hall Stanford University Stanford, California 94305 (415) 497-4879 LEVINTHAL, Elliott C., Ph.D. Departments of Mechanical and Electrical Engineering Building 530 Stanford University Stanford, California 94305 (415) 497-9037 SHORTLIFFE, Edward H., M.D., Ph.D. E. H. Shortliffe Principal Investigator - SUMEX Medical Computer Science, TC135 Stanford University Medical Center Stanford, California 94305 (415) 497-6970 192 5P41-RR00785-13 Scientific Subproject Abstracts Appendix B Scientific Subproject Abstracts The following are brief abstracts of our collaborative research projects. 193 E. H. Shortliffe Scientific Subproject Abstracts 5P41-RR00785-13 Stanford Project: GUIDON/NEOMYCIN -- KNOWLEDGE ENGINEERING FOR TEACHING MEDICAL DIAGNOSIS Principal Investigators: William J. Clancey, Ph.D. 701 Welch Road Department of Computer Science Stanford University Palo Alto, California 94304 (415) 497-1997 (CLANCE Y@SUMEX-AIM) Bruce G. Buchanan, Ph.D. Computer Science Department 701 Welch Road Stanford University Palo Alto, California 94304 (415) 497-0935 (BUCHANAN@SUMEX-AIM) SOFTWARE AVAILABLE ON SUMEX GUIDON--A system developed for intelligent computer-aided instruction. Although it was developed in the context of MYCIN's infectious disease knowledge base, the tutorial rules will operate upon any EMYCIN knowledge base. NEOMYCIN--A consulation system derived from MYCIN, with the knowledge base greatly extended and reconfigured for use in teaching. In contrast with MYCIN, diagnostic procedures, common sense facts, and disease hierarchies are factored out of the basic finding/disease associations. The diagnostic procedures are abstract (not specific to any problem domain) and model human reasoning, unlike the exhaustive, top-down approach implicit in MYCIN's medical rules. This knowledge base will be used in the GUIDON2 family of instructional programs, being developed on D- machines. REFERENCES Clancey, W.J.: Overview of GUIDON. In A. Barr and E.A. Feigenbaum (Eds.), THE HANDBOOK OF ARTIFICIAL INTELLIGENCE, Vol. 2. William Kaufmann Assoc., Los Altos, CA, 1982. (Also to appear in Journal of Computer-based Instruction) Clancey, W.J.: Methodology for building an intelligent tutoring system. In Kintsch, Polson, and Miller, (Eds.), METHODS AND TACTICS IN COGNITIVE SCIENCE. L. Erlbaum Assoc., Hillsdale, NJ. 1984. (Also STAN-CS-81-894, HPP 81-18) Clancey, W.J.: Acquiring, representing, and evaluating a competence model of diagnosis. In Chi, Glaser, and Farr (Eds.), THE NATURE OF EXPERTISE. In preparation. HPP-84-2. E. H. Shortliffe 194 §P41-RR00785-13 Scientific Subproject Abstracts Stanford Project: MOLGEN -- AN EXPERIMENT PLANNING SYSTEM FOR MOLECULAR GENETICS Principal Investigators: Edward A. Feigenbaum, Ph.D. Department of Computer Science Stanford University Charles Yanofsky, Ph.D. (YANOFSK Y@SUMEX-AIM) Department of Biology Stanford University Stanford, California 94305 (415) 497-2413 Contact: Dr. Peter FRIEDLAND@SUMEX-AIM (415) 497-3728 The goal of the MOLGEN Project is to apply the techniques of artificial intelligence to the domain of molecular biology with the aim of providing assistance to the experimental scientist. Previous work has focused on the task of experiment design. Two major approaches to this problem have been explored, one which instantiates abstracted experimental strategies with specific laboratory tools, and one which creates plans in toto, heavily influenced by the role played by interactions between plan steps. As part of the effort to build an experiment design system, a knowledge representation and acquisition package--the UNITS System, has been constructed. A large knowledge base, containing information about nucleic acid structures, laboratory techniques, and experiment-design strategies, has been developed using this tool. Smaller systems, such as programs which analyze primary sequence data for homologies and symmetries, have been built when needed. New work has begun on scientific theory formation, modification, and testing. This work will be done within the domain of regulatory genetics. We plan to explore fundamental issues in machine learning and discovery, as well as construct systems that will assist the laboratory scientist in accomplishing his intellectual goals. SOFTWARE AVAILABLE ON SUMEX SPEX system for experiment design. UNITS system for knowledge representation and acquisition. SEQ system for nucleotide sequence analysis. REFERENCES Friedland, P.E.: Knowledge-based experiment design in molecular genetics, (Ph.D. thesis). Stanford Computer Science Report, STAN-CS-79-771. Friedland, P.E. and Iwasaki, Y.: The concept and implementation of skeletal plans, Journal of Automated Reasoning, 1(2):161-208, 1985. Friedland, P.E. and Kedes, L.: Discovering the secrets of DNA, Communications of the ACM, 28(11):1164-1186, November, 1985. Stefik, MJ.: An examination of a frame-structured representation system, Proc. Sixth IJCAI, Tokyo, August, 1979, pp. 845-852. Stefik, M.J.: Planning with constraints, (Ph.D. thesis). Stanford Computer Science Report, STAN-CS-80-784, March, 1980. 195 E. H. Shortliffe Scientific Subproject Abstracts 5P41-RRO0785-13 Stanford Project: ONCOCIN -- KNOWLEDGE ENGINEERING FOR ONCOLOGY CHEMOTHERAPY CONSULTATION Principal Investigator: Edward H. Shortliffe, M.D., Ph.D. Departments of Medicine and Computer Science Stanford University Medical Center Medical School Office Building Stanford, California 94305 (415) 723-6979 (SHORTLIFFE@SUMEX-AIM) Project Director: Dr. Lawrence M. Fagan The ONCOCIN Project is overseen by a collaborative group of physicians and computer scientists who are developing an intelligent system that uses the techniques of knowledge engineering to advise oncologists in the management of patients receiving cancer chemotherapy. The general research foci of the group members include knowledge acquisition, inexact reasoning, explanation, and the representation of time and of expert thinking patterns. Much of the work developed from research in the 1970's on the MYCIN and EMYCIN programs, early efforts that helped define the group's research directions for the coming decade. MYCIN and EMYCIN are still available on SUMEX for demonstration purposes. The prototype ONCOCIN system is in limited experimental use by oncologists in the Stanford Oncology Clinic. Thus much of the emphasis of this research has been on human engineering so that the physicians will accept the program as a useful adjunct to- their. patient care activities. ONCOCIN has generally been well-accepted since its introduction, and we are now testing a version of the program which runs on professional workstations (rather than the central SUMEX computer) so that it can be implemented and evaluated at sites away from the University. SOFTWARE AVAILABLE ON SUMEX MYCIN-- A consultation system designed to assist physicians with the selection of antimicrobial therapy for severe infections. It has achieved expert level performance in formal evaluations of its ability to select therapy for bacteremia and meningitis. Although MYCIN is no longer the subject of an active research program, the system continues to be available on SUMEX for demonstration purposes and as a testing environment for other research projects. EMYCIN-- The “essential MYCIN” system is a generalization of the MYCIN knowledge representation and control structure. It is designed to facilitate the development of new expert consultation systems for both clinical and non-medical domains. ONCOCIN-- This system is in clinical use but requires lisp machines to be run. Much of the knowledge in the domain of cancer chemotherapy is already well-specified in protocol documents, but expert judgments also need to be understood and modeled. E. H. Shortliffe 196 5P41-RR00785-13 Scientific Subproject Abstracts REFERENCES Shortliffe, E.H., Scott, A.C., Bischoff, M.B., Campbell, A.B., van Melle, W. and Jacobs, C.D.: ONCOCIN: An expert system for oncology protocol management. Proc. Seventh IJCAI, pp. 876-881, Vancouver, B.C., August, 1981. Duda, R.O. and Shortliffe, E.H.: Expert systems research. Science 220:261-268, 1983. Langlotz, C.P. and Shortliffe, E.H.: Adapting a consultation system to critique user plans. Int. J. Man-Machine Studies 19:479-496, 1983. Bischoff, M.B., Shortliffe, E.H., Scott, A.C., Carlson, R.W. and Jacobs, C.D.: Integration of a computer-based consultant into the clinical setting. Proceedings 7th Annual Symposium on Computer Applications in Medical Care, pp. 149-152, Baltimore, Maryland, October 1983. Hickam, D.H., Shortliffe, E.H., Bischoff, M.B., Scott, A.C., Jacobs, C.D. A study of the treatment advice of a computer-based cancer chemotherapy protocol advisor (Memo KSL-85-21). Annals of Internal Medicine 103(6 pt 1):928-936 (1985). Kent, D.L., Shortliffe, E.H., Carlson, R.W., Bischoff, M.B., Jacobs, C.D. Improvements in data collection through physician use of a computer-based chemotherapy treatment consultant (Memo KSL-85-22). Journal of Clinical Oncology 3:1409-1417 (1985). Tsuji, S. and Shortliffe, E.H. Graphical access to a medical expert system: I. Design of a knowledge engineer's interface (Memo KSL-85-11). Meth. Inf. Med., Vol. 25, April 1986. Lane, C.D., Differding, J.C., Shortliffe, E.H. Graphical access to a medical expert system: II. Design of an interface for physicians (Memo KSL-85-15). To appear in Meth. Inf. Med., July 1986. Langlotz, C., Fagan, L., Tu, S., Williams, J., Sikic, B. ONYX: An architecture for planning in uncertain environments (Memo KSL-85-10). Proceedings of the Ninth International Joint Conference on Artificial Intelligence, pp.447-449, Los Angeles, CA, August 1985. 197 E. H. Shortliffe Scientific Subproject Abstracts 5P41-RR00785-13 Stanford Project: PROTEAN Project Principal Investigators: Oleg Jardetzky (JARDETZK Y @SUMEX-AIM.STANFORD.EDU) Nuclear Magnetic Resonance Lab, School of Medicine Stanford University Medical Center Stanford, California 94305 Bruce G. Buchanan, Ph.D. (BUCHANAN@SUMEX-AIM.STANFORD.EDU) Computer Science Department Stanford University Stanford, California 94305 Contact Person: Bruce G. Buchanan The goals of this project are related both to biochemistry and artificial intelligence: (a) use existing Al methods to aid in the determination of the 3-dimensional structure of proteins in solution (not from x-ray crystallography proteins), and (b) use protein structure determination as a test problem for experiments with the AI problem solving structure known as the Blackboard Model. Empirical data from nuclear magnetic resonance (NMR) and other sources may provide enough constraints on structural descriptions to allow protein chemists to bypass the laborious methods of crystallizing a. protein and using X-ray crystallography to determine its structure. This problem exhibits considerable complexity; yet there is reason to believe that AI programs can be written that reason much as experts do to resolve these difficulties. A prototype knowledge-based system has been developed that assembles major secondary structures of a protein into families of structures compatible with a given set of distance constraints. REFERENCES 1. Brinkley, J., Cornelius, C., Altman, R., Hayes-Roth, B. Lichtarge, O., Duncan, B., Buchanan, B.G., Jardetzky, O. Application of Constraint Satisfaction Techniques to the Determination of Protein Tertiary Structure. Report KSL-86-28 2. Buchanan, B.G., Hayes-Roth, B., Lichtarge, O., Altman, A., Brinkley, J., Hewett, M., Cornelius, C., Duncan, B., Jardetzky, O.:.The Heuristic Refinement Method for Deriving Solution Structures of Proteins. Report KSL-85-41. October 1985. In review for Proceedings of the National Academy of Sciences. 3. Hayes-Roth, B.: The Blackboard Architecture: A General Framework for Problem Solving? Report HPP-83-30, Department of Computer Science, Stanford University, 1983. 4. Hayes-Roth, B: BB/: An Environment for Building Blackboard Systems that Control, Explain, and Learn about their own Behavior. Report HPP-84-16, Department of Computer Science, Stanford University, 1984. 5. Hayes-Roth, B.: A Blackboard Architecture for Control. Artificial Intelligence 26:251-321, 1985. E. H. Shortliffe 198 5P41-RR00785-13 Scientific Subproject Abstracts 6. Hayes-Roth, B. and Hewett, M.: Learning Control Heuristics in BB/. Report HPP-85-2, Department of Computer Science, 1985. 7. Jardetzky, O.: A Method for the Definition of the Solution Structure of Proteins from NMR and Other Physical Measurements: The LAC-Repressor Headpiece. Proceedings of the International Conference on the Frontiers of Biochemistry and Molecular Biology, Alma Alta, June 17-24, 1984, October, 1984. 8. Hayes-Roth, B., Buchanan, B.G., Lichtarge, O., Hewett, M., Altman, R., Brinkley, J., Cornelius, C., Duncan, B., and Jardetzky, O.:. PROTEAN: Deriving protein structure from constraints. To appear in Proceedings of the AAAIT, 1986. 199 E. H. Shortliffe Scientific Subproject Abstracts 5P41-RR00785-13 Stanford Project: RADIX -- DERIVING KNOWLEDGE FROM TIME-ORIENTED CLINICAL DATABASES Principal Investigators: Robert L. Blum, M.D. Departments of Medicine and Computer Science Stanford University Stanford, California 94305 (415) 497-9421 (BLUM@SUMEX-AIM) Gio C.M. Wiederhold, Ph.D. Department of Computer Science Stanford University Stanford, California 94305 (415) 497-0685 (WIEDERHOLD@SUMEX-AIM) The objective of clinical database (DB) systems is to derive medical knowledge from the stored patient observations. However, the process of reliably deriving causal relationships has proven to be quite difficult because of the complexity of disease states and time relationships, strong sources of bias, and problems of missing and outlying data. The first goal of the RADIX Project is to explore the usefulness of knowledge-based computational techniques in solving this problem of accurate knowledge inference from -non-randomized, non-protocol patient records. Central to RADIX is a knowledge base (KB) of medicine and statistics, organized as a taxonomic tree consisting of frames with attached data and procedures. The KB is used to retrieve time-intervals of interest from the DB and to assist with the statistical analysis. Derived knowledge is incorporated automatically into the KB. The American Rheumatism Association DB containing records of 1700 patients is used. The second goal of the project is to develop a program and set of techniques for automated summarization of patient records. The summarization program is designed to automatically create patient summaries of arbitrary and appropriate complexity as an aid for tasks such as clinical decision making, real-time patient monitoring, surveillance of quality of care, and eventually automated discovery. A prototype summarization module has been implemented in KEE on the Xerox 1108 workstation. SOFTWARE AVAILABLE ON SUMEX RADIX--(excluding the knowledge base and clinical database) consists of approximately 400 INTERLISP functions. The following groups of functions may be of interest apart from the RADIX environment: SPSS Interface Package -- Functions which create SPSS source decks and read SPSS listings from within INTERLISP. Statistical Tests in INTERLISP --_ Translations of the Piezer-Pratt approximations for the T, F, and Chi-square tests into LISP. Time-Oriented Data Base and Graphics Package -- Autonomous package for maintaining a time-oriented database and displaying labelled time-intervals. E. H. Shortliffe 200