PB89-133722 SURVEY OF INDIVIDUAL USERS OF MEDLINE U. S. DEPARTMENT OF HEALTH AND HUMAN SERVICES National Institutes of Health National Library of Medicine Bethesda, Maryland November 1988 U.S. DEPARTMENT OF COMMERCE National Technical Information Service 49^.f,M3e) 30377-101 REPORT DOCUMENTATION 1 »• report no. 1 2. PAQE NLM-MED-88-04 3- Recipient's. Accession No. PB89-133722 4. Title and Subtitle ^SURVEY OF INDIVIDUAL USERS OF MEDLINE ON THE NLM SYSTEM 5. Report Date 30 November 1988 6. 7. Authors) Karen T. Wallineford. Nancv E. Selineer. Betsv L. HumDhrevs. 8. Performing Organization Rept. No. 9. Performing Organization Nam* and Address Elliot R Si es?el Division of Library Operations National Library of Medicine Bethesda, MD 20894 10. Proiect/Tssk/Work Unit No. 11. Contract(C) or Grant(G) No. (G) 12. Sponsoring Organization Nam* and Address same as 9 above 13. Typo of Report & Period Covered 14. IS. Supplementary Notas 14. Abstract (Limit: 200 words) In order to provide an online information retrieval service that individuals can use easily and effectively, NLM needed to obtain detailed information on the growing and important end-user population, their information needs, and their level of satisfaction with MEDLINE as available on the NLM system. This survey was an initial attempt to determine the demographic characteristics of individual users of MEDLINE, their methods of accessing the system, their reasons for searching, and their level of satisfaction Witt current system features and capabilities. The information gathered will be used to identify system improvements that can provide a better level of information service to U.S. health professionals. 17. Document Analysis a. Descriptors Information Retrieval b. Identifiers/Open-Ended Terms MEDLINE MEDLARS REPRODUCED BY National Library of Medicine ,K,cIl£^i.IECHNICAl information Services INF?s^tJcLPoFNco«SWfRR«V,CE c COSAT1 Field/Group SPRINCflElO. VA. 22161 Ik. Availability Statement Release unlimited 19. Security Class (This Report) UNCLASSIFIED 21. No. of Pages 20. Security Class (This Page) UNCLASSIFIED 22. Pries (See ANSI-Z39.il) See Instructions on Reverse OPTIONAL FORM 272 (4-77) (Formerly NTIS-3S) Department of Commerce SURVEY OF INDIVIDUAL USERS OF MEDLINE ON THE NLM SYSTEM Report prepared by: Karen T.Wallingford Nancy E. Selinger Betsy L. Humphreys Elliot R. Siegel 30 November 1988 CONTENTS Executive summary. Introduction_____________________________________. 1 Purpose of the survey_____________________________1 Methodology____________________________________2 Questionnaire development__________________2 Population surveyed 2 Data collection procedures_______;____________3 Questionnaire return rates___________________3 Data analysis procedures____________________4 Findings_________________________________________4 Demographic characteristics of survey respondents________________________4 General searching behavior__________________6 Characteristics of searches^______________9 Opinions about the system__________________13 Access and training________________________14 Comments on the system___________________17 Conclusions_____________________________________18 Key Points______________________________________23 Tables_________________________________________25 EXECUTIVE SUMMARY INTRODUCTION The National Library of Medicine's basic mission is to assist the progress of medicine and public health by improving the dissemination of biomedical information. In support of this goal, NLM developed MEDLARS, the automated Medical Literature Analysis and Retrieval System, and was a pioneer in offering online retrieval services over nationwide telecommunications networks. For the first 20 years of the system's existence, the principal users of MEDLARS were medical librarians who acted as search intermediaries for researchers and Eractitioners who needed to locate pertinent information in the health sciences terature. As more and more of these health professionals began to have access to personal computers, NLM took several steps to encourage them to search NLM's databases directly, including the provision of special short training courses in online searching and the development of GRATEFUL MED, a user-friendly interface to the NLM system. In the past three years the number of individual health professionals searching MEDLINE directly on the NLM system has increased dramatically. It is continuing to grow at the rate of over 200 users per month. In order to provide a retrieval service that individual health professionals can use easily and effectively, NLM needs to obtain more detailed information about this growing and important end-user population, their information needs, and their level of satisfaction with MEDLINE as available on the NLM system. This survey was an initial attempt to determine the demographic characteristics of individual users of MEDLINE, their methods of access to the system, their reasons for searching, and their level of satisfaction with current system features and capabilities. The information gathered will be used to identify system improvements that can provide a better levelof information service to U.S. health professionals. METHODOLOGY An NLM study team developed a questionnaire to elicit the types of information desired with assistance from Market Dynamics, Inc., a contractor hired to assist in tabulating the survey results. After a pre-test involving a small number of users, the questionnaire was revised and submitted for review by the U.S. Office of Management and Budget as required by the Paperwork Reduction Act. The final version of the questionnaire appears as Attachment 1. The survey population included all who were presumed to be individual "end-users" of MEDLINE on the NLM system as of the end of July 1987. Questionnaires were mailed to these 4,311 individuals on October 2, 1987. There were two follow-up mailings, and returns were accepted until December 10,1987. The gross return rate was 70.1% (2,970 responses); the usable return rate was 68% (2,716 responses). Since the entire universe of individual MEDLINE users was surveyed, sampling error does not affect the results. Because the return rate was high, non-response bias is negligible. We believe the survey findings to be representative of all individual users of MEDLINE as of July 1987. i DEMOGRAPHIC CHARACTERISTICS OF SURVEY RESPONDENTS Professions Nearly two-thirds (65.5%) of the respondents (2,710 individuals, 99.8% of the entire respondent group) identified themselves as physicians and more than a quarter (27.7%) as scientists. 286 (10.5%) said they were both physicians and scientists. More than 40 different medical specialties were listed by the 2,197 respondents (80.9% of the entire respondent group) who provided specialty information. Education Most respondents hold either MX). (1,760) or Ph.D. (570) degrees. The majority of the respondents (2,667 individuals, 98.2% of the entire respondent group) received their highest degree within the last 18 years: 40.1% during the 1970's and 27.1% in the 198(ys. Primary Work Place Nearly all of the individuals surveyed (2,701 individuals, 99.4% of the entire respondent group) indicated a primary work place. Almost half of the respondents (4&8%) work in an academic setting (college, university, or medical school). The remaining respondents are split rather evenly among private/solo practice (12.6%), group practice (123%) and hospital/clinic settings (14.6%). Only 5.4% work in a government agency and 33% in a private company or business. GENERAL SEARCHING BEHAVIOR Frequency of Searching More than two-thirds (68.6%) of the respondents (2,686 individuals, 98.9% of the entire respondent group) do all their own searches. Half of these perform two or fewer searches in an average month. For the respondents who perform at least one search in the typical month, the average number of searches performed is 43 searches per month. About one-third of the respondents have someone else search for them at least occasionally. These respondents request an average of 2.5 searches from others each month. Searches Performed by Others A substantial majority (81.9%) of the searches respondents request from others are done by librarian/information specialists. 42.4% of the entire respondent group (1,151 individuals) indicated the reasons they occasionally have someone else search for them. The most frequently indicated reasons were: lack of time to search personally (593% of those who gave any reasons); the need for different expertise or system knowledge (38.7%); and lack of satisfaction with their own results for a particular search (27.5%). 70.6% of those who have others search for them (1,121 individuals, 413% of the entire respondent group) expressed satisfaction with the results, 8.9% are dissatisfied and 20.5% are neutral. ii Searches Performed by Self Of the 2,171 individuals (79.9% of the entire respondent group) who indicated one or more factors that influenced them to do their own searches, 83.1% selected greater familiarity with the subject matter, 82.4% marked the ability to get results Faster, and 65.5% enjoyment of searching. Level of Experience/Pattern of Use Less than 10% of the respondents (2^61 individuals, 943% of the entire respondent group) regard themselves as very experienced users of online databases. The large majority consider themselves to be somewhat experienced or not very experienced. Less than 8% indicated that they were not at all experienced. Of the 2^09 respondents (92.4% of the entire respondent group) who indicated how long they had been searching MEDLINE with their own codes, nearly two-thirds had had their codes for a year or less at the time of the survey. 27% had had their codes for one to two years. Only 10% had had their codes for more than 2 years. A substantial majority (82%) of the respondents (2,661 individuals, 98.0% of the entire respondent group) do not share their code with anyone else. More than half of the respondents (2,545 individuals, 93.7% of the entire respondent group) said their level of usage of the system had remained relatively constant since they received their own access codes. 30.0% indicated increased use and 123% said their use had decreased. Cost Considerations 67.6% of the 2^54 respondents (94.0% of the entire respondent group) who provided an answer to the question on cost considerations indicated that cost rarely (26.6%) or never (41.0%) keeps them from performing a search. 23.1% said that costs considerations occasionally keep them from searching. Only 9.2% reported that cost considerations frequently keep them from searching. Cost is even less a consideration for those in group practice, private/solo practice, and private business than for respondents as a whole. CHARACTERISTICS OF MEDLINE SEARCHES General Search Purpose Of the 2^56 individuals (94.1% of the respondent group) who provided an answer, 81.7% indicated that they typically search MEDLINE to satisfy an immediate information need; 10% typically search for information to stay current in their field. Only 5% typically search to learn about new areas. Numbers of Citations Desired/Retrieved Overall, more than half of the respondents (2,550 individuals, 93.9% of the entire respondent group) typically wish to retrieve all relevant citations from a particular time period rather than just a few relevant citations. The results vary substantially according to the respondents' work place and their most common search purpose, however. 65.2% of those who work in academic settings and 67.1% of those who iii commonly use search information for research/testing purposes are interested in retrieving all relevant citations as opposed to 47.6% of those in group practice and 49.8% of those who most commonly use search information for patient care. More than half of the respondents (2,510 individuals, 92.4% of the'entire respondent group) typically retrieve about the right number of citations. Overall, 15% of the respondents indicated that they typically retrieve too few citations and 23.6% indicated that they typically retrieve too many. A higher percentage (23.6%) of those who always search using GRATEFUL MED form screens retrieve too few citations compared to individuals who always search using command language (102%). 2^399 individuals (88.3% of the entire respondent group) indicated the percentage of retrieved citations that they typically judge to be relevant to their inquiry. 58.6% of these respondents said that typically fewer than half of the citations they retrieve are relevant Use of Information Retrieved Over 90% of the respondents (2j552 individuals, 94.0% of the entire respondent foup) typically search for a subject rather than for an author or a journal title. espondents were asked to rank the three primary areas in which they use MEDLINE search information. Most frequently mentioned were: research/testing (by 71% of respondents), patient care (69.1%), and education (66.0%). Overall, 45.8% ranked research/testing as their single most common use; 32.8%, patient care; and only 12.4%, education. The results varied substantially by work place. 68.8% of those inprivate/solo practice and 75.8% of those in group practice most commonly use MEDLINE for patient care. In contrast 66.9%. of mdividuals in academic settings most commonly use MEDLINE for research/testing, purposes. UseofMeSH About two-thirds of the respondents (2^44 individuals, 93.7% of the entire respondent group) always use (24.1%) or usually use (43.6%) MeSH terms when searching for specific subjects. Similar percentages of respondents find MeSH terms very useful or useful. Length of Searches For almost two-thirds of the respondents (2,533 individuals, 933% of the entire respondent group) a tvpical search takes less than 10 minutes. Another quarter takes 10-15 minutes. Nearly 90% of respondents think the length of time to conduct a search is reasonable. OPINIONS ABOUT THE SYSTEM Level of Satisfaction 1,318 individuals (48.5% of the entire respondent group) indicated a level of satisfaction with the searches they perform for themselves. Of these, 72.5% are satisfied with their searches. iv 933% of the entire respondent group (2,539 individuals) rated their level of overall satisfaction with MEDLINE on the NLM system. 81.4% of this group were satisfied, and only 4.1% dissatisfied. The rest were neutral. Desirable Additions/Improvements to MEDLINE 60.8% of respondents (1,977 individuals, 72.8% of the entire respondent group) would find no additional types of information valuable. Of those who would like to see additional information, 79.1% indicated the full-text of articles, 37.0% author's address, 31.4% research design, and 23.6% dosage information. 78.8% of the entire respondent group (2,139 individuals) indicated features they would like to see added to MEDLINE by selecting from a list provided in the Questionnaire. 71.8% of these selected improved MEDLINE backfile searching and 692% improved printing capabilities. These two capabilities were also the single most desired features, by 35.9% and 27.9% of respondents respectively. ACCESS AND TRAINING Method of Access 96% of respondents (2,708 individuals, 99.7% of the entire respondent group) have access to a microcomputer in their work places, and a similar percentage use a microcomputer to search MEDLINE. 68.9% of the entire respondent group (1,870 individuals) always use a single method to access MEDLINE. Of the 2,474 individuals (91.1% of the entire respondent group) who provided information on access methods, 432% always use the command language, 26.9% always use the GRATEFUL MED form screens, 4.2% always use another front-end package, and 1.8% the GRATEFUL MED direct option. The remainder use more than one method to access MEDLINE. Higher percentages of physicians and those in private/solo and group practice always use GRATEFUL MED form screens in comparison to the respondent group as a whole. The percentage of those who always use command language increases as age (indicated by year highest degree received) decreases. The reverse is true for use of GRATEFUL MED form screens. Of the 2^92 individuals (88.1% of the entire respondent group) who answered the question regarding access problems, 53.8% reported none. The most frequent problem encountered was busy telecommunications lines (19.5%). Learning to Search MEDLINE 2,203 individuals (81.1% of the entire respondent group) checked the methods they used to learn to search, with 2,059 individuals (75.8% of the entire respondent group) also indicating the single most helpful method. The frequently mentioned methods were: NLM sponsored training course (49.7%); GRATEFUL MED (45.3%); and self-taught (45.0%). The most helpful methods were NLM-sponsored training (42.7%) and GRATEFUL MED (31.3%). Of the 1,368 individuals who said they took either NLM's 3-5 day training class or the 6-hour Basics of Searching MEDLINE Course, over 83% were satisfied with the training, less than 4% were dissatisfied, and the rest were neutral. v COMMENTS ON THE SYSTEM Most Satisfactory Aspects 747 individuals (27.5% of the entire respondent group) provided comments on their perceptions of the most satisfactory aspects of MEDLINE. 32.7% of these people mentioned the content of MEDLINE; 25.7% hours and availability; 24.6% speed and efficiency; 19.9% GRATEFUL MED; and 18.7% cost The results varied based on access method. Higher percentages of those who always use the comrnand language mentioned the content of the database and cost as compared to GRATEFUL MED form screen users. GRATEFUL MED itself was the most frequently mentioned satisfactory aspect by GRATEFUL MED form screen users. Least Satisfactory Aspects 874 individuals (322% of the entire respondent group) provided comments on aspects of MEDLINE that they considered to be least satisfactory. Most frequently mentioned were GRATEFUL MED limitations, by 172% of those who provided comments; backfile searching, by 14.0%; and MeSH vocabulary/ indexing, by 13.7%. Again the results varied based on access method. CONCLUSIONS The survey data reveal two different groups of individual users. The first group is concentrated in academic settings, uses MEDLINE primarily in support of research, and is typically interested in comprehensive retrieval. Most of the people in this group are physicians or physician scientists. . In July 1987, the majority in this group used the command language. The second group is made up predominantly of physicians who work in various clinical practice settings. These individuals use " MEDLINE primarily in support of patient care and are more likely than the "research" users to wish to retrieve just a few relevant citations. In July 1987, a larger percentage of this group already used GRATEFUL MED as compared to the "research" group. At the time the survey was conducted, the "research" group was slightly larger than the "patient care" group; but there is evidence that the "patient care" group is growing at a faster pace. Although the "research" and "patient care" groups have some key differences, they also have many important attributes in common. In general, they are relatively young physicians or scientists with access to rmcrocomputers. They are likely to do their own searches. Overwhelmingly they use MEDLINE to satisfy immediate information needs rather than to stay current in their fields or to learn about new areas. They typically search by subject In general, they are quite satisfied with MEDLINE. The results of the survey provide objective data on several key questions about online searching by individuals. NLM staff views about the system improvements most desired by individual users are generally corroborated by the users themselves In general, individual users appear to have a very positive view of NLM's online service. This is evidenced by the high return rate of the survey and the willingness of a large majority of respondents to participate in follow-up studies, as well as bv explicit indications of satisfaction. vi 1 Report on the Survey of Individual Users of MEDLINE on the NLM System INTRODUCTION The National Library of Medicine's basic mission is to improve the dissemination of information important to the progress of medicine and to public health. In support of this goal, NLM developed MEDLARS, an automated Medical Literature Analysis and Retrieval System, in the 1960's and was a pioneer in offering online retrieval services over nationwide telecommunications networks in the 1970*s. MEDLINE, a file of indexed citations to articles in biomedical journals, was the first NLM database to be made available online and continues to be the most heavily used MEDLARS file, accounting for more than 75% of the over 4 million searches done annually. For the first 20 years of its existence, MEDLARS was used principally by medical librarians who acted as search intermediaries for health professionals (both researchers and practitioners) who needed to locate pertinent information in the health sciences literature. To ensure that health professionals received good, cost-effective MEDLARS service, NLM required searchers to attend an online services training course as a condition of obtaining an access code for system use. As more and more individual health professionals began to have access to personal computers, NLM took several steps to encourage these individuals to access NLM's databases directly as "end-users." These steps included: developing a special one-day course in searching MEDLINE for health professionals, training medical librarians throughout the U.S. to provide this course in their local settings, stieamlining procedures for obtaining online access codes, making the system available around the clock, and developing a user-friendly microcomputer interface to the NLM system. On July 1, 1986, the training requirement was dropped entirely. As a result of these actions, the number of individual health professionals accessing MEDLINE directly on the NLM system has increased dramatically in the past three years and is continuing to grow at the rate of well over 200 users per month. PURPOSE OF THE SURVEY In order to provide a retrieval service that is both easy to use and effective, NLM needed more detailed information about this growing and important end-user population, their information needs, and their level of satisfaction with the present system, and with MEDLINE, the most frequently used database. As a first step, the Library decided to survey the individual users of MEDLINE to obtain answers to the following questions: 1. What are the demographic characteristics of the health professionals (researchers and practitioners) who perform their own searches on MEDLINE on the NLM computer system? 2. How do these users enter the system (e.g., directly using the command language or via a microcomputer front-end) and why do they use that method? 2 3. For what purposes are they accessing the system (research, patient care, etc.)? 4. What is their level of satisfaction? (a) with the content of the material identified (e.g., relevance of citations, completeness of retrieval, data elements available) (b) with the search mechanism used 5. What changes or improvements would make the system more useful to them? We hope to use the information gathered to identify system improvements that will provide a better level of information service to U.S. health professionals. METHODOLOGY Questionnaire Development In the summer of 1986, a Study Team* was convened to determine what information was needed to answer these basic questions and how it might best be obtained. The Study Team began by outlining broad categories of interest (e.g., demographics, level of tiaining, means of accessing the system, purpose of search, and level of satisfaction), and then formed subgroups to develop lists of questions within each of the categories. The Study Team developed a draft questionnaire that was circulated to other NLM staff for comment With the assistance of a contractor, Market Dynamics, Inc., the preliminary questionnaire was restructured and reformatted. A pre-test was then conducted on a small group of individual users of MEDLINE on the NLM system. Following the pre-test, the questionnaire was revised again and sent to the Office of Management and Budget for review as required by the Paperwork Reduction Act. The final version of the questionnaire appears as Attachment 1. Population Surveyed The survey population was defined as individual "end-users" of MEDLINE on the NLM system. Smce there was no reliable way to define sample strata, the decision was made to survey all individuals who had obtained access codes for use of NLM's online system in order to obtain as much richness in the data as possible. We were particularly interested in qualitative comments that individuals might provide in response to the open-ended questions. The only mechanism available to identify the target group was a classification system used by NLM's MEDLARS Management Section to process applications for online access codes. In cases of doubt, we erred on the side of being too inclusive, rather than restrictive, in deciding which classification codes to use.** As a result the mailing included, as best we could determine, all domestic individual end- users of MEDLINE on the NLM system as of July 1987. •Members of the Study Team were: Patricia Buchan, Betsy Humphreys, Charles Kalina, Sheldon Kotzin, T Scott Plutchak, Nancy Selinger, EUiot Siegel, John Starkweather, Carolyn Tilley, Karen Wallingford, and Rose Marie Woodsmall. •• Attachment 5 shows the classification categories and the numbers of mdividuals in each, and outlines the strategy used to define the survey population. 3 Instructions were included in both the cover letter and the follow-up letter for those who do not search MEDLINE themselves (i.e., who always have an intermediary search). They were asked to write across the top of the questionnaire that they do not search MEDLINE personally and to return the unanswered questionnaire to NLM. In cases in which they had begun filling out the questionnaire, they were instructed to stop at the bottom of page 2 after completing only the demographic questions. These individuals were not included in the data analyses. Data Collection Procedures The questionnaires, with a cover letter from the Director, National Library of Medicine (Attachment 2), were mailed to 4,311 individuals on October 2, 1987. A reminder postcard (Attachment 3) was sent to the entire survey population on October 14. Four weeks after the initial mailing, November 3, a follow-up letter (Attachment 4) and a duplicate copy of the questionnaire was mailed to those who had not yet responded. Returns were accepted until December 10,1987. Questionnaire Return Rates The following table shows the questionnaire return rates: Mailed Returned % Returned 4311 -73* 4238 2970 70.1% -243* -243** 3995 2716 Usable=68.0% * Undeliverable * * Ineligible because not individual users of MEDLINE **'Unanswered questionnaires returned by eligible individuals The return rate is based on the number who actually received the questionnaire. The 73 undeliverable questionnaires were subtracted from the original number mailed on the assumption that they were no longer active individual users of MEDLINE. The 243 who identified themselves as non-users were also subtracted from the population. The 11 individuals who erroneously considered themselves as ineligible (because they use GRATEFUL MED) were, however, not subtracted from the survey population because they are individual users of MEDLINE on the NLM system. The usable return rate was thus calculated to be 68.0%. Since the universe of individual MEDLINE users was surveyed, sampling error does not affect the results. Because the return rate was high, non-response bias is negligible. We believe the survey findings to be representative of all individual users of MEDLINE as of July 1987. 4 Data Analysis Procedures Questionnaires that were returned to NLM by the individuals surveyed were in turn sent to the contractor, Market Dynamics, Inc. Market Dynamics was responsible for designing a database format to store the questionnaire data and for coding and keying the data, with the exception of some of the open-ended questions. Coding of open- ended responses that required technical knowledge of the NLM system was done by NLM staff Market Dynamics used the SPSS statistical package for the data analyses. Frequency distributions were done for each of the questions and at the outset a number of cross-tabs were specified by NLM. As the data were examined, additional cross-tabs and t-tests were performed. FINDINGS The findings from the survey are presented in the following sections: o Demographic Characteristics of Survey Respondents o General Searching Behavior o Characteristics of MEDLINE Searches o Opinions about the System o Access and Training o Comments on the System o Willingness to Participate in Follow-up Studies o Conclusions o Key Points Unless, otherwise noted, percentages are based on the number of respondents who answered the particular question; mdividuals who did not answer the question are not reflected in the percentages presented. The number of non-responses to each of the questions is shown in the tables. On several questions, respondents were instructed to give more than one response, if appropriate. In these cases, percentages will sum to more than 100%. Demographic Characteristics of Survey Respondents Professions (Table 1) Question 1 asked respondents to indicate their profession(s) by circling all that applied. 2,710 individuals (99.8% of the entire survey group) responded to the question. Nearly two-thirds identified themselves as physicians (653%) and more than a quarter (27.7%) as scientists. 482 individuals did indicate more than one profession; 286 (103%) said that they were both physicians and scientists. Thus, 82.7% of the respondents who indicated their profession are physicians or scientists, or both. Nurses made up 3.7% of the respondent group. Another 6.6% said they were "other health professional." 7.6% listed their profession as "other." Included in this group were diverse professionals such as lawyers, statisticians, engineers, etc., but the numbers in the various "other" professions are too small to break out into separate categories 4.4% said they were students. Although the survey was intended for the population of users who search MEDLINE as individuals ("end-users"), 3.2% of the respondents indicated that they were librarian/information specialists. Of these, 28% indicated 5 more than one profession: nine were also "other health professionals," four were physicians, four nurses, four "other," and three students. Because the respondents were able to indicate more one than profession, the percentages total more than 100%. Specialties (Table 2) Question 4 asked the respondents who are health professionals to indicate their specialties. 2,197 (80.9% of the entire respondent group) answered this question. Those who responded are involved in a broad range of medical specialties. More than 40 specialties were listed, with some respondents indicating more than one specialty. The specialties listed most often are: internal medicine (282), medical oncology (147), pathology (125), and pediatrics (109). The full distribution of specialities is given in Table 2. Education (Table 3) Question 2 asked the respondents to indicate the highest degree held. The majority of the respondents hold either M.D. or Ph.D. degrees. 1,760 said that they hold M.D. degrees, and 570 hold Ph.D. degrees. Although the question asked for the highest degree (implying a single response), some respondents did list more than one degree. Year Highest Degree Was Received (Table 4,5) Question 3 asked the respondents to indicate the year in which they received their highest degree. Of the group, 982% (2,667 individuals) answered this question. Answers spanned a range of 60 years, with one person receiving the highest degree in 1929, and one expecting a degree in 1989. The majority received their highest degree within the last 18 years: 40.1% during the 1970's and 27.1% in the 1980's. Thus, the majority of individual users of MEDLINE appear to be relatively young. Just under 20% (19.6%) received their highest degree in the 1960's and 10.1% m the 1950's. Only 3.1% received their highest degree prior to 1950. Among the various professional groups, over half of the nurses (533%) and almost three-fourths of the students (72.8%) received their highest degrees in the 1980's. Primary Work Place (Table 6,7) Question 5 asked respondents to indicate their primary work place by circling only one of the choices provided or by specifying an "other." Nearly the entire group (99.4%, 2,701 individuals) answered, almost half (46.8%) indicating they work in an academic setting (college, university, or medical school). The remaining respondents are split evenly among private/solo practice (12.6%), group practice (123%), and hospital/clinic (14.6%). Only 5.4% work in a government agency, and 3.3% in a private company or business. Nearly half of the physicians (49.7%) work in a clinical practice setting (group practice 17.7%, private/solo practice 16.6%, and hospital/clinic 15.4%). 40.4% work m academic settings. Over three-fourths of the scientists work in academic settings. The largest percentage (45.8%) of the "other health professional" group also work in academic settings. Nurse respondents tend to work in either an academic setting (41.4%) or hospital/clinic setting (323%). 6 General Searching Behavior Frequency of Searching Question 8 asked the respondents to indicate the number of MEDLINE searc: es y do by themselves in the average month. 98.9% (2,686) of the respondents ar^ereuu question. More than two-thirds (68.6%) indicated that they do all searcnj» uy themselves. Half (52.7%) report that they perform two or fewer searches lnuieavciag month. Included in this group are the 12% who report that mey don tpertormd^y searches in the average month. These individuals were included ui tne J^a^ respondent group because, while they may not perform any searches in in 5 month, they do occasionally search MEDLINE by themselves. More ^ * ™J" (34.1%) perform 4 or more searches in the average month. Of the respondents1 wno reported that they perform at least one search in the^ average month, tne average number of searches performed is 43 searches per month.* Question 9 asked respondents to indicate the number of searches, thev have sorneone else do for them in the average month. Again, 98.9% (2,686 mdividuals) answered the question. The majority (68.6%) entered a zero, indicating that they do all of then* own searches. Of the one-third who at times have others perform searches for tnem, i:>.i /o have one search per month, 102% have 2-3 searches, and only 6.0% have 4 or more searches done by someone else in the average month. For those who have ottiers perform searches for them, the average number is 23 per month. Searches Performed by Others (Table 10-A/B, 11-A/B, 12,13,14) Question 10 was a multi-part question that asked respondents who occasionally or always have someone else search for them to indicate: A) who generally does the searches; B) under what circumstances is it preferable to have someone else search; C) what is the level of satisfaction with searches done by self and by others; and D) if generally not satisfied with searches done by others, what are the reasons for dissatisfaction. In response to the first part of the question, 1,103 individuals (40.6% of the entire respondent group) indicated that they at least occasionally have someone else perform searches for them. The majority (81.9%) of the searches performed by others are done by librarian/information specialists. Librarians tend be selected most frequently by those working in a government agency (89.7%), hospital/clinic (89.5%), group practice (89.4%), and private/solo practice (853%). Respondents in an academic setting have searches performed by student/research assistants (10.5%) and colleagues (6.7%), as well as librarians (74.1%). In response to part B of the question, 1,151 individuals (42.4% of the entire respondent group) provided the reasons why they, at least occasionally, have someone else search for them. "When I don't have time to do it myself" was the most frequently cited reason for having someone else perform MEDLINE searches. 593% of the responses •Because we were concerned that the few librarian/information specialists included in the respondent group might have inflated the average number of searches per month, we did some further analyses that excluded the librarian/information specialists. These analyses indicated that although the librarian/information specialists do perform far more searches (13.2) in an average month, they have a negligible effect on overall results. In addition, as noted earlier, over a quarter (28.0%) of the individuals who listed their profession as librarian/information specialist also indicated another profession. 7 indicated this reason. Slightly more than a third (38.7%) had someone else search when they needed different expertise or system knowledge. 27.5% had someone else search arter they had tried a search and not been satisfied with the results. Respondents to this question were instructed to circle all that apply, and in this case percentages again exceed 100%. 1,121 individuals (41.3% of the entire respondent group) indicated their level of satisfaction with searches done for them by someone else on a scale of 1 (very satisfied) to 5 (not at all satisfied). 70.6% (819 individuals) expressed satisfaction with the results by indicating 1 or 2, 8.9% (100 individuals) are dissatisfied (responses of 4 or 5), and 203% (230 mdividuals) are neutral (response of 3). Compared to the percentage of satisfied individuals from the respondent group as a whole, higher percentages of individuals in government (80.6%), private/solo practice (74.2%) and group practice (74.1%) reported being satisfied with searches done by others. Slightiy lower percentages of individuals satisfied with searches done by others were found in academic (68.5%) and hospital/clinic settings (65.5%). Private business not only had the highest percentage of satisfied individuals (80.7%), but also the highest percentage of dissatisfied respondents (12.9%). 150 individuals indicated reasons for dissatisfaction with searches done by others. Respondents to this part of the question were instructed to circle all that applied, and as a result percentages exceed 100%. Almost two-thirds of this group (62.0%) cited dissatisfaction with the results of the search. 393% stated that they have to wait too long for search results. "Other" was the next most frequent reason given for dissatisfaction (30.7%). The fourth most frequently listed reason was cost (24.0%). Searches Performed by Respondents Whv individuals search (Table 15) Question 11 asked respondents to circle all factors that influenced their decision to search MEDLINE themselves rather than having someone else search for them, and then to check the single most influential factor in their decision. 2,171 individuals (79.9% of the entire respondent group) answered this question; because multiple answers were allowed, percentages exceed 100%. Greater familiarity with the subject matter was selected by 83.1% of the respondents, and the ability to get results faster by 82.4%. Almost two-thirds of the respondents (65.5%) indicated enjoyment of searching as a factor. Nearly a third (31.6%) said that one of their reasons for searching by themselves is that it is more cost effective than using an intermediary. Only 14.7% cited lack of an intermediary as one of the reasons for doing their own searches. 2,074 individuals (76.4% of the entire respondent group) indicated a most influential factor. As the single most influential factor, respondents indicated greater familiarity with the subject matter than search intermediaries possess (47.9%) or the ability to get search results faster (322%). 7.5% indicated enjoyment of searching as the most influential factor in the decision to search for themselves. Level of experience (Table 16, 17) Question 12 asked respondents to indicate how experienced a user of online databases they considered themselves to be. Choices ranged from 1 (very experienced) to 4 (not at all experienced). 2,561 individuals (94.3% of the entire respondent group) answered this question. Less than 10% of the respondents felt themselves to be very experienced 8 users of online databases. The majority (84.5%) felt they were somewhat experienced or not very experienced. Less than 8% (7.2%) indicated that they were not at all experienced. Pattern of use of MEDLINE (Table 18, 19, 20,21, 22) Question 13 asked respondents to indicate how long they have been searching MEDLINE using their own code. 2^09 (92.4% of the entire respondent group) individuals responded. Nearly two-thirds (63.1%) have had their codes for a year or less. Just under another third (27.0%) have had their codes between one and two years. Only 10.0% have had access codes for more than two years. Individuals who always use command language to search have, on average, had their codes twice as long (134 years vs. 0.8 years) as those who always search using the GRATEFUL MED form screens. Question 7 asked respondents to indicate how many people (including themselves) share their access code. Respondents were instructed to enter a "1" if they were the only person who uses the code. 2,661 individuals (98.0% of the entire respondent group) answered this question. Overall, the respondents do tend to be true individual users of the system with the majority (82.0%) indicating that they don't share a code with anyone else. 8.6% share a code with one other person and another 8.7% share a code with two or more people. Question 14 asked respondents whether their use of MEDLINE has increased or decreased since they received their own access codes. 2,545 individuals (93.7% of the entire respondent group) answered this question. More than half (57.7%) said that their use has stayed the same. 30.0% indicated increased use and 123% said that their use has decreased. Question 15 was an open-ended response question that asked individuals whose usage had either increased or decreased to indicate the reasons for the change. The two most cited reasons given by the 678 individuals who reported increased use: increased familiarity with the system and greater need. Decreased need and lack of time were the two reasons cited most frequently by the 288 individuals who indicated their use of MEDLINE had decreased. Cost considerations (Table 23,24) Question 16 asked respondents to indicate how often cost considerations keep them from doing a MEDLINE search. 2^54 individuals (94.0% of the entire respondent group) responded. Cost seems not to be of overwhelming importance in individuals' use of MEDLINE on the NLM system. The majority (67.6%) indicated that cost considerations rarely (26.6%) or never (41.0%) keep them from performing a search Almost one-fourth (23.1%) said that cost considerations occasionally keep them from searching. Only 9.3% reported that cost considerations frequently keep them from searching. Those most constrained by cost considerations reported having no formal work Dlace with 46.1% of that group citing cost as a frequent or occasional constraint Slightlv ovS a third of those individuals in an academic (36.0%) or hospital/clinic (34 6%} settino said that cost was frequently or occasionally a consideration in searching. c"mg 9 Cost seems to be less of a consideration for those working in group practice, private business, and private/solo practice, than it is for those in academic and hospital/clinic settings. Under a third ot those working in private business (20.6%), group practice (22.6%) and private/solo practice (27.4%) reported that cost considerations occasionally or frequently are constraints on searching. Characteristics of MEDLINE Searches General Search Purpose (Table 25,26) Question 18 was a multi-part question that asked respondents about their most typical reasons for searching MEDLINE. 2^56 individuals (94.1%) answered the first part of the question. Four-fifths (81.7%) of the respondents mdicated they typically search MEDLINE to satisfy an immediate information need. The highest percentages in this category are found in group practice (883%), followed by those who indicated "other" work place (852%), pnvate/solo practice (83.6%), and government agency (83.2%). Ten percent of all respondents typically search for information to stay current in their field. Those in private business had the highest percentage (17.2%) indicating this as their typical reason for searching MEDLINE. People who indicated no formal work place (15.4%) and individuals in academic settings (12.6%) ranked second and third. Only 7-8% of those in private/solo or group practice or in a hospital/clime are typically interested in searching for information in order to stay current in their fields. Only 5% of all respondents indicated they typically search to learn about new areas. The highest percentage (6.6%) are individuals in a government agency; only 1.9% of those in group practice and "other" work places 'typically search to learn about new areas. Type of Retrieval Desired (Table 27, 28, 29) The second part of the question asked respondents to indicate the number of citations they typically wish to retrieve (all relevant or iust a few relevant citations). 2,550 individuals (93.9%) responded. The majority (58.9%) wish to retrieve all relevant citations from a particular time period rather than just a few relevant citations. Among the various work place groups, the percentages of individuals who typically want to retrieve all relevant citations range from a high of 65.2% of those in academic settings to a low of 47.6% of those in group practice. Analyzed by primary area in which search information is used, the percentages of individuals interested in all relevant citations ranges from a high of 67.1% of those who most commonly use search information for research/testing purposes to a low of 49.8% of those who most commonly use search information for patient care. Numbers of Citations (Table 30,31,32,55) The third part of question 18 asked respondents whether they typically retrieve too few citations, about the right number of citations, or too many citations. 2,510 individuals (92.4% of the entire respondent group) answered this part of the question. 61.1% of the respondents indicated that they typically retrieve about the right number of citations. When analyzed by work place, the groups with the highest percentages indicating retrieval of the right number of citations are: "other" work place (73.1%) and government agency (69.7%). 10 Overall, 15% of the respondents feel that they typically retrieve too few citations. The highest percentages choosing this response are found in private business (23.5%), pnvate/solo practice (21.7%), and group practice (20.6%), even though almost half of the respondents in these same work places expressed interest in retrieving only a few relevant citations. Just under a quarter (23.6%) of all respondents feel that they retrieve too many citations. A slightly higher percentage of those individuals in private business (27.1%) indicated this. Over 70% of all respondents always use either the command language or GRATEFUL MED form screens (43.2% and 26.9%, respectively) to search MEDLINE. Of the individuals who always search using command language, 64.8% indicated that they typically retrieve the right number of citations, compared to 56.9% of the individuals who always use the GRATEFUL MED form screens. A higher percentage of those who always search using GRATEFUL MED form screens retrieve too few citations compared to individuals who always search using command language (23.6% vs. 10.2%). In contrast 25.0% of the respondents who always search using command language indicated that theyretrieve too many citations compared to 19.6% of the searchers who always use GRATEFUL MED form screens. Relevance of Citations Retrieved (Table 33,34) The final part of question 18 asked respondents to indicate what percentage of the citations they typically retrieve are relevant to their inquiry. 2^99 individuals (88.3% of the entire respondentgroup) answered this portion of the question. More than half of all respondents (58.6%) feel that of the citations they typically retrieve, fewer than half are relevant. Analyzed by work place, an even higher percentage of those in group and private/solo practice feel this way (63.4% and 61.1%). Of the group of respondents as a whole, 41.4% feel that half or more of the citations typically retrieved are relevant. Individuals working in "other" places (52%) private £SF5J?? (48.8%), government agency (47.3%), and those with no formal work place (453%) were slightly more inclined to report retrieving greater percentages of relevant citations. Types of Searches Conducted (Table 35) Question 19 asked respondents whether they most often search for an author a iournal title, or a subject Instructions specified that only one choice should be circled 2 552 individuals (94.0% of the entire respondent group) answered this question 96 0% of the respondents said that they most often search for a subject. Only 3 2% indicated th*t they most often search for an author and less than 1% said that they most often search for a journal title. J "»■«=« acarcn Use of Information Retrieved (Table 36-A/B/C, 37,38) Question 20 asked respondents to rank the three primary areas in whirh tu* MEDLINE search information (1 = most common^ =1LTd m£^u^e££ 11 Regardless of the rank assigned, the areas mentioned most frequently were the following: research/testing (71%), patient care (69.1%), and education (66%). 2^92 individuals (95.4% of the entire respondent group) assigned a rank of 1 to indicate the area in which search information is most commonly used. 45.8% ranked research/testing as the most common use, followed by 382% who indicated patient care as tne most common use. Educational use was selected by 12.4%. Of all individuals who listed research/testing as one of their uses of MEDLINE information, 613% indicated it as the most common use. Similarly, of all individuals who ranked patient care as one of their uses, 52.8% ranked patient care as the most common. In the area of education the results are very different: of the individuals who listed education as one of their uses of MEDLINE information, only 17.9% indicated it is their most common use. Of the individuals in private/solo practice, over two-thirds (68.8%) indicated that their most common use of the information is for patient care. Similarly, 75.8% of those in group practice most commonly use MEDLINE information for patient care. One-half of those in a hospital/clinic setting indicated patient care as their most common use. In contrast of the individuals in academic settings, two-thirds (66.9%) indicated their most common use of MEDLINE information is for research/testing purposes. Over half of the individuals in private companies/businesses and government agencies also indicated that their most common use is for research/testing (52.8% and 56.6%, respectively). Only about one-tenth of the individuals in private/solo practice (10.6%), group practice (83%), and academic settings (103%) listed educational purposes as their most common use of MEDLINE information. Higher percentages of individuals in hospital/clinic setting (192%) and private company/business (15.7%) indicated educational purposes as their most common use. Of all respondents who listed patient care as their primary use of MEDLINE information, 40.8% always use command language to search while 31.6% always use the GRATEFUL MED form screens. Twice as many individuals who always use command language search for research/testing purposes as compared to GRATEFUL MED form screen users (45.6% vs. 23.1%). And far more of the respondents whose primary use of MEDUNE information is for management/administration purposes indicated that they always use command language compared to individuals who always use the form screens (60.7% vs. 10.7%). Similarly, of all respondents who use MEDLINE information for regulation purposes, 50% always use the command language vs. 12.5% who always use the GRATEFUL MED form screens. Use of MeSH (Table 39,40-A/B, 41) Question 21 asked individuals to indicate how often they use MeSH terms when searching for specific subjects. Choices provided were: 1. Always, 2. Usually, 3. Occasionally, 4. Rarely, 5. Never. 2,544 individuals (93.7% of the entire respondent group) provided a response to this question. Over two-thirds of the respondents (67.7%) always use (24.1%) or usually use (43.6%) MeSH terms for searching for subjects in MEDLINE. 8.8% rarely and 5.1% never use MeSH. There is a similar distribution of responses to question 22 which asked respondents to indicate how useful they find the MeSH terms to be. The scale provided ranged from 1 (very useful) to 5 (not at all useful). 2,475 individuals (91.1% of the entire respondent group) provided an answer to this question. Two-thirds find MeSH terms very useful (26.7%) or useful (39.0%), while 7.8% say that they're not useful and 3.2% say they're not at all useful. The remaining 23.2% are neutral regarding the usefulness of the MeSH terms. 12 When considered by work place, 73.2% of those in group practice and 71.7% of those in a hospital/clinic tend to use MeSH. Correspondingly, higher percentages of individuals in group practice (70.0%) and hospital/clinic settings (73.3%) feel that MeSH terms are useful. Among the few who indicated that they rarely or never use MeSH, the highest percentages are found in government agencies (20.6%) and in private business (193%). Higher percentages of the individuals in these same work places (government agencies 19.4% and private business 15.3%), along with individuals with no formal work place (16%) said that MeSH terms are not very or not at all useful. A t-test was done to compare the individuals who always search using the command language (N=1,060) with the individuals who always search using the GRATEFUL MED torm screens (N=653) with regard to their use of MeSH terms. The mean for the group of individuals who always search using the command language is 2.08, indicating that the individuals who always use the command language usually use MeSH terms to search, while the mean for the group who always search using the GRATEFUL MED form screens is 2.46 (p = .000), indicating that among these individuals MeSH terms are used less frequently in searching MEDLINE. These two groups were also compared on how useful they find the MeSH terms to be. In the group who always search using the command language and who answered question 22 (N=1,051), the mean is 2.14 indicating that they generally find MeSH terms to be useful, while for the group of individuals who always use GRATEFUL MED form screens (N=620), the mean is 226 (p < .05), indicating that these individuals, on average, think MeSH terms are useful, but less so than the group who search using the command language. Question 23 was an open-ended question that asked those individuals who think MeSH terms are generally not useful or who never use MeSH to indicate why. Responses were provided by 454 individuals (16.7% of the entire respondent group). The two reasons given most frequently for not using MeSH are: urjfamiiiarity with MeSH (N = 116) and that MeSH terminology is cumbersome to use (N=107). The next two most frequently given reason for not using MeSH is that "there are no terms in my area" (N=57), followed by "terms are too general" (N=49). Length of Searches (Table 42) Question 24 asked how long it typically takes to search MEDLINE for citations on a particular subject 2333 individuals (933% of the entire respondent group) answered this question. For almost two-thirds of the respondents (64.1%), a typical search takes less than 10 rninutes. Another quarter (25.2%) typically take between 10-15 minutes to conduct a search. 10.7% typically take more than 15 minutes. Question 25 then asked whether the time was too long, reasonable or Quicker thnn expected. 2326 individuals (93.0% of the entire respondent group^ ariwered thS question. The majority of the respondents (89.1%) feel that the length o7 timeto conduct a search is reasonable, including 5.7% who find it to be quicker than »™^-a Only 10.9% think it takes too long. Hearer man expected. 13 Opinions about the System Level of Satisfaction with Searches (Table 43,44) Question 8 asked individuals to indicate on a scale of 1 (very satfsfied) to 5 (not at all satisfied) how satisfied they are with searches they do themselves. 1,318 individuals £483% of the entire respondent group) answered this question. Almost three-quarters (723%) of these respondents are satisfied with searches they do for themselves. Compared to the group overall, higher percentages of those with no formal work place (90.0%), those in a government agency (793%), and those in an academic setting (763%) express satisfaction with searches they do themselves. Just under ten percent (9.0%) say that they are not satisfied with searches they do for themselves. Those in private/solo practice (14.7%) had the highest rate of dissatisfaction with their own searches. Overall Satisfaction with MEDLINE (Table 45,46,47) Using a scale of 1 (very satisfied) to 5 (not at all satisfied), question 17 asked the respondents to rate their overall satisfaction with MEDLINE on the NLM system. 23^9 individuals (933% of the entire respondent group) answered this question. Overall, 81.4% of the respondents indicated satisfaction with MEDLINE. Only 4.1% indicated that they are not satisfied. The remainder are neutral, neither satisfied nor dissatisfied. Analyzed by work place, the highest percentages of satisfied individuals are those with no formal work place (923%) and those working in a government agency (90.5%). The groups expressing the highest percentages of dissatisfaction are those in private/solo practice (6.7%) and group practice (5.8%). Individuals who always use GRATEFUL MED form screens and individuals who always use the command language to search appear to be equally satisfied with MEDLINE (823% and 823% respectively). Similarly, 3.6% of the individuals who always use command language report being dissatisfied with MEDLINE compared to 33% of those who always use the GRATEFUL MED form screens. Desirable Additions to MEDLINE (Table 48,49,50) Question 26 asked respondents whether there are types of information that they would find valuable that they cannot routinely find in a MEDLINE citation. 60.8% of the respondents answered no, indicating that MEDLINE citations are acceptable in their current form. Individuals who answered yes to the question were then presented with a list of information types from which to choose. Respondents were instructed to circle all that would be valuable, and to check the single most valuable type of information. Among the respondents who indicated that there are other types of information they would find valuable (N= 1,019, 37.4% of the entire respondent group), full text of articles was the overwhelming choice; 79.1% listed it as one of their choices and 65.2% as the single most valuable information to add to MEDLINE. Author address was the next most frequently mentioned (37.0%), with 112% of the respondents indicating it as the most valuable piece of information. 31.4% said that research design would be valuable, but only 7.9% chose it as the single most valuable. Similarly, 23.6% said they'd find dosage information valuable, but only 4.5% thought it would be the single most valuable type of information to add to MEDLINE citations. 14 When viewed by primary purpose of search (patient care, education, research/testing), just under two thirds of those primarily interested in patient care (64.0%, N73IH) and education (62.4%, N=83) selected full text as most valuable as opposed to just under half of those interested in research/testing (48.0%, N=271). Dosage information was the next piece of information most frequently identified as most valuable by those witn a primary interest in patient care (63%, N=30) and education (5.3%, N=30, tiedI witA journal section); those interested in research/testing selected author address (10.1/0, N=91). The piece of information which ranked third as most valuable was research design, chosen by 9.4% (N=53) of those interested in research/testing and 5.3% (N=25) of those in patient care. Desirable Improvements to MEDLINE (Table 51,52,53) Question 27 asked respondents to indicate the features or capabilities they would like to see added to MEDUNE by selecting from a list provided and to check the one feature they would most like to see. 2,139 individuals (78.8% of the entire respondent group) provided an answer to this question. 71.8% of the respondents indicated unproved MEDLINE backfile searching and 69.2% want improved capabilities for printing citations. These same two capabilities were chosen most frequently as the single most desired capabilities individuals would like to see added to the NLM system: 35.9% listed improved backfile searching as the most desired capability, and 27.9% chose improved capability for printing citations as the most desired. Improved backfile searching and improved print capability were the overwhelming first and second choices irrespective of the respondents' typical purpose of search. Among the individuals who typically search for educational purposes, 322% indicated more "didactic" literature as one among the features they would like to have added to MEDLINE and 9.8% considered it the most desirable additional feature. In the group who typically search for patient care purposes, 213% listed more "didactic" literature among their choices for features to add to MEDLINE. Individuals who always search using command language listed improved backfile searching (70.1%) and improved print quality (71.1) most frequently among all desired capabilities. Individuals who always search using GRATEFUL MED form screens also chose the same two features: backfile searching (63.8%) and printing (55.6%), although in smaller percentages. Access and Training Equipment (Table 54) Question 6 asked whether individuals have microcomputers available in their work Ria^ V? ind™duals. (9?.7% of the entire respondent group) answered this question, 96.0% of them affirmatively. Question 28 then asked respondents to indicate whether they primarily use a microcomputer or a terminal to access MEDLINE. Of the 2 556 mdividuals (94.1% of the entire respondent group) who provided an answer 96 0% said that they use a microcomputer, only 4.0% indicating that they use a terminal 15 Command Language Searching vs. Use of GRATEFUL MED (Table 55,56,57,58) Question 29 asked respondents to indicate the percent of MEDLINE searches they performed using various access methods. Instructions stated that the percent should add to 100 and that if a method was not used, respondents should enter a "0" in the blank provided. 2,474 individuals (91.1% of the entire respondent group) answered this auestion. 1,870 individuals (75.5% of those who answered the question) indicated that ley always use a single method to access MEDLINE, almost half (43.2%) always using the command language, while over one-fourth (26.9%) always use the GRATEFUL MED form screens. Smaller percentages of individuals always use another front-end package (42%) or GRATEFUL MED direct option (1.8%). Only 604 individuals mdicated that they use more than one method to access MEDLINE. Of these, 504 use GRATEFUL MED form screens some of the time, 359 use GRATEFUL MED direct option some of the time, 268 use command language some of the time, and 124 use another front-end package some of the time. The remainder of this Section presents data only on those mdividuals who either always search using the command language or who always search using the GRATEFUL MED form screens. Higher percentages of students (503%), other health professionals (49.4%), nurses (47.6%), and scientists (46.6%) always use the command language to search than of physicians (39.7%). More of those reporting no formal work place (57.7%) or "other" work place (47.1%) always search using command language as compared to individuals in other specified work places. A larger percentage of the physician group (30.1%) always uses the GRATEFUL MED form screens in comparison to the other professional groups ("other health professional" 24.8%,.scientist 21%, nurse 18.5% and student 17.1%). One-third of the respondents in private/solo and group practice use GRATEFUL MED form screens all of the time. A greater percentage of those who are younger (receiving degrees in the 1970's and 1980's) search MEDLINE using the command language, while a higher percentage of those receiving their degrees earlier search using the GRATEFUL MED form screens. Half of those receiving degrees in the 1980's indicated that they always search MEDLINE using the command language, whereas only 25.0% of those who received their highest degree before 1950 do so. The percent who search using command language increases as age decreases: only 41.2% of those receiving degrees before 1950 sometimes search using the command language, as opposed to 613% of those who received degrees in the 1980's. The reverse is true for use of the GRATEFUL MED form screens. 60.9% of those receiving degrees before 1950 use them at least some of the time, compared to 30.7% of those receiving degrees in the 1980's. Difficulties Accessing MEDLINE (Table 59) Question 31 asked respondents about the types of problems, if any, they have in accessing the NLM computer. 2,392 individuals (88.1% of the entire respondent group) provided an answer to this question. Respondents were instructed to circle all choices that applied. More than half of the respondents (53.8%) reported no problems in accessing the NLM computer. The most frequent problem encountered is busy telecommunications lines (19.5%). 16 Learning to Search MEDLINE (Table 60, 61-A/B, 62-A/B, 63-A/B, 64-A/B) Question 32 asked individuals to indicate all the methods they used to learn to search MEDLINE on the NLM system, and to check the one method that was tne mow helpful. 2,203 individuals (81.1% of the entire respondent group) affiweiw ^ question, with 2,059 individuals (75.8% of the entire respondent group) listing a meinuci that they found most helpful The most frequently mentioned methods were, m-ivi- sponsored framing course (49.7%), using GRATEFUL MED (453%), and self-taught (45.0%). These same three methods were most frequently listed as most neipmi. NLM-sponsored training (42.7%), use of GRATEFUL MED (313%), and self-taugnt (11.8%). Among the individuals in both private/solo practice and group prartice,G^TCFUL MED ranked first among the methods used to learn to search MEDLINE (52.2% and 512% respectively). Self-taught (443% and 443%) and attended NLM training (412% and 44.1%) were the next most frequently used methods by individuals m nnvQtA /c«1« a«d mrmn r»rar*iV* Individual* in academic and hOSPltal/ClmiC Settings Among the individuals who always search using the command language, NLM training was most frequently (72.8%) listed as a method used to learn to search MEDLINE, followed by self-taught (44.4%), and taught by a co-worker (14.0%). In this group of individuals, NLM training was most frequently selected as the most helpful method by 683% of the respondents. In the group of individuals who always search using the GRATEFUL MED form screens, GRATEFUL MED was by far the most frequently cited (93.1%) among all methods used to learn to search, followed by self-taught (38.7%) and NLM-sponsored ttaining (17.7%). In this group, 82.2% selected GRATEFUL MED as the most helpful method in learning to search. A t-test was done to compare the group of individuals who always search using the command language with those who always search using the GRATEFUL MED form screens in regard to whether they had attended either the 3-5 day training class offered by NLM or the 6-hour Basics of Searching MEDLINE course. 19.9% of the individuals who always search using the command language had attended the 3-5 day class compared to 2.7% of tne individuals who search using GRATEFUL MED form screens. And twice as many individuals who always search using the command language attended a 6-hour Basics class compared to those individuals who always use the GRATEFUL MED form screens (29.7% vs. 14.7%). Question 33 asked those respondents who had attended the 3-5 day training course and/or the 6-hour Basics of Searching MEDLINE course to indicate their level of satisfaction with the training. Again, the scale ranged between 1 (very satisfied) and 5 (not at all satisfied). Of those who said that they took the 3-5 day training course (328 individuals, 12.1% of the entire respondent group), 83.8% indicated that they were satisfied with the training. Less than 3% were dissatisfied. The rest were neutral. Satisfaction with training does relate to overall satisfaction with MEDLINE; 83.7% indicated that they are satisfied with both the training and with MEDLINE. Similarly, 2.8% indicated dissatisfaction with both the training and with MEDLINE. 13.6% were neutral on both. 17 Of those who took the 6-hour Basics of Searching MEDLINE course (1040 individuals, 383% of the entire respondent group), 83.5% were satisfied with the fraining they received. Less than 4% of this group indicated dissatisfaction. The rest were neutral. Again, satisfaction with fraining relates to overall satisfaction with MEDLINE; 83.7% indicated satisfaction with both the 6-hour course and with MEDLINE; 33% indicated dissatisfaction with both the fraining and with MEDLINE. 12.8% were neutral on both. Comments on the System Most Satisfactory Aspects (Table 65, 66) Respondents were given space to comment on their perceptions of the most satisfactory aspects of MEDLINE. Comments were provided by 747 individuals (27.5% of the entire respondent group). Percentages are based on the number of individuals responding, rather than the number of aspects they cited. The most frequently cited aspect is the content of MEDLINE (32.7%). Hours and availability (25.4%) and speed and efficiency (24.6%) were the next most frequently mentioned. GRATEFUL MED (19.9%) was fourth on the list followed closely by cost (18.7%). Among the individuals who always search using the command language and who provided comments on the aspects they find most satisfactory (N=305, 11.2% of the entire respondent group), the following were most frequently cited: content of the database (39.8%), speed and efficiency (293%), hours and availability (27.6%), and cost (24.7%). Of those who always search using the GRATEFUL MED form screens and who provided comments on the most satisfactory aspects (N=194, 7.1% of the entire respondent group), the aspects most frequently cited are: GRATEFUL MED (45.9%), hours and availability (253%), content of the database (23.2%), and speed and efficiency (21.6%). Cost was mentioned by only 12.9% in this group. Again, because a single respondent may have indicated more than one aspect percentages total to more than 100%. Least Satisfactory Aspects (Table 67, 68) Respondents were also given space to comment on aspects of MEDLINE that they considered to be least satisfactory. Comments on least satisfactory aspects were provided by 874 individuals (322% of the entire respondent group). Topping the list were GRATEFUL MED limitations (17.2%), backffle searching (14.0%), and MeSH vocabulary/indexing (13.7%). Among those who always search using the command language, and who provided comments on the aspects they find least satisfactory (N=387, 142% of the entire respondent group), the aspects cited most frequently are: MeSH vocabulary/indexing (17^3%), command language searching (16.0%), printing (16.0%), and backfile searching (13.2%). "GRATEFUL MED not available for other types of microcomputers" was mentioned as a least satisfactory aspect by 12.9% of the command language users who provided comments. In the list of least satisfactory aspects cited by individuals who always use GRATEFUL MED form screens (N = 196, 7.2% of the entire respondent group) are: GRATEFUL MED limitations (37.2%), backfile searching (15.3%), and MeSH vocabulary/indexing (10.7%). 18 Willingness to Participate in Follow-Up Studies (Table 69) 86.1% of the 2342 individuals (862% of the entire respondent group) who answered this question indicated that they would be willing to participate in follow-up stuaies, while only 13.9% declined. 374 individuals (13.8%) did not answer the question. CONCLUSIONS In the two years before the survey was conducted, NLM took two major steps to encourage individuals to search MEDLINE directly. The first was the est^jshrnerii m 1985 of a program to train health professionals in the basics of searching MEDLINE on the NLM system. To support this program the Library developed a 6-hour course and trained medical librarians throughout the United States to teach it The widespread availability of this short course led to the first sharp increase in the number of individual users of MEDLINE. About 25% of all survey respondents (27% of those who supplied the information) indicated that they obtained their codes during the first year the course was given, i.e., 13 to 24 months prior to July 1987. The second step was the introduction in March 1986 of GRATEFUL MED, a PC software package that allows users without special fraining to search MEDLINE and other databases on the NLM computer. About 58% of all survey respondents (63% of those who provided the information) obtained their codes in the year after GRATEFUL MED was introduced, i.e., during the 12 months ending in July 1987. The survey data on methods used to learn to search MEDLINE and to access the database indicate that this tremendous 12-month increase in the number of individual code holders is due to the combination of the introduction of GRATEFUL MED and the continuing effects of the short courses in command language searching. Overall, 40% of all respondents (49.7% of those who provided training information) indicated that they had taken an NLM sponsored training course and 49% (54.1% of those who answered the question) indicated that they always or sometimes used the command language. 37% (453% of those who supplied the information) stated that they had used GRATEFUL MED as one method for learning to search and 42% (47.4% of those who answered the question) indicated that they always or sometimes used GRATEFUL MED form screens when searching MEDLINE. As respondents could indicate multiple fraining methods and some use both the command language and GRATEFUL MED form screens to search, there is some overlap in these figures. Nonetheless, at the time the survey was taken there were slightly more individuals using command language to access NLM's system than were using GRATEFUL MED. This situation has changed dramatically since July 1987. Of the more than 3,500 individuals who have obtained codes since the survey was taken, about 75% indicated that they intended to use GRATEFUL MED. In addition to the general increase in publicity for the software package, the emphasis in many of the search training courses provided by medical librarians has shifted from the command language to the use of GRATEFUL MED. In assessing the significance of the survey results, it is important to remember that individual command language searchers are now almost certainly a minority and that the percentage of individuals using the command language will probably continue to decrease. While the split between command language and GRATEFUL MED searchers represented in the survey results can be viewed as a historical artifact the data also reveal a more basic division of individual users into two different groups that can be 19 expected to persist. The first group is more heavily concentrated in academic settings, uses MEDLINE primarily in support of research, and is typically interested in comprehensive retrieval. Most of the people in this group are physicians or physician- scientists. At the time of the survey, higher percentages of individuals in academic settings had taken formal online search training and were using the command language as compared to individuals in other work settmgs. This is not surprising since to date the majority of the special short fraining classes have been given in academic settings. The second group is made up predominantly of physicians who work in private/solo practice, group practice, or hospital or clinic settings. These individuals use MEDLINE primarily in support of patient care and are more likely (than "research" users) to wish to retrieve just a few relevant citations. At the time the survey was conducted, the majority of users in private/solo or group practice was already using GRATEFUL MED, and a higher percentage of users in hospitals or clinics was using GRATEFUL MED as compared to users in academic settings. Individuals in these settings were less likely to have access to the short courses in command language searching. The lines between the "research" and "patient care" groups blur, particularly in the research hospital environment and it is obvious that the same individual may be a "research" user and a "patient care" user at different times. Nevertheless, the survey data support the traditional view that there are two basic types of MEDLINE users and that they do look for different things from the system. At the time the survey was conducted, the "research" group was slightly larger than the "patient care" group. The the latter has much greater potential for growth, however, and can be expected to become the predominant group of individual users, if indeed it has not already become so. The survey results show that the percentage of new individual users coming from academic settings had been declining slowly during the 18 months prior to the survey, as the combined percentage of new users from patient care settings was increasing slowly. Although more than half of both groups typically retrieve what they perceive as the "right number of citations," the survey revealed that retrieval of too few citations is perceived as a more frequent problem by "patient care" and GRATEFUL MED users than by "research" and command language users. Thisperception is supported by NLM system statistics showing that over 30%> of GRATEFUL MED searches result in no retrieval. Analyses of system traffic files of GRATEFUL MED searches indicate that non-retrieval most frequently results from combining too many search terms together or through misunderstanding of basic indexing policy (i.e., that the most specific terms are assigned by indexers). This is probably due to the fact that fewer GRATEFUL MED users receive any formal online search training. The survey findings provide additional justification for NLM's efforts to provide some sort of automated assistance to users who have retrieved nothing in response to a GRATEFUL MED search. Although the "research" and "patient care" groups have some key differences, they also share many attributes. Respondents in both groups appear to be relatively young, as evidenced by the fact that 67% of all respondents received their highest degree in the last 18 years.* Essentially all respondents have access to personal computers and use •We compared the age distribution of physician respondents with the age distribution of all U.S. physicians as reported in a recent AMA survey (Physician Characteristics and Distribution in the U.S. 1987 edition. Chicago, Division of Survey and Data Resources, American Medical Association, 1987, p. 22). For purposes of comparison, we assumed that the "average" physician obtains the M.D. degree at age 26. This suggests that those who received their MJD. degree before 1950 are under-represented among MEDLINE access codes holders, while those who received their M.D. degree in the 1960's or 1970's are disproportionately more numerous in the NLM sample than in the U.S. physician population as a whole. 20 them to perform MEDLINE searches. The majority do all their own searches, although this amounts to only a few searches in the average month. Most cite greater farniuanty with the subject or the ability to get search results faster as the prmripal reason tney search for themselves. Overwhelmingly they typically use MEDLINE to satisry immediate information needs, rather than to stay current in their fields or to learn about new areas. They also typically search for subjects rather than authors or jonyn^ titles. Overall, they are quite satisfied with MEDLINE, although the majority thin* tnat fewer than half the citations they retrieve are relevant Use of features such as starred terms and subheadings can help to exclude irrelevant citations, but analyses ot tne system traffic files show that these features are rarely used by individual users. Additional improvements to GRATEFUL MED and the development of the CUACri expert search assistant should therefore focus on the problem of refining searches to reduce irrelevant retrieval. Most respondents are satisfied with the types of information currently available in MEDLINE citations, and provided no data on additional information they would like to find in the file, even though full-text of articles was one of the choices presented. Of the 40% who thought that additional types of information would be useful, nearly two- thirds did indicate that the full text of articles would be the single most useful addition. Although only a minority of both groups mentioned full text a greater percentage of the "patient care" group is interested in full text than of the "research" group, who tend to be in academic settings with better access to the literature. Again, not surprisingly, more members of the "research" group than of the "patient care" group listed author's address as the single most valuable addition to the MEDLINE record. As of January 1988, the principal author's address is carried in MEDLINE records. The majority of respondents did indicate some features or capabilities (as opposed to information) that would be useful enhancements to MEDLINE. Those respondents who specified any desirable improvements generally agree on the principal features or capabilities they would like to see added to the NLM system. These are improved MEDLINE backfile searching and improved capabilities for printing citations. As might be expected, improved print capabilities were of greatest interest to command language searchers. Here the survey findings corroborate the opinions of NLM staff. Prior to the survey, work had been initiated to improve backfile searching capabilities in GRATEFUL MED. GRATEFUL MED users can now indicate at the beginning of a search that they wish to search MEDLINE and the backfiles or they can perform a backfile search without first searching MEDLINE. These changes were among others introduced in Version 3.0 which was mailed to users early in 1988. Also prior to the survey, improvements were planned for backfile search capabilities for command language searchers. A multi-tile search capability was implemented in late 1988. An improved print capability was implemented for command language searchers in November 1987. About 70% of the respondents did not provide any free form comments on the most and least satisfactory aspects of MEDLINE. 27% of the respondent group (n=747) provided a total of 1,260 comments on the most satisfactory aspects of MEDLINE. The pattern of comments was very different for command language searchers and GRATEFUL MED users. Command language searchers mentioned the content of MEDLINE most frequently, followed in order by the speed and efficiency of the system, its hours and availability, and cost. The appreciation for the content of MEDLINE is gratifying, considering the resources NLM devotes to selecting the contents and ensuring the quality of the database. The MEDLINE database is available from a variety of commercial online vendors, and in CD-ROM products, however, so it does not represent a particular incentive to use the NLM system. NLM's 21 system also does not differ appreciably from those of other online services in terms of speed or accessibility. The widespread availability of short fraining courses in the use of the NLM system and relative cost are probably the factors that led these command language searchers to choose NLM's system over other alternatives. For GRATEFUL MED searchers who provided comments, convenience appears to be primary. GRATEFUL MED itself, which can only be used on NLMfs system, is mentioned most frequently, followed in rank order by hours and availability, content and speed and efficiency. Cost is mentioned much less frequently by GRATEFUL MED users. Older respondents are much more likely to use GRATEFUL MED than the command language, another indication that it is attractive to people with relatively little previous exposure to computers. These data support the view that GRATEFUL MED is making online searching attractive to people who previously thought it too inconvenient time-consuming and cumbersome. 31% of respondents (n=874) provided a total of 1,218 comments about the least satisfactory aspects of the system. Again, there were substantial differences between command language searchers and GRATEFUL MED users. The least satisfactory aspect mentioned most frequently by command language searchers was MeSH vocabulary/indexing (by 173% of those commenting), followed by command language searching itself (16.0%), printing citations (16.0%), and backfile searching (132%), and the fact that GRATEFUL MED is currently available for IBM-compatible micros only (12.9%). This list matches the Library's view of the principal difficulties with command language searching by individuals. Solving problems caused by a lack of understanding of MeSH and indexing principles is a niajor objective of NLM's Unified Medical Language System (UMLS) project. GRATEFUL MED was developed to obviate the need for health professionals to learn the command, language. As mentioned previously, NLM has also taken steps to improve printing and backfile capabilities. A Macintosh version of GRATEFUL MED wiil be available in 1989. The small percentage of GRATEFUL MED searchers who provided comments on the least satisfactory aspects mentioned specific limitations of the GRATEFUL MED package most frequently (37.2%), followed by backfile searching (15.3%), and MeSH vocabulary/indexing (10.7%). It should be noted that the survey was conducted before the introduction of version 3.0 of GRATEFUL MED which incorporated many of the desired changes. Also, many of the individuals who cited GRATEFUL MED software limitations as a least satisfactory aspect also listed GRATEFUL MED as the most satisfactory aspect of the system. Thus, it seems that GRATEFUL MED users are generally quite satisfied with the software, but would like to have continued improvements made in its capabilities. The results of this survey provide some objective data on key aspects of individual search behavior that have been discussed and debated since the introduction of online systems. There has been speculation about whether individuals would stop doing their own searching after the novelty wore off. The overwhelming majority of the respondents to this survey indicated that their use of MEDLINE had remained the same or increased since they obtained their online codes. As most respondents had obtained their codes within the year prior to the survey it will be interesting to see whether this pattern continues. Tne likelihood of continued use is increased by the fact that the overwhelming majority of survey respondents indicated that they typically search for immediate information needs rather than to stay current in their field or to learn about new areas. This is the case irrespective of whether respondents use the information for research or patient care purposes. It suggests that these individuals are integrating at least a modest amount of online searching into their regular work habits. 22 Another topic of discussion is whether the opportunity for direct online searching would attract new users or simply alter the mode of access of previous users of search intermediary services. The fact that two-thirds of the survey respondents do all their own searches suggests that new users are being atfracted by the convenience of direct online searching. This view is supported by recent evaluations of MEDLINE CD-ROM products which also indicate that each new form of access to MEDLINE attracts some people who previously did not use the system in any form. Controlled vocabularies, and the disparities among them, are often cited as barriers to effective use of online systems by individuals users. Over two-thirds of the respondents to the survey indicated that they always or usually use MeSH terms when searching for subjects in MEDLINE. A similar percentage find MeSH terms very useful or useful. The high rate of use of MeSH is a logical consequence of the percentage of respondents who had attended at least a 6-hour course in MEDLINE searching and the fact that GRATEFUL MED assists the user in locating appropriate MeSH terms. NLM's continuing efforts to improve and expand the MeSH cross-reference structure have also made the terminology more accessible to those without extensive search fraining. It appears that individual users are willing to use at least a single controlled vocabulary if such use is made relatively easy for them. There has been considerable debate on whether (and how much) individuals would be willing to pay for direct online searching. About two-thirds of the respondents to the survey indicated that cost considerations rarely or never keep them from doing a MEDLINE search. Less than 10% reported that cost considerations frequently keep them from searching. Cost seems to be even less of a consideration for those in group practice and private/solo practice than for those in academic and hospital settings. These data (plus the continuing growth in the numbers of individual users that NLM has experienced since the survey was conducted) indicate that the current cost of searching MEDLINE on NLM's system is acceptable to many individuals. While the survey provided new and interesting information related to online searching by individuals, it also provided objective data to support previously held opinions about these users and their views of the NLM system. In particular, NLM staff views about the system improvements most desired by individual users are generally corroborated by the users themselves. It is encouraging that enhancements recently implemented or planned by the Library will address several of these users' concerns with present system features. In general, respondents to the survey appear to have a very positive view of NLM's online service. This is evidenced not only by their explicit indications of satisfaction with MEDLINE but also by the high return rate of the survey and by the willingness of the large majority of the respondents to participate in follow-up studies. The respondents to the survey represent relatively early adopters of direct online searching Included in the survey population are; 1) the very small number of individuals who were willing to take the extensive fraining offered to search intermediaries in the earlier days of online retrieval; 2) the much larger groun whn were willing to take a 6-hour fraining course when it began to be offered in 1985- and Vi another substantial group who were the first individuals to decide that GRATFFTTT MED made direct online searching practical and convenient for them Th* r^Ti^zh growth in the number of individual GRATEFUL MED users since the time the I™!„ was conducted suggests that there is a large pool of people who will find direct nni3 searching feasible and attractive when they are made aware of the technoloev rnrr^nti available. It remains to be seen whether these later adopters of direct online vr1^ will differ in significant ways from the users described in this survey searching 23 Key Points 1. Nearly two-thirds of respondents identified themselves as physicians (65.5%) and more than a quarter (27.7%) as scientists. o Respondents were instructed to circle all professions that applied and 103% indicated that they were both physicians and scientists. 2. The majority of individual users of MEDLINE appear to be relatively young. o The majority received their highest degree within the past 18 years: -40.1% during 1970's - 27.1% during 1980's 3. More than two thirds of respondents (68.6%) indicated they do all searches by themselves. o On average, they perform 43 searches/month by themselves. 4. Lack of time was the most frequent reason given for having someone else perform MEDLINE searches (by 593% of those who have others search for them) 5. The two main reasons given for performing their own searches were: o greater familiarity with subject matter (47.9%) o ability to get search results faster (322%) 6. Among all reasons selected (regardless of rank assigned), two-thirds of the respondents (653%) indicated enjoyment of searching as a factor. 7. The majority of respondents have had their codes for two years or less (90.1%). The majority of the respondents also do not share their code with anyone else (82%). 8. The majority of respondents (843%) feel they are somewhat experienced or not very experienced in the use of online databases. 9. For over half of all respondents (57.7%), online usage has remained about the same since receiving their access codes. o 30% indicated usage has increased o 123% indicated usage has decreased 10. Cost seems not to be of overwhelming importance in individuals' use of MEDLINE. Tne majority (67.6%) indicated that cost considerations rarely (26.6%) or never (41.0%) keep them from performing a search. 11. The overwhelming majority (96.0%) say they most often search for a subject. 81.7% indicated they typically search to satisfy an immediate information need. 24 12. Those whose primary use of MEDLINE search information is in research/testing are more likely to want all relevant citations (67.1%) compared to those whose primary use of MEDLINE search information is in education (533%) or patient care (49.8%). 13. Those in group and private/solo practice are somewhat more likely to feel that less than half of the citations they typically retrieve are relevant. (63.4% of those in group practice and 61.1% of those m private/solo practice compared to 58.6% of all respondents.) 14. Almost two-thirds of all respondents (60.8%) indicated MEDLINE citations are acceptable in their current form (i.e., answered "NO" to additional information needed in a citation). o of the 39.2% who said additional types of information would be valuable, full text was overwhelmingly chosen as the single most valuable piece of information not presently available in MEDLINE citations. 15. Eighty percent of respondents expressed an overall satisfaction with MEDLINE. o Individuals also are satisfied with their chosen method of accessing MEDLINE 16. Two-thirds of the respondents (64.1%) took less than 10 minutes on a typical search. o majority of respondents (83.4%) feel that length of time to conduct a search on MEDLINE is reasonable. 17. Of the group of individuals who indicated research/testing as one of their uses of MEDLINE information, 613% ranked it as their most common use. Of those who indicated patient care, 52.8% ranked it as the most common use. 18. A somewhat larger portion of the physician group (30.1%) always use GRATEFUL MED form screens as compared to the other professional groups (other health professional 24.8%, scientist 21%, nurse 183% and student 17.1%). o one third of respondents in private/solo and group practice always use GRATEFUL MED form screens to search. TABLE 1 PROFESSION OF RESPONDENTS N = 2710 Frequency Total 3219 Percent* Physician 1776 65.5% Nurse 99 3.7% Other Health Professional 180 6.6% Scientist 753 27.8% Student 118 4.4% Librarian/Info Specialist 86 3.2% Other* 207 7.6% No answer: *Although 2710 individuals answered this questions, some indicated more than one profession; therefore, the frequency column is greater than 2710 and the percent column exceeds 100%. Table 2 Specialty of Survey Respondent N = 2197* Specialty of Respondent Internal Medicine Medical Oncology Pathology Pediatrics Family Practice Surgery Psychiatry Cardiovascular Disease Hematology Anesthesiology Radiology Dermatology Neurology and/or Child Neurol. Pulmonary Disease Obstetrics and Gynecology Gastroenterology Ophthalmology Endocrinology and Metabolism Infectious Disease Orthopaedic Surgery Neurological Surgery Emergency Medicine Nephrology Psychiatry and Neurology Allergy Urology Rheumatology Pediatric Hematology-Oncology Pediatric Endocrinology Physical Medicine and Rehab. Neonatal-Perinatal Medicine Pediatric Nephrology Nuclear Medicine Otolaryngology Plastic Surgery Preventive Medicine Thoracic Surgery Pediatric Cardiology Colon and Rectal Surgery Blood Banking Child Psychiatry Other Frequency % of Cases 282 12.8 147 6.7 125 5.7 109 5.0 99 4.5 95 4.3 85 3.9 75 3.4 64 2.9 58 2.6 56 2.5 53 2.4 53 2.4 51 2.3 50 2.3 44 2.0 44 2.0 40 1.8 39 1.8 35 1.6 29 1.3 25 1.1 22 1.0 21 1.0 20 0.9 18 0.8 16 0.7 13 0.6 11 0.5 10 0.5 8 0.4 8 0.4 7 0.3 7 0.3 7 0.3 6 0.3 6 0.3 5 0.2 4 0.2 3 0.1 1 0.0 469 21.3 Total 2320 No answer: 572 *Some respondents indicated more than one specialty; therefore, precent exceeds 100 and frequency exceeds N of 2197. TABLE 3 HIGHEST DEGREE RECEIVED N=2733** % of Frequency* Cases M.D. 1760 64.4 Ph.D. 570 20.9 Masters 222 8.1 Bachelor's 149 5.5 Pharm.D. 39 1.4 Nursing 33 1.2 Dentistry 30 1.1 D.O. 23 .8 Other Doctorate 21 .7 Other 17 .6 DVM 15 .5 High School 9 .3 * This question allowed for multiple responses. **The total number of respondents exceeds 2716 because this question was answered by some individuals who were later determined to be ineligible and were then excluded from further computations. TABLE 4 YEAR RECEIVED HIGHEST DEGREE N=2667 Cumulative Frequency Percent Percent Pre-1950 83 3.1 3.1 1950s 270 10.1 13.2 1960s 523 19.6 32.8 1970s 1069 40.1 72.9 1980s 722 27.1 100.0 Total 2667 No answer: 49 TABLE 5 PROFESSION* BY YEAR HIGHEST DEGREE RECEIVED Physician (n=1776) Nurse (n=99) Other Health Professional (n=180) Scientist (n=753) Student (n=118) Librarian/ Info Spec. (n=86) Other (n=207) Row Total Pre-1950 n=83 71 4.0% -- 1 .6% 18 2.4% -- 2 2.4% 5 2.6% 97 3.1% 1950s n=270 224 12.8% 2 2.0% 7 4.0% 72 9.7% 1 .9% 2 2.4% 13 6.7% 321 10.1% 1960s n=523 378 21.5% 6 6.1% 23 13.0% 156 21.0% 5 4.4% 9 10.8% 44 22.6% 621 19.6% 1970s n=1069 707 40.3% 38 38.4% 76 42.9% 307 ' 41.3% 25 21.9% 36 43.4% 72 36.9% 1261 40.1% 1980s n=722 376 21.4% 53 53.5% 70 39.5% 190 25.6% 83 72.8% 34 41.0% 61 31.3% 867 27.1% Column Total 1756 100.0% 99 100.0% 177 100.0% 743 100.0% 114 100.0% 83 100.0% 195 100.0% 3167* 100.0% No answer 20 0 3 10 4 3 12 *The number of respondents for each profession is based on a question which allowed multiple responses; therefore, the grand total of 3167 is greater than our population of 2716. The cross tab is based on two sets of data with different N's. The n's reported here reflect the // of responses to the individual question. TABLE 6 PRIMARY WORKPLACE OF RESPONDENTS N=2701 Frequency 340 Percent Cumulative Percent Private/Solo Practice 12.6 12.6 Group Practice 331 12.3 24.8 Hospital/Clinic 395 14.6 39.4 College/Univ. Medical School 1264 46.8 86.3 Private Company/ Business 89 3.3 89.6 Government Agency 145 5.4 94.9 No Formal Workplace 28 1.0 96.0 Other Workplace 109 4.0 100.0 No answer: 15 TABLE 7 PROFESSION* BY PRIMARY WORKPLACE Physician (n-1776) Nurse (n»99) Other Health Professional (n-180) Scientist (n-753) Student (n-118) Librarian/ Info Spec. (n=86) Other (n=207) Row Total Private/Solo Practice n=340 Group Practice n=331 295 16.6X 314 17.7% 3 3.0% 6 6.3% 16 < 8.9% 7 3.9% 12 1.6% 6 .8% 2 1.7% 5 5.9% 2 2.4% 28 13.7% 6 2.9% 359 12.6% 343 12.3% Hospital/Clinic n-395 273 15.4% 32 32.3% 44 24.6% 34 4.5% 17 14.5% 24 28.2% 28 13.7% 452 14.6% Coll/Univ/Med School n=1264 717 40.4% 41 41.4% 82 45.8% 602 80.3% 79 67.5% 14 16.5% 80 39.2% 1615 46.8% Private Company/ Business n=89 Government Agency n=145 21 1.2% 102 5.8% 3 3.0% 2 2.0% 14 7.8% 4 2.2% 31 4.1% 48 6.4% 1 .9% 1 .9% 11 12.9% 4 4.7% 21 10.3% 12 5.9% 102 3.3% 171 5.4% No Formal Workplace n=28 4 .2% 5 5.1% 3 1.7% -- 16 13.7% 6 7.1% 6 2.9% 40 1.0% Other Work Place n=109 47 2.7% 7 7.1% 9 5.0% 17 2.3% 1 .9% 19 22.4% 23 11.3% 123 4.0% Column Total 1773 100.0% 99 100.0% 179 100.0% 750 100.0% 117 100.0% 85 100.0% 204 100.0% 3207 100.0% No answer 3 0 1 3 1 1 3 *The number of respondents for each profession is based on a question which allowed multiple responses; therefore, the grand total of 3207 is greater than our population of 2716. The cross tab is based on two sets of data with different N's. The n's reported here reflect the // of responses to the individual question. TABLE 8 # OF SEARCHES PERFORMED BY RESPONDENTS IN A TYPICAL MONTH N=2686 Cumulative Frequency Percent Percent None 323 12.0 12.0 One 570 21.2 33.2 Two 523 19.5 52.7 Three 355 13.2 65.9 Four 249 9.3 75.2 Five 242 9.0 84.2 Six or more 424 15.8 100.0 Total 2686 No answer: 30 TABLE 9 // OF SEARCHES PERFORMED BY OTHERS IN A TYPICAL MONTH N=2682 Cumulative Frequency Percent Percent None 1842 68.6 68.6 One 405 15.1 83.7 Two 197 7.3 91.0 Three 78 2.9 93.9 Four 54 2.0 95.9 Five 55 2.0 98.0 Six or more 55 2.0 100.0 Total 2682 No answer: 30 TABLE 10-A OTHERS WHO PERFORM SEARCHES N=1103 Librarian/Info. Specialist Student/Research Assistant Secretary/ Admin. Assist. Colleague Family Member Other Total 11<>3 No answer: 1613* *Note that not all respondents were eligible to answer this question since many never used a search intermediary. Cumulative Frequency Percent Percent 903 81.9 81.9 58 5.3 87.1 40 3.6 90.8 51 16 35 4.6 95.4 1.5 96.8 3.2 100.0 TABLE 10-B OTHERS WHO PERFORM SEARCHES BY PRIMARY WORKPLACE Librarian/ Info Spec Student/ Res Assn Secretary/ Admin Assn Colleague Family Member Other Column Total Priv/Solo Practice (n=340) 118 85.5% 2 1.4* 6 4.3% 2 1.4% 1 .7% 9 6.5% 138 100.0% Group Practice (n=331) 144 89.4% 2 1.2% 5 3.1% 5 3.1% 3 1.9% 2 1.2% 161 100.0% Coll/Univ/ Med School (n»1264) 352 74.1% 50 10.5% 17 3.6% 32 6.7% 6 1.3% 18 3.8% 475 100.0% Hospital/ Clinic (n«395) Priv Co/ Business (n-89) Gov't Agency (n-145) No Formal Workplace (n«28) Other Workplace (n=109) Row Total 170 89.5% 26 83.9% 61 89.7% 4 80.0% 27 81.8% 902 81.9% -- 1 3.2% 2 2.9% -- -- 57 5.2% 8 4.2% 1 3.2% -- -- 3 9.1% 40 3.6% 5 2.6% 1 3.2% 4 5.9% 1 20.0% 1 3.0% 51 4.6% 4 2.1% 1 3.2% 1 1.5% -- — 16 1.5% 3 1.6% 1 3.2% -- -- 2 6.1% 35 3.2% 190 100.0% 31 100.0% 68 100.0% 5 100.0% 33 100.0% 1101 100.0% No answer 202 170 789 205 58 77 23 76 TABLE 11-A REASONS FOR HAVING OTHERS SEARCH MEDLINE N=1151 When someone else can do it as easily as I can When I don't have time to do it myself When I need different expertise/ system knowledge When my search results have not been satisfactory Other FREQUENCY PERCENT* 220 19.1% 682 59.3% 445 38.7% 317 27.5% 137 11.9% Total 1801 No answer 1565 *Respondents were instructed to circle all reasons that applied; therefore, percentages total to more than 100% and frequencies total more than the N of 1151. TABLE 11-B REASONS TO HAVE OTHERS SEARCH BY PRIMARY WORKPLACE Priv/Solo Practice (n=340) Group Practice (n-331) Coll/Univ/ Med School (n-1264) When someone else can do it as easily as I can 25 17.2% 38 23.5% 99 19.8% When I don't have time to do it myself 84 57.9% 104 64.2% 280 56.0% When I need different 46 expertise/system 31.7% knowledge 67 41.4% 198 39.6% When my search results 41 have not been 28.3% satisfactory 54 33.3% 126 25.2% Other 18 12.4% 11 6.8% 66 13.2% ospital/ Clinic (n-395) Priv Co/ Business (n-89) Gov't Agency (n-145) 33 16.8% 7 21.9% 11 15.3% 133 67.9% 18 56.3% 39 54.2% 68 34.7% 18 56.3% 30 41.7% 53 27.0% 15 46.9% 17 23.6% 19 9.7% 4 12.5% 14 19.4% No Formal Other Workplace Workplace (n-28) (n«109) 6 17.1% 5 71.4% 3 42.9% 2 28.6% 18 51.4% 14 40.0% 9 25.7% 4 11.4% Total Respondents 145 162 500 196 32 72 35 No answer 195 169 764 199 57 73 21 74 ^Respondents were asked to circle all reasons which applied; therefore, percentages exceed 100 and frequencies exceed number of total respondents for each column. TABLE 12 SATISFACTION WITH SEARCHES DONE BY OTHERS N=1121 Satisfied Frequency Percent Cumulative Percent 1 383 34.2 34.2 2 408 36.4 70.6 Neutral 3 230 20.5 91.1 Dissatisfi .ed 4 86 7.7 98.8 5 14 1.2 100.0 Total 1121 No answer: 1595* *Note that not all respondents were eligible to answer this question, since many never used a search intermediary. TABLE 13 REASONS FOR DISSATISFACTION WITH SEARCHES DONE BY OTHERS N=150 FREQUENCY PERCENT* Inconvenient location 17 11.3% Inconvenient hours 24 16.0% Have to wait for search 59 39.3% Cost 36 24.0% Unsatisfactory results 93 62.0% Other 46 30.7% Total 150 No answer 2566 ^Respondents were instructed to circle all reasons that applied; therefore, percentages exceed 100% and frequencies are greater than the N of 150. Note that not all respondents were eligible to answer this question since many never used a search intermediary. TABLE 14 SATISFACTION WITH SEARCHES DONE BY OTHERS BY PRIMARY WORKPLACE Priv/Solo Practice (n=340) Group Practice (n-331) Coll/Univ/ Med School (n-1264) Satisfied 1 54 40.9% 60 38.0% 151 30.6% 2 44 33.3% 57 36.1% 187 37.9% Neutral 3 25 18.9% 28 17.7% 114 23.1% Dissatisfied 4 7 5.3% 11 7.0% 36 7.3% 5 2 1.5% 2 1.3% 6 1.2% Column Totals 132 100.0% 158 100.0% 494 100.0% No answer 208 173 770 Hospital/ Clinic (n-395) Priv Co/ Business (n-89) Gov't Agency (n-145) No Formal Workplace (n-28) Other Workplace (n=109) Row Total 63 32.3% 11 35.5% 30 41.7% 1 25.0% 11 33.3% 381 34.0% 65 33.3% 14 45.2% 28 38.9% 1 25.0% 12 36.4 408 36.5% 45 23.1% 2 6.5% 9 12.5% 1 25.0% 6 18.2% 230 20.6% 19 9.7% 3 9.7% 5 6.9% 1 25.0% 4 12.1% 86 7.7% 3 1.5% 1 3.2% -- -- -- 14 1.3% 195 100.0% 31 100.0% 72 100.0% 4 100.0% 33 100.0% 1119 100.0% 200 58 73 24 76 TABLE 15 FACTORS WHICH INFLUENCE INDIVIDUALS TO DO THEIR OWN SEARCHING More familiar with subject than intermediary I get the information faster I enjoy searching More cost effective than intermediary No intermediary was available Other Total N=2171 INFLUENTIAL FACTOR FREQUENCY PERCENT* 1805 1788 1423 687 319 129 6151 83.1% 82.4% 65.5% 31.6% 14.7% 5.9% N=2074** MOST INFLUENTIAL FACTOR FREQUENCY PERCENT 994 667 156 62 96 99 2074 47.9% 32.2% 7.5% 3.0% 4.6% 4.8% 100.0% -'•Multiple reasons were allowed. Total % exceeds 100% and frequency exceeds N of 2171. **Not all respondents indicated a most influential factor. TABLE 16 EXPERIENCE WITH ONLINE DATABASES N=2561 Cumulative Frequency Percent Percent Very Experienced 212 8.3 8.3 Somewhat Experienced 1244 48.6 56.9 Not Very Experienced 920 35.9 92.8 Not Experienced 185 7.2 100.0 Total 2561 No answer: 155 TABLE 17 EXPERIENCE WITH ONLINE DATABASES BY PRIMARY WORKPLACE Priv/Solo Practice (n=340) Group Practice (n-331) Coll/Univ/ Med School (n-1264) Very Experienced 34 10.8% 19 6.1% 78 6.5% Somewhat Experienced 128 40.6% 137 43.9% 644 • 53.7% Not very Experienced 121 38.4% 128 41.0% 418 34.8% Not Experienced 32 10.2% 28 9.0% 60 5.0% Column Totals 315 100.0% 312 100.0% 1200 100.0% Hospital/ Clinic (n-395) Priv Co/ Business (n-89) Gov't Agency (n-145) No Formal Workplace (n-28) Other Workplace (n-109) Row Total 34 9.2% 15 17.2% 14 10.2% 3 11.5% 14 13.0% 211 8.3% 160 43.5% 43 49.4% 69 50.4% 12 46.2% 47 43.5% 1240 48.6% 138 37.5% 26 29.9% 41 29.9% 7 26.9% 39 36.1% 918 36.0% 36 9.8% 3 3.4% 13 9.5% 4 15.4% 8 7.4% 184 7.2% 368 100.0% 87 100.0% 137 100.0% 26 100.0% 108 100.0% 2553 100.6% No answer 25 19 64 27 TABLE 18 LENGTH OF TIME HAVE HAD MEDLINE CODE N=2509 Cumulativ Frequency Percent Percent 6 Months or less 808 32.2 32.2 7 to 12 Months 776 31.0 63.1 13 to 18 Months 212 8.4 71.6 19 to 24 Months 464 18.5 90.1 25 to 30 Months 47 1.9 91.9 31 to 36 Months 114 4.6 96.5 37 to 42 Months 10 .4 96.9 More than 42 Months 78 2.7 100.0 Total 2509 No answer: 264 TABLE 19 LENGTH OF TIME HAVE HAD MEDLINE CODE BY PRIMARY WORKPLACE Priv/Solo Practice (n-340) Group Practice (n-331) Coll/Univ/ Med School (n-1264) Hospital/ Clinic (n-395) Priv Co/ Business (n-89) Gov't Agency (n-145) No Formal Workplace (n-28) Other Workplace (n-109) Row Total 6 Months or less 110 35.9% 105 34.3% 354 30.3% 121 33.6% 19 22.1% 32 23.4% 11 44.0% 40 37.0% 792 31.7% 7 to 12 Months 94 30.7% 98 32.0% 360 30.8% 107 29.7% 36 41.9% 49 35.8% 9 36.0% 34 31.5% 787 31.5% 13 to 18 Months 21 6.9% 14 4.6% 117 10.0% 34 9.4% 6 7.0% 8 5.8% -- 11 10.2% . 211 8.4% 19 to 24 Months 41 13.4% 51 16.7% 247 21.1% 62 17.2% 14 16.3% 33 24.1% 4 16.0% 12 11.1% 46*4 18.6% 25 to 30 Months 3 1.0% 3 1.0% 28 2.4% 8 2.2% 2 2.3% 3 2.2% -- -- 47 1.9% 31 to 36 Months 20 6.5% 16 5.2% 42 3.6% 17 4.7% 3 3.5% 9 6.6% 1 4.0% 4 3.7% 112 4.5% 37 to 42 Months -- 2 .7% 4 .3% -- 2 2.3% 1 .7% -- 1 .9% 10 .4% More than Months 42 17 5.6% 17 5.6% 18 1.5% 11 3.1% 4 4.7% 2 1.5% 6 5.6% 75 3.0% Column Totals 306 100.0% 306 100.0% 1170 100.0% 360 100.0% 86 100.0% 137 100.0% 25 100.0% 108 100.0% 2498 100.0% 25 94 35 TABLE 20 NUMBER OF PEOPLE SHARING MEDLINE CODE* N=2661 Frequency Percent Cumulative Percent None 17 .6 .6 One 2183 82.0 82.7 Two 229 8.6 91.3 Three to Five 167 6.3 97.6 Six or more 65 2.4 100.0 Total 2661 No answer: 55 ^'Respondents, were instructed to write "1" if they were the only person to use the code. TABLE 21 USAGE OF 1 MEDLINE N=2545 Cumulative Usage Has: Frequency Percent Percent Increased 763 30.0 30.0 Stayed the Same 1469 57.7 87.7 Decreased 313 12.3 100.0 Total 2545 No answer: 171 TABLE 22-A REASONS GIVEN FOR INCREASE IN USE OF MEDLINE ______ BY WHETHER USE HAS INCREASED OR DECREASED N=678* Frequency Percent Lack of time 2 .3% Increases productivity 124 18.3% The more I learn, the 25 3.7% more I use it Search for other people 18 2.7% Enjoy using 13 1.9% Increased familiarity 292 43.1% Increased work load/need 169 24.9% Money reasons 8 1.2% Other services unavailable 9 1.3% Better than others 12 1.8% Change in research needs 6 Change from different systems 4 .6% GRATEFUL MED 26 3.8% Saves time (easier to use) 11 1.6% Better computer hardware 3 .4% Total 722 *0f the 763 individuals who indicated that their usage had increased 678 indicated reason(s) why this was so. Content of respondent answers was sometimes coded with more than one code; therefore, total % exceeds 100% and frequencies are greater than the N of 678. TABLE 22-B REASONS GIVEN FOR DECREASE IN USE OF MEDLINE BY WHETHER USE HAS INCREASED OR DECREASED N-288 Lack of time Decreased work load/need Money reasons Don't like using Confused when using Like others better Use other services Change in research needs GRATEFUL MED Hardware inoperable (bad telephone lines) Forget that it exists Limited access to computer Frequency Percent 77 26.7% 71 24.7% 30 10.4% 25 8.7% 28 9.7% 10 3.5% 19 6.6% 10 3.5% 3 1.0% 15 5.2% 1 .3% 2 .7% Total 291 *0f the 313 individuals who indicated that their usage had decreased, 288 indicated reason(s) why this was so. Content of respondent answers was coded with more than one code in a few instances; therefore, total % slightly exceeds 100% and frequencies are greater than the N of 288. TABLE 23 EFFECT OF COST CONSIDERATIONS ON SEARCHING N*2554 Cumulative Frequency Percent Percent Never a consideration 1047 41.0 41.0 Rarely a consideration 680 26.6 67.6 Occasionally a consideration 591 23.1 90.8 Frequently a consideration 236 9.2 100.0 Total 2554 No answer: 162 TABLE 24 EFFECT OF COST CONSIDERATIONS ON SEARCHING BY PRIMARY WORKPLACE Priv/Solo Practice (n=340) Group Practice (n-331) Coll/Univ/ Med School (n-1264) Never 152 48.4% 166 53.7% 429 35.8% Rarely 76 24.2% 73 23.6% 338 28.2% Occasionally 57 18.2% 52 16.8% 321 26.8% Frequently 29 9.2% 18 5.8% 110 9.2% Column 314 Totals 100.0% 309 100.0% 1198 100.0% No answer 26 22 66 Hospital/ Clinic (n-395) Priv Co/ Business (n-89) Gov't Agency (n-145) No Formal Workplace (n-28) 140 38.1% 43 49.4% 55 40.1% 8 30.8% 100 27.2% 26 29.9% 36 26.3% 6 23.1% 79 21.5% 17 19.5% 33 24.1% 7 26.9% 48 13.1% 1 1.1% 13 9.5% 5 19.2% 367 100.0% 87 100.0% 137 100.0% 26 100.0% Other Row Workplace Total (n-109) 52 1045 48.1% 41.0% 21 676 19.4% 26.6% 23 589 21.3% 23.1% 12 236 11.1% 9.3% 108 2546 100.0% 100.0% 28 TABLE 25 REASONS FOR WHICH INDIVIDUALS TYPICALLY SEARCH N=2556 Frequency Percent Cumulative Percent Immediate Information 2087 81.7 81.7 Stay Current 263 10.3 91.9 Learn New Areas 123 4.8 96.8 Other 83 3.2 100.0 Total 2556 No answer: 160 TABLE 26 REASONS FOR WHICH INDIVIDUALS TYPICALLY SEARCH BY PRIMARY WORKPLACE Priv/Solo Practice (n-340) Group Practice (n-331) Coll/Univ/ Med School (n-1264) Hospital/ Clinic (n-395) Priv Co/ Business (n-89) Gov't Agency (n-145) No Formal Workplace (n-28) Other Workplace (n-109) Row Total Immediate Information 266 83.6% 273 88.3% 944 79.1% 304 82.6% 68 78.2% 114 83.2% 19 73.1% 92 85.2% 2080 81.7% Stay Current 24 7.5% 23 7.4% 150 12.6% 29 7.9% 15 17.2% 12 8.8% 4 15.4% 5 4.6% 262 10.3% Learn New Areas 17 5.3% 6 1.9% 69 5.8% 15 4.1% 3 3.4% 9 6.6% 1 3.8% 2 1.9% 122 4.8% Other 11 3.5% 7 2.3% 31 2.6% 20 5.4% 1 1.1% 2 1.5% 2 7.7% 9 8.3% 83 3.3% Column Totals 318 100.0% 309 100.0% 1194 100.0% 368 100.0% 87 100.0% 137 100.0% 26 100.0% 108 100.0% 2547 100.0% No answer 22 22 70 27 2 8 2 1 ■v TABLE 27 # OF CITATIONS USUALLY INTERESTED IN RETRIEVING N=2550 -Cumulative Frequency Percent Percent A Few citations 973 38.2 38.2 All Relevant 1503 58.9 97.1 Citations 0ther 74 2.9 100.0 Total 2550 No answer: 166 TABLE 28 AMOUNT OF INFORMATION RESPONDENTS TYPICALLY INTERESTED IN RETRIEVING BY PURPOSE OF THE SEARCH* Patient Care Row As most common As 2nd most common As 3rd most common Total Typically interested in Freq % Freq % Freq % Freq % receiving A Few Citations 469 47.6% 191 32.5% 82 27.5% 742 39.7% All Relevant Citations 491 49.8% 377 64.2% 204 68.5% 1072 57.3% Other 25 2.5% 19 3.2% 12 4.0% 56 3.0% Total 985 100.0% 587 100.0% 298 100.0% 1870 100.0% Education * Row As most common As 2nd most common As 3rd most common Total . Typically interested in Freq % Freq % Freq % Freq % receiving A Few Citations 135 42.3% 409 41.0% 141 30.1% 685 38.4% All Relevant Citations 170 53.3% 570 57.1% 311 66.5% 1051 58.9% Other 14 4.4% 19 1.9% 16 3.4% 49 2.7% Total 319 100.0% 998 100.0% 468 100.0% 1785 100.0% TABLE 28 (cont.) AMOUNT OF INFORMATION RESPONDENTS TYPICALLY INTERESTED IN RETRIEVING BY PURPOSE OF THE SEARCH Typically interested in receiving A Few Citations All Relevant Citations Other Total As most Freq common % 361 30.5% 794 67.1% 29 2.4% 1184 100.0% Research/Testing As 2nd most common As 3rd most common Freq________% Freq_________% 136 33.5% 253 62.3% 17 4.2% 406 100.0% 152 45.6% 173 52.0% 8 2.4% 333 100.0% Row Total Freq_______% 649 33.7% 1220 63.4% 54 2.8% 1923 100.0% *Patlent Care. Education, and Research/Testing are each analyzed Independently of one another. TABLE 29 NUMBER OF CITATIONS INTERESTED IN RETRIEVING BY PRIMARY WORKPLACE Priv/Solo Practice (n-340) Group Practice (n=331) Coll/Univ/ Med School (n-1264) A Few Citations 139 44.1% 155 50.5% 386 32.4% All Relevant 170 Citations 54.0% 146 47.6% 778 65.2% Other 6 1.9% 6 2.0% 29 2.4% Column Totals 315 100.0% 307 100.0% 1193 100.0% Hospital/ Clinic (n-395) Priv Co/ Business (n-89) Gov't Agency (n-145) No Formal Workplace (n-28) Other Workplace (n-109) Row Total 146 39.7% 38 43.7% 52 38.0% 10 38.5% 43 39.8% 969 38.1% 204 55.4% 46 52.9% 81 59.1% 15 57.7% 58 53.7% 1498 59.0% 18 4.9% 3 3.4% 4 2.9% 1 3.8% 7 6.5% 74 2.9% 368 100.0% 87 100.0% 137 100.0% 26 100.0% 108 100.0% 2541 100.0% No answer 25 24 71 27 TABLE 30 NUMBER OF CITATIONS TYPICALLY RETRIEVED N-2510 Frequency Too Few Citations 385 The Right Number 1533 Too Many Citations 592 Valid Cumulative Percent Percent 15.3 15.3 61.1 76.4 23.6 100.0 Total 2510 No answer: 206 TABLE 31 NUMBER OF CITATIONS TYPICALLY RETRIEVE PRIMARY WORKPLACE Priv/Solo Practice (n-340) Group Practice (n-331) Coll/Univ/ Med School (n-1264) Hospital/ Clinic (n-395) Priv Co/ Business (n-89) Gov't Agency (n-145) No Formal Workplace (n-28) Other Workplace (n-109) Row Total Too Few Citations 67 21.7% 63 20.6% 152 12.9% 51 14.3% 20 23.5% 18 13.6% 1 3.8% 11 10.5% 383 15.3% The Right Number 176 57.0% 182 59.5% 734 62.2% 217 60.8% 42 49.4% 92 69.7% 19 73.1% 68 64.8% 1530 61.2% Too Many Citations 66 21.4% 61 19.9% 295 25.0% 89 24.9% 23 27.1% 22 16.7% 6 23.1% 26 24.8% 588 23.5% Column Totals 309 100.0% 306 100.0% 1181 100.0% 357 100.0% 85 100.0% 132 100.0% 26 100.0% 105 100.0% 2501 100.0% No answer 31 25 83 38 4 13 2 4 « TABLE 32 METHOD OF ACCESS BY NUMBER OF CITATIONS TYPICALLY RECEIVED Access Method Always use: Direct Command Language GM Form Screens Too Few The Right Citations Number Freo % Freq % 107 10.2 676 64.8 153 23.6 369 56.9 Too Many Citations Freq % Row Total Freq 261 25.0 1044 100 127 19.6 649 100 TABLE 33 PERCENTAGE OF CITATIONS RELEVANT TO SEARCH QUERY N=2399 Cumulative Frequency Percent Percent 0% to 25% 566 23.6 23.6 26% to 50% 841 35.1 58.6 51% to 75% 541 22.6 81.2 76% to 100% 451 18.8 100.0 Total 2399 No answer: 317 TABLE 34 PERCENTAGE OF CITATIONS RELEVANT TO SEARCH QUERY BY PRIMARY WORKPLACE Priv/Solo Practice (n-340) Group Practice (n-331) Coll/Univ/ Med School (n-1264) Hospital/ Clinic (n-395) Priv Co/ Business (n-89) Gov't Agency (n-145) No Formal Workplace (n-28) Other Workplace (n-109) Row Total 0 TO 25% 77 27.2% 73 25.4% 258 22.6% 83 24.3% 22 26.8% 23 17.8% 7 31.8% 21 20.6% 564 23.6% 26 TO 50% 96 33.9% 109 38.0% 423 37.0% 111 32.5% 20 24.4% 45 34.9% 5 22.7% 28 27.5% 837 35.0% 51 TO 75% 55 19.4% 55 19.2% 263 23.0% 86 25.1% 20 24.4% 27 20.9% 4 18.2% 31 30.4% 541 22.6% 76 TO 1003 ! 55 19.4% 50 17.4% 199 17.4% 62 18.1% 20 24.4% 34 26.4% 6 27.3% 22 21.6% 448 18.7% Column Totals 283 100.0% 287 100.0% 1143 100.0% 342 100.0% 82 100.0% 129 100.0% 22 100.0% 102 100.0% 2390 100.0% No answer 57 44 121 53 16 TABLE 35 TYPE OF SEARCH TYPICALLY DONE N=2552 Cumulative Frequency Percent Percent An Author 81 3.2 3.2 A Journal Title 22 .9 4.0 A Subject 2449 96.0 100.0 Total 2552 No answer: 164 TABLE 36-A AREAS IN WHICH SEARCH INFORMATION IS USED BY INDIVIDUALS (Includes ranking of 1, 2, or 3) N-2716 Frequency Percent Patient Care 1877 69.1% Education 1793 66.0% Research/Testing 1929 71.0% Management/ Administration 201 7.4% Regulation 79 2.9% Other 110 4.0% TABLE 36-B MOST COMMON AREA FOR WHICH SEARCH INFORMATION IS USED BY INDIVIDUALS N=2592 Frequency Percent Patient Care 991 38.2% Education 321 12.4% Research/Testing 1186 45.8% Management/ 30 1.2% Administration Regulation 9 .3% Other 55 2.1% Total 2592 100.0% No answer 124 TABLE 36-C AREAS IN WHICH SEARCH INFORMATION IS USED BY INDIVIDUALS (Breakout of usage within each area of interest) Row Most Common 2nd Most Common 3rd Most Common Total % Freq % Freq. % Frea % Freq Patient Care 991 52.8 588 31.3 298 15.9 1877 100.0 Education 321 17.9 1003 55.9 469 26.2 1793 100.0 Research/Testing 1186 61.5 407 21.1 336 17.4 1929 100.0 Management/Administration 30 14.9 67 33.3 104 51.7 201 100.0 q 11.4 30 38.0 40 50.6 79 100.0 Regulation -* 55 50.0 23 20.9 32 29.1 110 100.0 Other Patient 1* 1 Care ,2 or 3 EducatJ 1 1, on 2 or 3 Pvt/Solo Practice** (N-340) 234 68.82% 293 86.18% 36 10.59% 243 71.47% Group Practice (N-331) 251 75.83% 308 93.05% 28 8.47% 243 73.41% Coll/Univ Med School (N-1264) 208 16.45% 729 57.67% 130 10.28% 800 63.29% Hospital/ Clinic (N-395) 199 50.38% 336 85.06% 76 19.24% 301 76.20% Pvt Co/ Business (N-89) 17 19.10% 45 50.56% 14 15.73% 36 40.45% Government Agency (N-145) 31 21.38% 82 56.55% 18 12.41% 83 57.24% No Formal Workplace (N-28) 9 32.14% 18 64.29% 4 14.29% 15 53.57% Other Workplace 39 35.78% 61 55.96% 15 13.76% 68 62.39% * 1 - Most Common; 1, 2, Or 3 **%s are computed as percent of Combined Ranking total respondents in TABLE 37 PRIMARY WORKPLACE BY PURPOSE OF SEARCH Research/Testing I 1,2 or 3 42 139 12.35% 40.88% 30 142 9.06% 42.90% 846 1126 66.93% 89.08% 85 243 21.52% 61.52% 47 66 52.80% 74.16% 82 116 56.55% 80.00% 11 21 39.29% 75.00% 38 69 34.86% 63.30% Mgt/Adn 1 1 in ,2 or 3 3 0.08% 23 6.76% 3 0.91% 27 8.16% 9 0.71% 42 3.32% 4 1.01% 48 12.15% 1 1.12% 21 23.60% 3 2.07% 17 11.52% -- 2 7.14% 7 6.42% 21 19.27% Regulation 1 1.2 or 3 0.00% 14 4.12% 1 0.30% 6 1.81% 3 0.24% 25 1.98% — 5 1.27% 2 2.25% 13 14.61% 2 1.38% 9 6.21% 1 3.57% 7 25.00% each workplace TABLE 38 METHOD OF ACCESS USED TO SEARCH MEDLINE* BY PURPOSE OF THE SEARCH Direct Command Language Never Use Sometimes Use Always Use Total Patient Care Most Freq Common % 2nd Most Freq Common % 3rd Most Freq Common % 462 48.6 253 44.5 128 44.0 100 10.5 65 11.4 32 11.0 388 40.8 251 44.1 131 45.0 950 100.0 569 100.0 291 100.0 Row Total Freq % 843 46.6 197 10.9 770 42.5 1810 100.0 GM Form Screens Never Use Sometimes Use Always Use Total Patient Care Most *££fl Common % 2nd Most Freq Common % 3rd Most Freq Common % 457 48.1 303 53.3 163 56.2 193 20.3 127 22.4 57 19.7 300 31.6 138 • 24.3 70 24.1 950 100.0 568 100.0 290 100.0 Row Total Freq % 923 51.1 377 20.9 508 28.1 1808 100.0 TABLE 38 (cont) METHOD OF ACCESS USED TO SEARCH MEDLINE BY PURPOSE OF THE SEARCH Direct Command Language Never Use Sometimes Use Always Use Total Education Most Freq Common % 2nd Most Freq Common % 3rd Most Freq Common % 155 50.5 475 48.8 180 40.2 30 9.8 95 9.8 66 14.7 122 39.7 403 41.4 202 45.1 307 100.0 973 100.0 448 100.0 Row Total Freq % 810 46.9 191 11.1 727 42.1 1728 100.0 GM Form Screens Never Use Sometimes Use Always Use Total Education Most Freq Common % 2nd Most Freq Common % 3rd Most Freq Common % 152 49.7 500 51.5 239 53.5 68 22.2 202 20.8 106 23.7 86 28.1 269 27.7 102 22.8 306 100.0 971 100.0 447 100.0 Row Total Freq % 891 51.7 376 21.8 457 26.5 1724 100.0 TABLE 38 (cont) METHOD OF ACCESS USED TO SEARCH MEDLINE BY PURPOSE OF THE SEARCH Research/Test ing Direct Command Language Never Use Sometimes Use Always Use Total Most Freq Common % 2nd Most Freq Common % 3rd Most Freq Common % 488 42.8 165 42.3 149 45.7 132 11.6 49 12.6 33 10.1 520 45.6 176 45.1 144 44.2 1140 100.0 390 100.0 326 100.0 Row Total Frea % 802 43.2 214 11.5 840 45.3 1856 100.0 GM Form Screens Never Use Sometimes Use Always Use Total Research/Testing Most Freq Common % 2nd Most Common % 3rd Most Freq Common % 646 56.9 211 54.1 173 53.2 228 20.1 81 20.8 71 21.8 262 23.1 98 . 25.1 81 24.9 1136 100.0 390 100.0 325 100.0 Row Total Eisa % 1030 55.6 380 20.5 441 23.8 1851 100.0 TABLE 38 (cont) METHOD OF ACCESS USED TO SEARCH MEDLINE BY PURPOSE OF THE SEARCH Management/Administration Direct Command Language Never Use Sometimes Use Always Use Total Most Common Freq % 6 21.4 5 17.9 17 60.7 28 100.0 2nd Most Ziea Common % 3rd Most Freq Common % 23 35.9 48 47.5 11 17.2 12 11.9 30 ■ 46.9 41 40.6 64 100.0 101 100.0 Row Total Freq % 77 39 .9 28 14, .5 88 45, .6 193 100, .0 GM Form Screens Never Use Sometimes Use Always Use Management/Administration Most Freq Common % 2nd Most Common Freq % 3rd Most Freq Commoi % 19 67.9 36 57.1 49 48.5 6 21.4 12 19.0 25 24.8 3 10.7 15 23.8 27 26.7 Row Total Freq % 104 54.2 43 22.4 45 23.4 Total 28 100.0 63 100.0 101 100.0 192 100.0 TABLE 38 (cont) METHOD OF ACCESS USED TO SEARCH MEDLINE BY PURPOSE OF THE SEARCH Direct Command Language Never Use Sometimes Use Always Use Total Most Common Freq % 3 37.5 1 12.5 4 50.0 8 100.0 Regulation 2nd Most Common Freq % 8 28.6 3 10.7 17 60.7 28 100.0 3rd Most Common Freq % 12 33.3 8 22.2 16 44.4 36 100.0 Row Total Eisa % 23 31.9 12 16.7 37 51.4 72 100.0 GM Form Screens Never Use Sometimes Use Always Use Total Most Common Freq % 5 62.5 2 25.0 1 12.5 8 100.0 Regulation 2nd Most Common Freq % 18 64.3 5 17.9 5 17.9 28 100.0 3rd Most Common Freq % 20 55.6 9 25.0 7 19.4 36 100.0 Row Total Freq % 43 59.7 16 22.2 13 18.1 72 100.0 TABLE 39-A USE OF MESH N-2544 Cumulative Frequency Always 612 Usually 1109 Occasionally 470 Rarely 224 Never 129 Total 2544 Percent percent 24.1 24.1 43.6 67.6 18.5 86.1 8.8 94.9 5.1 100.0 No answer: 172 TABLE 39-B PERCEIVED "USEFULNESS OF MESH N-2475 Very Useful Useful Neutral Not Useful Not at all useful Cumulative Frequency Percent Percent 662 26.7 26.7 966 39.0 65.8 575 23.2 89.0 192 7.8 96.8 80 3.2 100.0 Total 2475 No answer: 241 TABLE 40-A USE OF MESH FOR SEARCHING SUBJECT BY PRIMARY WORKPLACE Priv/Solo Practice (n-340) Group Practice (n-331) Coll/Univ/ Med School (n-1264) Always 77 24.4% 87 28.4% 273 22.9% Usually 142 44.9% 137 44.8% 523 43.9% Occasionally 41 13.0% 53 17.3% 233 19.5% Rarely 31 9.8% 16 5.2% 112 9.4% Never 25 7.9% 13 4.2% 51 4.3% Total 316 100.0% 306 100.0% 1192 100.0% No answer 24 25 72 Hospital/ Clinic (n-395) Priv Co/ Business (n-89) Gov't Agency (n-145) No Formal Workplace (n-28) Other Workplace (n-109) Row Total 88 24.2% 19 21.8% 25 18.4% 7 26.9% 34 31.5% 610 24.1% 173 47.5% 35 40.2% 52 38.2% 10 38.5% 32 29.6% 1104 43.6% 59 16.2% 16 18.4% 31 22.8% 5 19.2% 31 28.7% 469 18.5% 26 7.1% 11 12.6% 16 11.8% 3 11.5% 8 7.4% 223 8.8% 18 4.9% 6 6.9% 12 8.8% 1 3.8% 3 2.8% 129 5.1% 364 100.0% 87 100.0% 136 100.0% 26 100.0% 108 100.0% 2535 100.0% 31 TABLE 40-B PERCEIVED USEFULNESS OF MESH BY PRIMARY WORKPLACE Priv/Solo Practice (n-340) Group Practice (n-331) Coll/Univ/ Med School (n-1264) Useful 1 94 31.2% 90 30.0% 281 24.2% 2 109 36.2% 120 40.0% 455 39.2% Neutral 3 65 21.6% 64 21.3% 295 25.4% Not Useful 4 17 5.6% 15 5.0% 103 8.9% 5 16 5.3% 11 3.7% 28 2.4% Total 301 100.0% 300 100.0% 1162 100.0% No answer 39 31 2 Hospital/ Clinic (n-395) Priv Co/ Business (n-89) 106 30.1% 20 23.5% 152 43.2% 31 36.5% 69 19.6% 21 24.7% 17 4.8% 11 12.9% 8 2.3% 2 2.4% 352 100.0% 85 100.0% 43 Gov't Agency (n-145) 27 20.1% 50 37.3% 31 23.1% 13 9.7% 13 9.7% 134 100.0% 11 No Formal Workplace (n-28) Other Workplace (n-109) Row Total 9 36.0% 33 30.6% 660 26.8% 7 28.0% 38 35.2% 962 39.0% 5 20.0% 24 22.2% 574 23.3% 4 16.0% 11 10.2% 191 7.7% -- 2 80 1.9% 3.2% 25 100.0% 108 100.0% 2467 100.0% TABLE 41 REASONS WHY INDIVIDUALS DO NOT USE MESH N«454 Unfamiliar with MeSH Cumbersome terms Terms too specific Terms too general No terms for my area Inappropriate retrieval Lose relevant citations Indexing inconsistency Text word is easier/better No manual/don't want to use Other Frequency Percent* 116 25.6% 107 23.6% 6 1.3% 49 10.8% 57 12.6% 11 2.4% 14 3.1% 7 1.5% 48 10.6% 45 9.9% 59 13.0% Total 519 '^Content of respondent answers was somtimes coded with more than one code; therefore, percentages exceed 100% and frequencies exceed the N of 454. Percentages used are based on the number of individuals responding to this question (454). TABLE 42 TYPICAL TIME TO SEARCH/REASONABLENESS OF TIME TO SEARCH Typical time to perform search on MEDLINE N-2533 Cumulative Frequency Percent Percent Less than 5 min. 578 22.8 22.8 5 to 10 min. 1047 41.3 64.2 10 to 15 min. 638 25.2 89.3 More than 15 min. 270 10.7 100.0 Total No answer: 2533 183 Reasonableness of time to search N-2526 Frequency Percent Too long 275 10.9 Reasonable amt. of time 2106 83.4 Quicker than expected 145 5.7 Total No answer: 2526 190 Cumulative Percent 10.9 94.3 100.0 TABLE 43 SATISFACTION WITH SEARCHES DONE BY SELF N-1318 Satisfied Frequency Percent Cumulative Percent 1 358 27.2 27.2 2 597 45.3 72.5 Neutral 3 257 19.5 92.0 Dissatisfied 4 77 5.8 97.8 5 29 2.2 100.0 Total 1318 No answer: 1398 TABLE 44 SATISFACTION WITH SEARCHES DONE BY SELF BY PRIMARY WORKPLACE Priv/Solo Practice (n-340) Group Practice (n-331) Coll/Univ/ Med School (n-1264) Hospital/ Clinic (n-395) Priv Co/ Business (n-89) Gov't Agency (n-145) No Formal Workplace (n-28) Other Workplace (n-109) Row Total Satisfied 1 45 27.6% 44 23.8% 155 26.8% 60 27.9% 8 20.5% 25 32.5% 4 40.0% 15 32.6% 356 27.1% 2 57 35.0% 79 42.7% 288 49.7% 92 42.8% 20 51.3% 36 46.8% 5 50.0% 18 39.1 595 45.3% Neutral 3 37 22.7% 42 22.7% 105 18.1% 41 19.1% 11 28.2% 10 13.0% 1 10.0% 10 21.7% 257 19.6% Dissatisfied 4 15 9.2% 14 7.6% 27 4.7% 14 6.5% -- 4 5.2% -- 3 6.5% 77 5.9% 5 9 5.5% 6 3.2% 4 .7% 8 3.7% -- 2 2.6% -- -- 29 2.2% Total 163 100.0% 185 100.0% 579 100.0% 215 100.0% 39 100.0% 77 100.0% 10 100.0% 46 100.0% 1314 100.0% No answer 177 146 685 180 68 Satisfied 1 2 Neutral Dissatisfied 4 5 TABLE 45 OVERALL SATISFACTION WITH MEDLINE N-2539 Cumulative Frequency Percent Percent 773 30.4 30.4 1293 50.9 81.4 367 90 16 14.5 3.5 .6 95.8 99.4 100.0 Total 2539 No answer: 177 TABLE 46 OVERALL SATISFACTION WITH MEDLINE BY PRIMARY WORKPLACE Priv/Solo Practice (n-340) Group Practice (n-331) Coll/Univ/ Med School (n-1264) Hospital/ Clinic (n-395) Priv Co/ Business (n-89) Gov't Agency (n-145) No Formal Workplace (n-28) Other Workplace (n-109) Row Total Satisfied 1 112 35.9% 99 32.1% 346 29.0% 109 29.9% 23 27.1% 42 30.9% 9 34.6% 30 28.6% 770 30.4% 2 115 36.9% 147 47.7% 648 54.3% 187 51.2% 45 52.9% 81 59.6% 15 57.7% 54 51.4% 1292 51.0% Neutral 3 64 20.5% 44 14.3% 160 13.4% 53 14.5% 15 17.6% 9 6.6% 2 7.7% 17 16.2% 364 14.4% Dissatisfied 4 15 4.8% 14 4.5% 38 3.2% 13 3.6% 2 2.4% 4 2.9% *""" 3 2.9% 89 3.5% 5 6 1.9% 4 1.3% 2 .2% 3 .8% 1 1.0% 16 .6% Total 312 100.0% 308 100.0% 1194 100.0% 365 100.0% 85 100.0% 136 100.0% 26 100.0% 105 100.0% 2531 100.0% No answer 28 23 70 30 4 9 2 4 TABLE 47 METHOD OF ACCESS BY OVERALL SATISFACTION WITH MEDLINE Overall satisfaction with MEDLINE Method of Access Always use Satisfied 1 Neutral Dissatisfied 4 Row Total Direct Command 330 Language 31.2% GM Form 192 Screens 29.3% 543 51.3% 348 53.0% 147 13.9% 93 14.2% 33 3.1% 21 3.2% 5 1058 5% 100.0% 2 656 3% 100.0% Yes No TABLB 48 NEED FOR ADDITIONAL INFORMATION IN A MEDLINE CITATION N=1977 Frequency 775 1202 Percent 39.2 60.8 Cumulative Percent 39.2 100.0 No answer: Author address Dosage information Research design Journal section Full text of article Other Total 739* TABLE 49 VALUABLE INFORMATION NOT FOUND IN MEDLINE CITATIONS N-1019 N-1039*** Valuable Information Frequency Percent*' 377 37.0% 240 23.6% 320 31.4% 193 18.9% 806 79.1% 116 11.4% 2052 Most Valuable Information Frequency Percent 116 11.2% 47 4.5% 82 7.9% 31 3.0% 677 65.2% 86 8.3% 1039 100.0% * Some respondents did not answer this question but did select items they'd like to see in a citation (Table 49). ** Multiple reasons were allowed. Total exceeds 100% and frequency exceeds N of 1019. ***The number of respondents selecting a most valuable item is slightly larger than the number selecting valuable items because some respondents only indicated a most valuable item. TABLE 50 Priority Additional of Choice Information Desired 1 2 3 4 5 6 Total Full Text Dosage Information Research Design Author Address Journal Section Other HOST COMMON AREA FOR WHICH SEARCH INFORMATION IS USED BY INFORMATION WHICH WOULD BE VALUABLE IN A CITATION Patient Care N-475* Frequency P Percent** o: 331 69.7% 124 26.1% . 91 19.2% 90 18.9% 79 16.6% 43 9.1% Priority Additional of Choice Information Most Desired 1 Full Text 2 Dosage Information 3 Other 4 Research Design 5 Author Address 6 Journal Section Did not indicate information most desired Frequency Percent 304 64.0% 30 6.3% 30 6.3% 25 5.3% 15 3.2% 13 2.7% 58 12.2% 758 475 100.0% * 991 respondents indicated that they primarily searched for information for patient care (ranking it as ill in question 20). From this group, 475 indicated additional pieces of information which would be desirable in a MEDLINE citation. **Multiple responses were allowed; therefore percent exceeds 100 and frequency exceeds N of 475. TABLE 50 (cont.) Priority Additional of Choice Information Desired 1 2 3 4 5 6 Total Full Text Dosage Information Research Design Author Address Journal Section Other MOST COMMON AREA FOR WHICH SEARCH INFORMATION IS USED BY INFORMATION WHICH WOULD BE VALUABLE IN A CITATION Education N-133* Frequency Percent** 86 64.7% 29 21.8% 26 19.5% 25 18.8% 21 15.8% 14 10.5% Priority Additional of Choice Information Most Desired Frequency Percent 1 Full Text 83 62.4% 2 Other 13 9.8% 3 Dosage Information 7 5.3% 4 Journal Section 7 5.3% 5 Author Address 5 3.8% 6 Research Design 4 3.0% Did not indicate information 14 10.4% most desired 201 133 100.0%- * 321 respondents indicated that they primarily searched for information for Education (ranking it as #1 in question 20). From this group, 133 indicated additional pieces of information which would be desirable in a MEDLINE citation. **Multiple responses were allowed; therefore percent exceeds 100 and frequency exceeds N of 133. TABLE 50 (cont.) Priority Additional of Choice Information Desired 1 2 3 4 5 6 Total Full Text Author Address Research Design Journal Section Dosage Information Other MOST COMMON AREA FOR WHICH SEARCH INFORMATION IS USED BY INFORMATION WHICH WOULD BE VALUABLE IN A CITATION N-564* Pri Frequency Percent** of 358 63.5% 241 42.7% 193 34.2% 85 15.1% 74 13.1% 51 9.0% i Priority Additional of Choice Information Most Desired Frequency Percent Full Text 271 48.0% Author Address 91 16.1% Research Design 53 9.4% Other 40 7.1% Journal Section 11 2.0% Dosage Information 8 1.4% Did not indicate in [formation 90 16.0% most desired 1002 564 100.0% * 1186 respondents indicated that they primarily searched for information for Research/Testing (ranking it as III in question 20). From this group, 564 indicated additional pieces of information which would be desirable in a MEDLINE citation. **Multiple responses were allowed; therefore percent exceeds 100 and frequency exceeds N of 564. Ability to sort citations online Improved capability for printing citations Sort citations among different databases Improve MEDLINE backfile searching Improve methods for SDI service More non-English literature indexed More "didactic" literature indexed Adjacency of searched Text Words Other Total TABLE 51 ADDITIONAL CAPABILITIES WHICH WOULD BE DESIRABLE IN MEDLINE N-2139 Additional Capabilities Frequency Percent* 818 1480 481 1535 481 118 398 800 239 6350 38.2% 69.2% 22.5% 71.8% 22.5% 5.5% 18.6% 37.4% 11.2% N> 1992** Most Desirable Capability Frequency Percent 144 7.2% 555 27.9% 58 2.9% 716 35.9% 95 4.8% 15 .8% 65 3.3% 171 8.6% 173 8.7% 1992 100.0% * Respondents were asked to circle those additional capabilities which they would like to see in MEDLINE; therefore, percent exceeds 100% and frequency is greater than N of 2139. **Respondents were asked to check the additional capability most desired. Not all respondents indicated a most desired capability. TABLE 52 MOST COMMON AREA FOR WHICH SEARCH INFORMATION IS USED BY ADDITIONAL CAPABILITIES DESIRED Priority Additional of Choice Capabilities Desired 1 Improve MEDLINE backfile searching Improved capability for printing citations Ability to sort citations online Adjacency of searched text words Sort citations among different databases More "didactic" literature indexed Total Patient Care N-800* Frequency Percent** Priority of Choice Additional Capability Most Desired N-757 Frequency Percent 558 69.8% 1 Improve MEDLINE backfile searching 265 35.0% 543 67.9% 2 Improved capability for printing citations 216 28.5% 310 38.8% 3 Adjacency of searched text words 70 9.2% 298 37.3% 4 Ability to sort citations online 58 7.7% 180 22.5% 5 More "didactic" literature indexed 28 3.7% 172 21.5% 6 Improve methods for SDI service Did not indicate capability most desired 22 98 2.9* 13.0% 2061 757 100.0% * 991 respondents indicated that they primarily searched for information for patient care (ranking it as 01 in question 20). From this group, 800 selected additional capabilities which would be desirable on MEDLINE. **Multiple responses were allowed; therefore, percent exceeds 100 and frequency exceeds N of 800. TABLE 52 (cont.) MOST COMMON AREA FOR WHICH SEARCH INFORMATION IS USED BY ADDITIONAL CAPABILITIES DESIRED Priority Additional of Choice Capabilities Desired Improved capability for printing citations Improve MEDLINE backfile searching Ability to sort citations online Adjacency of searched text words More "didactic" literature indexed Sort citations among different databases Total Educatic.i N-261 Pi Frequency Percent oi 181 69.3% 177 67.8% 91 34.9% 90 34.5% 84 32.2% 57 21.8% 680 Priority Additional of Choice Capability Most Desired Improve MEDLINE backfile searching Improved capability for printing citations More "didactic" literature indexed Adjacency of searched text words Ability to sort citations online Sort citations among different databases Did not indicate capability most desired N-245 Frequency Percent 83 33.9% 71 24 16 12 30 245 * 321 respondents Indicated that they primarily searched for information for education (ranking it as #1 in question 20). From this group, 261 selected additional capabilities which would be destrabJe on MIDLINE? **Multiple responses were allowed; therefore, percent exceeds 100 and frequency exceeds N of 261. 29.0% 9.8% 6.5% 4.9% 3.7%. 12.2% 100.0% TABLE 52 (cont.) Priority Additional of Choice Capabilities Desired 1 Improve MEDLINE backfile searching Improved capability for printing citations Ability to sort citations online Adjacency of searched text words Improve methods for SDI service Sort citations among different databases MOST COMMON AREA FOR WHICH SEARCH INFORMATION IS USED BY ADDITIONAL CAPABILITIES DESIRED Research/Testing N-1012 Frequency Percent 762 75.3% 712 390 375 259 224 70.4% 38.5% 37.1% 25.6% 22.1% Priority Additional of Choice Capability Most Desired 1 Improve MEDLINE backfile searching 2 Improved capability for printing citations 3 Adjacency of searched text words 4 Ability to sort citations online 5 Improve methods for SDI service 6 Sort citations among different databases Did not indicate capability most desired Total 2722 N-938 Frequency Percent 357 38.1% 253 27.0% 73 7.8% 71 7.6% 64 6.8% 29 3.1% 91 9.6% 938 100.0% * 1186 respondents indicated that they primarily searched for Information for research/testing (ranking it as #1 in question 20). From this group, 1012 selected additional capabilities which would be desirable on MEDLINE. **Multiple responses were allowed; therefore, percent exceeds 100 and frequency exceeds N of 1012. TABLE 53 ADDITIONAL CAPABILITIES WHICH WOULD BE DESIRABLE IN MEDLINE BY INDIVIDUALS WHO ALWAYS USE DIRECT COMMAND LANGUAGE TO SEARCH N-963* Additional Capability Frequency Percent** Ability to sort citations online Improved capability for printing citations Sort citations among different databases Improve MEDLINE backfile searching Did not indicate most desired capability Total 355 675 208 685 2751 36.9% 70.1% 21.6% 71.1% Improve methods for SDI 180 18.7% service More non-English literature 40 4.2% indexed More "didactic" literature 137 14.2% indexed Adjacency of searched Text 357 37.1% Words Other 114 11.8% Most Desirable Capability Frequency Percent 59 256 17 299 963 6.1% 26.6% 1.8% 31.0% 29 3.0% 8 .8% 25 2.6% 84 8.7% 71 7.4% 115 12.0% 100.0% * 1063 respondents indicated that they always searched MEDLINE using direct/command language. From this group, 963 selected additional capabilities which they would like to see in MEDLINE. **Multiple reasons were allowed; therefore percent exceeds 100 and frequency exceeds N of 963. Ability to sort citations online Improved capability for printing citations Sort citations among different databases Improve MEDLINE backfile searching Improve methods for SDI service More non-English literature indexed More "didactic" literature indexed Adjacency of searched Text Words Other Did not indicate most desired capability Total TABLE 53 (cont.) ADDITIONAL CAPABILITIES WHICH WOULD BR DESIRABLE IN MEDLINE BY INDIVIDUALS WHO ALWAYS USE GRATEFUL MED FORM SCREENS TO SEARCH N-577* Additional Capability Frequency Percent** 189 321 101 368 119 34 107 177 60 1476 32.8% 55.6% 17.5% 63.8% 20.6% 5.9% 18.5% 30.7% 10.4% MOST DESIRABLE CAPABILITY Frequency Percent 40 6.9% 127 22.0% 14 2.4% 186 32.2% 28 4.9% 963 .5% 14 2.4% 44 7.6% 49 8.5% 72 12.5% 100.0% * 659 respondents indicated that they always searched MEDLINE using GRATEFUL MED form screens. From this group, 577 selected additional capabilities which they would like to see in MEDLINE. **Multiple reasons were allowed; therefore percent exceeds 100 and frequency exceeds N of 577. TABLE 54-A EQUIPMENT USED WHEN SEARCHING N-2556 Cumulative Frequency 2454 Percent Percent A PC 96.0 96.0 A Terminal 102 4.0 100.0 Total 2556 No answer: 160 TABLE 54-B PC AVAILABLE AT WORKPLACE N-2708 Yes No Total 2708 No answer: 8 Cumulative Frequency Percent Percent 2600 96.0 96.0 108 4.0 100.0 TABLE 55 METHOD OF ACCESS USED WHEN SEARCHING MEDLINE Access Method Used Direct/Command Language GRATEFUL MED Form Screens GRATEFUL MED Direct Mode Other front-end package Never Use This Method Frequency Percent Sometimes Use This Method Frequency Percent Always Use This Method Frequency Percent Row Total* Frequency Percent 1129 45.9% 268 10.9% 1063 43.2% 2460 100% 1291 52.6% 504 20.5% 659 26.9% 2454 100% 2046 83.5% 359 14.7% 45 1.8% 2450 100% 2222 90.7% 124 5.1% 103 4.2% 2449 100% *Ideally, all row totals should be the same; however, due to respondent error in allocating percentages for each access method, the totals differ slightly. TABLK 56 METHOD OF ACCESS USED WHEN SEARCHING MEDLINE BY PROFESSION* Direct Command Language Physician (n-1776) Nurse (n-99) Other Health Professional (n-180) Scientist (n-753) Student (n-118) Librarian/ Info Spec. (n-86) Other (n-207) Row Total Never Use (n-1129) 794 49.1% 35 42.7% 66 40.2% 286 40.9% 40 37.4% 9 11.8% 83 47.7% 1129 45.9% Sometimes Use (n-268) 182 11.2% 8 9.8% 17 10.4% 87 12.4% 13 12.1% 6 7.9% 19 10.9% 268 10.9% Always Use (n-1063) 642 39.7% 39 47.6% 81 49.4% 326 46.6% 54 50.5% 61 80.3% 72 41.4% 1063 43.2% Column Totals 1618 100.0% 82 100.0% 164 100.0% 699 100.0% 107 100.0% 76 100.0% 174 100.0% 2920 100.0% No answer 158 17 16 54 11 10 33 GRATEFUL MED Form Screens Physician (n»1776) Nurse (n-99) Other Health Professional (n-180) Scientist (n«753) Student (n-118) Librarian/ Info Spec. (n-86) Other (n-207) Row Total Never Use (n-1291) 774 47.8% 50 61.7% 98 60.9% 405 57.9% 70 66.7% 66 85.7% 92 53.5% 1291 52.6% Sometimes Use (n=504) 357 22.1% 16 19.8% 23 14.3% 147 21.0% 17 16.2% 8 10.4% 35 20.3% 504 20.5% Always Use (n=659) 487 30.1% 15 18.5% 40 24.8% 147 21.0% 18 17.1% 3 3.9% 45 26.2% 659 26.9% Column Totals 1618 100.0% 81 100.0% 161 100.0% 699 100.0% 105 100.0% 77 100.0% 172 100.0% 2913 100.0% No answer 158 18 19 54 13 9 35 ^Multiple responses were received for profession; therefore, some individuals are represented in more than one category. The n's reported here reflect the 0 of responses to the individual question. ! Direct Command Language Priv/Solo Practice (n-340) Never Use 161 54.0% Sometimes Use 32 10.7% Always Use 105 35.2% Column Total 298 100.0% No answer 42 Priv/Solo Practice GRATEFUL (n-340) MED Form Screens Never Use 123 41.4% Sometimes Use 74 24.9% Always Use 100 33.7% Column Totals 297 100.0% No answer 43 Group Coll/Univ/ Practice Med School (n-331) (n-1264) 151 49.3% 34 11.1% 121 39.5% 306 100.0% 25 144 46.8% 62 20.1% 102 33.1% 308 100.0% 23 499 43.1% 128 11.1% 530 45.8% 1157 100.0% 107 Group Coll/Univ/ Practice Med School (n-331) (n-1264) 660 57.4% 223 19.4% 267 23.2% 1150 100.0% 114 TABLE 57 METHOD OF ACCESS BY PRIMARY WORKPLACE Hospital/ Clinic (n-395) Priv Co/ Business (n-89) 160 46.5% 35 42.7% 34 9.9% 11 13.4% 150 43.6% 36 43.9% 344 100.0% 82 100.0% 51 Gov't Agency (n-145) 65 48.5% 17 12.7% 52 38.8% 134 100.0% 11 No Formal Other Row Workplace Workplace Total (n-28) (n-109) 9 34.6% 2 7.7% 15 57.7% 26 100.0% 45 1125 43.3% 45.9% 10 268 9.6% 10.9% 49 1058 47.1% 43.2% 104 2451 100.0% 100.0% Hospital/ Clinic (n-395) Priv Co/ Business (n-89) Gov't Agency (n-145) No Formal Workplace (n-28) Other Workplace (n-109) Row TotaJ 182 53.1% 44 53.7% 63 47.0% 18 69.2% 52 49.5% 1286 52.6% 66 19.2% 17 20.7% 34 25.4% 3 11.5% 23 21.9% 502 20.5% 95 27.7% 21 25.6% 37 27.6% 5 19.2% 30 28.6% 657 26.9% 343 100.0% 82 100.0% 134 100.0% 26 100.0% 105 100.0% 2445 100.0% 52 11 TABLE 58 YEAR METHOD OF ACCESS BY IN WHICH HIGHEST DEGREE RECEIVED Direct Command Language Pre-1950 (n-83) 1950s (n-270) 1960s (n-523) 1970s (n-1069) 1980s (n-722) Row Total Never Use 40 58.8% 125 52.1% 242 52.2% 442 45.1% 260 38.7% 1109 45.8% Sometimes Use 11 16.2% 36 15.0% 47 10.1% 96 9.8% 75 11.2% 265 10.9% Always Use 17 25.0% 79 32.9% 175 37.7% 442 45.1% 336 50.1% 1049 43.3% Column Totals 68 100.0% 240 100.0% 464 100.0% 980 100.0% 671 100.0% 2423 100.0% No answer 15 30 59 89 51 GRATEFUL MED Form Screens Pre-1950 (n-83) 1950s (n-270) 1960s (n-523) 1970s (n-1069) 1980s (n-722) Row Total • Never Use 27 39.1% 98 40.8% 213 46.0% 528 54.0% 408 61.3% 1274 52.7% Sometimes Use 24 34.8% 71 29.6% 102 .22.0% 182 18.6% 116 17.4% 495 20.5% Always Use 18 26.1% 71 29.6% 148 32.0% 268 27.4% 142 21.3% 647 26.8% Column Total 69 100.0% 240 100.0% 463 100.0% 978 100.0% 666 100.0% 2416 100.0% No answer 14 30 60 91 56 TABLE 59 PROBLEMS ACCESSING MEDLINE ON THE NLM COMPUTER N-2392 FREQUENCY PERCENT* No problems Remembering the connect/ disconnect protocols Busy telecommunication lines NLM computer not available Other 1287 386 466 281 252 53.8% 16.1% 19.5% 11.7% 10.5% Total 2672 No answer: 324 *Respondents were asked to circle all reasons that applied; therefore, percentages exceed 100% and frequency exceeds N of 2392. TABLE 60 METHODS USED BY INDIVIDUALS IN LEARNING TO SEARCH MEDLINE Using GRATEFUL MED Using other front-end software NLM-sponsored training course Course from academic curriculum Other non-NLM training course Self-taught Learned from co-worker Other Total N-2203 Methods Used to Learn Frequency Percent* 999 45.3% 108 4.9% 1095 49.7% 95 4.3% 205 9.3% 992 45.0% 316 14.3% 138 6.3% 3948 N-2059** Most Helpful Method Used Frequency Percent 645 31.3% 28 1.4% 879 42.7% 39 1.9% 88 2059 4.3% 242 11.8% 72 3.5% 66 3.2% 100.0% * Respondents were asked to circle those methods which they used to learn to search MEDLINE; therefore, percent exceeds 100 and frequency is greater than N of 2203. **Respondents were asked to place a checkmark by the most helpful method used in learning to search. Not all respondents indicated a most helpful method. TABLE 61-A SATISFACTION WITH 3-5 DAY INITIAL TRAINING COURSE N=328* Cuxnulati1 Satisfied Frequency Percent Percent 1 170 51.8 51.8 2 105 32.0 83.8 Neutral 3 44 13.4 97.3 Dissatisfi ed 4 6 1.8 99.1 5 3 .9 100.0 Total 328 No answer 2388 *Of the 1095 individuals who indicated they had received some form of NLM sponsored training, 328 had taken a 3-5 day training course. TABLE 61-B SATISFACTION WITH 3-5 DAY INITIAL TRAINING COURSE BY OVERALL SATISFACTION WITH MEDLINE Overall satisfaction with MEDLINE 3-5 Day Training Satisfied 1 (n-773) Satisfied 1 (n=170) 88 72.1% 2 (n-105) 23 18.9% Neutral 3 (n=44) 10 8.2% Dissatisfied 4 (n=6) -- 5 (n-3) 1 .8% Column Total 122 100.0% (n-1293) 67 45.3% 54 36.5% 22 14.9% 3 2.0% 2 1.4% 148 100.0% Neutral 3 (n-367) 10 23.3% 21 48.8% 10 23.3% 2 4.7% Dissatisfied Row 4 (n-90) 5 (n-T6) Total 3 168 30.0% 51.9% 5 __ 103 50.0% 31.8% 1 10.0% 1 10.0% 1 100.0% 43 100.0% 10 100.0% 1 100.0% 44. 13.6% 6 1.9% 3 .9% 324 TABLE 62-A SATISFACTION WITH 6 HOUR BASICS OF SEARCHING MEDLINE COURSE N-1040* Satisfied Frequency Percent Cumulative Percent 1 464 44.6 44.6 2 404 38.8 83.5 Neutral 3 133 12.8 96.3 Dissatisfied 4 31 3.0 99.2 5 8 .8 100.0 Total 1040 No answer: 1676 *0f the 1095 individuals who indicated they had received some form of NLM sponsored training, 1040 had taken the 6 hour Basics of Searching MEDLINE course. TABLE 62 B SATISFACTION WITH 6 HOUR BASICS OF SEARCHING MEDLINE COURSE BY OVERALL SATISFACTION WITH MEDLINE Overall satisfaction with MEDLINE 6 Hour Basics Satisfied 1 (n-773) Satisfied 1 (n=464) 196 61.3% 2 (n«404) 93 29.1% Neutral 3 (n«133) 20 6.3% Dissatisfied 4 (n-31) 8 2.5% 5 (n=8) 3 .9% Column Totals 320 100.0% (n-1293) 218 41.0% 238 44.7% 66 12.4% 9 1.7% 1 .2% 532 100.0% Neutral 3 (n-367) 37 26.2% 58 41.1% 35 24.8% 10 7.1% 1 .7% 141 100.0% Dissatisfied Row 4 (n-90) 5 (n«T6) Total 8 459 25.8% 44.6% 12 1 402 38.7% 20.0% 39.1% 9 2 132 29.0% 40.0% 12.8% 1 1 29 3.2% 20.0% 2.8% 1 1 7 3.2% 20.0% .7% 31 5 1029 100.0% 100.0% 100.0% TABLE 63-A METHODS USED TO LEARN TO SEARCH MEDLINE* BY PRIMARY WORKPLACE Priv/Solo Practice (n-340) Group Practice (n-331) Coll/Univ/ Med School (n-1264) GRATEFUL MED 133 52.2% 130 51.2% 440 42.1% Other Front-End 7 2.7% 8 3.1% 60 5.7% NLM Training 105 41.2% 113 44.5% 561 53.6% School course 12 4.7% 5 2.0% 54 5.2% Other Non-NLM Training 15 5.9% 14 5.5% 125 12.0% Self-taught 113 44.3% 112 44.1% 458 43.8% Learned from co-workers 18 7.1% 24 9.4% 153 14.6% Other Method 23 9.0% 18 7.1% 47 4.5% Total Respondents 255 254 1046 No answer 85 77 218 Hospital/ Clinic (n-395) Priv Co/ Business (n-89) Gov't Agency (n-145) No Formal Workplace (n-28) Other Workplace (n-109) 145 44.3% 32 42.1% 65 53.7% 8 33.3% 43 46.7% 16 4.9% 8 10.5% 6 5.0% -- 3 3.3% 168 51.4% 35 46.1% 55 45.5% 12 50.0% 43 46.7% 13 4.0% 4 5.3% 4 3.3% -- 3 3.3% 28 8.6% 7 9.2% 10 8.3% 1 4.2% 5 5.4% 150 45.9% 44 57.9% 57 47.1% 13 54.2% 41 44.6% 52 15.9% 14 18.4% 26 21.5% 6 25.0% 22 23.9% 19 5.8% 4 5.3% 9 7.4% 2 8.3% 16 17.4% 327 76 121 24 92 68 13 24 17 *Respondents were asked to circle all methods which were used; therefore, percentages exceed 100 and frequencies exceed number of total respondents for each column. TABLE 63-B MOST HELPFUL METHOD USED TO LEARN TO SEARCH MEDLINE BY PRIMARY WORKPLACE Priv/Solo Practice (n-340) Group Practice (n-331) Coll/Univ/ Med School (n-1264) GRATEFUL MED 102 41.3% 97 38.8% 262 27.1% Other Front-End 3 1.2% 2 .8% 14 1.4% NLM Training 82 33.2% 92 36.8% 459 47.5% School course 8 3.2% 1 .4% 20 2.1% Other Non-NLM Training 9 3.6% 5 2.0% 56 5.8% Self-taught 24 9.7% 40 16.0% 99 10.2% Learned from co-workers 6 2.4% 4 1.6% 36 3.7% Other Method 13 5.3% 9 3.6% 21 2.2% Column 247 Totals 100.0% 250 100.0% 967 100.0% No answer 93 81 297 Hospital/ Priv Co/ Gov't No Formal Other Row Clinic (n-395) Business (n-89) Agency (n-145) Workplace (n-28) Workplace (n-109) Total 96 31.7% 15 22.1% 37 34.9% 6 25.0% 28 31.5% 643 31.3% 4 1.3% 3 4.4% 2 1.9% -- -- 28 1.4% 135 44.6% 26 38.2% 37 34.9% 10 41.7% 35 39.3% 876 42.7% 6 2.0% 1 1.5% 1 .9% 1 4.2% 1 1.1% 39 1.9% 11 3.6% 3 4.4% 3 2.8% -- 1 1.1% 88 4.3% 34 11.2% 15 22.1% 16 15.1% 4 16.7% 10 11.2% 242 • 11.8% 12 4.0% 3 4.4% 4 3.8% 2 8.3% 5 5.6% 72 3.5% 5 1.7% 2 2.9% 6 5.7% 1 4.2% 9 10.1% 66 3.2% 303 100.0% 68 100.0% 106 100.0% 24 100.0% 89 100.0% 2054 100.0% 92 21 39 20 TABLE 64-A METHODS USED TO LEARN TO SEARCH MEDLINE BY INDIVIDUALS WHO ALWAYS SEARCH USING DIRECT COMMAND LANGUAGE Always Use Direct Command Language N= =906* Methods Used to Learn Frequency Percent** GRATEFUL MED 21 2.3% Other Front-End 12 1.3% NLM Training 660 72.8% School course 53 5.8% Other Non-NLM 95 10.5% Training Self-taught 402 44.4% Learned from 127 14.0% co-workers Other method 63 7.0% Total 1433 N= 829 Most Helpful Method Used Frequency Percent 3 .4% 2 .2% 568 68.5% 26 3.1% 49 5.97 119 14.4% 32 3.9% 30 829 3.6% 100.0% * 1063 respondents indicated that they always searched MEDLINE using direct/command language. From this group, 906 indicated which methods were used in learning to search. **Respondents were asked to indicate all methods used; therefore, percent exceeds 100 and frequency exceeds N of 906, TABLE 64-B METHODS USED TO LEARN TO SEARCH MEDLINE BY INDIVIDUALS WHO ALWAYS SEARCH USING GRATEFUL MED FORM SCREENS Always Use GRATEFUL MED Form Screen N-576* Methods Used to Learn Frequency Percent** GRATEFUL MED 536 93.1% Other Front-End 18 3.1% NLM Training 102 17.7% School course 19 3.3% Other Non-NLM 28 4.9% Training Self-taught 223 38.7% Learned from 65 11.3% co-workers Other method 17 3.0% Total 1008 N= 555 Most Helpful Method Used Frequency Percent 456 82.2% 1 .2% 44 7.9% 4 .7% 8 1.4% 22 4.0% 11 2.0% 9 1.6% 555 100.0% * 659 respondents indicated that they always searched MEDLINE using GRATEFUL MED form screens. From this group, 576 indicated which methods were used in learning to search. **Respondents were asked to indicate all methods used; therefore, percent exceeds 100 and frequency exceeds N of 576 TABLE 65 MOST SATISFACTORY ASPECTS OF MEDLINE N=747 Frequency Percent* Great service/system 64 8.6% Good content/depth/information 244 32.7% GRATEFUL MED 149 19.9% Cost reasonable 140 18.7% Availability/Hours/Convenience 190 25.4% Speed/efficiency/time-saving 184 24.6% Ease of use/user-friendly 86 11.5% MeSH vocabulary/indexing 52 7.0% Other NLM databases available 14 1.9% Training courses 14 1.9% Service desk/assistance 47 6.3% Documentation/manuals 18 2.4% Other 58 7.7% No answer 1969 *Content of respondent answers was frequently coded with more than one code; therefore, percentages exceed 100% and frequencies exceed the N of 747. TABLE 66 MOST SATISFACTORY ASPECTS OF MEDLINE BY ACCESS METHOD USED TO SEARCH MEDLINE Method of access Always Use: Great service/system Good content/depth/info. GRATEFUL MED Cost Availability/hours/ convenience Speed/efficiency/ time-saving Ease of use/user-friendly MeSH vocabulary/indexing Other NLM databases available Training courses Service Desk/assistance Documentation/manuals Other r Direct Command Language N-305 Frequency Percent* 1 GM Form Screens N-194 Frequency Percent* 27 8.9% 13 6.7% 121 39.8% 45 23.2% 3 1.0% 89 45.9% 75 24.7% 25 12.9% 84 27.6% 49 25.3% 89 29.3% 42 21.6% 26 8.6% 29 14.9% 17 5.6% 17 8.8% 6 2.0% -- 7 2.3% -- 13 4.3% 15 7.7% 11 3.6% 1 .5% 28 9.2% 8 4.1% Total 507 333 *Content of respondent answers was frequently coded with more than one code; therefore, percentages exceed 100% and frequencies exceed the N's of 305 and 194. TABLE 67 LEAST SATISFACTORY ASPECTS OF MEDLINE N=874 Frequency Percent* Backfile searching 122 Ik.07. Direct/command language searching 96 11.0% Printing [selective] 89 10.2% Other NLM system limitations 83 9.5% Full text not available 63 7.2% No abstracts for some citations 14 1*6% Poor coverage in area Cost 48 5.5% MeSH vocabulary/indexing 120 13.7% 50 5.7% GRATEFUL MED limitations 150 17.2% GRATEFUL MED not available 68 7.8% for all microcomputers Need for more training 29 3.3% Need less technical newsletter 3 .3% Documentation/manuals 40 4.6% NTIS billing system 29 3.3% User hardware/software problems 25 2.9% Service desk/assistance 6 .7% Other NLM databases 12 1.4% Other Total 171 19.6% 1218 *Content of respondent answers was frequently coded with more than one code; therefore, percentages exceed 100% and frequencies exceed the N of 874. TABLE 68 LEAST SATISFACTORY ASPECTS OF MEDLINE BY ACCESS METHOD USED TO SEARCH MEDLINE Method of access Always Use: Backfile searching Direct/command language searching Printing [selective] Other NLM system limitations Full text not available No abstracts for some citations Poor coverage in area MeSH vocabulary/indexing Cost GRATEFUL MED limitations GRATEFUL MED not available for all microcomputers Need for more training Need less technical newsletter Documentation/manuals NTIS billing system User hardware/software problems r Direct Command Language N-387 Frequency Percent* GM Fo Frequency rm Screens N*196 Percent* 51 13.2% 30 15.3% 62 16.0% 3 1.5% 62 16.0% 8 4.1% 49 12.7% 3 1.5% 18 4.7% 16 8.2% 6 1.6% 2 1.0% 21 5.4% 11 5.6% ; 67 17.3% 21 10.7% 22 5.7% 9 4.6% ; 10 2.6% 73 37.2% >le 50 12.9% 4 2.0% 10 2.6% 6 3.1% 3 .8% -- 23 5.9% 4 2.0% 20 5.2% 1 .5% 9 2.3% 4 2.0% (continued on next page) TABLE 68 (cont.) LEAST SATISFACTORY ASPECTS OF MEDLINE BY ACCESS METHOD USED TO SEARCH MEDLINE Method of access Always Use: Service Desk/Assistance Other NLM databases Other Total Direct Command Language N-387 Frequency Percent* 1 5 68 557 .3% 1.3% 17.6% GM Form Screens N=196 Frequency Percent* 3 1 49 248 1.5% .5% 25.0% *Content of respondent answers was frequently coded with more than one code; therefore, percentages exceed 100% and frequencies exceed N's of 387 for Direct Command Language and 196 for GM Form Screens. TABLE 69 WILLINGNESS TO PARTICIPATE IN FOLLOW-UP STUDY N-2342 Cumulative Frequency Percent Percent Yes 2016 86.1 86.1 No 326 13.9 100.0 Total 2342 No answer: 374 ATTACHMENT 1 NATIONAL LIBRARY OF MEDICINE 8600 Rockville Pike Bethesda,MD 20894 r L INSTRUCTIONS: The purpose of this questionnaire is to find out how individuals use MEDLINE on the National Library of Medicine computer system, their level of satisfaction with the system, and their views on how it can be improved. The NLM hopes to use this information to provide better service to its users. Unless-othcrwise indicated, answer each question bv either writing your answer in the space provided, or by circling the number in front of the appropriate answer. All your answers will be available only to the study investigators, unless otherwise required by law. If you have any questions about this study, please contact Karen Wallingford at (301) 496-3261. SECTION I. GENERAL INFORMATION 1. What is your profession? (Circle all that apply) 1. Physician 2. Nurse 3. Other Health Professional 4. Scientist 5. Student 6. Librarian/Information Specialist 7. Other (specify) ______________________ 2. What is the highest educational degree you hold? 3. What year did you receive that degree? _______ 4. If you are a health professional, what is your specialty? (If you are not a health professional, please skip this question.) 5. What is your primary work place? (Circle only one answer) ""17 Private solo practice 2. Group practice 3. College or university or medical school 4. Hospital or clinic 5. Private company or business 6. Government agency 7. No formal work place (ie, student or otherwise unaffiliated) 8. Other (specify) _______,__________________________________________ 6. Do you have a microcomputer (PC) or is one available in your work place? 1. No 2. Yes, type: _______________________________________________________ 7. How many people (including yourself) share the MEDLINE User ID Code you use? (If you arc the only person who uses this code, please write T.) ________ user(s) 8. How many MEDLINE searches do you do on the NLM computer in the average month? (Do not include searches done for you by someone else.) ________searches 9. How many MEDLINE searches do you have someone else do for you in the average month? (Ifyou do all of your searches, enter a zero and skip to question 11.) ________ searches 10. If other people occasionally or always search MEDLINE for you: A. Who generally does the searches for you? (Circle only one answer) 1. Librarian/Information Specialist 2. Student/Research Assistant 3. Secretary/Administrative Assistant 4. Colleague 5. Family Member 6. Other (specify)______________________________________________;______ B. Under what circumstances do you prefer to have someone else search MEDLINE for you? (Circle all that apply) 1. When someone else can do it as easily as I can 2. When I don't have time to do it myself 3. When I need different expertise/system knowledge 4. When I've tried a search myself and have not been satisfied with the results 5. Other (specify) -______________________________________;_______ C. Please rate how satisfied you generally are with searches you do yourself, and searches that are done for you by others. (If you never search MEDLINE yourself, please leave that response blank.) Very Not At All Satisfied Satisfied Searches done by: 1. Yourself 12 3 4 5 2. Others 12 3 4 5 D. If you are generally not satisfied with MEDLINE searches done for you, (if you circled 4 or 5) please indicate why. (Circle all that apply) 1. Inconvenient location 2. Inconvenient hours 3. Have to wait to get search done 4. Cost 5. Unsatisfactory results 6. Other (specify) _____________________________________________________ IF YOU NEVER SEARCH MEDLINE YOURSELF, DO NOT FILL OUT THE REST OF THIS QUESTIONNAIRE. PLEASE RETURN IT IN THE ATTACHED POSTAGE PAID ENVELOPE. THANK YOU FOR YOUR COOPERATION. IF YOU DO SEARCH MEDLINE YOURSELF, PLEASE COMPLETE THE REST OF THIS QUESTIONNAIRE. SECTION n. SYSTEM USE 11. What factors influence you to search online databases yourself, instead of having someone else do the search for you? (Circle all that apply, and check the single most influential factor.) Influential Factor Factor 1- _____ I am more familiar with the subject matter than a search intermediary 2. _____ I get the information faster 3. _____ I enjoy searching 4. _____ It's more cost effective than using a search intermediary 5. _____ No one else is available to do the search for me 6. _____ Other (specify)______________________________________ 12. How experienced a user of online databases do you consider yourself to be? 1. Very experienced 2. Somewhat experienced 3. Not very experienced 4. Not at all experienced 13. How long have you been searching MEDLINE on your User ID code? ________years ________ months 14. During the time you have been searching, would you say that your use of MEDLINE has: 1. Increased 2. Stayed about the same 3. Decreased 15. If your usage has increased or decreased, please indicate the reasons for the change. 16. How often do cost considerations keep you from doing a MEDLINE search on the NLM computer? 1. Never 2. Rarely 3. Occasionally 4. Frequently 17. How would you rate your overall satisfaction with MEDLINE on the NLM computer system? Very Not At All Satisfied Satisfied 12 3 4 5 SECTION in. MEDLINE SEARCHES 18. When you search MEDLINE- A. Is it typically for (Circle only one answer) 1. An immediate information need 2. Staying current in your field 3. Learning about new areas 4. Other (specify)____________________________________________________ B. Are you typically interested in retrieving: (Circle only one answer) 1. Just a few relevant citations 2. All relevant citations from a particular time period 3. Other (specify) ___________________________________________________ C Do you typically retrieve: (Circle only one answer) 1. Too few citations 2. About the right number of citations 3. Too many citations D. What percent of these citations are typically relevant to your inquiry? ________% 19. When you search MEDLINE, do you most often search for (Circle only one answer) 1. an author 2. a journal title 3. a subject 20. Please indicate the primary areas in which you use MEDLINE search information, rank ordered so that your most common use is #1, second most common is #2, etc Please give no more than three answers. Rank Order _____Patient Care ____ Education _____ Research/Testing _____ Management/Administration _____ Regulation _____Other (specify)___________________________________________________ 21. How often do you use the Medical Subject Headings (MeSH) terms when searching for specific subjects? 1. Always 1 Usually 3. Occasionally 4. Rarely 5. Never 22. How useful do you find the MeSH terms to be? Very Useful 1 2 3 Not At All Useful 4 5 23. If you think the MeSH terms are generally not useful, or if you never use MeSH terms, please indicate why. 24. How long does it typically take you (at the terminal or microcomputer) to search MEDLINE on the NLM system for citations on a particular subject? 1. Less than 5 minutes 2. 5 to 10 minutes 3. 10 to 15 minutes 4. More than 15 minutes 25. Do you feel that this is: 1. Too long 2. A reasonable amount of time 3. Quicker than expected 26. Are there any types of information that would be valuable to you that you cannot routinely find in a citation? 1. No (Skip to question 27) 2. Yes- (Please circle all of the following types of information that would be valuable to you, and check the single most valuable type of information.) Most Valuable Valuable 1. _____ Author address 2. _____ Dosage information 3. _____ Research design 4. _____ Journal section (ie, Brief Communications) 5._____ Full text of article 6. _____ Other (specify)__________________________________ 27. Which of the following features or capabilities would you most like to see added to the NLM system? (Circle as many as you like, and check the one feature you would most like to see.) Most Wanted Wanted 1. _____ Ability to sort citations online 2. _____ Improved capability for selecting which citations to print 3. _____ Ability to sort citations among different databases 4. _____ Improved capability for searching MEDLINE Backfiles at one time 5. _____ Improved methods for SDI (automated monthly update search) service 6. _____ More non-English literature indexed 7. _____ More "didactic" (ie, educarional/instructional/teaching. etc) literature indexed 8. _____ Ability to specify the "adjacency" of searched Text Words 9. _____ Other (specify)_______________________________________________ SECTION TV. ACCESSING MFDMNF 28. When you search MEDLINE, do you primarily use: 1. A microcomputer (PC) 2. A terminal 29. Please write in the percent of MEDLINE searches you perform with each of the following methods. Note that your percents should add up to 100. (If you do not use a method, please write "0".) _____ % Direct/command language (no user-friendly front-end) _____ % GRATEFUL MED, using form screens _____ % GRATEFUL MED, using option 3, direct mode _____ % Other user-friendly front-end package (specify) ________________________ 100% Total 30. If you use more than one method of searching MEDLINE, under what circumstances do you choose one method over another? 31. What types of problems, if any, do you have accessing the NLM computer? (Circle all that apply) 1. No problems 2. Remembering connect/disconnect protocols 3. Busy telecommunication lines 4. NLM computer not available 5. Other (specify) _____________________________________________________ 32. How did you learn to search MEDLINE on the NLM computer system? (Please circle all the methods that you used, and check the one method that was most helpful to you in learning how to search MEDLINE.) JJsed Most Method Helpful 1. _____ Using GRATEFUL MED 2. _____ Using other front-end software (eg, SCI-MATE) 3. _____ Attended NLM-sponsored training course 4. _____ Attended a course as part of an academic curriculum 5. _____ Attended other, non-NLM sponsored training course 6. _____ Self-taught 7. _____ Learned from a co-worker 8. _____ Other (specify)___________ 33. If you have attended an NLM-sponsored training course (choice 3 in question 32), please circle the course! s) you attended, and how satisfied you were with the course(s). Very Attended? Satisfied 1. 3-5 day Initial Training Course 12 3 4 2. 6 hour Basics of Searching MEDLINE 12 3 4 Not At All Satisfied 5 5 At some future point the NLM may conduct additional research on topics related to MEDLINE and the MEDLINE search system. Would you be willing to participate in a follow up study? 1. Yes (Please fill out your name, address and phone number below.) 2. No Name: __________________________________ Address: Phone: If you have any additional comments that you would like to make about MEDLINE, please do so in the space below. We are particularly interested in knowing those aspects of MEDLINE with which you are most satisfied, and those aspects of MEDLINE with which you are least satisfied. Please continue on the back of this form if you need additional space. Most Satisfactory Aspects: Least Satisfactory Aspects: Please return this survey in the enclosed postage paid envelope. Thank you for your time and cooperation. ATTACHMENT 2 DEPARTMENT OF HEALTH & HUMAN SERVICES Public Health Service National Institutes of Health National Library of Medicine Bethesda MO 20894 Dear Colleague: The National Library of Medicine's basic mission is to improve the dissemination of information important to the progress of medicine and to the public health. In support of this mission, NLM developed MEDLINE, the first online bibliographic database available via a nationwide telecommunications network, in 1971. In the first decade of its existence, MEDLINE's principal users were medical librarians who acted as search intermediaries for practitioners and researchers needing to locate pertinent information in the biomedical and health care literature. In the past few years, however, the number of individuals who search MEDLINE for themselves has increased dramatically. To provide better service to these individual end-users, NLM needs to know more about them, how they use MEDLINE, their level of satisfaction with the current system, and their views on changes that would make the system more effective. In an effort to obtain this information, the Library is surveying all individuals who search MEDLINE directly on the NLM computer without the aid of a search intermediary. I hope you will take the approximately 15 minutes needed to fill out the enclosed questionnaire and return it in the postage paid envelope provided. While participation is voluntary, we would appreciate receiving your completed questionnaire by October 16. If you do not search MEDLINE yourself, that is, if you always use a search intermediary, please write across the top of the questionnaire that you do not search MEDLINE yourself, and return it in the postage paid envelope. If you have any questions, contact Karen Wallingford at (301) 496-3261. In appreciation for your time and participation, a $5.00 credit will be applied to your invoice for MEDLINE system use during the month of December 1987. The information collection in this study is authorized under Section 465 of the Public Health Services Act. Your responses will be confidential and your answers will be available only to the study investigators, unless otherwise required by law. Survey results will be presented in the aggregate. NLM is working to provide information services which can assist you in your work. Please help us to make MEDLINE more useful to you. Thank you for your cooperation. Enclosure WH LiJL, Donald A. B. Lindberg, M.4k'** Director ATTACHMENT 3 Dear Colleague: Recently you were sent a questionnaire regarding your use of MEDLINE. If you have mailed it to us, thank you for your participation. If you have not yet returned your completed questionnaire, please mail it as soon as possible to: NLM, 8600 Rockville Pike, Bethesda MD 20894. Attn: K. Wallingford, Bldg. 38, B1W-28. If you need another copy of the questionnaire, call Karen Wallingford at (301) 496-3261. Thankypu for your cooperation. DomildA. B. Lindberg, MJ). -^ Director, National Library of Medicine Q- DEPARTMENT OF HEALTH & HUMAN SERVICES ATTACHMENT 4 Public Health Service National Institutes of Health National Library of Medicine Bethesda MO 20894 Dear Colleague: Approximately four weeks ago you were sent a questionnaire that asked about your use of MEDLINE. The National Library of Medicine is very interested in your views, but has not yet received a completed questionnaire from you. As you may recall, the study is intended to assist us in making MEDLINE responsive to the information needs of persons like yourself who search MEDLINE directly without the aid of a search intermediary. Participation is voluntary, but if you have not yet completed and returned the questionnaire, please take a few minutes and complete the enclosed duplicate copy today. If you have already returned the questionnaire, thank you for participating. If you do not search MEDLINE yourself, that is, if you always use a search intermediary, please write across the top of the questionnaire that you do not search MEDLINE yourself, and return it in the postage paid envelope. If you have any questions, contact Karen Wallingford at (301) 496-3261. In appreciation for your time and participation, a $5.00 credit will be applied to your invoice for MEDLINE system use during the month of December 1987. The information collection is authorized under Section 465 of the Public Health Services Act. Your responses will be confidential and your answers will be available only to the study investigators, unless otherwise required by law. Survey results will reported in the aggregate. Thank you for your cooperation. You>6 truly, J. Donald A. B. Lindberg, M.D. Director Enclosure ATTACHMENT 5 Explanation of Queries Used to Pull Respondent Sample and Breakout of Sample by MMS User Classification Code There were three queries run against the MEDLARS Management Section's (MMS's) USERS file in which basic information on users of the NLM databases is stored. In each query, we were only interested in those users who had an active billing code. This did not ensure that they actively used their code, but, at least, they had not asked for it to be canceled. MMS uses a set of classification codes (see attached) to identify the different types of users on the system. There are also a set of administrative codes originally developed to identify the type of institution -- e.g., federal, state, non-government/not-for-profit, for profit/commercial -- which were refined in 1986 to try to track the growing number of individuals requesting codes. Both sets of codes have undergone various revisions over the years which had to be taken into consideration in refining queries for pulling the respondent sample. The following is a breakout of the respondent sample according to the MMS classification codes: Class. // in Code Classification Type Sample 100's Direct Health Care 3112 200's Health Education 411 300's Health-Related Research or Resource 602 400's Legislative, Regulatory, Planning Agencies - 19 500's Scientific or Technical Products and Services 48 600's General Products and Services 97 700's Information resource/Library 6 800's Other educational institutions/Personnel 18 Total 4313 **** NLM MEDLARS USER TYPE CLASSIFICATION **** 0 - No Classification Type Assigned (includes NLM in-house and foreign codes 100 - DIRECT PATIENT CARE 101 - Hospital/inpatient facility (inc. osteopathic, psychiatric) 102 - Outpatient facility/HMO 103 - Practitioner, physician (inc. osteopaths and M.D.'s) 104 - Practitioner, dentist 105 - Practitioner, veterinarian 106 - Practitioner, nurse 107 - Other direct care 108 - Students in health education 200 - HEALTH EDUCATION 201 - Academic Health Science Center 202 - Medical School 203 - Osteopathy School 204 - Dental School 205 - Veterinary School 206 - Nursing School 207 - Pharmacy School 208 - Optometry School 209 - Podiatry School 210 - Chiropractic School 211 - Public health/health administration program 212 - Allied health program 213 - AHEC/continuing professional education program 214 - Other health-related education program 215 - University/college not specified above 216 - Prof., teacher, instructor in health education 300 - HEALTH-RELATED RESEARCH OR RESOURCE 301 - Medical/biomedical research institution 302 - Scientific research (primary functions outside the scope of biomedicine 303 - Medical/scientific society or association 304 - Medical/scientific library or information resource 305 - Health care consumer/patient support group 306 - Health insurance 307 - Other support activity 308 - Medical/biomedical research 400 - LEGISLATIVE, REGULATORY, PLANNING AGENCIES 401 - Health administration/health planning/HSA 402 - Health care/drug regulation or legislation 403 - Scientific regulation/legislation/administration 404 - Environment/energy/space technology/agriculture/ safety regulation, legislation, or administration 405 - Military application 406 - Law enforcement 407 - General regulation/legislation/administration 500 - SCIENTIFIC OR TECHNICAL PRODUCTS AND SERVICES 501 - Pharmaceuticals 502 - Health care supplies/equipment/services 503 - Chemicals/plastics/petroleum/cosmetics 504 - Food/agriculture 505 - Energy/environment/space technology 506 - Computers/electronics/engineering 507 - Other scientific/technical product or service 600 - GENERAL PRODUCTS AND SERVICES 601 - (Not currently in use for classification) 602 - Information broker 603 - Publisher/media 604 - Law firm/lawyer 605 - Insurance other than health insurance 606 - Trade/welfare 607 - Other 608 - Science writer 700 - INFORMATION RESOURCE OR LIBRARY 701 - Information resource/library 800 - OTHER EDUCATIONAL INSTITUTIONS/PERSONNEL NATIONAL .IBPARY Or MfOlCINI NLfl DD7D7flSi 0 i I NLM007078510