AVIATION CADET QUALIFYING EXAMINATION Psychological Division Office of the Air Surgeon Headquarters of the Army Air Forces Washington October. 1942 3-2857,AF The aviation cadet qualifying examination A Report on the . V. Purpose, Development and Validation of Test AC-IO-A Prepared by: Psychological Division Office of the Air Surgeon Headquarters of the Army Air Forces Washington October, 1942 3-2857,AF. TABLE OF CONTENTS Page Section I; Procedures Followed In the Selection of Aviation Cadets during the Period 1918 to 1942 1 A. Survey of the Selection and Training Program for Pilots 1 B. Procedures Followed in the Initial Selection of Aviation Cadets 4 C. Selection Problems Resulting from Expansion of the Aviation Cadet Training Program 6 Section II: Adoption of a Uniform Qualifying Examination for the Selection of All Aviation Cadets 8 A. Purpose of Aptitude Testing 8 B. Assignment of Responsibility for Developing an Aptitude Test for Aviation Cadets 8 C. Research Contributing to the Development of the Aviation Cadet Qualifying Examination 8 Section III: Outline and Specifications of the Aviation Cadet Qualifying Examination 10 A. Purpose of the Examination 10 B. General Specifications 10 C. Specifications for the Various Parts of the Examination 10 Section IV: Development and Approval of the Aviation Cadet Qualifying Examination 13 A. Construction of Preliminary Forms 13 B. Approval of the Qualifying Examination by the Army Air Forces 13 Section V: Standardization of the Aviation Cadet Qualifying Examination 14 A. Scores Made by Aviation Cadets and by Aviation Students Who Were Selected under Former Requirements 14 B. Scores Made by First Classmen at West Point 16 C. Scores Made by High-School Seniors 16 D. Scores Made by College Men 18 E. Test Scores of Applicants 18 1 5-2857,AF Table of Contents — continued Page Section VI: Success of Test AC-IO-A in Predicting Eliminations from Air-crew Training 23 A. Procedures Followed In the Validation of Test AC-IO-A 23 B. Comparison of Men Who Scored 90 or Above on Test AC-IO-A with Men Who Scored 89 or Below 23 C. Research Findings on Various Parts of the Qualifying Examination 25 D. Additional Statistical Results 25 Section VII: Improvements that Have Been Made in the Aviation Cadet Qualifying Examination since the First Form (Test AC-IO-A) Was Released 34 A. Improvements in Part 1 — Vocabulary 34 B. Improvements in Part 2 — Reading Comprehension 34 C. Improvements in Part 3 — Judgment 34 D. Improvements in Part 4 — Mathematics 34 E. Improvements in Part 5 — Knowledge of Recent Developments . 34 F. Improvements in Part 6 — Mechanical Comprehension .... 35 G. Summary of Improvements that Have Been Made 35 Section VIII: Summary 36 Appendix 39 FIGURES Page Figure 1 Scores Made on Test AC-IO-A by Men Already Enlisted as Aviation Cadets for Training as Pilots, Bombardiers, or Navigators 15 Figure 2 Scores Made on Test AC-IO-A by Enlisted Men Already Selected for Pilot Training 15 Figure 3 Scores made on Test AC-IO-A by First Classmen at the United States Military Academy 17 Figure 4 Scores Made on Test AC-IO-A by a Representative Group of High School Seniors 17 Figure 5 Scores Made on Test AC-IO-A by a Representative Group of 1,000 Applicants 19 Figure 6 Percent of Freshmen and Sophomores In Four Different Colleges Who Passed Test AC-IO-A 19 Figure 7 Comparison of the Education of All Applicants for Aviation Cadet Training with the Education of Men Who Pass the Aviation Cadet Qualifying Examination 21 Figure 8 Relation between Score on Part IV, Mathematics Section, Test AC-IO-A, and Elimination Rate In Navigation Training 27 Figure 9 Relation between Score on Part I, Vocabulary Section, Test AC-IO-A, and Elimination Rate In Navigation Training 28 Figure 10 Relation between Score on Part II, Reading Comprehension Section, Test AC-IO-A, and Elimination Rate In Bombardier Training 29 Figure 11 Relation between Score on Part V, Knowledge of Recent Developments, Test AC-IO-A, and Elimination Rate In Pilot Training 30 Figure 12 Relation between Score on Part III, Judgment Section, Test AC-IO-A, and Elimination Rate In Pilot Training 31 Figure 13 Relation between Score on Part VI, Mechanical Comprehension Section, Test AC-IO-A, and Elimination Rate In Pilot Training 32 5-2857,AF TABLES Page Table I Summary of the Pilot Selection and Training Program . . 2 Table II Summary of the Pilot Training Program for Officers and Aviation Cadets 3 Table III Results of the Special Educational Examination for Aviation Cadets 5 Table IV Comparison of the Elimination Rate for Men Passing Test AC-10-A With Elimination Rate for Men Failing the Test 24 Table 1 Relation between Scores on Eaoh Section of Test AC-10-A and Success In Pilot Training for Class 42-G, Aviation Cadets 39 Table 11 Relation between Scores on Each Section of Test AC-10-A and Success in Pilot Training for Class 42-G, Aviation Students 40 Table 111 Relation between Scores on Each Section of Test AC-10-A and Success In Navigation Training for Classes 42-6, 42-7, and 42-8, Aviation Cadets .... 41 Table Iv Relation between Scores on Each Section of Test AC-10-A and Records In Bombardier School 42 Table v Intercorrelations of the Various Sections of Test AC-10-A 43 Table vl Reliabilities of the Various Sections of Test AC-10-A 44 Iv 3-2857,AF SECTION I PROCEDURES FOLLOWED IN THE SELECTION OF AVIATION CADETS DURING THE PERIOD 1918 TO 1942 A. Survey of the Selection and Training Program for Pilots Flying training and Instruction was at a low ebb for a number of years following the end of the first World War. At one school (Carlstrom Field in Florida) a select group of young men were trained In the early 20 fs for Army flying. This Army Air Corps school was moved to Brooks Field In 1922. At Brooks, prospective pilots went through six rigorous months of training. Those who completed this elementary stage were sent on to Kelly Field for six months of advanced training. These early schools were of Importance not so much because of the highly trained flyers they turned.out as because of the lessons ithey taught regarding the training of flyers and the Information they provided concerning the type of, man that makes a successful pilot. Much of the present program for the selection and training of men for the Array Air Forces tts based on the lessons learned In the last war and In the post-war training schools for flyers. During the years after the last war, standards for Initial selection and training of pilots were high and the training was rigorous. Only a number of men successfully completed the full year of flying training. In 1923, five years after the Armistice, authority was given for the exam- ination of 631 men who were applying for pilot training. Out of this num- ber, only 127 men, or 20%, met the rigid entrance requirements. In the same year only 45 of those who had entered In previous classes were gradu- ated from the advanced course at Kelly Field. In Table I Is summarized the number of applicants In 1923 to 1939, Inclusive, who were authorized to appear for examination during each fiscal year. The number of men who qual- ified, who entered school, and who graduated from the advanced course are also shown. In Table II Is contained a more complete summary of the progress of men In the training schools, showing the number of men who completed and who were eliminated in Primary and Advanced Schools. From the beginning of 1923 to the end of 1938, about 37,000 appli- cants for aviation cadet training were given permission to appear for exam- ination. From this number approximately 8,000, or 22%, were found to be qualified for pilot training. Of the 8,000 men who were accepted, a number failed to report for training and less than half of those who did start primary training during this 16-year period were successful In completing both the primary and advanced flying course. By the end of the fiscal year 1939, a sharp acceleration In the avia- tion cadet training program had taken place. Over 8,000 applicants and 397 graduates are reported for that year. During the twenty years from the Armistice Of*1918 until the German armies marched Into Czechoslovakia In 1938, the largest number of cadets that had been graduated as pilots from Air Corps schools during any one year was 306. The pilots who were graduated during these twenty years were only 10%- of the 37,000 applicants who had been examined originally. In addition to pilots who had been trained as aviation cadets, about 1,000 regular officers had completed the pilot training courses. In terms of present war needs, this group of highly selected and highly trained pilots was only a small nucleus upon which to' build the Army Air Forces that are now In formation. 1 3-2857, AF TABLE I SUMMARY OF THE PILOT'SELECTION AND TRAINING PROGRAM Fiscal Years 1923 to 1939 Inclusive Fiscal Year Number of Applicants Approved for Examination* Applicants Qualified for Aviation Cadet Training Cadets Entering Flying Schools Cadets Graduating From Advanced Schools Total Graduates: Cadets and Officers Combined 1923 631 127 124 45 78 1924 915 229 220 34 73 1925 1,057 287 234 53 131 1926 1,550 435 287 36 114 1927 1,640 349 342 38 111 1928 4,010 682 624 67 156 1929 4,095 807 564 220 275 1930 3,738 962 611 226 306 1931 2,081 504 637 195 250 1932 2,609 552 513 246 299 1933 2,624 503 454 225 287 1934 1,587 321 359 164 216 1935 1,735 251 324 146 204 1936 1,764 269 288 130 180 1937 2,166 427 360 120 170 1938 4,807 1,418 689 192 246 1939 8,146 1,722 903 397 479 * These figures- are for men applying for Aviation Cadet training only, exclusive of officers or enlisted men applying for training In grade. These data were compiled from records of the Aviation Cadet Section, Office of the Director of Personnel, Headquarters, Army Air Forces; and Statistical Records Section, Headquarters, Flying Training Command. 5-2857,AF TABLE II SUMMARY OF THE PILOT TRAINING PROGRAM FOR OFFICERS AND AVIATION CADETS Fiscal Years 1923 to 1939 Inclusive PRIMARY SCHOOLS ADVANCED SCHOOLS Fiscal Year Total In Training (Reported and held over) Num- ber Grad- uated Num- ber Elim- inated Under Training End of Year Total In Training (Reported and held over) Num- ber Grad- uated Num- ber Elim- inated Under Training End of Year 1923 124 48 71 5 103 45 12 46 1924 246 56 64 126 107 26 18 63 1925 363 88 131 144 157 58 43 56 1926 418 103 160 155 154 39 55 60 1927 519 176 262 81 264 111 94 59 1928 728 166 443 119 183 156 27 0 1929 808 315 368 125 408 275 9 124 1930 914 291 520 103 415 306 14 95 1931 875 275 475 125 370 250 4 116 1932 903 303 372 228 419 299 11 109 1933 641 275 280 86 384 287 6 91 1934 567 219 273 75 310 216 11 33 1935 463 199 196 68 282 204 10 68 1936 407 192 172 43 260 180 6 74 1937 473 190 205 78 264 170 2 92 1938 869 310 353 206 402 246 8 148 1939 1,224 526 498 200 674 479 11 184 3-2857, AF A summary of the methods used In the Initial selection of the 8,000 men chosen for pilot training prior to 1939 will supply the background.!or a-better understanding of the present Army Air Forces selection problem. Naturally, the selection procedures used prior to 1939 when only a few hun- dred pilots were being trained each year were quite different from the pro- cedures used now when thousands upon thousands of pilots are being trained as quickly as possible. B. Procedures Followed In the Initial Selection of Aviation Cadets Aviation cadets have always been required to meet especially high men- tal and physical standards. The physical examination will not be considered In this report, however, which Is concerned only with problems of selection on the basis of psychological qualifications. A summary of previous require' ments, both educational and mental, will"furnish a background for evaluating psychological aspects of the present selection program. The purpose of the Initial procedures of selection has always been to pick for training only those men who possess the special aptitudes and qual- ifications required in military aviation and In this way to reduce the num- ber of men who are eliminated after training Is begun. The ultimate goal, selection only of men who are capable of graduation, has not been reached. In 1920, It was decided that hlgh-school graduation or an equivalent amount of education would be required of every applicant for aviation cadet train- ing. Examining boards were authorized to apply practical tests or oral examinations In order to determine whether the applicant met educational re- quirements. Written examinations of the essay or discussion type, covering hlgh-school subjects, were authorized. By 1925, two scheduled examinations were held each year for the purpose of qualifying applicants who could not submit evidence of graduation from high school or attendance at college within one year preceding the date of examination. Because of the popular Impetus given aviation by the early transoceanic flights It was found possible to raise these educational requirements. Beginning In 1927, only men with two years of college training or Its equi- valent were qualified. If an applicant had not completed two years of college, he still could qualify by passing a special examination on 9 col- lege subjects. The first special examination of this nature was held In August, 1927. The two-year college requirement remained In effect from 1927 until January 15, 1942. Minor changes were made from time to time In the subjects covered and In the type of special examinations given. In Table III there Is a summary of the results of the special educa- tional examinations that were given In the fiscal years 1923 to 1941, In- clusive, and In the first half of the fiscal year 1942. It will be noted that In the years preceding 1940, an average of 35% of the men who took the tests obtained passing marks. Those who passed the special educational ex- amination constituted only about 6% of the total number who were qualified for training. With the acceleration of aviation cadet training In 1939 and 1940, an Increasing proportion of applicants, as well as a much larger total number, met the educational requirements by passing the special educational test. Five times as many applicants took the test In the fiscal year 1941 as had taken It during the preceding nine years, and almost four times as many passed it as had qualified In this manner In the preceding nine-year period. In the first half of the fiscal year 1942, an additional 9,000 men took the educational examination. Of these,.5,562 qualified. The marked Increase In the number of men passing the tests reflected a lowering of standards due 5-2857,AF TABLE III RESULTS OF THE SPECIAL EDUCATIONAL EXAMINATION FOR AVIATION CADETS Fiscal Years 1923 to 1942 Fiscal Year Number of Applicants Number of Examinations Held Number Passing Examinations Per cent Passing 1923 17 1 5 29.4 1924 92 3 28 30.4 1925 105 2 15 14.3 1926 149 3 41 27.5 1927 116 1 25 21.6 1928* 363 3 5 1.4 1929 246 3 24 9.8 1930 176 2 48 27.3 1931 119 2 14 11.8 1932 75 3 25 33.3 1933 57 3 24 42.1 1934 47 3 8 17.2 1935 35 3 11 31.4 1936 53 3 20 37.7 1937 83 3 12 14.5 1938 237 3 68 28.7 1939 332 3 157 47.3 1940 928 4 355 38.3 1941 9,272 4 2,478 26.7 First Half of 1942 8,992 2 5,562 61.9 * Educational requirements changed from 4 years of high school to 2 years of college. 5-2857,AF to a growing demand for manpower. This demand for men could not be met by recruitment from college ranks alone and therefore the educational standards were gradually lowered so that more non-college men could qualify. The educational examinations were constructed and graded at Randolph Field until late In 1941, when, at the request of the Air Corps, the Per- sonnel Procedures Section of the Adjutant General’s Office constructed a set of objective examinations for use In Initial selection. These exami- nations Included five required subjects (English composition, arithmetic, geometry, trigonometry, and algebra) and five additional subjects from which two options could be chosen (general history, U. S. history, physics, chemistry, or a language). The first objective-type educational examina- tion was given during November, 1941. This objective examination was more reliable and required much less time to score than the older essay tests. It was a carefully constructed test which was designed to measure the level of education and training of applicants. It was not constructed to measure special aptitude for flying, but to select men with special educational qualifications. The educational examination was supplanted by the Aviation Cadet Qualifying Examination In January, 1942. c. Selection Problems Resulting from Expansion of the Aviation Cadet training Program Recent training reports of the Flying Training Command emphasize the extent of the expansion^that has taken place In the aviation cadet training program. In the summer of 1939, first use was made of elementary flying schools under civilian direction. Following the plan Introduced In that year, aviation cadets who were to be trained as pilots were sent first to civilian Elementary Flying Schools, then to Air Forces Basic Flying Schools, and finally to Air Forces Advanced Flying Schools. The entire flying train- ing period was shortened from one year to about seven months. Specialized bombardier ana navigator training was Introduced a year later In the summer of 1940. The first bombardier and navigator classes were composed of men selected from among cadets who had been eliminated from pilot training. Some men were sent to Florida for navigation training under the supervision of Pan-American Airways instructors'. Others were sent to Lowry Field for training as bombardiers. When these cadets were graduated and commissioned, some were retained to serve as instructors for the ever Increasing stream of men who were sent for bombardier or navigator training. In the two years since these specialized schools were opened In the summer of 1940, the number of bombardiers and navigators graduated has been as large as the total number of pilots graduated during the preceding sixteen years. Such an*expansion has brought with It many problems and resulted In numerous changes In selection and training procedures. The selection of men for the bombardier and navigator schools was at first the responsibility of the Faculty Boards that eliminated men from pilot training. These boards made an effort to select men for navigation training who had been trained In engineering and mathematics. It was also the practice not to recommend for navigation training any cadet who had been eliminated from pilot training for failure in ground school. The selection of bombardier and navigator students from lists of elim- inated pilots was criticized by certain officers In charge of these types of training. It was asserted that this practice tended to place the bom- bardier and navigator- below the pilot In Importance in the air-crew. It also was believed to have an adverse effect on morale In bombardier and nav- igator schools. For these and other reasons the policy was changed, and late in 1941 enlistment for bombardier and navigator training was opened to men with hlgh-school training who could make satisfactory scores on a battery of three aptitude tests. These three aptitude tests (a physics test, the Army General Classification Test, and the Army Mechanical Aptitude Test) were selected for use on the basis of research conducted by the Classification Section of the Technical Training Command with assistance from the Personnel 3-2857,AF Procedures Section of the Adjutant General's Office. Acceptance of the applicant was determined on the basis of the total score on the three tests A total of 1,507 applicants took these special aptitude tests prior to January 15, 1942 and 75$ were qualified for bombardier or navigator train- ing. The use of an aptitude test designed to select qualified men from ap- plicants throughout the nation possessing a hlgh-school education was a major step In Increasing the efficiency of selecting men for air-crew train- ing. For the first time, young men who had not attended college were per- mitted to compete for appointment as aviation cadets on an equal basis with college men. However, this procedure for drawing cadets from a larger pool of manpower could not for long be limited only to bombardiers and naviga- tors. It was soon apparent that there was not a sufficient number of col- lege men to meet the increasing demand for pilots. These and other considerations led to the adoption In January, 1942, of a single qualifying examination that was designed to measure the apti- tude and proficiency of all aviation cadet applicants rather than their knowledge of formal educational subject matter. The reasons for adoption of this test are presented In the following section, with a summary of the advantages accruing from the use of a test of this type. 5-2857,AF SECTION II ADOPTION OF A UNIFORM QUALIFYING EXAMINATION FOR THE SELECTION OF ALL AVIATION CADETS A. Purpose of Aptitude Testing Aptitude testing for the purpose of selecting men who possess the spe- cial talents or abilities required for Important Job assignments Is a rela- tively new and characteristically American development. In recent years a steadily Increasing number of industrial plants, schools and colleges, divisions of the government and of the Army, and other progressive organi- zations have adopted this practical and efficient application of psycholog- ical testing to the selection or classification of men. The purpose of aptitude testing Is to determine the abilities and ca- pacities of each man, and to select and assign each Individual to the kind of work In which he will be most efficient and satisfied. This effort to match men and Jobs has been well received by the men who have been tested and by the directors or officers who supervise their work. B. Assignment of Responsibility for Developing an Aptitude, Test for Aviation Cadets After a series of conferences during November, 1941, representatives of the Military Personnel Division, Training and Operations Division, and ■fhe Office of the Air Surgeon recommended that responsibility for the de- velopment and refinement of an aptitude test for selection of aviation cadets should be assigned to the Psychological Division of the Office of the Air Surgeon, Headquarters, Army Air Forces. Official action on this rec- ommendation was taken by General Stratemeyer, then Assistant Chief of the Air Corps, who signed a directive on December 20, 1941 which established responsibility for air-crew selection and classification procedures. An Im- portant reason for assigning responsibility for developing the Qualifying Examination to the Office of the Air Surgeon was the fact that a staff of Individuals with professional training In the development and application of practical testing procedures had been built up In this office, and an extensive research program, Involving study of air-crew requirements and the development of aptitude tests for air-crew members, had already been started. C. Research Contributing to the Development of the Aviation Cadet Qualifying Examination “ * Specifications for the Avlat-ion Cadet Qualifying Examination were based in part on data collected during several months of research on pilots by the staff of psychologists In the Office of the Air Surgeon. On June 14, 1941, the Chief of the Army Air Corps approved a plan 'for the establishment of a Psychological Research Agency In the Medical Division, Office, Chief of the Air Corps, to determine the characteristics necessary for success in pilot training schools, and to develop aptitude tests for measuring these charac- teristics. In order to facilitate this research the Training Section of the Train- ing and Operations Division, Office, Chief of the Air Corps, modified the program of Instruction at Pilot Replacement Training Centers to Include an allowance of six hours for psychological testing. The first Psychological Research Section was established at Maxwell Field, Alabama, on September 22, 1941 and examining of the second class (42-E) to enter the Replacement Training Center was begun on October 11. On December 18, 1941, the Office of the Air Surgeon was made responsible for research on the selection and 5-2857,AF classification of bombardiers and navigators, in addition to research on pilots. In making plans for the original research program,it had been antic- ipated that a full year would be devoted to the accumulation of data, analysis of results in relation to flying training records, and tryout of preliminary tests. The pressure of the emergency training program and en- try into the war made it necessary to proceed immediately with plans for the application of selection and classification tests. As early as August 20, a directive had been sent from the Chief of the Air Corps to the Military Personnel Division, directing that an intelligence examination be prepared for use in the selection of aviation cadets. This directive was referred to the Medical Division. By the time that war was declared an outline for the Qualifying Examination had been developed and much work had been done on the construc- tion of the first form of the test. In developing the outline and speci- fications for this test all available data on pilot selection were considered Information from tha last war and from research studies since that time was carefully studied and evaluated. The data secured by the Committee on Selection and Training of Aircraft Pilots of the National Research Council, the results of recent studies conducted by the United States Navy, and findings of the Technical Training Command were carefully examined and con- sidered together with the results obtained from the study of aviation cadets by the Psychological Research Sections in the Air Forces Replacement Training Centers. In developing the outline and specifications for the Qualifying Exami- nation, important consideration was given to the suggestions and opinions offered by experienced flying officers who had been associated with the avia- tion cadet training program. Numerous conferences were held with officers within the Headquarters, Flying Training Command and with flying instructors in order to secure information about the abilities that they considered necessary for air-crew members and the aptitude and proficiencies that they thought should be measured by the test. As a supplement to the obtained opinions of flying Instructors, a de- tailed analysis was made of Faculty Board proceedings in the cases of 1,000 aviation cadets who were eliminated from flying training schools in 1941. The board proceedings contained a summary of the reasons why each cadet failed. Analysis of these reasons furnished an accurate cross section of instructors’ opinions regarding-the deficiencies of unsuccessful cadets and provided valuable information regarding selection. In summary, the outline and specifications for the initial form of the Qualifying Examination were based on the practical suggestions of flying officers, the study of reasons for elimination from training schools, and upon data from job analyses and special research studies. A summary of the specifications for Test AC-10-A is given in the following section. 3-2857,AF SECTION III OUTLINE AND SPECIFICATIONS OF THE AVIATION CADET QUALIFYING EXAMINATION A. Purpose of the Examination The Aviation Cadet Qualifying Examination Is designed to qualify for aviation cadet training those men who are sufficiently alert and Intelli- gent to be capable of learning an air-crew assignment and who measure up to the Intellectual and leadership standards required of officers In the Army. The purpose of the test Is to measure aptitude rather than specific techni- cal Information, formal educational achievement, or specialized training. In order that It might be suitable for use with men applying for military service and meet accepted standards of reliability and validity, the follow- ing general specifications were adhered to In constructing the parts of the first form of the examination, Test AC-10-A. B. General Specifications All parts of the examination have some practical relation to aviation or military affairs and measure the sort of behavior or Information that Is required of aviation cadets. No questions or problems are Included that are Impractical or unrelated to air-crew requirements. The test Is free from Items or topics that are matters of controversy or personal opinion. Successful navigators, bombardiers, and pilots undoubtedly differ in Interests, aptitudes, and temperament from the average man In the street. In so far as possible, therefore, the examination Is constructed so that It favors men who have the Interests, aptitudes, and temperament characteris- tic of successful air-crew members. It emphasizes nonverbal rather than verbal material; It proposes problems of action rather than of theory; It fa-'rrs the individual who has been Interested In mechanics, engineering, ana outdoor activities, and who has shown Initiative and aggressiveness, rather than the Individual who has been Interested in the study of such top- ics as history, literature, economics, or politics, or who has chosen the role of a spectator more often than the role of an active participant. In addition, the material In the test favors men who can be described as Inde- pendent and resourceful, who have educated themselves and developed the ability to make sound Independent Judgments, In contrast with men who may be equally Intelligent, but who are less agresslve, Independent, and re- sourceful. Instructions for the examination are quite simple and are read by the applicant before he begins the test. Most Individuals are able to finish the test In less than two hours. Speed Is not considered In the final score, as the test Is designed to measure capacity or level of ability, rather than speed. Various types of Items are Included for the purpose of Insuring that Individuals who pass the test will have the well-balanced aptitude that Is required for success in air-crew assignments at the present time. C. Specifications for the Various Parts of the Examination 1. Part I — Vocabulary Each Item In this section consists of a short clause, phrase, or sen- tence (the stem) followed by five choices, one of which Is the answer. The stem usually contains a single word whose meaning must be Interpreted, but It may call for Interpretation of a phrase or clause. One of the five possible answers Is clearly the best Interpretation of the test; word or phrase. 3-2857,AF The purpose of the vocabulary section is to make possible the selec- tion of men who have good general Intelligence and are able to comprehend and understand written directions. Vocabulary tests have been found to predict the ability to understand and remember the sort of material that Is covered In air-crew ground schools, where the student must remember what he reads and hears. All words tested in Part I are taken from Army Technical Manuals or Army Field Manuals, chiefly from Army Air Forces Manuals. Words that fall Into any of the following categories are not used: (1) Words belonging to a highly specialized field (e.g., oscillograph, mesomorphic) (2) Words with a special aeronautical or military usage (e.g., platoon, azimuth) (3) Words of a definitely unmilItary, unpatriotic or otherwise undesirable connotation (e.g., sadistic, boondoggle) (4) Words that are chiefly literary in connotation (e.g., decorum, saffron) 2. Part II — Reading Comprehension This section consists of reading passages of from 200 to 250 words each, with several questions on each passage. Five choices are given for each question. The purpose of this section is to select Individuals who can read and comprehend the sort of material that they must study and apply In aviation training. This section, like the vocabulary section, Is a measure of general intellectual ability. All reading passages used in Part II are taken from Army Technical Manuals or Field Manuals. Most of the passages develop or explain some pro- cedure, principle, or generalization. Passages that are entirely of a literary, descriptive, enumeratlve, or factual nature are not Included. 3. Part III — Practical Judgment This section contains Items measuring Judgment In practical situations. The Items describe a problem situation In a few brief sentences and offer five alternative procedures or solutions, one of which is clearly the best answer. The problems favor the individual who has had a wide background of practical experience. Persons taking the test are not able to choose the best solution by applying any single rule, or by following any strictly logical or deductive reasoning process-. The answers do not depend entirely on simple reasoning or logic, although reasoning should be employed In test- ing various solutions. The problems do not require any specific Information. Choice of the correct answer to each Item does require, however, that the Individual weigh the practical advantages and disadvantages of each proce- dure and select the most efficient solution from the five presented. The purpose of this section Is to pick out Individuals who are re- sourceful and who are able to "size up" a situation and exercise sound In- dependent Judgment. It Is designed to measure some of the requisite char- acteristics of men who are to be placed In positions of responsibility and leadership. 11 3-2857,AF 4. Part IV — Mathematics This section contains Items covering simple mathematics. Each Item consists of a mathematics problem and five choices, only one of which Is the correct answer. Questions cover such topics as: (l) facility In handling numbers, figures, and quantitative relations; (2) knowledge of the mechanics of computation; (3) accuracy of computation; and (4) simple mathematical reasoning. The nature of the Items Is such that they can be answered by very capable boys with as little as years of high-school mathematics. This section is of special significance for selecting navigation stu- dents. However , .It Is designed to measure many of the skills and quanti- tative abilities that are required of other air-crew members. The section also contains problems requiring the ability to use and interpret data con- tained In graphs, tables, and charts. 5. Part V — Alertness to Recent Developments This section contains Items that measure Information about recent de- velopments in the world. Recent developments from each of the following fields are chosen for the Items: (1) aviation, (2) science, and (3) mili- tary affairs. Only happenings or facts that are of general significance and that have been given wide publicity are used, so that all persons who are alert and Interested in these fields will have had an opportunity to acquire the Information. This section of the test Is designed to measure: (1) motivation and Interest, as evidenced by the recall of Important facts, events, discoveries, etc., In fields In which aviation cadets are normally Interested; (2) gen- eral alertness to what Is happening In the world; and (3) general level of information in fields other than those covered by formal study In school. The section measures Indirectly certain qualities of personality, habits of noticing and keeping up with recent developments, and other characteristics that are indicative of alertness, Independent thinking, and capacity for leadership. 6. Part VI — Mechanical Comprehension This section contains a number of diagrams, pictures, or perspective drawings. Each diagram Is followed by an explanatory paragraph and several questions to determine whether or not the person talcing the test comprehends the operation of the device, the principle Involved in the drawing, and the explanation. The diagrams are largely self-explanatory, so that individuals with good aptitude but with no training In mechanics or mechanical drawing and no previous experience with the devices portrayed can select the correct answers to the questions. All essential parts are labeled, so that a man unfamiliar with conventional drawing symbols can Identify fixed and movable parts, bearings, openings, etc. Machines or devices with which some Indi- viduals could be expected to be very familiar are not used. This section measures the ability to comprehend spatial relations, the ability to understand mechanical movements, pressure relations, and the principles of operation of simple mechanical devices. It measures chiefly non'verbal ability and favors men with natural mechanical aptitude. 5-2857 ,AF SECTION IV DEVELOPMENT AND APPROVAL OF THE AVIATION CADET QUALIFYING EXAMINATION The first form of the Aviation Cadet Qualifying Examination (Test AC-IO-A) was constructed hy the Psychological Division, Office of the Air Surgeon, It was released to the Adjutant General's Office for printing and distribution to Aviation Cadet Examining Boards on December £9, 1941*. A. Construction of Preliminary Forms The examination was written in conformity with detailed specifications, part of which are Included In the preceding section. The original construc- tion of Items was accomplished by the staff of the Test Construction Section of the Psychological Division, Office of the Air Surgeon. After they were written, these preliminary Items were criticized and revised by members of the staff and prepared for further criticism by special experts. On December 18th and 19th, 1941, a number of Special Consultants, all of whom are recognized specialists and leaders In the field of test con- struction and test validation, were called in to assist with the final editing and preparation of Test AC-IO-A. The final form of the test was completed and ready for try-out by January 1st, 1942. B. Approval of the Qualifying Examination by the Army Air Forces Preceding the release of the first form of the Qualifying Examination, It was submitted for examination and approval to a number of Army officers in the Offices and Commands most directly concerned with the selection and training of aviation cadets. Final acceptance of the examination and approval of the procedures for Its use was made by a special board appointed by che Commanding General of the Army Air Forces. The board was composed of General Walter R. Weaver, General Ralph P. Cousins, Colonel Walter F. Kraus, and Colonel Luther S. Smith. On January 13th and 14th, this board met with Colonel Harris Jones, United States Military Academy; Dr. Jerome Hunsaker, Massachusetts Institute of Technology; and Dr. Walter V. Bingham, Chief Psy- chologist, Personnel Procedures Section, Adjutant General’s Office. The group considered the procedure for the appointment of aviation cadets and inspected Form AC-10-A of the Aviation Cadet Qualifying Examination. In its report, the board stated, ’’These requirements, a major feature of which is the use of ’screening tests’ In lieu of formal educational requirements, are considered a distinctive step forward.” (Paragraph 2 a). Following the recommendation of this board, Test AC-10-A was officially put Into use on January 15, 1942. About two hundred Aviation Cadet Examin- ing Boards throughout the country Immediately began to administer the test to applicants. Test AC-IO-A was used for approximately two and one half months, after which It was replaced by Test AC-IO-B. Other Improved forms have been released periodically. 5-2857,AF SECTION V STANDARDIZATION OF THE AVIATION CADET QUALIFYING EXAMINATION One of the advantages of an objective examination, such as the Aviation Cadet Qualifying Examination, Is that it can be standardized on different populations. Once this has been done, a number of accurate estimations can be made of the scores which other men will make on the test. The number of men from a given population who will make any given score can be predicted with considerable accuracy. After proper standardization of a test, the passing mark can be adjusted to admit any desired number of applicants; that Is, the effect of changing the passing mark can be predicted. In order to collect data for such standardization purposes, Test AC-10-A was administered to several groups of men, Including aviation cadets who had been selected for bombardier, navigator, or pilot training under the former two-year college requirement, enlisted men selected for pilot training by their officers, hlgh-school seniors, and West Point First Classmen. Informa- tion was also secured regarding the number of men in selected colleges who can qualify as aviation cadets. In addition to data secured from these spe- cial studies, test scores are available on over 100,000 applicants who took Test AC-10-A at Aviation Cadet Examining Boards. A. Scores Made by Aviation Cadets and by Aviation Students Who Were Selected Under Former Requirements Early In January a group of 1,104 aviation cadets was given the trial edition of Test AC-10-A at the Air Forces Replacement Training Center, Maxwell Field, Alabama. This group contained 683 cadets who were scheduled to be trained as pilots, 221 who were to be trained as navigators, and 201 who were to be trained as bombardiers. All of these cadets were men who had met the 2-year college requirement or had passed either the 2-year college educational examination for pilot candidates or the special examination given applicants for bombardier and navigation training. In the group tested there were a few cadets who had been eliminated from pilot training schools, and who had been accepted for training as bombardiers or navigators on the recommendation f Faculty Boards at the schools from which they were eliminated. A total of 1,360 cadets were tested at Kelly Field, Texas, at about the same time. The distribution of scores made by the total group of 1,104 cadets at Maxwell Field Is shown In Figure 1. At the same time that aviation cadets were examined at Maxwell Field, 184 enlisted men who had been selected for pilot training were given Test AC-10-A. The distribution of scores for this group Is shown In Figure 2. It will be noted that as a group the enlisted men made scores very much be- low those made by the aviation cadets. The results on 1,360 cadets and 286 students tested at Kelly Field were similar. After careful consideration of these data and estimation of the probable number and quality of future applicants, the passing mark on Test AC-10-A was set at a score of 90*. It was believed that the passing mark of 90 would give fair assurance that the quality of men selected by the test would be as high as It was when men were selected on the basis of education, and that It would also Insure sC sufficient Increase in the number of applicants qual- ified. The shaded areas on Figures 1 and 2 Indicate the number of men who met the former requirements for aviation cadet training but who did not meet the *The score is the number right plus one-fifth of the omitted items. 14 5-2857,AF % 90 or above = 83.32 % below 90 = 16.68 Percent of Cases FIGURE L SCORES MADE ON TEST AC-10-A BY MEN ALREADY ENLISTED AS AVIATION CADETS FOR TRAINING AS PILOTS, BOMBARDIERS OR NAVIGATORS % 90 or above = 49.46 % below 90 = 50.54 Percent of Cases Scores FIGURE 2 SCORES MADE ON TEST AC-10-A BY ENLISTED MEN ALREADY SELECTED FOR PILOT TRAINING 5-2857, AF passing mark of 90 on the Qualifying Examination. This amounted to 16.7% of the aviation cadets and 50.5% of the enlxsted men. ■ # The standards established by the Qualifying Examination are such that they prevent the acceptance for training of aviation cadets who are as low In aptitude as the ’’owest groups In former classes. Applicants admitted under present who have not attended college may lack some of the specialized Information and some of the social graces that are acquired during attendance at a college or university. However, In aptitude for fly- ing they rank as high or higher than applicants admitted prior to January 15th, 1942. The difference between aviation cadets and student pilots Is striking. The group of student pilots Included enough men who made high aptitude scores however, to indicate that a large number of satisfactory applicants can be selected from the ranks of enlisted men In the Army. B. Scores Made by First Classmen at West Point The distribution of scores made by First Classmen at the United States Military Academy Is shown In Figure 3. The highest man made a score of 145 and the lowest a score of 105. No man failed the test. Some comparisons between these West Point Cadets and other groups are of Interest. The lowest West Point Cadet made a score that was better than the scores of almost half of the aviation cadets, and better than scores made by 69% of men applying for aviation cadet training during the winter of 1942. On the other hand, about 2% of the enlisted pilot students and 3% of the hlgh-school boys made scores that equalled or surpassed the score of the average West Point Cadet. C. Scores Made by High School Seniors Data on a group of hlgh-school senior boys are given In Figure 4. The significant fact to be noted from this figure Is that approximately 40% made scores of 90 or above. The test was administered at two high schools which are considered to be fairly representative of schools throughout the country. If this assumption Is correct, then 40% of hlgh-school seniors In the United States can now meet the mental qualifications for air-crew training. The average hlgh-school senior Is 17.5 years of age at the time of graduation and, therefore, will be eligible to apply for aviation cadet training the next year. Assuming that half of the boys who finish high school are Inter- ested in air-crew training, this means that, so far as the mental qualifi- cations are concerned, a steady flow of approximately 120,000 men In the 18-year-old group will be available each year. This number will be reduced In proportion to the number of men who are disqualified on physical grounds.* At the present time, In addition to last year’s hlgh-school graduating classes, the classes of earlier years contain many men between 18 and 26 who could pass the Qualifying Examination. Also, there are some men In each age bracket who do not have a hlgh-school education but who are Interested In aviation and could pass the Qualifying Examination. These estimates are based on the following data: Approximately 600,000 men have been graduated from high school each year recently. Of these. It Is estimated that 40%, or 240,000, could pass the Qualifying Examination. No exact figures are available on the percent of hlgh-school students who can meet the physical requirements for aviation cadets, or on the percent who are likely to apply for training. 5-2857,AF % 90 or above = 100 Percent of Cases FIGURE 5 SCORES MADE ON TEST AC-10-A BY FIRST CLASSMEN AT THE UNITED STATES MILITARY ACADEMY % 90 or above = 39.06 % below 90 = 60.12 Percent of Cases Scores FIGURE 4 SCORES MADE ON TEST AC-10-A BY A REPRESENTATIVE GROUP OF HIGH SCHOOL SENIORS 3-2857,AF D. Scores Made by College Men The Aviation Cadet Qualifying Examination* has been tried out in a num- ber of colleges. In Figure 6 is shown the percent of freshmen and sophomores who passed the test in four different colleges. The marked difference that was found in the percent passing in the four different colleges is very striking. This is explained in terms of the nature of the schools. College A is a large, class I institution, where entrance requirements are very high. Colleges B and C are typical state universities. College B has a large pro- portion of Engineering students. College D has very low entrance require- ments. The true figure representing the number of college freshmen or sopho- mores throughout the country who could qualify on the test probably lies somewhere between the figures for Colleges B and C. The percent passing is considerably higher for those college students who volunteer, because volun- teers on the whole are superior in aptitude and interest to those who do not volunteer. E. Test Scores of Applicants In Figure 5 Is shown the distribution of scores made by a group of 1,000 men who are representative of those who applied for training during the first two months following release of the Aviation Cadet Qualifying Examination on January 15, 1942. There is some evidence to indicate that the men who applied during these early months of war were better qualified and better educated than men who have applied since that time. A large number of these early applicants were college men. Since a smaller number of college men are now applying, the percent of applicants who pass the examination has decreased slightly. It will be seen that 60% of this group of 1,000 applicants passed the Qualifying Examination (l.e., obtained a score of 90 or above). This indi- cates that the men who are sufficiently interested in air-crew training actually to try to enlist are a superior group. On the average they made better test scores than boys in high-school senior classes. They were also superior to the group of enlisted men who had been selected for pilot train- ing. Their scores ranged from as low as 30 to above 140. About one percent made a score higher than 130. The most frequent score was one between 100 and 109. Another way of evaluating the quality of applicants and of the men who now are qualified for air-crew training is to determine how high their scores are on some other test that has been standardized on a different population. The Army General Classification Test has been given to a large number of cadets and it has been determined that a score of 119 on that test is equivalent to a passing mark of 90 on the Qualifying Examination. The correlation between the two tests, when given to an aviation cadet population, is only 0.64. This means that they are measuring fairly different sets of abilities. Al- though the two tests are different, it is reasonable to assume that since approximately one-fourth of the men who are inducted into the general army • obtain scores of 119 or higher on the G. C. Test about an equal proportion could pass the Aviation Cadet Qualifying Examination. A point of interest, which may be considered here, is the relation of test score to education. The Aviation Cadet Qualifying Examination was de- veloped to select men with aptitude for air-crew assignments regardless of the extent of their formal education or their mastery of school subjects. *The test used In this case was Test AC-10-B, but the percentage passing AC-10-A would be quite similar. 5-2857,AF 1% 90 or above = 60.00 1% below 90 = 40.00 Percent of Cases Scores FIGURE 5 SCORES MADE ON TEST AC-10-A BY A REPRESENTATIVE GROUP OF 1,000 APPLICANTS College A College B College C College D FIGURE 6 PERCENT OF FRESHMEN AND SOPHOMORES TN FOUR DIFFERENT COLLEGES WHO PASSED TEST AC-10-A The difference in percent of students passing at each of these col- leges is striking, and indicates the differences between schools. The four colleges ranged from one with very high entrance require- ments to one with very low entrance standards. 5-2857 .AT Yet It Is known that those who seek the best education tend to be the more able Individuals, while those who drop out of school earliest tend to be Individuals of less ability. Therefore, It Is to be expected that college men will tend on the average to be more Intelligent and to make higher scores on aptitude tests as well as on subject-matter tests than men who do not go to college. The uniformly high scores made at the United States Military Academy represent a typical example of this relation between edu- cational level and test score. Studies of the relation between education and score on the Qualifying Examination have been reported by several of the Aviation Cadet Examining Boards. In Figure 7 Is shown the results of examinations given to 1,670 men by the Philadelphia Aviation Cadet Examining Board during the period from January 15 to April 1, 1942. The upper left-hand circle in Figure 7 represents'the total number of men applying for examination. The different sectors of this circle represent the percent of the total that is made up of those men who had less than a hlgh-school education, those with one year of college training, and those with two or more years of college training. The circle on the right represents the men who successfully passed the Avia- tion Cadet Qualifying Examination. This circle Is smaller, because only 59% of the applicants qualified. The sectors of this smaller circle Indi- cate the educational level of the men who succeeded in qualifying. It can be seen that about 30% of the applicants qualified for aviation cadet train- ing by this board were men who had attended college. About two-thirds of those qualified were men who had graduated from high school, while only 4% had less than 4 years of hlgh-school training. The bars at the lower part of Figure 7 represent the percent of men In each educational group who passed the examination. It will be noted that the percent passing varied from 80% for college men to 20% for men with less than a hlgh-school education. As was Indicated earlier In this report, there are not enough college men to supply the manpower needed by the Air Forces. The majority of men who are now selected for aviation cadet training are hlgh-school graduates. It Is significant that a small number of college men now fall to qualify as aviation cadets. Many hlgh-school graduates show higher aptitude than this lower group of college men. An interesting report on the Qualifying Exami- nation, made by one of the Examining Boards,* contains the following inter- pretations of the relation between education and test scores: "As might be expected, the average over-all grade made by the applicants taking this examination Increases with the number of years'of formal schooling which they have completed . . . . Practically all applicants with more than two (2) years of college training passed. "Perhaps the most Interesting thing about the report is that we secured 179 qualified applicants out of the hlgh-school group and an additional 73 out of the one-year college group, or a total of 252 which we would not have secured If the qualifications for appointment were still held at the two- year college requirement. "It Is also Interesting to note that 15 out of 107 men with two years of college or more were disqualified. It is per- haps safe to assume that these 15 men would not have been satisfactory as officer material Inasmuch as they could not pass a relatively easy screening test." of the Aviation Cadet Examining Board, San Francisco, California to the Commanding General, West Coast Air Force Training Center, March 25, 1942. 3-2857,AF COMPARISON OF THE EDUCATION OF ALL APPLICANTS FOR AVIATION CADET TRAINING WITH THE EDUCATION OF MEN WHO PASS THE AVIATION CADET QUALIFYING EXAMINATION EDUCATION OF MEN WHO QUALIFIED EDUCATION OF ALL APPLICANTS than four years of high school Two or more years of college Less than four years of high school One year of college 11% -fwo" or more years of college 4$ i 12% 11% f One \ year of college IfljC W High school graduate eefo High school graduate 66/0 EDUCATION PERCENT OF EACH EDUCATIONAL GROUP PASSING TWO OR MORE YEARS OF COLLEGE (N = 178) 80% ONE YEAR OF COLLEGE (N = 193) 79% HIGH SCHOOL GRADUATE (N = 1119) 59% LESS THAN FOUR YEARS OF HIGH SCHOOL (N = 180) 20% FIGURE 7 These- data are based on all men who were given Test AC-IO-A by the Phila- delphia Aviation Cadet Examining Board from January 15, 1942 to March 1, 1942. The reaction of this examining board Is characteristic of that of other boards and of officers concerned with aviation cadet selection. The final decision regarding the value of the Qualifying Examination, however, rests upon the degree to which It selects men who make successful pilots, bombar- diers, and navigators. In the next section, results of studies of the re- lation between scores on Test AC-10-A and success In air-crew training are presented. 3-2857,AF SECTION VI SUCCESS OF TEST AC-10-A IN PREDICTING ELIMINATIONS FROM AIR-CREW TRAINING The real test of the value of any practical procedure is how well it works. The degree to which the Aviation Cadet Qualifying Examination is selecting men who are qualified for air-crew training is attested by the results that are presented below. A. Procedures Followed In the Validation of Test AC-10-A Before the first form of the Qualifying Examination was released for general use, the test was given to a large number of men (2,934 in all) who had already been selected for air-crew training. It should be em- phasized that these men had already been assigned to specific types of training and that the scores that they made on the Qualifying Examination did not affect their classification In any way. The Qualifying Examination was given to these men while they were still in pre-flight school. Their records through specialized Air Forces schools were followed carefully. The number who successfully completed navigation school, bombardier school, or primary, basic, and advanced flying schools was determined. Records likewise were secured of the men who were eliminated from training. Only records of men eliminated for flying reasons, because of academic difficulty, or at their own request, were Included. Men eliminated for physical reasons were not Included. Various statistical analyses have been made In determining the validity of the total score on Test AC-10-A and the validity of the scores on each of Its six component sections. One type of study has been a comparison In the original try-out groups of the elimination rate for men who passed the test with that for men who failed the test. Scores on the total examination and scores for each part have been broken up into different score levels and comparisons have been made between the men scoring at these different levels and rate of elimination from training schools. Bl-serlal coefficients of correlation have been computed between part scores and success In training schools. The intercorrelations of the various sections of the test have been computed and a multiple coefficient of correlation for predicting pilot suc- cess has been determined. The reliability coefficients of the various sec- tions of the test have been computed. Item analyses have been made to deter- mine the validity of each Item In the examination. Item analyses have In- cluded the determination of the number of successful and of eliminated cadets who chose each one of the five choices to each Individual Item In the exam- ination. The more significant results of these studies are presented In the present section of this report and In the appendix. B. Comparison of Men Who Scored 90 or Above on Test AC-1Q-A with Men Who Scored 59 or Below ' The majority of the aviation cadets who were given Test AC-10-A for re- search purposes passed the examination (with a score of 90 or above). About half of the aviation students who were tested for research purposes passed the examination. As one Indication of the practical value of the Aviation Cadet Qualifying Examination, a study has been made of the elimination rate among men who passed and those who failed when it was given for research purposes. The results of this analysis are given in Table IV. It will be noted that 37.7% of the aviation cadets who scored 90 or better on the test were eliminated from pilot training, whereas the elimination rate for cadets who scored below 90 was 57.8%. For aviation students, the comparable figures were 32.2% and 55.2%. The prediction of navigation-school success was even more striking. Only 18% of the.cadets scoring 90 or above were eliminated 3-2857,AF TABLE IV COMPARISON OF THE ELIMINATION RATE FOR MEN PASSING TEST AC-IO-A WITH ELIMINATION RATE FOR MEN FAILING THE TEST Score of 90 or (Passing) Above Score of 89 or (Falling) Below Air-crew Specialty Total Number Number Eliminated Percent Eliminated Total Number Number Eliminated Percent Eliminated Pllots-Cadets 443 167 37.7 102 59 57.8 Pilots- Student s 87 28 32.2 87 48 55.2 Pllots-Total 530 195 36.8 189 107 56.6 Navigators- Cadets 200 36 18.0 21 11 52.4 Bombardlers- Cadets 156 14 9.0 35 4 11.4 These data are based on the complete training records of Class 42-G for pi- lots, and Classes 42-6, 42-7, and 42-8 for navigators and bombardiers. These men were tested at Maxwell Field, Alabama, during the First part of January, 1942. All cadets and students already had been selected and clas- sified for air-crew training. Test results were used for research purposes only. 5-2857,AF while 52.4% of the cadets who scored below 90 were eliminated. The test was less successful In predicting eliminations from bombardlerlng school. One reason for this lower validity for bombardiers may have been the fact that only a little over 9% of the bombardier students were eliminated In the classes that were studied. In addltlop to the data Included In Table IV, which are for men tested at Maxwell Field, data are also available on 1,360 aviation cadets and 286 aviation students tested at Kelly Field. The elimination rates for Kelly Field Aviation Cadets who passed and failed the examination were 41.1% and 60.4%, respectively. The elimination rate for aviation students who passed the examination was 39.8% as compared with 56.3% for students who failed the examination. The Kelly Field data are thus In close agreement with those obtained at Maxwell Field. A consideration of these data must lead to the conclusion that the first form of the Qualifying Examination was a very useful test for select- ing applicants for aviation cadet training. The results of preliminary re- search testing indicate that men who score 90 or above on this examination have a much greater chance of graduating from training than do the men who score 89 or below. Since these results were obtained on aviation cadets, all of whom were college men, and on aviation students, all of whom had been carefully selected for pilot training, it is safe to conclude that If the test could be validated on an unselected group of students, it would be found to be an even better selection device than Is Indicated by the present findings. C. Research Findings on Various Parts of the Qualifying Examination Studies have been made of the elimination rate in comparison with scores on the six parts of the Qualifying Examination. In order to simplify this analysis, scores on each part of the test have been recorded as "A”, "BM, ”0", MDU , or "E". These scores are defined as follows: A = highest 7% B = next highest 24% C = middle 38% D = next lowest 24% E = lowest 7% The percent of eliminations for the group of men Included In each one of these different grades Is shown In Figures 8 to 13. The various parts of the examination are of varying Importance for pre- dicting success In pilot, bombardier, and navigator schools. The relation between each of the parts of the examination and a particular air-crew spe- cialty Is shown in these figures. 1. Mathematics: The mathematics part gave a very good prediction of success In navigation training. It was less accurate in predict- ing the success or failure of pilots and bombardiers. From Figure 8 it can be seen that well over 90% of the aviation cadets who scored “A” or "BH on the mathematics part later graduated from training school, whereas less than 20% of the cadets who scored MEH were graduated. It can be stated definitely that men who are deficient in the mathematics skills measured by this test should not be assigned to navigation training. . ! 2. Vocabulary: The vocabulary part was one of the more extensive sec- tlons of Test AC-10-A. Present results show that it was of special value only for the prediction of success in navigation school. The relation between the vocabulary score and navigation training is shown in Figure 9. One interesting thing about this figure is that the lowest elimination rate was for men who scored "BM on the 3-2857,AF vocabulary part rather than for men who scored HAM. At present, It cannot be said definitely whether this is a true relationship or whether it is a chance difference. While the vocabulary part was not of value in differentiating between successful and elimi- nated aviation cadets in a group selected under the two-year col- lege requirement, it did give a fair prediction for aviation students (Table 11). Since aviation students tested much below aviation cadets, these results might be Interpreted as showing that when a man's vocabulary is very low, this deficiency is a handicap in pilot training schools. 3. Heading Comprehension* This part showed a positive correlation with success in pHot training and a very high correlation with navigation training. In addition, it was the best part for pre- dicting success in bombardierini. As shown in Figure 10, none of the aviation cadets who scored ffAtt on this section were eliminated from bombardlerlng training, while 25% of the cadets scoring MEH were eliminated. The elimination rate from bombardier school was only 9% for all the cadets tested. 4. Knowledge of Recent Developments; This part was designed to measure rntTfest. ' It was Ibuhd’"’that"~scores on this part predict success in pilot training with a fair degree of success. This relationship is shown in Figure 11. A more detailed analysis of the specific questions in this part has shown that knowledge of recent develop- ments in the field of aviation is much more important in the selec- tion of aviation cadets than is knowledge of recent developments in related fields. 5. Judgment: The Judgment part has been determined to be the best sec- tion for predicting pilot success. The results are shown in Figure 12. They indicate that if it were possible to assign for pilot training only men who scored MAtt on Judgment, and present standards were maintained, the elimination rate could be reduced to 18%. On the other hand, if only men scoring MEM were assigned for pilot training, then the elimination rate would be 76%. The Judgment part is of special interest since it has a relatively low correlation with navigation training and a zero correlation with success in bombardlerlng. These findings indicate that the kind of Judgment measured by this test is essential for success in pilot training but not for training for the other air-crew positions. 6. Mechanical Comprehension: This part was highly useful in the selec- t'l'on of pilots. The degree to which it predicts pilot success is indicated in Figure 13. it selects pilots almost as well as the Judgment part. The elimination rate for men who scored **E” was 3 times as great as for men who made a grade of "A". These results indicate that mechanical comprehension is one of the more Important aptitudes required of the pilot. Scores on this part of the Quali- fying Examination also correlated positively with success in navi- gation and in bombardlerlng. D. Additional Statistical Results Detailed statistical results obtained on Class 42 G and on Classes 42-6, 42-7, and 42-8, tested at Maxwell Field, are presented In Tables 1 to vl In the appendix. In Table 1 are given the mean test scores made by cadet pilot graduates and elimineas on various parts of the Qualifying Exam- ination and the correlation between test scores and school records of avia- tion cadets In pilot training. In Table 11, corresponding data are given for aviation students. In Table 111 are summarized the data on a group of aviation cadets who had been classified for navigation training. In Table iv, the data for bombardiers are presented. 5-2857,AF RELATION BETWEEN SCORE ON PART IV, MATHEMATICS SECTION, TEST AC-10-A, AND ELIMINATION RATE IN NAVIGATION TRAINING TEST GRADE ELIMINATIONS FROM TRAINING A 7% B 6% 19% C r 28% E 81% FIGURE 8 These data are based on 221 Navigators of Classes 42-6, 42-7, and 42-8, tested for research purposes at Maxwell Field, Alabama. The tests were given in January, 1942 before navigation training was begun, but the results were not used for classification. RELATION BETWEEN SCORE ON PART I, VOCABULARY SECTION, TEST AC-IO-A, AND ELIMINATION RATE IN NAVIGATION TRAINING TEST GRADE ELIMINATIONS FROM TRAINING A 19% B 9% C 21% D 27% E 45% FIGURE 9 These data are based on 221 Navigators of Classes 42-6, 42-7, and 42-8, whorwere tested for research purposes at Maxwell Field, Alabama. The tests were given In January, 1942 before navigation training was begun, but the results were not used for classification. 5-2857,AF RELATION BETWEEN SCORE ON PART II, READING COMPREHENSION SECTION, TEST AC-IO-A, AND ELIMINATION RATE IN BOMBARDIER TRAINING TEST GRADE ELIMINATIONS FROM TRAINING A 0% 6% B C 10% D 10% 25% E FIGURE 10 These data are based on 191 Bombardiers of Classes 42-6, 42-7, and 42-8 who were tested for research purposes at Maxwell Field, Alabama. The tests were given in January, 1942 before bombardier training was begun, but the results were not used for classifica- tion. 3-2857,AF RELATION BETWEEN SCORE ON PART V, KNOWLEDGE OF RECENT DEVELOPMENTS, TEST AC-IO-A, AND ELIMINATION RATE IN PILOT TRAINING TEST GRADE ELIMINATION RATE IN PRIMARY, BASIC, AND ADVANCED PILOT SCHOOLS A 26% B 34% C 41% D 49% E 64% FIGURE 11 These data are based on 719 Pilots (Aviation Cadets and Aviation Students) of Class 42-G, who were tested for research purposes at Maxwell Field, Alabama. The tests were given in January, ±942 before flying training was begun, but the results were not used for classification. 5-2857,AF RELATION BETWEEN SCORE ON PART III, JUDGMENT SECTION, TEST AC-IO-A,AND ELIMINATION RATE IN PILOT TRAINING TEST GRADE ELIMINATION RATE IN PRIMARY, BASIC, AND ADVANCED PILOT SCHOOLS 18% A B 33% C 38% 55% D E 76% FIGURE 12 'These data are based on 719 Pilots (Aviation Cadets and Aviation Students) of Class 42-G, who were tested for research purposes at Maxwell Field, Alabama. The tests were given in January, 1942 before flying training was begun, but the results were not used for classification. 3-2857,AF RELATION BETWEEN SCORE ON PART VI, MECHANICAL COMPREHENSION SECTION, TEST AC-IO-A, AND ELIMINATION RATE IN PILOT TRAINING TEST GRADE ELIMINATION RATE IN PRIMARY, BASIC, AND ADVANCED PILOT SCHOOLS A 22% B 33% C 39% D 55% 67% E FIGURE 15 These data are based on 719 Pilots (Aviation Cadets and Aviation Students) of Class 42-G, who were tested for research purposes at Maxwell Field, Alabama. The tests were given in January, 1942 before flying training was begun, but the results were not used for classification. 8-2857,AF The Intercorrelations of the various sections of Test AC-10-A are given In Table v of the appendix. On the whole, the Intercorrelations of the various parts of the examination are low. Considering the reliability coefficients of the parts, we may Justifiably conclude that the examina- tion Is not testing one unitary ability or trait, but rather a number of fairly distinct traits. On the basis of the bl-serial correlations reported In Table 1 and the Intercorrelations reported In Table v, it has been determined that If each part of the Qualifying Examination were weighted In the optimum manner, a multiple correlation of .48 between the total test score and success In pilot training would be secured. The reliability of the various parts of the examination, l.e., the in- ternal consistency of each part Is given In Table vl. The reliability co- efficient for the total test Is 0.93. This Is a highly satisfactory reli- ability In view of the wide variety of material Included In the examination 33 5-2857 ,AF SECTION VII IMPROVEMENTS THAT HAVE BEEN MADE IN THE AVIATION CADET QUALIFYING EXAMINATION SINCE THE FIRST FORM WAS RELEASED Statistical analyses carried on during the last 6 months have furnished much vital Information about the aptitudes required for success In the va- rious air-crew assignments. Each part of the Qualifying Examination has been studied exhaustively on the basis of all of the research findings and it has been possible to Incorporate a large number of Improvements in the examination. The policy followed by the Psychological Division has been to apply new research findings to the development of each new form of the examination. Since the release of AC-10-A, forms AC-IO-B, AC-10-C, AC-10-D, AC-IO-E, AC-IO-F, AC-10-G, and AG-10-H have been constructed. A summary of the im- provements that have been Incorporated In forms G and H of the examination follows. A. Improvements In Part 1 — Vocabulary This part has been greatly reduced In length, so that It now makes a much smaller contribution to the total score on the test. This Is done because the part was found to contribute chiefly to the prediction of suc- cess In navigation training. The specifications for the part have also been changed. In place of general words taken from Army Air Forces manuals, words are now included that are related to practical mechanics or science. The test is now measuring the sort of vocabulary that is most highly developed in men with special interests in aviation, mechanics, and related activities, and it is believed that this will definitely improve its validity. B. Improvements in Part 2 — Reading Comprehension The same sort of reading comprehension items that were included in the original examination have been retained in the most recent forms. The length of the section has been doubled by the addition of items requiring the in- terpretation of data that are presented in maps, tables, graphs, and charts. The data that applicants are required to read and interpret are of the type that they will encounter if they are trained as aviation cadets. This type of material has been included on the basis of research findings from the ad- ministration of tests of this type in Army Air Forces Classification Centers. Such test items have been found to correlate significantly with pilot success. C. Improvements in Part 3 — Judgment This part has not been modified. The same type of items that were found to be so satisfactory in Test AC-10-A are still included in the new forms. An effort has also been made to introduce aspects of practical Judgment into other parts of the examination. D Improvements in Part 4 — Mathematics The length of the mathematics part has been reduced, since it was found to be less valuable for selecting pilots and bombardiers than other parts of the examination. The general types of items are the same as those that were included in early forms. E. Improvements in Part 5 — Knowledge of Recent Developments The length of this part of the examination has been increased. About half of the items are intended primarily to test the interests of pilots, 5-2857,AF and about one-fourth of the Items are Intended to test the Interests of bombardiers and navigators. The material covered In this part Is now re- stricted more closely to aviation items since this type of item was found to be most significant in predicting, air-crew success. Interests in and information concerning certain.recreational activities, such as sports, hobbles, model airplane building, etc., have been included. These activ- ities have been found to be characteristic of successful aviators. F. Improvements in Part 6 — Mechanical Comprehension The length of this part has been Increased. Some additional types of items designed to measure practical mechanical abilities have been intro- duced. More diagrams and drawings have been Included and less emphasis is now placed on the interpretation of written descriptions of mechanical devices. G. Summary of the Improvements that Have Been Made The changes in the 6 parts of the examination are such that there is much less emphasis than formerly on vocabulary or purely verbal ability, and somewhat less emphasis upon mathematics. Conversely, there is much more emphasis upon the ability to interpret data, interest in aviation, and mechanical insight. Since all of the changes are based on actual research findings and repeated validation studies, there is every reason to believe that these changes in the content and emphasis of the Qualifying Examination will greatly improve its validity as a selection test for aviation cadets. It is believed that research studies now in progress will result in addi- tional Improvements in future forms of the examination. 3-2857,AF SECTION VIII SUMMARY 1. Preceding the present emergency, the number of men selected for a- viatlon cadet training was small. The men who were selected were required to meet very exacting physical standards and to show evidence of education equivalent to 2 years of college work. With the Inauguration of the accel- erated aviation cadet training program, it became necessary to change the selection standards. 2. In order that aviation cadets could be selected from the total num- ber of men in the country who were mentally suited for air-crew training, it was decided to adopt a uniform examination for selecting men. 3. Before the first form of the Aviation Cadet Qualifying Examination was constructed, a comprehensive set of specifications was developed. These specifications called for a test with 6 parts, designed to measure a number of the traits that were considered to be most Important for aviation cadets. 4. The examination was constructed by the Psychological Division, Office of the Air Surgeon, Headquarters, Army Air Forces, according to the specifications that had been drawn up. Expert consultants were employed to assist with the final preparation and editing of the test. The first form of the examination, Test AC-10-A, was formally approved by a committee appointed by the Commanding General, Army Air Forces, and was officially adopted on January 15, 1942. Immediately, Aviation Cadet Examining Boards throughout the country began to use the test in selecting aviation cadets. It was used until Test AC-10-B was released on April 1, 1942. 5. In order to determine the percent of men in this country who can qualify for avlhtlon cadet training, the Qualifying Examination has been given to various groups for standardization purposes. The scores of high- school students and applicants at large have been determined. The examina- tion also has been given to aviation cadets and to aviation students who were selected before the Qualifying Examination was officially approved. pproxlmately half of the men who have applied for aviation cadet training in the past 9 months have passed the Qualifying Examination. One hundred percent of the graduating class at West Point passed the test, whereas slightly less than half of a group of high-school senior men made passing marks. 6. Extensive statistical analyses have been made to determine the re- lation between scores on the Qualifying Examination and subsequent success or failure in Army Air Force training schools. The results have shown that the elimination rate is much higher for men who make poor scores on the examination than for men who make high scores. Detailed analyses of each of the parts of the examination have revealed that the best parts for se- lecting pilots are those measuring judgment, mechanical comprehension, and knowledge of recent developments. The parts on mathematics and reading com- prehension are the best parts for selecting navigators. The reading comprehension part gives the best prediction of success in bombardier train- ing. The vocabulary part is not as important as other parts, but gives satis factory predictions of success in navigation training. 7. The results of statistical studies of Test AC-IO-A have been in- corporated in revised specifications for later forms of the examination. Since the changes that have been made have all been based upon research findings and have always been of such a nature as to Increase the probable validity of the examination, it is safe to conclude that the Qualifying Examination has been steadily improved. Further improvements will be made on the basis of research studies now in progress. 3-2857,AF 8. It can be concluded that the Qualifying Examination is a satis- factory device for selecting aviation cadets, since it has made it possible to Increase greatly the numbers while at the same time maintaining the in- trinsic quality of the men available for air-crew training. 3-2857,AF APPENDIX TABTJC 1 RELATION BETWEEN SCORES ON EACH SECTION OF TEST AC-IO-A AND SUCCESS IN PILOT TRAINING FOR CLASS 42-G, AVIATION CADETS Section Mean Score: Graduates Mean Score: Ellminees Standard Dev. Total Group Biserlal Correlation 1. Vocabulary 28.20 28.61 7.16 -.04 2. Beading Comprehension 12.49 12.03 2.02 .14 3. Judgment 10.28 9.10 2.06 .36 4. Mathematics 24.00 23.06 4.21 .14 5. Knowledge of Recent Developments 20.32 18.70 4.15 .24 6. Mechanical Comprehension 8.53 7.02 3.24 .29 Total Score (Rights plus credit) 105.72 100.56 15.68 .20 The part scores are based on number right without credit for omits. All data are for complete records in primary, basic and advanced training for all men in class 42-G, Aviation Cadets, tested at Maxwell Field in January, 1942. Number of cases = 545, including 319 graduates and 226 ellminees. No men who were eliminated for physical reasons are Included in the eliminated group. 3-2857,AF TABLE 11 RELATION BETWEEN SCORES ON EACH SECTION OF TEST AC-10-A AND SUCCESS IN PILOT TRAINING FOR CLASS 42-G, AVIATION STUDENTS Section Mean Score: Graduates Mean Score: Ellmlnees Standard Dev. Total Group Blserlal Correlation 1. Vocabulary 22 .'73 20.01 6.86 .25 2. Reading Comprehension 11.69 10.41 2.90 .28 3. Judgment 9.88 8.91 1.96 .31 4. Mathematics 20.53 18.87 4.78 .22 5. Knowledge of Recent Developments 18.78 17.04 6.00 .18 6, Mechanical Comprehension 7.42 5.51 3.23 .37 Total Score (Rights plus credit) 93.37 83.33 17.40 .36 The part scores are based on number right without credit for omits. All data are for complete records in primary, basic, and advanced training for all men in class 42-G, Aviation Students, tested at Maxwell Field in January, 1942. Number of cases = 174, including 98 graduates and 76 ellmlnees. No men who were eliminated for physical reasons are included in the eliminated group. 3-2857,AF TABLE 111 RELATION BETWEEN SCORES ON EACH SECTION OF TEST AC-IO-A AND SUCCESS IN NAVIGATION TRAINING FOR CLASSES 42-6, AND 42-8, AVIATION CADETS Section Mean Score:. Graduates Mean Score: Elimlness Standard Dev. Total Group Biserlal Correlation 1. Vocabulary 31.87 27.85 7.30 .32 2. Reading Comprehension 13.05 11.13 2.15 .52 3. Judgment 10.32 9.74 2.19 .15 4. Mathematics 26.26 22.66 3.26 .64 5. Knowledge of Recent Developments 20.99 19.91 3.84 .16 6. Mechanical Comprehension 8.36 5.85 3.22 .45 Total Score (Rights plus credit) 112.34 99.45 15.39 • 00 The part scores are based on number right without credit for omits. All data are for complete records In Navigation training of ail Aviation Ca- dets tested In Classes 42-6, 42-7, and 42-8 at Maxwell Field In January, 1942. Number of cases = 221, Including 174 graduates and 47 ellmlnees. Eliminations for physical reasons are not Included In the eliminated group. 5-2857,AF TABLE iv RELATION BETWEEN SCORES ON EACH SECTION OF TEST AC-10-A AND RECORDS IN BOMBARDIER SCHOOL Section Mean Score: Graduates Mean Score: Ellminees Standard Dev. Total Group Blserlal Correlation 1. Vocabulary 29.56 27.39 6.09 .18 2. Reading Comprehension 12.38 11.06 2.20 .31 3. Judgment 9.66 9.67 1.91 .00 4. Mathematics 22.60 21.00 4.35 .19 5. Knowledge of Recent Developments 20.29 19.56 3.85 .10 6. Mechanical Comprehension 7.11 6.33 3.13 .13 Total Score (Rights plus credit) 103.34 96.83 14.97 .22 These data are for 191 Aviation Cadets who were tested at Maxwell Field In January, 1942. All of these men were sent to Bombardier School and either graduated or were eliminated for bombing deficiency, unsatisfac- tory progress or at their own request. Part scores are based on the num- ber of right answers without credit for omits. TABLE v INTERCORRELATIONS OF THE VARIOUS SECTIONS OF TEST AC-10-A Section 1 2 3 4 5 6 1. Vocabulary- .49 .32 .40 .42 .43 2. Reading Comprehension .49 .38 .40 .30 .36 3. Judgment .32 .38 .45 .30 .49 4. Mathematics .40 .40 .45 .24 .50 5. Knowledge of Recent Developments .42 .30 .30 .24 .39 6. Mechanical Comprehension .43 .36 .49 .50 .39 These intercorrelatlons are based on 545 Aviation Cadets, and are for the number of right answers, without credit for omits. The Cadets in this group were all classified for pilot training. 5-2857,AF TABLE vl RELIABILITIES OF THE VARIOUS SECTIONS OF TEST AC-10-A Section Number of Items Reliability 1. Vocabulary 45 CO 00 • 2. Reading Comprehension 15 .58 3. Judgment 15 .36 4. Mathematics 30 88 5. Knowledge of Recent Developments 30 .75 6. Mechanical Comprehension 15 .71 Total Test 150 .93 These reliability coefficients were computed on a group of 370 appli- cants tested by the Minneapolis Aviation Cadet Examining Board. The reliabilities given are odd-even reliabilities corrected by applica- tion of the Spearman-Brown formula. 44 5-2857, AF