---- -----✓ -· I PL 86-36 50 USC 3605 1 •· r m DVElDWullk i lBIWWf 100'11Bf 111DWV l IDWlill5 lB11 wauJIDU f WUIW lalWUI WVCk1JBJ f • • • • • • • I No n - Responsive I ' ' - • • c nMP 'l'J mnr 1-1T nu p O _F I ru -- I T 7 l 1'AST WORD ON I A T S • • • • • Cecil Philll£S • lZ I • • 111111 B88UPll IT 88Uft1Ul I 88BBIIO b liikidtiNL TOP IICIIT Approved for Release by NSA on 08-30 202 MDR Cas ' Ir 110141 Non - Responsive Published Monthly by Pl Techniques and Standards for the Personnel of Operations VOL IV No 4 APRIL 1977 I Non - Respons i ve l · i I Non - Responsive Non - Responsive I Non - Responsive I I Non - Responsive r l j j I I j I J I I Non - Responsive I Non - Responsive QO f Jlll B RHFfll1 li Russian Professional Qualification Exam nation PQE is a topic guaranteed o stimulate lively discussion aJROng NSA's Russlan lin11uists And if you want to get into a really heated conversation spend an hour or so with an aspirant who has failed a portion of he PQE more than once -- he will have a dcfinite opinion on what can be done with the PQE The frustration despair and anger of such an aspirant are understandable for it seems to him hi career advancement and promotion are inse rably linked with passing the test To some •particularly those who have been with NS I fo' only a few years the PQE becomes the added frus ration that makes working at NSA almost unbearab lc Unfortunately perhaps fortunately in some eases some linguists eventually begin boycdtting the tests· others seek job reclassification to pursue a' career elsewhere The loseor in such cases is NSA because usually lar investments in time and money have been made fo train the aspirant to become what he will net be if he abandons the language career field • For this reason and because language professionalization is a particularly timely topic I would like to-share with CRYPTOLOG readers a sampling of the opinions expressed by discouraged aspirants Some expressed feelings which are no more than vague co111plaints about the ''unfairness and irrelevancy of the PQE others have voiced more compelling evaluations and arguments Jt is my intention here to note only those valid and reasonable complaints which are in my opinion worthy of the Language Career Panel's consideration The audience I have in mind is those gency linguists who are receptive to criticism and who are in a position to effect meaningful changes in PQE policy I also wish to address the aspirants those who are finding it difficult to pass all parts of the PQE and those who will soon be taking the tests for the first time to ask for their forbearance as well as their suggestions and to reassure them that the testmakers are aware of their frustration and are striving to devise better tests I-------------· PL 86-36 50 USC 3605 1 2'f0 i73 7 April 77 CRYPTOLOG • Page 7 III PiPJ L a There is clearly a need for a testing program Parts II and 118 of the PQE are the major roadblocks to increasing the total nU111ber of L3 to determine which Agency linguists are of a Russian linguists it behooves all of us to professional caliber Academic credentials scrutinize the testing policy and to determine alone are not enough However the effectiveness of Parts IIA and IIB in particular in fill- what can be done to increase the annual yield ing this need is clouded by the following factors of L3 linguists Ostensibly the purpose of the Russian PQE is • the majority of aspirants have had no to identify those linguists whose language work practical experience in working with the can reliably be said to be of a professional kinds of materials on which they are being level and who clearly demonstrate an ability to tested -- in many cases they may never be handle a diversity of language tasks The work required to work with such materials of such a linguist should in theory require little or no language check and could conceiv• there is no existing training program to ably be a releasable NSA product But in pracprepare aspirants for Part IIA classified tice this is not what happens NSA generally translation does not publish translations and we are for the • present NSA reporting policy discourages 1110st part specifically instructed to conceal the publishing of translations and those the fact that our reports are derived from comthat are published from operational units munications intelligence Too it is unlikely are generally subjected to as 111any as three that any Russian linguist would be required to levels of language check be solely responsible for the transcription or translation of an important conversation When • available training packets for Part llA do not include model translations with which faced with a difficult language task a truly the aspirants studying for the examination professional linguist will always seek the advice and help of others equally or better qualican compare their own efforts fied But in a test situation no such recourse • an alarming number of professionalized lin- is available nor can an aspirant in the course guists are unwilling pe rhaps unable to of an examination avail himself of a technical help aspirants prepare for the examinati n library The aspirant willy-nilly decides which dictionaries and aids to bring with him to • there is at present no standardized method the examination If he guesses wrong about what of establishing a pass-fail threshold for Part IIA -- thus giving rise to the likeli- technical fields will be reflected in the test materials he will discover that he has the hood of test grading being reduced to a wrong aids for the particular test placed before subjective exercise him We have all seen aspirants carrying • it is becoming increasingly difficult co mountains of books to a PQE in the hope of cover• find suitable materials for the testing ing all contingencies No true test of a linguist • r program due to compartmentalization and ability should hinge on luck This problem can other problems of course be easily remedied by specifying what 'aids an aspirant should bring to the test and In this light we need to ask ourselves a few then ensuring that those aids deal with all the searching questions Such as does a pass on Russian language problems to be encountered in either Part IIA or JIB tell us with any certain- the examination vocabulary technical terminol ty that an aspirant is a professional linguist ogy place names etc There are indications Or does it tell us only that the aspirant has that this will be the case in future PQEs finally succeeded in passing the test And A major problem with Part IIA stems from finally how many professionalized linguists the lack of an organized course of instruction could pass the test a second time There are no aimed at preparing aspirants to take the test clear answers to such questions but on the A transcriber-aspirant can prepare for Part 11B basis of the large nUJ1ber of linguists who re classified transcription by completing course peatedly fail it ·is obvious that for many RS220 Intermediate Russian Transcription He Agency Russian linguists Parts IIA and l B in can also make use of the training materials their present form are a formidable perhaps an available in A64 In the case of Part JJA unfair barrier to the professionalization which however no comparable training program exists has become an important criterion for promotion RS200 Intermediate Russian Translation once included a segment on classified translation There are of course many who feel that but this was discontinued several years ago the ranks of certified linguists should in fact be elitist i e few in number and that Parts A classified translation course could either be llA and IIB are especially crucial in determin- offered by the National Cryptologic School or conceivably it could be organized and taught ing which linguists are the professionals Others including myself feel that most Agency within production organizations by professionalized language analysts A likely source of inRussian linguists should ultimately be professtructors would be those who have recently resionalized even if extraordinary efforts must ceived certificates of professionalization By be 1118de to bring them up to the desired level of competence Since it certainly appears that requiring a professionalized linguist to teach April 77 CRYPTOLOG Page 8 IIIIIPIS 121 22171317 2H I t EC 3 3b 3J PL 86-36 50 USC 3605 80HPIBRHTl11JL a course•ci • 1assified translaUon would be together In grading we relied heavily on the insuring •hat they do indeed knew the 1 sand I translation grading systeM reconvnended by James outs of • lconversa't ions and R Child and Emery Tetrault in their article messages and hopefully this '11ould neg a t e- -t e- -- Scoring Trandlations AQitR'4L November 1973 frequentli voiced allegation th At many profes - 23 However in the case of SIGINT translasionalized linguists the11selve do not know a tion I feel there is a need to draw up some adgreat deal about these things and lOuld not pass ditional guide lines dealing specifically with the PQE a •se·cond time And of ' 'tourse the asclassified tra1islation I am in fact trying pirants w uld be much better prt ared for the test to do just this at the present time But as The translator- aspirant can•also use the Part Child and Tetrault point out tot al obj ectivity IIA study•packet available in tire A62 library to in test grading is impossible We are therestudy for the test But there are not enough fore temporarily stalemated in grading Part IIA packets t go around and invaiv bly there are of the PQE and can only do our very best to renaspirants who do not manage to obtain a packet der a fair pass-fail decision These packets however are only•riarginally helpWhen I first picked up my Skilcraf -u s Govful -- though obviously better•than nothing - - i ernment ballpoint pen it was my intention to that ther are no model translat o s with which an write a scathing polemic on why Parts IIA and aspirant an compare his own elfQrts Many asIIB of the Russian PQE should be eliminated I pirants c mplain that they truly do not know wha was convinced that this was the best solution to a finishe4 translation should o3k like Thus a bothersome problem But later when I read to elimin4te this problem we sfto ld provide ever over the doien or so handwritten pages I realized aspirant iith his own personal s udy packet and that I was suggesting that the problem be eliminmake 1110del translations availabl so that an as- ated but was not addressing the reasons for the pirant can learn by doing It would be advisable problem Also in discussing the matter with to keep the models separate frem'the study packe others many of whom also felt that these parts and to sujgest to the aspirant thlt he first should be abolished I finally decided that Parts translate the sample texts and thin obtain the IIA and 118 should not be eliminated because model to iee how well he has done NSA as a producer of SIGINT must have some system of identifying its best linguists Until a Many aipirants argue that Pirti Ila and IIB actually measure experience ratheT than language better method is developed we are stuck with the ability a11d that the responsibi tit1 for evaluating PQE for establlshing he standard experience rightly belongs to tKe 02erational unit I sheepishly admit that my initial reaction t o which t linguist is assigne4 Tl lere is clearly was prompted in part by my reluctance to get some merit in this line of rea oni ng but on involved in grading test papers and my even the other hand it is clearly ad wantageous to the greater reluctance to face irate aspirants in Agency to•have a corps of linguist S who can deal counseling sessions with a difersity of language tdks The nature of language work at NSA is in con tan flux and we But even though I have changed y mind about • • • must be ptepared to move linguJsts from one job the value of Parts IIA and JIB I am still conto anothew when necessary Theref re I feel vinced that in their present form they leave much that this•argument is not valid and that if a to be desired The Language Career Panel and the comprehen ive training program is developed to PQE committees must take steps in the very near prepare aipirants for Part II the hopefully future to improve the PQE The percentage of the basis for this kind of co1111 lai t should dis- passes on Part lIA is far too low to be acceptappear able A 2S-30 percent pass rate should be our target and this goal should be achieved not by Tiedt this matter are aspiran s' charges lowering our standards but by raising the skill that they•have failed Part IIA•because they do level of aspirants through training and guidance not know the correct format While' I cannot This does not mean of course that every Agency speak for past Part lIA tests I do not feel that Russian linguist can or should become a profesthis is c rrently true I Imo that the panel of sionalized linguist it means only that every which I a11 a member would not ail en aspirant Agency linguist should be given ample opportunity on forma1 errors alone In my e rent model to become a professional Once the Language translatidns of the sample PQE ' wilt clarify the Career Panel has fully met its obligations to the format exllected from them aspirant by offering him training and guidance Test gradinR which might s perf cially seem the final responsibility rests with the individual to prove himself to be a straightforward matter is ln fact a sticky bus iness In the case f the Part IIA I hope that this article will prompt CRYPTOLOG panel to which I belong we discussed at great readers to respond with criticism and ideas No length whtt we intended to look for in the papers system is so good that it cannot be improved we were tci grade Our unanimo s dec'sion was and surely the collective effort of NSA's linin the translation f the guists can lead to substantial improvements in I_ _conversation ai id the 'l message the PQE program e gave somewhat less weight tote translationgist which was a new concept in the test we put I ellenc April 77 • CRYPTOLOG Page 9 - - - -- - - i -- - ·'-i Non-Responsive Re c ord r I ---------------------------------------------------- I - - No n - Resp o n sive l' 8HPIDDf liili THE lflST W• R • r i flTS' CECil PH ll PS C• 3 qvhen I wrote the ar icle about the AG-22 and IATS CRYPTOLOG March 1976 it did not occur to me that what I was saying would be so provocative although I knew many people disagreed with my point of view -- especially the idea of having the operator exercise more judgment and format the data more rigidly Perhaps my problem was that I was working from the 'WOnderful state of being an expert without any actual experience as a Morse intercept operator At any rate it is clear that both ex-operators and traffic analysts used to working directly with the traffic felt I was off-base in any idea of further gisting or formatting of traffic What provoked me to write the article was the innumerable complaints I have heard about some of the Horse data bases gfnerated automatically from IATS input I will stick my neck out aaain and say that I believe the quality of many of these data bases is very poor In some cases this poor quality is obscured because manually prepared inputs from U S and 2nd and 3rd party sources in the form of TI CSUMs STRUMs etc is of 11K1Ch higher quality Further the poor quality may be unnoticed in some cases because relatively little attention is paid to the Morse data base vis-a-vis the voice and printer parts of the data base Still another factor may be that the portions of the Morse problem of particularly high interest are singled out for special attention- · such as either the hand-prepared reports mentioned above or just direct printing of the lATS input which means that a large and costly system was built to replace a much simpler one with little advantage except sorting It seems to me that the cost of the IATS-GAPS Generalized AG-22 Processing System implemented on the IBM 370s the al1110st endless SPECOL manipulation of the data base the very high cost of on-line storage and the staggering amounts of printed output mean that we ought to be getting something good out of the whole thing Hy doubts that the system is good that is has analytical integrity and is timely are based on the informal comments of analysts and the more formal complaints of external users who get feedback f-rom the process A related concern is that we are treating the whole input in such a gross manner that we obscure some quality portion of the files and that the sheer bulk of all of this processing causes delays in critical feedback processes and output to the analyst A lot of data processing increases geometrically as the file size increases so there is a chance of much better processing performance with smaller and more specific files Since SPECOL retrievals are sequential-search the same rule does not apply i e run times vary arithmetically with file size but more selective treatment of these files could probably cut average runs to less than one-tenth of present run times In sununary I believe that more can be done than we are now doing without an increase of resources Better computer edits and more manmachine interaction will require additional un and machine resources but r think 110st of these could be saved out of existing processes Perhaps a series of sessions chaired by a senior analyst from a senior technical staff could co e up with approaches better than those I have suggested above or alternatively perhaps a few sessions of the Agency's 110st senior technical people would be able to assess the need for change Will Pl accept the challenge to lead sueh sessions 11 1111111 L II 1• April 77 • CRYPTOLOG • Pzie 12 e@tHillBllftRtHs llltlPI 2 171 an I Non - Responsive - •• - - - - - - - _ _ _ - · - - - - _ _ ¥ - -- - - - - · - - - - - - - - - - - - - - - lr j J - ' I No n - Responsive I No n - Responsive a • • No n - Responsive I Responsive Non - Responsive Non - Responsive Non - Responsive - - - - -· -- -- - ----- Non - Responsive I No n - Responsive I