Understanding Cognitive Stress in Tactical Cyber Operations Celeste Lyn Paul and Josiah Dykstra National Security Agency SPEAKERS CELESTE LYN PAUL JOSIAH DYKSTRA Senior Researcher and Technical Advisor at NSA Research Deputy Technical Director of NSA Cybersecurity Operations Ph D in Human-Centered Computing Ph D in Computer Science Hackers are people too Late to the party AGENDA • About NSA • Tactical Cyber Operations • Stress and Hacking at NSA • Putting these Results to Work NATIONAL SECURITY AGENCY CYBERSECURITY Defend national security systems SIGNALS INTELLIGENCE Intercept and exploit foreign signals CYBERTHREATS RUSSIA Their aggressive cyber behavior resembles the show of force we have seen displayed in their geopolitical actions IRAN Remains very sensitive to international political events which can influence target selection and level of malicious activity DPRK Has always viewed cyber as an effective tool of state power every conflict will have a cyber dimension CHINA Continues to use cyber espionage as a prime enabler to acquire transformative technologies as part of their long-term plan to be a global superpower PARTNERS INTELLIGENCE COMMUNITY INDUSTRY ACADEMIA COMBATANT COMMANDS U S GOV SECTORS FOREIGN PARTNERS LAW ENFORCEMENT COUNTER INTEL NCTOC National Cybersecurity WHERE WE OPERATE On-Network Operations Computer Network Exploitation o o o o o ADVERSARY NETWORKS U S GOV NETWORKS Vulnerability Assessments Authorized Hacking Targeted Hunting Incident Response Comms Security Monitoring DEPARTMENT OF DEFENSE INFORMATION NETWORK STRESS HACKING @ Just before this talk Holding on for Tonight Addiction in InfoSec Now Stress and Hacking Understanding Cognitive Stress in Tactical Cyber Operations This afternoon Mental Health Hacks Fighting Burnout Depression and Suicide in the Hacker Community Tomorrow morning Demystifying PTSD in the Cybersecurity Environment Let s talk about stress TYPES OF STRESS ACUTE Temporary ‘fight or flight’ response EPISODIC Repetitive stress with little time to recover CHRONIC Enduring situations with no sense of control WHY IS HACKING SO COMPLEX UNPREDICTABLE HIGH RISK REWARD STUDYING STRESS - Hard to study - Multi-faceted - Dependent on environment - Very subjective STRESS HAS EFFECTS ON… FATIGUE FRUSTRATION COGNITIVE WORKLOAD Physical and mental feeling of tiredness Anxiety and annoyance over lack of control Amount of mental effort needed to use memory FATIGUE Samn-Perelli Fatigue Scale Fatigue How awake or tired are you before the operation Fully alert wide awake Very responsive but not at peak Okay somewhat fresh A little tired less than fresh Moderately tired Extremely tired very Exhausted unable to let down difficult to concentrate function effectively FRUSTRATION COGNITIVE WORK NASA Task Load Index TLX Very Low Very High Mental Demand How mentally demanding was the operation Physical Demand How physically demanding was the operation Time Demand How hurried or rushed was the pace of the operation Overall Performance How successful were you in accomplishing what you were asked to do Frustration Level How insecure discouraged irritated stressed and annoyed were you Effort How hard did you have to work to accomplish your level of performance BASELINE Normalizing Individual Differences Fatigue How awake or tired are you before the operation Fully alert wide awake Very responsive but not at peak Okay somewhat fresh A little tired less than fresh Moderately tired Extremely tired very Exhausted unable to let down difficult to concentrate function effectively Fatigue How awake or tired are you after the operation Fully alert wide awake Very responsive but not at peak Okay somewhat fresh A little tired less than fresh Moderately tired Extremely tired very Exhausted unable to let down difficult to concentrate function effectively BASELINE Normalizing Individual Differences Frustration Level How insecure discouraged irritated stressed and annoyed are you Very Low Very High Frustration Level How insecure discouraged irritated stressed and annoyed were you Very Low Very High Cyber Operations Stress Survey Cyber Operations Stress Survey PRE-OP Complete this part before you start the operation POST-OP Complete this part after you complete the operation I Name or Participant ID Date Operation end time What time did you arrive at the office today When was your last operation Fatigue How awake or tired are you after the operation 7 Operation type or go i 7 fully Alert aggregate am somewhat A ihi le ur less M d i iy 'Ex rr hieiyk bd ctriatj aftin'a'ii wrde awake but not at peak fresh than fresh lured let down very difficult to to function concentrate ellectlvely Mental Demand How mentally demanding was the capera ion 5 Study-specific questions can be added as 3 I i I i Very Low Very Hugh I Physical Demand How physically demanding was the operation I Verytow VeryHigh Tim_e Demand Ho_w hurried or ru'shedrwasnthe pace oftheo-peration Very low Very High Fati ue How awake or tired are ou before the erat'onOverall Performance How successful were you In accomplishing what you were asked to do I 1 i 1 Fully alert Very responswe Okay somewhat A little tired less Moderately Extremely tired Exhausted unable A Wide awake but not at peak fresh than fresh tired let down very dif cult to to function Very low WW concentrate effectively Frustration Level How insecure discouraged irritated stressed andiannoyed were you Frustration Level How insecure discouraged irritated stressed and annoyed are you right I I I Very Low Very High Very Low Very _E1fort How hard did you have to work to accomplish your level of performance Complete this section only if you have never completed a version of this survey before y ww Job Role 7 7 VSy E'g h' Team Synergy How well did your team work together I V How long have you worked in this job 4 Very Low Very Hugh What are your other work duties or responsibilities Did you complete your objective Yes Is there anyt g else you would like to tell us Operation start time 1 Complete the back page after the operation is complete - 1 1 4le 1 9 IIigs wl ll I II I PARTICIPANTS 4 NSA Locations 126 Tactical Cyber Operators 361 Operation Surveys Both CIV and MIL RESULTS Tactical Cyber Operations cause stress FATIGUE 15 20 1O Pre-operation Post-operation FRUSTRATION A12cyo m_ 52 9 02 6 57 0 L0 - Pre-operation Post-operation 20 COGNITIVE WORKLOAD 10 15 14 2 10 01 9 49 9 02 6 41 0 5 4 33 Mental Physical Time Performance RTLX 44 5 SD 28 1 Effort Frustration Is this a lot Hint That s not the right question COGNITIVE WORKLOAD Mental Physical 479 Physical Time 547 541 Time Performance - 034 - 012 - 022 Performance Effort 686 486 509 - 009 Effort Frustration 468 334 429 - 315 469 p 001 Frustration FATIGUE FRUSTRATION Mental Physical Time Performance Effort Frustration ∆Fatigue 263 225 162 - 078 227 173 ∆Frustration 238 194 201 - 184 277 p 01 OPERATION LENGTH Operation Length Mental Physical Time Performance Effort Frustration 376 253 271 032 296 176 p 001 OPERATION LENGTH Operation Length ∆Fatigue ∆Frustration 361 210 p 001 5 Hours ∆10% 5 Hours Failure is not an option Locus of control The extent to which a person feels that they have control over the outcome of events in their lives Fatigue How awake or tired are you before the operation Fully alert Very responswe Okay somewhat Alittletired less Moderately Extremelytired very Exhausted unable to wide awake but notat peak fresh than fresh tired letdown difficult to concentrate function effectively 23M Frustration Level How Insecure discouraged Very Low Significantly dropped after the operation SUMMARY Tactical cyber operations increase fatigue frustration and cognitive work Longer operations are more tiring frustrating and mentally demanding Fatigue and frustration begin to compound after 5 hours However operators always pull through with performance but at what cost TAKEAWAYS Use the Cyber Operations Stress Survey to evaluate your own operations Review policies on breaks scheduling and operation length Empower operators with happy healthy work environments Journal of Information Warfare 2017 https www jinfowar com journal volume-16-issue2 understanding-operator-fatigue-frustration-cognitiveworkload-tactical-cybersecurity-operations Cyber Security Experimentation and Test 2018 https www usenix org conference cset18 presentation dykstra Understanding Cognitive Stress in Tactical Cyber Operations For more information contact NSA PUBLIC AFFAIRS OFFICE mediarelations@nsa gov 1 301 688-6311