The National Cyber Range A Systems Engineering Resource for Cybersecurity R D S T Testing and Training Prepared for the 18th Annual Systems Engineering Conference October 27 and 28 2015 Presented by Pete Christensen Director National Cyber Range peter h Christensen civ@mail mil 571-372-2699 What Why How o What do we want to accomplish - Provide an overview of T E Policy and Guidance - Provide an overview of the National Cyber Range NCR - Discuss how programs and organizations can benefit from using the NCR o Why is this important - Cyberspace Threats are proliferating - Systems Security Engineering SSE and Risk Management Framework RMF - Recent policies are emphasizing the importance of increased realism in cybersecurity testing and training - TRMC and the NCR can help o How will we do it - - - - - Cover some existing DoD cybersecurity guidance and policies Explain some of the history behind the NCR Provide an overview of NCR technical capabilities Discuss what you can do with the NCR and types of events that it supports Describe NCR event planning and how customers can get engaged 2 3 New Ongoing Cybersecurity Policy and Guidance Activities o Revision of DoDI 5000 02 Issued 6 Jan 2015 - New better guidance for both developmental and operational testing of IT o Revision of DoD 8500 01 Cybersecurity 14 Mar 2014 - Expanded scope and specificity o DoDI 8510 01 - Risk Management Framework RMF for DoD IT 14 Mar 2014 - Provides policy clarity and guidance on the RMF and compliance o o Six Phase Cybersecurity T E Process Planned Aug 2015 Incorporated into Defense Acquisition Guidebook Chapter 9 OSD DOT E- Procedures for Operational Test and Evaluation of Cybersecurity in Acquisition Programs 01 Aug 2014 - Formalizes OT E Phases o Cybersecurity Implementation Guidebook for PMs In Draft - Address Cybersecurity T E across the acquisition lifecycle o Cybersecurity T E Guidebook Issued July 2015 - Address Cybersecurity T E across the acquisition lifecycle 3 DoD Cybersecurity Test Posture and Emerging Requirements o o o o o Also in 2014 my office conducted 16 cybersecurity assessments in conjunction with Combatant Command and Service exercises Despite the improved defenses my office found that at least one assessed mission during each exercise was at high risk to cyber-attack from beginner to intermediate cyber adversaries DOT E found significant vulnerabilities on nearly every acquisition program that underwent cybersecurity OT E in FY14 The cyber threat has become as real a threat to U S military forces as the missile artillery aviation and electronic warfare threats which have been represented in operational testing for decades Operational Test Agencies OTAs will include cyber threats among the threats to be encountered in operational testing for DOT E oversight systems with the same rigor as other threats All oversight systems capable of sending or receiving digital information are required to conduct cybersecurity testing 4 Cybersecurity T E Shift Left - Six Phased Process Operations and Sustainment O S Pre MS A B Requirements and Systems Security Engineering Analysis SE DT E Evaluate Software and Systems Security Architecture DT E OT E RMF DT E Training Exercises Evaluate Mission Capabilities Verify Baseline Cybersecurity Evaluate TTPs in a and Interoperability in a Requirements and Contested Environment Contested Environment Vulnerability Assessment OT Focus - Codified in OSD DOT E Memo dated 01 Aug 2014 5 National Cyber Range - Background o Originally developed by Defense Advanced Research Projects Agency DARPA in the 2009-2012 timeframe o Transitioned from DARPA to the DoD Test Resources Management Center TRMC in October 2012 o TRMC was charged with operationalizing the capabilities for use by the DOD test training and experimentation communities 6 What is a Cyber Range o o o o o Traditional Ranges Physical Environment for Weapon Testing Live Training TTP Development Range Assets Change slowly o Cyber Range Place to Evaluate o o o o o o Effectiveness of Cyber Defenses Effectiveness of Cyber Weapons Train Cyber Warfighters Rehearse Mission TTP Development Range Assets Change Rapidly NCR provides a range solution that can span the entire spectrum of cyber test evaluation training needs 7 DASD DT E Director TRMC USD AT L HON Frank Kendall ASD R E Mr Stephen Welby Acting Staff Director DASD DT E Director TRMC Col Bohenek USAF Dr C David Brown Chief of Staff Vacant Principal Deputy DT E Principal Deputy Director TRMC Dr Brian Hall SES Acting Mr Derrick Hinton SES Deputy Director T E Competency Development Deputy Director Air Warfare Deputy Director Land and Expeditionary Warfare Deputy Director Naval Warfare Deputy Director Cyber and Information Systems Tom Simms Mike Ginter Steven Lopes Patrick Clancy Andrew Pahutski NCR is here Deputy Director Space and Missile Defense Systems Darlene MosserKerner Director National Cyber Range Deputy Director Corporate Operations Deputy Director T E Range Oversight MRTFB Deputy Director Test Capabilities Development CTEIP Deputy Director Cyber and Interoperability JMETC Deputy Director Technology Development T E S T Pete Christensen Sheila Wright Bruce Bailey Gerry Christeson Chip Ferguson George Rumford 8 NCR - Vision and Mission o Vision - Be recognized as the cyberspace test range of choice for providing mission tailored hi-fidelity cyber environments that enable independent and objective testing and evaluation of advanced cyberspace capabilities o NCR Mission Statement - Provide secure facilities innovative technologies repeatable processes and the skilled workforce - Create hi-fidelity mission representative cyberspace environments - Facilitate the integration of the cyberspace T E infrastructure through partnerships with key stakeholders across DoD DHS industry and academia 9 BLUF - NCR Key Capabilities o Multiple concurrent tests at varying classification levels are supported using a Multiple Independent Levels of Security MILS architecture - Accredited for testing up to Top Secret Sensitive Compartmented Information - Currently support up to 4 events at varying classification concurrently o Rapid emulation of complex operationally representative network environments - Can scale up to 40K high-fidelity virtual nodes - Red Blue Gray support including specialized systems e g weapon systems o Automation provides significant efficiencies that enable more frequent and more accurate events - Reduces timelines from weeks or months to hours or days - Minimizes human error and allows for greater repeatability o Sanitization to restore all exposed systems to a known clean state - Allows assets to be reused even when they are exposed to the most malicious and sophisticated uncharacterized code o Supports a diverse user base by accommodating a wide variety of event types R D OT E information assurance compliance malware analysis etc and communities testing training research etc 10 What is the National Cyber Range Computing Assets Facility LMCO Orlando FL Encapsulation Architecture Operational Procedures Cyber Test Team Integrated Cyber Event Tool Suite Secure Connectivity via JIOR and JMETC Realistic Mission Environments RSDPs PSDPs JMN 11 Facility Overview On-site or Remote Access o Fully accredited SCIF o Supports at least two independent concurrent events on-site o Test suites can be utilized at different security levels and contain - Two test rooms - Conference room o o Remote access currently provided through the Joint IO Range JIOR and JMETC MLS o Unclassified Range Support Center Wireless Testing Environment 12 Facility Overview Support for Wireless Testing o Facility o Wireless environment that supports classified testing TS SCI o Support for mobile computing iOS Android Windows 8 on tablets cell phones and multimedia devices 1313 Automation Toolkit End to End Support Tools to support event planning Tools to define and manage resource requirements Tools to automatically o Build verify and sanitize your environment o Support event execution Faster more reliable event environment creation and execution 14 NCR Automated Cyber Test Process Start with a common pool of HW SW Resources and Cyber Tool Set Step 6 Sanitization Tool sanitizes HW and virtually puts HW resources back in pool Running a Cyber Test Evaluation Sanitize Resources Step 5 Test Execution Tools are used by the event team along with event-specific systems for execution and data collection analysis Step 1 Utilize Test Spec Tool to define end to end aspects of test Define Test Allocate Resources Run Test Configure the SW Step 4 Range Configuration ACORN tools automatically configure the SW you need to run the event Step 2 Resource Allocation determines what resources from the pool are needed and allocates them to Event Configure the HW Step 3 Range Provisioning Tools automatically wire HW to the appropriate configuration 15 Cybersecurity T E As A Service o TRMC Government FFRDCs Lockheed Martin SETA Contractors Services Include But Are Not Limited To - - - - - - End-to-End Test Support Test Bed Design Support Cyber and Testing Expertise Threat Vector Development Custom Traffic Generation Custom Sensor and Visualization Support - Custom Data Analysis - Integration of Custom Assets - - - - Software Hardware Wired and Wireless Remote Red Blue Team Support The NCR's Most Valuable Resource Is A Diverse and Experienced World Class Cybersecurity Workforce 1616 Why Use a Cyber Range o o o o Requirements to conduct testing that cannot or should not occur on open operational networks due to potential catastrophic consequences for example full execution of extremely malicious threats on realistic representations of systems and networks e g releasing self-propagating malware Requirements to test advanced cyberspace tactics techniques and procedures that require isolated environments of complex networked systems e g movement on the Internet The need to rapidly and realistically represent operational environments at different levels of security fidelity and or scale e g Blue friendly force Red adversary force and Gray neutral networks The need for precise control of the test environment that allows for rapid reconstitution to a baseline checkpoint reconfiguration and repeat of complex test cases this would include the need for rapid variation of conditions to quickly evaluate hundreds of scenarios 17 When To Use a Cyber Range Across the Acquisition Life Cycle Operations and Sustainment O S Pre MS A B Requirements and Systems Security Engineering Analysis NCR Event Cybersecurity Architecture Evaluation SE DT E Evaluate Software and Systems Security Architecture NCR Event Cybersecurity Verification and Validation DT E OT E RMF DT E Training Exercises Evaluate Mission Capabilities Verify Baseline Cybersecurity Evaluate TTPs in a and Interoperability in a Requirements and Contested Environment Contested Environment Vulnerability Assessment NCR Event Mission Thread Testing with Blue Team NCR Event Mission Thread Testing with Red Team in a Realistic Threat Environment NCR Event Large-scale Simulation to Train Cyber Mission Forces and Evaluate Cyber Defensive and Offensive Operations 18 What You Can Do With the NCR 1 of 4 Question Does Product A close a requirements gap - Does it mitigate a particular set of threats within my operational system - How well - What is my residual risk What you get - Empirical evidence showing how the technology or product closes the requirements gap in your operational environment How does adding a technology to my existing environment reduce my threat surface Commercial Product Emerging Technology Evaluation 1919 What You Can Do With the NCR 2 of 4 Question Will my architecture scale in the field - Will it handle the expected user load - What are potential issues that can only be discovered at scale normally only found very late in the testing process What you get - Minimize unexpected performance failures late in the DT or early OT process - Reduce costly rework - Empirical data to show whether or not the system operates as predicted in a realistic environment Will this architecture scale to support the mission Results provide insight into system performance before the design is finalized 2020 What You Can Do With the NCR 3 of 4 Question How resilient is my system to cyber attacks and faults when connected into the overall system of systems - System is a distributed sensing system that has a dependency on an external service to interconnect platforms to ground stations - How does my system behave when there are problems with external systems What you get - Increased resilience to cyber attack and failures - Reduce costly rework - Empirical data to show whether or not the system operates as predicted in a realistic environment - Understand how the dependencies on the broader DoD environment affect the ability to meet the mission System Testing During Development Graphic Source http fm cnbc com applications cnbc com resources img editorial 2013 02 14 100460031-server-room-cyber-security-gettyp 1910x1000 jpg 2121 What You Can Do With the NCR 4 of 4 Question How do I generate realistic cyber mission effect within a large scale training exercise safely and Be able to use securely unrestricted TTPs - OCO is destructive - Cyber weapons and TTPs are often classified at security levels higher than the rest of the exercise Operate on realistic and complex network topologies What you get Integrate home - Realistic operator training base and remote - Repeatability to evaluate relative training effectiveness of multiple TTPs - On-demand low-cost evolution of the environment to represent salient real-world environments Have access to interactive web sites A safe environment for safely conducting realistic cybersecurity training Graphic Source http www npr org 2014 04 30 307963996 whats-the-nsa-doing-now-training-more-cyber-warriors 2222 NCR Supports Many Different Types of Events o NCR supports a wide variety of cyber event types - R D testing - Product evaluation - Training events - System emulation - Target emulation - - - - - - - o o Mission rehearsal Risk reduction activities Architecture analysis DT E OT E Malware analysis Forensic analysis Events can occur exclusively at NCR or in conjunction with other Joint Mission Environment Test Capability or Joint Information Operations Range nodes Level of support from NCR is dependent on customer needs 23 NCR Operational Support Models We deliver a verified range and support sanitization at end consumer does everything else We work with consumer to define tests and then NCR personnel do everything else with periodic review Plan Test Plan Test Construct Verify Testbed Construct Verify Testbed Execute Test Execute Test Closeout Minimal Consumer Participation Range Staff Consumer Minimal Range Staff Participation Closeout You Select the Desired Level of Support from NCR Staff 2424 NCR Planning and Scheduling Procedures o NCR Director - Coordinates with the JMETC PM to review schedules and make decisions - Owns the NCR Event Planning List and the NCR Range Schedule o o o The NCR Event Planning List describes the events that are currently in the discussion planning phase and scheduled but not yet run NCR Range Schedule describes the events to be held on the range Monthly Review held to - Formally add move events to the schedule - Review customer feedback on tests - Review Event Planning Port 25 NCR Event Planning Stages o Event Pre-Planning Planning - - o Event Design - - - o Red Team Operations Environment Build verification Event Execution - - o Goals Objectives Assumptions Outputs Data Collection Plan Environment Design Event Development - - o Discussions Use Case Development Conduct tests and data Review results adapt as needed Event Completion - - Data Analysis Reporting Briefings Next Event Planning Example Generalized from Actual NCR Event 26 How to get engaged Technical Interchange Meetings Technical Resource Planning Scope Identify New Definition Development NCR provides SME support automated tools libraries NCR Environment Development Testbed Construction Integration with Remote Assets on JMETC or JIOR 27 Summary o Cyberspace threats to DoD systems are proliferating at an unprecedented rate - Leadership has recognized that current cybersecurity testing and training needs further improvements - Leadership is placing increased emphasis on the need to consistently incorporated realistic cybersecurity testing and training at all levels and phases - Early identification of system vulnerabilities can make them easier and cheaper to fix o NCR provides customers with a unique set of cybersecurity test evaluation and training capabilities - NCR enables acquisition organizations to conduct system specific cybersecurity test and evaluation events that are tailored to meet program requirements throughout the systems acquisition lifecycle - NCR enables operational organizations to conduct realistic cybersecurity training in environments that closely replicate the real world o NCR capabilities have been independently validated and have successfully supported a wide variety of cyber events including - Developmental Testing - Operational Testing - Training Exercise o NCR is institutionally funded and cost effective - Customers only pay for their own personnel travel systems under test special equipment etc 28 Questions Peter H Christensen Director National Cyber Range TRMC Office Phone 571-372-2699 TRMC Email peter h christensen civ@mail mil Address 4800 Mark Center Drive Suite 07J22 Alexandria Va 22350                     National Security Archive    Suite 701  Gelman Library  The George Washington University    2130 H Street  NW  Washington  D C  20037    Phone  202 994‐7000  Fax  202 994‐7005  nsarchiv@gwu edu