United States Government Accountability Office Report to Congressional Requesters September 2016 INFORMATION TECHNOLOGY Agencies Need to Improve Their Application Inventories to Achieve Additional Savings GAO-16-511 September 2016 INFORMATION TECHNOLOGY Agencies Need to Improve Their Application Inventories to Achieve Additional Savings Highlights of GAO-16-511 a report to congressional requesters Why GAO Did This Study What GAO Found The federal government is expected to spend more than $90 billion on IT in fiscal year 2017 This includes a variety of software applications supporting agencies’ enterprise needs Since 2013 OMB has advocated the use of application rationalization This is a process by which an agency streamlines its portfolio of software applications with the goal of improving efficiency reducing complexity and redundancy and lowering the cost of ownership Most of the 24 Chief Financial Officers CFO Act of 1990 agencies in the review fully met at least three of the four practices GAO identified to determine if agencies had complete software application inventories To be considered complete an inventory should 1 include business and enterprise information technology IT systems as defined by the Office of Management and Budget OMB 2 include these systems from all organizational components 3 specify application name description owner and function supported and 4 be regularly updated Of the 24 agencies 4 the Departments of Defense Homeland Security and Justice and the General Services Administration fully met all four practices 9 fully met three practices 6 fully met two practices 2 fully met one practice and 3 did not fully meet any practice see figure GAO’s objectives were to determine 1 whether agencies have established complete application inventories and 2 to what extent selected agencies have developed and implemented processes for rationalizing their portfolio of applications To do this GAO assessed the inventories of the 24 CFO Act agencies against four key practices and selected six agencies— the Departments of Defense Homeland Security the Interior Labor and NASA and NSF—due to their IT spending among other factors to determine whether they had processes addressing applications What GAO Recommends GAO is recommending that 20 agencies improve their inventories and five of the selected agencies take actions to improve their processes to rationalize their applications more completely The Department of Defense disagreed with both recommendations made to it After reviewing additional evidence GAO removed the recommendation associated with improving the inventory but maintained the other The other agencies agreed to or had no comments on the draft report View GAO-16-511 For more information contact David Pow ner at 202 512-9286 or pownerd@gao gov Assessment of Whether Agencies Fully Met Practices for Establishing Complete Software Application Inventories A January 2016 OMB requirement to complete an IT asset inventory by the end of May 2016 contributed to most of the agencies fully meeting the first three practices Agencies that did not fully address these practices stated among other things their focus on major and high risk investments as a reason for not having complete inventories However not accounting for all applications may result in missed opportunities to identify savings and efficiencies It is also inconsistent with OMB guidance regarding implementation of IT acquisition reform law referred to as the Federal Information Technology Acquisition Reform Act which requires that Chief Information Officers at covered agencies have increased visibility into all IT resources Not accounting for all applications also presents a security risk since agencies can only secure assets if they are aware of them Each of the six selected agencies relied on their investment management processes and in some cases supplemental processes to rationalize their applications to varying degrees However five of the six agencies acknowledged that their processes did not always allow for collecting or reviewing the information needed to effectively rationalize all their applications The sixth agency the National Science Foundation NSF stated its processes allow it to effectively rationalize its applications but agency documentation supporting this assertion was incomplete Only one agency—the National Aeronautics and Space Administration NASA —had plans to address shortcomings Taking action to address identified weaknesses with agencies’ existing processes for rationalizing applications would assist with identifying additional opportunities to reduce duplication and achieve savings United States Government Accountability Office Contents Letter 1 Background Most Agencies Fully Met at Least Three of the Four Practices for Establishing Complete Application Inventories Agencies Rationalize Some but Not All Applications through Existing Investment Management Processes Conclusions Recommendations for Executive Action Agency Comments and Our Evaluation 12 20 21 22 Appendix I Objectives Scope and Methodology 28 Appendix II GAO’s Evaluation of Agencies’ Application Inventories 34 Appendix III Comments from the Department of Commerce 50 Appendix IV Comments from the Department of Defense 52 Appendix V Comments from the Department of Education 55 Appendix VI Comments from the Department of Energy 56 Appendix VII Comments from the Department of Health and Human Services 58 Appendix VIII Comments from the Department of Homeland Security 60 Page i 3 7 GAO-16-511 Federal Agencies’ Use of Application Rationalization Appendix IX Comments from the Department of Housing and Urban Development 62 Appendix X Comments from the Department of the Interior 63 Appendix XI Department of Justice 65 Appendix XII Comments from the Department of Labor 67 Appendix XIII Comments from the Department of State 68 Appendix XIV Comments from the Department of Veterans Affairs 70 Appendix XV Comments from the Environmental Protection Agency 72 Appendix XVI Comments from the National Aeronautics and Space Administration 74 Appendix XVII Nuclear Regulatory Commission 76 Appendix XVIII Comments from the Office of Personnel Management 77 Appendix XIX Comments from the Social Security Administration 79 Page ii GAO-16-511 Federal Agencies’ Use of Application Rationalization Appendix XX GAO Contact and Staff Acknowledgments 82 Tables Table 1 GAO Assessment of Agencies’ Efforts to Establish a Complete Application Inventory Table 2 Agency Ranking by Number of Fully Met and Partially Met Practices Table 3 Number of Agencies that Fully Met Partially Met or Did Not Meet the Practices for Establishing a Complete Application Inventory Table 4 Department of Agriculture Table 5 Department of Commerce Table 6 Department of Defense DOD Table 7 Department of Education Table 8 Department of Energy Table 9 Department of Health and Human Services Table 10 Department of Homeland Security DHS Table 11 Department of Housing and Urban Development Table 12 Department of the Interior Interior Table 13 Department of Justice Table 14 Department of Labor Labor Table 15 Department of State Table 16 Department of Transportation Table 17 Department of the Treasury Table 18 Department of Veterans Affairs Table 19 Environmental Protection Agency Table 20 General Services Administration Table 21 National Aeronautics and Space Administration NASA Table 22 National Science Foundation NSF Table 23 Nuclear Regulatory Commission Table 24 Office of Personnel Management Table 25 Small Business Administration Table 26 Social Security Administration Table 27 U S Agency for International Development 11 34 34 35 36 36 37 37 38 39 39 40 41 41 42 43 44 45 45 46 47 47 48 48 49 Figure 1 Assessment of Whether Agencies Fully Met Practices for Establishing Complete Software Application Inventories 9 8 9 Figure Page iii GAO-16-511 Federal Agencies’ Use of Application Rationalization Abbreviations CFO CIO DHS DOD FISMA FITARA Interior IT Labor NASA NSF OMB Chief Financial Officer Chief Information Officer Department of Homeland Security Department of Defense Federal Information Security Modernization Act of 2014 Federal Information Security Management Act of 2002 Federal Information Technology Acquisition Reform Act Department of the Interior information technology Department of Labor National Aeronautics and Space Administration National Science Foundation Office of Management and Budget This is a work of the U S government and is not subject to copyright protection in the United States The published product may be reproduced and distributed in its entirety without further permission from GAO However because this work may contain copyrighted images or other material permission from the copyright holder may be necessary if you wish to reproduce this material separately Page iv GAO-16-511 Federal Agencies’ Use of Application Rationalization Letter 441 G St N W Washington DC 20548 September 29 2016 The Honorable Ron Johnson Chairman Committee on Homeland Security and Governmental Affairs United States Senate The Honorable Jason Chaffetz Chairman Committee on Oversight and Government Reform House of Representatives In fiscal year 2017 the federal government is expected to spend more than $90 billion on information technology IT including software applications Applications are software components and supporting software hosted on an operating system that create use modify share or store data in order to enable a business or mission function to be performed This includes custom commercial off-the-shelf government off-the-shelf or open-sourced software In a memorandum issued in March 2013 the Office of Management and Budget OMB advocated the use of application rationalization—streamlining the portfolio with the goal of improving efficiency reducing complexity and redundancy and lowering the cost of ownership Through this process agencies can identify duplicative wasteful and low-value applications and identify opportunities for savings You asked us to review federal agencies’ efforts to rationalize their portfolio of applications Our objectives were to determine 1 whether agencies have established complete application inventories and 2 to what extent selected agencies have developed and implemented processes for rationalizing their portfolio of applications For consistency we defined applications as those commodity IT assets associated with Page 1 GAO-16-511 Federal Agencies’ Use of Application Rationalization enterprise IT systems and business systems commodity IT categories identified in OMB guidance 1 To address the first objective we identified four practices for establishing complete inventories We derived them primarily from our guide for assessing the reliability of computer-processed data and best practices identified in our 2014 report on federal software licenses These practices are 1 including the business and enterprise IT systems defined by OMB 2 including systems from all organizational components 3 specifying basic application attributes—namely application name description owner and function supported— and 4 regularly updating the inventory with quality controls to ensure the reliability of the data in the inventory The 24 Chief Financial Officers CFO Act of 1990 agencies provided us with their software application inventories We analyzed the inventories reviewed documentation and interviewed agency staff to determine the extent to which agencies implemented the practices we identified To address the second objective we selected six agencies—the Departments of Defense DOD Homeland Security DHS Labor Labor and the Interior Interior the National Aeronautics and Space Administration NASA and National Science Foundation NSF We selected them based on their fiscal year 2015 IT spending—we selected two large agencies two medium agencies and two small agencies—and whether they claimed to have an application rationalization process We also included agencies recognized for effective rationalization efforts based on our research and OMB observations We identified a set of common application rationalization practices reviewed documentation and interviewed agency officials to determine whether the agencies had processes addressing these practices We conducted this performance audit from May 2015 to September 2016 in accordance with generally accepted government auditing standards Those standards require that we plan and perform the audit to obtain 1 According to OMB’s memorandum Chief Information Officer Authorities M-11-29 Washington D C Aug 8 2011 enterprise IT systems include e-mail identity and access management IT security web infrastructure and collaboration tools Business systems include finance human resources and other administrative functions In addition while commodity IT assets represent a range of applications systems and investments we are using the term application to address them all Page 2 GAO-16-511 Federal Agencies’ Use of Application Rationalization sufficient appropriate evidence to provide a reasonable basis for our findings and conclusions based on our audit objectives We believe that the evidence obtained provides a reasonable basis for our findings and conclusions based on our audit objectives See appendix I for a more detailed discussion of our objectives scope and methodology Background In March 2012 OMB launched the PortfolioStat initiative which required agencies to conduct an annual review of their commodity IT portfolio to among other things achieve savings by identifying opportunities to consolidate investments or move to shared services 2 For PortfolioStat OMB defined broad categories of commodity IT • enterprise IT systems which include e-mail identity and access management IT security web infrastructure and collaboration tools • business systems which include finance human resources and other administrative functions and • IT infrastructure which includes data centers networks desktop computers and mobile devices Of those categories the first two include software applications which are software components and supporting software hosted on an operating system that create use modify share or store data in order to enable a business or mission function to be performed This includes custom commercial off-the-shelf government off-the-shelf or open-sourced software The memorandum establishing the PortfolioStat initiative also required agencies to develop a commodity IT baseline including the number types and costs of investments for all commodity IT categories In a subsequent memorandum OMB advocated the use of application rationalization to inform data center optimization efforts 3 Application rationalization is the process of streamlining the portfolio to improve efficiency reduce complexity and redundancy and lower the cost of 2 OMB Implementing PortfolioStat M-12-10 Washington D C Mar 30 2012 3 OMB Fiscal Year 2013 PortfolioStat Guidance Strengthening Federal IT Portfolio Management M-13-09 Washington D C Mar 27 2013 While OMB advocated the use of application rationalization in its memorandum it did not define a process or propose steps for carrying it out Page 3 GAO-16-511 Federal Agencies’ Use of Application Rationalization ownership It can be done by retiring aging and low-value applications modernizing aging and high-value applications eliminating redundant applications standardizing on common technology platform and version as is the case for moving to shared services or consolidating applications 4 OMB stated in its memorandum that application rationalization would be a focus of PortfolioStat sessions and required agencies to describe their approach to maturing the IT portfolio including rationalizing applications in the information resource management plans and enterprise roadmaps that are required to be updated annually In December 2014 the law commonly referred to as the Federal Information Technology Acquisition Reform Act FITARA was enacted and required covered executive branch agencies except for DOD to ensure that Chief Information Officers CIO have a significant role in the decision making process for IT budgeting as well as the management governance and oversight processes related to IT 5 The act also required that CIOs in each covered agency except DOD review and approve 1 all contracts for IT services prior to their execution and 2 the appointment of any other employee with the title of CIO or who functions in the capacity of a CIO for any component organization within the agency OMB issued guidance in June 2015 that reinforces the importance of agency CIOs and describes how agencies are to implement the law 6 In that same memorandum OMB changed PortfolioStat from being an annual review session to quarterly reviews including a discussion of portfolio optimization efforts and focus on commodity IT Specifically the memorandum stated that agencies are to discuss how they use category management to consolidate commodity IT assets eliminate duplication between assets and improve procurement and management of 4 Oracle An Oracle White Paper in Enterprise Architecture Application Portfolio Rationalization How IT Standardization Fuels Growth Redwood Shores CA May 2010 Reprinted with permission from Oracle 5 Federal Information Technology Acquisition Reform provisions of the Carl Levin and Howard P ‘Buck’ McKeon National Defense Authorization Act for Fiscal Year 2015 Pub L No 113-291 div A title VIII subtitle D 128 Stat 3292 3438-3450 Dec 19 2014 6 OMB Management and Oversight of Information Technology Memorandum M-15-14 Washington D C June 10 2015 Page 4 GAO-16-511 Federal Agencies’ Use of Application Rationalization hardware software network and telecom services during the sessions Furthermore agencies are to share lessons-learned related to commodity IT procurement policies and efforts to establish enterprise-wide inventories of related information The memorandum also specified key responsibilities for CIOs—including having increased visibility into all IT resources—and required agencies to develop plans to implement these responsibilities by December 2015 Further during the course of our review in January 2016 OMB updated guidance to agencies requiring that they provide information regarding their IT asset inventories when making integrated data collection submissions 7 The guidance required agencies to provide a preliminary inventory by the end of February 2016 and a complete IT asset inventory including information on systems sub-systems and applications by the end of May 2016 to OMB Finally federal law and guidance specify requirements for protecting federal information and systems Specifically the Federal Information Security Management Act FISMA of 2002 8 among other things requires agencies to maintain and update an inventory of major information systems at least annually and the National Institute of Standards and Technology specifies that this should include an accurate inventory of software components including the software applications which are the subject of our review OMB plays a key role in monitoring and overseeing agencies’ security activities and their FISMA implementation This includes tracking how well agencies are managing their inventories of hardware and software assets and protecting them 7 OMB M-13-09 created the integrated data collection approach to streamline agency reporting functions and reduce agency burden 8 The Federal Information Security Modernization Act of 2014 FISMA 2014 Pub L No 113-283 Dec 18 2014 largely superseded the Federal Information Security Management Act of 2002 FISMA 2002 enacted as Title III E-Government Act of 2002 Pub L No 107-347 116 Stat 2899 2946 Dec 17 2002 Page 5 GAO-16-511 Federal Agencies’ Use of Application Rationalization GAO Has Reported on Efforts Related to Application Rationalization In November 2013 we reported that agency commodity IT baselines were not all complete and recommended that 12 agencies complete their commodity IT baselines 9 As of March 2016 6 of the 12 agencies—the Departments of Agriculture Commerce Housing and Urban Development and Labor the Social Security Administration and the U S Agency for International Development—reported that they had completed their commodity IT baseline The remaining 6 agencies reported making progress towards completion In May 2014 in a review examining federal agencies’ management of software licenses which are types of enterprise IT applications we determined among other things that only 2 of the 24 CFO Act agencies—the Department of Housing and Urban Development and the National Science Foundation—had comprehensive software license inventories Twenty had partially complete inventories and two did not have any inventory 10 We recommended that agencies complete their inventories We also recommended that OMB issue a directive to help guide agencies in managing licenses and that the 24 agencies improve their policies and practices for managing licenses In June 2016 OMB issued a memorandum that is intended to improve agencies’ acquisition and management of enterprise software consistent with our May 2014 recommendation 11 The memorandum contains elements related to having a comprehensive policy such as developing and implementing a plan for centralizing the management of software licenses 9 GAO Information Technology Additional OMB and Agency Actions Are Needed to Achieve Portfolio Savings GAO-14-65 Washington D C Nov 6 2013 10 GAO Federal Software Licenses Better Management Needed to Achieve Significant Savings Government-Wide GAO-14-413 Washington D C May 22 2014 11 OMB Category Management Policy 16-1 Improving the Management and Acquisition of Common Information Technology Software Licensing Memorandum M-16-12 Washington D C Jun 2 2016 Page 6 GAO-16-511 Federal Agencies’ Use of Application Rationalization Most Agencies Fully Met at Least Three of the Four Practices for Establishing Complete Application Inventories We identified four practices to determine whether agencies had a complete software application inventory To do so we primarily relied on best practices used in our recent report on federal software licenses which determined among other things whether agencies had a comprehensive software license inventory 12 and our guide for assessing the reliability of computer-processed data We determined that to be considered complete agencies’ inventories should • include business systems and enterprise IT systems as defined by OMB • include these systems from all organizational components • specify basic attributes namely application name description owner and function supported and • be regularly updated with quality controls in place to ensure the reliability of the information collected Most of the agencies fully met at least three of the four practices Specifically • 4 agencies fully met all four practices • 9 agencies fully met three practices and 8 of these partially met the fourth • 6 agencies fully met two practices and 5 of these partially met the others • 2 agencies fully met one practice and partially met the three others and • 3 agencies did not fully meet any practice Of the three agencies that did not fully meet any practice one partially met all four practices and two partially met three practices and did not meet the fourth Table1 lists the 24 agencies and shows whether they fully met partially met or did not meet each of the four practices and figure 1 graphically depicts this status 12 GAO-14-413 Page 7 GAO-16-511 Federal Agencies’ Use of Application Rationalization Table 1 GAO Assessment of Agencies’ Efforts to Establish a Complete Application Inventory Agency Includes business and enterprise IT systems Includes systems from all organizational components Specifies basic application attributes Is regularly updated with quality controls to ensure reliability ● ● ● ● ● ● ● ● ● ● ◐ ● ● ● ● ● ◐ ● ◐ ○ ● ◐ ◐ ◐ ● ● ● ◐ ● ◐ ● ◐ ◐ ● ◐ ● ◐ ● ● ● ● ● ● ● ◐ ● ◐ ● ● ● ● ◐ ● ● ◐ ◐ ◐ ◐ ● ◐ ● ● ○ ● ○ ◐ ◐ ◐ ◐ ● ● ◐ ● ● ● ◐ ● ● ● ● ● ◐ ◐ ◐ ● ● ● ◐ ● ● ◐ ◐ ◐ ○ ◐ ◐ Department of Agriculture Department of Commerce Department of Defense Department of Education Department of Energy Department of Health and Human Services Department of Homeland Security Department of Housing and Urban Development Department of the Interior Department of Justice Department of Labor Department of State Department of Transportation Department of the Treasury Department of Veterans Affairs Environmental Protection Agency General Services Administration National Aeronautics and Space Administration National Science Foundation Nuclear Regulatory Commission Office of Personnel Management Small Business Administration Social Security Administration U S Agency for International Development Key ● Fully met—the agency provided evidence that it fully or largely addressed the key practice for establishing a complete application inventory Partially met—the agency provided evidence that it addressed some but not all of the key practice for establishing a complete application ◐ inventory Not met—the agency provided evidence that it largely did not meet the key practice for establishing a complete application inventory or did not ○ provide any evidence that it addressed the key practice Source GAO analysis of agency documentation I GAO-16-511 Page 8 GAO-16-511 Federal Agencies’ Use of Application Rationalization Figure 1 Assessment of Whether Agencies Fully Met Practices for Establishing Complete Software Application Inventories Table 2 ranks the agencies first by the number of fully met practices then by the number of partially met practices Table 2 Agency Ranking by Number of Fully Met and Partially Met Practices Agency Number of fully met practices Number of partially met practices Number of not met practices Department of Defense 4 0 0 Department of Homeland Security 4 0 0 Department of Justice 4 0 0 General Services Administration 4 0 0 Department of Education 3 1 0 Department of Health and Human Services 3 1 0 Department of Veterans Affairs 3 1 0 Environmental Protection Agency 3 1 0 National Science Foundation 3 1 0 Nuclear Regulatory Commission 3 1 0 Office of Personnel Management 3 1 0 Department of Agriculture 3 1 0 Department of Commerce 3 0 1 Department of State 2 2 0 Department of the Treasury 2 2 0 National Aeronautics and Space Administration 2 2 0 Social Security Administration 2 2 0 U S Agency for International Development 2 2 0 Page 9 GAO-16-511 Federal Agencies’ Use of Application Rationalization Agency Number of fully met practices Number of partially met practices Number of not met practices Department of the Interior 2 1 1 Department of Energy 1 3 0 Department of Housing and Urban Development 1 3 0 Department of Transportation 0 4 0 Department of Labor 0 3 1 Small Business Administration 0 3 1 Source GAO analysis I GAO-16-511 The following are examples of how we assessed agencies against our practices See appendix II for a detailed assessment of all the agencies • The Environmental Protection Agency fully met three practices and partially met one The agency fully met the first practice because its inventory includes enterprise IT and business systems with the exception of very small systems In addition it included applications from all offices and regions in the organization The agency partially met the practice for including application attributes in the inventory because although it identifies the application name and description component managing the applications and the business function associated with its applications it does not identify the business function for every application Officials stated that they are working to have this information populated for all applications Lastly the agency fully met the fourth practice of regularly updating the inventory because it has processes to update its inventory through the agency’s software life cycle management procedure and provided evidence of the annual data call issued by the CIO to ensure that the inventory is current • The U S Agency for International Development fully met two practices and partially met two Specifically the agency’s inventory includes business and enterprise IT systems and the inventory includes basic application attributes However the agency’s inventory does not include systems from all organizational components because officials stated that coordination and communication in the geographically-widespread agency is difficult In addition the agency has processes for updating its inventory however it relies on manual processes to maintain it • The Department of Transportation partially met all four practices While the department’s inventory for the common operating environment includes all business and enterprise IT systems and its inventory of applications includes business systems the inventory of applications does not include all enterprise IT systems Furthermore Page 10 GAO-16-511 Federal Agencies’ Use of Application Rationalization both of its inventories do not include applications used by all of its components Specifically the inventory does not include applications used by the Federal Highway Federal Railroad and Federal Transit Administrations among others and the inventory for its common operating environment does not include applications used by the Federal Aviation Administration The department also partially met the practice of including basic application attributes because although the department’s inventory includes these attributes its common operating environment does not provide the business function that the applications support Further while the Department of Transportation has a process for its partners to provide information on its individual inventories in order to update the inventory of applications it does not have processes in place to ensure the reliability and accuracy of the reported information and thus partially met this practice Regarding the four practices the majority of the agencies fully met the practices of including business systems and enterprise IT system including these systems from all organizational components and specifying the application name description owner and business function supported Only five agencies fully met the practice of regularly updating the inventory and implementing quality controls for ensuring the reliability of the inventory data because they provided evidence of performing both of these activities Table 3 shows the number of agencies who fully met partially met and did not meet the practices Table 3 Number of Agencies that Fully Met Partially Met or Did Not Meet the Practices for Establishing a Complete Application Inventory Includes business and enterprise IT systems Includes systems from all organizational components Specifies basic application attributes Is regularly updated with quality controls to ensure reliability 20 16 16 5 Partially met 4 8 8 15 Not met 0 0 0 4 Rating Fully met Source GAO analysis I GAO-16-511 OMB’s requirement for agencies to complete an IT asset inventory by the end of May 2016 greatly contributed to most of the agencies including business systems and enterprise IT systems for all of their organizational components and specifying key attributes for them Those agencies that did not fully address these practices provided various reasons for not doing so For example one agency stated that it has not made its software application inventory a priority because it has been focusing on major and high risk investments while delegating applications to the component level Others noted that the lack of automated processes Page 11 GAO-16-511 Federal Agencies’ Use of Application Rationalization make collecting complete inventory information difficult Further others noted that it is challenging to capture applications acquired by components in the department-wide inventory While it is reasonable to expect that priority be given to major and high risk investments applications are nevertheless part of the portfolio and should be accounted for as such Not accounting for them may result in missed opportunities to identify savings and efficiencies It is also inconsistent with OMB guidance for implementing FITARA which requires that CIOs have increased visibility into all IT resources In addition the lack of a comprehensive inventory presents a security risk If agencies are not aware of all of their assets they cannot secure them resulting in a vulnerable posture Given the importance of securing federal systems and data to ensuring public confidence and the nation’s safety prosperity and well-being we designated federal information security as a government-wide high-risk area in 1997 13 In 2003 we expanded this area to include computerized systems supporting the nation’s critical infrastructure In our high risk update in February 2015 we further expanded this area to include protecting the privacy of personal information that is collected maintained and shared by both federal and nonfederal entities Agencies Rationalize Some but Not All Applications through Existing Investment Management Processes As previously noted application rationalization is the process of streamlining the portfolio to improve efficiency reduce complexity and redundancy and lower the cost of ownership It can be done in many ways including retiring aging and low-value applications modernizing aging and high-value applications eliminating redundant applications standardizing on common technology platform and version as is the case for moving to shared services or consolidating applications Based on common practices identified in technical papers from industry experts to effectively perform rationalization an agency should first establish a complete inventory of applications It should then collect and review cost technical and business value information for each application and use that information to make rationalization decisions These practices are 13 GAO High-Risk Series An Overview GAO HR-97-1 Washington D C February 1997 GAO High-Risk Series An Update GAO-15-290 Washington D C February 2015 Page 12 GAO-16-511 Federal Agencies’ Use of Application Rationalization consistent with those used to manage investment portfolios Therefore an agency can achieve application rationalization through established practices related to investment management including budget formulation security or enterprise architecture 14 Each of the six selected agencies relied on their investment management processes and in some cases supplemental processes to rationalize their applications to varying degrees However five of the six agencies acknowledged that their processes did not always allow for collecting or reviewing the information needed to effectively rationalize all their applications The sixth agency NSF stated its processes allow it to effectively rationalize its applications but we found supporting documentation to be incomplete Only one agency NASA had plans to address shortcomings The following describes the six selected agencies’ processes for rationalizing their applications provides rationalization examples identifies weaknesses and challenges and addresses plans if any the agencies have for addressing them • DOD The department uses its investment management process for defense business systems 15 to annually review its applications Officials noted that the department’s enterprise architecture is also used to identify duplication and overlap among these applications In addition the department has identified eight enterprise common services for collaboration content discovery and content delivery it is requiring its components to use to among other things improve warfighting efficiency and reduce costs One example of rationalization that DOD provided resulting from its efforts with Executive Business Information System that was replaced 14 An architecture is a “blueprint” that describes how an organization operates in terms of business processes and technology how it intends to operate in the future and how it plans to transition to the future state 15 Pursuant to title 10 U S C § 2222 a “defense business system” is an information system that is operated by for or on behalf of DOD including any of the following a financial system a financial data feeder system a contracting system a logistics systems a planning and budgeting system an installations management system a human resources management system a training and readiness system The term does not include a national security system or an information system used exclusively by and within the defense commissary system or the exchange system or other instrumentality of the Department of Defense conducted for the morale welfare and recreation of members of the armed forces using nonappropriated funds Page 13 GAO-16-511 Federal Agencies’ Use of Application Rationalization by the Navy Enterprise Resource Planning system in a full migration in 2014 Estimated cost savings or avoidances were estimated at $268 000 in fiscal year 2012 and almost $200 000 per year in fiscal years 2013 through 2015 In addition in an effort to improve its financial management systems the department has efforts underway to reduce the number of financial management systems from 327 to 120 by fiscal year 2019 However officials acknowledged that its processes do not address all applications Specifically according to information provided by the department about 1 200 enterprise IT and business systems which are associated with the Enterprise Information Environment Mission Area are not reviewed by the department—though they are reviewed by components—because they do not meet the definition of a defense business system Officials cited several challenges with implementing systematic rationalization efforts including the department’s organizational structure and contractual agreements As an example they noted that the Navy’s Next Generation e-mail system is being procured through a contract with a particular vendor and as such would be difficult to consolidate with other department e-mail systems They also noted that the cost of collecting additional cost technical and business value information along with maintaining even more data at greater granularity may outweigh the benefits The department does not have plans at this time to further enhance its processes to rationalize its applications While we recognize the challenges and costs that may be associated with systematic rationalization efforts the Enterprise Information Environment Mission Area could be considered as a near-term target for rationalization given the large number of enterprise IT and business systems associated with it Modifying existing processes to allow for the collection review and evaluation of cost technical and business information of these systems at the department level could help identify opportunities for savings and efficiencies • DHS DHS has several processes for rationalizing applications For example through its investment management process portfolios are regularly assessed against criteria which help identify duplication In Page 14 GAO-16-511 Federal Agencies’ Use of Application Rationalization addition the department uses its DHS Collaborative Architecture Methodology in conjunction with its segment architectures to help identify duplication and fragmentation at different levels including at the application level 16 The DHS IT Duplication Reduction Act of 2015 mandated the department to report on a strategy for reducing duplicative IT systems 17 and the department used the DHS Collaborative Architecture Methodology process to address this mandate including about 700 commodity IT and back-office applications in the scope of the effort Further the department recently established an Application Services Council chaired by its Enterprise Business Management Office According to its charter the council is a cross-component and crossdisciplined leadership team responsible for developing maintaining and overseeing the Enterprise Information Technology Services Portfolio Lifecycle Governance Model and Roadmap It is expected to take a strategic approach to evaluating existing and future IT service offerings—including software platform and infrastructure services—and provide a forum to identify strategies best practices processes and approaches for enterprise IT services cloud computing and shared service challenges For example officials reported the council is currently developing a standard service level agreement template and guidance as well as a cloud adoption strategy The department also reported other mechanisms related to rationalization include its Joint Requirements Council strategic sourcing initiatives IT acquisition reviews and executive-level portfolio reviews In addition it reported that it uses its DHS Enterprise Architecture Information Repository Technical Reference Model to track application products and software versions—mainly consisting of commercial off-the-shelf software The product information is gathered through the use of continuous network discovery scans 16 DHS’s Collaborative Architecture Methodology is the department’s multi-disciplinary analysis approach that results in recommendations formed in collaboration with leaders stakeholders planners and implementers for segment architecture planning These analyses support portfolio-based decisions and include functionally-based Executive Steering Committees as the primary decision-making authorities for segment architecture planning and governance 17 Pub L No 114-43 129 Stat 470 Aug 6 2015 Page 15 GAO-16-511 Federal Agencies’ Use of Application Rationalization Examples of rationalization include the consolidation of learning management systems and the consolidation of site services including help desk operations The consolidation of learning management systems was identified through the segment architecture process and is expected to result in projected savings of 10 to 20 percent in fiscal year 2016 after transition costs are addressed The modernization of the department’s help desk and on-site operations resulted in savings that cumulatively accrued to $202 million by fiscal year 2015 due to similar efforts among all department components However DHS’s processes do not address all applications because while the components may carry out their own rationalization efforts the department does not always collect the application-level cost technical or business information for applications used by its components Specifically officials reported challenges tracking product level information for deployed applications and difficulty gaining visibility into all the supporting application products for large systems Officials particularly noted they have been challenged to collect such information and cited a general lack of visibility into the components’ budget and their spending They also noted it was not clear whether there was a good return on investment for the resources needed to collect additional technical cost and business value data for systematic application rationalization efforts Officials reported the department had a financial systems modernization effort underway which would provide greater visibility into components’ spending but they did not have a plan to address the collection and review of technical and business value information While we recognize that collecting additional details on all applications may not be cost-beneficial the department could consider taking a segmented approach and initially identify one high-cost function it is currently not collecting or reviewing detailed cost technical and business information for across the department It could then modify existing processes to collect and review this information These actions would assist the CIO in gaining visibility into all IT resources as specified in the OMB implementation guidance for FITARA and also help identify additional opportunities for savings and efficiencies • NASA NASA uses its current investment management process—the Capital Planning and Investment Control process—and its configuration management tools—to review its applications Page 16 GAO-16-511 Federal Agencies’ Use of Application Rationalization NASA reported examples of rationalization resulting in significant savings according to NASA officials These included the NASA gov Portal Cloud Transition which resulted in estimated savings of $4 million and the Enterprise Business Portal Transition Consolidation which resulted in estimated savings of about $184 000 per year However NASA officials acknowledged that their current processes do not provide the level of detail needed to effectively rationalize the agency’s applications In terms of challenges to rationalizing applications officials stated that it is difficult to obtain transparency on all applications since each of the agency’s centers runs independently In addition officials stated that determining application business value is currently subjective to users because the agency’s process for obtaining this information is to ask the application owner the impact on the agency if the application did not exist whereas application technical health information is more concrete Furthermore NASA officials stated that there is no systematic process to review applications facing end-of-life issues due to flat budgets and budget cuts NASA has developed a plan for a supplemental process the annual capital investment review process that is to allow the agency to among other things collect detailed data about its applications The agency has begun to implement the plan and has completed the first milestone of the process which included conducting a data call to gather and validate application information provided by the various centers and agency stakeholders At the time of our review NASA had also performed an initial review and analysis of the information collected and identified optimization opportunities including developing a plan to consolidate decommission or invest to achieve maximum cost efficiencies and process effectiveness across the application program Fully implementing the annual capital investment review process could better position the agency to identify additional opportunities for savings and efficiencies • Interior As part of its budget formulation process Interior performs rationalization through annual reviews of its portfolio of investments and supporting applications against criteria which measure business value and technical fit Reported examples of application rationalization include Interior’s cloud e-mail and collaboration services initiative which consolidated Page 17 GAO-16-511 Federal Agencies’ Use of Application Rationalization 14 disparate systems into a single enterprise system and achieved a cost savings avoidance of $13 56 million and the consolidation of the Enterprise eArchive System with the eMail Electronic Records and Document Management System which resulted in cost savings avoidance of $6 1 million However the department reported that its portfolio review process is not standardized because it has not been fully defined or established in policy In addition it has only been used at the department level not at the bureaus or offices and there is a lack of confidence in the data that is collected to support the analyses In comments on a draft of this report the department noted that it has also yet to document a plan to implement policy associated with these efforts which they believe would establish a standard analytical technique for rationalizing the investment portfolio Such a plan would also help secure the commitment needed to carry out planned efforts The department reported several challenges to rationalizing its applications including 1 ensuring the quality and accuracy of data collected since it relies largely on manual processes for collecting information and 2 the lack of standard portfolio evaluation techniques to support information resource management decisionmaking across the department The department has efforts underway which should help address these challenges Specifically it is making changes to its information resource management governance According to the department these changes combined with efforts to implement the CIO responsibilities specified in FITARA should help to address the challenges to rationalizing its applications and allow for rationalization of all applications However while the department has defined and begun to implement criteria to assess whether or not an investment and its underlying applications are wasteful low-value or duplicative it has not documented its plan for improving its governance—which according the department would support application rationalization Such a plan would help secure the commitment needed to carry out planned efforts • Labor Similar to the other agencies the department uses its investment management process to review the majority of its business and enterprise IT applications In addition officials stated that the department initiated an enterprise-wide budget formulation and Information Technology Acquisition Review Board approval function beginning in fiscal year 2013 which has helped with rationalization Page 18 GAO-16-511 Federal Agencies’ Use of Application Rationalization Officials stated that their efforts have resulted in rationalization of commodity applications and on a case-by-case basis the rationalization of other applications such as for a case management platform and an acquisition management system Additional examples of application rationalization include the deployment of a web-based conferencing and collaboration shared service to employees which resulted in cost avoidance of travel costs of about $2 3 million The department also noted benefits of moving to a cloud e-mail solution such as saved time and increased user satisfaction However officials identified weaknesses and challenges with rationalizing their applications Specifically they reported that in most cases IT investments are associated with a group of IT assets including applications and individual application information is therefore not reviewed making it difficult to effectively rationalize In addition officials stated that the fact that each bureau-level agency has had authority and responsibility for managing its own applications and that the department has over 600 locations present challenges Further though senior officials including the CIO agreed with the benefits of rationalization they did not have any plans to rationalize They questioned the value of developing such plans stating that 1 maintaining mission critical applications and the department’s aging infrastructure are current priorities and 2 funding may not be available to implement rationalization plans While we agree that mission critical applications should be given priority rationalizing mission support applications including enterprise IT and business systems could result in solutions which allow agencies to focus more on mission capabilities and at the same time generate savings which could be reinvested As we noted for DHS the department could consider taking a segmented approach to further rationalize and identify a function for which it could modify existing processes to collect and review detailed application cost technical and business value information • NSF NSF also uses its investment management processes and supporting budget formulation process—with key stakeholders such as the Executive IT Resources Board Capital Planning and Investment Control Working Group and Enterprise Architecture Working Group—to collect and review information for its investments In addition NSF’s Enterprise Modernization Roadmap—which is updated annually—identifies applications along with their associated business segment and modernization status and plans Page 19 GAO-16-511 Federal Agencies’ Use of Application Rationalization NSF identified its e-mail migration to a new platform which was completed in July 2013 as an example of an application rationalization effort with the highest savings According to the agency’s November 2015 integrated data collection submission to OMB the migration effort resulted in cost avoidances of $60 000 in 2014 Other examples of application rationalization include modernization and consolidation of NSF’s grant systems the 2014 retirement of the financial functions of a legacy system and the implementation of its financial system modernization initiative However while officials told us that evaluations for all applications meeting the scope of our review would be included in the roadmap we only identified half of the applications 9 out of 18 In addition cost information was only provided in the roadmap for three individual applications NSF officials told us that because they are a relatively small agency with a single mission in a single location many of their processes are handled informally and not thoroughly documented but they are able to discuss all the applications with each other on a regular basis and as a result there is no duplication Nevertheless consistently documenting the evaluations and costs for all applications in the roadmap would improve transparency Conclusions While it is encouraging that 13 of the 24 CFO Act agencies fully met at least three of the four practices for establishing a complete software application inventory most could improve their software applications inventories—albeit to varying degrees—by taking steps to fully meet the practices we identified as being either partially met or not met Doing so would better position them to identify opportunities to rationalize their applications which could lead to savings and efficiencies In addition they would be better positioned to comply with OMB issued implementation guidance for the recent IT acquisition reform law which requires that CIOs have increased visibility into all IT resources and ensure they are effectively securing their IT assets Six selected agencies used their investment management processes and sometimes supplemental processes to rationalize their applications Of the six agencies one—NSF—had processes that allowed it to rationalize all applications though the supporting documentation was not always complete In addition while the remaining five agencies’ processes did not allow for rationalizing all applications only one—NASA—had plans to Page 20 GAO-16-511 Federal Agencies’ Use of Application Rationalization address identified weaknesses While these agencies all had examples of rationalization resulting in savings and efficiencies modifying their existing processes to more completely address their applications would help identify additional opportunities to achieve such savings and efficiencies which even small would add up across agencies and over time Recommendations for Executive Action To improve federal agencies’ efforts to rationalize their portfolio of applications we are recommending that • the heads of the Departments of Agriculture Commerce Education Energy Health and Human Services Housing and Urban Development the Interior Labor State Transportation the Treasury and Veterans Affairs and heads of the Environmental Protection Agency National Aeronautics and Space Administration National Science Foundation Nuclear Regulatory Commission Office of Personnel Management Small Business Administration Social Security Administration and U S Agency for International Development direct their CIOs and other responsible officials to improve their inventories by taking steps to fully address the practices we identified as being partially met or not met and • the Secretaries of Defense Homeland Security the Interior and Labor and the Director of the National Science Foundation to direct the CIOs and other responsible officials to modify existing investment management processes to address applications more completely Specifically • the Secretary of Defense should direct the responsible official to modify the department’s existing processes to collect and review cost technical and business information for the enterprise and business IT systems within the Enterprise Information Environment Mission Area applications which are currently not reviewed as part of the department’s process for business systems • the Secretary of Homeland Security should direct the department’s CIO to identify one high-cost function it could collect detailed cost technical and business information for and modify existing processes to collect and review this information • the Secretary of the Interior should direct the department’s CIO to document and implement a plan for establishing policy that would define a standard analytical technique for rationalizing the investment portfolio Page 21 GAO-16-511 Federal Agencies’ Use of Application Rationalization Agency Comments and Our Evaluation • the Secretary of Labor should direct the department’s CIO to consider a segmented approach to further rationalize and identify a function for which it would modify existing processes to collect and review application-specific cost technical and business value information and • the Director of the National Science Foundation should direct the CIO to consistently document evaluations for all applications and report cost information for them in the roadmap or other documentation We provided a draft of this report to the 24 CFO Act agencies in our review for comment and received responses from all 24 Of the 24 17 agreed with the recommendations directed to them one the Department of Defense disagreed with the recommendations directed to it five the Department of the Treasury the National Science Foundation the Nuclear Regulatory Commission the Small Business Administration and the and U S Agency for International Development stated that they had no comments and one the Department of Justice agreed with the assessment and conclusion for three of the four practices associated with establishing a complete software application inventory and provided clarifying information on the two other practices Several agencies also provided technical comments which we incorporated as appropriate The agencies’ comments and our responses are summarized below • In e-mail comments the Department of Agriculture’s Senior Advisor for Oversight and Compliance Enterprise Management stated that the department concurred with our recommendation The department also provided technical comments which we incorporated as appropriate As a result of these comments and additional documentation provided we changed our evaluation of the practice associated with updating the software application inventory from not met to partially met • In written comments the Department of Commerce concurred with our recommendation and stated that the department is committed to implementing a more efficient process by regularly updating its application inventory to ensure the reliability of the data collected The department also specified actions it plans to take to provide regular updates of its inventory The department’s comments are reprinted in appendix III • In written comments the Department of Defense disagreed with both of our recommendations to the department For the first recommendation the department provided evidence showing that it Page 22 GAO-16-511 Federal Agencies’ Use of Application Rationalization updated its inventory subsequent to us sending the report for comment As a result we changed the rating for the related practice from partially met to fully met and removed the associated recommendation For the second recommendation the department stated that 53 percent of the inventory records for the Enterprise Information Environment Mission Area we focused on were IT infrastructure assets specifically network enclaves or circuits and not applications subject to rationalization The mission area nevertheless includes enterprise and business IT applications which could benefit from rationalization as evidenced by the example of e-mail system consolidation provided in the comments Given the number of systems involved at least 1 200 collecting and reviewing cost technical and business information for them would help identify additional rationalization opportunities which could yield savings and efficiencies We therefore believe a recommendation to address these systems is still warranted The department also stated that our draft implied that major IT infrastructure modernization efforts many of which involve the Enterprise Information Environment Mission Area were not reviewed or properly managed by the department However as noted in our report we did not include IT infrastructure assets in the scope of our review and therefore made no comment on how these assets are being managed We have restated our emphasis on enterprise and business IT systems as it relates to the mission area where appropriate Finally in its comments the department stated that our report ignored significant Enterprise Information Environment Mission Area application rationalization efforts such as the Pentagon IT consolidation under the Joint Service Provider the Business Process and System Review and ongoing efforts concerning public-facing websites and associated systems While we were not informed of these efforts during our review our intent was to highlight additional opportunities for rationalization not discount any that might have already been implemented The department also provided technical comments which we incorporated into the report as appropriate The department’s comments are reprinted in appendix IV • In written comments the Department of Education concurred with our recommendation and described actions it plans to take to address it The department’s comments are reprinted in appendix V Page 23 GAO-16-511 Federal Agencies’ Use of Application Rationalization • In written comments the Department of Energy concurred with our recommendation In addition the department stated that it partially met the four practices associated with establishing a complete software application inventory and provided the IT Asset Inventory it submitted to OMB in May 2016 and other documentation supporting this claim Our review of the documentation found that the inventory includes business and enterprise IT systems however it does not include those systems from all organizational components and it is missing the business function code for a large number of systems Furthermore while the department is updating the IT Asset inventory in response to OMB guidance for the fiscal year 2016 integrated data collection submission process it has not implemented quality control processes to ensure the reliability of the data within the inventory As a result we changed the department’s rating for the practice associated with including business and enterprise IT systems from not met to fully met and from not met to partially met for the remaining three practices We modified sections of the report specific to the department accordingly The department’s comments are reprinted in appendix VI • In written comments the Department of Health and Human Services concurred with our recommendation and stated that that it would review the feasibility of fully addressing the practices it partially met The department’s comments are reprinted in appendix VII • In written comments the Department of Homeland Security concurred with our recommendation and described actions it plans to take to implement it The department’s comments are reprinted in appendix VIII • In written comments the Department of Housing and Urban Development concurred with our recommendation and stated that more definitive information with timelines will be provided once the final report has been issued The department’s comments are reprinted in appendix IX • In written comments the Department of the Interior stated that it would agree with the recommendations if we made its requested changes However we disagreed with the request to change the rating for the practice associated with regularly updating the inventory from not met to partially met because while the department provided evidence supporting its claim that it recently updated its inventory the evidence was not sufficient Specifically the department provided an e-mail requesting the bureaus and offices to complete an inventory survey However the department did not show how the survey resulted in updates to the inventory We incorporated the remaining Page 24 GAO-16-511 Federal Agencies’ Use of Application Rationalization requested changes in the report as appropriate The department’s comments are reprinted in appendix X • In written comments the Department of Justice stated that it concurred with our assessment and conclusions The department also provided clarifying information regarding its procedures to ensure application inventory accuracy and provided documentation showing that it updates the inventory and implements quality controls to ensure its reliability As a result we changed the rating for the related practice from partially met to fully met and removed the recommendation made to the department The department’s comments are reprinted in appendix XI • In written comments the Department of Labor concurred with our recommendations to the department and stated that it would take the necessary steps to address the recommendations The department’s comments are reprinted in appendix XII • In written comments the Department of State concurred with our recommendation to the department and described current and planned actions to fully address it The department’s comments are reprinted in appendix XIII • In e-mail comments the Department of Transportation’s Audit Liaison stated that the department concurred with our findings and recommendation • In e-mail comments the Department of the Treasury’s Audit Liaison stated that the department did not have any comments • In written comments the Department of Veterans Affairs concurred with our conclusions and recommendation The department also provided information on the actions it plans to take to address the recommendation The department’s comments are reprinted in appendix XIV • In written comments the Environmental Protection Agency generally agreed with our recommendation The agency also asked that we include some of the language from the detailed evaluation in appendix II of the report to the example we have in the body to provide the full context of its practices We added the language as requested The agency’s comments are reprinted in appendix XV • In e-mail comments the General Services Administration’s Associate CIO of Enterprise Planning and Governance concurred with the report The agency also provided evidence of its processes to update the inventory and ensure the reliability of the data in the inventory Page 25 GAO-16-511 Federal Agencies’ Use of Application Rationalization including the coordination between its Enterprise Architecture Team and subject matter experts As a result we changed the agency’s rating for the related practice from partially met to fully met and removed our recommendation to the agency • In written comments the National Aeronautics and Space Administration concurred with our recommendation and stated that it would utilize the capital investment review process it is currently implementing to improve its inventory The agency’s comments are reprinted in appendix XVI • In e-mail comments the National Science Foundation Office of Integrated Activities’ Program Analyst stated that it had no comments on the draft report • In written comments the Nuclear Regulatory Commission stated that it is in general agreement with the report The agency’s comments are reprinted in appendix XVII • In written comments the Office of Personnel Management concurred with our recommendation and described plans to fully address it The agency’s comments are reprinted in appendix XVIII • In e-mail comments the Small Business Administration Office of Congressional and Legislative Affairs’ Program Manager stated that the Office of the Chief Information Officer believes the report captures its current posture • In written comments the Social Security Administration agreed with our recommendation to the agency but disagreed with the partially met rating for regularly updating the inventory including implementing quality controls stating that it had provided evidence supporting its implementation of the practice However as noted in the report the Social Security Administration reported that its systems development lifecycle contains steps for maintaining the inventory but did not provide evidence showing that it is using this process to regularly update the inventory Therefore we did not change our rating The agency’s comments are reprinted in appendix XIX • In an e-mail the U S Agency for International Development Audit Performance and Compliance Division’s Management Analyst stated that the agency did not have any comments Page 26 GAO-16-511 Federal Agencies’ Use of Application Rationalization We are sending copies of this report to interested congressional committees the heads of the Departments of Agriculture Commerce Defense Education Energy Health and Human Services Homeland Security Housing and Urban Development the Interior Justice Labor State Transportation the Treasury and Veterans Affairs the Environmental Protection Agency the General Services Administration the National Aeronautics and Space Administration the National Science Foundation the Nuclear Regulatory Commission the Office of Personnel Management the Small Business Administration the Social Security Administration the U S Agency for International Development the Director of the Office of Management and Budget and other interested parties This report will also be available at no charge on our website at http www gao gov If you or your staff have any questions on matters discussed in this report please contact me at 202 512-9286 or pownerd@gao gov Contact points for our Offices of Congressional Relations and Public Affairs may be found on the last page of this report GAO staff who made major contributions to this report are listed in appendix III David A Powner Director Information Technology Management Issues Page 27 GAO-16-511 Federal Agencies’ Use of Application Rationalization Appendix I Objectives Scope and Methodology Appendix I Objectives Scope and Methodology Our objectives were to determine 1 whether agencies have established complete application inventories and 2 to what extent selected agencies have developed and implemented processes for rationalizing their portfolio of applications For the first objective we reviewed the 24 major agencies covered by the Chief Financial Officers CFO Act of 1990 1 To ensure consistency we decided to focus on the software applications associated with the business and enterprise information technology IT commodity IT categories defined in the Office of Management and Budget OMB guidance since they would be familiar to the agencies in our scope 2 OMB defines enterprise IT systems as e-mail identity and access management IT security web infrastructure and collaboration tools and business systems as finance human resources and other administrative functions 3 We then identified practices to assess whether agencies had a complete software application inventory To identify these practices we primarily relied on our guide for assessing the reliability of computer-processed data which addresses questions about the currency of the data and how often it is updated procedures for ensuring the completeness of the data and quality control processes in place to ensure the accuracy of the data and on criteria used in our recent report on federal software licenses which determined whether agencies had a comprehensive software 1 The 24 major federal agencies covered by the Chief Financial Officers Act of 1990 are the Departments of Agriculture Commerce Defense Education Energy Health and Human Services Homeland Security Housing and Urban Development the Interior Justice Labor State Transportation the Treasury and Veterans Affairs Environmental Protection Agency General Services Administration National Aeronautics and Space Administration National Science Foundation Nuclear Regulatory Commission Office of Personnel Management Small Business Administration Social Security Administration and U S Agency for International Development 2 OMB Chief Information Officer Authorities M-11-29 Washington D C Aug 8 2011 3 OMB also defined a third commodity IT category—infrastructure—but we excluded it from our scope because it is primarily made up of hardware assets In addition while commodity IT assets represent a range of applications systems and investments we used the term application to address them all Page 28 GAO-16-511 Federal Agencies’ Use of Application Rationalization Appendix I Objectives Scope and Methodology license inventory among other things 4 To be considered complete we determined an inventory should • include business systems and enterprise IT systems as defined by OMB • include these systems from all organizational components • specify basic attributes namely application name description owner and function supported and be regularly updated with quality controls in place to ensure the reliability of the information collected Following the identification of these four practices we asked the 24 CFO Act agencies for their software application inventories We used a set of structured questions to determine whether the agencies implemented the practices and identify lessons learned and challenges faced in establishing a complete software application inventory We analyzed supporting documentation such as agency and departmental guidance policies and procedures for updating the inventories and interviewed relevant agency officials as needed We compared the information received to the four practices We determined a practice to be fully met if agencies provided evidence that they fully or largely implemented the practice for establishing a complete application inventory partially met if agencies provided evidence that they addressed some but not all of the practice for establishing a complete application inventory and not met if the agencies did not provide any evidence that they implemented the practice for establishing a complete application inventory To verify the inclusion of business and enterprise IT systems we analyzed agencies’ inventories and looked for examples of each type of system identified by OMB in the business and enterprise IT commodity categories We followed up with agencies when we were not able to identify a type of system to determine the reason for the omission We considered the practice to be fully met if agencies’ inventories included all of the business and enterprise IT system types or if agencies provided valid reasons for excluding them We considered the practice to be partially met if agencies acknowledged they were missing applications or if we determined system types to be missing and agencies did not provide 4 GAO Federal Software Licenses Better Management Needed to Achieve Significant Savings Government-wide GAO-14-413 Washington D C May 22 2014 Page 29 GAO-16-511 Federal Agencies’ Use of Application Rationalization Appendix I Objectives Scope and Methodology a valid reason for this Although we followed up with agencies to determine whether they maintained separate inventories of software licenses when they were not included in the inventories provided we did not consider the inclusion of these applications in determining our rating because software licenses are expected to be tracked separately by OMB To verify the inclusion of systems from all organizational components we analyzed agencies’ inventories against the list of organizational components to determine whether they were included We followed up with agency officials to determine causes if any for missing components We considered the practice to be fully met if inventories included applications from all organizational components or if agencies provided valid reasons for excluding them We considered the practice to be partially met if agencies acknowledged they were missing organizational components or if we determined several components to be missing and agencies did not provide a valid reason for this Regarding application attributes we determined that at a minimum agencies should have a name a description an owner and function supported for each application We considered the practice to be fully met if inventories included these attributes for all or most applications or the agencies provided evidence that attributes not included in the inventory provided were being tracked separately We determined the practice to be partially met if agencies acknowledged that they were missing any of the attributes or if we determined them to be missing from the inventory and agencies did not provide alternate sources for them For the last practice we determined whether agencies 1 used relevant methods to update and maintain the application inventory and 2 implemented controls to ensure the reliability of the information collected Regarding these controls we looked for the use of automated tools to collect and track information as their use increases reliability We determined the practice to be fully met if agencies provided evidence that they regularly updated the inventory and had controls for ensuring the reliability of information collected including the use of automated tools or if agencies had mitigating factors when these processes were not in place We determined the practice to be partially implemented when agencies provided policies and procedures but no evidence of actual inventory updates or quality controls We also determined the practice to be partially implemented if agencies provided evidence of either regular updates or controls for ensuring reliability but not both or did not make use of automated tools for collecting or maintaining information and had Page 30 GAO-16-511 Federal Agencies’ Use of Application Rationalization Appendix I Objectives Scope and Methodology no mitigation factors Finally we also determined the practice to be partially implemented if agencies provided draft policy and guidance of their processes For our second objective we selected 6 of the 24 CFO Act agencies—the Departments of Defense Homeland Security the Interior and Labor and the National Aeronautics and Space Administration and National Science Foundation—to assess their application rationalization plans and efforts to implement them We selected the agencies based on three factors • whether they had an application rationalization process in our initial set of structured questions to agencies we asked whether they had a plan or process for rationalizing applications and selected those that reported having one • the size of the agency based on fiscal year 2015 IT spending we selected two large agencies i e with spending equal to or greater than $3 billion two medium agencies i e with spending between $1 billion and $3 billion and two small agencies i e with spending of less than $1 billion for a full range of IT spending and • if they were known for effectively rationalizing their applications based on OMB observations and our research on IT acquisition reform recognizing agencies for their application rationalization efforts We identified key practices for effectively rationalizing applications To do so we reviewed OMB guidance 5 on federal IT management We also reviewed technical reports on application rationalization from industry experts We synthesized the information collected looked for themes and determined that to effectively rationalize applications agencies should have a process addressing the following four key practices • establish an application inventory • collect information on each application such as total cost technical details and business value 5 OMB Memorandum for Heads of Executive Departments and Agencies Fiscal Year 2013 PortfolioStat Guidance Strengthening Federal IT Portfolio Management M-13-09 Washington D C Mar 27 2013 and Memorandum for Heads of Executive Departments and Agencies Management and Oversight of Federal Information Technology M-15-14 Washington D C June 10 2015 Page 31 GAO-16-511 Federal Agencies’ Use of Application Rationalization Appendix I Objectives Scope and Methodology • evaluate the portfolio and make application rationalization decisions based on a review of collected information and determine what applications to retain retire replace eliminate modernize or consolidate move to shared services and • execute and manage the process by implementing decisions from the evaluation and evaluate process outcomes against defined metrics and adjust as needed While our research identified specific processes for rationalizing applications the principles of collecting application information and reviewing it to inform decision making are consistent with those used to manage investment portfolios Therefore we considered established practices related to investment management budget formulation security or enterprise architecture 6 Since the first key practice was addressed in our first objective we focused on the last three practices To do so we interviewed relevant officials using a structured set of questions that were developed in conjunction with internal experts We also reviewed documentation to determine the extent to which agencies had processes addressing these practices We also asked agencies to provide their two best examples of application rationalization in terms of savings or cost avoidance—to illustrate the results of rationalization When agencies did not provide two examples meeting these conditions—the case for DOD DHS and NSF— we drew examples from other documentation they had provided Finally we interviewed staff from OMB’s Office of the Federal Chief Information Officer to determine whether and how the office monitors agencies’ efforts to rationalize their portfolio of applications as recommended in OMB guidance 7 We also interviewed the staff to determine the impetus for the IT asset data inventory guidance and the planned used for the information collected 6 An architecture is a “blueprint” that describes how an organization operates in terms of business processes and technology how it intends to operate in the future and how it plans to transition to the future state 7 OMB M-13-09 Page 32 GAO-16-511 Federal Agencies’ Use of Application Rationalization Appendix I Objectives Scope and Methodology We conducted this performance audit from May 2015 to September 2016 in accordance with generally accepted government auditing standards Those standards require that we plan and perform the audit to obtain sufficient appropriate evidence to provide a reasonable basis for our findings and conclusions based on our audit objectives We believe that the evidence obtained provides a reasonable basis for our findings and conclusions based on our audit objectives Page 33 GAO-16-511 Federal Agencies’ Use of Application Rationalization Appendix II GAO’s Evaluation of Agencies’ Application Inventories Appendix II GAO’s Evaluation of Agencies’ Application Inventories The following tables provide our evaluation of the 24 agencies’ application inventories Table 4 Department of Agriculture Practice Rating Includes business and enterprise IT systems Includes these systems from all organizational components ● ● Specifies basic application attributes ● Agency regularly updates the application inventory ◐ Explanation The department provided its fiscal year 2016 IT Asset Inventory that includes enterprise IT and business systems The inventory includes enterprise IT and business systems from all organizational components The inventory includes application name owner description and business function The department updated its IT Asset Inventory to coincide with OMB’s integrated data collection submission process The department provided emails requiring it components to respond to a data call that informed the update to the IT Asset Inventory for its final submission to OMB in May 2016 The department is expected to regularly reconcile and verify the data in its Enterprise Architecture Repository which maintains the IT Asset Inventory against data in the Cyber Security Administration and Management system to ensure the data within the repository is accurate and officials provided examples of memoranda to component Chief Information Officers addressing this reconciliation However officials stated that the department has not completed the reconciliation process for the IT Asset Inventory since it was moved to the Enterprise Architecture Repository In addition department officials stated that they have initiatives in place to improve the department’s IT Asset Inventory including conducting crosswalks of the data in the IT Asset Inventory Cyber Security Administration and Management system and Capital Planning and Investment Control inventories Source GAO analysis of agency documentation I GAO-16-511 Table 5 Department of Commerce Practice Rating Includes business and enterprise IT systems Includes these systems from all organizational components ● ● Specifies basic application attributes ● Agency regularly updates the application inventory ○ Explanation The department provided its fiscal year 2016 IT Asset Inventory that includes enterprise IT and business systems The inventory includes enterprise IT and business systems from all organizational components The department’s inventory includes application name owner description and business function The department did not provide any evidence of a process to regularly update its inventory or quality controls to ensure the reliability of the data collected Source GAO analysis of agency documentation I GAO-16-511 Page 34 GAO-16-511 Federal Agencies’ Use of Application Rationalization Appendix II GAO’s Evaluation of Agencies’ Application Inventories Table 6 Department of Defense DOD Practice Rating Includes business and enterprise IT systems Includes these systems from all organizational components ● ● Specifies basic application attributes ● Agency regularly updates the application inventory ● Explanation The department has a DOD IT Portfolio Repository which includes the department’s business systems a Two of the repository’s mission areas—the enterprise information environment area and the business mission area—include the business and enterprise IT systems in the scope of our review The inventory includes enterprise IT and business systems from all DOD’s organizational components The inventory includes system name owner description and business function The inventory is expected to be updated on a real-time basis and during annual reviews of business systems and DOD officials provided evidence of DOD officials provided an example of a system update as supporting evidence Department officials also stated that data quality reviews are performed on its inventory To support this they provided evidence of validations of the system update example and an investment decision memo showing that investments are reviewed before they are certified The department also provided metrics related to other data quality reviews that it performs For example in 2014 officials identified 178 systems that were potentially categorized in the enterprise information environment mission area instead of business mission area Source GAO analysis of agency documentation I GAO-16-511 a Pursuant to title 10 section 2222 a “defense business system” is and information system that is operated by for or on behalf of DOD including any of the follow ing a financial system a financial data feeder system a contracting system a logistics systems a planning and budgeting system an installations management system a human resources management system a training and readiness system The term does not include a national security system or an information system used exclusively by and w ithin the defense commissary system or the exchange system or other instrumentality of the Department of Defense conducted for the morale w elfare and recreation of members of the armed forces using nonappropriated funds Page 35 GAO-16-511 Federal Agencies’ Use of Application Rationalization Appendix II GAO’s Evaluation of Agencies’ Application Inventories Table 7 Department of Education Practice Rating Includes business and enterprise IT systems Includes these systems from all organizational components ● ● Specifies basic application attributes ● Agency regularly updates the application inventory ◐ Explanation The Department of Education provided its IT Asset Inventory which officials stated was created in response to the Office of Management and Budget’s request to establish one The inventory includes business and enterprise IT systems The department’s IT inventory includes systems associated with all organizational components with the exception of the Office of Legislative and Congressional Affairs International Affairs Office and Office of Education Technology which according to agency officials use shared services and enterprise IT and do not own any IT systems The department’s IT inventory specifies the name description owner and executive sponsor and business function code associated with the systems Department officials stated that they have not established policy for updating the IT Asset Inventory however they plan to publish inventory maintenance procedures specific to the Cyber Security Asset Management System which the department plans to merge the inventory into by December 2016 In regards to quality control processes officials stated that it is assumed that all identified systems have an authorization to operate because they would not be considered an official system without it However there is no policy explicitly stating this Officials stated that they will work with the department’s Information Assurance team to document the operating assumption for the systems inventory Source GAO analysis of agency documentation I GAO-16-511 Table 8 Department of Energy Practice Rating Explanation Includes business and enterprise IT systems Includes these systems from all organizational components ● ◐ ◐ Specifies basic application attributes Page 36 The department’s IT Asset Inventory includes business and enterprise IT systems The inventory does not include systems associated with all organizational components Department officials stated that the inventory is not representative of the entire department The department’s IT Asset Inventory includes system name description owner and executive sponsor contact information and identifies the business function code for some but not all of the systems listed in the inventory GAO-16-511 Federal Agencies’ Use of Application Rationalization Appendix II GAO’s Evaluation of Agencies’ Application Inventories ◐ Agency regularly updates the application inventory The department updated its IT Asset Inventory in conjunction with OMB’s integrated data collection submission for fiscal year 2016 and conducted a data call requesting that program offices submit their inventories in support of this update In regards to quality controls to ensure the reliability of the inventory department officials stated that they plan to use governance boards to review and validate the information included in the IT Asset Inventory as they have done in years past Officials also stated that they are working to make the inventory more comprehensive and accurate Source GAO analysis of agency documentation I GAO-16-511 Table 9 Department of Health and Human Services Practice Rating Includes business and enterprise IT systems Includes these systems from all organizational components ● ● Specifies basic application attributes ● Agency regularly updates the application inventory ◐ Explanation The department provided its fiscal year 2016 IT Asset Inventory that includes enterprise IT and business systems The inventory includes enterprise IT and business systems from all organizational components The inventory includes system name description owner and business function The department provided its policies and guidelines establishing requirements for systems to be entered into the inventory However it did not provide any evidence showing that it has implemented them In addition the department did not provide any evidence that it has implemented quality control processes to ensure the reliability of the data in the inventory Source GAO analysis of agency documentation I GAO-16-511 Table 10 Department of Homeland Security DHS Practice Rating Includes business and enterprise IT systems Includes these systems from all organizational components Specifies basic application attributes Page 37 ● ● ● Explanation The department provided an inventory that includes enterprise IT and business systems The inventory contains enterprise IT and business systems from the DHS’s organizational components The inventory contains application name description business function and owner GAO-16-511 Federal Agencies’ Use of Application Rationalization Appendix II GAO’s Evaluation of Agencies’ Application Inventories Practice Rating Agency regularly updates the application inventory ● Explanation The inventory is updated through routine change management and an annual refresh and both of these activities are expected to be performed by the Inventory Management Team as part of the FISMA compliance process For the change management activity components are required to submit change requests to the Inventory Management Team when there is a change in systems’ status The department provided change control forms as evidence that it regularly updates the inventory using this process For the annual refresh process the department reported that the Inventory Management Team works with the components to identify errors or omissions in the inventory and to make the changes and provided evidence of a refresh performed in 2016 DHS also has quality control processes for ensuring the reliability of the data in the inventory In addition to the annual refresh mentioned earlier they include the process for discovering hidden applications for which DHS provided a Software Approval Report According to the department this process entails comparing software associated with devices to the software in the inventory to ensure it is approved and resolving cases where components are using “prohibited” or “not approved” software Source GAO analysis of agency documentation I GAO-16-511 Table 11 Department of Housing and Urban Development Practice Rating Includes business and enterprise IT systems Includes these systems from all organizational components Specifies basic application attributes Agency regularly updates the application inventory ● ◐ ◐ ◐ Explanation The department provided its Inventory of Automated Systems which includes enterprise IT and business systems in its managed service environment While the department’s inventory includes enterprise IT and business systems it does not include all systems from its local offices The inventory includes application name owner and description however it does not include business function The department provided its IT security policy inventory user guide and a description of past validations as support that it regularly updates its inventory Nevertheless the department did not provide any artifacts to corroborate that it actually regularly updates its inventory Furthermore it reported that it does not currently have in place any quality assurance processes to ensure the reliability of the data in the inventory Source GAO analysis of agency documentation I GAO-16-511 Page 38 GAO-16-511 Federal Agencies’ Use of Application Rationalization Appendix II GAO’s Evaluation of Agencies’ Application Inventories Table 12 Department of the Interior Interior Practice Rating Includes business and enterprise IT systems Includes these systems from all organizational components ◐ ● Specifies basic application attributes ● Agency regularly updates the application inventory ○ Explanation Interior provided a spreadsheet of its applications associated with IT investments in the mission delivery and management support area Department officials stated that the inventory likely does not include all applications supporting business functions In addition it does not include enterprise IT systems The inventory contains business systems from all organizational components Interior’s inventory includes system name description function and business sponsor The department stated that it reviews the inventory data on at least an annual basis However officials said it is reliant on manual data collection and lacks robust automated tools to manage and analyze the data Officials also reported that the department updated its inventory through its annual inventory update process subsequent to a February 2016 meeting that was held with its organizational components to discuss the future collection of application information However while they provided evidence of the meeting they did not provide evidence of updates to the inventory or quality control processes to ensure its reliability Source GAO analysis of agency documentation I GAO-16-511 Table 13 Department of Justice Practice Rating Includes business and enterprise IT systems Includes these systems from all organizational components ● ● ● Specifies basic application attributes Page 39 Explanation The department and the Federal Bureau of Investigation provided inventories that include enterprise IT and business systems a Together the inventories include enterprise IT and business systems from all organizational components The inventories each include system name description owner and business function GAO-16-511 Federal Agencies’ Use of Application Rationalization Appendix II GAO’s Evaluation of Agencies’ Application Inventories Practice Rating Agency regularly updates the application inventory ● Explanation The department updates its Cyber Security and Assessment Management inventory system continuously as the condition of systems change It also performs quality control processes to ensure the reliability of the data in the inventory It provided evidence of these processes and its Security Authorization and Assessment Handbook which includes requirements for the department’s inventory to be updated regularly and validated through system assessments for security purposes The Federal Bureau of Investigation’s inventory is updated through user submitted change requests The bureau also performs quality control processes on its inventory on a daily weekly monthly annual and ad hoc basis The Federal Bureau of Investigation provided evidence of these quality control processes and its Data Quality Procedures and Checklist document Source GAO analysis of agency documentation I GAO-16-511 a According to officials the Federal Bureau of Investigation’s inventory—known as the Bureau IT Know ledge Repository—was created to manage the FBI’s IT portfolio identify systems for consolidation replacement or retirement and facilitate responding to data calls from DOJ and OMB It is currently use to capture and register information about IT systems w ithin the FBI in a central repository Table 14 Department of Labor Labor Practice Rating Includes business and enterprise IT systems Includes these systems from all organizational components ◐ ◐ Specifies basic application attributes ◐ Agency regularly updates the application inventory ○ Explanation Labor provided a list of applications which it stated includes 22 business mission support applications but does not include all of its enterprise IT and business systems According to officials including the Chief Information Officer there is no comprehensive inventory of enterprise IT and business systems The department does not have a comprehensive list of applications from all its organizational components The list includes application name description and owner However it does not include business function The department did not provide any evidence of a process to regularly update its inventory or quality controls to ensure the reliability of the data collected Source GAO analysis of agency documentation I GAO-16-511 Page 40 GAO-16-511 Federal Agencies’ Use of Application Rationalization Appendix II GAO’s Evaluation of Agencies’ Application Inventories Table 15 Department of State Practice Rating Includes business and enterprise IT systems Includes these systems from all organizational components ● The department provided an inventory that includes enterprise IT and business systems ● Specifies basic application attributes ◐ Agency regularly updates the application inventory ◐ Explanation The inventory includes enterprise IT and business systems from all the department’s organizational components The inventory includes application name description and owner However it does not include a business function for the majority of inventory entries The department provided documentation of one of its periodic data calls to investment owners and program managers as evidence that it regularly updates the inventory However it did not provide evidence that quality control processes are in place to ensure the reliability of the data in the inventory Source GAO analysis of agency documentation I GAO-16-511 Table 16 Department of Transportation Practice Rating Includes business and enterprise IT systems Includes these systems from all organizational components ◐ ◐ ◐ Specifies basic application attributes Page 41 Explanation The Department of Transportation provided a spreadsheet containing its list of applications and investments which officials stated are associated with enterprise IT and business systems We verified that the list included business systems but was missing some enterprise IT systems associated with e-mail and security The department also provided an inventory for its common operating environment which includes commodity IT applications The department’s list of applications and investments includes software applications from some but not all components In addition the inventory for the common operating environment does not include software owned by the Federal Aviation Administration The department’s inventory has basic attribute information to include the component using it the application name and description department enterprise architecture segment it is mapped to and the federal enterprise architecture business function it supports among other things However the inventory for its common operating environment does not identify the business function supported GAO-16-511 Federal Agencies’ Use of Application Rationalization Appendix II GAO’s Evaluation of Agencies’ Application Inventories Practice Rating Agency regularly updates the application inventory ◐ Explanation In 2014 the department’s application inventory was updated through the update to the Enterprise Architecture Roadmap and it was determined that this effort would continue annually however officials did not provide evidence of any efforts to inform the next update to the Enterprise Architecture Roadmap In addition while officials stated that they rely on their Operating Administration partners to provide up-to-date and accurate information on their individual inventories in order to develop the department’s list of applications and investments the department does not have processes for ensuring the reliability of the reported information Source GAO analysis of agency documentation I GAO-16-511 Table 17 Department of the Treasury Practice Rating Includes business and enterprise IT systems Includes these systems from all organizational components ● ● Specifies basic application attributes ◐ Agency regularly updates the application inventory ◐ Explanation The Department of the Treasury provided its inventory as contained in the Treasury FISMA Inventory Management System which officials stated includes most of the applications associated with business and enterprise IT systems with the exception of e-mail which according to officials are part of a general support system and not required to be listed individually The department’s inventory includes applications from all of its bureaus and departmental offices The inventory identifies the component using the system and system name However it does not include a basic description of the applications or the business segment function they support While officials stated that the department’s FISMA Inventory Management System includes these attributes they did not provide supporting evidence According to officials and department policy the inventory is updated continuously as applications are deployed upgraded or decommissioned However the department did not provide evidence of actual updates to the inventory In addition officials stated that they conduct quality control processes to ensure the reliability of the inventory data through the annual FISMA audit performed by the department’s Office of Inspector General However our review of the 2014 FISMA evaluation report found that the audit addressed the inventory compliance with FISMA and other related information security policies procedures standards and guidelines but did not include quality control processes to ensure the reliability of the data collected Such processes are critical given that the department relies on information provided by its bureaus Source GAO analysis of agency documentation I GAO-16-511 Page 42 GAO-16-511 Federal Agencies’ Use of Application Rationalization Appendix II GAO’s Evaluation of Agencies’ Application Inventories Table 18 Department of Veterans Affairs Practice Rating Includes business and enterprise IT systems Includes these systems from all organizational components ● ● Specifies basic application attributes ● Agency regularly updates the application inventory ◐ Explanation The department provided a table of support systems contained in its Systems Inventory which includes business and enterprise IT systems The list provided includes IT support systems used by all of the department’s organizations with the exception of the Office of Inspector General—due to statutory independence—and the Office of Government Relations which does not sponsor any IT systems The inventory provided specifies basic attribute information to include the system name the parent organizations a basic description of the applications and the business function they support The department provided a table of support systems contained in its Systems Inventory which includes business and enterprise IT systems The list provided includes IT support systems used by all of the department’s organizations with the exception of the Office of Inspector General—due to statutory independence—and the Office of Government Relations which does not sponsor any IT systems The inventory provided specifies basic attribute information to include the system name the parent organizations a basic description of the applications and the business function they support The department updates the inventory continuously as changes occur Furthermore department policy requires that system inventory information be updated or validated during operational assessments or any IT system reviews The department also has quality control processes to ensure the reliability of the information collected For example officials stated that the Enterprise Architecture Management Suite environment enables reporting against information in the enterprise architecture In addition the department reports on inventory performance metrics which officials stated are to ensure that leadership has visibility into any dated or missing information However while officials stated that their repository of systems is viewed as complete the information within the repository is still maturing and work is being done to automate data capture and integration with other sources Source GAO analysis of agency documentation I GAO-16-511 Page 43 GAO-16-511 Federal Agencies’ Use of Application Rationalization Appendix II GAO’s Evaluation of Agencies’ Application Inventories Table 19 Environmental Protection Agency Practice Rating Includes business and enterprise IT systems Includes these systems from all organizational components ● ● Specifies basic application attributes ◐ Agency regularly updates the application inventory ● Explanation The agency provided its Registry of Environmental Protection Agency Applications Models and Databases system which contains its application inventory The commodity IT categories of enterprise IT systems and business systems are included in the inventory with the exception of very small systems e g local office carpool tracking systems which agency officials stated are not required to be registered but can be added at the discretion of component offices The inventory includes software applications from all offices and regions of the organization The inventory includes the component or region managing the application the application name and application description The inventory also includes the Primary Business Reference Model name and code that identify the primary business function of the application Although the inventory does not identify the business function associated with all applications officials stated that they are working to have this information populated for all applications The agency has processes to routinely update its application inventory and does so consistent with policy Officials stated that program offices use the annual data call issued by the Chief Information Officer CIO to ensure their respective portfolios in its registry are current They provided an example e-mail from the CIO which requests that the program offices perform their annual data call and update and it includes requirements and instructions for doing so In addition the agency implements quality control processes to ensure the reliability of the inventory in its registry For example although officials stated that they rely on selfreported information from program offices and Information Management Officers to update the inventory there is a steward for each record who officials stated is knowledgeable about the particular system and has edit rights to update that record In addition a notification is sent to the officers when a record that they are responsible for is updated or changed and agency officials stated that a report can be generated that will show which records are missing information The agency provided sample e-mails from stewards to the officers requesting changes to records and from the registry to the officers notifying them of record updates Source GAO analysis of agency documentation I GAO-16-511 Page 44 GAO-16-511 Federal Agencies’ Use of Application Rationalization Appendix II GAO’s Evaluation of Agencies’ Application Inventories Table 20 General Services Administration Practice Rating Includes business and enterprise IT systems Includes these systems from all organizational components ● ● Specifies basic application attributes ● Agency regularly updates the application inventory ● Explanation The agency provided its inventory of applications which officials stated is contained in IBM’s System Architect tool The inventory includes enterprise IT and business systems The inventory includes software applications from all components bureaus of the organization except the Board of Contract Appeals which does not have any applications of its own The inventory specifies the application name and description the office associated with the application and the business capability each application is mapped to The agency updates its application inventory on an ongoing basis based on customer input the Enterprise Architecture team’s interaction with subject matter experts and other methods To ensure the reliability of the data in the inventory the Enterprise Architecture Team reaches out to the subject matter experts at a minimum quarterly to review their applications The agency also makes use of scanning tools to discover technologies and software components on its networks—some of which make up the business and enterprise IT applications in the inventory Officials told us they are working to map the IT technologies and software components to the business applications Source GAO analysis of agency documentation I GAO-16-511 Table 21 National Aeronautics and Space Administration NASA Practice Rating Includes business and enterprise IT systems Includes these systems from all organizational components Page 45 ● ◐ Explanation NASA provided a spreadsheet containing a list of applications derived from its System for Tracking and Registering Applications and Websites for external facing applications and various tools that capture application information for internal facing systems The list included business and enterprise IT systems with the exception of email because according to officials the agency acquires enterprise licenses for applications such as these and they are tracked separately by NASA’s Enterprise License Management Team The inventory does not include all software applications from all the agency components NASA officials stated that where available center-specific software applications are included in the inventory provided however they are currently working with agency business owners through the preliminary Annual Capital Investment Review to ensure complete inventories of center-specific applications GAO-16-511 Federal Agencies’ Use of Application Rationalization Appendix II GAO’s Evaluation of Agencies’ Application Inventories Practice Rating Specifies basic application attributes ● Agency regularly updates the application inventory ◐ Explanation The inventory specifies application name and description In addition the agency component using the listed application and business segment function that the applications support are tracked in a separate tool The agency described its continuous and annual update processes and provided documentation of external facing systems being added or updated in the System for Tracking and Registering Application and Websites Agency officials stated that they are also currently updating its processes for including internally facing applications as part of the Annual Capital Investment Review process In regards to implementing quality control processes agency officials stated that for external systems they reconcile security scan data with the inventory However they did not provide documentation showing that they actually perform this reconciliation Source GAO analysis of agency documentation I GAO-16-511 Table 22 National Science Foundation NSF Practice Rating Includes business and enterprise IT systems Includes these systems from all organizational components ● ● Specifies basic application attributes ● Agency regularly updates the application inventory ◐ Explanation NSF provided its inventory of business support applications contained in the NSF FISMA inventory The inventory includes the agency’s business and enterprise IT applications NSF’s FISMA inventory includes systems from all organizational components The inventory includes the application sponsor i e owner name and class e g minor major or general support system In addition the agency’s Enterprise Modernization Roadmap provides a basic description for and identifies the business segment function supported by most applications NSF’s inventory is updated as needed through IT governance enterprise architecture management budgeting and planning and the system development life cycle NSF provided documentation of the retirement of its legacy financial system and replacement with another system which informed the update to the NSF FISMA inventory as an example Regarding quality control processes agency officials stated that they conduct a major validation review of the inventory annually as a part of its FISMA processes to ensure the reliability of the information in it however they did not provide any evidence of the validation review Source GAO analysis of agency documentation I GAO-16-511 Page 46 GAO-16-511 Federal Agencies’ Use of Application Rationalization Appendix II GAO’s Evaluation of Agencies’ Application Inventories Table 23 Nuclear Regulatory Commission Practice Rating Includes business and enterprise IT systems Includes these systems from all organizational components ● ● Specifies basic application attributes ● Agency regularly updates the application inventory ◐ Explanation The agency provided a spreadsheet containing its application inventory maintained in the NRC System Inventory Control Database The inventory includes custom-developed systems and shared services that are associated with business and enterprise IT systems The systems inventory includes software applications from all offices in the agency with the exception of the offices which according to officials do not use custom developed systems or own any software applications These offices include the Office of Commission Appellate Adjudication Office of Congressional Affairs and Office of Small Business and Civil Rights its Office of International Programs and Region IV do not own any software applications The list includes the application name a brief description associated office business segment it supports and the Federal Enterprise Architecture business function name The agency has processes to routinely update its application inventory however documentation associated with these processes has not been finalized Further according to officials the inventory is updated monthly through software detection In addition the agency implements quality control processes to ensure the reliability of the application information collected through NRC’s Cybersecurity Program and Information Security Continuous Monitoring process Specifically it requires continuous application scanning to be performed to identify changes to any systems and environments in which those systems operate in order for them to maintain authorization to operate In addition NRC’s Cybersecurity Assessment process includes policies and procedures for manually examining system and information integrity Source GAO analysis of agency documentation I GAO-16-511 Table 24 Office of Personnel Management Practice Rating Includes business and enterprise IT systems Includes these systems from all organizational components ● ● ● Specifies basic application attributes Page 47 Explanation The agency provided its application inventory which includes business and enterprise IT systems The repository includes systems from all organizational components with the exception of the Offices of Diversity and Inclusion Procurement Operations and Small and Disadvantaged Business Utilization which are small offices The application inventory provided includes the application name description and associated organization The business function mappings are included in the IT asset inventory required by OMB GAO-16-511 Federal Agencies’ Use of Application Rationalization Appendix II GAO’s Evaluation of Agencies’ Application Inventories Practice Rating Agency regularly updates the application inventory ◐ Explanation The agency described the process for updating the inventory including procedures for adding new applications however it did not provide evidence of actual updates to the inventory In addition the agency stated it relies on manual reviews of the data in the application to ensure that it is complete and current The agency reported it is taking steps to implement scanning tools to verify the reliability of the data in the inventory Source GAO analysis of agency documentation I GAO-16-511 Table 25 Small Business Administration Practice Rating Includes business and enterprise IT systems Includes these systems from all organizational components ◐ ◐ Specifies basic application attributes ◐ Agency regularly updates the application inventory ○ Explanation The agency provided a list of applications and systems that includes some not all of its business and enterprise IT systems The list provided only includes systems from some headquarters offices In addition the agency stated that a number of field offices are running unsupported applications to help them with their work tasks Officials reported that the agency has begun an initiative to identify and document unreported systems The list includes system name description and owner however it does not include business function The agency reported that it has begun using automated tools in efforts to develop a complete application inventory but provided no supporting evidence In addition the agency stated it will continue to refine its automation efforts and work on its draft Software Asset Lifecycle Management framework with criteria that will be applied to each application in the portfolio SBA reported that it hopes to publish this framework by the end of the year Source GAO analysis of agency documentation I GAO-16-511 Table 26 Social Security Administration Practice Rating Includes business and enterprise IT systems Includes these systems from all organizational components ● ◐ ● Specifies basic application attributes Page 48 Explanation The agency provided its Application Portfolio Management Inventory that includes applications aligned with IT and mission support functions The inventory includes enterprise IT and business systems The inventory contains enterprise IT and business systems developed within its systems organization however it does not include applications developed by its operational components The inventory provides the application name description function owner and business sponsor GAO-16-511 Federal Agencies’ Use of Application Rationalization Appendix II GAO’s Evaluation of Agencies’ Application Inventories Practice Rating Agency regularly updates the application inventory ◐ Explanation The agency reported that its systems development lifecycle contains steps to register and maintain application information but did not provide evidence to show that it is using this process to regularly update the inventory Regarding quality control processes the agency reported that applications are discovered through the architecture review board processes and provided a data discovery report as supporting evidence It also reported that it has efforts underway to catalog applications outside its systems organization and has begun implementing an additional data discovery process using interviews with project teams Source GAO analysis of agency documentation I GAO-16-511 Table 27 U S Agency for International Development Practice Rating Includes business and enterprise IT systems Includes these systems from all organizational components ● ◐ Specifies basic application attributes ● Agency regularly updates the application inventory ◐ Explanation The agency provided a list of applications that includes the business and enterprise IT commodity IT categories Officials stated that coordination and communication in their geographically widespread agency presents challenges to including systems from all organizational components For example identifying the appropriate points of contact and receiving a timely response from all bureaus and Independent Offices makes it difficult to do so The application inventory includes the system name owner description and the service area aligned to the applications The agency has processes for updating its inventory however it relies on manual processes for doing so Specifically officials stated that the application list is updated through data calls and research conducted by the enterprise architecture team Also the agency implements quality control processes to ensure the reliability of application information For example it provided documentation of its FISMA data collection conducted on every computer system Source GAO analysis of agency documentation I GAO-16-511 Page 49 GAO-16-511 Federal Agencies’ Use of Application Rationalization Appendix III Comments from the Department of Com merce Appendix III Comments from the Department of Commerce Page 50 GAO-16-511 Federal Agencies’ Use of Application Rationalization Appendix III Comments from the Department of Com merce Page 51 GAO-16-511 Federal Agencies’ Use of Application Rationalization Appendix IV Comments from the Department of Defense Appendix IV Comments from the Department of Defense Page 52 GAO-16-511 Federal Agencies’ Use of Application Rationalization Appendix IV Comments from the Department of Defense Page 53 GAO-16-511 Federal Agencies’ Use of Application Rationalization Appendix IV Comments from the Department of Defense Page 54 GAO-16-511 Federal Agencies’ Use of Application Rationalization Appendix V Comments from the Department of Education Appendix V Comments from the Department of Education Page 55 GAO-16-511 Federal Agencies’ Use of Application Rationalization Appendix VI Comments from the Department of Energy Appendix VI Comments from the Department of Energy Page 56 GAO-16-511 Federal Agencies’ Use of Application Rationalization Appendix VI Comments from the Department of Energy Page 57 GAO-16-511 Federal Agencies’ Use of Application Rationalization Appendix VII Comments from the Department of Health and Hum an Services Appendix VII Comments from the Department of Health and Human Services Page 58 GAO-16-511 Federal Agencies’ Use of Application Rationalization Appendix VII Comments from the Department of Health and Hum an Services Page 59 GAO-16-511 Federal Agencies’ Use of Application Rationalization Appendix VIII Comments from the Department of Hom eland Security Appendix VIII Comments from the Department of Homeland Security Page 60 GAO-16-511 Federal Agencies’ Use of Application Rationalization Appendix VIII Comments from the Department of Hom eland Security Page 61 GAO-16-511 Federal Agencies’ Use of Application Rationalization Appendix IX Comments from the Department of Housing and Urban Development Appendix IX Comments from the Department of Housing and Urban Development Page 62 GAO-16-511 Federal Agencies’ Use of Application Rationalization Appendix X Comments from the Department of the Interior Appendix X Comments from the Department of the Interior Page 63 GAO-16-511 Federal Agencies’ Use of Application Rationalization Appendix X Comments from the Department of the Interior Page 64 GAO-16-511 Federal Agencies’ Use of Application Rationalization Appendix XI Department of Justice Appendix XI Department of Justice Page 65 GAO-16-511 Federal Agencies’ Use of Application Rationalization Appendix XI Department of Justice Page 66 GAO-16-511 Federal Agencies’ Use of Application Rationalization Appendix XII Comments from the Department of Labor Appendix XII Comments from the Department of Labor Page 67 GAO-16-511 Federal Agencies’ Use of Application Rationalization Appendix XIII Comments from the Department of State Appendix XIII Comments from the Department of State Page 68 GAO-16-511 Federal Agencies’ Use of Application Rationalization Appendix XIII Comments from the Department of State Page 69 GAO-16-511 Federal Agencies’ Use of Application Rationalization Appendix XIV Comments from the Department of Veterans Affairs Appendix XIV Comments from the Department of Veterans Affairs Page 70 GAO-16-511 Federal Agencies’ Use of Application Rationalization Appendix XIV Comments from the Department of Veterans Affairs Page 71 GAO-16-511 Federal Agencies’ Use of Application Rationalization Appendix XV Comments from the Environm ental Protection Agency Appendix XV Comments from the Environmental Protection Agency Page 72 GAO-16-511 Federal Agencies’ Use of Application Rationalization Appendix XV Comments from the Environm ental Protection Agency Page 73 GAO-16-511 Federal Agencies’ Use of Application Rationalization Appendix XVI Comments from the National Aeronautics and Space Administration Appendix XVI Comments from the National Aeronautics and Space Administration Page 74 GAO-16-511 Federal Agencies’ Use of Application Rationalization Appendix XVI Comments from the National Aeronautics and Space Administration Page 75 GAO-16-511 Federal Agencies’ Use of Application Rationalization Appendix XVII Nuclear Regulatory Commission Appendix XVII Nuclear Regulatory Commission Page 76 GAO-16-511 Federal Agencies’ Use of Application Rationalization Appendix XVIII Comments from the Office of Personnel Management Appendix XVIII Comments from the Office of Personnel Management Page 77 GAO-16-511 Federal Agencies’ Use of Application Rationalization Appendix XVIII Comments from the Office of Personnel Management Page 78 GAO-16-511 Federal Agencies’ Use of Application Rationalization Appendix XIX Com ments from the Social Security Administration Appendix XIX Comments from the Social Security Administration Page 79 GAO-16-511 Federal Agencies’ Use of Application Rationalization Appendix XIX Com ments from the Social Security Administration Page 80 GAO-16-511 Federal Agencies’ Use of Application Rationalization Appendix XIX Com ments from the Social Security Administration Page 81 GAO-16-511 Federal Agencies’ Use of Application Rationalization Appendix XX GAO Contact and Staff Acknowledgments Appendix XX GAO Contact and Staff Acknowledgments GAO Contact David A Powner 202 512-9286 or pownerd@gao gov Staff Acknowledgments In addition to the individual named above the following staff made key contributions to this report Sabine Paul Assistant Director Chris Businsky Rebecca Eyler Dan Gordon James MacAulay Lori Martinez Paul Middleton and Di’Mond Spencer 100090 Page 82 GAO-16-511 Federal Agencies’ Use of Application Rationalization GAO’s Mission The Government Accountability Office the audit evaluation and investigative arm of Congress exists to support Congress in meeting its constitutional responsibilities and to help improve the performance and accountability of the federal government for the American people GAO examines the use of public funds evaluates federal programs and policies and provides analyses recommendations and other assistance to help Congress make informed oversight policy and funding decisions GAO’s commitment to good government is reflected in its core values of accountability integrity and reliability Obtaining Copies of GAO Reports and Testimony The fastest and easiest way to obtain copies of GAO documents at no cost is through GAO’s website http www gao gov Each weekday afternoon GAO posts on its website newly released reports testimony and correspondence To have GAO e-mail you a list of newly posted products go to http www gao gov and select “E-mail Updates ” Order by Phone The price of each GAO publication reflects GAO’s actual cost of production and distribution and depends on the number of pages in the publication and whether the publication is printed in color or black and white Pricing and ordering information is posted on GAO’s website http www gao gov ordering htm Place orders by calling 202 512-6000 toll free 866 801-7077 or TDD 202 512-2537 Orders may be paid for using American Express Discover Card MasterCard Visa check or money order Call for additional information Connect with GAO Connect with GAO on Facebook Flickr Twitter and YouTube Subscribe to our RSS Feeds or E-mail Updates Listen to our Podcasts Visit GAO on the web at www gao gov To Report Fraud Waste and Abuse in Federal Programs Contact Congressional Relations Katherine Siggerud Managing Director siggerudk@gao gov 202 512-4400 U S Government Accountability Office 441 G Street NW Room 7125 Washington DC 20548 Public Affairs Chuck Young Managing Director youngc1@gao gov 202 512-4800 U S Government Accountability Office 441 G Street NW Room 7149 Washington DC 20548 Strategic Planning and External Liaison James-Christian Blockwood Managing Director spel@gao gov 202 512-4707 U S Government Accountability Office 441 G Street NW Room 7814 Washington DC 20548 Website http www gao gov fraudnet fraudnet htm E-mail fraudnet@gao gov Automated answering system 800 424-5454 or 202 512-7470 Please Print on Recycled Paper