Friday, September 13, 2013

How I Learned to Stop Worrying about Security and Love Incremental Development

This is a follow-up to my previous post: Agile Development and Security in Government.

The security authorization processes that U.S. Government agencies implement to comply with guidelines defined by the National Institute of Standards and Technology (NIST) fail to support incremental development methodologies like agile and spiral. Instead, I argue that they promote "big-bang" system releases that are impractical in contemporary budgetary conditions and generally seem to fail more than they succeed. Agency Chief Information Officers (CIOs) can fix the problem, but only if they reinvent their authorization processes and redefine process responsibilities.

Like Dr. Strangelove's bomb, agency Information System Security Managers (ISSMs) generally discount the importance of incremental development to streamlining government. When I met with a security policy executive at one agency to discuss a reconciliation between my project's agile incremental management methodology and the agency's security process, she was none-too-pleased with the notion.

"Projects use agile to bypass security requirements. I will not allow that to happen."

I don't doubt that there is a lot of truth in that statement. Government project managers bypass security requirements (or really any policy requirements) more often than someone complains about government inefficiency. But, I think that her refusal only makes the problem worse. My project manager was not going to reconsider his use of agile because he (and I) believed that incremental development was critical to the project's success following a nearly catastrophic multiyear "big-bang" failure in the system's prior life. So, the only plausible path was for the project to proceed "under-the-radar," putting me in the unenviable position of having to manage against two competing processes.

I believe that there is a better way.

When incremental development processes such as agile development are employed, this typically results in an incremental assessment as each cycle is conducted. A similar process is used for assessing security controls in COTS information technology products employed within the information system. Even when incremental development is not employed, organizations may choose to begin assessing security controls prior to the complete implementation of all security controls listed in the security plan. This type of incremental assessment is appropriate if it is more efficient or cost-effective to do so. [NIST SP 800-37rev1, Page 31 under Security Control Assessment]

With this statement, NIST grants agencies the flexibility to redefine their processes. While NIST fails to provide any specific guidance to help government agencies effectively and securely implement incremental development methodologies, it grants the agencies license to define their own security policies with relation to incremental development. Doing so will require that agencies address several challenges to integrate security effectively and may force some experimentation. I submit that by going through the effort, agencies will not only gain the cost efficiency and productivity enhancements promised by incremental development practitioners, they will likely enhance their overall security in the process.

Agencies face some general challenges to effectively integrate incremental development methodologies into their security programs. Since an emphasis on rapid development tends to favor meeting functional requirements over process execution, agencies will need to adjust their security compliance activities to promote broader integration potential. Because incremental development efforts often avoid addressing potential security risks to minimize their impact on release delivery, practitioners can inappropriately delay implementation of some security controls, resulting in vulnerabilities that they could avoid with careful planning. Also, the limited (usually nonexistent) focus on developing documentation throughout incremental phases or sprints represents a real barrier to satisfying compliance needs.

Reconciling the apparently diverging desires of incremental development and security practitioners is possible, but doing so will demand effort on both sides. Project managers must accept that incremental releases may not go into production without receiving an Authority to Operate (ATO). Understanding that will force them to consider incorporating security phases into releases. But, cutting through a culture of dissonant acceptance that security is less important than core functionality will be a herculean task in and of itself.

Making security a high priority is just the start. Anyone who has ever worked for the U.S. Government knows that the bureaucracy seems to be designed to discourage good ideas (and good people, for that matter) from succeeding. But, for incremental development to succeed (in the open), agencies will likely need to adjust their security authorization processes to the reality that an incrementally developed system will be under constant development. Project managers already know this to be the case as their contemporary users have little patience for multi-year development cycles. They demand functional enhancements to arrive in frequent updates.

Agency officials, starting with the CIO and then down the reporting chain, also need to prepare themselves to grant conditional security authorization based on a security control design versus an operational system. I've been against doing this in the past and would still advocate against it for a more traditional waterfall development effort. But, for secure incremental development to succeed, I believe that the Information Security Officer (ISO) needs to be more involved in the development. Adding a formal design authorization step would give project managers more flexibility in how they implement to the design over time while also forcing more frequent ISO reviews. With more frequent reviews, the agency benefits from being able to identify and remediate new security risks more proactively.

To execute a release, the project manager would then need to scope a Security Control Assessment (SCA) in parallel with release frequency. While normally an arduous task in every agency that I've had the privilege of supporting, I believe that agencies can execute SCA's more incrementally themselves with just some subtle changes. With the design already authorized, the agency could design the SCA to avoid lengthy documentation reviews and focus instead on testing the implementation of security controls. With greater involvement over process, the ISO should be able to identify security control changes and additions made during the course of release development. Then, the agency could focus the SCA on only those changes rather than retesting everything every time. It could then authorize the release based on the SCA results, push minor and moderate findings as requirements into the next increment or release, or push release to the next increment until major findings are addressed.

I am probably making this sound easy. It's not. It will take commitment by project managers and agency CIOs alike to change in some significant ways. Policies will need to change, contracts will have to be modified, personnel will have to adjust their skill sets. In the government, all of those changes will be painful.

However, for those who make the commitment, the rewards will be great. Project managers would be able to better meet the needs of their user populations without constantly defending themselves to the agency Inspector General (IG) and/or Congress. Agencies would gain the benefit of catching security weaknesses early and often, allowing them to engage more than command, and reduce the expense of fixing problems.

Pushing security authorization to be more "incremental" itself is a win/win for everyone.

For much of my career, I served as an enterprise security consultant for the U.S. government. I derived much of the basis for this post from my experiences on the inside. My experience includes a stint as a trusted advisor to a Treasury Department Chief Information Security Officer (CISO) whom I assisted in developing a broad agency-wide security program, a couple years as a security program manager for a major infrastructure consolidation effort at the Department of Homeland Security (DHS), and as a security program manager for a large-scale project at the Department of Veterans Affairs (VA) where we applied an Agile methodology modified to support system integration management. I currently manage an applied research development project where I designed another modified Agile methodology, this time focused on incremental development to support short-term research objectives within a multi-year strategic research vision. Whereas my current work requires no interaction with security certification authorities (yet), I worked closely with ISSOs/ISSMs in my past work and the current project resides in a domain with strong security requirements. I speak from my general experiences and nothing that I discuss in this post should be attributed to any specific project or other engagement.