Security by Design: An Annotated Resource List

In August 2023, Lawfare announced a new multiyear project that aims to evaluate several elements related to “security by design,” including secure-by-design principles and how legal and policy processes could require or incentivize security by design from software developers. As the White House explores how to develop a software liability framework, Lawfare’s project aims to be a timely and useful contribution to these discussions.

Upcoming work within this project seeks to answer four questions: 

• Is there a useful working definition of “security by design”? 
• Can security by design be measured? If so, how? 
• How can security-by-design principles be translated into articulable standards?
• How can standards best be incentivized or imposed?

This resource list is a point-in-time effort, last updated in December 2023, to capture relevant background information that is informing our work, and offers a convenient resource to access government documents, guidelines, corporate practice, and analysis on the subject of security by design. It is not our intention to offer a comprehensive bibliography of all such documents. We have organized them under two general categories: policy and guidance from the U.S. government, which encompasses both public procurement considerations on secure software and broader policy directed at the private sector; and development best practices from a variety of stakeholders. You can find these and other documents on our dedicated Security by Design page.

Software Security and the U.S. Government 

This section includes policy and guidance from the U.S. government on the matter of software security. One way in which the U.S. government creates incentives for secure software is through the power of the purse. By setting conditions in the public procurement process, more companies will adopt those standards to access profitable government contracts or grants. It becomes easier for most companies to have only one development process, and so the benefits of secure software development trickle down to the consumer market. Additionally, the Biden administration has taken a more direct stance on the matter, by announcing that it will begin considering a software liability regime that holds companies accountable through civil liability for security results in their products. The Cybersecurity and Infrastructure Security Agency (CISA) security-by-design and -default guidelines offer information on the type of behaviors that can be expected from responsible software developers.

Procuring Secure Software

Executive Order 14028 on Improving the Nation’s Cybersecurity (May 12, 2021)

This executive order outlines different ways in which the federal government should respond to “persistent and increasingly sophisticated malicious cyber campaigns” that affect all sectors. Although the order does not discuss security by design and default, it does dedicate an entire section to enhancing software supply chain security. In fact, it stresses: 

The development of commercial software often lacks transparency, sufficient focus on the ability of the software to resist attack, and adequate controls to prevent tampering by malicious actors. There is a pressing need to implement more rigorous and predictable mechanisms for ensuring that products function securely, and as intended.

To improve the security of its own supply chain, the executive order requires the National Institute of Standards and Technology (NIST) to conduct consultations and later publish guidance on the practices that “enhance the security of the software supply chain.” The guidance should cover (1) secure software development environments, (2) generation and provision of conformance artifacts, (3) processes to maintain trusted source code supply chains, (4) processes that check for known and potential vulnerabilities and remediate them, (5) artifacts attesting to the execution of (3) and (4), (6) knowing the origin of code and components, (7) providing an software bill of materials (SBOM), (8) participating in a vulnerability disclosure program, (9) attesting to conformity with secure software development practices, and (10) ensuring and attesting to the integrity and provenance of open-source software used.

NIST is also tasked with “identify[ing] secure software development practices or criteria for a consumer software labeling program, and shall consider whether such a consumer software labeling program may be operated in conjunction with or modeled after any similar existing government programs, consistent with applicable law.”

Software Supply Chain Security Guidance (NIST, Feb. 4, 2022)

Issued in accordance with Executive Order 14028, the guidance includes an explanation of NIST’s approach to addressing the order and provides guidelines for federal agency staff involved in procuring and acquiring software. The guidance does not distinguish between on-premises or cloud-hosted software. The NIST guidance provides four minimum recommendations for staff to attest the conformity of software with secure development practices: (1) use the Secure Software Development Framework (SSDF) terminology and structure to organize communications about secure software development requirements; (2) require attestation to cover secure software development practices performed as part of processes and procedures throughout the software life cycle; (3) accept first-party attestation of conformity with SSDF practices unless a risk-based approach determines that second- or third-party attestation is required; and (4) when requesting artifacts of conformance, request high-level artifacts.

Secure Software Development Framework (SSDF) Version 1.1: Recommendations for Mitigating the Risk of Software Vulnerabilities (NIST, February 2022)

The NIST SSDF focuses on the outcomes of practices that can be integrated and implemented throughout the software development life cycle to enhance software security. Notably, the paper begins with a recognition that few software development life cycles explicitly address software security in detail. The recommendations are technology and producer neutral, since each product and service will have unique needs and requirements, and focus on organizational processes and actions. The SSDF practices are structured into four groups: (1) prepare the organization, (2) protect the software, (3) produce well-secured software, and (4) respond to vulnerabilities. Together, these practices cover the integration of software security at different stages, from creating new roles and providing secure development training to using secure development practices and addressing the root causes or vulnerabilities.

M-22-18, Memorandum on Enhancing the Security of the Software Supply Chain through Secure Software Development Practices (Office of Management and Budget, September 14, 2022)

The memorandum requires federal agencies to follow the NIST guidance on software supply chain security and the SSDF. The memorandum outlines specific actions and responsibilities of agencies to certify and evidence that the software complies with secure development practices. Namely, they will be required to obtain from software producers (1) a self-attestation and (2) “artifacts that demonstrate conformance to secure software development practices, as needed.” While these mandates are mainly applicable to third-party software programs, agency-developed software is encouraged to comply with the NIST guidelines and Executive Order 14028.

M-23-18, Memorandum on the Administration Cybersecurity Priorities for the FY 2025 Budget (Office of Management and Budget, June 27, 2023)

This memorandum describes the cross-agency cybersecurity investment priorities for fiscal year 2025. Following the direction set in Executive Order 14028 and a national security memorandum published on Jan. 19, 2022 (NSM-8), “[a]gency investments should lead to durable, long-term solutions that are secure by design” (emphasis added). Accordingly, a key element in the modernization of federal defenses is advancing zero trust cybersecurity principles. For federally funded programs, the budget submissions should show that the agency is supporting projects that are “designed, developed, fielded, and maintained with cybersecurity resilience in mind.”

Shifting the Balance of Cybersecurity Risk

Easterly, Jen, & Goldstein, Eric, Stop Passing the Buck on Cybersecurity (Foreign Affairs, February 1, 2023)

Penned by top CISA officials, this article offers a starting point to the public discourse on shifting the balance of cybersecurity risk. To solve a culture that accepts unsafe and indefensible software products, technology providers must offer products that are “secure by default” and “secure by design.” Modifying the development cycle to incorporate these principles will place the “burden of security” on the software vendors. The article defines these concepts as follows:

• Secure by design: The expectation that technology is purposely designed, built, tested, and maintained to significantly reduce the number of exploitable flaws before it is introduced to the market for broad use.
• Secure by default: Products have strong security features at the time of purchase, without additional costs.

The Cost of Unsafe Technology and What We Can Do About It (CISA, March 10, 2023)

This post summarizes the speech on technology product safety by CISA Director Jen Easterly at Carnegie Mellon University. It outlines three principles for manufacturers to build product safety: (1) taking ownership of security outcomes, (2) radical transparency to better understand the scope of consumer safety challenges, and (3) explicit focus on building safe products.

National Cybersecurity Strategy (March 2023)

One of the stated objectives of the Biden administration’s National Cybersecurity Strategy is to shift liability for insecure software products and services away from the end user and onto the vendors that fail to reasonably secure their software. Per the strategy’s text, legislation to that effect will need to be developed, and the administration will also drive “development of an adaptable safe harbor framework to shield from liability companies that securely develop and maintain their software products and services.” This safe harbor will draw from “best practices for secure software development.” 

The strategy outlines a need for the federal government to use grants and other incentives to “drive investment in critical products and services that are secure- and resilient-by-design, and sustain and incentivize security and resilience throughout the life cycle of critical infrastructure.” Security-by-design principles play a role in the administration’s international strategy: “Through these and other partnerships, the United States and international counterparts can advance common cybersecurity interests by sharing cyber threat information, exchanging model cybersecurity practices, comparing sector-specific expertise, driving secure-by-design principles, and coordinating policy and incident response activities.”

Shifting the Balance of Cybersecurity Risk: Principles and Approaches for Security-by-Design and -Default (CISA, April 13, 2023) + Update (October 2023)

This report builds on the sentiment articulated in the previous documents and encourages software manufacturers to “revamp their design and development programs to permit only Secure-by-Design and -Default products to be shipped to customers.” The guidance is subscribed to by CISA, the National Security Agency, and the FBI along with cybersecurity agencies from Australia, Canada, the United Kingdom, Germany, the Netherlands, and New Zealand.

There are three security-by-design principles: (1) the burden of security should not be exclusively on the customer, (2) need for radical transparency and accountability, and (3) building organizational structure and leadership focused on security. The report offers operational tactics and development practices that can help organizations enhance their secure development process (these reference SSDF practices).

In October, the authoring agencies released an update to the report, in response to feedback from interested individuals, companies, and trade associations. Primarily, the update expands on how the three principles apply to different software manufactures and their customers. The original agencies are joined by the cybersecurity authorities from Norway, Korea, Israel, Japan, the Organization of American States, Singapore, and the Czech Republic.

The definitions of “secure-by-design” and “secure-by-default” shifted slightly compared to the ones provided by Easterly and Goldstein’s op-ed in February 2023.

• Secure-by-Design: Technology products are built in a way that reasonably protects against malicious cyber actors successfully gaining access to devices, data, and connected infrastructure.

• Secure-by-Default: Products are resilient against prevalent exploitation techniques out of the box without additional charge.

National Cybersecurity Strategy Implementation Plan (July 2023)

In accordance with the pillars outlined in the cybersecurity strategy, the implementation plan assigns responsibility for most of the originally listed initiatives to different agencies, and outlines how and when they must be completed. For example, the Office of the National Cyber Director will establish an initiative to promote secure open-source software and adoption of memory-safe programming languages. CISA will culminate public and private relationships to bolster adoption of secure-by-design and secure-by-default practices with manufacturers, nonprofit organizations, academics, and open-source software platforms; and the Department of Energy is required to factor secure- and resilient-by-design practices into upcoming federal projects.

CISA Announces Secure by Design Alert Series: How Vendor Decisions Can Reduce Harm at a Global Scale (CISA, November 29, 2023) 

Advancing security by design requires not just looking at software development practices but also identifying recurring classes of defects for software manufacturers to address. This process includes software manufacturers performing a root cause analysis and making systemic changes to eliminate vulnerabilities of that type. “Such recurring patterns should lead to improvements in the product that make secure settings the default, not stronger advice to consumers in ‘hardening guides.’” CISA’s new alert series will focus on this issue set and how vendors can reduce harm at the global scale by tackling classes of vulnerabilities. For this first publication, CISA draws attention to malicious cyber activity against web management interfaces and how manufacturers should proactively embrace the security-by-design principles of (1) take ownership of customer security outcomes and (2) embrace radical transparency and accountability.

Secure Software Development Best Practices

This section offers an array of secure development practices recommended by different organizations. As the NIST framework highlights, each product’s needs will drive the adoption of some or all tools to satisfy requirements for secure-by-design software. While the selection of resources is not comprehensive, the goal is to provide a sampling of perspectives on the different approaches—and their similarities—that have been proposed for organizations and developers to improve software security. The sampling of sources further underscores that different approaches vary in their sophistication, reach, and definitions and conceptions of the same terms.

Saltzer, Jerome H., & Schroeder, Michael D., The Protection of Information in Computer Systems (1975)

In the absence of methodical techniques that can “systematically exclude flaws,” Saltzer and Schroeder identify “useful principles that can guide the design and contribute to an implementation without security flaws.” These principles include (1) economy of mechanism, (2) fail-safe defaults, (3) complete mediation, (4) open design, (5) separation of privilege, (6) least privilege, (7) least privilege, (8) least common mechanism, and (9) psychological acceptability.

Fundamental Practices for Secure Software Development (SAFECode, March 2018)

This guide focuses on “practiced practices” that make up a “foundational set of secure development practices that have been effective in improving software security in real-world implementations.” When it comes to design, it refers to Saltzer and Schoeder but adds three more key principles: (1) defense-in-depth, (2) fail securely, and (3) design for updating.

The BSA Framework for Secure Software (BSA, September 2020)

The BSA Framework attempts to consolidate “best practices in a manner that can be effectively measured, regardless of the development environment or the purpose of the software.” It identifies organization processes and product capabilities at different stages of the software life cycle. The framework is organized under six headings: Functions, Categories, Subcategories, Diagnostic Statements, Implementation Notes, and Informative References. The five guiding principles are (1) risk-based, (2) outcome-focused, (3) flexible, (4) adaptable, and (5) aligned with internationally recognized standards.

Kelly, Judy, & Sastre, David, Security by Design: Security Principles and Threat Modeling (Red Hat, February 20, 2023)

Because there is “no complete checklist” that one can follow to guarantee product security, security-by-design principles guide threat modeling—which is used to improve product and network cybersecurity. RedHat draws on the OWASP Development Guide for inspiration. The 10 security-by-design principles are (1) defense in depth, (2) secure by default, (3) least privilege, (4) separation of duties, (5) minimize attack surface, (6) complete mediation, (7) open design, (8) isolated compartment, (9) evidence production, and (10) application of coding best practices. Each principle corresponds to different combinations of process, technical control, and other approaches. For instance, “secure by default” includes denying access to systems by default through a least privilege model, designing services to fail securely (“meaning that when they fail, they fail ‘closed’”), and using allowlists as the baseline for authorized entities, rather than denylists. Minimizing the attack surface includes keeping designs as simple and small as possible, favoring minimum installs, and emphasizing economy and frugality in design to simplify analysis and reduce errors and oversights.

Best Practices for OSS Developers Working Group, Concise Guide for Developing More Secure Software (OpenSSF, June 14, 2023)

Developed by the OpenSSF Best Practices Working Group, an open group dedicated to advancing open-source security education and best practices, this guide focuses on secure software development, building, and distribution: (1) ensure all privileged developers use multi-factor authentication (MFA) tokens, (2) learn about secure software development, (3) use a combination of tools in your CI pipeline to detect vulnerabilities; (4) evaluate software before selecting it as a direct dependency, (5) use package managers, (6) implement automated tests, (7) monitor known vulnerabilities in your software’s direct & indirect dependencies, (8) keep dependencies reasonably up-to-date, (9) do not push secrets to a repository, (10) review before accepting changes, (11) prominently document how to report vulnerabilities & prepare for them, (12) make it easy for your users to update, (13) sign your project’s important releases, (14) earn an OpenSSF Best Practices badge for your open-source project, (15) improve your OpenSSF Scorecards score, (16) notify the community of vulnerabilities in your project, (17) improve your Supply chain Levels for Software Artifacts (SLSA) level, (18) publish and consume a SBOM, (19) onboard your project into LFX Security if you manage a Linux Foundation project, (20) apply the CNCF Security TAG Software Supply Chain Best Practices guide, (21) implement ASVS and follow relevant cheatsheets, (22) apply SAFECode’s Fundamental Practices for Secure Software Development, (23) complete a third-party security code review/audit, (24) continuously improve, (25) manage succession, and (26) prefer memory-safe languages. 

Australian Cyber Security Centre, IoT Secure-by-Design Guidance for Manufacturers (Updated September 21, 2023)

This guidance was originally developed in support of the Department of Home Affairs’s Securing the Internet of Things code of practice. The 13 secure-by-design principles are (1) no duplicated default or weak passwords, (2) implement a vulnerability disclosure policy, (3) keep software securely updated, (4) securely store credentials, (5) ensure that personal data is protected, (6) minimize exposed attack surfaces, (7) ensure communication security, (8) ensure software integrity, (9) make systems resilient to outages, (10) monitor system telemetry data, (11) make it easy for consumers to delete personal data, (12) make installation and maintenance of devices easy, and (13) validate input data.