A leap forward in software security assurance

AI is Helping Prove Software is Secure

AI is making a big step forward in software assurance. It can now do more than just helping with testing.

A group recently published a paper AlphaProof Code: Verifying Signal’s Security at the Machine-Code Level. The paper shows how AI can now create formal proofs that software works as intended.

Below is a brief, non-AI expert summary.

The Problem

Signal (and apps that use its protocol, such as WhatsApp and Google Messages) protects the private messages of billions of people. It does so with end-to-end encryption. The core library (libsignal) providing encryption has advanced features. These include a post-quantum handshake (PQXDH) and the “Triple Ratchet.” The Triple Ratchet provides forward secrecy and can help recovery after a device compromise.

The gold standard for assurance is formal verification. Formal verification is a mathematical proof that the software does exactly as intended. In this case, that it fits its security specification claims with no hidden bugs. In the past this type of verification has been very expensive.

Prior projects like the verified operating system seL4 took years of expert work. However, those results were usually achieved on abstract models or rewritten code. The paper reports results on real, unmodified production software.

What the Team Did

They built AlphaProof Code, a multi-agent AI framework that works with humans:

  • Humans (with AI help) define what should be proved. This includes the security properties and the formal model of the processor.
  • Teams of AI agents then generate the detailed proofs.
  • Every proof is checked by the Lean theorem prover’s kernel. The AI cannot simply claim success. The mathematics must check out.

They applied this to the real, unmodified libsignal code. The code was compiled with ordinary tools into AArch64 (ARM 64-bit) machine code. The AI agents produced roughly 6.5 million lines of machine-checked proof. This rests on about 26,000 lines of human-reviewed specifications and processor semantics. The whole effort took a small team only about four weeks.

Key Results

The proofs show, for one-to-one sessions, that the software met important security properties:

  • Forward secrecy (past messages stay protected even if keys are later compromised)
  • Authentication
  • Post-compromise security (the system can “heal” after a temporary compromise)
  • Handshake secrecy

They proved this with two complementary mathematical models. One was a symbolic “ideal crypto” model. The other was a computational model with concrete hardness assumptions. They also deliberately re-introduced two previously fixed bugs from real libsignal history. They showed that the security theorems fail on those buggy versions. This provided evidence that the specifications are strong enough to catch real problems.

This is the first time anyone has produced validated, machine-checked proofs.

What It Means for a Leader

  • Higher practical assurance. Signal’s CTO noted that this expands formal verification far beyond previous limited efforts. It can increase confidence in provider privacy guarantees.
  • Proof that AI can make formal verification tractable for real production code. The bottleneck shifts from “writing millions of lines of proof by hand” to “writing and reviewing the right specifications.”
  • A path toward verifying other critical systems (communications infrastructure, medical devices, power grids, etc.). The greatest returns will be where the cost of a flaw is high.

Important caveats to note:

  • The proofs cover a defined scope (one-to-one sessions, certain assumptions about underlying crypto primitives and the processor model).
  • Some external functions are “hooked” under contracts rather than fully proved.
  • The computational proofs do not cover quantum attackers. The trusted base (what must still be assumed correct) is substantially smaller than “trust the entire compiler and runtime,” but it is not zero.

In short, this is a concrete demonstration that AI-assisted formal methods can verify security claims. The results come at a cost and on a timeline that are practical for industry. The proofs and tools are open source for independent review.

Implications Beyond Implementing Formal Proof Methods

Proof generation may no longer the scarce resource. The scarcer, high-value work is now defining the right security specification. “Right” means precise, strong enough to be useful, not vacuous, and small enough for review. Then it must be rigorously reviewed.

Implications of the Shift
  • Assurance quality is now gated by the quality of specifications, not by the size of the proof.
  • Errors or omissions in the specification become the dominant risk. A weak or incomplete spec can produce a valid proof of the wrong thing.
  • The required human expertise is no longer ability to write large formal proofs. It is now ability to state security properties clearly and judge whether a formalization captures the real intent.
  • Formal verification becomes more practical, but only if produce and maintain high-quality specifications.
What to Do to Address the Coming Bottleneck

1.     Make security specification writing a first-class engineering process.

Use specialized AI agents to draft and critique specifications. However, keep final approval with humans who understand the threat model.

2.     Validate specifications aggressively and independently of the proofs.

  • Re-introduce known historical bugs and confirm the theorems fail.
  • Run mutation testing / bug-injection campaigns.
  • Check for vacuity (theorems that hold for trivial or unintended reasons).
  • Require independent reviewer agents or humans to attack the specification before finalized.

3.     Build and reuse libraries of trusted specifications.

Invest in reviewed formalizations of common cryptographic primitives, protocol patterns, and machine models. Every project that starts from scratch multiplies the bottleneck and creates unnecessary risk.

4.     Keep the trusted base small and auditable.

Prefer binary-level or low-level verification when it reduces what must be trusted. Hook external code under clear contracts rather than expanding the trusted surface. Document exactly what remains assumed.

5.     Create a lightweight but mandatory human review loop for any change to the specification.

Allow agents to propose amendments when proofs reveal problems. Treat any issues as a bug that requires explicit justification and review. Freeze a reviewed specification before launching large proof campaigns.

6.     Develop the new skill set.

Train (or hire) people who can operate as “specification architects” … domain experts who can read formal statements and decide if the specification matches security goals. Full manual proof expertise is less critical than it used to be.

In summary: start optimizing the specification lifecycle (drafting, stress-testing, reviewing, versioning, and reusing). That is how to turn AI-assisted formal verification into reliable, repeatable security assurance.

About the author

Mike Russell helps organizations succeed without compromising speed, quality, or regulatory compliance. His 40+ years of leadership and advisory experience ranges from startups to Fortune 20 companies and national governments. He expertise includes product and software development, cybersecurity, AI, and leadership. Mike served on the AAMI group that created TIR45, Guidance on the use of Agile practices in the development of medical device software. He is also an AAMI expert faculty member for software validation (product and non-product) and agile. For SoftwareCPR, he co-created and delivers training on compliant agile and medical device cybersecurity. As an executive, Mike led organizations of up to 1,100 people with over $100 million budgets.

Upcoming Public Courses

No public courses are planned at this time.

If you would like to discuss a private, onsite course, please fill out the form below.

Or just email training@softwarecpr.com for more info.

Corporate Office

15148 Springview St.
Tampa, FL 33624
USA
+1-781-721-2921
Partners located in the US (CA, FL, MA, MN, TX) and Canada.