Provably Awesome.

This is a curated list of links and resources related to mathematical proofs about the properties of computer programs.

Table of Contents

• Languages - Languages with good ability to use formal type safety
• Proof Assistants - Interactive theorem provers used to prove properties of programs.
• Projects - Projects involving provably correct code.
• Books - Textbooks commonly referred to
• Courses - Online courses (youtube, university courses)
• More Links - Video presentations about formal proof of code topics

Languages

• Idris : Idris is a general purpose pure functional programming language with dependent types. Dependent types allow types to be predicated on values, meaning that some aspects of a program’s behaviour can be specified precisely in the type. It is compiled, with eager evaluation. Its features are influenced by Haskell and ML.

  • Idris docs
  • Idris tutorial
  • Theorem proving with Idris tutorial

• Agda : Dependently typed functional programming language. It has inductive families, i.e., data types which depend on values, such as the type of vectors of a given length. It also has parametrised modules, mixfix operators, Unicode characters, and an interactive Emacs interface which can assist the programmer in writing the program.

  • Agda Github
  • Agda User Manual

• UR/Web : Ur/Web is Ur plus a special standard library and associated rules for parsing and optimization. Ur/Web supports construction of dynamic web applications backed by SQL databases. The signature of the standard library is such that well-typed Ur/Web programs “don’t go wrong” in a very broad sense. Not only do they not crash during particular page generations, but they also may not:

  • Suffer from any kinds of code-injection attacks
  • Return invalid HTML
  • Contain dead intra-application links
  • Have mismatches between HTML forms and the fields expected by their handlers
  • Include client-side code that makes incorrect assumptions about the “AJAX”-style services that the remote web server provides
  • Attempt invalid SQL queries
  • Use improper marshaling or unmarshaling in communication with SQL databases or between browsers and web servers

• Haskell : An advanced, purely functional programming language.

• Liquid Haskell : LiquidHaskell (LH) refines Haskell’s types with logical predicates that let you enforce critical properties at compile time. (Guarantee functions are total, keep pointers within bounds, avoid infinite loops, enforce correctness properties, prove laws by writing code)

• Elm : A type-safe functional programming language for declaratively creating web browser-based graphical user interfaces. Implemented in Haskell. It generates JavaScript with great performance and no runtime exceptions.

Proof Assistants

• Coq : Coq is a formal proof management system. It provides a formal language to write mathematical definitions, executable algorithms and theorems together with an environment for semi-interactive development of machine-checked proofs. [current stable version] [reference manual]
  • https://github.com/Ptival/PeaCoq
  • https://math-comp.github.io/mcb/
• Isabelle : Isabelle is a generic proof assistant. It allows mathematical formulas to be expressed in a formal language and provides tools for proving those formulas in a logical calculus. [overview]
• HOL : The HOL interactive theorem prover is a proof assistant for higher-order logic: a programming environment in which theorems can be proved and proof tools implemented. Built-in decision procedures and theorem provers can automatically establish many simple theorems (users may have to prove the hard theorems themselves!) An oracle mechanism gives access to external programs such as SMT and BDD engines. HOL is particularly suitable as a platform for implementing combinations of deduction, execution and property checking. [Other HOLS]
• LEAN : Lean is a new open source theorem prover being developed at Microsoft Research, and its standard library at Carnegie Mellon University. Lean aims to bridge the gap between interactive and automated theorem proving, by situating automated tools and methods in a framework that supports user interaction and the construction of fully specified axiomatic proofs. The goal is to support both mathematical reasoning and reasoning about complex systems, and to verify claims in both domains. Online version
• K Framework : The K framework is a rewrite-based executable semantic framework in which programming languages, type systems and formal analysis tools can be defined using configurations, computations and rules. Configurations organize the state in units called cells, which are labeled and can be nested. Computations carry computational meaning as special nested list structures sequentializing computational tasks, such as fragments of program. Computations extend the original language abstract syntax. K (rewrite) rules make it explicit which parts of the term they read-only, write-only, read-write, or do not care about. This makes K suitable for defining truly concurrent languages even in the presence of sharing. Computations are like any other terms in a rewriting environment: they can be matched, moved from one place to another, modified, or deleted. This makes K suitable for defining control-intensive features such as abrupt termination, exceptions or call/cc.
  • K Tutorial by Grigore Rosu, video playlist
  • (https://runtimeverification.com/) : Company formed from K Framework people. Runtime Verification Inc. is currently developing three core products:
    • RV-Predict is a predictive runtime analysis tool focused on automatically detecting concurrency errors in your programs.
    • RV-Monitor is a development methodology and library generation tool allowing for the monitoring and enforcement of user-selected properties at runtime.
    • RV-Match is a tool allowing for exhaustive runtime verification to be performed symbolically on all possible program paths, proving certain properties correct for all possible executions of a given program.
• Viper : Viper (Verification Infrastructure for Permission-based Reasoning) is a language and suite of tools developed at ETH Zurich, providing an architecture on which new verification tools and prototypes can be developed simply and quickly. It comprises a novel intermediate verification language, also named Viper, and automatic verifiers for the language, as well as example front-end tools. The Viper toolset can be used to implement verification techniques for front-end programming languages via translations into the Viper language. ETH Zurich has built several verifiers on top of Viper, including the Gobra verifier for Go, Nagini for Python and Prusti for Rust.

Projects

• seL4 : The world’s first operating-system kernel with an end-to-end proof of implementation correctness and security enforcement is available as open source. [brochure] [white paper]
• Certikos : Certified Kit Operating System.
  • Certified OS Kernels: Clean-slate design with end-to-end guarantees on extensibility, security, and resilience. Without Zero-Day Kernel Vulnerabilities.
  • Layered Approach: Divides a complex system into multiple certified abstraction layers, which are deep specifications of their underlying implementations.
  • Languages and Tools: New formal methods, languages, compilers and other tools for developing, checking, and automating specs and proofs.
• Compcert : The CompCert project investigates the formal verification of realistic compilers usable for critical embedded software. Such verified compilers come with a mathematical, machine-checked proof that the generated executable code behaves exactly as prescribed by the semantics of the source program. [C compiler]
• Bedrock [tutorial pdf] : Bedrock is a library that turns Coq into a tool much like classical verification systems (e.g., ESC, Boogie), but niftier. In particular, Bedrock is:
  • Low-level: You can verify programs that, for performance reasons or otherwise, can’t tolerate any abstraction beyond that associated with assembly language.
  • Foundational: The output of a Bedrock verification is a theorem whose statement depends only on the predicates you choose to use in the key specifications and on the operational semantics of a simple cross-platform machine language. That is, you don’t need to trust that the verification framework is bug-free; rather, you need to trust the usual Coq proof-checker and the formalization of the machine language semantics.
  • Higher-order: Bedrock facilitates quite pleasant reasoning about code pointers as data.
  • Computational: Many useful functions are specified most effectively by comparing with “reference implementations” in a pure functional language. Bedrock supports that model, backed by the full expressive power of Coq’s usual programming language.
  • Structured: Bedrock is an extensible programming language: any client program may add new control flow constructs by providing their proof rules. For example, adding high-level syntax for your own calling convention or exception handling construct is relatively straightforward and does not require tweaking the core library code.
  • Mostly automated: Tactics automate verification condition generation (in a form inspired by separation logic) and most of the process of discharging those conditions. Many interesting programs can be verified with zero manual proof effort, in stark contrast to most Coq developments.

• HACMS

  High-Assurance Cyber Military Systems (HACMS) . Dr. Raymond Richards. Technology for cyber-physical systems, functionally correct and satisfying appropriate safety and security properties. Clean-slate, formal methods, semi-automated code synthesis from executable, formal specifications. HACMS seeks a synthesizer capable of producing a machine-checkable proof that generated code satisfies functional specs as well as security and safety policies, and development to ensure proofs are composable, allowing components. [more Darpa “formal” tagged links] [verigames-crowdsourced formal verification]

• Genode : Genode is a novel OS architecture that is able to master complexity by applying a strict organizational structure to all software components including device drivers, system services, and applications.

Books

• The Little Prover The Little Prover introduces inductive proofs as a way to determine facts about computer programs.

• Certified Programming with Dependent Types by Adam Chlipala. Textbook about practical engineering with the Coq proof assistant. The focus is on building programs with proofs of correctness, using dependent types and scripted proof automation. [Latest draft]

• Software Foundations by Benjamin Pierce and others. The Software Foundations series is a broad introduction to the mathematical underpinnings of reliable software.

  • Vol. 1: [read] [download] Logical Foundations, serves as the entry-point to the series. It covers functional programming, basic concepts of logic, computer-assisted theorem proving,and Coq.

  • Vol. 2: [read] [download] Programming Language Foundations, surveys the theory of programming languages, including operational semantics, Hoare logic, and static type systems.

  • Vol. 3: Verified Functional Algorithms [read] [download] Learn how to specify and verify (prove the correctness of) sorting algorithms, binary search trees, balanced binary search trees, and priority queues.

• HoTT : Homotopy Type Theory: Univalent Foundations of Mathematics [pdf] Foundations of Mathematics is Vladimir Voevodsky’s new program for a comprehensive, computational foundation for mathematics based on the homotopical interpretation of type theory. The type theoretic univalence axiom relates propositional equality on the universe with homotopy equivalence of small types. The program is currently being implemented with the help of the automated proof assistant Coq.

• https://math-comp.github.io/mcb/

Courses

• DeepSpec Summer School 41 Videos about deep specification. Coq videos, examples, tutorials.

• Adam Chlipala Videos:

  • 2017-12-29 CCC Presentation: Coming Soon Machine-Checked Mathematical Proofs in Everyday Software and Hardware Development

  • Adam Chlipala Lecture 1, OPLSS 2015

  • Bedrock: A Software Development Ecosystem Inside a Proof Assistant

  • CACM August 2016 - Ur/Web: A Simple Model for Programming the Web

  • Proof engineering Adam Chlipala

  • 2015 Coq Proof Assistant and Its Applications to Programming-Language Semantics

• Type-Drive Development in Idris - Edwin Brady

• Benjamin Pierce - Software Foundations Course

• Learning Automated Theorem Proving : Stackexchange post about learning

More

• Curry-Howard : Curry–Howard correspondence refers to the direct relationship between computer programs and mathematical proofs. This is also the idea of “proofs as programs”, and “propositions (formulas)-as-types”.

• Hoare logic Hoare logic (also known as Floyd–Hoare logic or Hoare rules) is a formal system with a set of logical rules for reasoning rigorously about the correctness of computer programs.

• Designing A Theorem Prover (Paulson, Cambridge, 1990)

• Rolf Rolles Program Synthesis in Reverse Engineering …Assume you generate all possible programs…

• The Open-Source seL4 Kernel. Military-Grade Security Through Mathematics - SFO17-417

• DARPA Hack Proof Drones Uses SEL4, other RTOS

• Pentagon Wants Unhackable Helicopters

• Hacker-Proof Code Confirmed : Computer scientists can prove certain programs to be error-free with the same certainty that mathematicians prove theorems. The advances are being used to secure everything from unmanned drones to the internet.

• CertiKOS enables creation of secure system kernels [certikos project] Computer system security stinks, because our software is buggy and untestable in full. Great for cyber criminals, but not for us. So why doesn’t someone build a mathematically verified, secure, concurrent kernel that can run on x86 and ARM? A team at Yale has.

• seL4 Is Free – What Does This Mean For You?

• From L3 to seL4 what have we learnt in 20 years of L4 microkernels?

• seL4 introduction: Capability–based Access Model <- Chinese, translation?

• seL4 playlist

• Creating drones that can’t be hacked

• HACMS: Protecting Military Systems from Hackers

• Formal Methods for Avionics Software Verification pt1

• TODO: find Heartbleed Example with Agda Showing Proofs