Low-Level Software Security: Exploiting Memory Safety Vulnerabilities and Assumptions

By Stephen Checkoway.

A dissertation submitted in partial satisfaction of the requirements for the degree of Doctor of Philosophy in Computer Science at UC San Diego.

Abstract

The security of computer systems depends in a fundamental way on the validity of assumptions made by the systems’ designers. Assumptions made about attacker capabilities have a tendency to turn out false and many computer systems are insecure as a direct consequence. This is especially true with memory-safety vulnerabilities whereby an attacker is able to violate the memory-safety guarantees of a software system. Here, system designers have assumed that defenses against code injection or certain other forms of data corruption are sufficient to stop a determined attacker.

In this dissertation, I will examine several instances where a system’s designer incorrectly assumed that an ad hoc defense against attackers was sufficient to defend the system. First, I show how to defeat the Sequoia AVC Advantage voting machine’s hardware defense against code injection. To that end, I construct a proof-of-concept, vote-stealing program by extending return-oriented programming to the Z80. Next, I show that several proposed defenses against return-oriented programming attacks are insufficient by demonstrating Turing-complete, return-oriented programming without returns on the x86. Finally, I turn to systems that attempt to prevent a malicious operating system kernel from interfering with the execution of a protected application. To do so, I introduce Iago attacks: attacks a malicious kernel can mount to subvert the execution of the protected program.

Material

• Dissertation in PDF.

Reference

@PhDThesis{checkoway:dissertation:2012,
  title =  {Low-Level Software Security: Exploiting Memory Safety
            Vulnerabilities and Assumptions},
  author = {Stephen Checkoway},
  school = {University of California, San Diego},
  year =   2012,
  month =  jun,
  url =    {https://checkoway.net/papers/dissertation},
}