Low-Level Software Security: Exploiting Memory Safety Vulnerabilities and Assumptions

By Stephen Checkoway.

A dissertation submitted in partial satisfaction of the requirements for the degree of Doctor of Philosophy in Computer Science at UC San Diego.


The security of computer systems depends in a fundamental way on the validity of assumptions made by the systems’ designers. Assumptions made about attacker capabilities have a tendency to turn out false and many computer systems are insecure as a direct consequence. This is especially true with memory-safety vulnerabilities whereby an attacker is able to violate the memory-safety guarantees of a software system. Here, system designers have assumed that defenses against code injection or certain other forms of data corruption are sufficient to stop a determined attacker.

In this dissertation, I will examine several instances where a system’s designer incorrectly assumed that an ad hoc defense against attackers was sufficient to defend the system. First, I show how to defeat the Sequoia AVC Advantage voting machine’s hardware defense against code injection. To that end, I construct a proof-of-concept, vote-stealing program by extending return-oriented programming to the Z80. Next, I show that several proposed defenses against return-oriented programming attacks are insufficient by demonstrating Turing-complete, return-oriented programming without returns on the x86. Finally, I turn to systems that attempt to prevent a malicious operating system kernel from interfering with the execution of a protected application. To do so, I introduce Iago attacks: attacks a malicious kernel can mount to subvert the execution of the protected program.



	title =		{Low-Level Software Security: Exploiting Memory Safety
			 Vulnerabilities and Assumptions},
	author =	{Stephen Checkoway},
	school =	{University of California, San Diego},
	year =		2012,
	month =		jun,
	url =           {https://checkoway.net/papers/dissertation},