Iago Attacks: Why The System Call API Is a Bad Untrusted RPC Interface

By Stephen Checkoway and Hovav Shacham.

In Proceedings of ASPLOS 2013. ACM Press, March 2013.


In recent years, researchers have proposed systems for running trusted code on an untrusted operating system. Protection mechanisms deployed by such systems keep a malicious kernel from directly manipulating a trusted application’s state. Under such systems, the application and kernel are, conceptually, peers, and the system call API defines an RPC interface between them.

We introduce Iago attacks, attacks that a malicious kernel can mount in this model. We show how a carefully chosen sequence of integer return values to Linux system calls can lead a supposedly protected process to act against its interests, and even to undertake arbitrary computation at the malicious kernel’s behest.

Iago attacks are evidence that protecting applications from malicious kernels is more difficult than previously realized.

This work is based on an older technical report.



  author =    {Stephen Checkoway and Hovav Shacham},
  title =     {Iago Attacks: Why the System Call {API} is a Bad Untrusted {RPC} Interface},
  booktitle = {Proceedings of ASPLOS 2013},
  editor =    {Rastislav Bodik},
  publisher = {ACM Press},
  year =      2013,
  month =     mar,
  url =       {https://checkoway.net/papers/iago2013},