UC Irvine

Remote code execution (RCE) exploits are considered among the more destructive intrusions in today’s software security landscape because they allow the adversary to execute arbitrary code with the permissions of the victim process. Today’s leading software defense systems are able to eliminate significant portions of this attack surface, but many of these approaches such as control flow integrity (CFI) and intrusion detection systems (IDS) address a limited scope of attack vectors, while others like diversification have not received wide user adoption. Consequently, user applications remain vulnerable to RCE attacks. One limitation that is common among these approaches is an overcommitment to preventing the adversary’s takeover strategy. Since the domain of potential attack vectors may be infinite, the adversary may always be able to devise a new takeover strategy that cannot be detected or prevented by any of the existing takeover-oriented defenses. After an attack, current monitoring systems often do not provide sufficient information about how the adversary took control or what vulnerability was compromised. Forensic tools can derive this information from an instance of the attack, but it may take multiple days or even weeks to isolate an instance for analysis.

Motivated by the need for a program monitor that is equally effective against unforeseen takeover strategies as well as an exploit payload, Introspective Intrusion Detection (IID) combines the advantages of traditional intrusion detection and CFI by distinguishing anomalies in execution without making absolute judgments about malicious intent. This approach begins with a profiling phase that distills observed execution paths into a compact data representation called a trusted profile that constitutes a “ground truth” of safe application behavior. When deployed for online monitoring, IID reports any deviation from the Trusted Profile as a potential intrusion. Software developers can benefit from IID as a complement to deployed defenses by gaining visibility into unforeseen malicious behaviors or evasive variations of known attacks. Debugging of field failures and error reports can also benefit from the deep introspective logging provided by IID. Where deployed software has ample performance headroom and sufficient technical support, IID can also be deployed as a comprehensive realtime monitor to detect advanced persistent threats (APT) and other evasive intrusions.

Experimental evaluation of IID prototypes for x86 binary executables and PHP applications show that this technique (a) identifies unforeseen takeover strategies along with exploit payloads, (b) accurately distinguishes benign anomalies from those associated with real attacks, (c) incurs low overhead with minimal false positives during normal execution of server applications such as Microsoft IIS and web applications such as WordPress, and (d) incurs moderate overhead for desktop applications such as Microsoft Office, Google Chrome and Adobe PDF Reader with a false positive rate suitable for expert analysis.