MARPLE: Mitigating APT - Northwestern University

Detecting Missing RAT Attacks with Semantics on Windows Xutong Chen and Yan Chen Northwestern University Outline Overview (APT and RAT) Motivation (comparison with 5D) Keylogging case study Overall flowchart graphs (already have) System Design: API + discriminative sigs Evaluation Keylogging for accuracy: API only, API+discri results Overhead: speed and memory/CPU consumption 2 Overview Initial Initial Compromise

Compromise Gaining Gaining Foothold Foothold Lateral Lateral Movement Movement High High Value Value Asset Asset Acquisition Acquisition Malware (e.g. RAT) Phishing Attacker

Malicious Web ork Netw Exploit vulnerability Victim scan Malw a prop re agat ion Code Repo

Database Exploit browser Behavior Behavior based based Malware Malware detection detection Design a detection mechanism that targets at the post-compromise steps in the APT life-cycle 3 Overview Remote Access Trojan (RAT)

Based on the study of 300+ APT whitepapers, RAT is a core component in an APT attack, and >90% are Windows based. Allows an adversary to remotely control a system A complex set of potentially harmful functions (PHFs) E.g., keylogger, remote desktop, remote shell, audio grab A Windows RAT typically embodies10~40 PHFs. 4

Motivation Observations Ways of implementing a PHF are limited too, in terms of Core APIs & security-relevant events , and such sequences to exactly define a PHF Propose PHF-based RAT detection when a program exhibits A sufficient number of PHFs RAT-specific resource access characteristics Advantages: evasion-resilient and semantic-aware

Hard to evade unless attackers find new ways of implementing PHFs Know exactly what activity is going on 5 Motivation Keylogging Case Study Process of Keylogging Data NU Data API 5D Data Capture keystrokes

GetKeyState GetKeyboardState GetKeyboardLayout ToUnicodeEx Capture foreground window information GetForegroundWindow GetWindowTextW GetWindowThreadProcessId Missing Save results to file WriteFile

WriteFile Missing 6 Motivation Deficiency of Traditional Malware Detection Even if traditional detection tool captures all artifacts left by the attack, administrators still cannot understand consequences of the attack. 7 Motivation Advantage of Our Method With our semantic-aware detection system, administrators can easily identify fine-grained semantic behaviors and understand consequences of the attack. 8

System Design 9 System Design Discriminative Features Discriminative features is used to reduce False Positive, we propose several Discriminative features as follows: API Types: Assume that a sequence matches a behavior graph, we consider the number of unique APIs, except APIs included in the behavior graph, occurring in the same sequence. User Interaction: When a process matches a behavior graph, we judge whether there are user interactions on this process.

Network Pattern: The frequency and the payload size of network traffic. 10 System Design Discriminative Feature Example on Keylogging Y axis Normal ized # of API Types Point with other colors is Benign Red point is RAT X axis : Graph Matching Score 11

Evaluation 12 Evaluation Deploy the system on three physical machines, which last three days. TPR(API only) TPR(with discriminativ e features) FPR(API only) FPR(with discriminativ e features)

Keylogging 89.9% 89.9% 3.2% 0.05% RemoteDesktop 84.6% WIP 4% WIP RemoteShell

84.6% WIP 1.5% WIP Download&Exec % WIP 2.3% WIP 13

Evaluation The throughput and the overhead of the detection system The average # of APIs generated per host is 6K-10K APIs/sec, after filtering The average matching speed of all API behavior graphs per core is 38K APIs /sec and the memory consumption is 7GB (without optimization) 14 Conclusions 15 Evaluation To test the space overhead of out collector, we deploy our collector in two individuals' computer for several days. One is an officer who frequently uses text editing applications (Word, Excel,

PowerPoint), communication tools (Outlook and Skype) and Browsers (Chrome and Firefox). The other is a software developer who often uses IDE (Visual Studio), Browsers (Chrome and Firefox) and command line tools (CMD and PowerShell). Top-layer API Original Data Size 14 GB / hour Disk size after compression 2 MB / hour 16