How Anti Malware Works
Anti Malware software works in two broad categories: 1a) classification, and 2b) clustering.
…uses two broad techniques: signature and behavior
Signature: code sets that are in a database and the anti malware just references a code set against fields within a packet. This makes it fast, but relies on existing definitions within the software. It won’t stop zero day attacks.
Behavior: looking at the processes, files, registry records, and network events, and running the actual malware and seeing what it does in a confined environment
Why Classification Is Important?
Because you CAN’T analyze all the malware by hiring analysts! You need some way to model the data. So you ‘hand analyze’ a small set and then extrapolate that into a larger set
…groups similar behavior together into samples and helps the anti malware scale
The ability to automatically and effectively cluster analyzed malware samples into families with similar characteristics is beneficial for the following reasons: First, every time a new malware sample is found in the wild, an analyst can quickly determine whether it is a new malware instance or a variant of a well-known family. Moreover, given sets of malware samples that belong to different malware families,
it becomes significantly easier to derive generalized signatures, implement removal procedures, and create new mitigation strategies that work for a whole class of programs. Source Here
vs Classification: to fight against the signatures attackers use polymorphic obfuscation, packing, and code rearranging (Sharif et. all., 2009)
Polymorphic Malware: malware has an embedded encryption engine and uses that with a random key to generate a new instance of itself. The content of the new instance is quite different from its original version. The content is decrypted in execution time by a decryption engine. Source Here
Metamorphic Malware: is able to generate an instance of itself with a different internal structure and probably disparate code. Moreover, since a signature-based detector has to use a separate signature for each malware variant, the database of signatures also grows at an exponential rate. Source Here
Classification: leveraging behavior analysis because this technique doesn’t care about the underlying code but cares about the behavior it exhibits.
Security Art – Malware
We have 3 types of malware coming through the Internet here: 1) metamorphic, 2) boring ol’ malware, 3) polymorphic. The signature captures the boring ol’ but can’t do much for the metamorphic. The behavior catches the metamorphic malware, and then lastly the trickiest of all: polymorphic – signature and behavior work together to capture polymorphic.
The clustering is collecting all the malware and creating samples. The malware engine will leverage these samples to deny malware because it can’t infinitely scale its signature engine.