Immunity Debugger PeDetect and the art of signature generation

hello to you all

i,m really sorry for our late  in posting we really working on lots of things … before starting about our subject i should  tell you about our advisories and exploits we are not  really full-disclosure believers but still we will post some more exploits and advisories at  :

http://www.exploit-db.com/author/abysssec

so stay tuned.

OK let’s start  ….

=========================================

before start if you are not familiar with PE  : The Portable Executable (PE) format is a file format for executables, object code, and DLLs, used in 32-bit and 64-bit versions of Windows operating systems. The term “portable” refers to the format’s versatility in numerous environments of operating system software architecture.

for more information  :  http://en.wikipedia.org/wiki/Portable_Executable

- now the first question is what is a signature ?

a signature actually  is what that means but in computer world and more specific in reverse engineering and binary auditing  world a signature is a sequence of  unique instructions (actually their representation op-codes) in target binary.

for better understanding please watch figure 1

figure 1 – a c++ compiled binary opened in immunity debugger

reminiscence : an opcode (operation code) is the portion of a machine language instruction that specifies the operation to be performed.

in above figure it have tree red rectangular :

  • first rectangular are RVA (relative virtual address) of instructions
  • second rectangular are OP-Codes (will be execute)
  • third rectangular are  readable assembly instructions

so we will search for a sequence of unique op-codes (so sequence of instructions)  in our target binary and those byte will be signature of our binary. simple enough eh ?

- what and who need to use a signature ?

  • most of anti-virus (and other anti-things)
  • and almost all of PE Detection tools

so now you can imagine how  an anti-virus company can detect a malware and how  PE-Detection tools  (witch areused for detecting signature in compiled binary and determine compiler / packer / compressor and … )  works .

- next question is why we need care about signatures:

  • before starting any fuzzing / reversing / auditing project we need to about our target binary
  • identify binaries those have not any signatures
  • with them we can speed up our reversing and we can find available tools against our target binary

-how we can find signatures in binaries ?

we should search for static and constant location (static instructions) in our file but how we can find them? for answer to this question please watch PE file layout again :

figure 2 – PE file layout

we can search for signatures in a few areas :

  • around program entry point (where program instructions will start execution …)
  • from offset (from top to bottom)

each executable file have some other locations can be good for generating signature those are :

  • around import table (where functions will be import)
  • start and end of sections (optional section specially)
  • name of optional / static sections
  • ….

so we can just open the executable  under debugger and copy a few OP-Codes from entry point and we are done ? of course not ! because in lots of situations entry point could be change  refer to various factors like :

  • initializing addresses / variables with state of program
  • if we are in fighting against a packer / compressor / cryptor / there are several technologies they can use for hiding / changing instructions …

note : these changes are more on not “just compiled binaries” it means those have a packer / protector and ….

so how we can find reliable signatures ?

we need to research about variant program situations  and then we can understand which bytes/instructions are constant and which are not then we  can ignore dynamic bytes and rely to static bytes.

before a  real case study i just want explain how packer/protectors works :

a packer will do what it sounds : packing a program. think  about winzip it will comperes the program and actually will decrease size of program .

elementary packers just will compress the portable executable and will change entry point to decompression section for better understanding just watch below figure.

figure 3 How typical packer runtime works

1. Original data is located somewhere in the packer code data section
2. Original data is uncompressed to the originally linked location
3. Control is transferred to original code entry point (OEP)

Ok now you know how a basic packer works but today modern packers are not just compressor they will use a lots of anti-debugging  technologies against debugger / disassembler to make reverser life harder. this technologies are out of scope of  this post.

Ok for example if we want to make  a signature for a new packer / protector we need to pack / protect variant  executable (it’s better  to test on different compiler / size)  and then watch which byte of files are changed and which one are static !

you can use binary copy option in immunity debugger for starting our test

figure 4 binary copy

this program is  packed with a really simple and good packer named FSG.

and my first signature will be :

87 25 5C AD 41 00 61 94 55 A4 B6 80 FF 13 73 F9 33 C9 FF 13 73 16 33

so now i need to pack more files and check my selected Op-codes to know which one are changed and then we will replace changed op codes with ?? .  after a few try we will get a signature like  :

87 25 ?? ?? ?? ?? 61 94 55 A4 B6 80 FF 13 73 F9 33 C9 FF 13 73 16 33

so if i search for these bytes i can find i can find them in any program those are packed with FSG v2 !

this example is really really simple for advanced packer we need really test more bytes to be sure our signature is good enough but from my experience  length between 30-70 byte  from entry point are good enough.

if you be smart you will select good instructions like sections those have 16-bit registers and instructions those are not used all times. so an example of really good signature can be below figure (taken from symantec slides) :

figure 5 ( a really good signature )

OK. now you can make you own signatures just by spending a few time on each target . there are several tools can be use for detecting  signatures if executable most popular of them are :

  • PEiD
  • RDG Packer Detector
  • PE Detective

but all of them have a same problem not so update signatures ! so if you have a program that is packed by a really new packer or just a few byte take changed from their signature  most of them will fail (intelligent signature detection is out of scope of this post) . so what we can do ? we should have our own database for our job .

so i collect all of existing signature database (those i found) in internet and i removed stupid and duplicated signature from the list those are :

  • BoB at Team PEiD signature database
  • Panda Security customized signature database
  • Diablo2002 signature database
  • ARteam members signature database
  • SnD members signature database
  • Fly signature database
  • and …

after i combined all of their signature databases i changed a few of important signature to be more general and i added some new signature to my list  and my final list right now have around 5064 unique and 4268 from entry point signature.

PEiD can parse external signatures and it’s nice but i liked to have detection in my debugger so i searched for a signature detection library in python (i like python) and with a quick search i found nice Pefile coded by Ero Carrera can handle all of our requirement in working with PE file not only handling signatures you can download it at :

http://code.google.com/p/pefile/

so i decide to use this library to write a pycommand for immunity debugger fortunately i found a copy of a pefile in immunity debugger lib ! so all i have to do is writing a few line of code that can read my database and test it against my binary and tell me the output .
so here is my complete script also have a option for auto-update  .

#!/usr/bin/env python
 
'''
 This script is for identify packer/protector and compiler used in your target binary
 the first version have more than about 5000 signatures ... we will try to updates signatures monthly
 and for now it will use entry point scaning method ...
 
 Tree Important Notes :
 First  the database signatures are reaped by lots of people we should thanks them : BoBSoft at Team PEID  , fly , diablo2oo2 and others you can find their name in list ...
 Second A big thanks to Ero Carrera for his nice python pefile lib the hard part of processing singanutes is done by his library .
 Third  we updated some of signatures and will keep update them monthly  for detection newer version of packers / comprassion algorithm (hopefully) 
 
 thanks to nicolas waisman / Muts (offsec) and all of abysssec memebers ...
 
 Feel free to contact me with admin [at] abysssec.com
'''
 
#import python libraries
import os
import sys
import getopt
import pefile
import immlib
import peutils
import hashlib
import shutil
import urllib
 
__VERSION__ = '0.2'
 
DESC= "Immunity PyCommand PeDectect will help you to identfy packer / protection used in target binary"
USAGE = "!PeDetect"
 
#global
downloaded = 0
 
#Using debugger functionality
imm = immlib.Debugger()
 
# pedram's urllib_hook
def urllib_hook (idx, slice, total):
    global downloaded
 
    downloaded += slice
 
    completed = int(float(downloaded) / float(total) * 100)
 
    if completed > 100:
        completed = 100
 
    imm.Log("   [+] Downloading new signatures ... %d%%" % completed)
 
# Downloader function
def get_it (url, file_name):
    global downloaded
 
    downloaded = 0
    u = urllib.urlretrieve(url, reporthook=urllib_hook)
    #imm.Log("")
    shutil.move(u[0], file_name)
 
# Calculate MD5Checksum for specific file
def md5checksum(fileName, excludeLine="", includeLine=""):
    m = hashlib.md5()
    try:
        fd = open(fileName,"rb")
    except IOError:
        imm.Log("Unable to open the file in readmode:", filename)
        return
    content = fd.readlines()
    fd.close()
    for eachLine in content:
        if excludeLine and eachLine.startswith(excludeLine):
            continue
        m.update(eachLine)
    m.update(includeLine)
    return m.hexdigest()
 
# Simple Usage Function
def usage(imm):
    imm.Log("!PeDetect -u (for updating signature ... )" )
 
# Auto-Update function
def update():
 
    # Using urlretrieve won't overwrite anything
    try:
        download = urllib.urlretrieve('http://abysssec.com/AbyssDB/Database.TXT')
    except Exception , problem:
        imm.Log ("Error : %s"% problem)
 
    # Computation MD5 cheksum for both existing and our current database
    AbyssDB = md5checksum(download[0])
    ExistDB = md5checksum('Data/Database.TXT')
 
    imm.Log(" [!] Checking for updates ..." , focus=1, highlight=1)
    imm.Log("")
    imm.Log(" [*] Our  database checksum : %s "%AbyssDB)
    imm.Log(" [*] Your database checksum : %s "%ExistDB)
    imm.Log("")
 
    if AbyssDB != ExistDB:
 
        imm.Log("[!] Some update founds updating ....")        
 
        # Removing existing one for be sure ...
        if os.path.exists('Data/Database.txt'):
            os.remove('Data/Database.txt')
 
        # Download latest database
        try:
            get_it("http://abysssec.com/AbyssDB/Database.TXT", "Data/Database.txt")
        except Exception,mgs:
            return " [-] Problem in downloading new database ..." % mgs
 
        imm.log("   [+] Update Comepelete !")
 
    else:
        imm.Log(" [!] You have our latest database ...")
 
# Main Fuction
def main(args):
 
    if args:
        if args[0].lower() == '-u':
            update()
        else:
            imm.Log("[-] Bad argumant use -u for update ...")
            return  "[-] Bad argumant use -u for update ..."
    else:
        try:
            # Getting loded exe path
            path = imm.getModule(imm.getDebuggedName()).getPath()
        except Exception, msg:
            return "Error: %s" % msg
 
        # Debugged Name
        name = imm.getDebuggedName()
 
        # Loading loaded pe !
        pe = pefile.PE(path)
 
        # Loading signatures Database
        signatures = peutils.SignatureDatabase('Data/Database.TXT')
 
        # Mach the signature using scaning entry point only !
        matched = signatures.match(pe , ep_only=True)        
 
        imm.Log("===================  Abysssec.com  =======================")
        imm.Log("")
        imm.Log("[*] PeDetect By Shahin Ramezany" , focus=1, highlight=2)
        #imm.Log("=============================================================")
        imm.Log("[*] Total loaded  signatures : %d" % (signatures.signature_count_eponly_true + signatures.signature_count_eponly_false + signatures.signature_count_section_start))
        imm.Log("[*] Total ep_only signatures : %d" % signatures.signature_count_eponly_true)
        #imm.Log("=============================================================")
        imm.Log("")
 
        # Signature found or not found !
        if matched:
            imm.log("[*] Processing : %s " % name)
            imm.Log("[+] Signature Found  : %s "   % matched , focus=1, highlight=1)
            imm.Log("")
        else:
            imm.log("[*] Processing   %s !" % name)
            imm.Log("   [-] Signatue Not Found !" , focus=1, highlight=1)
            imm.Log("")
 
    # Checking for arguements !
        if not args:
            usage(imm)
 
        return "[+] See log window (Alt-L) for output / result ..."

for using this script you just need copy PeDetect.py in you PyCommand directory in immunity debugger python then copy Database.TXT in DATA folder in immunity debugger. after this you just need run it from immunity debugger command bar using !PeDetect you can see the output of this script against some files…


figure 6 – output of PeDetect against not packed file


figure 7 – output against  packed file

also this have an argument !PeDetect -u for updating your signature to our latest database. notice that my script will use md5checksum so your changes meaning it won’t be same as my database and your database will be update automatically.

figure 8 – update command

PS : after i wrote this i saw another PyCommand named scanpe wrote by BoB at PeiD it’s really good and have PE scan option but have not update update so no more new signatures …

references :

  • Automatic Generation of String Signatures for Malware Detection
  • Signature Generation by korupt (http://korupt.co.uk)
  • Team PEiD forums
  • Immunity Debugger online documentation
  • FSecure – reverse engineering slides
  • My time

download PeDetect (database + pycommand) from : (please read the ReadMe.txt for installation guide)

http://abysssec.com/files/PeDetect.zip

happy new years !

cheers

The Portable Executable (PE) format is a file format for executables, object code, and DLLs, used in 32-bit and 64-bit versions of Windows operating systems. The term “portable” refers to the format’s versatility in numerous environments of operating system software architecture.

Categories


Get Adobe Flash player