Malware analysis

Common malware behavior

Dan Virgillito
September 4, 2019 by
Dan Virgillito

As malware threats continue to grow in both sophistication and frequency, it is increasingly critical for information security professionals to develop effective mitigation and reverse-engineering techniques. A good starting point is identifying and understanding key behaviors common to modern malware intrusions. This also helps ensure that IT defense teams and engineers have the knowledge necessary to dissect, block and undo malware.

Become a certified reverse engineer!

Become a certified reverse engineer!

Get live, hands-on malware analysis training from anywhere, and become a Certified Reverse Engineering Analyst.

Beginning with downloaders and backdoors, this article breaks down a number of behaviors that indicate the potential presence of malware. Learn not just the “whats” but the “whys” in the essential section on the objectives of network analysis and get familiar with some reversing techniques for common attack vectors such as credential stealers, keyloggers and more.

Let’s jump in, shall we?

Key behaviors of various types of malware

Downloaders and backdoors

During a malware attack, the threat actor will often use a range of Trojans to infiltrate a vulnerable system. The infiltration is followed by the creation of a downloader or backdoor that allows the attacker to gain remote access over the targeted system. 

Reversing these applications requires analyzing how downloaders and backdoors run in a sandbox environment as well as understand their processes, registries, network activities and file systems. The reverse engineer will also use a debugger and a disassembler, which could be supported by a decompiler and a range of helpful tools.

Credentials stealers

Credentials stealers are a type of malware that searches for passwords saved on a target machine and transfers them remotely to an attacker (using HTTP, email). Malware authors typically use software that waits for users to log in. Other common techniques involve the use of programs that log keystrokes and tools that dump credentials stored in Windows, like password hashes, for offline cracking. 

Process injection

This is a malware attack technique that gives adversaries the ability to deploy malicious code that mimics legitimate applications. Running code in the environment of another application may grant access to its process memory, network/system resources, and even authorized privileges. The code is typically inserted into common processes (svchost.exe, regsvr32.exe, etc.), granting the attacker an improved level of persistence and stealth. 

Some kernel modules can be configured to prevent some forms of process injections, based on behavioral patterns that are identified during the analysis of the attack. 

DLL and direct injection 

DLL (dynamic-link library) injection is a type of process injection that requires the threat actor to write the path to a DLL inside an application’s process and then using a remote thread to invoke execution. Code injection methods like SetWindowsHookEx API or manipulation tactics like CreateRemoteThread can be used for DLL injections on Windows. In contrast, a direct injection involves the direct placement of shellcode into another process.

Process replacement (aka process hollowing) is where malicious code will substitute an illicit process for a genuine one, helping the code to hide among legitimate processes. This is usually done by hollowing the genuine code from the memory of the target process using a kernel function like NtUnmapViewOfSection, and then assigning a new block of memory for hosting illicit code. 

Hook injection

Malware creators may also use a hook injection to load and run malicious code within the environment of another program, mimicking the original execution while also gaining access to memory processes. In addition, the technique may also be used to intercept API calls that feature input and output parameters.

Once APIs are hooked, the threat actor can control the relevant program’s execution path and point it to the malicious process of his or her choice. 

APC injection

APC, or Asynchronous Procedure Call injection, describes the technique of attaching malicious code to the APC queues of a thread inside a process. Infected APCs will then tell the thread to execute some other command before initiating its standard execution path. 

An example of this is AtomBombing. AtomBombing involves the utilization of queued APC functions to invoke malware previously used to add character strings to the global atom table.

Registry persistence 

After a malware occupies the processes of a system it aims to stay there for a long period. This is normally done by modifying the registry keys to collect details about the system, save configuration information and achieve persistence on the infiltrated machine. This enables the adversary to attack once and the malicious software will continue to function even after log-offs, reboots and restarts. 

Using SvchostEXE for malware persistence

Svchost.exe is a basic host process for programs that function from DLLs, and systems with Windows OS have several instances of svchost.exe active at once. Using svchost.exe for malware persistence allows the threat’s code to blend with the process list and registry better than the usual service. In its setup, the host process may also delete the randomly named program and create a new one. 

Trojanized system binaries 

Malware may also gain persistence by Trojanizing system binaries. This is where the malware patches the threat bytes of a system binary to encourage the system to run the malware the next time the compromised binary is loaded or activated. 

Threat actors will usually go after a system binary that is regularly utilized for everyday Windows programs, such as DLLs. Typically, the binary is modified through the patching of the entry function so that it directs to malicious code. 

DLL load order hijacking 

Windows systems typically look for DLLs required by executable in a certain order. When the executable isn’t searching for a DLL through hardcoded paths, then a malicious code can be placed in the DLL search order and be loaded by the executable. 

Malware creators may also modify DLL loading capability of a program by substituting a current DLL or modifying a directory, junction or manifest file to cause the program to execute in another DLL, allowing it to maintain privilege or persistence growth.

Analyzing malware network behavior 

When it comes to analyzing the behavior of malware, analyzing and interpreting its network activity can help complement dynamic code and contemporary static analysis. Because it can be challenging for malicious software to significantly modify its network behavior and still achieve its objectives, analyzing malware network behavior may provide an opportunity to identify malware on affected hosts. The process requires extracting features from network records and then creating patterns that help identify malware intrusions. 

All of these insights can be used to create effective countermeasures against malicious attacks.

Networking basics for reverse engineers 

Put simply, network reverse-engineering is the art of extracting network/application-level protocols utilized by either an application or a client server. This section covers the fundamentals of how networking works and how to use various techniques and tools to mitigate a network-based malware attack. Ideally, a reverse engineer should strive to develop an understanding of: 


Protocols are a set of rules that a network needs to follow. The rules are made up of formats and procedures that outline communication between the devices on a network. 

Essentially, most networks have three kinds of protocols: security, like SSH (Secure Shell), management, like SMTP (Simple Mail Transfer Protocol), and communication, like Ethernet. Several advanced operating systems contain integrated software services that are designed to execute some of these network protocols. 

PDU (Protocol Data Unit)

Data transmitted via each layer of the Internet Protocol Suite is known as a Protocol Data Unit. Each layer’s PDU holds the payload data that is being routed. 

PDUs typically prefix a “heard” that contains information needed for the data to be routed, such as the destination nodes and addresses of the receiving device. A footer with values required to ensure correct routing may also be suffixed to the data being transmitted. 

Network packet

This is a unit of data that is transmitted between a destination and an origin over a packet-switched network like the World Wide Web. Packets vary in function and structure, based on the protocols executing them. 

For example, an Ethernet network uses 1.5 KBS packets and routes them via Ethernet frames. On a VoIP network, an IP protocol is used to transmit data over IP/TCP networks. 

Packet structure

The structure of a network packet is made up of three different parts: the header, the payload and the trailer. The header is composed of the instructions about the data hosted on the packet, the payload is the actual data that the packet is carrying at a certain point in time and the trailer is the couple of bits that inform the destination device that it has reached the packet’s end. 


That’s it. You’ll successfully gained an understanding of the key behaviors a malware would exhibit. Note that a malware may exhibit various other behaviors like COM object hijack and MBR, but the abovementioned tactics are some of the most commonly utilized strategies used by malware to gain persistence.


Dan Virgillito
Dan Virgillito

Dan Virgillito is a blogger and content strategist with experience in cyber security, social media and tech news.