Network security

Intrusion Prevention System: First Line of Defense

Sudhanshu Chauhan
July 2, 2012 by
Sudhanshu Chauhan

In the past few years, advancements in technology have grown hand in hand with a substantial increase in hacking and cyber espionage. The whole world is deeply interconnected through the medium of the Internet, and one vulnerable machine can be the gateway to a series of undesirable events. The motivation for such events could be anything from financial gain, to political statements, or simply for fun. But for the victim of such an attack, it might cause a much more significant loss than intended. For example, a technology company whose website is successfully hacked results in the loss of reputation, which is certainly very crucial for any company. Enterprises, governments, and even individuals face this problem on a daily basis without knowing how to deal with it.

So what is a solution that could help considerably reduce, if not eradicate, this threat? The answer is the implementation of an intrusion prevention system (IPS). An IPS can be understood as a security mechanism that monitors a network and/or system for any malicious action and tries to prevent it.

Learn Network Security Fundamentals

Learn Network Security Fundamentals

Build your skills with seven hands-on courses covering network models and protocols, wireless and mobile security, network security best practices and more.

Learn Network Security Fundamentals

Learn Network Security Fundamentals

Build your skills with seven hands-on courses covering network models and protocols, wireless and mobile security, network security best practices and more.

As explained on Wikipedia ( the core functions of an IPS are:

  • Identify malicious activity
  • Log information related to such activity
  • Attempt to block/stop such activity
  • Report the activity

Often, an intrusion prevention system (IPS) is confused with an intrusion detection system (IDS), but in reality an IPS can be considered an extension of IDS. They do have some features in common like traffic monitoring, observing system activity, creating log etc., but what makes an IPS different from an IDS is the IPS's ability to prevent/block any detected intrusion attempt.

Differences between IPS and IDS:


It provides an active prevention security solution.    

An IPS sits in-line with the network traffic flow.

Core function is to prevent an intrusion. 

An IPS slows down the traffic (analysis time).

Example: SNORT  


It provides a passive detection security solution. 

An IDS generally does not sit in-line (although it can) with the network traffic flow.

Primary purpose is monitoring and reporting of any malicious activity.

An IDS does not affect the traffic.

Example: OSSEC HIDS  

Because of the services provided by an IPS, it is also known as an intrusion detection and prevention system (IDPS). Basically, an IPS sits in-line with the network traffic and monitors it.

Through analysis of intrusion signatures, generic behavior, and heuristic methods, it looks out for any malicious action on the network/system and takes the appropriate action of dropping the packet and/or blocking the specific traffic. It further sends an alert to the administrator when any such event occurs.

Intrusion prevention systems can be classified as:

  • Network intrusion prevention system (NIPS): This type of IPS is placed at specific point(s) on the network to monitor the entire network for any malicious traffic and attempts to stop it.
  • Host intrusion prevention system (HIPS): A host intrusion detection system is implemented on individual hosts (or devices) to monitor the inbound and outbound traffic for that specific host/device and perform the required action after detecting any malicious activity.
  • Network behavior analysis (NBA): This kind of IPS inspects the traffic for suspicious or unusual flow that could lead to an attack like DoS/DDoS.
  • Wireless intrusion prevention system (WIPS): This monitors the wireless network for malicious behavior.

Some common types of threats that an IPS usually is capable of handling:

  • DoS/DDoS attack: An attack that aims to make a service unusable for legitimate users by flooding the service with a huge number of connection requests.
  • ARP spoofing/poisoning: ARP stands for address resolution protocol, which is utilized to find a MAC address when the IP is known in a local network. If successful, this kind of attack allows the attacker to intercept all the traffic between two hosts.
  • SSL evasion: Most new IPSs are capable of stopping attacks on the SSL protocol.
  • Port scanning: Generally IPSs are capable of stopping any attempt to find which ports are open on specific hosts.
  • OS fingerprinting: During the recon phase of an attack, one of the most important tasks is to identify a target machine's operating system so that specific exploits can be launched to compromise that machine. Modern IPSs are capable of detecting and stopping any such attempt.
  • Buffer overflow: This is one of the most widely exploited attacks that can lead to total system compromise. This can be handled by an advanced IPS.

An IPS generally utilizes the follow detection methods:

  • Signature based threat detection: An IPS contains a huge repository of attack signatures from known exploit/vulnerability patterns and utilizes these to detect any attempt at intrusion. If a match is found based on the signature, suitable action is taken by the IPS to prevent the attack.
  • Anomaly based threat detection: This detection method uses the baseline established by average network traffic conditions and compares current traffic conditions with that baseline to identify any abnormal or unsafe behavior. If any such activity is found, the IPS takes the appropriate action to safeguard against it. This technique can provide a defense against unknown threats.
  • Stateful protocol analysis detection: This methods looks out for difference in the protocol states as compared to standard predefined profile.
  • Passive monitoring: In this technique the IPS simply sits and monitors abnormal or suspicious behavior, such as unusual amount of traffic from the same IP address (attempt of DoS), and takes the required action against it.

An IPS must not be confused or mistaken as a substitute for another network security measure, namely: the firewall. The job of a firewall is to limit access to a network/system, and it is generally not capable of detecting and preventing an intrusion.

Firewalls basically work based on filtering rules, whereas an IPS uses signature-based threat-detection (and many other techniques) to safeguard the network/system. A firewall's job is to allow or block traffic, whereas an IPS's job is to determine if there is something malicious in the traffic allowed by the firewall. An IPS is usually deployed behind a firewall on a network. Neither of these two tools should be considered a replacement for the other; instead, they must be deployed in conjugation with each other to implement defense in depth.

Some of the best tools of the trade are described below:

The first free and open source NIDPS (network intrusion detection and prevention system) is called as Snort. Originally released by Martin Rosh (Sorcefire founder and CTO) in 1998, Snort has become one of the most deployed IDPS. Snort can perform real time analysis of the traffic and logging of the packets on an IP network. It analyses the protocol, searches/matches the content, and is capable of detecting different kinds of attacks described earlier in the article (port scanning, OS fingerprinting, buffer overflow, etc.).

The latest version of the application can be downloaded from Figure 1 shows a standard example of the Snort interface.

Figure 1. Snort Interface

Snort has three primary uses:

  • Packet sniffer: Read and display network packets.
  • Packet logger: Logs the packets to the disk, which can be used for network traffic debugging.
  • Intrusion prevention system (IPS): Monitors and analyses the traffic based on the predefined rule-set and performs the specified action.

Snort rules are at the core of the detection of any intrusion attempt. They can be defined as: The approach to perform detection. Snort rules are defined based on vulnerabilities, unlike signatures, which are based on exploits and can be bypassed by modifying the exploit. Figure 2 shows the output of the packet dump mode.

Figure 2. Snort packet dump output

Next in the list is another free and pen source tool called OSSEC. OSSEC is a host-based intrusion detection system, but that's not all; it also contains aspects of a log monitor and security information and event manager (SIM/SIEM), which makes it a unique tool in its arena. A security event manager can be understood as a tool which can be used on huge data network with the aim to centralize the storage and interpretation of events/logs that are generated by other applications.

OSSEC is capable of performing:

  • Log analysis
  • Registry monitoring
  • File integrity check
  • Rootkit detection
  • Real-time alerting
  • Active response

OSSEC can be easily implemented across multiple platforms such as Linux, Mac, Windows, BSD, VMware ESX, etc. OSSEC contains a cross platform architecture that allows it to monitor and manage multiple systems easily through a centralized management server. It can easily be integrated with an existing system to perform centralized event reporting. Figure 3 shows the initiation of OSSEC.

Figure 3. OSSEC initiation

OSSEC contains a manager, which is the center of the deployment. It stores and performs all the functions and hence, acts as a server. Agents are installed on the machines that need to be monitored. Agents send the information in real-time to the manager for the purpose of analysis. OSSEC allows performance integrity checks on devices that do not allow installation of the agent such as routers, firewalls, etc. OSSEC agents can also be installed on a virtual machine (VMware ESX) and can help monitor the virtualization platform. Figure 4 displays the output log.

Figure 4. OSSEC output log

Last in the list is Honeyd, which is also an open source program. It is not actually an IPS/IDS but a honeypot which allows its user to set up virtual hosts. These virtual hosts can act as decoys for attackers and distract them from legitimate systems. They can also help to catch attackers after distracting them from the intended target. Figure 5 shows the Honeyd interface. Honeyd in action is displayed by Figure 6.

Figure 5. Honeyd interface

Figure 6. Honeyd in action

Some IPS/IDS evasion techniques as explained on Wikipedia ( are:

  • Obfuscating attack payload: Encoding the payload of the attack in a manner such that the target machine will be able to reverse it but the IPS/IDS will not.
  • Polymorphic code: Creating the same attack in different (unique) forms so that the IPS/IDS does not detect it.
  • Packet fragmentation: Breaking the attack payload into many small packets so that the IPS/IDS is not capable of reassemble them to detect the attack.
  • Overlapping fragments: Crafting a series of packets with overlapping TCP sequence number.
  • Traffic insertion at the IPS/IDS: Sending packets crafted with TTL (time to live) such that it only reaches the IDS/IPS.
  • Denial of Service/Distributed Denial of Service (Dos/DDoS): Exploiting a vulnerability/bug the IPS/IDS can be flooded with traffic overloading all its computational resources, which will make it unusable.

One tool that needs to be mentioned here is fragroute. Fragroute is a tool that is capable of intercepting, modifying, and rewriting outbound traffic for specified host. The capabilities of this tool make it one of the standard tools used for IPS/IDS evasion. The latest version of the tool (i.e. frageroute-1.2.5-ipv6) is compatible with IPv6 networks. The tool can be downloaded from The non IPv6 version can be found at Figure 7 shows the frageroute interface.

Figure 7. Fragroute interface


Today there are many technologies present in the market designed to safeguard systems/networks from being attacked and to prevent data loss. From firewalls to anti-virus to encryption systems etc., there are many protective measures we can use to hide our precious data.

Yet even with the plethora of security technology, we have to face intrusion events and worry about our data being stolen. The main source of such occurrences is the lack of awareness among common people and even people who are supposed to look after our data. People confuse different security measures and consider one a substitute for another, which can lead to data compromise.

Intrusion detection and prevention systems provide us with the first line of defense against malicious intruders. We must not consider an IPS as a substitute for IDS (it's a different case if the same tool provides all the functions of both tools). IPS and IDS provide the control and visibility required by any enterprise to protect its infrastructure from being attacked and taken over.

One of the most important factors to take into consideration when utilizing an IPS/IDS is: proper inclusion into the infrastructure through deployment. Improper deployment will lead to a huge slowdown of the network and improper management.

Some best-practices that should be followed regarding IPS/IDS:

Learn Network Security Fundamentals

Learn Network Security Fundamentals

Build your skills with seven hands-on courses covering network models and protocols, wireless and mobile security, network security best practices and more.

Learn Network Security Fundamentals

Learn Network Security Fundamentals

Build your skills with seven hands-on courses covering network models and protocols, wireless and mobile security, network security best practices and more.
  • Identify and prioritize risk areas for proper deployment.

  • Ensure complete coverage of the network.

  • Plan appropriate incidence response in case an intrusion is detected.

  • Awareness and training for the staff.

Sudhanshu Chauhan
Sudhanshu Chauhan

Sudhanshu Chauhan is a researcher at InfoSec Institute. He is a B.Tech (CSE) graduate from Amity University. His areas of interest include (but are not limited to) Web Application Security and Bypasssing Security Measures(IDS/IPS, AV etc.).