NMAP Scripting Engine and Categories

1. Introduction

We all know what Nmap is and what we can do with it, but for those of you who don't, here's a short overview. Nmap is an open source security scanner that can scan for open ports and also do other functions as well, such as auditing the operating system, detecting the application version, etc.

FREE role-guided training plans

Get 12 cybersecurity training plans — one for each of the most common roles requested by employers.

Download Now

However, we often forget about the most powerful feature of Nmap scanner. We've not talking about host discovery, operating system enumeration, report generation or firewall evasion. We're also not talking about Zenmap, which is a graphical user interface for Nmap. What we are talking about is the Nmap scripting engine. The Nmap scripting engine allows us to write scripts that we can use with Nmap to automate any task. When running these scripts, we can expect them to be quite powerful, since they are using the underlying Nmap software program that is famous for speed and efficiency.

2. The Nmap Scripting Engine (NSE)

Before diving straight into the NSE, we'll say a few words about what NSE can do. We need to mention that NSE is capable of performing a wide variety of tasks. It can discover the devices on the network, scan for open ports, run a whois query, and more. It can scan a network to discover a newly disclosed vulnerability and while it isn't as good as a vulnerability scanner that was written for that sole purpose, it gets the job done. It can even be used to actually exploit the vulnerability rather than just detect it.

The NSE scripts are written in a programming language called LUA. The scripts can be run with the command below:

[bash]

# nmap -sC

# nmap --script

[/bash]

The -sC option enables the most common scripts, while we can choose the scripts to run with the --script option. The --script option takes comma-separated values as arguments which specify the scripts to be run upon starting Nmap. We don't necessarily need to specify the script names, but we can specify the following: filenames, categories and directory names.

The Nmap main page contains the following regarding the Nmap scripting engine commands:

[plain]

SCRIPT SCAN:

-sC: equivalent to --script=default

--script=: is a comma separated list of

directories, script-files or script-categories

--script-args=: provide arguments to scripts

--script-trace: Show all data sent and received

--script-updatedb: Update the script database.

[/plain]

We already mentioned the -sC and --script command, but there are others as well. The --scripts-args command provides the arguments to each of the scripts, while the --script-trace command is used for debugging purposes to show all sent and received data. The --script-updatedb command is used to update the script database, which is needed if we added/removed a script or if we changed the categories. We must also mention that script scanning is automatically done if we pass the option -A (aggressive scan) to the Nmap command. There's also a --script-help command that displays the help page of the script.

There are multiple commands we can use in combination with script scanning. Normally, Nmap first discovers if ports are online, or else it will run a port scan now and then to discover open ports on the active targets. To disable host discovery, we can use the -Pn option. To disable port scanning, we can use the -sn option.

To run a script without host or port discovery, run the command below (every host is assumed to be up):

[bash]

# nmap -sC -Pn -sn

[/bash]

2.1. Script Categories

Each NSE script belongs to a category based on what it does. Current categories are the following:

- auth: scripts that work with authentication credentials

- broadcast: scripts that discover active hosts by broadcasting on a local network and adding them to a target list

- brute: scripts that brute force the credentials of the remote service

- default: scripts that are automatically run with -sC or -A options

- discovery: scripts that try to acquire more information about the target network

- dos: scripts that may crash the target application and therefore cause a denial of service to the target

- exploit: scripts that may be able to exploit the target application

- external: scripts that send data to a third party server over the network (whois)

- fuzzer: scripts that send invalid random data to the target to find undiscovered bugs

- intrusive: scripts that can cause the target to fail

- malware: scripts that test whether the target is infected by malware or backdoors

- safe: scripts that can be run safely, so they will not crash a server

- version: scripts that can determine the version of the application running on a target (they are run only when -sV option is specified)

- vuln: scripts that can check whether the target is vulnerable to specific attacks

An example of running a script scan using the default script scan against the target www.google.com is shown below:

[plain]
nmap --script=default www.google.com

Starting Nmap 5.51 (http://nmap.org) at 2012-09-26 00:16 CEST

Nmap scan report for www.google.com (173.194.35.147)

Host is up (0.059s latency).

Other addresses for www.google.com (not scanned): 173.194.35.145 173.194.35.148 173.194.35.144 173.194.35.146

rDNS record for 173.194.35.147: muc03s01-in-f19.1e100.net

Not shown: 998 filtered ports

PORT STATE SERVICE

80/tcp open http

|_http-methods: No Allow or Public header in OPTIONS response (status code 405)

| http-robots.txt: 245 disallowed entries (15 shown)

| /search /sdch /groups /images /catalogs /catalogues

| /news /nwshp /setnewsprefs? /index.html? /? /?hl=*&

|_/addurl/image? /pagead/ /relpage/

| http-title: 302 Moved

|_Did not follow redirect to http://www.google.si/

|_http-favicon:

443/tcp open https

|_http-methods: No Allow or Public header in OPTIONS response (status code 405)

| http-robots.txt: 245 disallowed entries (15 shown)

| /search /sdch /groups /images /catalogs /catalogues

| /news /nwshp /setnewsprefs? /index.html? /? /?hl=*&

|_/addurl/image? /pagead/ /relpage/

| http-title: 302 Moved

|_Did not follow redirect to https://www.google.si/

|_http-favicon:

Nmap done: 1 IP address (1 host up) scanned in 7.22 seconds
[/plain]

We can see that there are multiple IP addresses for the domain name www.google.com and that we're scanning only one of them, 173.194.35.147. Nmap scanned 1000 ports, 998 of which are filtered and two of which are opened: port 80 and port 443.

Nmap also performed a number of additional scans when determining the open ports. We can see that it first sent the "OPTIONS / HTTP/1.0" request to the web server on both ports and received that the command OPTIONS is not allowed and it can't be executed. Afterwards, it checked the contents of the robots.txt file that contained 15 entries, which are also listed. It did exactly the same scans for port 80 and port 443, since they are both running HTTP (except 443 is encrypted HTTP).

2.2. The NSE Script API

First, we must be aware that the NSE script is written in LUA. To use Nmap features in the LUA NSE script, there is an Nmap API accessible under the Nmap namespace, which we can use to call functions and access Nmap resources.

When beginning to write NSE scripts, there are quite a few variables that we can define in the beginning of the script to better describe it in detail. An example of the description, author, license, categories and action fields is presented below (this is taken from the banner script accessible here).

[plain]

description = [[

A simple banner grabber which connects to an open TCP port and prints out anything sent by the listening service within five seconds.

The banner will be truncated to fit into a single line, but an extra line may be printed for every increase in the level of verbosity requested on the command line.
]]

author = "jah"

license = "Same as Nmap--See http://nmap.org/book/man-legal.html"

categories = {"discovery", "safe"}

action = function( host, port )

local out = grab_banner(host, port)

return output( out )

end

[/plain]

We can see that the description field provides a description about what the script does, the author specifies the author of the script, etc. The following fields are available when writing NSE scripts:

- description: describes what the script does.

- categories: categorizes the script into the predefined categories.

- author: displays the scripts' authors.

- license: displays the license the script was published under.

- dependencies: lists the dependencies as an array containing the names of scripts that need to be run before this script. The dependencies just force the ordering when running the scripts specified with --script option, and it doesn't actually run the scripts.

- rules: defines the rules that are used to determine whether the script will be run against the target. Each rule must return either true or false and the action is only performed if the rule is evaluated to be true. A script must use one of the functions below to determine whether the rule has evaluated to true or not: prerule, hostrule, portrule or postrule. The prerule script is run before any hosts are scanned, while hostrule and portrule are run after each batch of hosts is scanned and then the postrule is run after all of the hosts have been scanned. Any data activity should be done in a prerule, while the postrule should be used for reporting. An example of a portrule taken from the previous banner example is below:

[plain]

---

-- Script is executed for any TCP port.

portrule = function( host, port )

return port.protocol == "tcp"

end

[/plain]

- action: defines a function to be run when the script's prerule, portrule, hostrule or postrule is evaluated to true. The action function accepts the same arguments as the rule and returns a table of name-value pairs, a string or a nil (which is automatically parsed when being written to screen). Again, let's take an example from the previous banner script, which defines the action as can be seen in the output below:

[plain]

---

-- Grabs a banner and outputs it nicely formatted.

action = function( host, port )

local out = grab_banner(host, port)

return output( out )

end

[/plain]

Each script also has three environment variables local to the script. The SCRIPT_PATH specifies the path of the script, the SCRIPT_NAME specifies the name of the script and the SCRIPT_TYPE specifies the rule that activated the script. Those variables are automatically created when the script is run and we can use the variables in our script normally.

There are also a bunch of NSE libraries, which can be included and used just by requiring them in the NSE script LUA code. The libraries provide a powerful mechanism, since they can greatly enhance the power of an Nmap script with ease. All of the modules are shown on this web page: http://nmap.org/book/nse-library.html.

3. Conclusion

We've looked at the basics that we need to know when writing Nmap NSE scripts. In the next tutorial, we'll take a look at the Nmap API and write a basic NSE script.

Further Resources:

FREE role-guided training plans

Get 12 cybersecurity training plans — one for each of the most common roles requested by employers.

Download Now

Nmap's commands and usage options