Nmap cheat sheet: From discovery to exploits, Part 3: Gathering additional information about host and network

As we discussed before, this is our third installment in our Nmap series.

Nmap is well known for port scanning, port discovery, and port mapping. But we can do many more things by the Nmap NSE script. We can do email fingerprinting, retrieve a Whois record, use UDP services, etc.

FREE role-guided training plans

Get 12 cybersecurity training plans — one for each of the most common roles requested by employers.

Download Now

Discovering Geographical Location

Gorjan Petrovski submitted Nmap NSE scripts that help us geo locate a remote IP address: ip-geolocation-maxmind, ip-geolocation-ipinfodb, and ipgeolocation-geobytes.

This will show us how to set up and use the geo location scripts included with Nmap NSE.

ip-geolocation-maxmind

For the NSE script to be run under Nmap, download Maxmind's city database from http://geolite.maxmind.com/download/geoip/database/GeoLiteCity.dat.gz. Extract it to your local Nmap data folder ($NMAP_DATA/nselib/data/).

Fire up the command line and enter the command to download the scripts:

Wget http://geolite.maxmind.com/download/geoip/database/GeoLiteCity.dat.gz

Put into the /usr/local/share/nmap/nselib/data and run the following command:

nmap --script ip-geolocation-* ip

Submitting a new geo-location provider

Sometimes the location may be wrong, because the location depends upon the maxmind database. So we have to submit a new database to Nmap [NSE script which is Nmap Dev, so that we can develop our own Nmap].

Getting information from WHOIS records

WHOIS records often contain important data such as the registrar name and contact information. System administrators have been using WHOIS for years now, and although here are many tools available to query this protocol, Nmap proves itself invaluable because of its ability to deal with IP ranges and hostname lists. Open up the command line and write the commands like:

nmap --script whois target

The argument --script whois tells Nmap to query a Regional Internet Registries WHOIS database in order to obtain the records of a given target. This script uses the IANA's Assignments Data to select the RIR and it caches the results locally. Alternatively, we could override this behavior and select the order of the service providers to use in the argument whodb:

nmap --script whois --script-args whois.whodb=arin+ripe+afrinic <target>

This script will query, sequentially, a list of WHOIS providers until the record or a referral to the record is found. To ignore the referral records, use the value nofollow:

nmap --script whois --script-args whois.whodb=nofollow <target>

To query the WHOIS records of a hostname list (-iL <input file>) without launching a port scan (-sn). We can also use other options for Nmap. Enter the following Nmap command:

nmap -sn --script whois -v -iL hosts.txt (hosts.txt contains a list of hosts or IP addresses)

Disabling cache

Sometimes cached responses will be preferred over querying the WHOIS service, and this might prevent the discovery of an IP address assignment. To disable the cache we could set the script argument whodbto nocache:

nmap -sn --script whois --script-args whois.whodb=nocache scanme.nmap.org

Checking if a host is known for malicious activities

Nmap allows us to systematically check if a host is known for distributing malware or being used in phishing attacks, with some help from the Google Safe Browsing API.

This recipe shows system administrators how to check if a host has been flagged by Google's Safe Browsing Service as being used in phishing attacks or distributing malware.

We have to use the script http-google-malware which depends on Google's Safe Browsing service and it requires you to register to get an API key. Register at:

http://code.google.com/apis/safebrowsing/key_signup.html

Open your favorite terminal and type:

nmap -p80 --script http-google-malware --script-args http-google-malware.api=<API> <target>

The script http-google-malwarequeries Google Safe Browsing Service determines if a host is suspected to be malicious. This service is used by web browsers such as Mozilla Firefox and Google Chrome to protect its users, and the lists are updated very frequently. Check the following command:

nmap -p80 --script http-google-malware -v scanme.nmap.org

Collecting valid e-mail accounts

Checking for an email address is very useful for penetration tester, since this information can also useful for further attacks like phishing attack, brute force attack, etc. Nmap gives the facility to perform discovery of email address.

To run these methods we have to run the NMAP script. The script http-google-email is not included in Nmap's official repository. So we need to download it from http://seclists.org/nmap-dev/2011/q3/att-401/http-google-email.nse and copy it to our local scripts directory. As you can see, the following screenshot has details for the email collector script:

But as a security researcher we must understand the script, because the script may contain malicious things. So the following covers details regarding the script. I have already downloaded the script below so that you do not have to download again, you can directly use this script:

description = [[

http-google-email queries the Google web search engine and Google Groups for e-mails pertaining to a specific domain.

]]

---

-- @usage

-- nmap -p80 --script http-google-email <host>

--

-- @output

-- PORT STATE SERVICE

-- 80/tcp open http

-- | http-google-email:

-- | nmap-dev () insecure org

-- | nmap-svn () insecure org

-- |_fyodor () insecure org

--

-- @args http-google-email.domain Domain to search for.

-- @args http-google-email.pages The number of results pages to be requested from Google Web search and Google Group search respectively. Default is 5.

---

author = "Shinnok"

license = "Same as Nmap--See http://nmap.org/book/man-legal.html"

categories = {"discovery", "safe", "external"}

-- This tells about HTTP protocol:

require "http"

require "shortport"

portrule = shortport.http

-- Declare the variable to perform the search:

--Builds Google Web Search query

-- () param domain

-- () param page

-- () return Url

-- HTTP query method to perform the search:

local function google_search_query(domain, page)

return string.format("http://www.google.com/search?q=%%40%s&hl=en&lr=&ie=UTF-8&start=%s&sa=N", domain, page)

end

-- Building Google group search:

--Builds Google Groups Search query

-- () param domain

-- () param page

-- () return Url

local function google_groups_query(domain, page)

return string.format("http://groups.google.com/groups?q=%s&hl=en&lr=&ie=UTF-8&start=%s&sa=N", domain, page)

end

-- Here the main program starts. It will retrieve 50 pages per search:

---

--MAIN

---

action = function(host, port)

local pages = 50

local target

local emails = {}

if(stdnse.get_script_args("http-google-email.pages")) then

pages = stdnse.get_script_args("http-google-email.pages")*10

end

-- Check if we have the domain argument passed

if(stdnse.get_script_args("http-google-email.domain")) then

target = stdnse.get_script_args("http-google-email.domain")

else

-- Verify that we have a hostname available

if not(host.targetname) then

return string.format("[ERROR] Host can not be resolved to a domain name.")

else

target = host.targetname

end

end

stdnse.print_debug(1, "%s: Checking domain %s", SCRIPT_NAME, target)

-- Local search through Google:

-- Google Web search

for page=0, pages, 10 do

local qry = google_search_query(target, page)

local req = http.get_url(qry)

stdnse.print_debug(2, "%s", qry)

stdnse.print_debug(2, "%s", req.body)

body = req.body:gsub('<em>', '')

body = body:gsub('</em>', '')

if body then

local found = false

for email in body:gmatch('[A-Za-z0-9%.%%%+%-]+@' .. target) do

for _, value in pairs(emails) do

if value == email then

found = true

end

end

if not found then

emails[#emails+1] = email

end

end

end

end

-- Google Groups search

for page=0, pages, 10 do

local qry = google_groups_query(target, page)

local req = http.get_url(qry)

stdnse.print_debug(2, "%s", qry)

stdnse.print_debug(2, "%s", req.body)

body = req.body:gsub('<b>', '')

body = body:gsub('</b>', '')

if body then

local found = false

for email in body:gmatch('[A-Za-z0-9%.%%%+%-]+@' .. target) do

for _, value in pairs(emails) do

if value == email then

found = true

end

end

if not found then

emails[#emails+1] = email

end

end

end

end

if #emails > 0 then

return "n" .. stdnse.strjoin("n", emails)

end

end

The easiest method to find all nse scripts is to use the find command like below.

find / -name '*.nse'

The above command will enlist all .nse files for the directory. In general the path may be usr/share/nmap/scripts. Please find the below commands and screenshot for reference:

Clearly it mentions all scripts and relevant directory paths. After getting the path, just download the script with the "wget" command. You can also directly copy the script to the relevant directory.

Check the script for confirmation. Follow the below screenshot.

After copying http-google-email.nse, we should update the script database with the following command:

nmap --script-updatedb

After updating the scrip,t fire up the command line with the following script:

nmap -p80 --script http-google-email <target>

NSE Script Argument

The flag –script--args argument is used to set the argument of NSE script. For example:

nmap -sV --script http-title --script-args http.useragent="Mozilla 999" <target>

The above will work on HTTP Library argument "useragent".

Additional Host Information And Further Pentesting

As a professional security tester we should go a further step to gain additional information about the network or host which will boost our pentesting.

The –O parameter used to detect the target operating system:

Command: nmap –O target

Operating system detection is performed by analyzing responses from the target for a set of predictable characteristics which can be used to identify the type of OS on the remote system. In order for the OS scan work perfectly there must be at least one open and one closed port on the target system. When scanning multiple targets, the --osscan-limit option can be combined with -O to instruct Nmap not to OS scan hosts that do not meet this criteria.

Multiple options for nmap can be used, like –v.

Ex: nmap –v –O <target>

In some cases, Nmap will not be able to determine the OS. it will provide a fingerprint which can be submitted to Nmap' s OS database at www.nmap.org/submit/

By submitting the fingerprint generated and correctly identifying the target system's operating system, we can improve the accuracy of Nmap' s OS detection feature in future releases.

Guessing the Operating System

If Nmap is unable to determine the operating system, we can use the –osscan option to force Nmap into discovering the OS.

Note: This option is useful when Nmap is unable to determine the discovered OS

Command: nmap -O --osscan-guess target

It will list all possible matches of operating system in Nmap's script database.

The –fuzzy option can be used as a shortcut for the above.

--osscan-limit (Limit OS detection to promising targets)

OS detection is far more effective if at least one open and one closed TCP port are found. Set this option and Nmap will not even try OS detection against hosts that do not meet this criteria. This can save substantial time, particularly on -Pn scans against many hosts. It only matters when OS detection is requested with -O or -A.

--max-os-tries (Set the maximum number of OS detection tries against a target)

When Nmap performs OS detection against a target and fails to find a perfect match, it usually repeats the attempt. By default, Nmap tries five times if conditions are favorable for OS fingerprint submission, and twice when conditions aren't so good. Specifying a lower --max-os-tries value (such as 1) speeds Nmap up, though we miss out on retries which could potentially identify the OS.

Command: map –O --max-os-tries target

The above options show the extra discovery options for OS discovery. We can easily discover the difference between the above options.

Troubleshooting Version Scans

The --version-trace option can be enabled to display verbose version scan activity.

Usage syntax: nmap -sV --version-trace [target]

The --version-trace option can be helpful for debugging problems or to gain additional information about the target system.

Perform an RPC Scan

The -sR option performs a RPC (Remote Procedure Call) scan on the specified target.

Usage syntax: nmap -sR [target]

The output of the -sR scan above displays information about RPC services running on the target system. RCP is most commonly associated with Unix and Linux systems specifically for the NFS (Network File System) service.

Nmap Scanning with --hostmap version

The script hostmap depends on external services, and the official version only supports BFK's

DNS Logger. Previously I told how to download and update the database with the local directory.

Download hostmap.nse with Bing support at https://secwiki.org/w/Nmap/.

External_Script_Library.

After copying it to our local script directory, update your script database by running the following command:

#nmap --script-updatedb

Open a terminal and enter the following command:

nmap -p80 --script hostmap nmap.org

The arguments --script hostmap -p80 tell Nmap to start the HTTP script hostmap and limit port scanning to port 80 to speed up this task.

This version of hostmap.nse queries two different web services: BFK's DNS Logger and ip2hosts.com. BFK's DNS Logger is a free service that collects its information from public DNS data and ip2hosts. Both of these services are free, and abusing them will most likely get you banned from the service.

Different search engine arguments are used for pentesting about hostname as follows:

nmap -p80 --script hostmap --script-args hostmap.provider=BING <target>

nmap -p80 --script hostmap --script-args hostmap.provider=BFK <target>

nmap -p80 --script hostmap --script-args hostmap.provider=ALL <target>

To save a hostname list for each IP scanned, use the argument hostmap.prefix. Setting this argument will create a file with a filename of <prefix><target> in our working directory:

nmap -p80 --script hostmap --script-args hostmap.prefix=HOSTSFILE <target>

Brute forcing DNS record

This is used for attempts to enumerate DNS hostnames by brute force guessing of common subdomains. With the dns-brute.srv argument, dns-brute will also try to enumerate common DNS SRV records.

Spoofing the origin of IP port scan

Idle scanning is a very powerful technique, where Nmap takes advantage of an idle host with a predictable IP ID sequence number to spoof the origin IP of a port scan.

This technique illustrates how to find zombie hosts and use them to spoof your IP address when scanning a remote host with Nmap.

To launch an idle scan we need a zombie host. A zombie host is a machine with a predictable

IP ID sequence number that will be used as the spoofed IP address. A good candidate must not be communicating with other hosts, in order to maintain the correct IP ID sequence number and avoid false positives. To find hosts with an incremental IP ID sequence, you could use the script ipidseq as follows:

#nmap -p80 --script ipidseq < ip>/24

#nmap -p80 --script ipidseq -iR 1000

Possible candidates will return the text incrementally in the script's output section:

To launch an idle scan, open your terminal and type the following command:

#nmap -Pn -sI <zombie host> <target>

I have already discussed idle scan in the previous part. Please go through it.

Idle scanning should work if the zombie host meets the previously-discussed requirements. If something did not work as expected, the returned error message should give us an idea of what went wrong.

Timing options for Nmap

What are timing options and why?

As a pentester we are using timing options for Nmap, but we should know why we are using timing options and why.

When we are doing Nmap many times we should come up across a firewall which may block our request for a certain time response. To speed up Nmap scanning and for good performance we should use timing options.

These timing options can be used to speed up or slow down scanning operations, depending on our needs.

When scanning a large number of hosts on a fast network, we may want to increase the number of parallel operations to get faster results. Alternatively, when scanning slow networks (or across the Internet) you may want to slow down a scan to get more accurate results or to evade intrusion detection systems. Below are some timing options for Nmap.

Timing parameter

By default when we scan using Nmap it is scanning in seconds. But we can further increase the performance by setting up timing format. Nmap can be used to with the following timing parameters:

m-minutes

s-seconds

ms-miliseconds

h-hours

Sometimes while choosing timing options, we may be confused about how much time we will set for the scanning. To resolve these issues, Nmap offers a variety of timing options for scanning as below.

Commands: nmap -T[0-5] [target]

There are six templates (numbered 0-5) that can be used to speed up scanning (for faster results) or to slow down scanning (to evade firewalls).

0-paranoid

1-sneaky

2-polite

3-normal

4-aggressive

5-insane

With 0 option:

With 0 option we can do a paranoid scan for Nmap, which is a very slow scanning option so that the firewall or IDs are not able to block that request and will decrease the noise for the Nmap probe.

Command:nmap –T0 target

With 1 option:

The sneaky option is used for firewall bypass or IDS evade options.

Nmap –T1 target

While -T0 and -T1 may be useful for avoiding IDS alerts, they will take an extraordinarily long time to scan thousands of machines or ports.

With 2 option:

This is used for the polite option, which is to interfere with the target system.

Polite mode slows down the scan to use less bandwidth and target machine resources.

Nmap –T2 target

With 3 option:

This is a normal scan, as every time Nmap uses this template as a default scan method.

Nmap –T3 target

With 4 and 5 option:

The t4 and t5 option is a very fast and aggressive scan.

Aggressive (4) mode speeds scans up by making the assumption that you are on a reasonably fast and reliable network. Finally, insane mode (5) assumes that you are on an extraordinarily fast network or are willing to sacrifice some accuracy for speed.

Nmap –T4 target

Nmap –T5 target

Parallel option

As a pentester we should not waste our time by scanning one by one. Instead we can do optimization by scanning many at a time. Nmap does this by dividing the target IP space into groups and then scanning one group at a time. In general, larger groups are more efficient. The downside is that host results can't be provided until the whole group is finished.

There are two options for Nmap to do parallelism, like min and max.

Min:

The --min-parallelism option is used to specify the minimum number of parallel port scan operations.

nmap --min-parallelism [number of operation] [target]

While manually setting the --min-parallelism option may increase scan performance, setting it too high may produce inaccurate results.

Max:

The –max-parallelism operation is used to specify the maximum number of parallel port scan operations.

Nmap –max-pallelism [number of operation] [target]

Host group size options:

Nmap has the ability to port scan or version scan multiple hosts in parallel. It is used for setting the number of hosts in the targets. It has also two options, max and min.

Max:

The --max-hostgroup option is used to specify the maximum number of targets Nmap should scan in parallel.

nmap --max-hostgroup [number] [targets]

Min:

The --min-hostgroup option is used to specify the minimum number of targets Nmap should scan in parallel.

nmap --min-hostgroup [number] [targets]

Nmap will perform scans in parallel to save time when scanning multiple targets such as a range or entire subnet. By default, Nmap will automatically adjust the size of the host groups based on the type of scan being performed and network conditions. By specifying the --min-hostgroup option, Nmap will attempt to keep the group sizes above the specified number.

The max option is helpful if you want to reduce the load on a network or to avoid triggering any red flags with various network security products.

RTT TIME-OUT

In the TCP connection, RTT or Round Trip Timeout is a measurement for timeout value for the sliding window protocol in the communication, and which depends on the below points.

If the timeout value is too small, the source will time out too fast, resulting in unnecessary retransmissions. On the other hand, if the timeout value is too large, the source will take too long to recover from errors.

For details regarding this follow the link:

http://www.pcvr.nl/tcpip/tcp_time.htm

Nmap maintains a running timeout value for determining how long it will wait for a probe response before giving up or retransmitting the probe. This is calculated based on the response times of previous probes.

During the scan, Nmap's internal processes calculate these response time values:

srtt – Smoothed Round Trip Time – This estimate of response time is based on the observed traffic between the Nmap station and the remote device. This value is provided in microseconds.

rttvar – Round Trip Time Variance – Network communication can be very unpredictable, and Nmap compensates for this uncertainty by creating a range of timeout values. If a response isn't received within this variance, then Nmap concludes that a response isn't likely to ever appear.

Initial option for Nmap

It controls initial timeout for network response:

nmap --initial-rtt-timeout [time] [target]

Increasing the time value will reduce the number of packet retransmissions due to timeouts. By decreasing the value we can speed up scans, but we have to do so with caution. Setting the RTT timeout value too low can negate any potential performance gains and lead to inaccurate results.

Max options for RTT value

The --max-rtt-timeout option is used to specify the maximum RTT (Round-Trip Time) timeout for a packet response.

nmap --max-rtt-timeout [time] [target]

Nmap dynamically adjusts RTT timeout options for best results by default. The default maximum RTT timeout is 10 seconds. Manually adjusting the maximum RTT timeout lower will allow for faster scan times (especially when scanning large blocks of addresses). Specifying a high maximum RTT timeout will prevent Nmap from giving up too soon when scanning over slow/unreliable connections. Typical values are between 100 milliseconds for fast/reliable networks and 10000 milliseconds for slow/unreliable connections.

Max retries

This option is used to perform a maximum number of probes by Nmap for pentesting.

nmap --max-retries [number] [target]

By default, Nmap will automatically adjust the number of probe retransmissions based on network conditions. The --max-retries option can be used if we want to override the default settings or troubleshoot a connectivity problem. Specifying a high number can increase the time it takes for a scan to complete, but will produce more accurate results. By lowering the --max-retries we can speed up a scan, although we may not get accurate results if Nmap gives up too quickly.

The TTL option

Every time doing pentesting while doing reconnaissance we came across a TTL value, that is time to live value option. But we should know what a TTL value is.

TTL

Time To Live is a value in an Internet Protocol (IP) packet that tells a network router whether or not the packet has been in the network too long and should be discarded. From the perspective of a pentester, a TTL value can help to determine a lot of information about a target.

Nmap can be used as great measure to find all hosts with regards of TTL value. With the help of TTL value, Nmap will do a more comprehensive and reliable scan against the target.

The --ttl option is used to specify the TTL (time-to-live) for the specified scan (in milli seconds).

nmap --ttl [time] [target]

Packets sent using this option will have the specified TTL value. This option is useful when scanning targets on slow connections where normal packets may time out before receiving a response.

Host timeout option:

The --host-timeout option causes Nmap to give up on slow hosts after the specified time.

nmap --host-timeout [time] [target]

A host may take a long time to scan if it is located on a slow or unreliable network. Systems that are protected by rate limiting firewalls may also take a considerable amount of time to scan. The --host-timeout option instructs Nmap to give up on the target system if it fails to complete after the specified time interval. In the above example, the scan takes longer than one minute to complete (as specified by the 1m parameter), which causes Nmap to terminate the scan. This option is particularly useful when scanning multiple systems across a WAN or Internet connection.

Minimum scan delay

The --scan-delay option instructs Nmap to pause for the specified time interval between probes.

syntax: nmap --scan-delay [time] [target]

Some systems employ rate limiting, which can hamper Nmap scanning attempts. Nmap will automatically adjust the scan delay by default on systems where rate limiting is detected. In some cases we need our own scan delay if any rate limiting or IDS are in the actions.

Maximum scan delay

The –max-scan-delay is used to specify the maximum amount of time Nmap should wait between probes.

syntax: nmap --max-scan-delay [time] [target]

The --max-scan-delay option can be used to provide an upper limit to the amount of time between probes. This can speed up a scan, but comes at the expense of accurate results and added network stress.

Minimum packet rate

The --min-rate option is used to specify the minimum number of packets Nmap should send per second.

syntax: nmap --min-rate [number] [target]

Maximum packet rate

The --max-rate option is used to specify the maximum number of packets Nmap

should send per second.

syntax: nmap --max-rate [number] [target]

In the example above, specifying --max-rate 30 instructs Nmap to send no more than 30 packets per second. This can dramatically slow down a scan but can be helpful when attempting to avoid intrusion detection systems or a target that uses rate limiting.

Defeat reset rate limits

The --defeat-rst-ratelimit is used to defeat targets that apply rate limiting to RST (reset) packets.

syntax: nmap --defeat-rst-ratelimit [target]

The --defeat-rst-ratelimit option can be useful if you want to speed up scans on targets that implement RST packet rate limits. It can, however, lead to inaccurate results, and as such, it is rarely used.

This is the end of the document. I will cover "Evading Firewall, Pentesting with Nmap, Web Service Auditing, Web Application Pentesting, Nmap Script Engine development" in the upcoming installment.

FREE role-guided training plans

Get 12 cybersecurity training plans — one for each of the most common roles requested by employers.

Download Now

Sources

Hacking Nmap

Nmap man performance