Incident response

Network traffic analysis for IR: TCP protocol with Wireshark

Howard Poston
January 14, 2020 by
Howard Poston

The Transmission Control Protocol (TCP) is one of the most commonly-used protocols on the internet. Unlike the User Datagram Protocol (UDP), TCP is not a “fire and forget” protocol. TCP tracks the packets that are sent, confirms that they are received and retransmits copies if necessary.

As a result, TCP is ideally suited to applications that need a high level of reliability in their communications channels. A common use of TCP is carrying hypertext transfer protocol (HTTP) packets that make requests for and serve web pages.

Learn Incident Response

Learn Incident Response

Get hands-on experience with incident response tools and techniques as you progress through nine courses.

TCP in Wireshark

TCP differs from other protocols since it is intended to provide reliable data transfer. As a result, a TCP communication is very formalized, using several different types of packets denoted by different TCP flags. A TCP communications channel is set up using the TCP handshake and provides a number of guarantees to the sender and recipient.

TCP flags

One of the primary differentiators between TCP and UDP packet is the use of flags in TCP. Like ICMP types and codes, TCP flags describe the purpose of the packet. The TCP flags are:

  • SYNchronization: Request a connection
  • ACKnowledgement: Acknowledges receipt of a packet
  • FINish: Gracefully closes a connection
  • ReSeT: Immediately terminates a connection
  • PuSH: Tells recipient to immediately process a packet (instead of buffering)
  • URGent: Process a packet before all other packets

Different types of packets are used at different times within the TCP session. Some are intended to be used a set time during the connection (like SYN and FIN), others are used throughout (like ACK) and the rest are used only in unusual scenarios (RST, PSH and URG).

TCP connections and use of flags is very formulaic. However, it is possible to abuse flags. A common reason to abuse flags is in scanning since different operating systems have different responses to certain errors.

TCP handshake

In TCP, the data intended for the final recipient is never sent in the first packet. Instead, the TCP users first go through the TCP handshake protocol to establish a channel before sending actual data. [CLICK IMAGES TO ENLARGE]

The steps in the TCP handshake are fairly straightforward, as shown in the screenshot from Wireshark above. First, the sender initiates the communication by sending a SYN packet, expressing their interest in connecting. The recipient responds with a SYN/ACK packet that expresses their interest as well and acknowledges receipt of the sender’s SYN packet. Finally, the sender sends an ACK packet to acknowledge the recipient’s SYN/ACK.

Now the TCP channel between the sender and recipient is initiated. At this point, the primary purpose of the TCP packets switches over from achieving TCP goals (i.e., establishing a connection) to being a carrier for another protocol. For example, the next packet after the TCP handshake may be a packet from the sender requesting an HTTP webpage.

TCP guarantees

The TCP protocol is primarily intended to provide certain guarantees to the higher-level protocol using it. These guarantees include packet ordering, reliability and error correction.


Another advantage that TCP has over other protocols is that it provides built-in support for packet ordering. While different packets may be sent in a certain sequence, they can travel over different paths in the network and reach the recipient out of order.

The image above shows a screenshot of Wireshark with packet sequence numbers indicated. As shown, the sender increments its sequence number for each packet. This way, the recipient can reorder packets as needed if they arrive jumbled.


Unlike UDP, TCP is not a “fire and forget” protocol. As shown above in the TCP handshake, each packet sent results in an acknowledgement from the other party. A lack of acknowledgement will result in a retransmission of the dropped packet, ensuring that the recipient gets a copy of all data that is transmitted by the sender.

The image above shows a Wireshark screenshot with the acknowledgement information marked. In packet 2, the ACK flag is set and the packet contains an acknowledgement number of one, which is one more than the sender’s previous sequence number (zero). This tells the sender that the packet with sequence number zero has been received and that the receiver is ready for the next packet. Packet 3 has a similar structure, with an incremented sequence number for the sender and acknowledging the receiver’s packet with sequence number of zero.

The acknowledgement of every packet in a TCP transmission provides a high level of reliability. It is impossible for a packet to be dropped without being noticed.

Error correction

Finally, TCP has integrated error correction functionality. While traversing the network, it is possible that a bit in the packet could have been flipped, changing the message that it contains. TCP error correction helps to identify if this has occurred.

As shown in the screenshot above, TCP implements error correction by embedding a checksum in each packet. The recipient can then verify the checksum to ensure that the packet was received correctly. If not, the recipient can request a retransmission.

TCP analysis for incident response

As one of the most common protocols on the internet, TCP can be used for carrying a wide variety of attack traffic. However, some attacks are performed at the TCP level itself, including using the protocol for scanning and DDoS amplification.


A common misuse of the TCP protocol is for scanning. The reason for this is that different operating systems respond in different ways when someone breaks the “rules” of a TCP connection. By observing how a system responds to something, it is possible to determine if a certain machine is up or down or if a port is open or closed.

The screenshot above is of a SYN or half-open scan in Wireshark. In this type of scan, the scanner sends SYN packets to the target. A SYN/ACK in response means that the port is open, while a closed port would result in a RST response. For open ports, the scanner will then send a RST packet, closing down the connection. If an organization only monitors and logs connections that complete the TCP handshake, this type of scan may go undetected.

The screenshot above is of an Xmas tree scan, named because the packet’s flags look “lit up” like an Xmas tree. This type of scan can be used to determine if a port is open, since RFC-compliant operating systems will send a RST packet if the port is closed and no response if it is open.

TCP reflection and DDoS amplification

As their name suggests, DDoS amplification is designed to amplify the impact of a Distributed Denial of Service (DDoS) attack. These attacks are designed so that the attacker can send a small amount of traffic to an “amplifier” and have a much larger volume of traffic sent to the intended target.

Most DDoS attacks use UDP, but attackers have recently been taking advantage of TCP reflection for DDoS amplification. In a TCP reflection attack, the attacker sends a TCP SYN packet to an amplifier while spoofing their IP address of that of the target. The amplifier, believing that the sender is trying to initiate a connection, will send a SYN/ACK packet to the target.

If the target does not respond as expected, the amplifier will continue retransmitting SYN/ACK packets, achieving rates of up to 5,000 packets per minute. As the target, TCP reflection DDoS amplification can be detected as SYN/ACK packets without a corresponding SYN. As the amplifier, the attack can be detected by systems within the network sending large volumes of SYN/ACK packets and receiving no response.

Conclusion: Investigating TCP traffic in Wireshark

TCP is a very structured protocol, which allows it to provide certain guarantees to the applications using it. When looking at TCP traffic in Wireshark, any deviation from the normal structure of a TCP conversation might be worth a closer look.

Learn Incident Response

Learn Incident Response

Get hands-on experience with incident response tools and techniques as you progress through nine courses.


Howard Poston
Howard Poston

Howard Poston is a copywriter, author, and course developer with experience in cybersecurity and blockchain security, cryptography, and malware analysis. He has an MS in Cyber Operations, a decade of experience in cybersecurity, and over five years of experience as a freelance consultant providing training and content creation for cyber and blockchain security. He is also the creator of over a dozen cybersecurity courses, has authored two books, and has spoken at numerous cybersecurity conferences. He can be reached by email at or via his website at