Tunneling, Crypto and VPNs

Dawid Czagan
April 23, 2015 by
Dawid Czagan

1. Introduction

The idea of Virtual Private Network (VPN) is to simulate a private network over a public network. A VPN tunnel can be used to securely connect LANs of the company over an insecure Internet (VPN gateways are responsible for making the connection secure).

Learn Applied Cryptography

Learn Applied Cryptography

Build your applied cryptography and cryptanalysis skills with 13 courses covering hashing, PKI, SSL/TLS, full disk encryption and more.

This article describes how tunneling and cryptography can be used to build VPN tunnels without going into the details of existing VPN protocols.

2. TCP/IP model and encapsulation

One needs to understand these topics first before tunneling is discussed. There are four layers in the TCP/IP model:

Layer 4: Application layer

Layer 3: Transport layer

Layer 2: Internet layer

Layer 1: Network access layer

From the point of view of the sender, the data goes through layers 4 → 1 (1 → 4 from the perspective of the receiver). L4PDU (Layer 4 Protocol Data Unit) is sent from the application layer to the transport layer. A TCP header is appended to L4PDU and L3PDU (Layer 3 Protocol Data Unit) is created. L3PDU is called segment. Then L3PDU is sent from the transport layer to the Internet layer. IP header is appended to L3PDU and L2PDU (Layer 2 Protocol Data Unit) is created. L2PDU is called datagram.

This simplified description shows that L3PDU (segment) becomes a part of the L2PDU (datagram). In fact, the segment is included in the datagram and this inclusion is called encapsulation. Then the datagram is appended with another header and L1PDU (Layer 1 Protocol Data Unit) is created. L1PDU is called frame. Finally the frame is sent via transmission medium in the form of zeros and ones.

From the perspective of the receiver, the exact reverse process occurs (layers 1 → 4) and is called four step decapsulation.

3. VPN tunnel

Normally the data of the application layer is encapsulated into the segment of the transport layer which is further encapsulated into the datagram of the Internet layer. Then the frame of the network access layer encapsulates the datagram and finally the bits are transferred via a physical medium. When VPN tunnel is concerned, one datagram (the internal one) is encapsulated in the another datagram (the external one).

This encapsulation is used to carry private addresses through the tunnel. We want to carry private IP addresses through the tunnel, because the goal is to connect local area networks (LANs) at both ends of the tunnel. That's why the external IP (the one which is not tunneled) is a public address used to connect to the VPN gateway and the internal IP (the one which is tunneled) is a private address.

Let's analyze the real world analogy of tunneling to better understand how it works. The car wants to drive from city C1 to city C2 and these cities are separated by the river. The car is loaded into the ship and transported from C1 to C2. This is exactly how the tunneling works. The internal datagram is tunneled inside another datagram that reaches the VPN gateway. Here the internal datagram is extracted and can be sent to another host. Although the private addresses are not routable in the Internet, they can be traversed using this approach.

There is one thing missing. We need to make the tunneling secure, and cryptography is used for this purpose.

4. Using crypto to secure the tunnel

The intention of this part of the article is to present briefly how crypto can be applied to make the tunnel secure without going into the cryptographic details.

First of all we want the authentication to be achieved. Digital certificates can be used for this purpose.

Moreover, the communication should be confidential so that unauthorized users can't see it. The confidentiality can be achieved by symmetric encryption. Before the symmetric encryption happens, the symmetric key needs to be securely distributed. Asymmetric encryption is used for the purpose of key distribution. Let's assume that A is communicating with B. The symmetric key is generated by A, encrypted with the public key of B and sent to B. Only B can decrypt it, because B is the only one that has the corresponding private key.

In addition to this, we want to be sure that the communication has not been modified. HMAC is used for this purpose (hash of the message sent and the symmetric key).

The symmetric key can be regenerated periodically. Then it is called a session key (randomly generated and valid only for one session). If an attacker learns the session key, then he can only decipher the messages sent after the last regeneration of the key and before the next regeneration. This is the way Forward Secrecy is achieved.

As far as symmetric encryption is considered, some encryption mode is needed to change the ciphertext in a random way in order not to weaken the encryption key. The solution is a cipher block chaining (CBC) mode of encryption.

5. Summary

Learn Applied Cryptography

Learn Applied Cryptography

Build your applied cryptography and cryptanalysis skills with 13 courses covering hashing, PKI, SSL/TLS, full disk encryption and more.

Remote work via VPN is a standard nowadays. VPN simulates a private network (secure) over the public one (insecure). TCP/IP model and encapsulation were presented first. Then it was described how tunneling works. Finally, we've seen how cryptography can be used to make the VPN tunnel secure.

Dawid Czagan
Dawid Czagan

Dawid Czagan (@dawidczagan) has found security vulnerabilities in Google, Yahoo, Mozilla, Microsoft, Twitter, BlackBerry and other companies. Due to the severity of many bugs, he received numerous awards for his findings.

Dawid is founder and CEO at Silesia Security Lab, which delivers specialized security auditing services with a results-driven approach. He also works as Security Architect at Future Processing.

Dawid shares his bug hunting experience in his workshop entitled "Hacking web applications - case studies of award-winning bugs in Google, Yahoo, Mozilla and more". To find out about the latest in Dawid's work, you are invited to visit his blog ( and follow him on Twitter (@dawidczagan).