IPsec Tunnel Mode vs. Transport Mode

IPsec Tunnel Mode and Transport Mode are both security protocols used to protect data sent over a network. They differ in how the data is protected.

IPSec Protocols: AH vs ESP

IPsec AH Protocol

IPsec Authentication Header (AH) is a security protocol used to protect data sent over a network. Its core focus is around data integrity and authentication. It is also responsible for authenticating IP packets and helps protect against network attacks. 

IPsec ESP Protocol

The IPsec Encapsulating Security Payload (ESP) protocol protects data confidentiality and data origin authentication. Both IPsec AH and IPsec ESP focus on encryption with the difference coming in the use of both protocols in the IPsec modes which we will discuss below. IPsec Tunnel mode and IPsec Transport mode. 

In order to get a better understanding of the differences in each IPsec transport mode, let’s first discuss the use cases for them.

Looking to secure your remote workforce?

IPSec Tunnel Mode

IPsec AH Tunnel Mode

The IPsec AH tunnel mode sets up a secure connection between two communication endpoints on the internet. This is the most common mode to use when connecting to a VPN server. While the AH protocol establishes a VPN tunnel without encrypting data, it instead provides integrity of the data packets.

IPsec ESP Tunnel Mode

The IPsec ESP tunnel mode encrypts and encapsulates IP packets while also providing authentication and integrity. This protocol is used by VPN tunnels to see if data packets have been tampered with while in transit. This allows VPN connections to be routed through untrusted networks while maintaining encrypted data packets.

How to Configure IPsec Tunnel Mode

In order to configure IPsec tunnel mode, you will first need to use a different protocol such as IKE (Internet Exchange Key) to negotiate the parameters that will be used in order to secure the communication between peers. The IKE setup process is broken down into 2 phases: 

• IKE Phase 1: The initial IKE phase establishes a secure tunnel between channels. The main purpose in the first step is to authenticate IPsec peers and to negotiate security associations (SA). 
• IKE Phase 2: Once a security association has been established, the next step is to negotiate authentication and encryption, thus encrypting the entire packet which is then included in the payload or the transmission of data from the intended message.

IPSec Transport Mode

IPsec AH Tunnel Mode

IPsec AH transport mode is a security protocol used to protect data through your network, but it doesn’t make a secure connection. It encrypts the data being sent without checking for integrity or authentication, which makes it faster than IPSec AH Tunnel Mode. However, it is far less secure.

IPsec ESP Transport Mode

IPsec ESP transport mode secures data sent over a network, providing privacy by encrypting it, and this protocol provides authentication and integrity. It is used by VPN tunnels to ensure that data is secured while in transit without having to establish a secure connection between two points on the internet. 

The final destination in ESP transport mode is typically the host. The other point to take into consideration is that ESP transport mode encrypts the data only and not the original headers.

How to Configure IPsec Transport Mode

IPsec transport mode secures traffic from one system to another. There is also a two-step configuration process similar to tunnel mode.

• IKE Phase 1: The key focus here is on the negotiation of the secure channel between two systems using an ISAKMP security association or Internet Security Association and Key Management Protocol. 
• IKE Phase 2: In this step, the IKE peers dynamically negotiate the authentication and encryption algorithms to secure the payload.

Transport mode is seen as less secure than tunnel mode because the IP header is not encrypted.

Looking to secure your remote workforce?

What is The Difference Between IPsec Tunnel and Transport Mode?

IPsec tunnel mode sets up a secure connection, while IPsec Transport Mode only encrypts the data being sent without establishing a secure connection.

In transport mode, the sending and receiving hosts establish a connection before exchanging data. In tunnel mode, a second IP packet is sent in a completely different protocol. This protects data packets from being inspected or modified in transit. 

The advantages of tunnel mode over transport mode are that it can work through Network Address Translation (NAT) and that the entire original IP packet is hidden. NAT maps a private IP address to a public IP address by modifying network address information in the IP header of packets across a traffic routing device while in transit.  

The major disadvantages of tunnel mode are additional overhead from encapsulation, an inability to defend against attacks on weak integrity protocols, and that transport mode may be more compatible with some firewalls. 

Examples of such attacks include SYN floods which is a type of distributed denial-of-service (DDoS) attack. SYN floods send massive requests to overwhelm a server, rendering the system unavailable to receive legit traffic. It also prevents the completion of the TCP three-way handshake between client and server needed for a secure connection. 

In general, tunnel mode is better when both endpoints are behind a NAT device, and transport mode is preferable when there is no NAT or if the network uses pre-NAT devices with address translation only at the IP packet level. In most cases, transport mode will provide better security with less overhead.

IPsec Tunnel Mode vs Transport Mode: When to Use Each Mode

In order to know when to use either tunnel mode or transport mode, you should consider where each endpoint is located in relation to the internet. If you are both behind the NAT device, then tunnel mode is better because it establishes a connection while transport mode simply encrypts packets.

However, if only one of the endpoints is behind the NAT device, you’ll need to use transport mode so both hosts can communicate securely with each other.

Both tunneling and transport mode encrypts data, but when implementing one over the other, you should consider whether there are NAT devices between two connected networks. If no NAT device exists, use transport mode. If a NAT or pre-NAT firewall exists, use tunnel mode.

For example, if you’re using a pre-NAT firewall and your endpoint is located in the same private network as the server, use transport mode. If your endpoint is located behind a NAT device or on a different network than the server you’ll be connecting to through IKEv2, then use tunnel mode.

Transport mode works best for firewalls that do not translate IP addresses in the packet header and for cases where transports mode is more compatible with certain firewalls.

Advantages and Disadvantages of IPsec Transport and Tunnel Mode

IPsec Transport Mode

The main advantage of IPsec transport mode is that it is more compatible with certain firewalls and it offers higher levels of security. In addition, transport mode does not require a secure connection to be established between two endpoints and has less overhead because it does not encapsulate packets.

The main disadvantage of IPsec transport mode is the difficulties it has with NAT traversal or UDP encapsulation. The User Datagram Protocol (UDP) is a technique of adding network headers to the packets and helps with load balancing to better distribute network traffic.

IPsec Tunnel Mode

The main advantage of IPsec tunnel mode is that it creates a secure connection between two endpoints by encapsulating packets in an additional IP header. Tunnel mode also provides better security over transport mode because the entire original packet is encrypted.

The main disadvantage of the IPsec tunnel mode is that it requires a secure connection to be established between two endpoints and tends to create more overhead because the entire original packet must be encapsulated. In addition, transport mode may perform better than tunnel mode on some types of networks and with certain firewalls.

In order to know which mode is best for you, consider your network environment. You might also want to consider an IPsec VPN to create encrypted tunnels and secure remote access to an entire network, whether on-premises or from corporate headquarters.

Perimeter 81’s IPsec VPN: The Next Level of Encryption & Security

Perimeter 81’s IPsec VPN enables organizations to work safely from anywhere in the world by establishing a secure connection between devices.

Perimeter 81’s IPsec VPN leverages the principles of Zero Trust to provide a stronger level of security across the network. This allows admins to create policies based on authentication factors such as Multi-Factor Authentication (MFA) and 256-bit encryption.

Instantly deploy your entire network with Perimeter 81’s IPsec VPN. See how radically simple it is for yourself. Request a demo today.