IPsec Protocol OverviewIPsec is a framework for security that operates at the Network Layer by extending the IP packet header (using additional protocol numbers, not options). This gives it the ability to encrypt any higher layer protocol, including arbitrary TCP and UDP sessions, so it offers the greatest flexibility of all the existing TCP/IP cryptosystems. Flexibility, however, often comes at the price of complexity, and IPsec is not an exception. Configuring which addresses and ports to encrypt using which IPsec options often begins to look like configuring packet filtering, then add in the additional complexities of key management. While conceptually simple, setting up IPsec is much more complex that installing SSH, for example.
IPsec also has the disadvantage of requiring operating system support, since most O/S kernels don't allow direct manipulation of IP headers. This presents legal issues, since cryptographic software is restricted by many governments. Linux IPsec support (the FreeS/WAN project), for example, isn't included in the standard kernel distribution for this reason, and has to be applied as an add-on. Furthermore, putting the cryptography in the kernel isolates it from the application, making it more difficult to code crypto-aware software. Using SSL, for example, simply requires linking a library into the application and allows the application to easily query what certificates have been used to authenticate a client (for example). Doing the same thing with IPsec requires the application to query the kernel using some kind of API.
Figure 1. IPsec Document Roadmap
For example, one SA might protect all traffic through a gateway, while another SA would protect all traffic to a particular host. The packets finally routed across the network would be encapsulated in an SA bundle consisting of both SAs. The other side of the connection could be identical in design, consisting of a gateway implementing a tunnel SA, followed by a host implementing a transport SA, or the entire bundle could be terminated in a single host, which would then implement both SAs.
A common use of IPsec is the construction of a Virtual Private
Network (VPN), where multiple segments of a private network are linked
over a public network using encrypted tunnels. This allows applications
on the private network to communicate securely without any local
cryptographic support, since the VPN routers perform the encryption and
decryption. IPsec is well suited for this environment, more so than
tunneling PPP over SSL or SSH, since it operates directly on the IP
packets and preserves a one-to-one correspondence between packets
inside and outside the network. In the case of tunneling PPP over
an encrypted TCP connection, any packet loss in the public network
would trigger a TCP retransmission, stalling the link until the packet
was delivered. In particular, running Voice Over IP (VoIP) traffic
through a TCP/PPP tunnel would largely defeat the RTP protocol used
for VoIP; IPsec is better suited in this case.