How Does a VPN Work?

When a packet is transmitted from a client, it sends it through the VPN router or gateway, which adds authentication and optionally encrypts the data. IPsec has various modes. The most commonly used configuration uses Tunnel Mode with Encapsulating Security Payload (ESP). Some proprietary VPN clients use Transport Mode with ESP, despite the difficulties when Transport Mode is deployed with NAT.

The deployment of Authentication Header (AH) is uncommon these days and is mostly used between core routers where the overhead of ESP is undesirable and encryption is not required. ESP as deployed by Red Hat Enterprise Linux always includes authenticated headers, although older versions and other racoon software based implementations are sometimes mistakenly deployed with ESP + AH, resulting in a double authentication layer.

The receiving VPN router strips the header information, decrypts the data, and routes it to its intended destination (either a workstation or other node on a network). Using a network-to-network connection, the receiving node on the local network receives the packets already decrypted and ready for

processing. The encryption and decryption process in a network-to-network VPN connection is transparent to a local node.

With such a heightened level of security, an attacker must not only be able to intercept a packet, but needs to be able to decrypt the packet as well. Intruders who employ a man-in-the-middle attack between a server and client need access to at least one of the private keys for authenticating sessions.

Additionally, obtaining such a private key only compromises current traffic and not traffic sent encrypted in the past. Because they employ several layers of authentication and encryption, VPNs are a secure and effective means of connecting multiple remote nodes to act as a unified intranet.

2.7.2. Openswan

Openswan is an open source, user space IPsec implementation available in Red Hat Enterprise Linux.

It employs the key establishment protocol IKE (Internet Key Exchange) v1 and v2, implemented as a user-level daemon. Manual key establishment is also possible via ip xfrm commands, however this is not recommended. Openswan interfaces with the Linux kernel using netlink to transfer the encryption keys. Packet encryption and decryption happen in the Linux kernel.

Cryptographic Support

Openswan uses the NSS (Network Security Services) cryptographic library, which is required for FIPS security compliance. More information on the FIPS (Federal Information Processing Standard) can be found in Section 8.2, “Federal Information Processing Standard (FIPS)”.

Installation

Run the yum install openswan command to install Openswan.

2.7.2.1. Configuration Locations

This section lists and explains important directories and files used for configuring Openswan.

/etc/ipsec.d - main directory. Stores Openswan related files.

/etc/ipsec.conf - master configuration file. Further * .conf configuration files can be created in /etc/ipsec.d for individual configurations.

/etc/ipsec.secrets - master secrets file. Further * .secrets files can be created in /etc/ipsec.d for individual configurations.

/etc/ipsec.d/cert* .db - Certificate database files. The old default NSS database file is cert8.db. From Red Hat Enterprise Linux 6 onwards, NSS sqlite databases are used in the cert9.db file.

/etc/ipsec.d/key* .db - Key database files. The old default NSS database file is key3.db. From Red Hat Enterprise Linux 6 onwards, NSS sqlite databases are used in the key4 .db file.

/etc/ipsec.d/cacerts - The old Location for Certificate Authority (CA) certificates. It is

recommend to import CA certificates into the NSS database files. This location will be obsoleted in the next version of Red Hat Enterprise Linux.

/etc/ipsec.d/certs - The old location for machine certificates. It is recommend to import

machine certificates into the NSS database files. This location will be obsoleted in the next version of Red Hat Enterprise Linux.

/etc/ipsec.d/policies - Opportunistic Encryption groups policies. These are not in use in Red Hat Enterprise Linux 6.

/etc/ipsec.d/nsspassword - NSS password file. This file does not exist by default, and is required if the NSS database in use is created with a password.

Global Configuration Parameters

This section lists some of the configuration options available for the global config setup section in /etc/ipsec.conf.

protostack - defines which protocol stack is used. The default option in Red Hat Enterprise Linux 6 is netkey. Other valid values are klips and mast. These alternative IPsec kernel stacks require a third-party kernel module which is not supported by Red Hat Enterprise Linux. Alternative stacks are mostly in use with embedded Linux devices.

nat_traversal - defines if NAT-traversal (RFC 3947) for connections are accepted. Default is yes.

dum pdir - defines the location for core dump files. Changing this will require changing the SELinux policies to match the new location.

nhelpers - defines the number of helper threads used for cryptographic operations.

virtual_private - subnets allowed for the client connection. Ranges that may exist behind a NAT router through which a client connects. This should contain the RFC 1918 address space with an exception for any LAN range used by the server.

plutorestartoncrash - set to yes by default.

plutostderrlog - When enabled, the log file name to use instead of the syslog facility.

Connection Specific Parameters

This section lists some of the configuration options available per connection in a conn section.

auto - defines the startup profile for this connection. The default is ignore. Other valid values are start (initiate), add (respond-only) and route (Initiate on-demand).

type - defines the type of tunnel policy used. The default is tunnel for Tunnel Mode. Other valid values are transport for Transport Mode (used with L2TP) and passthrough which bypasses IPsec completely.

authby - defines the type of authentication used. The default is rsasig for raw RSA keys. . Other valid values are secret for Pre-Shared Key (PSK) and never for pass-through connections.

rekey - defines whether the connection should rekey when it reaches its expiry time. The default is yes and is usually used for static VPNs. The value no is mostly used for supporting dynamic VPN clients.

Further details about Openswan configuration can be found in the ipsec.conf(5) and ipsec.secrets(5) manual pages.

2.7.2.2. Commands

This section provides examples of commands used for Openswan. Note that except for checking the status of a service, these commands must be run as root.

Service Commands for Openswan

service ipsec start service ipsec stop service ipsec status

Loading and Unloading a Connection

ipsec auto --add connection_name ipsec auto --delete connection_name Initiating, On-demand and Terminating a Connection

ipsec auto --up connection_name

ipsec auto --route connection_name ipsec auto --down connection_name

Checking the IPsec Kernel Subsystem ip xfrm policy

ip xfrm state ip -s xfrm monitor

Checking the IPsec User Space Subsystem

ipsec auto --status ipsec verify

Raw RSA for IPsec

To enable an IPsec host to use raw RSA keys (without a Certificate Authority), the NSS database must be initialized and a new raw RSA key generated.

Note

If the Openswan IPsec service has been started before initializing the NSS database, you must shutdown the IPsec service and remove the * .db files in /etc/ipsec.d/ before you can add raw RSA keys or certificates to the NSS database.

certutil -N -d /etc/ipsec.d

Note

Commands that generate raw RSA keys depend on the system's entropy. Depending on system activity and key size, these commands can take several minutes to complete.

Generate raw RSA keys:

ipsec newhostkey --configdir /etc/ipsec.d --output /etc/ipsec.d/name-of-file.secret

List raw RSA keys for use in IPsec configuration files ipsec showhostkey --left | --right List raw RSA keys for publication in DNS

ipsec showhostkey --ipseckey

Configuring Certificates with IPsec

Creating and maintaining a Certificate Agency should be done on a separate machine. Only the created PKCS #12 files need to be transferred off this machine onto the IPsec servers. Exporting the PKCS #12 (.p12) files requires the CA administrator to use a password to encrypt the .p12 file. Importing these files requires the system administrator to know this password. This allows the .p12 files themselves to be safe to email, provided the password is shared via a different method, such as a phone call.

Create a directory in your home directory to hold the certificates and initialize the database:

mkdir ~/CAstore

certutil -N -d ~/CAstore

Create a (self-signed) CA certificate:

certutil -S -k rsa -n ca.example.com -s \ "CN=ca.example.com" -w 12 -t

"C,C,C" -x -d ~/CAstore

Create a host certificate signed by this CA:

certutil -S -k rsa -c ca.example.com -n west.example.com -s

"CN=west.example.com" -w 12 -t "u,u,u" -d ~/CAstore

Create an export file in PKCS #12 format for transfer to a target machine:

pk12util -o west.p12 -n west.example.com -d ~/CAstore

Import an exported machine certificate to the IPsec NSS database:

pk12util -i west.example.com.p12 -d /etc/ipsec.d

List the contents of the NSS database (IPsec gateway or CA):

certutil -L -d /etc/ipsec.d certutil -L -d ~/CAstore