Creating a VPN appliance with Ubuntu and Openswan

Make an enterprise-ready, scalable VPN appliance on Ubuntu Linux via Openswan with this expert's instructions.

While Virtual Private Networks (VPNs) are great providers for secure communication over networks, they are often...

very hard to set up. VPN appliances erase that problem, but can be expensive. With a do-it-yourself VPN appliance, you erase both problems.

This article will guide readers in installing and configuring a Linux VPN appliance that will allow clients running Linux, Windows XP SP2, Vista and OS X (10.3+) to connect with their native clients. Any IT admin can take on this project, as it doesn't require any developer experience.

Also, this isn't just a VPN appliance for small organizations. This VPN appliance is far more scalable than something from, say, Cisco. Consider that a Cisco VPN may only have a 200 MHz processor, but with this project you are using a server that may have a Xeon processor. Of course, network cards present a bottleneck, but you can always drop in some quad-port PCI NICs.

VPN basics
As the name implies, a VPN creates a virtual, private network between two end-points. These end-points can be a computer-to-router, a router-to-router, or even a computer-to-computer (Microsoft Active Directory Internet Protocol Security, or IPsec, implementations are largely computer-to-computer.) The "virtual" in VPN describes the type of network that is being created: a virtual network not defined by a dedicated physical medium. This type of virtualization is analogous to virtual local area networks (VLANs). "Private" refers to the private communication between the two aforementioned end-points, which cannot be viewed by any device on a network between the two network devices participating in the VPN connection. However, the privacy of the Ethernet frames before they enter the VPN and after they exit cannot be guaranteed.

There are several types of VPNs:

  • PPTP A Point to Point Tunneling Protocol (PPTP) VPN is a proprietary standard created by Microsoft (and other companies). It uses what is widely thought to be questionable encryption to create an encrypted transport for a Point to Point Protocol (PPP) connection between two end points. PPTP VPNs are highly discouraged due to their weak security, but are often still in use because of how easy it is to create them.
  • SSL Secure Socket Layer (SSL) VPNs use SSL to encrypt network traffic via SSL tunnels. SSL VPNs use certificates much like IPsec.
  • Pure IPsec Pure IPsec VPNs create IPsec encrypted tunnels between two network end points. In a pure IPsec tunnel, the entire IP packet is being encrypted
  • L2TP over IPsec While pure IPsec VPNs create tunnels between two network end points, IPsec can also be used to transport other types of generic routing encapsulated (GRE) protocols. When operating in transport mode, IPsec only encrypts the data portion of the IP packet. The data portion itself is Layer 2 Transport Protocol (L2TP). L2TP is just another level of GRE that is used to create a PPP connection between the two network end points (much like PPTP). Although this type of VPN creates additional authentication and encryption overhead compared to an SSL or Pure IPsec VPN, this article describes how to create an L2TP over IPsec VPN because it is the type supported out of the box by the four major operating systems (OSs) today: Windows XP, Windows Vista, Linux, and OS X.

Selecting the VPN server and OS
The first step to creating your VPN appliance is finding a server to use. Any x86 or x86_64 OS will do. For the purposes of this guide I used a white-box x86_64 server running Ubuntu 7.04 x86_64 (Feisty Fawn). A newer version of Ubuntu is available, 7.10 Gutsy Gibbon, but its stability has received questionable reviews, so I am sticking with Feisty until the next version, Herdy, since it will be a Long-Term Support (LTS) version that includes 5 years of support on the server side of things. It is always a good idea to secure the OS as tightly as possible, especially since this server is going to be acting as your gateway to your internal network. There is more information on how to secure Ubuntu on SearchEnterpriseLinux.com.

Downloading the VPN software
Stability aside, selecting Feisty Fawn as a server distribution has a few downsides. The latest Openswan package available via the Ubuntu repositories is 2.4.6, which includes only l2tpd, a great L2TP daemon, but not very useful with regard to development. To that end,  I have packaged the latest release of Openswan, 2.4.11, as an Ubuntu-ready Debian-style package. You can also find the Debian package for the latest release of xl2tpd. The creators of Openswan, recognizing that development on l2tpd seems to have halted, have forked the project and call their fork xl2tpd. Download the software and place the packages in '/usr/local/src/vpn' so you can easily remember where they are (you will need to create this directory.)

Openswan installation and configuration
You need to install three pieces of software in order to facilitate a VPN appliance. The first piece, a PPP daemon, is already installed for you by default with even the most basic server installation from Ubuntu (we'll get to that later.) After downloading the other two pieces of software you need, place them in '/usr/local/src/vpn'. Go ahead and install Openswan 2.4.11 and xl2tpd 1.1.12 by typing:

sudo dpkg --install /usr/local/src/vpn/*.deb

The Openswan package comes with an installation wizard. It does not really matter what you choose since we will be overwriting the default values anyway. Hopefully, both packages installed without errors. If for some reason you encountered errors, feel free to  follow the steps that Nate Carlson provides at his website. Even with Nate's fantastic site, creating user certificates can be a bit of a pain, so to make it easier I have provided a script that will help.

Creating the certificate for the VPN server itself can be extremely tricky, especially if you want to make sure that it supports OS X clients. OS X places very specific rules on a VPN server's certificate. These rules are documented at Jacco's website. If you use the script I provided above in conjunction with Jacco's instructions you will do just fine.

Once the certificates are created, place them in either your /etc/ssl directory or in the Openswan certificate directories. I store the certificates inside the standard /etc/ssl directory. I recommend this method since other applications that are compiled against libssl will look in that directory (its sub-directories actually) for certificates.

Now that the certificates are in place, it is time to configure Openswan. Open up /etc/ipsec.conf with your favorite text editor. Note that the real ipsec.conf file does not support comments inside stanzas, such as the ones below which I provide. Any line that begins with "config" or "conn" is the start of a stanza and does not end until an empty line. The other thing to know is that the indentations beneath these section headers are required or the configuration file will not work.

    # This line is required or else the IPsec daemon will complain that the configuration file is out of date. Version 2.0 # This is the only global configuration section in the ipsec.conf file. Its settings apply to the entire configuration file. config setup interfaces=%defaultroute # This line tells the IPsec daemon to encapsulate ISAKMP packets inside universal datagram protocol (UDP) packets on port 4500 in order to traverse network address translation (NAT) devices. nat_traversal=yes # This line tells the IPsec daemon which NATd subnets it is allowed to accept connections from. For example, the ! character in front of means that the IPsec daemon should not authorize any incoming connections from clients with an IP address in this range. The reason for this is that this is the subnet that the internal network behind the VPN is operating on. virtual_private=%v4:,%v4:,%v4:! # Setting this value will cause debug output to be generated. plutodebug=none # This is where to log the IPsec daemon debug output. plutostderrlog=/var/log/pluto.log # This is the default connection. Its settings are applied to all of the subsequent connections. conn %default # Authenticate incoming connections by X.509 secret. You would set this to 'secret' to authenticate connections by a pre-shared key. authby=rsasig keyingtries=1 compress=yes disablearrivalcheck=no leftrsasigkey=%cert # Path to the server's VPN certificate. leftcert=/etc/ssl/mycerts/vault.lostcreations.com.crt rightrsasigkey=%cert # This connection is used for "roadwarriors. A roadwarrior is a roaming network device, such as a laptop.conn roadwarrior type=transport # This forces all connections to act as if they are using NAT-T. forceencaps=yes # This is the IP address of the VPN server. left= # This is the host name of the VPN server. [email protected] # Because this VPN server is not a router in my network topology I must tell the VPN server to send all incoming packets to my router, the "next hop" for the packets. Thus this value is my router's IP address. leftnexthop= # This is port on which the IPsec daemon should connect to the xl2tpd daemon. leftprotoport=17/1701 right=%any rightsubnet=vhost:%no,%priv # This value must be 17/0 to support OS X clients. rightprotoport=17/0 pfs=no # Turn on Dead Peer Detection to drop stale IPsec connections. dpddelay=40 dpdtimeout=130 dpdaction=clear # Automatically bring up this IPsec connection when the IPsec daemon starts. auto=add # Include an external configuration file that tells the IPsec daemon to disable opportunistic encryption. include /etc/ipsec.d/examples/no_oe.conf


The best way to learn the options in the ipsec.conf file is to read the main page for ipsec.conf. It explains every option in detail. One other note when configuring Openswan: If something is not working the way you think it should, turn on debug logging and check the logs. Also, you can run the command 'ipsec verify' and it will do a sanity check on your system to see if it is properly set up to be a VPN server. For example, because my VPN server was not previously a part of my network topology, it did not have packet forwarding turned on in the Kernel. Before I ran 'ipsec verify' I could connect to my VPN but I could not travel my internal network. The verify command made me aware that packet forwarding was not turned on.

Configuring xl2tpd
The next step to configuring the VPN is to configure xl2tpd, our L2TP server. An L2TP server is necessary because the major OS VPN clients expect to see L2TP in their VPN stack. Eventually, the native clients will all support pure IPsec VPNs, but until that time comes, L2TP is necessary. As mentioned earlier, xl2tpd is xelerance's fork of l2tpd, with new security patches and features. They actually make it very easy to configure xl2tpd since you can use your old l2tpd.conf files. As with Openswan, here are the file locations for xl2tpd:

 /etc/init.d/xl2tpd - This is the service initialization file /etc/xl2tpd/xl2tpd.conf - This is the configuration file. /usr/sbin/xl2tpd - This is the daemon.

To get xl2tpd up and running we only have to edit its configuration file. Here is an example:

    [global] # Tell the daemon to only bind to the local network interface. This is a must, so unencrypted, unauthenticated clients cannot connect to the L2TP daemon. listen-addr = [lns default] # The range of IP addresses to dish out to connecting clients. ip range = # The IP address of the L2TP server local ip = # We are refusing authentication since the IPsec daemon is doing that for us. require authentication = no # The name of the VPN host. hostname = vault.lostcreations.com # The name of the L2TP server. name = vault.lostcreations.com # Set to "yes" to turn on debugging. Output will be logged to /var/log/ debug. ppp debug = no # The path to the PPP options file. pppoptfile = /etc/ppp/options.l2tpd length bit = yes


Configuring pppd
Configuring xl2tpd was easy enough. The final step is to configure PPP. PPP comes with most Linux distributions. Ubuntu Linux's PPP package places files in these locations:

 /etc/ppp/ - The PPP configuration directory. /usr/sbin/pppd - The PPP binary used to initiate client and server connections.

The only file we need to configure is /etc/ppp/options.l2tpd. This file does not exist yet, so create it and then use the following configuration options:

 # Dump all the variables to the debug log to ensure that the configuration was picked up properly. dump # Output debugging information to /var/log/debug debug # Do not support BSD compression. nobsdcomp passive lock # Allow all usernames to connect. name * proxyarp ipcp-accept-local ipcp-accept-remote lcp-echo-failure 3 lcp-echo-interval 5 nodeflate # Do not authenticate incoming connections. This is handled by IPsec. noauth refuse-chap refuse-mschap refuse-mschap-v2 # Set the DNS servers the PPP clients will use. ms-dns mtu 1450 mru 1450

Launching your VPN
Before the VPN can go live, you need to restart all your services. Type:

 /etc/init.d/xl2tpd restart /etc/init.d/ipsec restart

Congratulations! You have a working VPN appliance. If you have any questions or concerns do not hesitate to the definitive site for information creating VPNs with Openswan. Good luck creating your VPN.

About the author: Andrew Kutz is a Microsoft Certified Solutions Developer (MCSD) and a SANS/GIAC Certified Windows Security Administrator (GCWN). An avid fan of .NET, open source, Terminal Services, and coding, Andrew's current focus is on virtualization. Andrew graduated from the University of Texas at Austin with a B.A. in Ancient History and Classical Civilization and currently lives in Austin, Texas with his wife Mandy and their two puppies, Lucy and CJ.

Dig Deeper on Linux servers