Using a Linux L2TP/IPsec VPN server with Mac OS X and iPhone

eXTReMe Tracker
Last update: May 24, 2010



1.1 Introduction

I have made the following webpages on using L2TP/IPsec with Linux:

The page you are now reading describes how you can use the built-in VPN client of Mac OS X 10.3 ("Panther"), 10.4 ("Tiger") and Mac OS X 10.5 ("Leopard") with a Linux Openswan VPN server. Mac OS X 10.6 ("Snow Leopard") has not been tested by me personally but is expected to work. Panther and Tiger are no longer supported by Apple with security updates, so I would not recommend using these old versions on the Internet as VPN clients. If you are looking for information about the Mac's built-in L2TP/IPsec client, you can find some here. You can also find a few remarks about the L2TP/IPsec client included with the Apple iPhone. I will not cover the configuration of Windows 2003/2008 VPN servers or Mac OS X Servers for use with Mac based clients (contact Apple and/or Microsoft Support for that). If you are not familiar with setting up L2TP/IPsec on a Linux server, it's probably a good idea to start with reading this page. It provides information on setting up the Linux side. The other pages listed above contain specifics on several L2TP/IPsec clients which are available for Windows.

In addition to L2TP/IPsec, Mac OS X 10.3 and higher also supports pure IPsec (i.e. without L2TP). But for this feature you have to edit text files manually or use one of the third party graphical user interface (GUI) clients. Pure IPsec makes configuration on the Linux server much easier. If you can use pure IPsec, you don't have to jump through all the hoops as described here on this webpage. I refer you to the Openswan Wiki for more information on pure IPsec interoperability between Openswan and Mac OS X.

Apple's L2TP/IPsec implementation supports multiple authentication mechanisms. For "Machine authentication" (the IPsec part of the L2TP/IPsec protocol) there are basically two methods: Preshared Keys (PSKs) and X.509 machine certificates. The GUI of Mac OS X 10.4 and higher supports both of these methods but Mac OS X 10.3's GUI only supports PSKs. PSKs are easier to configure than certificates but certificates provide better security and certificates are they are recommended for supporting Road Warriors. So, Mac OS X 10.3 not supporting certificates for L2TP/IPsec is a pretty severe limitation.

Once the IPsec authentication succeeds, the next step is "User authentication" (the L2TP/PPP part of the protocol). Mac OS X 10.3+ supports a number of user authentication options, including MS-CHAP and RSA SecurID. This depends on the Mac OS X version that you are using. Mac OS X 10.3 supports only a small number of user authentication options and (the latest point updates for) Mac OS X 10.4 and higher support more.

Mac OS X 10.4.4 and higher support NAT traversal: both the official RFC 3947 standard and Microsoft's draft-02 implementation. So Mac OS X 10.4+ should interoperate with Cisco, Openswan, Windows 2003 and other servers when NAT-T is involved. But older versions of Mac OS X (10.3.x and 10.4.0-10.4.3) do not interoperate that well with other IPsec implementations when NAT is involved. A work-around for this problem has been added to Openswan 2.4.5+.

1.2 Author

The author of this document is Jacco de Leeuw. Corrections, additions, extra information etc. are much appreciated.



2. Contents
3. Background information

Mac OS X 10.3 and higher ship with an L2TP/IPsec client. The Mac's IPsec implementation is a fork based on KAME which is known to interoperate with Openswan. I also received a report from Chris Andrews that Mac OS X's VPN client interoperates with a setup that consists of the native IPsec implementation of the Linux kernel 2.6, plus l2tpd and  ipsec-tools (racoon).

Back to Contents



4. L2TP discussion

The big question of course is: why would you want to use L2TP with the Mac? L2TP/IPsec has the advantage that it is an official IETF standard. Furthermore, IPsec is generally considered to be more secure than PPTP. As Apple writes, "L2TP is Mac OS X Server's preferred VPN protocol due to its superior transport encryption and its ability to be authenticated via Kerberos". These are indeed valid points: a Mac client and an Openswan server can in principle agree upon the use of strong encryption (AES, SHA-1 etc.). I have not used Kerberos on the Mac so I can't comment on that.

Some users prefer Mac OS X's L2TP/IPsec client simply because it is free. Others may prefer a third-party client, because of the support that the third-party will provide.

Back to Contents



5. Installation (Mac OS X)

There is no installation required. The L2TP/IPsec client is installed by default on Mac OS X 10.3 and higher.

Back to Contents



6. Configuration (Linux)

The installation and configuration on the Linux side is basically the same as described on my main L2TP/IPsec page. There are however a few subtleties that need extra attention because they are different in Mac OS X 10.3 and higher, compared to other L2TP/IPsec clients.
  1. Like any other L2TP/IPsec client, Mac OS X requires the use of left/rightprotoport parameters in ipsec.conf which is your Openswan configuration file. These parameters are not available in vanilla FreeS/WAN. Strongsec's X.509 patch for FreeS/WAN provides these parameters. This patch is included with Openswan and strongSwan. The problem is that Mac OS X uses a 'floating port'. Normally UDP port 1701 is used through the following Openswan parameters:

    left=x.x.x.x
    leftprotoport=17/1701
    right=y.y.y.y
    rightprotoport=17/1701


    But these do not work with Mac OS X. They result in the following error:

      "cannot respond to IPsec SA request because no connection is known for x.x.x.x:17/1701...y.y.y.y:17/57937"

    Where 57937 (or any other number) is the ephemeral ("floating") UDP port. According to the L2TP/IPsec standard, this seems to be allowed (Apple might use this for QoS, load balancing or possibly even NAT-T?). The server has no way of knowing the UDP port in advance, so you need to to accept all possible UDP source ports. This is supported by Openswan 1.0.2+, Openswan 2.2.0+ and strongSwan 2.0.0+. You shouldn't be using versions older than that anyway. (If you want to know the details: you need at least version 0.9.38 of Strongsec's X.509 patch for Openswan 1.x/ FreeS/WAN 1.99 or version 1.5.3 for Openswan 2.x/ FreeS/WAN 2.04/2.05. Pluto's startup messages in /var/log/messages indicate which version of the X.509 patch you are using (e.g.: "Starting Pluto FreeS/WAN Version 2.04 X.509-1.5.4"). What you need is the %any port wildcard parameter:

    left=x.x.x.x
    leftprotoport=17/1701
    right=y.y.y.y
    rightprotoport=17/%any

    (This supersedes the EXPERIMENTAL patch that I made for other FreeS/WAN versions, where you needed to specify 17/0 instead of 17/%any).

  2. If you are using KLIPS, you may want to run the L2TP daemon only on the internal interface using the listen-addr parameter and then use iptables to forward all packets to the daemon. But because of the floating port mentioned above, a small modification to the iptables rule is needed. You need to forward (DNAT) all ports to the interface where the L2TP server is listening (e.g. the internal network interface):

    iptables -t nat --append PREROUTING -i ipsec0 -p udp --dport 1701 -j DNAT --to-destination 192.168.1.98

    (Where 192.168.1.98 is the IP address of the internal interface). This is similar to what you can find on my main L2TP/IPsec page. The difference is the removal of --sport 1701, so that all source ports are forwarded, not just port 1701. Unfortunately this trick does not work with the kernel 2.6 IPsec implementation ("NETKEY") because NETKEY does not have ipsec0 style interfaces and NAT-after-IPsec is currently broken on vanilla kernel 2.6. There might be a few ways to solve this problem on 2.6 kernels. The first two mentioned on that link should work, although I have not tested these myself yet.

  3. I noticed that Mac OS X's L2TP client does not supply a hostname during the L2TP connection negotiation (our_params.host_name[0] = 0?). This is not a problem, as long as you use at least version 8jdl of my l2tpd RPM (which contains this hostname patch). The MSL2TP client has the same issue.

  4. If you happen to use l2tpns as your L2TP daemon, you may need to apply a patch to the source code because l2tpns' HELLO messages apparently confuse Mac OS X. More information can be found on this webpage by Wolfgang Hennerbichler.

In most cases you do not know the client's IP address in advance, so using right=y.y.y.y is out of the question then. Instead you would typically use right=%any which results in the following:

left=x.x.x.x
leftprotoport=17/1701
right=%any
rightprotoport=17/%any
rightsubnet=vhost:%priv,%no
forceencaps=yes

You may notice the parameter forceencaps=yes. This was added because of a bug in Openswan (incorrect routing) which rears its head when you use right=%any in combination with rightprotoport=17/%any. Unfortunately this is a very common configuration when you want to support Mac clients. There are four workarounds but each one has its drawbacks: A) use right=client.ip.add.ress instead of %any: this only supports 1 client with a fixed IP address, not a very attractive prospect. B) use 17/1701 instead of 17/%any: Mac clients can no longer connect. C) remove the parameter rightsubnet=vhost:%priv,%no : NAT is no longer supported D) add the parameter forceencaps=yes: this forces NAT traversal, so it introduces unnecessary overhead if clients are not behind NAT. Another drawback of option D is that Windows XP/Vista/7 clients will then need a registry modification because the server appears to be behind NAT. Nevertheless, option D is probably the easiest solution, if you happen to have Mac clients.

Back to Contents



7.1 Connecting with a PSK (Mac OS X 10.5 and higher)

The GUI in Mac OS X 10.5 and higher supports PSKs. However, the application "Internet Connect" is no longer available because its functionality has been integrated with the "System Preferences" application. The procedure is now as follows (based on the instructions by Alan Whinery):
If you click on the button "Advanced..." you can find the option "Enable VPN on demand". Presumably it does exactly what it says: the VPN connection is automatically initiated when a particular hostname (or subnet?) is accessed by whatever program. I have not looked into this, as I am not a big fan of things that happen behind my back :-).

7.2 Dead Peer Detection


One problem is noticed is that Mac OS X Leopard does not appear to send a “Delete SA” message when the user disconnects. Previous versions of Mac OS X had their problems, but not this one. The IPsec connection remains up. The Mac client may not be able to reconnect, and an error is reported: "The server does not respond. Please verify your server address and try again." The problem is resolved when Dead Peer Detection (DPD) times out, the SA itself times out (if DPD is disabled) or the Openswan daemon is restarted. For this reason it is highly recommended to enable DPD on the Openswan VPN server by adding these parameters to your Openswan configuration (suggested time-out values):
    dpddelay=40
dpdtimeout=130
dpdaction=clear
Support for Dead Peer Detection is only supported on Mac OS X 10.5 and higher. It is not supported on Mac OS X 10.3 and 10.4.

7.3 "User Authentication" options

Mac OS X supports a number of PPP user authentication options, depending on the version of Mac OS X that you use. If you want to use a particular PPP authentication option then you also need support for that option on the Linux server. All Mac OS X versions support CHAP and RSA SecurID. As mentioned above I recommend that you first try to get CHAP passwords working. They are easier to use and things are easier to troubleshoot. If you want to use RSA SecurID hardware tokens (EAP type "fe" or 254, according to pppd): I understand that there is a Linux version of the RSA SecurID server but this is outside the scope of this webpage.

Starting from 10.4.4(?) Mac OS X also supports PAP and MS-CHAP(v2) password authentication. You don't actually have to tell the Mac what type of password authentication to use, it automatically uses the type of authentication requested by the PPP server.

Starting from 10.4.3(?) Mac OS X also supports user certificates for PPP authentication (EAP-TLS). This is the option "Certificate (Select)" under "User Authentication". Don't confuse this with certificates based IPsec authentication which is the option "Certificate (Select)" under "Machine Authentication". It appears that you can select the same certificate for both User and Machine Authentication, unlike Windows 2000/XP. I have not looked at EAP-TLS authentication yet but you can find some pointers here.

Starting from 10.4.3(?) Mac OS X also supports Kerberos for PPP authentication (EAP type "Windows 2000", according to pppd). I haven't looked at this. It is outside the scope of this webpage.

Starting from 10.4.6 Mac OS X also supports CRYPTOCard hardware tokens. I understand that there is a Linux version of the CRYPTOCard server but again, this is outside the scope of this webpage.

7.4. MPPE problems

Some users have reported the following error: "MPPE required but peer negotiation failed" (read this thread). This is odd because MPPE is not required for L2TP/IPsec, only for PPTP.

Back to Contents



8.1 Connecting with a PSK (Mac OS X 10.3 and 10.4)

The GUI in both Mac OS X 10.3 and 10.4 supports PSKs. Panther and Tiger are no longer supported by Apple with security updates, so I would not recommend using these old versions on the Internet as VPN clients. For archival purposes, the procedure is as follows:

If everything is OK then "Status" should say: "Connected To", followed by the IP address of the PPP server running on the internal interface.

There is also a field "Group Name (optional)" in Mac OS X 10.4.6+. This option appears to used for connecting to Cisco VPN servers (XAUTH and/or Hybrid Mode?) with a PSK. When used with a certificate the option probably does not have any effect. I did not look into this because Cisco modes are unsafe anyway.

The client also has an option "Enable VPN on demand".

Back to Contents



9. Connecting with a certificate (Mac OS X 10.5 and higher)

The VPN client GUI in Mac OS X 10.4 and higher supports both certificates and PSKs for IPsec authentication. Mac OS X 10.3's GUI only supported PSKs. There are two steps involved: first you import your PKCS#12 user certificate and then you add a VPN configuration that uses this certificate.

9.1 Passwords overview

First some remarks on passwords. Passwords are used for several tasks on Mac OS X. At several stages you will be prompted to enter a password. It may not always be clear what kind of password will be required at a certain point in time. Here is a short overview of the different types of passwords in Mac OS X:
9.2. Creating user and server certificates

Certificates will have to be created for the Openswan server and the L2TP/IPsec clients. There are some general instructions on my other webpage. I do not provide detailed information on generating certificates because it is outside the scope of this webpage. There are however two requirements that are very important when you want Mac clients to connect successfully to your Openswan VPN server:
The "ikeIntermediate" EKU is a bit of an annoyance on the part of Apple. Its use is discouraged by the IETF. Anyway, don't add any EKUs to a server certificate. Or, if you really want to, add the "ikeIntermediate" EKU by including the following option in the [usr_cert] section of your openssl.cnf before you generate the server certificate:

        extendedKeyUsage=1.3.6.1.5.5.8.2.2

This particular EKU can be combined with other EKUs that you might need. For example, if you also want to support Windows 2000/XP/Vista/7 clients, then you might want to use:

        extendedKeyUsage=1.3.6.1.5.5.8.2.2,serverAuth

The subjectAltName identifier (ID) must match what was entered in the "Server Address" field in the Mac's "Internet Connect" (see below). Examples of a hostname and an IP address are vpnserver.example.com and 123.123.123.123, respectively. Obviously, using a numeric IP address as the ID is not very flexible so in most cases you would want to use a hostname in the server certificate. Note that Mac OS X 10.4+ clients will not allow you to connect to servers that present a server certificate containing a Distinguished Name ("DER_ASN1_DN") as its ID, such as "C=CH, O=ACME, OU=Research, OU=Special Effects, CN=Bart Simpson". But Mac OS X clients are not fussy about client certificates: they are allowed to contain any type of ID. There are no restrictions for client certificates, unlike server certificates.

The exact procedure for adding a hostname or an IP address to a server certificate depends on the software that you use to generate the certificate. If you use OpenSSL, you have to add one of the following options to the [usr_cert] section in openssl.cnf before you generate the server certificate:

        subjectAltName=DNS:vpnserver.example.com
or:
          subjectAltName=IP:123.123.123.123

One of these two lines have to be added.

Without a subjectAltName in the server certificate Openswan will report the error: "ignoring informational payload, type INVALID_CERTIFICATE". If there is an EKU in the server certificate without the EKU "ikeIntermediate", then Openswan will report the error "ignoring informational payload, type INVALID_CERT_AUTHORITY". More information about using certificates with IPsec can be found in the Strongsec documentation.

9.3 Modifying the Openswan configuration

As mentioned above, the server's certificate contains an ID that is an IP address (e.g. 123.123.123) or a hostname (vpnserver.example.com). The Openswan configuration file ipsec.conf should match this and the "Server Address" field:

         leftid=@vpnserver.example.com
or:
         leftid=123.123.123.123

Obviously, an IP address is less flexible.

9.4.1 Importing the user's PKCS#12 machine certificate on Mac OS X 10.5 and higher

Apple has released very limited information on importing machines certificates for VPN use. The following procedure is a bit more elaborate (based on the work by Alan Whinery):
If you did not see any items in the "System" keychain, then click "All Items" in "Category". It works as a filter so perhaps the certificate was actually imported but simply not shown because of the filter. 

If you get this error: "An error has occurred. Unable to import an item. CSP_INVALID_DATA" then you typed an incorrect PKCS#12 password.

Unfortunately there is no indication about which private key belongs to which certificate. So should you want to delete a certificate be very careful about which corresponding private key you delete.

9.5 Adding a VPN configuration with certificate authentication

The procedure for adding an L2TP VPN configuration with authentication based on certificates is very similar to the procedure described above for PSKs. Except you can now select a certificate instead of a PSK. Important: in the "Server Address:" field you have to enter the hostname or IP address that is contained as an ID in the server's certificate (as mentioned above).

If the PKCS#12 certificate was not added to the "System" keychain but you tried to select one anyway in "Machine Authentication:" you get the following error:
No machine certificates found.

Certificate authentication cannot be used because your keychain does not contain any suitable certificates. Use Keychain Access to import the appropriate certificates into your keychain. If you do not have the certificates required for authentication, contact your network administrator.
9.6 Final remarks

The first time you click "Connect" you may get an IPsec connection but the L2TP connection may not work properly. If you disconnect and reconnect it should work. You may also have to reboot (or sudo /usr/bin/killall -9 racoon) before the machine certificate is correctly installed into the racoon of Mac OS X 10.4 and higher.

Back to Contents



10. Connecting with a certificate (Mac OS X 10.4)

As with Mac OS X 10.5+, there are two steps involved: first you import your PKCS#12 user certificate and then you add a VPN configuration that uses this certificate.

10.1.1 Importing the user's PKCS#12 machine certificate on Mac OS X 10.4

I am not aware of documentation by Apple or a webpage which provides any information on how to import a client certificate on Mac OS X 10.4 for L2TP/IPsec authentication. So I had to come up with something myself. The following procedure worked for me but may be too complex for some (end-)users. I hope that some time Apple decides to fix it so that you don't have to jump through hoops like described below.

Anyway, the following procedure describes how you can import one or more PKCS#12 certificates for use with L2TP/IPsec on Mac OS X 10.4 and higher:
If you did not see any items in the "System" keychain, then click "All Items" in "Category". It works as a filter so perhaps the certificate was actually imported but simply not shown because of the filter. 

If you get this error: "An error has occurred. Unable to import an item. CL_INVALID_FIELD_POINTER" then you have started the Keychain Access application as a normal user. That will work fine for importing user certificates (in the "login" keychain) but not for machine certificates. You have to open a Terminal and enter the "sudo" command as mentioned above. (There is an option "Click to unlock the System keychain." but if you do that and import a certificate, you still get the error. You really have to use the sudo. I don't know if this is a bug in Mac OS X or if it is really as intended by Apple).

If you get this error: "An error has occurred. Unable to import an item. CSP_INVALID_DATA" then you typed an incorrect PKCS#12 password.

If you get this error when you drag the root certificate to X509Anchors: "An error has occurred. Unable to add an item to the current keychain. The specified item already exists in the keychain.", then apparently the root certificate already is available in X509Anchors. You can safely remove the root certificate from System by selecting it and selecting "Edit -> Delete" in the menu.

Unfortunately there is no indication about which private key belongs to which certificate. So should you want to delete a certificate be very careful about which corresponding private key you delete.

10.1.2 Alternative method of importing the machine certificate

If you are a die-hard command line fan you can use the utility 'certtool' that is included with Mac OS X. It works for me but I did not look into this much because the command line scares off most users. Anyway, here are the commands for importing a file in PKCS#12 format. This example assumes that the file is called 'yourcrt.p12'.
openssl pkcs12 -in yourcrt.p12 -cacerts -out ca.pem -nokeys
openssl pkcs12 -in yourcrt.p12 -clcerts -out client.pem -nokeys
openssl pkcs12 -in yourcrt.p12 -nocerts -out key.pem -nodes
cp /System/Library/Keychains/X509Anchors $HOME/Library/Keychains/X509Anchors.bkp
cp /Library/Keychains/System.keychain $HOME/Library/Keychains/System.keychain.bkp
certtool i ca.pem k=X509anchors.bkp v
certtool i client.pem r=key.pem f=1 k=System.keychain.bkp v
sudo cp $HOME/Library/Keychains/X509Anchors.bkp /System/Library/Keychains/X509Anchors
sudo cp $HOME/Library/Keychains/System.keychain.bkp /Library/Keychains/System.keychain
rm ca.pem client.pem key.pem
You will be asked three times for the certificate password. After that, you will be asked for the Keychain Access password and then for your login password.

The procedure works on Mac OS X 10.4 but I am not sure if it also works on Mac OS X 10.5+. It is probably better to use the GUI which does work, unlike the GUI in 10.4.

A word of advice: copy, paste and execute these lines one by one in a Terminal window. Typing them is error prone.

(With special thanks to the author of this Mac OS X hint).

Back to Contents



11. Alternative methods of connecting with a certificate (Mac OS X 10.3 and higher)

The GUI is not the only way to configure IPsec on Mac OS X. If you are familiar with KAME you could edit KAME's configuration files manually. I have not tried this because this is probably too much to ask for the typical Mac end-user. Advanced users who are able to configure KAME from the command line probably do not need the GUI and the L2TP protocol anyway. Wolfgang Hennerbichler has done this for a project. He writes:
"OS X creates config-files on the fly, but the main racoon.conf is not touched, instead there's a parameter in racoon.conf that says: include "/etc/racoon/remote/*.conf"  So I changed the racoon.conf just to my needs (Certificates and so on), and removed this include-line. With that, you can set the connection up via the GUI, and racoon will be called by the GUI with the correct parameters and the policies will be set correctly. This might be a problem if you have more than 1 network (different certificates) to connect to".
The relevant configuration file can be found on this page. "Agent Smith" provided a similar setup on the Openswan mailinglist.

Back to Contents



12. NAT-Traversal

For information about IPsec NAT-Traversal in general, see my other webpage.

Apple supports the IETF NAT-T standard (RFC 3947) in Mac OS X 10.4.4 and higher. These versions should interoperate fine with recent versions of Openswan and Cisco that also support RFC 3947. Windows clients and Windows Server 2003 should also interoperate because Mac OS X 10.4.4+ supports the draft NAT-T version implemented by Windows ("draft-ietf-ipsec-nat-t-ike-02").

For Mac OS X 10.3.6+ and Mac OS X 10.4.0-10.4.3 the situation is different. Apple implemented its own NAT-T variant which is is incompatible with most other IPsec implementations. Mac OS X sends the non-standard vendor ID string "draft-ietf-ipsec-nat-t-ike" instead of "RFC 3947". According to some reports Apple's version is a draft version 8 of the NAT-T standard which was the latest draft before the standard was ratified. This draft version is not the final version and, in fact, draft 8 jumped the gun a bit because it uses invalid ISAKMP payload types which were already allocated to RFC 3547 by IANA. Apple's NAT-T version does not interoperate with other IPsec implementations unless they specifically support this Mac OS X quirk. Apple's Mac OS X Server is one of these implementations; the Stinghorn L2TP/IPsec Gateway is another one. Recent versions of Openswan also support Apple's NAT-T version (see the next section). Apple's racoon modifications are available on their website (e.g. here for Mac OS X 10.4.9) but they are available under the Apple Public Source License which unfortunately means that you cannot use these modifications directly in Openswan (=GPL) or KAME (=BSD). Apple does not want to relicense the code either because of legal concerns.

Peter Van der Beken has created a patch for Openswan that supports Apple's oddball NAT-T version. This patch has been adapted by Michael Richardson of the Openswan team and incorporated in Openswan 2.4.5+. Paul Wouters of the Openswan team noted that a rekeying problem occurred after one hour. Note that even with Openswan 2.4.5+ you will still not be able to use NAT-T with a PSK if your Openswan server is using KLIPS. This is because the NAT-T patch for KLIPS does not support PSKs. You have to switch to NETKEY because NETKEY does support NAT-T with PSK authentication. Also note that NETKEY has problems supporting the Mac's floating UDP source port. I have also not tried to connect with multiple Macs behind the same NAT device so I don't know if that is supported.

Apple's racoon version is a fork of KAME's racoon which has been discontinued. Unfortunately Apple has decided to start their own racoon fork instead of using the ipsec-tools fork of racoon which is still in active development. Other BSD versions such as NetBSD did switch to ipsec-tools. This means that Apple is missing out on features such as IPCOMP unless they add them theirselves.

Back to Contents

13. Connecting with the Apple iPhone

The iPhone is based on Mac OS X. It ships with a built-in client that supports a number of VPN protocols including L2TP/IPsec. I have no first hand experience with the iPhone. Kim Hendrikse reports that the iPhone connects to an Openswan based L2TP/IPsec server but for some reason the iPhone  disconnects within a minute if there is no payload traffic, no matter if you use PPP/L2TP/IPsec's keep alive mechanisms. According to an Astaro press release, their Astaro Security Gateway appliance is compatible with the iPhone for both L2TP/IPsec and PPTP. The Astaro Security Gateway is based on Linux (Strongswan, l2tpd etc.) so I suppose it also works with the setup described on this webpage.

According to Apple, the iPhone supports the following VPN configurations:
The iPhone does not support:
The lack of certificate support is unfortunate, to say the least. Pure IPsec with a single PSK is not a good idea for corporate VPN use. But even after several iterations of the iPhone software Apple still does not support certificates (both Windows Mobile and Android for example support client certificates)...

Currently the iPhone can store only one PPTP and one L2TP/IPsec configuration. This too shows that there is room for improvement.

The iPhone's VPN client works over both Wi-Fi and EDGE connections. To configure a VPN connection, follow this procedure:
Once you have configured a VPN connection, a VPN on/off slider appears at the top of the Settings list. Tap this to turn the VPN on or off.

The iPhone appears to have an interactivity timeout problem, unlike Mac OS X. The connection is terminated by the iPhone after about one minute if there was no 'useful' network data sent over the VPN connection. It also does not send a "Delete SA" message to the server. DPD does not help. It does not matter if the user keeps the iPhone active by tapping the screen, for example. There must be data sent through the VPN tunnel to keep the iPhone from disconnecting, such as browsing websites. Very annoying. Keep-alive packets (IPsec, L2TP and PPP) seem to be ignored.

One final note: PublicVPN.com reports a couple of problems in the iPhone firmware. Version 1.0 has a bug where you have to type the password in every time you connect. Version 1.01 has a different problem: DNS resolution doesn't work if you're connected to a VPN. Without DNS support the VPN connection may not be very useful.

Back to Contents



14. Perfect Forward Secrecy (PFS)

Mac OS X does not propose Perfect Forward Secrecy (PFS) by default. I don't know how to do enable PFS through the Mac OS X GUI (e.g. the "Internet Connect" program). There is no mention of a "PFS" setting. I suppose the default racoon.conf can be modified so that PFS is enabled (similar to the procedure described above) but I haven't tried that.

There are valid reasons for using PFS. If you really need PFS and you don't want to modify the default racoon.conf, you could decide to switch to IPsec without L2TP (e.g. through a third-party IPsec client or configuration utility). Then it is possible to enable PFS.

Back to Contents



15. Advanced topics

15.1 DHCP Inform

According to Wolfgang Hennerbichler, Mac OS X also supports DHCP in order to retrieve settings such as DNS servers, domain names, static routes etc. from the VPN server. This is completely optional, the VPN connection should work without them. You need a DHCP server that supports DHCP INFORMATIONAL Messages, such as ISC DHCPD 3.x or higher.

15.2 AES, SHA-1 and IPCOMP

By default, Mac OS X proposes ISAKMP SAs with 3DES encryption, HMAC authentication based on SHA-1 hashes and DH group 2 (MODP1024). These are reasonable defaults and Openswan will accept them.

Mac OS X also proposes IPsec SAs with either 128-bit AES or 168-bit 3DES encryption and HMAC authentication based on SHA-1. These are used for the bulk encryption so they affect the throughput of the L2TP/IPsec connection. Older versions of Openswan will use 3DES and SHA-1 for IPsec SAs by default. These are good defaults for Windows clients that use the built-in IPsec stack. AES is much faster than 3DES and Mac OS X supports AES, so you might want to enable AES support in Openswan. You could for instance add the following lines to the connection section(s) in your ipsec.conf:

ike=aes-sha,3des-sha
esp=aes-sha1,3des-sha1


The Mac OS X configuration file racoon.conf contains a line that should enable the 'deflate' type of IPsec compression (IPCOMP) for its IPsec SAs. But it seems that the Mac OS X kernel does not support it. I don't see an IPCOMP header in the packets that the Mac sends. Openswan supports IPCOMP deflate but by default it does not enable it. You can enable deflate compression by compress=yes to the connection section(s) in your ipsec.conf. But this will not result in actual use of IPCOMP because the Mac OS X kernel does not seem to support it.

Back to Contents



16. Troubleshooting: examining the logs

There are two phases: the IPsec phase and the L2TP/PPP phase. Logging information can be found in several locations on Mac OS X.

Information about troubleshooting on the Linux side can be found on my main L2TP/IPsec page.

16.1 IPsec and PPP logs

The logfile of Racoon (the Mac's VPN daemon) can be found at /var/log/system.log (Examining this file requires superuser privileges so use something like "sudo less /var/log/system.log").

On Mac OS X 10.5 and higher, the "Network" option in "System Preferences" has a button called "Advanced". If you click this button, you find an option "Use verbose logging" which is supposed to capture more detailed log information in your VPN session. Problem is, Apple does not say where these details are logged.

On Mac OS X 10.3 and 10.4, the most basic logging information can be found in the "Internet Connect" application by opening the "Connection Log". You can find it in the menu under "Windows".

16.2 Even more logging

Alexandre Ghisoli provides a procedure for more verbose logging:
Alexandre also provides a procedure for even more logging by "tracing" racoon:
16.3 Example

Here is an example of a problem I ran into and how I used system.log to trace the cause. I had defined a connection on Mac OS X for a certain server. I had installed another server with the same configuration and I wanted to test it as well, so I simply changed the IP address in Internet Connect's main window. I did not change the username, password and shared secret. Since I had not changed them, I figured that the same settings would be used to connect to the second server. It turned out that this was not the case for the shared secret. When entered in the "Edit configurations..." window, the shared secret is specifically for the server whose IP address you entered there (apparently the Keychain application stores shared secrets in such a way that they are bound to an IP address). I got the following error in system.log:

Nov 15 12:02:17 localhost racoon: ERROR: oakley.c:2071:oakley_skeyid(): couldn't find the pskey for 192.168.0.111.

I had to go to "Edit configurations..." window and change the IP address there. You cannot change the IP address in Internet Connect's main window and expect it to work.

Back to Contents



17. Split tunnelling

If you don't want to send regular Internet traffic through the VPN tunnel you may want to enable split tunnelling. See this section for more details about split tunnelling and its advantages and disadvantages. Here is how to enable split tunnelling on Mac OS X:
Back to Contents



18. VPN alternatives for the Mac

The built-in VPN client is not the only VPN client available for Mac OS X. There are others as well. None of them supports L2TP/IPsec, though. The following products are basically front-ends for Mac OS X's built-in IPsec clients:
The clients above all support pure IPsec (without L2TP) which has its advantages and disadvantages. There are also other VPN alternatives:
Back to Contents

19. Acknowledgements

This page is sponsored by the HCC Amsterdam user group.

Alan Whinery demonstrated that certificates are now much easier to import on Leopard.

Thanks to Manny Veloso of PublicVPN.com for allowing me to test interoperability between Mac OS X Server and Linux L2TP/IPsec clients.

Thanks to Alexandre Ghisoli for bringing the INVALID_CERT_AUTHORITY problem to my attention, and to Daniel Bertolo for solving the problem.

Back to Contents



20. Revision history

May, 24 2010: Started section about (Snow) Leopard.
Mar 5, 2008: Started section about DHCP Inform.
Feb 26, 2008: Added section about split tunnellng.
Jan 8, 2008: iPhone compatible with the Astaro Security Gateway appliance.
Jan 4, 2008: There is a bug in Openswan which pops up when you want to support Mac clients (workarounds mentioned).
Oct 30, 2007: First tests with Mac OS X 10.5 (Leopard) are inconclusive: some users report success, others had problems.
Oct 25, 2007: Added more info about requirements for server certificates.
Sep 19, 2007: Added some remarks about the iPhone.
May 22, 2006: Mac OS X 10.4 supports PAP/CHAP (10.4.4+?). Also new: CryptoCard (10.4.6+) and EAP-TLS (10.4.4+?).
Feb 12, 2006: Mac OS X 10.4.4+ supports the IETF NAT-T standard (RFC 3947).
Dec 6, 2005: That patch is no good. Get Openswan 2.4.5 or higher.
Nov 21, 2005: Made patch (based on Peter's patch) that swaps the NAT-D hashes: needed for NAT-T.
Nov 12, 2005: Added command line procedure for importing machine certificates on Mac OS X 10.4. Update on DPD, IPCOMP.
Nov 7, 2005: Added procedure for importing machine certificates on Mac OS X 10.4.
Sep 29, 2005: (Updated) patch by Peter Van Der Beken available for NATed Mac clients.
Jun 26, 2005: Reportedly, the VPN problems in Tiger have been fixed in update 10.4.1.
May 15, 2005: s2svpnadmin ("site-to-site VPN admin") only included in Mac OS X 10.4 Server, not Client  :-(
May 3, 2005: Mac OS X 10.4 breaks several third-party VPN clients. Whoops. L2TP/IPsec (and PPTP) should work.
Apr 27, 2005: First report of Mac OS X 10.4: has a GUI for certs. NAT-T still non-standard.
Jan 16, 2005: Added mention of a method to use certificates on Mac OS X 10.3.
Jan 8, 2005: Mac OS X's NAT-T support is bogus. Apple implemented the RFC but mislabeled it.
Nov 23, 2003: Confirmed by Apple employee: no cert support yet.
Nov 16, 2003: Added screenshots and more details.
Nov 15, 2003: Succesfully tested with PSKs (had to make a FreeS/WAN patch though).
Nov 6, 2003: Page created.

Jacco de Leeuw