Escaping (CG)NAT hell: tunnel your way out
Recently my ADSL connection was down for a few days as some idiot had put a shovel or backhoe into a set of phone lines serving the area (no ADSL). To make matters worse, this happened while we were collecting a relative from the airport. The ability to communicate with relatives either by phone or applications like Skype is important, not to mention I need internet access just to entertain myself, so I set about finding a solution.
With ADSL down, the only other practical alternative at short notice is a pre-paid 3G connection. Unfortunately, in Australia, these are always behind layers of operator NAT's (commonly called CGNAT these days, to differentiate from the NAT done in many consumers CPE's), often with features designed to "help" feature phone users, such as transparent proxies. (i.e the Optus transparent proxy used to interfere with SyncML sessions).
The best deal I could obtain at short notice was Woolworths Mobile (powered by Optus), 5GB for $29, plus $2 for the SIM card. When I plugged in my 3G modem right into my Fritz!Box, it managed to connect, but our NodePhone VoIP service did not like the multiple layers of NAT imposed on it. (NodePhone usually has no issues doing NAT traversal, if it is just a single router between your IP phone and your internet connection). To make it work, I needed to present a full end-to-end IPv4 connection
The solution was to use the "cloud" - I could use a low end virtual server - which gives me a full IPv4 end point, as a VPN server and tunnel all my internet traffic through that. For this excercise, I used the recently launched Amazon EC2 cluster in Sydney, and a t1.micro instance - cheap and far more than adequate for this purpose. (Hint: If you don't mind having some occasional down time, having to relaunch an instance - set it up as a spot instance - which is cheaper still)
The components of the solution are:
- A server with a full IPv4 address assigned to it and the ability to use tun and iptables DNAT and SNAT within Linux (or the equivalent for your chosen OS). EC2 works, as would a 'lowendbox' provider using Xen or KVM.
- A Linux or similar router with the ability to terminate OpenVPN connections, and two interfaces: one to your WAN (3G) and to your LAN or existing router (i.e I configured my Fritz!Box to connect to the internet through the Linux box terminating the VPN
The Amazon Linux AMI is adequate for this purpose. You will need to install OpenVPN through the package manager, and configure it similar to the Static Key Mini-HOWTO. The server configuration needs to include a route back to the client network (omit if your VPN endpoint also does the routing for your local network), and I also chose to use LZO compression to reduce overhead (I only have a limited bandwidth allowance, after all).
In the end the server configuration looked like this:
dev tun ifconfig 10.8.0.1 10.8.0.2 secret static.key route 192.168.5.0 255.255.255.0 10.8.0.2 comp-lzoThe client configuration is the same as described in the HOWTO:
remote <your server IP> dev tun ifconfig 10.8.0.2 10.8.0.1 secret static.key comp-lzo
On the server, you need to enable IP forwarding:
vi /etc/sysctl.conf # set net.ipv4.ip_forward=1 sysctl -p /etc/sysctl.conf
Now we need to configure iptables:
# Notch out ports that should not be forwarded to the remote end iptables -t nat -A PREROUTING -m tcp -p tcp --dport 22 -j ACCEPT # SSH iptables -t nat -A PREROUTING -m udp -p udp --dport 1194 -j ACCEPT # OpenVPN # Forward all other traffic to the remote router iptables -t nat -A PREROUTING -d <server WAN IP> -j DNAT --to 192.168.5.11 # Static NAT all traffic from remote to the server iptables -t nat -A POSTROUTING -i 192.168.5.0/24 -j SNAT --to <server WAN IP> iptables -t nat -A POSTROUTING -i 10.8.0.2 -j SNAT --to <server WAN IP>Important Note: On Amazon EC2, the server WAN IP, for iptables purposes, is the internal IP address (10.x.x.x) - the address on the eth0 interface
Launch OpenVPN by changing to the directory with static.key, and running <openvpn> I prefer to do this in a <screen> session - a more permanent solution could be launched via an init script.
openvpn --config openvpn.conf
On the client end, change your routing table so packets to the VPN server go directly through your router, irrespective
route add <VPN server IP> gw <3G modem gateway IP>
Launch OpenVPN, as with the server I would recommend doing it via <screen>
openvpn --config openvpn.conf
Now change your routing tables so the <default> route is via the VPN:
route delete default route add default gw 10.8.0.1If the IP forwarding sysctl is not enabled, you would need to do this as you did on the server.
Don't forget to change your DNS by editing "/etc/resolv.conf", as it is probably set to your 3G modem or similar. I chose to use the DNS server on the EC2 cluster (check resolv.conf on the server):