VPN-Gateway with NAT


Initial Situation

A friend of mine has an Office in the Basement of his home. He quite successfully manages his IT himself but every now and then he needs a helping hand, just as we all do. For this Situations I had a VPN into his Office for quite some time.

After he switched Internet Providers we found that his new connection does not come with a Public IPv4 Address but instead his router receives a private IP-Address and his provider then does Carrier-Grade NAT. He also receives a public IPv6 Subnet which is publicly routed.

My friend has a AVM Fritz!Box, which is quite a versatile Modem/Router/Phone System/Wifi-AP/Dect-AP in itself, but it has one really huge drawback: It only support IPSec VPN.

AVMs VPN Implementation does not allow acting as a VPN-Server on an IPv6-Only Connection (according to this Info-Page):

IPv6 VPN connections are not supported by the FRITZ! Box, since no IPv4 data can be transmitted over them. In addition, such IPv6 connections could only be made if both subscribers have IPv6 Internet connectivity, which in most mobile networks and Wi-Fi hotspots is not the case.

Running an OpenVPN-Server On-Site

My first reaction was keeping the VPN-Server On-Site but switching to OpenVPN, because I knew that OpenVPN could tunnel IPv4 over IPv6. One of the devices in his Office, a Synology Disk-Station is able to act as an OpenVPN server and so –in a pinch– we went with that.

Although that simple Setup worked okay for some time, it had some annoying problems:

  • Now his Storage is connected to the Internet and acts as a VPN-Gateway
  • The Disk-Station can't NAT VPN connections, so that one has to do SSH-Hopping and SSH Port-Forwarding tricks to reach all devices in the Network
  • The Public IPv6 Net changes every now and then and so does the Disk-Station's IPv6 Address, so some DynDNS-Tricks are required to always know the Server's address
  • Connections are only possible from networks with working IPv6 connectivity, which unfortunately rules out my work Network. Meh…

A more robust solution

With this solution only working half of the time I decided I wanted a more robust solution, that would possibly also scale to other friends' networks.

I wanted a solution that

  • Is reachable via IPv4 and IPv6
  • Has a static Connection-Address
  • Does not require any configuration on the target Network
  • Thus allows the target Networks' owner to switch Uplink-Providers or physically move without me knowing.

What I came up with was an OpenVPN-Server running on my VPS hosted by NetCup which provides stable IPv4 and IPv6 connectivity with static IPs. On-Site I used an TP-Link TL-WR841N v13 Router because it resells at only about 16€. (!) The v13 is only supported in recent OpenWRT Snapshots any may or may not work as expected for you. If you want to be sure, you might want to get yourself a TL-WR1043ND which resells at around 32€ but is completely and officially supported.

The On-Site OpenWRT Router gets its Uplink via DHCP from whatever Router is present on the Network and acts as an OpenVPN Client through that Internet Connection. As long as it can connect to the OpenVPN-Server I can SSH into it and examine what's going on in the target network.

Setting up the OpenVPN Server

Setting up the OpenVPN-Server on Debian is quite well documented on the Debian-Wiki and I basically followed it to the word.

One thing I changed was using the name of my friends business (HTDuS) in all configuration- and folder-names, so that I can easily add another network in the future. Also I chose a non-default port and a usefuk tun-Device-Name.

; general server stuff
mode server
local ::0
port 1200
proto udp
dev tun_htdus

; directories
chroot /etc/openvpn

; mtu
link-mtu 1400
mssfix 0 ; disabled

; certs
ca ./easy-rsa-htdus/keys/ca.crt
cert ./easy-rsa-htdus/keys/mazdermind.de.crt
key ./easy-rsa-htdus/keys/mazdermind.de.key  # This file should be kept secret
dh ./easy-rsa-htdus/keys/dh2048.pem

; verify
crl-verify ./easy-rsa-htdus/keys/crl.pem
tls-auth ./easy-rsa-htdus/keys/ta.key 0
remote-cert-tls client
persist-key
persist-tun

; crypto
cipher AES-128-CBC
auth SHA512
tls-cipher "DHE-RSA-AES256-SHA256:DHE-RSA-AES128-SHA256:DHE-RSA-AES256-SHA:DHE-RSA-AES128-SHA"
tls-server

; server options
server 10.97.0.0 255.255.255.0
client-config-dir clients-htdus
keepalive 10 30
client-to-client
;mode server
topology subnet

user nobody
group nogroup

; logging
ifconfig-pool-persist ipp-htdus.txt
status /etc/openvpn/status-htdus.txt
#log /var/log/openvpn-htdus.log
verb 3

/etc/openvpn/htdus.conf

As a VPN-Net I chose 10.97.0.0/16. This is the IP-Pool the VPN Clients will be assigned an IP from. I basically wanted it to be visually far away from the IP-Range used in the target Network, which is 192.168.124.0/16, so I can easily identify them if I have to.

The client-config denotes which routes should be announced to the default_clients who's responsible for them (in this case: the OpenWRT):

root@mazdermind /etc/openvpn/clients-htdus $ ls -la
total 16
-rw-r--r-- 1 root root  102 Jun 13 10:56 default_clients
lrwxrwxrwx 1 root root   15 Jun 10 23:14 ray -> default_clients
lrwxrwxrwx 1 root root   15 Jun 10 23:14 spock -> default_clients
-rw-r--r-- 1 root root   70 Jun 13 10:55 wrt

root@mazdermind /etc/openvpn/clients-htdus $ cat wrt
iroute 192.168.124.0 255.255.255.0

root@mazdermind /etc/openvpn/clients-htdus $ cat default_clients 
push "route 192.168.124.0 255.255.255.0 10.97.0.1"

As you can see you can freely route some Subnets to one Client and other Nets to other Clients to support, for example, multiple Locations.

For the Clients I generated Keys with Passphrases, while I generated one without for the WRT, because I wanted it to connect on Power-Up without anyone being there to enter a Passphrase.

If you have an iptables-Firewall on your Server (you should!) you will need to allow the incoming VPN Connections:

sudo iptables  -A INPUT -i eth0 -p udp -m udp --sport 1024:65535 --dport 1200 -m comment --comment "openvpn htdus" -j ACCEPT
sudo ip6tables -A INPUT -i eth0 -p udp -m udp --sport 1024:65535 --dport 1200 -m comment --comment "openvpn htdus" -j ACCEPT

You can now enable and start the OpenVPN-Server as follows:

sudo systemctl enable openvpn@htdus.service
sudo systemctl start openvpn@htdus.service
sudo systemctl status openvpn@htdus.service

Generating Client-Configs

Testing the Connection requires generating client-Config-Files. I prefer the Config-Style with embedded keys, so I wrote me a small Shell-Script which generates the Client-Configs for me:

#!/bin/sh
set -e

CLIENT="$1"

if [ ! -e "keys/$CLIENT.key" ]; then
  echo "empty or invalid client: '$CLIENT'"
  exit 1
fi

cat <<EOT
#viscosity name HTDuS

; general
client
dev tun_htdus
topology subnet

; connection
remote mazdermind.de
port 1200
proto udp
keepalive 10 30
resolv-retry infinite
link-mtu 1400
mssfix 0
nobind
max-routes 1000

; crypto
cipher AES-128-CBC

; certificates
key-direction 1

; auth
tls-cipher "DHE-RSA-AES256-SHA256:DHE-RSA-AES128-SHA256:DHE-RSA-AES256-SHA:DHE-RSA-AES128-SHA"
tls-client
auth SHA512
remote-cert-tls server

; logging
;verb 0
EOT

echo '<ca>'
cat "./keys/ca.crt"
echo '</ca>'

echo '<cert>'
cat "./keys/$CLIENT.crt"
echo '</cert>'

echo '<key>'
cat "./keys/$CLIENT.key"
echo '</key>'

echo '<tls-auth>'
cat "./keys/ta.key"
echo '</tls-auth>'

/etc/openvpn/easy-rsa-htdus/generate-config-for-client.sh

Isolating the Host from the VPN

With suche a Client-Config you can now connect to the VPN from any Linux-System by simply calling sudo openvpn --config my-conf.ovpn. You'll get an Interface named tun_htdus and an IP from the 10.97.0.0/16 range and routes pointing 10.97.0.0/16 and 192.168.124.0/16 via the VPN-Server 10.97.0.1 into the tun_htdus-Interface.

You can now ping 10.97.0.1 which is you VPN-Server. Yay!

But depending on what your Server hosts, you can now also reach a lot of services on your Host-System by just adding appropriate routes on your client into the tun-device.

In my case I have some Docker-Container running on the host and Docker uses 172.17.0.0/16 for its internal containers. Simply adding this to the routes on the client would expose the internal Docker-Containers network' to all VPN Clients.

So some further iptables-Rules are required:

sudo iptables  -I INPUT 0  -d 10.97.0.0/16 -i tun_htdus -p icmp -m icmp --icmp-type 8 -m comment --comment "allow ipv4 pings from vpn to host on vpn-net" -j ACCEPT
sudo iptables  -I INPUT 1 -i tun_htdus -p tcp -m comment --comment "reject remaining tcp vpn-to-host" -j REJECT --reject-with tcp-reset
sudo iptables  -I INPUT 2 -i tun_htdus -p udp -m comment --comment "reject remaining udp vpn-to-host" -j REJECT --reject-with icmp-port-unreachable

sudo ip6tables -I INPUT 0 -i tun_htdus -p tcp -m comment --comment "reject remaining tcp vpn-to-host" -j REJECT --reject-with tcp-reset
sudo ip6tables -I INPUT 1 -i tun_htdus -p udp -m comment --comment "reject remaining udp vpn-to-host" -j REJECT --reject-with icmp-port-unreachable

They need to be on top of the INPUT-Chain, or some ACCEPT rules which do not explicitly specify interfaces or ip-ranges further down might allow the traffic anyway. I had, for example, some rules like those, which, unexpectedly allowed HTTP-Traffic from the VPN to Docker-Containers running on the Host.

sudo iptables -A INPUT -p tcp -m tcp --sport 1024:65535 --dport 80 -m comment --comment "allow http" -j ACCEPT

I added an explicit -i eth0 to them, to further harden the Firewall-Setup.

One thing to be aware of while configuring you Firewall: client-to-client in the OpenVPN Server-Config allows VPN-Clients to communicate with each other without the Packagets being processed by the OpenVPN Servers' Kernel, so they are not affected by iptables and do not require FORWARD rules.

This is great I you don't want to have the Host interfere with the OpenVPN but removes some control from your fingertips. Just be aware of this option when you need more control about which Client should be able to communicate with which net.

Installing OpenWRT

Setting up OpenWRT depends on which router you got. For the TL-WR841N v13 look up the TFTP Firmware-Link from the OpenWRT-Wiki, connect the TP-Link and your Computer to an Ethernet-Switch, add 192.168.0.66/24 to your Interface, start an TFTP-Server and make it serve the Image as wr841nv13_tp_recovery.bin. Hold the Reset-Key on the WRT and power it on. It fetches the image and Flashes itself.

This procedure is not trivial and it might need multiple tries and debugging to get it going. Most WRTs can usually be also flashed via the vendors Web-UI but this specific firmware-Version rejects non-vendor updates.

Give your self some time and search the Internet for hints, if it doesn't work. You can not really brick these devices, because the TFTP Recovery Mode is burned into their ROM and can not be over-flashed.

After Flashing the Device it will fetch an IP-Adresse via DHCP and open SSH with Username root and no Password. You can then install LUCI, the WebUI by calling opkk update; opkg install luci.

Configuring the OpenWRT

At this point you should be able to connect multiple Computers to the VPN and communicate between them using their 10.97.0.0/16 IPs. They should all have a route to 192.168.124.0/16 through the VPN but obviously nothing is responding to those Packets.

To get OpenVPN running on the WRT you need to install the required Packages: ovpn install openvpn-openssl luci-app-openvpn should give you what you need.

I chose to not migrate my working OpenVPN-Config to the uci-Syntax, so I uploaded my working OpenVPN Client-Config to the WRT as /etc/openvpn/htdus.ovpn and configured UCI via the SSH-Terminal as follows:

uci set openvpn.htdus='HTDuS'
uci set openvpn.htdus.enabled='1'
uci set openvpn.htdus.config='/etc/openvpn/htdus.ovpn'
uci commit

The LUCI-UI shows the VPN and can control it, although the Settings will not be editable via the UI.

The OpenWRT should now have an Interface tun_htdus with one of the 10.97.0.0/16 and if one of the other clients sends Packages to the 192.168.124.0/16 net, the OpenVPN Server will forward them to the OpenWRT, but it does not yet know what to do with them.

Your Interfaces-Section should look somehow like this: Interfaces-Section of the LUCI Config-UI

You should now add a Firewall-Zone that allows Forwarding into the LAN-Zone. The OpenWRT's Firewall-Concept basically only applies to who's allowed to initiate a connection. All Packets relating to an already established Connection are always allowed.

The Firwall-Zone should look something like this: Interfaces-Zone in the LUCI Config-UI

Now enable Masquerading on your LAN-Zone, like this:

The Firewall should now look something like this: Interfaces-Section of the LUCI Config-UI

  • VPN INPUT allows SSH & LUCI from the VPN
  • VPN OUTPUT allows connections from the Device into the VPN (ie. Pings)
  • LAN INPUT allows SSH & LUCI from the LAN
  • LAN OUTPUT allows connections from the Device into the LAN (ie. DNS, OpenVPN)

Why Masquerading on the LAN-Interface?

Imagine your Home-Router which has Computers on a LAN behind it, trying to communicate with Hosts on the Internet. The Hosts on the Internet always ever see the Routers IP-Address, so it is NATing all Packets from IPs not belonging to its external Net.

Our scenario here is quite similar, only that our VPN-Clients are behind the Router with their VPN-IPs being "internal" and the target-Network representing being "the internet": It's where the Packets need to be forwarded to it should never see the "internal" IP-Addresses but only the VPN-Gateways' Address.

Usually one would enable Masquerading on the WAN-Interface, so that packets going from LAN to WAN would be NATed, but we want the Packets going from VPN to LAN go be NATed, so we enable Masquerading on the LAN-Zone.

Testing the Solution

Testing requires us to have a Network with the targeted IP-Range present. I tried to simulate that by adding another IP to the lan-Interface with ip a a 192.168.124.42/24 dev br-lan but the MASEQERADE-Filter will only use the first IP it finds on an interface and not consider other IPS. Meh.

In the end I mis-used the Guest-Network provided by my Fritz!Box to simulate being in a different Network (and add it's IP-Range to the VPN for testing purposes).

Conclusion

I learned a ton about OpenVPN, Routing, OpenWRT and iptables. The whole Setup took me about 3 days and another on to write this Blog-Post.

I hope it saves you some days or gives you an idea how to setup your VPN-Gateway.