We lease a couple of machines at Hetzner.de and recently, I looked to improve my initial Easy Setup manual, so I could also use the IPv4 “main” address for a virtual machine. This leaves the main server (the “host”) without an IPv4 address. Please read below how to proceed.
Hetzner has a rather ingenious network setup, where the switch (or router) does not send ARP requests. Instead, all your traffic is directed to the MAC address of your primary NIC. This poses a few difficulties, but I found a way around them back in January, 2011. But now it’s 2013 and we don’t want the primary IP address to live in the host machine – just for nothing. We’d like to use that in a guest machine, too.
Here’s how to proceed.
The first thing to note is, that I am going to use an IP address for the host machine. Without an IP address, I would need a different way to resolve MAC addresses – most likely, a list of VM’s with their IP address and accompanying network card address. This is what ARP was invented for, so we want to use ARP.
On the host machine, this is what my /etc/network/interfaces file looks like:
### Hetzner Online AG - installimage
# Loopback device:
auto lo
iface lo inet loopback
auto br0
iface br0 inet6 static
address 2a01:xxxx:xxx:xx::x
netmask 64
gateway fe80::1
bridge_ports eth0
bridge_maxwait 0
iface br0 inet static
address 10.9.8.7
netmask 255.255.255.255
pointopoint $Hetzner-gateway-IP
As you’ll notice, I have a single bridge that connects the outside world (eth0) with the virtual machines inside the bridge. This has the rather odd side effect of seeing every packet twice: once when Hetzner sends it to your ethernet card (source MAC address from Hetzner, destination MAC is your card), then again when your host concludes that this packet is to be routed to one of your virtual machines (source MAC is your card, destination is your virtual network card). I have pondered the possibility of having a proper routed net, giving eth0 something like 10.9.8.7 and br0 something like 10.10.11.12. This would have the advantage of isolating the virtual network traffic (“arp who-has 10.10.11.12” from the virtual machines) from the outside. The disadvantage, however, is, that we would need to route our IPv6 network. Hetzner, as far as I can see, does use proper neighbour discovery for IPv6 – so it actually uses the virtual MAC addresses for IPv6 traffic.
So I will use a single bridge. Hetzner will receive the virtual machine’s “who has 10.9.8.7” but it ignores them anyway.
Our setup is almost ready. There’s a few things that won’t go quite right now. If the host sends an “arp who-has”, it will do so with the source address of 10.9.8.7. And Hetzner does not answer such a request, because it doesn’t recognize the source address. We need “arptables” for this, with a “post-up” command:
post-up arptables -A OUTPUT -s 10.9.8.7 -d $Hetzner-gateway-IP -j mangle --mangle-ip-s $Your-main-IP-address
Another issue is, that the host won’t know which IP addresses are link local – i.e. belong to a virtual machine. It will send all traffic to Hetzner – which, in turn, will send it back – unless you tell it otherwise. In order to send “arp who has” to br0, you’ll need to add routes. I’ll add another bunch of “post-up” commands to our br0-device in /etc/network/interfaces:
iface br0 inet6 static
[...]
post-up arptables -A OUTPUT -s 10.9.8.7 -d $Hetzner-gateway-IP -j mangle --mangle-ip-s $Your-main-IP-address
post-up ip ro a $your-extra-ip-address1/32 dev br0
post-up ip ro a $your-extra-ip-address2/32 dev br0
post-up ip ro a $your-extra-ip-address2/32 dev br0
… a “ip ro a” command for every IP address that you lease. Please note, that you cannot use “$your-ip-subnet/29”, because that would render the first and last IP addresses in the subnet useless.
Now you only need to setup your guest machine’s network interfaces. That’s fairly easy, once they’re installed:
auto eth0
iface eth0 inet static
address $your-hetzner-address
netmask 255.255.255.255
pointopoint 10.9.8.7
gateway 10.9.8.7
iface eth0 inet6 static
address 2a01:xxxx:xxxx:xxxx::xxxx
netmask 64
gateway fe80::1
A final note: if you are going to install an Ubuntu or Debian machine, the installer won’t be able to setup the network properly. Proceed as follows.
Run the installer. After trying to configure the network, choose “do not configure the network at this time”. Your next question is the “hostname”. Switch to a virtual terminal, by pressing “alt”-“right arrow” or “alt-F2”. Activate the console and enter the following commands (please note: 8.8.8.8 is the public Google-DNS, feel free to use another DNS-server):
ip address add $your-ip-address/32 dev eth0
ip route add 10.9.8.7 dev eth0
ip route add default via 10.9.8.7
echo 'nameserver 8.8.8.8' > /etc/resolv.conf
ping www.openoffice.nl
If everything went well, your ping command should say “www.openoffice.nl is alive!” You can shorten the “ip” commands: “ip address add” can be abbreviated by “ip a a”, and “ip route add” may be shortened to “ip ro add”.
Now return to the installer by pressing “Alt-F1” or “Alt”-“left-arrow”, and continue the installation. After the installation has finished, you need to enter the correct information in /etc/network/interfaces – see above.