Problems with IPv6 and Bridging/Xen

I’ve been using IPv6 on my rootserver for some time now. Last week I migrated to a new rootserver and copied my domU/VM instances over to the new hardware. Everything is working fine so far – except IPv6. The network setup is the same as on the old server: the external interface is eth0. The domU/VMs are hooked up to a bridge, called xenbr0. There’s another bridge for internal communication: xenbr1.

The dom0/Xen host itself seems reachable via IPv6. The /48 subnet is provided by Sixxs and is using a static 6-to-4 tunnel. But the VMs are not reliable reachable, although it is setup in the same way as it was on the old server and where it was working like a charme.

The configs of the hosts are these: 

Xen dom0
config old server new server
System Debian Etch
2.6.18-6-xen-amd64
linux-image-2.6.18-6-xen-amd64
linux-modules-2.6.18-6-xen-amd64
xen-hypervisor-3.0.3-1-amd64
xen-ioemu-3.0.3-1
xen-linux-system-2.6.18-6-xen-amd64
xen-tools
xen-utils-3.0.3-1
xen-utils-common

Debian Squeeze
2.6.32-5-xen-amd64
libxenstore3.0
linux-image-2.6.32-5-xen-amd64
xen-hypervisor-4.0-amd64
xen-linux-system-2.6.32-5-xen-amd64
xen-tools
xen-utils-4.0
xen-utils-common
xenstore-utils
xenwatch

/etc/sysctl.conf net.ipv4.ip_forward=1
net.ipv6.conf.all.forwarding=1
net.ipv4.conf.default.proxy_arp=1
net.ipv4.ip_forward=1
net.ipv4.ip_syncookies=1
net.ipv4.conf.all.rp_filter=1
net.ipv4.icmp_echo_ignore_broadcasts=1
/etc/xen/xend-config.sxp (network-script network-route)
(vif-script     vif-bridge)
(dom0-min-mem 196)
(dom0-cpus 0)
(vif-script vif-bridge)
(network-script network-route)
(dom0-min-mem 196)
(enable-dom0-ballooning yes)
(total_available_memory 0)
(dom0-cpus 2)
(vncpasswd ”)
/etc/network/interfaces

# device: eth0
auto eth0
iface eth0 inet static
  address 85.10.209.30
  broadcast 85.10.209.31
  netmask 255.255.255.224
  up route add -net 85.10.209.0 netmask 255.255.255.224 gw 85.10.209.1 eth0 || true
  up ip route add 85.10.209.0/28 via 85.10.209.1 src 85.10.209.30 || true
  up route add default gw 85.10.209.1 || true

auto xenbr0
iface xenbr0 inet static
        address 85.10.209.30
        netmask 255.255.255.224
        pre-up brctl addbr xenbr0
        up route add -net 78.47.85.144/29 dev xenbr0  || true
        up ip -6 r a 2001:6f8:90e:145::1/64 via 2001:6f8:90e:1:216:3eff:fe55:197c dev xenbr0 ||true
        up ip -6 r a 2001:6f8:90e:146::1/64 via 2001:6f8:90e:1:216:3eff:fe2f:481d dev xenbr0 || true
        up ip -6 r a 2001:6f8:90e:147::1/64 via 2001:6f8:90e:1:216:3eff:fe60:68be dev xenbr0 || true
        up ip -6 r a 2001:6f8:90e:a100::1/64 via 2001:6f8:90e:1:216:3eff:fe70:be dev xenbr0 || true

auto xenbr1
iface xenbr1 inet static
        address 192.168.x.254
        netmask 255.255.255.0
        pre-up brctl addbr xenbr1
        # some internal IPv4 routing

auto sixxs
iface sixxs inet6 v4tunnel
        address 2001:6f8:900:c6e::2
        netmask 64
        endpoint 212.224.0.188
        local 85.10.209.30
        ttl 64
        up ip link set mtu 1280 dev sixxs || true
        up ip route add default via 2001:6f8:900:c6e::1 dev sixxs || true
        up ip -6 route flush dev eth0 || true
        up ip -6 r a 2001:6f8:90e::/48 dev xenbr0 || true
        down ip -6 route flush dev sixxs || true

 # device: eth0
auto  eth0
iface eth0 inet static
  address   178.63.83.84
  broadcast 178.63.83.127
  netmask   255.255.255.192
  gateway   178.63.83.65
  # default route to access subnet
  up route add -net 178.63.83.64 netmask 255.255.255.192 gw 178.63.83.65 eth0

auto xenbr0
iface xenbr0 inet static
        address 178.63.83.84
        netmask 255.255.255.192
        pre-up brctl addbr xenbr0
        up route add -host 178.63.83.104 dev xenbr0  || true
        up route add -host 178.63.83.105 dev xenbr0  || true
        up route add -host 178.63.83.106 dev xenbr0  || true
        up ip route add 178.63.123.128/26 dev xenbr0 || true
        up ip -6 r a 2001:6f8:90e:145::1/64 via 2001:6f8:90e:1:216:3eff:fe89:6c31 dev xenbr0 ||true
        up ip -6 r a 2001:6f8:90e:146::1/64 via 2001:6f8:90e:1:216:3eff:fedc:af5f dev xenbr0 || true
        up ip -6 r a 2001:6f8:90e:147::1/64 via 2001:6f8:90e:1:216:3eff:fe08:8f40 dev xenbr0 || true
        up ip -6 r a 2001:6f8:90e:a100::1/64 via 2001:6f8:90e:1:216:3eff:fe70:be dev xenbr0 || true

auto xenbr1
iface xenbr1 inet static
        address 192.168.x.254
        netmask 255.255.255.0
        pre-up brctl addbr xenbr1
        # some internal IPv4 routing

     
auto sixxs
iface sixxs inet6 v4tunnel
        address 2001:6f8:900:c6e::2
        netmask 64
        endpoint 212.224.0.188
        local 178.63.83.84
        ttl 64
        up ip link set mtu 1280 dev sixxs || true
        up ip route add default via 2001:6f8:900:c6e::1 dev sixxs || true
        up ip -6 route flush dev eth0 || true
        up ip -6 r a 2001:6f8:90e::/48 dev xenbr0 || true
        down ip -6 route flush dev sixxs || true

The config of one of the Xen domUs is this: 

sample domU config
config old server new server
system Debian Lenny
2.6.18-6-xen-amd64
Debian Lenny
2.6.32-5-xen-amd64
/etc/sysctl.conf net.ipv4.conf.default.rp_filter=1
net.ipv4.tcp_syncookies=1
net.ipv4.conf.default.forwarding=1
net.ipv6.conf.default.forwarding=1
net.ipv4.conf.eth0.proxy_arp=1
net.ipv4.conf.default.proxy_arp=1
kernel.shmmax=268435456
net.ipv4.conf.default.rp_filter=1
net.ipv4.tcp_syncookies=1
net.ipv4.conf.default.forwarding=1
net.ipv4.conf.all.forwarding=1
net.ipv6.conf.default.forwarding=1
net.ipv4.conf.eth0.proxy_arp=1
net.ipv4.conf.default.proxy_arp=1
kernel.shmmax=268435456
/etc/network/interfaces

# The primary network interface
auto eth0
iface eth0 inet static
        address 78.47.85.145
        netmask 255.255.255.248
        up ip route add 85.10.209.0/27 dev eth0
        up route add default gw 85.10.209.30 dev eth0
        up ip -6 address add 2001:6f8:90e:145::1/64 dev eth0 || true
        up ip -6 route add default via 2001:6f8:90e:1::1 || true       
        up iptables -t nat -A POSTROUTING -s 192.168.x.96/27 -o eth0 -j MASQUERADE || true

iface eth0 inet6 static
        address 2001:6f8:90e:145::1
        netmask 64
        gateway 2001:6f8:90e:1::1

auto eth1
iface eth1 inet static
        address 192.168.x.1
        netmask 255.255.255.0

# The primary network interface
auto eth0
iface eth0 inet static
        address 178.63.83.104
        gateway 178.63.83.84
        netmask 255.255.255.192
        broadcast 178.63.83.127
        up ip -6 address add 2001:6f8:90e:145::1/64 dev eth0 || true
        up ip -6 route add default via 2001:6f8:90e:1::1 src 2001:6f8:90e:1:216:3eff:fe89:6c31 || true
        up iptables -t nat -A POSTROUTING -s 192.168.x.96/27 -o eth0 -j MASQUERADE || true

iface eth0 inet6 static
        address 2001:6f8:90e:145::1
        netmask 64
        gateway 2001:6f8:90e:1::1

auto eth1
iface eth1 inet static
        address 192.168.x.1
        netmask 255.255.255.0

The dom0 seems to be reachable via IPv6 just perfectly fine. When trying to reach the domU I see packets going through the xenbr0 bridge and reaching the domU eth0 interface. The ICMP6 echo request packets (proto 58) are unanswered there, no echo replies. Strange enough it seems to work from time to time, but mostly not. I’ve also tried to set a generic default route like ip -6 route add default dev eth0, but still no improvement.

Did I miss something when migrating to the new server? Is there any mistake in the configs? Any suggestions are appreciated! 🙂

UPDATE:
This is a tcpdump from the domU, doing a ping from dom0: 

07:35:29.308261 IP6 fe80::e46d:25ff:fe1f:317b > ip6-allnodes: ICMP6, router advertisement, length 56
07:35:29.329411 IP6 gate-hro.ip6.windfluechter.net > vserv.windfluechter.net: ICMP6, echo request, seq 45, length 64
07:35:30.329439 IP6 gate-hro.ip6.windfluechter.net > vserv.windfluechter.net: ICMP6, echo request, seq 46, length 64
07:35:30.451991 IP6 2001:418:4001:3::c657:b0c5 > vserv.windfluechter.net: ICMP6, echo request, seq 52836, length 64
07:35:30.481124 IP6 fe80::e46d:25ff:fe1f:317b > ff02::1:ff70:be: ICMP6, neighbor solicitation, who has 2001:6f8:90e:1:216:3eff:fe70:be, length 32
07:35:31.329906 IP6 gate-hro.ip6.windfluechter.net > vserv.windfluechter.net: ICMP6, echo request, seq 47, length 64
07:35:31.481645 IP6 fe80::e46d:25ff:fe1f:317b > ff02::1:ff70:be: ICMP6, neighbor solicitation, who has 2001:6f8:90e:1:216:3eff:fe70:be, length 32
07:35:32.329284 IP6 gate-hro.ip6.windfluechter.net > vserv.windfluechter.net: ICMP6, echo request, seq 48, length 64
07:35:32.481114 IP6 fe80::e46d:25ff:fe1f:317b > ff02::1:ff70:be: ICMP6, neighbor solicitation, who has 2001:6f8:90e:1:216:3eff:fe70:be, length 32
07:35:33.329951 IP6 gate-hro.ip6.windfluechter.net > vserv.windfluechter.net: ICMP6, echo request, seq 49, length 64
07:35:34.329705 IP6 gate-hro.ip6.windfluechter.net > vserv.windfluechter.net: ICMP6, echo request, seq 50, length 64

UPDATE #2:
The issue is finally solved. The solution can be found here.

Uncategorized

2 thoughts on “Problems with IPv6 and Bridging/Xen

  1. Are you missing this from
    Are you missing this from dom0: net.ipv6.conf.all.forwarding=1

    1. No, I don’t think so:
      No, I don’t think so:

      gate:~# cat /proc/sys/net/ipv6/conf/all/forwarding
      1

      So, it is set.

Comments are closed.