IPv6 in XCP 1.6

The intent of this post is to document how to enable IPv6 in XCP 1.6 and manage the host using IPv6 transport. I hope Google leads many people to this page, as I wasn’t able to find anything else on the web on the subject. I’d like to see more people experimenting with IPv6 on XCP hosts.

XCP 1.6 is built on an optimized Centos 6 dom0 kernel. Enabling IPv6 is not as simply a matter of editing files in /etc/sysconfig/ as you would for a typical Centos server. XCP takes over network configuration during system start-up. Fortunately, the process is very straightforward. To manage IPv6 networking on an XCP host, you must be comfortable with the ‘xe’ command line tools, as this cannot be performed using XenCenter.

Here are the steps for enabling IPv6 and configuring an IPv6 address.

  1. Log in to the XCP host as root and execute ‘/opt/xensource/bin/xe-enable-ipv6 enable’.
  2. Reboot.
  3. Verify that an IPv6 link-local address appears on xenbr0 using ‘ip -6 addr show dev xenbr0’. You should see a /64 address that begins with fe80 (e.g., fe80::20c:29ff:fe48:5bc9/64).
  4. Configure an IPv6 address on the host using ‘xe pif-reconfigure-ipv6’.

I’ll  provide some sample ‘xe pif-reconfigure-ipv6’ configuration commands.

Static IPv6 address – ‘xe pif-reconfigure-ipv6 mode=static uuid=<PIF_UUID> ipv6=<IPV6_ADDRESS>. The address is specified using the standard IPv6 notation with a slash followed by the prefix length (e.g., 2001:DB8::2/64). Fill in the UUID parameter with the Physical Interface (PIF) UUID of your management domain as provided by ‘xe pif-list’.

Stateless Autoconfiguration  (SLAAC) – ‘xe pif-reconfigure-ipv6 mode=autoconf uuid=<PIF_UUID> ipv6=<IPV6_ADDRESS>’. After executing this command the XCP host will create an IPv6 address using IPv6 Route Advertisements (RA). If there are no routers sending RAs on your network, the XCP host will not assign an address to xenbr0.

DHCPv6 – ‘xe pif-reconfigure-ipv6 mode=dhcp uuid=<PIF_UUID>’. XCP starts dhcp6c after the command is executed; however, the address assignment does not take place. If anyone wants DHCP on the hypervisor, feel free to fire up wireshark and track down the problem. I’ll update the post.

I’ve verified that the ‘xe’ commands can be run remotely using IPv6 transport. XenCenter also connects over IPv6. If you use an IPv6 literal, you must enclose it in brackets as you would in a web browser (e.g., [2001:DB8::2]). As I mentioned earlier in the post, XenCenter cannot configure IPv6 networking.

I want to thank Grant McWilliams for his tips that led me to figuring out IPv6 in XCP 1.6.

UPDATE 9/4/13 – This process also works in Xenserver 6.2.


Why I Use AWS EC2 Reserved Instances

Amazon Web Services (AWS) EC2 reserved instances provide a simple-to-use method for reducing AWS costs for small business like mine. When my free tier expired last year, I’d heard of reserved instances but didn’t recognize how the how simple it is to save money using them rather than on-demand instances.

We’re been told that EC2 instances are not servers. I agree completely. People who use EC2 instances as simple VPS-like servers are using a fraction of AWS’ capabilities. You can find very inexpensive VPSs from Joe’s Datacenter (btw, kudos to Joe’s for native IPv6). If you are not technically inclined, you’ll find VM with cpanel already pre-installed much easier to use than AWS EC2. I choose AWS because I wanted experience with the API and other AWS services.

If you run your company’s web page on EC2, the instance will be running 24×365. You can commit to a certain number of hours using reserved instances. AWS charges an upfront fee in addition to reduced hourly fees.

Let’s look at an example. You decide on a micro instance for a lightly utilized web server (I recommend you test your load on a micro instance before buying reserved instances. Some people are unhappy with the performance.). We’ll use a Linux instance in the Northern Virginia region.

All prices listed from 3/30/2013 in USD.

The cost for an on-demand Linux instance is $0.02/hour, or about $175/year.

The cost for a reserved Linux instance (light utilization) has an upfront cost of $23. The hourly pricing is $0.012/hour, or about $105/year. Add in the $23 upfront fee for a total of $128/year.

27% savings isn’t bad at all.  Increasing the commitment or selecting a bigger instance increases the discount. AWS claims on its web site that the savings can be up to 65%. In absolute terms, this is chump change for a person serious about running a business. Even so, I think awareness of the savings with reserved instances is good to know if you are setting up instances for friends, family members, and charities.

Check out AWS’s reserved instance page for additional information.

The AWS VPC and the Network Engineer

Amazon AWS is doing amazing things with its IaaS platform. As a networking guy, I find the networking features very impressive. AWS made a wise choice in using Layer 3 as the networking foundation. I suppose AWS engineers recognize what should be a widely held belief in networking–Layer 2 does not scale. The connection of the VPC to corporate data centers presents a compelling value proposition for customer interested in offloading work to the cloud. What I want to focus on in this post is how the integration of cloud and corporate network affects the network engineer.

I design IP networks for my clients. I know my way around basic Linux system administration and can probably figure most things out with patience and Google. I respect talented sys admins who understand the service that the IP network provides to their systems and can communicate simple network conditions (e.g., “I can’t ping the default gateway”). Who will be integrating the VPC and the corporate network? Clearly, both network engineers and sys admins will be involved. You wouldn’t want a sys admin making critical IP design decisions any more than you’d want me standing up a hadoop cluster.

Network engineers will have to adapt their thinking to the virtualized environment. This is a new way of thinking about moving packets. Networking components in the physical world are about as un-elastic resources as possible. I would argue more so than servers. Getting to a point in which network engineers can grasp the flexibility in VPC is going to require investment on their part in learning–the same way learning IS-IS would for an engineer who knows OSPF.

Educating network engineers in VPC networks is in Amazon’s best interests. It’s going to be guys like me who will get calls from potential clients wanting to tie their VPC into their network. The existing documentation does little to further that goal. I had to reach the VPC guide several times before obtaining a degree of comfort. Elastic Network Interfaces? Implied routers? Subnet routing tables? These concepts are not intuitive for network engineers.

Here’s how I recommend that Amazon could educate my networking brethren.

  1. Write a guide on the VPC intended for network engineers. Think about how Juniper write JUNOS documentation for engineers with an Cisco background. This is a very effective way to quickly get smart folks up-to-speed.
  2. Document use cases & recommended architectures for VPC that involve VPC to VPC and VPC to data center connectivity. Cisco excels in this area with its Cisco Validated Designs. Mimic their approach. Today, the documentation is limited to connecting a VPC gateway to a router with IPsec. This barely scratches the surface of how customers will use the networking capabilities of the VPC.
  3. Create online training that steps through the configuration of a VPC. Adding a hands-on component with “actual” VPCs shouldn’t be that difficult for a company that does virtualization at a massive scale.
  4. Talk to internal and external networking savvy engineers. I’ve met some sharp engineers who work on Amazon’s backbone. By engaging them and engineers outside of Amazon, the company could gain valuable insight on networking.

Migrating to the VPC should be as frictionless as possible for businesses. The accelerated set-up of a stable and scalable VPC will translate into more revenue for Amazon.

How to Share Content over IPv6 with AWS EC2

Although EC2 instances are not IPv6-capable as of this writing, Amazon has implemented IPv6 for its US East (Northern Virginia) and EU (Ireland) Elastic Load Balancers. I’ll demonstrate how to make IPv6 content available using EC2 and the load balancers. Please note that Amazon is currently offering new customers EC2 micro instances at no charge if you remain under certain thresholds.

Instance Set-up

  1. Install a Linux-based Amazon Machine Instance. If you want to follow along with this tutorial, use a Ubuntu 10.04 LTS instance that Canonical uploaded to the Community AMIs (AMI ID ami-63be790a). Use US East or EU (Ireland) servers. If this is your first time setting up an instance, I recommend viewing Greg Wilson’s tutorial on Youtube.
  2. Log in using the “ubuntu” user name. Use the ssh private key as described in the video.
  3. Install the packages required for a LAMP server. A simple way to do this is to “sudo tasksel --section server”. Select “LAMP server” in the graphical installer. Strangely, the LAMP selection does not install PHP. I did this manually with “sudo apt-get install php5-cli”.

Load Balancer Set-up

  1. Click on “Load Balancer” in the “Network & Security” left panel of the AWS Console. Click the “Create Load Balancer” button.
  2. Give your load balancer a name. I used the default HTTP entry. For the health check, I used the default settings.
  3. Add your instance to the load balancer.
  4. Now that the load balancer is created, place a check next to its entry so that detailed information appears in the bottom panel.
  5. Write down your IPv4, IPv6, and dual stack DNS names.
  6. Click on the Instances tab in the bottom panel. Make sure the instance’s status indicates “In Service”. Note:  I’ve noticed that the time required for the health check to add the instance into service can be 20 – 45 minutes.

Testing DNS and Load Balancer

  1. Use dig or nslookup to verify that you get A (IPv4) and AAAA (IPv6) records. This verification step is primarily for your information.
  2. ubuntu@ip-10-244-171-28:~$ nslookup
    > Jeff-LB-Test-1796974432.us-east-1.elb.amazonaws.com
    Non-authoritative answer:
    Name: Jeff-LB-Test-1796974432.us-east-1.elb.amazonaws.com
    > set type=AAAA
    > ipv6.Jeff-LB-Test-1796974432.us-east-1.elb.amazonaws.com
    Non-authoritative answer:
    ipv6.Jeff-LB-Test-1796974432.us-east-1.elb.amazonaws.com has AAAA address 2406:da00:ff00::3213:dcb8
    Authoritative answers can be found from:
    > dualstack.Jeff-LB-Test-1796974432.us-east-1.elb.amazonaws.com
    Non-authoritative answer:
    dualstack.Jeff-LB-Test-1796974432.us-east-1.elb.amazonaws.com has AAAA address 2406:da00:ff00::3213:dcb8
    Authoritative answers can be found from:

  3. Create a script called test.php with the following text.
    $headers = apache_request_headers();
    $ip = $headers["X-Forwarded-For"];
    if($ip) {
      print "X-Forwarded-For header is $ip";
    else {
      $ip =  getenv('REMOTE_ADDR');
      print "IP is $ip";

    Amazon’s Elastic Load Balancers will set the X-Forwarded-For header to the IPv6 source address. If the connection is made via IPv4, the X-Forwarded-For variable is undefined. Put this script in /var/www.

  4. Using your web browser, access http://yourIPv4DNS/test.php, http://yourIPv6DNS/test.php, and http://yourDualstackDNS/test.php. Assuming you are accessing from a dual stack IPv4/IPv6 end host that prefers IPv6, you will see an IPv4 address, an IPv6 address, and an IPv6 address respectively.


Congratulations! Your content is now available over IPv6. Now you can set the CNAME record for your domain to the dual stack DNS name so that users can type in your domain and reach your site via IPv4 or IPv6. For more information on how to use CNAME’s with Amazon EC2, see Using Domain Names with Elastic Load Balancing.

I hope this post encourages people to make content available over IPv6. The days of assuming all end hosts are reachable via IPv4 are over. Amazon’s EC2 and Elastic Load Balancers make transitioning content to IPv6 simple.

Virtualization in the Network Designer’s Toolbox

I’ve found virtualization increasingly useful in my work. I thought I’d share my observations on effectively using virtualization for feature testing, architecture validation, and learning. Virtualization is a very inexpensive way to accomplish tasks that previously required thousands of dollars in lab equipment.

The first decision in employing virtualization is selection where to establish your test environment. The advantage of using a dedicated server is that your applications aren’t competing for resources with the test environment. You can get a server with a lot of memory, which I would advise if you plan on using many virtual machines simultaneously. I prefer Ubuntu Server LTS for headless servers. Ubuntu provides a very stable host OS.

An alternative is creating the virtual environment on your laptop. This comes in very handy if you find yourself without Internet connectivity or you deliver a demo to customers. If you are at a customer site, do not expect to be able to reach your server. There are too many problems that can arise. For my needs, I maintain virtualized labs on both my laptop and office server.

For software, I recommend purchasing VMware Workstation 7.1. VirtualBox has its uses; however, Workstation is a better option. It has features not available in VirtualBox. Let’s take a look.

  • Teaming – Workstation lets you set up a group of VMs in a way that makes it easier to manage the virtual infrastructure. You do things such as start and stop all the VMs in the team. Over time, you end having numerous VMs for different purposes. The ability to group VMs into teams is a convenience.
  • VM Recording/Playback – This is an excellent feature for creating demos.
  • Virtual Network Editor – Creating the virtual network infrastructure is very simple in Workstation. When you combine teaming with the virtual network editor, you can set up new labs very quickly.
VMware Workstation has additional benefits. For those new to virtualization, the software introduces you to concepts and terminology that VMware uses across its product family. I use Workstation for one of the same reasons I use Ubuntu. When something breaks, you can almost inevitably find someone else who has encountered the problem by doing a web search. Don’t expect this if you use VirtualBox. Don’t get me wrong– I’m a proponent of open source software. In this case, the better product is clearly Workstation. On a related note, avoid qemu and its derivatives like the plague. Setting up bridging by hand and figuring out poorly documented command line flags is a hassle you don’t need.


Be very wary about connecting a virtualized environment to the old Cisco router you have in storage. I’ve made the mistake of trying to connect VMs and tangible networks. For performing testing, do this as a last resort. You don’t want to spend time when something breaks figuring out if the problem lies in the interconnection of physical and virtual gear.


To give you an idea of how I use virtualization, I’ll share several items on my to-do list. (Can you guess that I’m thinking about IPv6 a lot these days?)
  • Ecdysis NAT64/DNS64 – While I wouldn’t recommend beta software to clients, I don’t have commercial NAT64/DNS64 products in my lab. I want to investigate the IPv6-only user experience across various OSes.
  • Linux installation with IPv6-only connectivity– After doing some basic testing, I suspect that the developers of some distributions assume that end stations are dual stack. For example, I’ve been unable to get CentOS to install with only IPv6 connectivity. The installer sends DNS queries for A records only. I hope to write a report on the state of IPv6-only installations across the major distributions prior to end of year.
  • IPv6 Router Advertisement Option for DNS Configuration (RFC5006) – Recently there has been discussion on the v6ops list about replicating functionality in both DHCPv6 and SLAAC.  As a core guy, I haven’t worked extensively DHCPv6. I’d like to see DNS server assignment as explained in RFC5006. I believe only Linux supports the RFC. I’ll confirm.
If you are like most engineers, you enjoy taking things apart and understanding the details of how they work. Virtualization gives you the ability to do this without a big investment. Go forth and virtualize.