Ohai! Figured it was a good time to write a blog post and demo some of the building blocks that may help you to automate the stuff you have to deal with everyday as an infrastructure automation dude so there we go. I will demonstrate how to take advantage of Boxgrinder to create virtual appliances and provision them using the VMWare ESX/ESXi hypervisor and Opscode Chef, plus some glue that I created for the occasion. I’d love to use an open source hypervisor such as KVM or Xen Community but some small bits are missing in my toolbox (they’ll be ready soon though :). To make the tutorial short and palatable, I’ll assume you already know a little bit about Opscode Chef, VMWare ESXi and Ubuntu. This HOWTO will guide you to:

• Spin up a Boxgrinder meta appliance in ESXi using the ruby esx gem.
• Build a Ubuntu virtual appliance from scratch using the Boxgrinder meta appliance.
• Use the Ubuntu appliance generated to create a new virtual machine in VMWare ESXi.
• Bootstrap the Opscode Chef client in that VM, adding it to your Hosted/Community Chef Server.

Pre-requisites For this tutorial you will need:

• An ESXi hypervisor with SSH enabled and root access.
• An Opscode Hosted Chef Server account or a Community Chef Server available somewhere.
• Opscode chef knife configured and working.
• Ubuntu installed in your laptop/server. Most of the stuff will also work in Fedora or even MacOS X too.

Step 1. Install esx and knife-esx rubygems

Some of the gems that will be pulled have native extensions that require headers, compiler, make etc. We also need ruby and rubygems. Let’s install them:

$ sudo apt-get install ruby rubygems ruby-dev libxml2-dev \
            libxslt-dev build-essential git

Now we are ready to install the gems:

$ sudo gem install knife-esx

This will pull chef and the esx gem too.

Step 2. Spin up a Boxgrinder meta appliance

I’ve got a pre-built Ubuntu Boxgrinder meta-appliance that you can download from here.

$ wget http://download.frameos.org/appliances/boxgrinder-meta-ubuntu-oneiric-amd64.vmdk

Let’s create a virtual machine in ESX using the Boxgrinder meta-appliance VMDK you downloaded and the esx command provided by the esx rubygem.

$ esx create-vm --user root --password secret \
                --name boxgrinder-meta-vm  \
                --disk-file boxgrinder-meta-ubuntu-oneiric-amd64.vmdk \
                --datastore datastore1 \
                --memory 1024 \
                --poweron \
                  esx-test-host

(make sure you type the correct password for the root user in your ESX host)

This command will upload the Boxgrinder VMDK to the ESXi server (esx-test-host in my case) datastore1 using SCP. A new VM named boxgrinder-meta-vm is created with one virtual NIC and 1024 MB of RAM. After that, the guest is powered on and gets an IP address from a DHCP server (the Boxgrinder meta-appliance NIC is preconfigured to get the IP dynamically, via DHCP) available in our LAN.

VMware tools are installed in the Boxgrinder meta-appliance so we can list the VMs using knife-esx and see the IP address the VM received:

$ knife esx vm list --esx-host esx-test-host \
                    --esx-user root --esx-password secret
WARNING: No knife configuration file found
Connecting to ESX host esx-test-host...
+----------------------------+--------------+-------------+-----------+
| NAME                       | IPADDR       | POWER_STATE | VMW_TOOLS |
+----------------------------+--------------+-------------+-----------+
| ESXi QA1                   |              | poweredOn   | false     |
| ESXi QA2                   |              | poweredOn   | false     |
| boxgrinder-meta-vm         | 10.60.20.201 | poweredOn   | true      |
 +----------------------------+--------------+-------------+-----------+

Step 3. Create a Ubuntu appliance with vmware-tools installed

Login to the Boxgrinder meta-appliance. All the commands below need to be run inside the Boxgrinder meta-appliance.

$ ssh root@my-box-grinder  (root password is boxgrinder)

Get some appliance definitions from my github repo.

$ git clone http://github.com/rubiojr/boxgrinder-appliances

We’ll use a Ubuntu appliance definition to generate a new appliance:

$ cd boxgrinder-appliances/ubuntu-jeos

Now create the appliance using the boxgrinder-build command:

$ boxgrinder-build -p vmware --platform-config type:personal,thin_disk:true \
                   -l boxgrinder-ubuntu-plugin oneiric-with-vmware-tools.appl

This appliance definition creates a minimal Ubuntu appliance with openssh-server and vmware-tools installed. The Boxgrinder host is x86_64 so the appliance will be generated for that architecture. It will take a few minutes to generate the appliance. After the process is complete, you’ll be able to find the appliance inside the build/ directory in QCOW2 and VMDK formats. Now back to the laptop where knife-esx and esx gems are installed.

Step 4. Spin up a VM using the recently created Ubuntu appliance

Download the ubuntu-oneiric.vmdk file from the Boxgrind meta-appliance using SCP for example. Let’s bootstrap it in the ESXi host using knife-esx. First of all, make sure you’re in your chef-repo directory so knife can read the required configuration. Assuming you have the chef stuff under /home/myuser/chef-repo:

$ cd chef-repo
$ knife esx vm create \
               --template-file ~/.chef/bootstrap/ubuntu11.10-gems.erb \
               --vm-disk /home/rubiojr/tmp/ubuntu-oneiric.vmdk \
               --vm-name ubuntu-oneiric-chef \
               --datastore datastore1 \
               --esx-host esx-test-host \
               --ssh-user ubuntu \
               --ssh-password ubuntu \
               --esx-password temporal

This is similar to the esx command we used back in step 2, plus some extra stuff to bootstrap the chef client and execute the first chef-client run:

1. The ubuntu-oneiric.vmdk is uploaded to the ESX host (esx-test-host).
2. A new VM is created using that VMDK (in datastore1) as the VM disk.
3. The VM is powered on.
4. knife-esx waits for the VM to get an IP (via DHCP in my case) and uses the ubuntu11.10-gems.erb bootstrap template (which is a typical knife bootstrap template) to bootstrap (via SSH) the chef client inside the VM.
5. Finally the chef-client is run to register itself in your chef server.

–ssh-user and –ssh-password are the user credentials used when bootsraping chef (the where automatically created for you by Boxgrinder when the appliance was built). knife-esx accepts a few more parameters to tune the VM memory (default 512MB), add recipes/roles to the bootstrap process, etc. Have a look at the knife-esx Github project to get all the details.

Links to the relevant stuff

• Boxgrinder appliance builder: boxgrinder.org
• The knife-esx gem: github.com/rubiojr/knife-esx
• The esx gem: github.com/rubiojr/esx
• Opscode Chef documentation: wiki.opscode.org

Enjoy! P.S. The article is work in progress. It will be polished in the next couple of days.