ESXi 5 on Mac Mini – First Problem (with update)

A few weeks ago, I installed ESXi 5 on my new Mac Mini using the installation instructions from Paraguin. Everything was running perfectly until a few days ago. It was all working so well that I was getting ready

Mac Minito move other VMs I have off some old equipment to it and retire the machines.

I’ve discussed my fondness for this configuration and thanks to Chris for the mention on his blog site.

But to my surprise one morning, all of my VMs on the Mac Mini were all off line.

I had to log in directly to the Mac Mini and found that the management NIC (the only NIC, btw) was completely missing. I tried the DCUI method to completely reset the network settings to default and rebooted the Mac. After the reboot, the network card was available again and I was able to successfully reconfigure the management network NIC and everything was back online.

However, after about 10-12 hours, it disappeared again.  I have to start digging into log files next week to see if I can find any indication of what is happening, but there seems to be a pattern of after a period of time working perfectly, it drops off. I’m not sure about the driver loading or checking processes ESXi 5 uses, but it seems to indicate a pattern of trying to reload or re-read the loaded modules or drivers and it just can’t load the Broadcom NIC driver I installed vial Paraguin’s instructions.

What makes this make less sense to me is it ran perfectly for almost a month, with no power cycles or updates and just started failing.

I know from the response on twitter that this is a popular idea, so I’d like to leverage the VMware discussion forums to centralize anything that the community discovers about deploying and supporting ESXi on MacMini. I found a discussion thread that seems to be a good fit here

 

Update 1:

I started with a simple change and set the vmnic0 to 1000 Full Duplex instead of Autonegotiate. It hasn’t dropped off after 46 hours of normal use. I’ll continue to track it and update as things change.

Update 2:

Now you can add another NIC using the Thunderbolt port!  VirtuallyGhetto has a great post on how to do it.

 

Turnkey Linux Virtual Appliances – 10 minutes to Mediawiki

As most of you know that read my blogs or twitter posts, I work with more than one virtualization vendor’s products. VMware has a huge library of virtual appliances, but it’s a bit of a hassle to convert them to other platforms.

Recently I’ve been working on upgrading our training and demo kits.  Part of our upgrade is to have the setups self documented by using a dedicated internal wiki on the kits themselves. I looked at several options including doing a fresh build from scratch.

Eventually, I found Turnkey Linux appliances. I like them because they aren’t already built appliances for one vendor;  instead, they are Ubuntu server .iso files with the software already installed and configured. That way you can install on pretty much any virtualization vendor that supports Ubuntu or Debian as a guest OS. Then it’s just a matter of installing the tools.

I installed a Mediawiki appliance on ESX  (New Virtual Machine>Ubuntu 32-bit, defaults on CPU and RAM and whatever size disk you want) and here are the post install steps to get the  VMware tools installed.

Refresh to the latest repositories.

  • apt-get update

Install the missing packages:

  • apt-get install gcc
  • apt-get install make
  • apt-get install linux-headers-2.6.24-23-generic (check your version of the kernel and match it)

Insert virtual CD ROM image by selecting  ”install vmware tools”  from the VM’s Edit  menu.
mount /dev/cdrom /media/cdrom
cp /media/cdrom/VMwareTools-xxxx.xxx…tar.gz  /tmp
cd /tmp
tar -zxvf VMwareTools-xxxxx.xxx….tar.gz
cd vmware-tools-distrib/
./vmware-install.pl

Select the defaults, you may have to tell it to use a newer version of gcc on one of the questions.

You should have the VMWare tools installed at this point.

Upgrade the system

  • apt-get upgrade

Start wikiing.

Marathon Announces everRun 2G and VM Lockstep

I recently had the opportunity to talk with Steve Keilen and Michael Bilancieri from Marathon Technologies about their high availability products for XenServer. Marathon builds on to their existing HA products by releasing version 5 of everRun for XenServer.

I think that is an important point when it comes to product maturity. According to Steve, “everRun 2G stands for the second generation of our traditional everRun products. These are the products that have been very successful in vertical industries such as process automation and emergency dispatch where Windows has infiltrated and become business critical. What we are doing is bringing a load of features and functions to that traditional market. everRun 2G also adds in more supported platforms including Windows 2008 and 64-bit versions as well as allows for multiple applications to be protected on a single OS instead of just one.” Michael pointed out, “There really is no grey area for mission critical systems requiring fault tolerance. Our offering is version 5 that has been in production for years protecting mission critical applications such as emergency dispatch systems. What the competition is offering is a version 1 beta product. We’ve been working on this product for 15 years and solidified and hardened it, so it’s extremely stable and extremely robust.”

I remember teaching XenServer 3.2 class in Boston a few years ago and that is where I was introduced to Marathon Technologies. They had a couple of their people attending the class and we talked about their HA capabilities. I’ve been impressed by the stability and ease of use of the everRun product.

The second announcement is for the everRun VM Lockstep for Citrix XenServer. Michael said, “Ever since we partnered with Citrix and integrated our HA capability into XenServer, people have been asking us when we were going to add in our everRun FT capabilities as well because we still have the zero downtime requirements but would like to consolidate the server workloads on XenServer. This is the first software based system level fault tolerance for virtual environments. This is true system fault tolerance. We lock-step the virtual machines between one or multiple XenServer hosts. So if you lose a device, a network or storage component or the entire host, the application will continue to run with no interruptions, no loss of data or transactions.”

They have it set up where you can set the level of protection per virtual machine in XenServer. “We refer to this as Level 3 (where HA that comes with XenServer is Level 1). Now customers can deploy their most mission critical applications into their XenServer pools and are not limited to just two hosts. We support multiple servers at the same time running full Level 3 fault tolerance. This supports the free version as well as the essentials. It’s important to point out that VM Lockstep protects the entire stack, from the application, operating system, server hardware, network and storage. The standard approach is to try to create high availability from each of the components such as storage, network and the servers, and then get them all to work together. That’s a lot of work. If any one piece of that chain doesn’t work, your fault tolerant solution is no longer fault tolerant. We harden the whole system, from application, operating systems, storage, network and server. If something fails, the application keeps running.”

The architecture looks like the following graphic. everRun is installed and runs on XenServer under the virtual machines. “We install right on XenServer and it sits there pretty benign.” according to Michael.

I fall into the same redundant device thinking just as everyone else does that architects projects. The one point that jumped out to me was one Michael made, “You don’t need shared storage because we have it covered. You can deploy a completely fault tolerant system at a different geographic Disaster Recovery (DR) site on one server with local storage. If the primary site goes down, the application still runs with no data loss at the DR site. Many customers choose use because they don’t want to buy another SAN for their DR site just to protect a few mission critical systems.”

I think this is really a compelling product announcement that should be considered when designing for zero downtime fault tolerance systems for Windows. I’d like to thank Michael and Steven for their time.

Doubletake Product Suite Announcement

Whenever you are trying to come up with the best solution for physical to virtual (P2V) migrations, there are several things to consider. Depending on your downtime window, you sometimes have to get really creative. When the downtime window is very short, it also shortens the list of suitable methods to accomplish the P2V task. In that mix of options to have in your solution arsenal, Doubletake is a good one. I recently talked with Bob Roudebush from Doubletake about their new product suite.

Replication often happens at the hardware level with SAN mirroring but that can be costly or just not practical, depending on the project requirements. “Figuring out a way to replicate Exchange from one site to another is always a challenge. We do it with real time data replication with software. Because it’s software, some of the biggest benefits are the cost savings and no big costs with hardware.”

I asked about what host systems Doubletake works with currently and what is on their roadmap. “Our software works with virtual machines as well as physical. We’ve figured out with software what a SAN normally does. We have an edition of our product for ESX and one for Hyper-V, and XenServer is on the roadmap. The Linux version allows for block-level replication.”

Doubletake started out with a nice replication offering, but with this recent announcement, they’re seeking to become more. According to Bob, “The Company we’ve become over the years is much different than when we started. We aren’t just a point problem solver. We now have the ability to move data around in real time without the users knowing about it. For the first time, we are unveiling a workload product suite.”

The suite’s key capabilities include:

  • Workload Portability (Double-Take Move): Move workloads between any combination of physical and virtual hardware (P2P, P2V, V2P or V2V) within datacenters for hardware refreshes, or across locations for datacenter migrations and consolidations.
  • Workload Backup (Double-Take Backup): Continuously backup workloads and recover them on-demand to a new physical or virtual machine. Using CDP capabilities, recovery can also be done at the item level to any point in time.
  • Workload Availability (Double-Take Availability): Ensure the availability of critical IT workloads for disaster recovery and business continuity using real-time replication and failover that can protect individual applications, entire servers or virtualized workloads running on VMware ESX or Microsoft Hyper-V.
  • Workload Flexibility (Double-Take Flex): Easily manage any workload by booting it from iSCSI SANs, regardless of hardware.

I think Doubletake Flex is an interesting product. Bob said, “The idea is to make it easier to manage a workload to boot from network instead of local disks. Using the standard hardware you can boot to the iSCSI SAN. Blade servers are a good example. You may not have a lot of storage, and you can boot to an iSCSI LUN. We have a central console and assign MAC address to LUNs and it contacts to our management server to see where it needs to boot from and then it boots from the iSCSI SAN. ”

So, whenever you are faced with a ridiculous downtime window for your conversions, make sure to check out Doubletake’s solutions.

I’d like to thank Bob for taking his time to discuss the new Doubletake product suite.