Growing a Virtual RDM in ESX

11 Jan
2010

Short version:  To grow a VM with and RDM in virtual compatiblity mode, you must VMotion the VM after performing your storage rescans to recognize the additional space.  After the VMotion, the guest OS will see the additional space and be able to access it.  

Background
At work, we use raw disk mappings (RDM) some within our VMware virtual environment.   The last VMworld, I was inundated with folks saying that using an RDM for any reason other than a Microsoft cluster is a bad idea anymore.  None of them really gave concrete reasons why, other than saying that VMDK performance had improved so much that there was really no reason to use them.  Their recommendation was to use them only where required, such as clustering.  

In our datacenter, we have chosen to implement RDM’s for large filesystems, typically database applications.  All of our SQL server VM’s and all of our Oracle VM’s run raw disk mappings for their data storage.   Before we felt comfortable about running these workloads in VM’s and knowing how they’d perform, RDM’s gave us a way to quickly move back to physical server, if needed.  That reasons still stands, though our confidence in our virtual infrastructure has increased considerably.  We have no qualms about moving SQL into a VM, as a matter of fact, we prefer it in most cases for redundancy and disaster recovery reasons.   The only physical Microsoft SQL Server is our multi-instance cluster, something that has redundancy and disaster recovery covered.

Growing, growing, grown
We initially liked RDM’s for Windows VM’s so that we could grow our disks using the SAN array and then extend the partition using diskpart.  This always worked well with our physical database servers, so we continued the practice into the virtual world.  We initally implemented physical mode RDM’s for our SQL servers, but we quickly found out that physical RDM’s prohibit snapshots and other features, so we changed to virtual mode RDM’s. 

With the virtual mode RDM, we found that growing the filesystem and then getting the VM to recognize the new space was more difficult.  Most information I saw indicated that you’d need to reboot the VM to recongize the space, but then I happened upon a great post in the VMware forums.  This post said to rescan the host, VMotion the VM and then rescan the VM and the disk space would be recognized.  This worked great.

To use or not to use
I realize that today, growing a VMDK is an easy task.   ESX 3.5 and 4.0 allow you to grow a VMDK from the Edit Settings… menu, but when we began in the 3.0 days, it wasn’t an easy thing to grow a VMDK.   So, we went this direction and we continue today to keep things consistent.  I think its safe to recommend that others use VMDK’s, since I do agree that the performance hit is small for using a VMDK versus a raw LUN.   That advice is sound, but I wanted to post this for others who might be in our situation, with VM’s that were already setup with virtual RDM’s and trying to figure the way to grow their filesystems.


1 Response to Growing a Virtual RDM in ESX

Avatar

ACMComputers

January 27th, 2010 at 2:27 pm

Hi Philip,

Like you, I find myself managing large LUNs (1TB each) presented as RDMs only for the purpose of virtual file servers. The sole reason for being RDMs is that there is no real benefit of putting them onto VMFS due to the 2TB limit (excluding the use of Extents which I won’t go into). If you imagine my scenario if I did use VMFS, I would have 2 file servers each with a 1TB virtual disk hosted on a 2TB VMFS volume and attempts to do anything such as snapshotting or indeed volume growing of the 1TB virtual disks is almost impossible, unless of course extents are introduced. One way to bypass the volume sizing limit without an extent would be to create another virtual disk on a different VMFS volume and then diskpart them together at the OS layer!

How you manage the provision of storage varies between use cases although I find it hard to justify adding the VMFS layer if the use case (high capacity demands) can only offer 1 or 2 virtual disks from it.

Andrew

Comment Form

top