VMware is positioning itself as the developers’ friend, much in the same way Microsoft did when the desktop era dawned.  The 2009 acquisition of SpringSource that left many, including myself, wondering what VMware was thinking, but the technology is now emerging as a critical part of the VMware ecosystem and vision for the future.  The middleware for developers is a critical third piece of the VMware vision of cloud which will bridge the two other primary foci of the company.

VMware has long been the king of the virtual infrastructure game.  Their vSphere suite and the type 1 ESX hypervisor are a mainstay in nearly all datacenters around the world.  VMware also got its beginnings with its type 2 Workstation and now VMware Fusion hypervisors.  But as enabling the cloud, a true run-anywhere architecture and bring your own hardware initiatives have emerged, the two disparate product areas left a gap the company has been working to fill.

Based around the virtual infrastructure VMware has developed, the cloud posed a new set of problems that could only be solved by rewriting traditional client/server applications into cloud apps with distributed data models, distributed processing and distributed display logic across multiple datacenters.  And when thinking of cloud, the first thing to come to mind are applications that Google and Facebook have written and how they are architected to be highly available and redundant.

What is vFabric?

The vFabric line up is a multi-product library of solutions to provide developers with the frameworks and middleware needed to build out products on cloud architecture.  The focus is clearly on web applications using customized or VMware-branded versions of the Apache Web Server and Apache Tomcat, known as vFabric Web Server and vFabric tc Server, to develop Java web applications which use the Spring framework.  In addition to the presentation and processing layers, there are also products to handle messaging, monitoring and data storage.  In particular, this year VMware announced innovation to add to the GemFire data models, including a new in-memory SQL solution called SQLFire.  GemFire and SQLFire both seek to upend competitors like Oracle and Microsoft’s traditional relational database products, which VMware is calling out as bottlenecks.  Even while the bottleneck may be true, the real idea is a distributed data model where the data does not exist in a single instance database but rather in a mesh of systems on a cloud architecture.  To me, that is the real difference and news with GemFire.

Looking back & moving forward

While I don’t claim to be a Java developer, I can certainly see a clearer picture of the vision VMware has for cloud.  I was not fortunate enough to attend VMware 2010, so perhaps things would have began to be clearer had I attended, but 2011 certainly provided the clarity I needed.

The unfortunate side to all of this is that VMware is now attempting to engage a new audience of developers, which are catered to in a very different way than the systems administrators and infrastructure staff they have traditionally engaged with.  Saying it in a different way – they have moved past me a bit, as one of the systems guys, and are working on things that excite me less.  I realize we must have applications, but I’m not the guy who writes them, so the thrill of announcements at VMworld 2011 was less for me, than at past conferences.

VMware certainly has a learning curve of how to engage developers, too, but the SpringSource acquisition provided them with a solid foundation and a good set of roots in the open-source community.  For now, I hear that the user communities will remain separate with VMUG’s and Spring User Groups existing beside one other, which is smart.  Hopefully, the same will hold true for the future with VMworld and other events as well.  What I most hope is that their message does not become muddled and muddy because they now have two different audiences to cater to.

I think that the political issue is easier to explain, so I’ll begin there.  Unlike virtualization or virtual infrastructure, the decision to move to the cloud is not going to come from the systems group in most companies.  The systems groups may be able to advocate the change and provide the reasons for it, but it is going to have to be a strategic move from higher management.  The tangible monetary benefits of cloud are less clear than with virtualization, whose primary motivators were increasing utilization of physical machines and reducing the number of physical machines required which saved money.  In other words, the IT systems group is not going to be the primary advocate or decision maker when moving to the cloud.  Cloud is going to need to meet some higher business need rather than a technical need, although there are technical benefits.

I have made the mistake of calling my vSphere deployment at work a “private cloud.”  As I have written about before, I realized earlier this year how incorrect this actually is.  A cloud, by definition, includes automation and a service catalog and includes things like distributed file systems for storage.  Cloud apps are truly things like Google Apps, Facebook and Salesforce.com.  Cloud apps are written differently to make use of new data models and programming techniques to handle distributed computing across multiple machines or even datacenters.  My vSphere deployment at work does not meet these requirements to be a cloud.  We have simply virtualized client and server applications.  My inaccuracy was underlined during the keynotes of VMworld 2011, from both CEO Paul Martiz and CTO Dr. Steven Herrod.

In both keynotes this week, a large amount of time was spent understanding a middle layer which VMware is building to enable developers to build cloud software which is fundamentally different than our “legacy” client/server applications.  There is a push away from relational databases towards distributed models which can be spread over multiple sites.   With tools from the vFabric product line, VMware is seeking to enable developers to adapt their software to work in the cloud.  This week, VMware introduced SQLFIRE, an in-memory SQL product which can do very high performance, low latency data lookups in the cloud.  SQLFIRE joins GemFIRE in the data subset of vFabric tools.

Perhaps it is a lack of experience with the cloud, but it seems like the entire thing is foggy, if you’ll forgive the pun.  Unlike virtualization, that instantly clicked with me, cloud doesn’t make as much sense.  I see the reasons for distributed computing across multiple datacenters, both internal and external, makes sense.  It allows for flexibility in how companies deploy applications and services.  It enables users to consume these services on any device and instantly, but that brings me back to a political issue again.

In many corporate environments, the user’s personal devices are not allowed to be used for corporate access.  That is going to be another major political shift in companies between management and security officers.  I am glad to see that VMware is spending time in making tools to deploy these applications with IT’s controls setup in the beginning instead of an afterthought, but the entire model will make many security officiers cringe.

In addition to security officers, there are great leagues of developers engrained in ‘their’ way of coding who may balk at the new paradigm.  As Steven Herrod said during the Tuesday keynote, a lot of the code written for the cloud is going to come from those under the age of 35.  It is a key demographic for VMware who have yet to become so set in their ways and who are open to change.  But these folks face political resistance from management and from co-workers as they take the cloud journey.

With all of this said, however, I think it is absurd for systems admins and managers at VMworld to simply stick their head in the sand.  Cloud appears to be here to stay and all major vendors are embracing it.  Whether it is a completely private or hybrid cloud model, all companies should be investigating this.  We now have more information with which we can make educated decisions and the mass public can begin its journal upward to the cloud.  But it is not without great challenges, which I believe VMware saw early, and it seems that the vFabric part of the VMware portfolio is the critical piece as the years move forward.

VMworld 2011 will have a lot of new information to share about the newly released vSphere 5, which went live yesterday.  vSphere 5 has a slew of new features, including the introduction of Storage DRS, rearchitected high availblity (HA), port mirroring in vSwitches, VMFS5 and filesystem enhancements, and many more.  I hope to learn more and write about Storage DRS in more depth, as its one of the bigger feature sets added and it encompasses a lot of technology and capabilities.

The week is also one of the best networking (as in people, not wires) events of the year.  With over 17,000 attendees planned this year, it is by far the largest of events I have ever attended.  I’m looking forward to seeing some folks I met at HP Discover this year and hopefully meet some other folks I’ve run into on Twitter.

All in all, I’m really looking forward to next week.  I always come back to work with some grand ideas after VMworld, so I expect this year to be no different.

Early on, I got a taste of success (particularly for a blogger with NO following) by blogging during VMworld 2008 about the announcements being made from VMware.  It was the first time I had content picked up and linked to from others, and I’ll admit, it felt good.  To this day, my blog still gets some traffic from web searches about VMware’s Fault Tolerence feature.

Those early results did not turn into overnight success.  After VMworld 2008 and 2009, the blog has received relatively low numbers of visits, participation and readership. I allowed it to languish with only a few updates here and there.  The growth has been relatively slow, but recently doors are opening up.  Opportunities, like attending HP Discover and the Blogger Reality Contest, are allowing me to attend great networking and technical events,  to report on great new technology being introduced, and to refine my skills as a blogger.  The future for this blog is looking as bright as ever and I just wanted to to take time to thank those involved for the opportunities, support and encouragement.

1:30pm – Updated with revised judging panel.

When I think of a reality show, I think of people competing for prizes, being judged and getting voted out of the house, off the stage or off of the island.   Take that concept and apply it directly to tech bloggers and what would it be?  It would be a very cool contest that I’m going to be taking part it starting the first week in August.

Thomas Jones of NikeTown588.com is hosting a blogger reality show for up and coming tech bloggers. I was lucky enough to meet Thomas during my recent trip to HP Discover and I was really excited when he announced the Blogger Reality Show contest.  I was even more stoked when I got the email that said I had made it INTO the contest.

Thomas has put together a great program which will pit blogger against blogger, duking it out in a war of words – either written or video.  The contestants will attend three online classes, then formulate our blog entries about each class, one a week for three weeks.

The cool part is that you, our readers, will be assisting in the judging process.  Each week, my fellow bloggers and I will create our entries based on the class we attend and then you get to comment and vote for our posts.  Each of the blogger’s sites will have voting somehow built into the posts. In addition to your comments and votes, re-tweets of our information on Twitter will also count towards the scores.  Readers, you hold a lot of power in this competition!

In addition to our readers, my fellow bloggers and I will have a slate of more established bloggers judging our work to make up a second portion of our scores each week. John Obeto (@JohnObeto), Stu Miniman (@Stu), Kristi McDonald (@KRISLMC), Matt Davis (@Da5is), and S. Copeland (@TeckieGirl) will serve as our judging panel for the competition.  Each of our judges are well established bloggers with a significant following online.  They are obviously knowledgeable about social media, blogging and how to do it right.

Since we have judging and scoring going on, that also means, like American Idol, someone is going home each week.  Two bloggers, actually, with the lowest scores and will be leaving the contest each week.  When I read that, it was the first time I got nervous about the contest.  No one wants to fail, but such is life, we can’t all be winners.  Or can we?

The overall winner of the contest will be announced during VMworld 2011 in Las Vegas the last week of August.  Each of us has been awarded with the trip to Las Vegas for the announcement, as well as, attendance to the VMworld event, you could say we have all won already.   The contest is graciously being sponsored by HP and Ivy Worldwide.  There are several other prizes that will be awarded to players, in addition to the overall winner.

It is always good to know your opponent, but in this case, its my first time ‘meeting’ most of these other bloggers.  After reviewing their sites, it is clear that I’ve got some very good competition.  Some of the bloggers are more established and have a significant advantage in readership and experience.  I encourage you to check them out, also, and make sure you read their blogs during the competition.  And now for my competition, the bloggers are…

• Matthew Norwood (@matthewnorwood) (http://www.insearchoftech.com/)
• Maish Saidel-Keesing (@maishsk) (http://technodrone.blogspot.com/)
• Michael Letschin (@mletschin) (http://thesolutionsarchitect.com/)
• Luigi Danakos (@nerdblurt) (www.nerdblurt.com)
• Matthew Brender (@mjbrender). (http://itechthereforeiam.com/)
• David Hurst (@the_super_dave) (http://thesuperdave.com/)

As a side note about the sponsors, HP and Ivy Worldwide brought a significant number of bloggers to the HP Discover event to cover it.  It is extremely good to see a corporate sponsor realizing the value of tech bloggers and the role we can play in disseminating news, perspectives, solutions and our own experiences with their products.  I certainly wanted to send along my thanks to them for underwriting the competition and to Thomas Jones for envisioning and coordinating the opportunity that they have collectively presented to all of the bloggers involved.  I cannot wait to begin!  Let the competition begin! So, readers, help me stay in the competition and not get evicted, fired, or voted off the island.

Beyond internal consumption, all data points to Exchange evolving into a better workload to run within virtualization.  Much of that can probably be attributed to Microsoft’s own virtualization technology, but Exchange on ESX benefits just the same.  Performance gains out of ESX 4 make for a good combination with the improved I/O for Exchange 2007.   Initial data for Exchange 2010 continues the trend of making Exchange a better workload in general and making it more appropriate to virtualize.

Primarily, the session was a presentation of lab work dones to measure the work loads using load generators and test environments.  Those numbers are good to create best practice documents, but sometimes real world experience trumps these experiments.   So by ‘eating their own dogfood’, I think VMware is endorsing the solution whole heartedly.  No more, yes it should work with no problems.

But in the real world, we all realize there will be problems from time to time, and so support is a big issue.  Microsoft, for its part, does not endorse running much of anything in a virtual machine.  Of course, we all know it works – just look at Microsoft education services – all of their coursework is done in VirtualPC.

The session also covered support options, wisely, as that is a major barrier for adoption.  Avenues for support include a Microsoft Premier support contract, which gets you support in any configuration; Microsoft support via the SVVP (Microsoft Server Virtualization Validation Program) with ESX 3.5 U2, Win 2008 and Exchange 2007; or support through an OEM like HP, etc., who are large enough to be able to offer their own support based on experience.

The session also focused on ways to establish resiliency and failover.  This discussion covered Single Copy or Failover Clusters – the traditional MCSC clustering technique, Cluster Continuous Replication (CCR) for Exchange, VMware Fault Tolerence and protection utilizing VMware HA.  I have a chart below that outlines some of the benefits and drawbacks from each method:

Ultimately, though, of the many solutions offered, there is no one correct way to implement this. It ultimately depends on your needs and which configuration most benefits your scenario. But the session provided lots of practical information for helping make those determinations for your real-world deployment.

Virtual Connect is about consolidation by reducing the number of physical connections required.  From the fiber channel modules, two 4-port VC-Fiber modules connect an entire chassis to the fabric, and then using NPIV, the fiber channel traffic is sent to individual blade servers within the chassis.  The 16 blade slots all share the 4 ports of each VC module.

The new development on the ethernet Virtual Connect side is a technology known as Flex-10.  Flex-10  is a new technology which helps to overcome some of the traditional challenges of blades (such as lack of NICs) when looking at blades for virtualization projects.  Virtual Connect Flex-10 allows up to 4 times as many NIC connections per blade server.  In a simliar way to the fiber channel implementation, a single Flex-10 physical NIC (pNIC) on a blade server can be divided into 4 logical NICs or FlexNICs.  These FlexNICs can be configured to different port speeds and capacities.

A blade server with two integrated Flex-10 NICs will have 8 logicial FlexNICs on the motherboard, and each one of those has it’s own MAC address.  These MAC addresses, like earlier blade NICs, can be overridden in the Virtual Connect server profiles and can be assigned virtualized MAC addresses, so the Flex-10 servers get the same portability that earlier server profiles in Virtual Connect offered.

The Flex-10 technology is a two parter – requiring both Flex-10 Ethernet modules on the interconnect bays of the blade enclosure and Flex-10 NICs in the blade servers.  Fortunately for companies with existing investments, the Flex-10 Ethernet interconnect modules are backwards compatible with earlier integrated NICs on your existing blades.

Each FlexNIC can be sized from 100Mb up to a full 10Gb speeds in 100MB increments.  The incredible thing, is that with the two available mezzanine cards and the integrated pNIC’s, Flex-10 technology allows for up to 24 FlexNICs per blade – an incredible amount in a such a compact space.

We have plans of implementing this technology, so hopefully I can post a follow up from personal experiences once we begin that project.

Data Recovery, like VCB, is an agentless backup technology used to grab either full VM image or file level backups of virtual machines in ESX.  Data Recovery also includes de-duplication technology and backup to disk, where VCB is just a method of obtaining the thin VMDK file to be backed up by a third party solution.  Data Recovery also makes use of innovation in the virtual hardware version 7 which allows for block level change tracking.  Although, Data Recovery can apparently (not 100% sure) backup earlier versions of virtual hardware, it won’t be nearly as fast because they lack the block level changes.

Data Recovery deploys in two parts – a virtual appliance and a plug-in for vSphere client.  The virtual appliance is imported from OVF format and with some basic configuration is ready to begin backups.  An IP must be configured and a VMDK must be added to the virtual appliance as a target for the de-duplicated data.  You may have two destination storage locations of up to 1TB each for a total of 2TB per virtual appliance.

Creating jobs for back are pretty simple.  A nice feature for this is that you may choose folders, hosts or clusters as part of backup jobs – meaning that any new VM’s included in that folder will automatically be backed up in addition to the existing VM’s.  This is a nice function for future-proofing your backup strategy.

Backup jobs are scheduled with a backup window to work and there can be up to 8 jobs at a time running, but the virtual appliance does all the scheduling and deciding of when to run the backups.  After an initial backup is run with all data, incremental backups are run from that point on – grabbing on change blocks.  Retention policies are also set for your backup stores and then enforced to keep a number of versions for your backups so that you can go back to a point in time backup.

Destination storage may be unmounted and exported, backed up or otherwise saved, though this is a manual process.

As another note, this is only available on vSphere hosts and cannot backup ESX 3.5 or vCenter 2.5 infrastructures.

Why would you want to do a long distance VMotion?  In my case, we have two data-centers – geographically close to one another.  We currently stretch our cluster between the two locations and it allows us to float VM’s using VMotion between the two.  The problem is that if we lose our primary datacenter, all storage is presented from here.  Long Distance VMotion is the notion of having two separate clusters, one in each datacenter, and being able to VMotion between them.

What was really news to me from this session (I’ll get to what was presented) was that we can present the same data stores to two different clusters and have them recognized on both clusters.  I am pretty sure I tried this way back in the 3.0 days and it failed to work.  This must have been added in 3.5 or 4.0 – I have not tried in recent years.

So, what was presented?  The three companies worked together to identify and trial a solution to allow for long distance VMotion.  At this point, there is a very narrow set of criteria must be satisified to be support and for this to occur.  Much of the restriction comes on the storage side, but network also presents some problems.  Apparently, everything you need in vSphere is there, if you separate each datacenter into its own set of hosts.

Requirements

• Distance between datacenters must be less than 200 km
• A single instance of vCenter to control both clusters
• Each site must be configured in it’s own cluster – the cluster cannot be stretched
• Dedicated gigabit ethernet network
• A single VMware Distributed Switch stretched across both clusters (ok, didn’t know we could that either)
• Same IP subnet configured on both clusters for the VM to run
• Cisco DCI (Datacenter Interconnect) type technology – if you have something similiar by another vendor, you’ll be supported – this means that you should have a core network that can handle routing traffic to either location for the IP – the VM networks must be stretched between datacenters
• Datacenter storage should be R/W on both sides
• VMFS Storage is presented to both clusters – VM’s in each datacenter are run from the local storage LUNs.
• Must have less than 5 ms latency and at least 622 mbps bandwidth (OC12)
• No FT across sites!

Surprisingly, we have most of this configured in our environment and its been status-quo for us for several years.  The biggest difference between our environment and this spec configuration is that we run a stretched cluster to achieve this.  Our datacenters are very close to one another and we only present storage from our primary datacenter so that we don’t have a split-brain scenario.  But, it does give me new things to think about and talk about with co-workers.  We currently don’t run two clusters or SRM because we like the flexibility to VMotions between datacenters – with that now a possiblity, we may have something new to investigate…