I think that the political issue is easier to explain, so I’ll begin there.  Unlike virtualization or virtual infrastructure, the decision to move to the cloud is not going to come from the systems group in most companies.  The systems groups may be able to advocate the change and provide the reasons for it, but it is going to have to be a strategic move from higher management.  The tangible monetary benefits of cloud are less clear than with virtualization, whose primary motivators were increasing utilization of physical machines and reducing the number of physical machines required which saved money.  In other words, the IT systems group is not going to be the primary advocate or decision maker when moving to the cloud.  Cloud is going to need to meet some higher business need rather than a technical need, although there are technical benefits.

I have made the mistake of calling my vSphere deployment at work a “private cloud.”  As I have written about before, I realized earlier this year how incorrect this actually is.  A cloud, by definition, includes automation and a service catalog and includes things like distributed file systems for storage.  Cloud apps are truly things like Google Apps, Facebook and Salesforce.com.  Cloud apps are written differently to make use of new data models and programming techniques to handle distributed computing across multiple machines or even datacenters.  My vSphere deployment at work does not meet these requirements to be a cloud.  We have simply virtualized client and server applications.  My inaccuracy was underlined during the keynotes of VMworld 2011, from both CEO Paul Martiz and CTO Dr. Steven Herrod.

In both keynotes this week, a large amount of time was spent understanding a middle layer which VMware is building to enable developers to build cloud software which is fundamentally different than our “legacy” client/server applications.  There is a push away from relational databases towards distributed models which can be spread over multiple sites.   With tools from the vFabric product line, VMware is seeking to enable developers to adapt their software to work in the cloud.  This week, VMware introduced SQLFIRE, an in-memory SQL product which can do very high performance, low latency data lookups in the cloud.  SQLFIRE joins GemFIRE in the data subset of vFabric tools.

Perhaps it is a lack of experience with the cloud, but it seems like the entire thing is foggy, if you’ll forgive the pun.  Unlike virtualization, that instantly clicked with me, cloud doesn’t make as much sense.  I see the reasons for distributed computing across multiple datacenters, both internal and external, makes sense.  It allows for flexibility in how companies deploy applications and services.  It enables users to consume these services on any device and instantly, but that brings me back to a political issue again.

In many corporate environments, the user’s personal devices are not allowed to be used for corporate access.  That is going to be another major political shift in companies between management and security officers.  I am glad to see that VMware is spending time in making tools to deploy these applications with IT’s controls setup in the beginning instead of an afterthought, but the entire model will make many security officiers cringe.

In addition to security officers, there are great leagues of developers engrained in ‘their’ way of coding who may balk at the new paradigm.  As Steven Herrod said during the Tuesday keynote, a lot of the code written for the cloud is going to come from those under the age of 35.  It is a key demographic for VMware who have yet to become so set in their ways and who are open to change.  But these folks face political resistance from management and from co-workers as they take the cloud journey.

With all of this said, however, I think it is absurd for systems admins and managers at VMworld to simply stick their head in the sand.  Cloud appears to be here to stay and all major vendors are embracing it.  Whether it is a completely private or hybrid cloud model, all companies should be investigating this.  We now have more information with which we can make educated decisions and the mass public can begin its journal upward to the cloud.  But it is not without great challenges, which I believe VMware saw early, and it seems that the vFabric part of the VMware portfolio is the critical piece as the years move forward.

VMworld 2011 will have a lot of new information to share about the newly released vSphere 5, which went live yesterday.  vSphere 5 has a slew of new features, including the introduction of Storage DRS, rearchitected high availblity (HA), port mirroring in vSwitches, VMFS5 and filesystem enhancements, and many more.  I hope to learn more and write about Storage DRS in more depth, as its one of the bigger feature sets added and it encompasses a lot of technology and capabilities.

The week is also one of the best networking (as in people, not wires) events of the year.  With over 17,000 attendees planned this year, it is by far the largest of events I have ever attended.  I’m looking forward to seeing some folks I met at HP Discover this year and hopefully meet some other folks I’ve run into on Twitter.

All in all, I’m really looking forward to next week.  I always come back to work with some grand ideas after VMworld, so I expect this year to be no different.

Early on, I got a taste of success (particularly for a blogger with NO following) by blogging during VMworld 2008 about the announcements being made from VMware.  It was the first time I had content picked up and linked to from others, and I’ll admit, it felt good.  To this day, my blog still gets some traffic from web searches about VMware’s Fault Tolerence feature.

Those early results did not turn into overnight success.  After VMworld 2008 and 2009, the blog has received relatively low numbers of visits, participation and readership. I allowed it to languish with only a few updates here and there.  The growth has been relatively slow, but recently doors are opening up.  Opportunities, like attending HP Discover and the Blogger Reality Contest, are allowing me to attend great networking and technical events,  to report on great new technology being introduced, and to refine my skills as a blogger.  The future for this blog is looking as bright as ever and I just wanted to to take time to thank those involved for the opportunities, support and encouragement.

Intel created the Cloud Builder program to invite industry partners into its research facility to experiment and create reference architectures in the lab to publish to customers. Cloud Builders is not about certifying the hardware, it is about creating references for customers who are looking to be a particular type of cloud. One reason Intel is not seeking certification is that cloud technology is changing at a very rapid pace and certification process would take an extended amount of time, during which technology would have already changed.

Cox said that Intel has really focused on building the private cloud model. Intel surveyed its end-user enterprise customers and came up with a list of use cases to focus on with its partners.

While researching with its partners, Intel has been able to find some gaps in technology, particularly in the case of management software for the cloud. But the research has also turned up other gaps, such as trusted compute pools and missing audit information that would be required for compliance.

Intel has also been working on technology such as the hardware root of trust which allows for hypervisors to send a request to the Intel hardware in plain text and receive an encrypted, signed response certifying that the response is coming from the trusted hardware. This was technology that had not found a use case, but in compliance heavy environments, its finally finding implementation.

Auditing has also been difficult to produce for these environments. This is an area that Intel and its partners are still focusing.

For more information about Cloud Builders and to find details about the reference architectures published, head over to http://www.intelcloudbuilders.com.

In the interest of full disclosure, HP and Ivy Worldwide invited me and paid for my trip to HP Discover. Even though, I am trying to relay the information as impartially as possible.

Day to day, I work for a telephone company, though we are really much more – we sell cable TV, wireless, landline, Internet, security and home automation.  [Disclaimer - these are my views and not theirs.]  We have tossed around the idea of becoming a public cloud provider more than once.  Last week, during HP Discover, I got my first full view of how we might realistically do that.  We are not currently a HP CloudSystem customer, but I could see it fitting our needs.

HP unwrapped CloudSystem during the Discover conference in Las Vegas.  CloudSystem is a full stack solution from storage, to networking, to servers and software to enable the cloud, whether it be private, public (service provider) or hybrid.  So, where to begin on discussing CloudSystems – lets start with a definition.

What is the cloud?

To begin, this is a grey area.  Cloud means many things to different people.  Most agree, however, that cloud computing and virtualization are not the identically the same.  In many people’s minds, the two are synonymous – mine included – until I really dug into the topic in recent weeks.  I think the VMware, because it is pushing the cloud and because it is known for virtualization has helped further this incorrect concept.

The difference between cloud and virtualization is that cloud computing has a management software layer to enable provisioning of services.  Cloud computing includes a level of automation and management.  In HP’s solution, cloud doesn’t necessarily mean virtualization, although its heavily utilized.    The key to cloud is automation.

Basis of the HP CloudSystem

From talking with an HP engineer in the HP Discovery Zone, I got a clearer picture of what makes CloudSystem tick.  The core of the HP CloudSystem is the HP Server Automation (SA) software package and, optionally, HP’s Rapid Deployment Pack (based on Altiris).  These two packages are designed to bring a level of automation to lifecycle of systems.  SA is used to take requests from end users, create and build out the systems, add the software packages to the base OS, and deploy it to users.  For lifecycle, SA is also used for patch management and change monitoring/mitigation.   The HP engineer I spoke with showed me how an application stack is built out on a newly provisioned system and how applications are added after the fact from a service catalog.

The key here is that the server build process is no longer a manual task for administrators, but rather a standardized task performed by the automation software.  The service catalog is key to making this happen and this is a mentality switch for many IT operations.  Gone are the days of checklists and manual point and click build-outs.  These are replaced with scripted installs for modular components.

The Service Provider’s Role

From a service providers perspective, there are two roles that we can play for customers.  The first is that of a hosting provider – where we fully house their IT operation in our public cloud.  The second is burst public hosting, or a hybrid approach.

The first model, the purely public cloud model, has been around as long as commercial web hosting has been available.  Since shared hosting arrived in the 1990′s, this business model of a for-rent data center, rack or server has housed the majority of websites on the internet.  The difference in cloud hosting, whether its Amazon EC3 or Terremark’s vCloud hosting, is the automation layer for customers.

For many customers, a public cloud only model may work.  But for larger companies with established IT departments, there are many cases where data is too confidential to be hosted in a public cloud, either because of compliance rules or security concerns.  This is where the second model, the hybrid or burstable cloud is coming into play.

The idea of burst is that companies can more “right-size” their infrastructures to handle their normal workload and then push additional resources into a public cloud for the times when their demand exceeds what they are capable of handling in-house.  A good example of this might be a college or university.   Computing resources required during registration increase drastically and as a result, the university spins up several additional registration web servers in the public cloud to accommodate the increased demand.  After registration, the public extension is decommissioned and all computing brought back to their private cloud.

In some cases, such as steaming media providers, they may want a hybrid deployment with a source streaming server and redirector or proxy services in several public clouds to push the workloads out to be handled in several data centers and spread their overall workload.  These are a couple really high-level (simple) concepts, but I think they help illustrate how this could work.

HP CloudSystem’s unique take

I had the pleasure of sitting down with Steve Dietch, VP of CloudSystem, at the Blogger’s Studio last Thursday as they discussed CloudSystem with a group of bloggers.  HP has created CloudSystem as an open cloud stack.  It is hypervisor agnostic, running VMware ESX, Microsoft’s Hyper-V or Citrix XenServer.  And, you are no locked to a single choice per CloudSystem – you can mix and match hypervisors within a CloudSystem deployment, running two or all three side by side.   To go even further, HP is supporting third party x86 servers and storage with their CloudSystem solution.

Unlike other vendor solutions, like vBlock, CloudSystem is not a single size solution.  It is a stack of which customers can pick and choose which components they need to compliment their existing infrastructure or it can be an entire stack purchased as a single SKU.  Deploying CloudSystem does not require you to discard your current infrastructure.

As HP is approaching converged infrastructure like CloudSystem, it is building these solutions out on industry standard (tried and true) hardware with millions of operational hours already logged on this hardware.  In this case, CloudSystem is an extension built on VirtualSystem which in turn uses BladeSystem as a basis.  So, although these are new offerings, they are not truly new.   CloudSystem Matrix is the software used for orchestration and automation in the CloudSystem solution.  This software was formerly known as BladeSystem Matrix, but rebranded to CloudSystem as the overall solution was assembled.

For storage, users can select either a P4000 (Lefthand) iSCSI array or a 3PAR array as the basis for CloudSystem.  Both are modern, fresh approaches to storage and both were designed to handle the unpredictable workloads of virtualized systems.

HP is still working on CloudSystem API’s which will interact with other public clouds to allow bursting and hybrid hosting, so that work is to be released, but the vision is to be able to host with a VMware vCloud provider or any other cloud provider.  They are really seeking to make this solution work with as many other cloud solutions as they can, which is what I would want as a customer.  HP is working hard to avoid single vendor lock-in, in some ways hedging bets on which hypervisor will ultimately win the war.

Finally, and most importantly in a service provider’s case, HP is offering solutions were they can host a public cloud for you in a turnkey solution, where a service provider can begin marketing and selling the solution while having it hosted in HP’s datacenter.  HP can provision your public cloud and hand over the keys for you to use in one scenario.

Disaster recovery (DR) is another scenario where HP’s hosted public cloud might make sense.  In my company’s case, we could bring a Service Provider edition of CloudSystem onsite and offer it as a public cloud to our customers utilizing our high speed network connections in our service area, but we could also replicate the cloud data to an HP datacenter as DR solution.  Since we are in a hurricane prone area, this is an appealing offer where we could assure our customers of DR failover to another geographical area.

All in all, I stand by my assessment that this is the most complete solution that I have seen to date.  I know that VMware has been working very hard to build out the entire stack and vCloud Director has certainly made big steps towards the service model, but VMware is still working on issues such as VM data transit to the public cloud.  HP will likely have the same issues in moving large VM hard disk images from on-premise to the cloud.  This pain point can be lessened when using a local provider with lots of bandwidth available to you, such as the case with my company.  This is still a fast changing area, but I am impressed by what HP introduced last week.

In the interest of full disclosure, HP and Ivy Worldwide invited me and paid for my trip to HP Discover.  Even though, I am trying to relay the information as impartially as possible.

During the introduction to session 3839, we were all given a disclaimer by the presenter – a disclaimer that this sessions would be a very tape heavy session.  Sadly, I was hoping for some good information about the StoreOnce product.  But I should have keyed in on the word archiving in the title, because the session’s core was about utilizing tape for archival and not backup purposes.

Even while disk-to-disk backup has been making huge strides for backup in recent years, tape has continued to advanced and still offers the most economical media for large data stores.  Traditionally tape has not been a readily readable media since it is linear and moving from place to place takes a lot of time as tape has the wind in the cartridge, but as we found out on Friday morning, HP has not relegated tape to being past its prime.  

HP is moving forward with LTO 5 and new tape drives and libraries as users need additional capacity and large stores for video and imaging applications both for backup and archiving.  LTO continues to be an open standard with several media manufacturers to choose from and several tape drive manufacturers, as well.

Last week, HP introduced a new ESL G3 tape library which is capable of housing up to 15 Petabytes of data within a single library.  With LTO5 cassettes holding 1.5TB of uncompressed data, or 3TB of compressed data, these tapes are becoming more and more capable of housing immense amounts of data at a cost of about $.05 per GB.

But, why is tape still relevant?

HP sees tape with a future in archival.  Tape has a shelf life of approximately 30 years and these tapes are capable of housing data which will not be needed often and can be moved to a lower tier of storage where it is ok if it takes some time to retrieve.  A good example of archiving would be in a medical imaging system.  After 90 days (or even a year), the images are not necessarily needed to be online at all times and this data can be moved to the more economical tape, where it is still available should a request come in from a patient or doctor.

Archiving is different than backup, though I didn’t really think of it differently before this session.  Archiving is storing files in a format where they are readily available to retrieve in a normal file format (just like they were on a hard drive), whereas, backup generally uses some sort of application to perform both full, incremental or differential backups and stores the data in a proprietary file format in most cases.

One of the key innovations that is allowing for archival to tape is the introduction of the LTFS file system with the LTO5 generation of media.  This filesystem is written to a separate media area of the tape and allows for a tape to be mounted just like a USB drive or hard drive onto a computer system.  You view it in Windows Explorer or from the command shell just like any other disk media.   You can read and write to and from the tape like any other filesystem. This is really opening the avenues of allowing tape media to be used for archival purposes.

With traditional backup, you would normally have to restore the files from backup — with LTFS, you are able to read the list, find the file you want (through normal disk based search utilities) and manipulate the file as you need.  Although its only a slight difference, it is a difference.  No additional software is needed for an archive disk as opposed to most backup disks which require backup software of some kind to read the backups.

“Why fly your data first class when coach will do?”

One fact from the University of California at Santa Cruz that surprised me was that 95% of data is never access after 90 days. This fact helps to build a strong case for archival.  Why should companies spend top dollar to keep this data online if it will never be accessed again.  The fault of this argument is detailing which data should be kept online and will be needed or accessed and what will not.

Generally, when you talk about backup (or more aptly restore) you talk about recovery time objects and recovery point objectives.  Once you decide upon how long you can take to restore and then at what point your restore needs to bring you to, you can choose your technology.  Disk to disk, like HP StoreOnce, makes a lot of sense when it comes to backup as it improves speed of backup and speed to recover.  The downside to disk-to-disk solutions is that they burn exponentially more energy than tape.

HP has also developed a full work flow around tape for the video industry.  For videographers shooting in the field, they can backup their digital video onto tape, ship it to post production and then post production can use tape to ship rough cuts for review and even the final products to be output.  There still seems to be a lot of potential uses because of the high capacity and low cost that tape offers.

In the interest of full disclosure, HP and Ivy Worldwide invited me and paid for my trip to HP Discover.  Even though, I am trying to relay the information as impartially as possible.

A couple years ago at VMworld, HP announced a new plugin for vCenter, called Insight Control for vCenter, which pulled from HP System Insight Manager and on-board intelligence of ESX host Proliant servers and displayed system status within vCenter.  Since the introduction, the Insight Control for vCenter package has grown to include a second plugin for storage, introduced in May of 2010.

The two plugins are separate and users have the choice of installing one or the other on the vCenter server when running the installation.  As far as licensing is concerned, the good news is that the storage plugin is completely free of charge, but the server plugin is licensed as part of the Insight Control package, though there is no license check.  The server plugin is a separate package not included with the other Insight Control installation.  To obtain the installation media, go to http://www.hp.com/go/icvcenter.

The goal of Insight Control for vCenter is to enable vCenter administrators a single pane of glass, which administrators are already using to manage vCenter.  By integrating the server and storage components, HP is delivering a package which also adds its own alerts to the normal vCenter alarms framework.  The plugin is very vCenter centric and uses the same user and permissions, adding its own permission groups which can be given to users.

The latest version of Insight Control for vCenter includes the ability to look into Virtual Connect Ethernet on BladeSystem and allows you to visually see how VM’s map back within the Virtual Connect area and to the external ports on the interconnect bay.  This is a big improvement for customers running Virtual Connect and ESX together, which has been a bit of smoke and magic which we couldn’t peer into very much.

The plugin also provides administrators with hooks into the other management software for HP.  Within the server module, there are links for additional information into HP System Insight Manager (SIM), to the iLO card and to the OA from different locations to gain full visibility.  The plugin does not try and replicate all functionality of these other management interfaces.  The same is true for the storage plugin, which includes hooks into CommandView EVA (and I assume other management interfaces, although I have to say I have only seen it demoed on EVA).

The storage plugin includes the ability to provision storage (if allowed by security permission within the plugin and vCenter) directly within vCenter.  It allows the ability to create clones and offload the work of creating the clone to the array instead of ESX.  In addition, the storage plugin also reports paths and references LUNs back to their VMFS datastores and correlates data which was previously difficult to correlate between the storage management and vCenter.

Insight Control Storage for vCenter currently supports all of the current generation of P2000 MSA, P4000 Lefthand, P6000 EVA, XP and P9500 series storage arrays.  Many previous generation EVA products are also supported.  HP is working to integrate support for 3PAR arrays into the storage plugin.

To setup and configure the Insight Control for vCenter, the plugin adds a new icon to the home screen in vCenter where all the passwords and integration can be setup.  Configuration for the storage plugin is configured from within the storage tabs on the ESX hosts.  Once configured against the array or array management software (CommandView EVA, in our demo and our shop), the list of disk groups is presented and the administrator can allow or disallow write access and provisioning to the arrays from within the vCenter plugin.  Save that configuration and the plugin is ready to begin querying the management servers.  Once it populates a view of the storage environment, the tabs populate with a good amount of data.  Data is a cached local copy of data retrieved from the storage management, so a refresh button exists in the storage plugin to pull updated information if changes are made outside of the Insight Control for vCenter plugin.

Of note with the plugin, the system information pulls from both the iLO card on the servers, from HP System Insight Manager (SIM), and with the Onboard Administrators of BladeSystems.  Without HP SIM, some information is not populated.  If you have a host without the Insight Agents loaded, you will only get a partial view of the server and some details will not populate.  For ESXi hosts, the Insight Control plugins use CIM (Common Information Model) to pull the system information.  HP also releases a custom release of ESXi which includes all the Insight functionality baked into the ESXi distribution.

In the interest of full disclosure, HP and Ivy Worldwide invited me and paid for my trip to HP Discover.  Even though, I am trying to relay the information as impartially as possible.

I have found my seat and people are streaming in to find their seats.  The closing keynote is about to begin in a few minutes.  We are currently listening to a live DJ mixing video and music on stage.  You may also watch a live stream from the event at http://www.hp.com/go/discover. Live updates will appear below…

In the interest of full disclosure, HP and Ivy Worldwide invited me and paid for my trip to HP Discover.  Even though, I am trying to relay the information as impartially as possible.

(This went out a little earlier than I wanted, but since its already in my RSS, gonna go ahead and release it.)

I was fortunate to have been invited to HP Discover 2011 by Ivy Worldwide along with a group of other independent bloggers.  I am gracious for the opportunity to attend and blog about the event.  The HP and Ivy team was fantastic as arranging my travel and hotel, setting up an amazing blogger’s lounge to use while we were all on site at the event, and setting up a special bloggers area with tables, power and internet for us during the keynote sessions.

I have to try and describe the blogger’s studio to you.  For the more advanced (moreso that I), there were two video areas to meet with and shoot video with technologists throughout the week.  A lot of video was also shot out on the Discover Zone floor at booths and with representatives.  Refreshments and snacks were never far away.  Lots of counters, tables and plent of hard wired Internet and power were available to us in the secured area.  It was a staging area where we were able to work.  A picture is worth more than words, so see below.

Throughout the week, we were presented with opportunities for Coffee Talks in the blogger’s lounge to speak with HP staff and management, and few other special guests.  This offered us time to pick the brains of these decision makers and influencers within HP and get some details directly from the department heads and directors.

But, what an amazing week.  As we are heading towards the closing section this evening and tonight’s concert with Paul Mcartney, I cannot express how successful of a week this has been on so many levels.  I have been able to network and make contacts within the HP organization, which is invaluable to an HP IT shop like my employer.  I have been exposed to so much new in the HP technology portfolio that I will now bring back to my employer and readers alike.  I have been able to ask more questions and get more technical response about solutions that may be of interest in my company in the future.  I have made some great connections with other bloggers who will continue to contribute to my learning and hopefully the same for me to the community.

In the interest of full disclosure, HP and Ivy Worldwide invited me and paid for my trip to HP Discover.  Even though, I am trying to relay the information as impartially as possible.