vCenter Orchestrator is installed by default along side of vCenter Server Standard edition and can run on the same host, but in other environments, it may make more sense to deploy it as a vApp instead.  If you’re interested in deploying a vApp (which I did not do), see this post from Cody.  I’m going to detail my installation procedures as an example of the install, which is fairly simple.  These are by NO means the official procedures, just my experience and anything I learned along the way.  But first…

What does vCenter Orchestrator Do?

vCenter Orchestrator is, at heart, a workflow tool.  It is used to string together a series of tasks that can be kicked off repeatedly and performed on different objects within vCenter and with additional system, using additional plug-ins.  It is used to automate repeated tasks that are done in the environment, handle bulk operations and handle integration points.  It can be used to receive actions from help desk and monitoring systems and kick off actions based on tickets or alerts.

As I have written about before, I have come to define a cloud as an pool of compute, storage and networking resources with a self-service portal and lifecycle management to automate provisioning, management and decommissioning systems.  The key to this definition is in the automation and workflow of processes.  Although it is not as advanced, I think vCenter Orchestrator can represent a solid first step towards automation and workflowing deployments and management of datacenter assets.  Said another way, I think its a solid first step for many organizations towards the cloud.

My Installation Procedures (as an example)

In our environment (and I assume in most), vCenter Orchestrator is installed by default along with vCenter Server.  The primary requirement to bring up vCenter Orchestrator is an additional database, either Oracle or Microsoft SQL Server. Before you begin, you will want to know the following things:

• Database server, user account and password and database name for vCenter Orchestrator
• Basics of your LDAP configuration and have an account with read access to LDAP
• Have a LDAP group which can be assigned as the vCenter Orchestrator administrators group.
• The name of vCenter Server and an active user account to access the license for vCO.

General

To begin the installation process, you navigate to the VMware folder on the Start menu and choose vCenter Orchestrator Configuration.  A web page launches and you login with the generic vmware/vmware login to begin configuration.

• For security, the first step is to change the generic password to something specific for your environment.  To do this, click the “Change Password” tab under the General section.

Network

The next step is to configure the network configuration, which is fairly simple.  Choose Network on the left sidebar.  Once it loads, there is a drop-down list of IP addresses assigned to the server along with the DNS name to get to the vCenter Orchestrator configuration.  Click Apply Changes and we’re moving along.   In addition to the network, the SSL Certificate is also located here.

• I chose to use the same SSL certificate as my vCenter Server, so to do that use the “Import from URL” option, type in the URL of your vCenter server, and click Import.  The URL would be similar to “https://vcenterserver.domain.int“.

LDAP

After network, you should configure your LDAP integration, so choose the LDAP option on the left sidebar.  You will need an account capable of reading your LDAP (likely Active Directory in many shops).  You have a choice of Active Directory, eDirectory or Sun Java Directory as the LDAP client mode.  Fill in the server names of your LDAP servers (domain controllers).  You will fill in the LDAP root, for instance “DC=domain,DC=int“, depending on your configuration.  At this point, fill in your username. For Active Directory, it can be DOMAIN\username, username@domain.int, or full LDAP distinguished name format.  In my configuration, username@domain.int format worked.  Fill in a password and then apply changes.

Go back down to the “User lookup base:” option and then click Search located to the right of the text field.  If everything is configured correctly above, you should be able to query the LDAP server and locate the OU or CN where your users are located.

• One problem that I ran into with a generic AD configuration, with users stored in the default “Users” folder, is that Orchestrator expects an OU and not a “CN=Users,DC=domain,DC=int“.  I had to configure the User and Group Roots to be the same as my Root.  Your mileage my vary depending on your LDAP configuration.

Last, but not least, you must choose a group in LDAP that defines who is an administrator in vCenter Orchestrator.  So at the “vCO Admin group:” option, choose Search and find the group.  Apply changes, and then use the Test Login option to test everything.

Database

Next up is the Database configuration.  This is fairly straight forward.  Choose the type of database from the drop-down menu.  Enter the username and password to login to the database server.  Enter the FQDN of the database server and the port that the database is listening on.  Finally, enter the database name and instance (if needed).  For Windows Authentication, you may need to enter a domain and tick the “Use Windows authentication mode (NTLMv2)” option.  Apply Changes and if everything communicates, you will be presented the option at the top to install the database (initialize it with the vCO schema).  If there are communication errors, those will be presented.  Just continue to tweak the configuration until everything communicates.

License

Finally, license.  vCenter Orchestrator shares a vCenter license, so it will communicate with vCenter’s licensing service and pick up its license.  You will need to fill in the name of the vCenter server, the port that web services are running on (443 is default, unless you have modified the configuration).  The path will be “/sdk” and then you should add a username and password who can login to vCenter.

• I do not know what permissions this account needs in vCenter to read the license, but from a security perspective, I would try to limit it to only the permissions required.  I’ll take that as a to-do for myself.

Once the configuration is done, you’re ready to restart services and hopefully everything starts up with green lights all the way down the left-hand sidebar.  I found the installation very intuitive and simple to complete.  I was able to have vCenter Orchestrator up and running in just an hour or so.

Learning More

I have scheduled a couple of the out-of-box workflows to handle some weekly tasks in my environment.  It is fairly straight forward to do this, but I’m just getting my feet wet at this point.  Building a workflow seems daunting at this point, even if its just drag and drop.  So, I have pre-purchased and I would recommend Cody’s forthcoming book  to help you and I get a better handle on how to leverage vCenter Orchestrator.

I have, also, lamented in the past that the value of vSphere seems to be eroding, but I believe that a powerful workflow tool like Orchestrator could be a big boost to the value in many organizations.  It is up to customers, like myself, to leverage the weapons that VMware provides and bundles for free to increase the value that vSphere provides. 

Getting a few minutes to talk with Cody was one of the more useful conversations during my trip to VMworld.  I have found a great foundation of information through his blog that you will find helpful, too.  Check out all of his (free) resources on his blog at ProfessionalVMware.com.

Local bursting is a fairly simple concept.  The CloudSystem a customer is delivered is built larger than the customer needs or pays for, and the additional capacity is available to the customer to consume on a pay-per-use, utility basis.  This is a very similar concept that has existed with HP SuperDome and Integrity lines (along with IBM’s OnDemand strategy) for a long time, but its now being applied to cloud technology.   But this concept allows for a customer to have additional local resources to consume without the need for additional capital expenses.

In addition to local bursting, HP is working to make a network of service providers that are compatible with CloudSystem through its CloudAgile service provider program.  These partners will include lots of different cloud providers and customers are not locked into a specific technology, such a vCloud, to choose from.  Savvis is a current HP CloudAgile partner, but others mentioned include SHI and Verizon (who is a vCloud provider).  HP has reiterated on several occasions that it is seeking to create an open cloud architecture which can interoperate with multiple and varied third party cloud providers.

Since bursting is one of the more promising concepts within the cloud and we took some time last week to talk about some use cases that would benefit from bursting:

• School photographers have a large spike in traffic during ‘yearbook’ season when they handle the majority of their business.
• College registration produces a peak amount of traffic for schools trying to get students into their desired classes for the coming semester.
• Rendering farms for movies would also be a good use case, because the majority of the time, render farms are not processing on a movie – it is only really required during large render sessions.
• Scholastic testing is also another seasons activity when users needs must be met and performance is key, but which outside of the season is not nearly as great of a computing load.
• Black Friday for ANY retail e-commerce vendor.

But there was one concept which also came up in our bursting conversations – and its an interesting one I’d like to reiterate here – and it is the issue of capacity with service providers.  Bursting and service providers are a new frontier for many companies, so knowing who can and cannot handle our capacity and needs is an interesting concept to think about.  I would think most service providers are not going to advertise their exact capacity (or lack of) for competitive reasons.  So, in those cases, how does a customer gauge whether or not a service provider can handle their workload?  Its a good question and I don’t have the answer, but I would love to hear what readers think.  One interesting point, is that because HP’s CloudSystem is multi-vendor public cloud, if one vendor cannot handle your load, customers have choice to move to another provider.   While this exists for VMware vCloud providers, too, HP is working so that customers can choose from non-vCloud providers as well.

We spent the week diving deep into the HP CloudSystem solution – viewing the workflows and automation that make it tick, learning about the products included and how they are used, and learning about directions for networking and storage for cloud.  Coming out of the event, I have a stronger understanding of how we could transition our organization towards a cloud and a better idea of the tangible benefits it would enable, something I have struggled to understand thus far.

I was fortunate to be with a diverse and intelligent group of bloggers for the event.  Each of us came from a different background and represented a large cross section of the customers HP services, from small business to enterprise and government resellers.  It left for lots of great conversations and debate about products, offerings, use cases and other details for the businesses we represented.  I’ll encourge you to follow these folks below and their blogs:

VMware is positioning itself as the developers’ friend, much in the same way Microsoft did when the desktop era dawned.  The 2009 acquisition of SpringSource that left many, including myself, wondering what VMware was thinking, but the technology is now emerging as a critical part of the VMware ecosystem and vision for the future.  The middleware for developers is a critical third piece of the VMware vision of cloud which will bridge the two other primary foci of the company.

VMware has long been the king of the virtual infrastructure game.  Their vSphere suite and the type 1 ESX hypervisor are a mainstay in nearly all datacenters around the world.  VMware also got its beginnings with its type 2 Workstation and now VMware Fusion hypervisors.  But as enabling the cloud, a true run-anywhere architecture and bring your own hardware initiatives have emerged, the two disparate product areas left a gap the company has been working to fill.

Based around the virtual infrastructure VMware has developed, the cloud posed a new set of problems that could only be solved by rewriting traditional client/server applications into cloud apps with distributed data models, distributed processing and distributed display logic across multiple datacenters.  And when thinking of cloud, the first thing to come to mind are applications that Google and Facebook have written and how they are architected to be highly available and redundant.

What is vFabric?

The vFabric line up is a multi-product library of solutions to provide developers with the frameworks and middleware needed to build out products on cloud architecture.  The focus is clearly on web applications using customized or VMware-branded versions of the Apache Web Server and Apache Tomcat, known as vFabric Web Server and vFabric tc Server, to develop Java web applications which use the Spring framework.  In addition to the presentation and processing layers, there are also products to handle messaging, monitoring and data storage.  In particular, this year VMware announced innovation to add to the GemFire data models, including a new in-memory SQL solution called SQLFire.  GemFire and SQLFire both seek to upend competitors like Oracle and Microsoft’s traditional relational database products, which VMware is calling out as bottlenecks.  Even while the bottleneck may be true, the real idea is a distributed data model where the data does not exist in a single instance database but rather in a mesh of systems on a cloud architecture.  To me, that is the real difference and news with GemFire.

Looking back & moving forward

While I don’t claim to be a Java developer, I can certainly see a clearer picture of the vision VMware has for cloud.  I was not fortunate enough to attend VMware 2010, so perhaps things would have began to be clearer had I attended, but 2011 certainly provided the clarity I needed.

The unfortunate side to all of this is that VMware is now attempting to engage a new audience of developers, which are catered to in a very different way than the systems administrators and infrastructure staff they have traditionally engaged with.  Saying it in a different way – they have moved past me a bit, as one of the systems guys, and are working on things that excite me less.  I realize we must have applications, but I’m not the guy who writes them, so the thrill of announcements at VMworld 2011 was less for me, than at past conferences.

VMware certainly has a learning curve of how to engage developers, too, but the SpringSource acquisition provided them with a solid foundation and a good set of roots in the open-source community.  For now, I hear that the user communities will remain separate with VMUG’s and Spring User Groups existing beside one other, which is smart.  Hopefully, the same will hold true for the future with VMworld and other events as well.  What I most hope is that their message does not become muddled and muddy because they now have two different audiences to cater to.

I think that the political issue is easier to explain, so I’ll begin there.  Unlike virtualization or virtual infrastructure, the decision to move to the cloud is not going to come from the systems group in most companies.  The systems groups may be able to advocate the change and provide the reasons for it, but it is going to have to be a strategic move from higher management.  The tangible monetary benefits of cloud are less clear than with virtualization, whose primary motivators were increasing utilization of physical machines and reducing the number of physical machines required which saved money.  In other words, the IT systems group is not going to be the primary advocate or decision maker when moving to the cloud.  Cloud is going to need to meet some higher business need rather than a technical need, although there are technical benefits.

I have made the mistake of calling my vSphere deployment at work a “private cloud.”  As I have written about before, I realized earlier this year how incorrect this actually is.  A cloud, by definition, includes automation and a service catalog and includes things like distributed file systems for storage.  Cloud apps are truly things like Google Apps, Facebook and Salesforce.com.  Cloud apps are written differently to make use of new data models and programming techniques to handle distributed computing across multiple machines or even datacenters.  My vSphere deployment at work does not meet these requirements to be a cloud.  We have simply virtualized client and server applications.  My inaccuracy was underlined during the keynotes of VMworld 2011, from both CEO Paul Martiz and CTO Dr. Steven Herrod.

In both keynotes this week, a large amount of time was spent understanding a middle layer which VMware is building to enable developers to build cloud software which is fundamentally different than our “legacy” client/server applications.  There is a push away from relational databases towards distributed models which can be spread over multiple sites.   With tools from the vFabric product line, VMware is seeking to enable developers to adapt their software to work in the cloud.  This week, VMware introduced SQLFIRE, an in-memory SQL product which can do very high performance, low latency data lookups in the cloud.  SQLFIRE joins GemFIRE in the data subset of vFabric tools.

Perhaps it is a lack of experience with the cloud, but it seems like the entire thing is foggy, if you’ll forgive the pun.  Unlike virtualization, that instantly clicked with me, cloud doesn’t make as much sense.  I see the reasons for distributed computing across multiple datacenters, both internal and external, makes sense.  It allows for flexibility in how companies deploy applications and services.  It enables users to consume these services on any device and instantly, but that brings me back to a political issue again.

In many corporate environments, the user’s personal devices are not allowed to be used for corporate access.  That is going to be another major political shift in companies between management and security officers.  I am glad to see that VMware is spending time in making tools to deploy these applications with IT’s controls setup in the beginning instead of an afterthought, but the entire model will make many security officiers cringe.

In addition to security officers, there are great leagues of developers engrained in ‘their’ way of coding who may balk at the new paradigm.  As Steven Herrod said during the Tuesday keynote, a lot of the code written for the cloud is going to come from those under the age of 35.  It is a key demographic for VMware who have yet to become so set in their ways and who are open to change.  But these folks face political resistance from management and from co-workers as they take the cloud journey.

With all of this said, however, I think it is absurd for systems admins and managers at VMworld to simply stick their head in the sand.  Cloud appears to be here to stay and all major vendors are embracing it.  Whether it is a completely private or hybrid cloud model, all companies should be investigating this.  We now have more information with which we can make educated decisions and the mass public can begin its journal upward to the cloud.  But it is not without great challenges, which I believe VMware saw early, and it seems that the vFabric part of the VMware portfolio is the critical piece as the years move forward.

Unfortunately, when multiple vendors’ products are coupled together as a solution, support can become nasty as vendors point back and forth at each other while trying to get to a resolution.  The more complex the solution, for instance a SAN, the more difficult to troubleshoot through the multiple layers of software, firmware and hardware, even multiple vendors of the solution.  And, I believe, the hassle has made customers seek a better way.

Finding a better way

In my employer’s case, they chose to standardize with a single vendor long before I joined the staff.   We have stuck with servers and storage hardware from the single vendor, including their certified part upgrades (no third party upgrade components).  We chose to do this to simplify our support and avoid finger-pointing.

The vendor we standardized with was HP, and the reason was that they offered an entire line of products under their umbrella to meet our needs.  By the time I joined the staff in 2006, we were already HP heavy, except where a specific Unix was required by another vendor.   What we wanted as a customer was the quickest and easiest route  to a resolution, with the least resistance and finger-pointing, when a problem came up.  Even beyond the hardware solutions, HP has handled our software support for Microsoft, RedHat and VMware for many years.  We wanted this because the software companies could not finger point at the hardware or vice versa – HP was doing it all.  Sure, it might happen between teams in HP occasionally, but we could easily escalate our case and have a manager bring this to a resolution.  It has worked well for our needs.

Having all this expertise in-house is an advantage that HP is now branding under the name “Converged Systems” or the “Instant-On Enterprise”.  Earlier this week, I attended a webinar for the Blogger Reality Contest where HP unpacked more of its converged solutions strategies.  HP is bringing together all of the pieces spread throughout its portfolio into specialized solutions.  Its not a new concept, in my opinion, but one that some customers have been already using for years on their own.  HP has improved on this by tweaking configurations  to squeeze performance out of configurations and adding software to ease installation and management of the solutions.

Building Upwards – HP VirtualSystem

HP introduced VirtualSystem in June as a modular, easy and quick way to implement virtualization in customer datacenters.  The VirtualSystem solution is a full package of storage and compute resources plus the software tools to quickly and easily deploy a virtual stack in an environment.

For HP VirtualSystem, the key benefits are:

• Quick built out timeframe
• Automation through Insight Control suite components
• Monitoring through the Insight Dynamics suite components
• Improved virtual machine performance, cost and scale due to purpose built hardware
• Ability to upgrade to CloudSystem for fully automated IT
• Single point of contact for support – HP for compute, storage and software, including hypervisor

HP VirtualSystem comes in 3 levels (shown below).  The VS1 is built out using rack-mount, Proliant hardware for both the server hosts and for the storage and features a P4000 series iSCSI storage array.  It is rated to handle up to 750 virtual machines and can scale up to 8 physical hosts.  The VS2 is built out using HP BladeSystem with a P4800 iSCSI storage array (covered in depth last week).  It is rated for up to 2500 virtual machines and can scale up to 24 physical hosts.  The third offering is the VS3 which is built on HP BladeSystem and the 3PAR Utility Storage platform to provide ultimate scale and performance.  VS3 introduces fiber channel storage capability and scales up to 6000 virtual machines with up to 64 hosts.

In terms of choice, VirtualSystem supports all three major hypervisors from VMware, Microsoft and Citrix.  Using my company as an example again, the multi-hypervisor datacenter already exists.  We are utilizing VMware vSphere heavily and then some Citrix XenServer.  When it came to planning upgrades for our aging MetaFrame/XenApp farm, we looked at virtualization.  As we evaluated XenServer, we found it to be “good enough” for running Citrix XenApp on top of it.  XenApp has its own failover and redundancy built into the application layer, so many of the VMware advanced features did not matter.

For VirtualSystem, HP is also handling all support for both the hardware and software for these solutions.  Having experience with HP’s software support teams, I can report that they do a good job at it.  I would not say they are always perfect, but in general, they have solved our issues and advised us well, so in reality this is a big benefit.  For those who want not on break/fix support, HP offers Proactive 24 Services for an additional level of preventative support.

Building to the cloud – HP CloudSystem

As I learned at HP Discover, just because you have a large virtualization pool in your datacenter does not mean you have a private “cloud.”  The critical difference between a virtual infrastructure and a cloud is orchestration and automation.  Built on top of HP VirtualSystem, HP CloudSystem is a solution that offers all of the necessary orchestration, service catalog and workflows to turn virtual infrastructure into a true cloud.  There is a clear and clean upgrade path from VirtualSystem into CloudSystem.  And for those starting fresh or who want to evaluate the HP solution, there is even an HP CloudStart service which will deliver a rack with CloudSystem into their datacenter and have it fully operational in 30 days or less.

CloudSystem is offered in three levels: CloudSystem Matrix, CloudSystem Enterprise and CloudSystem Service Provider.  CloudSystem Matrix is targeted towards those looking to automate the private cloud, customers who are looking to add automation and orchestration to their existing virtual systems.  It provides infrastructure as a service (IaaS) and basic application provisioning in minutes.  CloudSystem Enterprise extends upon Matrix and allows for private and hybrid cloud, enabling the bursting of workloads to public cloud.  It is a platform for hosting not only IaaS, but Platform as a Service (Paas) and Software as a Service (SaaS).  CloudSystem Enterprise provides application and infrastructure lifecycle management and allows for management of traditional IT resources in addition to virtualized resources.   The CloudSystem Service Provider edition extends upon the Enterprise edition and allows for multiple tenants on a single infrastructure, securely without exposing customer data between customers.  It is intended to host public and hosted private clouds for customers.  The editions in CloudSystem are more about capabilities and less about limits, compared to VirtualSystem.

Since automation and orchestration is the key of CloudSystem, that is where I wanted to focus.  The base of CloudSystem is the Matrix Operating System, which is the same combination of HP software found in the HP VirtualSystem solution.  On top of the Matrix Operating System, the CloudSystem Matrix solution includes Cloud Service Automation for Matrix.  This software includes Server Automation for lifecycle management for physical and virtual assets via a single portal and set of processes and HP SiteScope, an agent-less monitoring solution for performance and availability.

The enterprise and service provider editions include a beefed up version called, simply, Cloud Service Automation.  It includes the entire orchestration, database and middleware automation pieces of the pie and a cloud controller software.  These additional pieces allow not only the automatic and streamlined provisioning of physical and virtual servers but also the provisioning of the required glue that sits in between the apps and the servers.  The diagram below from HP shows all the moving parts of Cloud Service Automation better than I can explain in words.  And because, Cloud Service Automation is total lifecycle management, there are the pieces for monitoring and performance management which would be needed.  In addition, the centralized portals serve as point for both end users and IT professionals to manage the cloud.

Cloud Maps are another feature of CloudSystem and these are predefined automation workflows for deploying software and platforms easily.  These are the piece of the puzzle that allows for improved deployment times and also allow for drag and drop creation of new workflows and processes in the cloud.  HP has worked with its software partners to create these maps of requirements and automate the process of deploying their solutions.

Beyond all of the capabilities, HP is working hard to make this an open solution by making it compatible to burst workloads into third party clouds, whether its Amazon’s EC3 or a vCloud service provider.  This was a point stressed during the announcements at HP Discover and during the call on Tuesday.

This is post number two for Thomas Jones’ Blogger Reality Show sponsored by HP and Ivy Worldwide. I ask that readers be as engaged and responsive as possible during this contest.  I would like to see comments and conversations that these entries spark, tweets and retweets if it interests you and I also request that you vote for this entry using the thumbs up/thumbs at the top of this page.  As I said earlier, our readers play a large part in scoring, so participate in my blog and all the others!

This isn’t the first time I’ve written about CloudSystem.  In June,  I posted about my take on CloudSystem Service Provider from a potential service provider’s perspective.  I encourage you to take a look at that post, too, after you take a minute to comment and/or vote on this post.

Day to day, I work for a telephone company, though we are really much more – we sell cable TV, wireless, landline, Internet, security and home automation.  [Disclaimer - these are my views and not theirs.]  We have tossed around the idea of becoming a public cloud provider more than once.  Last week, during HP Discover, I got my first full view of how we might realistically do that.  We are not currently a HP CloudSystem customer, but I could see it fitting our needs.

HP unwrapped CloudSystem during the Discover conference in Las Vegas.  CloudSystem is a full stack solution from storage, to networking, to servers and software to enable the cloud, whether it be private, public (service provider) or hybrid.  So, where to begin on discussing CloudSystems – lets start with a definition.

What is the cloud?

To begin, this is a grey area.  Cloud means many things to different people.  Most agree, however, that cloud computing and virtualization are not the identically the same.  In many people’s minds, the two are synonymous – mine included – until I really dug into the topic in recent weeks.  I think the VMware, because it is pushing the cloud and because it is known for virtualization has helped further this incorrect concept.

The difference between cloud and virtualization is that cloud computing has a management software layer to enable provisioning of services.  Cloud computing includes a level of automation and management.  In HP’s solution, cloud doesn’t necessarily mean virtualization, although its heavily utilized.    The key to cloud is automation.

Basis of the HP CloudSystem

From talking with an HP engineer in the HP Discovery Zone, I got a clearer picture of what makes CloudSystem tick.  The core of the HP CloudSystem is the HP Server Automation (SA) software package and, optionally, HP’s Rapid Deployment Pack (based on Altiris).  These two packages are designed to bring a level of automation to lifecycle of systems.  SA is used to take requests from end users, create and build out the systems, add the software packages to the base OS, and deploy it to users.  For lifecycle, SA is also used for patch management and change monitoring/mitigation.   The HP engineer I spoke with showed me how an application stack is built out on a newly provisioned system and how applications are added after the fact from a service catalog.

The key here is that the server build process is no longer a manual task for administrators, but rather a standardized task performed by the automation software.  The service catalog is key to making this happen and this is a mentality switch for many IT operations.  Gone are the days of checklists and manual point and click build-outs.  These are replaced with scripted installs for modular components.

The Service Provider’s Role

From a service providers perspective, there are two roles that we can play for customers.  The first is that of a hosting provider – where we fully house their IT operation in our public cloud.  The second is burst public hosting, or a hybrid approach.

The first model, the purely public cloud model, has been around as long as commercial web hosting has been available.  Since shared hosting arrived in the 1990′s, this business model of a for-rent data center, rack or server has housed the majority of websites on the internet.  The difference in cloud hosting, whether its Amazon EC3 or Terremark’s vCloud hosting, is the automation layer for customers.

For many customers, a public cloud only model may work.  But for larger companies with established IT departments, there are many cases where data is too confidential to be hosted in a public cloud, either because of compliance rules or security concerns.  This is where the second model, the hybrid or burstable cloud is coming into play.

The idea of burst is that companies can more “right-size” their infrastructures to handle their normal workload and then push additional resources into a public cloud for the times when their demand exceeds what they are capable of handling in-house.  A good example of this might be a college or university.   Computing resources required during registration increase drastically and as a result, the university spins up several additional registration web servers in the public cloud to accommodate the increased demand.  After registration, the public extension is decommissioned and all computing brought back to their private cloud.

In some cases, such as steaming media providers, they may want a hybrid deployment with a source streaming server and redirector or proxy services in several public clouds to push the workloads out to be handled in several data centers and spread their overall workload.  These are a couple really high-level (simple) concepts, but I think they help illustrate how this could work.

HP CloudSystem’s unique take

I had the pleasure of sitting down with Steve Dietch, VP of CloudSystem, at the Blogger’s Studio last Thursday as they discussed CloudSystem with a group of bloggers.  HP has created CloudSystem as an open cloud stack.  It is hypervisor agnostic, running VMware ESX, Microsoft’s Hyper-V or Citrix XenServer.  And, you are no locked to a single choice per CloudSystem – you can mix and match hypervisors within a CloudSystem deployment, running two or all three side by side.   To go even further, HP is supporting third party x86 servers and storage with their CloudSystem solution.

Unlike other vendor solutions, like vBlock, CloudSystem is not a single size solution.  It is a stack of which customers can pick and choose which components they need to compliment their existing infrastructure or it can be an entire stack purchased as a single SKU.  Deploying CloudSystem does not require you to discard your current infrastructure.

As HP is approaching converged infrastructure like CloudSystem, it is building these solutions out on industry standard (tried and true) hardware with millions of operational hours already logged on this hardware.  In this case, CloudSystem is an extension built on VirtualSystem which in turn uses BladeSystem as a basis.  So, although these are new offerings, they are not truly new.   CloudSystem Matrix is the software used for orchestration and automation in the CloudSystem solution.  This software was formerly known as BladeSystem Matrix, but rebranded to CloudSystem as the overall solution was assembled.

For storage, users can select either a P4000 (Lefthand) iSCSI array or a 3PAR array as the basis for CloudSystem.  Both are modern, fresh approaches to storage and both were designed to handle the unpredictable workloads of virtualized systems.

HP is still working on CloudSystem API’s which will interact with other public clouds to allow bursting and hybrid hosting, so that work is to be released, but the vision is to be able to host with a VMware vCloud provider or any other cloud provider.  They are really seeking to make this solution work with as many other cloud solutions as they can, which is what I would want as a customer.  HP is working hard to avoid single vendor lock-in, in some ways hedging bets on which hypervisor will ultimately win the war.

Finally, and most importantly in a service provider’s case, HP is offering solutions were they can host a public cloud for you in a turnkey solution, where a service provider can begin marketing and selling the solution while having it hosted in HP’s datacenter.  HP can provision your public cloud and hand over the keys for you to use in one scenario.

Disaster recovery (DR) is another scenario where HP’s hosted public cloud might make sense.  In my company’s case, we could bring a Service Provider edition of CloudSystem onsite and offer it as a public cloud to our customers utilizing our high speed network connections in our service area, but we could also replicate the cloud data to an HP datacenter as DR solution.  Since we are in a hurricane prone area, this is an appealing offer where we could assure our customers of DR failover to another geographical area.

All in all, I stand by my assessment that this is the most complete solution that I have seen to date.  I know that VMware has been working very hard to build out the entire stack and vCloud Director has certainly made big steps towards the service model, but VMware is still working on issues such as VM data transit to the public cloud.  HP will likely have the same issues in moving large VM hard disk images from on-premise to the cloud.  This pain point can be lessened when using a local provider with lots of bandwidth available to you, such as the case with my company.  This is still a fast changing area, but I am impressed by what HP introduced last week.

In the interest of full disclosure, HP and Ivy Worldwide invited me and paid for my trip to HP Discover.  Even though, I am trying to relay the information as impartially as possible.

Websites, such as Silicon Alley Insider and AppleInsider, have been reporting that iCloud will include a streaming music offering and have followed speculation and anonymous confirmation from music industry insiders that Apple has struck deals with three of the big four music companies for its cloud offering.  What is yet to be seen is whether the iCloud offering will consume MobileMe, compliment it, or replace it all together.   Speculation has run rampant that MobileMe will move to a reduced rate, free even, though all will be seen after the keynote.

Mac OS X Lion was previously demonstrated and a partial list of features and upgrades were shown at last year’s “Back to the Mac” event.  Lion is touted as the release where many iOS features were incorporated back into the Mac platform which gave birth to iOS, including autosave, instant-resume, full screen apps, the launchpad, and the Mac App Store which launched in January.  Lion also bundles the Mac OS X Server features formerly released as a separate software package, opening new possibilities to power home and business users at no additional cost.

Latest rumors indicate that Lion will be released through the Mac App Store, though that is still to be confirmed.  In addition to new distribution, the new release also appears to include a number of UI refinements which has been the case in each iteration of Mac OS X since its initial release.

Much of what is known about Mac OS X Lion has been divulged from developers who received pre-release versions of the software through the Apple Developers program.   On the other hand, iOS5 and its features should be surprise for attendees and watchers.  The new version of iOS has been kept under close guard and very little concrete information is available about what it may include.  But, with the launch of iCloud, its easy to speculate that both iOS5 and OS X Lion will have deep ties to the new service.

First, SafeCopy looks a lot like Mozy on the surface.  I immediately see that they are cross platform with both Windows and Mac compatibility, but they also list iPhone.  Digging a bit deeper, I have found that SafeCopy also has a search engine feature for finding and retrieving your data online and thus all your files are from the iPhone or other mobile phone.  I have to say, the Google-style search for my backup is cool.  The online demo shows that you can see versioning of the individual files, also.   Like Mozy, the backups are encrypted.  SafeCopy uses 448-bit Blowfish encryption according to their website.  

Second, their data caps for the yearly rates are really competitive and more realistic in my opinion.  They show two plans on their website a 200GB and a 300GB a year plan available for $50 and $70 a year, respectively.  I’m currently paying about $100 per year for two computers at Mozy, although both computers have unlimited storage (but only for now).  All of my data currently backed up would fit within the 300GB limit, even with versioning (a lot of my data does not change — music and photo libraries, etc.).

Third, its one account for all our home computers – meaning, while I’ve only ever run Mozy on two computers, I could potentially protect more of our home computers.  This is fundamentally different than how Mozy used to work — where Mozy used to charge by the number of computers, SafeCopy charges by data limit and you’re free to use as many computers as you want.  Mozy will limit its users in both number of computers and the amount of storage moving forward.

Beyond these three points, however, SafeCopy has some very appealing features to differentiate it from other providers that I have seen.  The first of these features is something they call “TurboCopy”.  Essentially, “TurboCopy” is deduplication, and this is one area I found Mozy to be lacking.  For instance, I have two iTunes libraries with the same music, but I backed up both on Mozy (yes, blame me for killing unlimited storage plans by wasting space like this).  With SafeCopy, it should only store a single copy of these files and then allow me pointers for both computers to access them.  That is also the best use for your storage capacity.  By only storing the single copy, you’re not consuming your storage allotment with all the same data.

The other major differentiator is real-time backup.  The SafeCopy software is constantly watching for changes on the file system and when it finds one, it moves that to the backup cloud.    There are no schedules to manage making it even simpler to setup and maintain a backup.  But, this feature also worries me.  I am concerned about saturating my internet connection — a problem I experienced with Mozy even though it had trottling.  Fortunately, SafeCopy offers throttling, also.  Saturating my connection has become a problem since I added a Microcell at home.  When saturated, this obviously affects my ability to make and receive phone calls.

So, I will be beginning my SafeCopy trial this weekend and I’ll be posting my additional thoughts and impressions after I begin the process of backing up to their service.  And that brings me to the best part…  SafeCopy offers a free account with 3GB capacity.  I’ll be starting with this initially to test and setup my computers.  Stay tuned.

And, you can sign up for Safecopy below:

First, lets start with what they changed.  Mozy moved from a backup plan of about $50 per year, per computer to a plan of $5.99 per month for 50 GB and one computer or $9.99 per month for 125 GB and up to three computers.

For many users, this is probably a non-issue.  I think of my friend Tammy who is using the service and this is probably sufficient for her — today.  But she’s a photographer, and the more she takes and imports, the larger her library will become.  I would not classify her as an excessive user, but what she was paying was fair for the service she gets.  And, she’s already gotten her money’s worth with one recovery under belt from the dead eMachine.  She will probably continue with the $5.99 a month plan (an an annual basis with one month free) and will net paying the same amount per year, until she goes over her 50GB.

But, as I said, I could be wrong.  Consumers behavior and habits have changed.  With larger megapixel cameras and lots of video and music files, the average consumer’s library of data is probably growing exponentially.  A remember an audiophile friend of mine had a 60GB iTunes library many years ago, and that was just converting his CDs into MP3 — before he started buying digital.

I’m am certainly in the top 10% of users.  I fit easily into the “blame me” category, as Stephen Shankland explains over at CNet, today.  I have a photo library of well over 100GB, a lot of iTunes music and video (to the tune of over 200GB ), and I have tons of personal video — much of it in 1080p — about 250GB in total.  The personal video and iTunes TV shows have never been backed up on Mozy, but even without them, I have well over 250GB backed up.  So, today’s change affects me greatly.

I would have a very difficult time choosing what to backup in attempts to get under a quota of 125GB.  Photos are most important to me, but equally important is all the music and video I’ve purchased from iTunes, things that I’ve spent my hard earned money on already.  [Sidenote: where is the iTunes cloud service, by the way?  Scan your library and provide access to the video online, on demand without having to push your video to the cloud -- Apple already has it there, after all.]

But cleaning up my Mozy backup is proving very difficult.  First, how do you go about removing the items you don’t want in the Mozy backup cloud?  Kinda tough without any tools to do so.  So far, I can only find one method.  If a file is deleted from your computer, it will be marked for deletion in the cloud.  I may not want to delete the stuff on my computer.  But in other cases, it may force the healthy practice of cleaning up the bad or useless photos in my photo library.  I am hoping that removing a file or set of files like (all M4V video files) will mark them for deletion in the cloud.  Time will tell, I guess, since I’m waiting on my backup to complete.

The other solution for cleaning up is deleting your existing backup and starting fresh – something I really don’t want to do.  If I start fresh, it’ll likely be with BackBlaze, Carbonite or another vendor offering unlimited backups.

Other vendors will likely follow suit.  Mozy was a pioneer of online backup and since that time many other vendors have cropped up.  Peer to peer backup is another possibility using a service like CrashPlan.  I’m already talking that over with my co-worker.

The good news for users is that your unlimited plan will continue until its expiration date.  So, you are still set for any period that you have pre-payed.  And for most users, the $5.99 plan will suffice for you in the future.  One important note, however, is that Mozy is requiring users to login and set their “renewal” plan for the future.  In ominous, bold, red lettering, there is a notice in the Mozy site that your data is at risk for being deleted if you do not select a plan…  You’ve been warned.