Its a very interesting time for the ole’ television.  For the majority of my life, and probably the lives of many older than me, TV has not changed much.  Sure, there was that big transition from black and white to color, but since then, TV has been largely the same.  In recent years, a big push of innovation has been directed towards the TV marketplace.  The advent of plasma and LCD has brought about flat panel TV’s, high definition clarity and broadcast, the decommission of analog signal, the introduction of LED for “green” sets, and now the push for IP delivery for shows and movies.   Read the rest of this entry »


For once, I was not the first of my friends with the shiny new Apple toy!  You see, I work for an AT&T reseller and we had an internal policy for the iPhone launch that employees would need to wait so that our customers could get iPhone 4′s in hand as quickly as possible.  I can really respect that — it speaks to what HTC is doing for the sake of customer service.   Read the rest of this entry »


HP added the idea of release sets for Bladesystem firmware starting in January.  On a conference call yesterday, we were alerted to the new release set certification process.  Previously, HP had been releasing firmware for the Bladesystem as components and firmware were updated, which they still do, but the idea of a release set adds an additional cross-testing process to ensure that firmware from each component works together correctly.  There was no publicly disclosed certification process prior to January.

To quote the HP engineer on the call — there were just too many combinations and possibilities to be able to certify all available firmwares — and so the idea of a release set began in January.  The release sets are available in a compatibility matrix on the HP Bladesystem page (http://h18004.www1.hp.com/products/blades/components/c-class.html) — look at the Compatibility tab.

The good news is that in our environment, we are close to compliance with the January release set.  The only thing out of compliance for the January release is our blade ROM, PMC and iLO firmware, but, as we were also informed, this is not a good situation to be in.

Something I already knew is that we have a bottom-up firmware topology for the HP Bladesystem — meaning that we have to update the bottom level components first and then move up.  I knew this applied to the interconnects and onboard administrator modules, but it also extends to blade servers also.   In addition, the blade servers are at the lowest level and should be updated first — particularly the iLO.

The reason for this is that new OA and interconnect firmware may introduce features and if the PMC or iLO is not aware of these features due to out-dated firmware, erroneous might be passed and all sorts of things could happen — at worst, random server reboots.

iLO firmware 1.81 is particularly susceptible to the random reboot for Windows blades.  If you are running 1.81 and have Windows OS loaded, you should upgrade to 1.82 as soon as possible.  See http://h20000.www2.hp.com/bizsupport/TechSupport/Document.jsp?locale=en_US&objectID=c01802766 for more information.  Other OSes – Linux, VMware, XenServer – are not affected.


I have had the privilege of working with close friends updating their website as they were working on a new album.  The husband approached me, asking my opinion about a couple different hosted website solutions.  After talking with him about what they’d like to do, I found that their biggest desire was getting a good-looking website together for the new album, but also finding something easy to maintain and update in the future.  I suggested WordPress.

A week after our conversation, I began playing with WordPress on my own webserver and started putting together a website mock up with a template from WooThemes and content from their existing site.  It was fairly easy and I like playing around with websites like that.  Wordpress worked pretty well for the basic site.

WordPress also provided them with an easy way to keep the website up-to-date from the road or from their iPhones as they traveled and snapped photos.  Wordpress is supported and updated as security problems were found and should help them stay secure in the future.

Useful Plugins for Musicians
After putting their existing information into the site, I started looking at what else would be good on the site.  I located the fantastic website of developer Dan Coulter, http://blogsforbands.com/.  He had developed several WordPress plug-ins for bands, including a gig calendar and discography.  I added the discography plug-in to a new install of WordPress and began adding songs from the past album that I had.  It worked wondefully — allowing me to post each album, the songs, words to the songs and links to buy the songs from iTunes, Napster or Amazon’s MP3 store.
It also allowed a link to buy a physical copy…  but from where?

Enter another plug-in – Tips & Tricks HQ – developers of the WordPress eStore.  I had used WP E-Commerce in the past, but it didn’t want to install on my musician’s web host and it didn’t easily offer digital downloads, but WordPress eStore did…   We added the plugin and began populating it items, like T-Shirts and CD’s.

Our musician friends utilize ReverbNation heavily and so I was able to find their widgets (which our friends were already using on their original site) and place those into text widgets on the sidebar.  The cool thing about this approach is that it allows them to update their music players, gig calendar and mailing list all within ReverbNation and have it feed their website.  Likewise, their website is feeding ReverbNation new mailing list addresses for future mailings and it is collecting stats of who is listening to their music.   ReverbNation is a free service, but offers enhanced and additional pay-for services.

As we were going live and testing everything, I found a couple things – like emails being sent from the eStore were sending from “WordPress”, but Tips & Tricks HQ had another plugin to allow us to customize the friendly name of the sender.

If any other bands or musicians are looking for a solution, I’ll be the first to recommend WordPress!


Apple quietly released a revision to the Mac Mini this morning while updating the online store.  The new version features a new unibody Aluminum enclosure for the Mac Mini, a slimmed profile, and best of all HDMI — making it the first Mac with a native HDMI port.

I had been contemplating getting a Mac Mini to replace my aging Apple TV at home and to offer us the capability of watching NetFlix or Hulu directly on our TV.   I had found dongle cables to offer video and audio to HDMI on the last revision of Mac Mini and was just waiting primarily on our house build to get going to see how our money looked as we were finishing the project.  I have been trying to be very good about my technology purchases – since I felt like we had higher priorities. Read the rest of this entry »


Last week, I found a deal I could not pass up.  B&H Photo has a deal on a Drobo for $299 though 6/30/2010.  If you’ve never heard of a Drobo, it is an external storage enclosure from Data Robotics that offers some enterprise-class, automated mirroring/striping for your data across multiple hard drives.  Data Robotics calls it Beyond-RAID because unlike a RAID set where drives should be the same size, their technology allows mix and match drive sizes and handles striping and leveling the data across whatever mix of SATA drives you buy.  If a drive fails, pull it and replace it and the device will rebuild.

I had been worried about losing my digital home movies.  That data is really too large to really push out to a backup service like Mozy and when I load new movies, its usually to the tune of 20 or 30GB at a time, which would take weeks to push up.  In addition to that data, I also have Movies and TV shows that we have purchased through iTunes. Read the rest of this entry »


Like many shops, we have finally attained buy-in from all our stakeholders for virtualization.  As a result, we’ve pushed more and more into our infrastructure.  And while VMware is the most datacenter ready solution for virtualization, it is not without its shortcomings — monitoring and visibility into the infrastructure being one of the biggest.

While we were first deploying VI3 and performing our consolidation, the primary focus was on the non-critical systems and moving them into the virtual infrastructure to get the best utilization of hardware.  Since completion of this phase, the next focus became moving some of our mission critical systems to VMware in order to establish disaster recovery for our non-clustered systems.  Disaster recovery through VMware is accomplished by 1) relocating the boot and data onto SAN storage which is replicated to our secondary data center and 2) by the ability to utilized VMware HA in the event of hardware failure to establish resiliency we do not have on a single-server, hardware deployment.

As we have expanded VMware’s role in our data center, new challenges have emerged.  First, when a network issue is occurring, we don’t have our traditional monitoring tools (like PRTG) in a position where they are able to alert for large changes in traffic.  In our physical environment, HP agents are run and PRTG is able to query against these systems with SNMP to retrieve information about traffic.  In the virtual environment, we don’t run these agents (because they are largely non-applicable since these are virtual boxes).  Our preferred way to monitor is through something that can look directly into the virtualization layer and retrieve information. Read the rest of this entry »


Since we began upgrading our clusters to ESX4, we have been having strange “failed physical path” messages in our vmkernel logs.  I don’t normally post unless I know the solution to a problem, but in this case, I’ll make an exception.  Our deployment has been delayed and plauged by the storage issues that I mentioned in an earlier post.  Even though we have fixed our major problems, the following type errors have persisted.

Our errors look like this:

vmkernel: 19:18:05:07.991 cpu6:4284)NMP: nmp_CompleteCommandForPath: Command 0x2a (0×410005101140) to NMP device “naa.6001438005de88b70000a00002250000″ failed on physical path “vmhba0:C0:T0:L12″ H:0×2 D:0×0 P:0×0 Possible sense data: 0×0 0×0 0×0.
vmkernel: 19:18:05:07.991 cpu6:4284)WARNING: NMP: nmp_DeviceRequestFastDeviceProbe: NMP device “naa.6001438005de88b70000a00002250000″ state in doubt; requested fast path state update…

After several cases with VMware and HP technical support, we are no closer to resolving the issues.  VMware support, for its part, has done a good job of telling us what ESX is reacting to and seeing.  HP support, on the other hand, has been circling around the problem but has made little progress in diagnosing the issue.  We have had an ongoing case for several months and our primary EVA resource at HP has continually examined the EVAperf information and SSSU output that we have sent to HP for analysis.  Those have turned up nothing, and yet the messages continue from VMware.

The errors in the log make sense to me – we are losing a path to a data disk (sometimes even a boot-from-SAN ESX OS disk!) – but why HP cannot see anything in our Brocade switches or within the EVA is beyond me.   Our ESX hosts, whether blade or rack-mounted hardware, are seeing the problems across the board.  The one cluster we waited to upgrade never saw the issues in ESX3.5, but sees them now in ESX4.  And perhaps it is a VMware issue that is just too sensitive in monitoring its storage, but I suspect its something else.   The messages don’t seem to affect operation on the hosts, but it certainly makes investigating problems difficult when trying to determine what is a real problem versus just another failed path message.  Anyone else seeing this?


Among the new technologies introduced with ESX4, I’m particuarly inpressed with the vNetwork Distributed Switch.  We have chosen to slowly introduce the dvSwitches into our environment and transition VM’s over to these switches.  The distributed switch allows us some new capabilities such as centralized management, individual port assignments, retained state after vMotions, port statistics and improved monitoring, and rate limiting.    That said, the distributed vSwitch has posed some challenges in the transition to it and from a design perspective.

Transition
The transition to the dvSwitches was an unexpected complication.  In vCenter, the port group names, although identical, are presented differently in vCenter.  Because of this, you could not simply VMotion from an ESX3.5 node to ESX4.  As a solution, I decided to make an temporary standard vSwitch on one node of the new ESX4 cluster as the destination for all my vMotions.  Each of my dvSwitches on the ESX4 cluster had two uplinks, so I stole one uplink for each of my temp standard vSwitches.  This allowed me to seemlessly change the network configuration of each VM after it vMotioned from the standard port group to the distributed port group.   Although it was more steps and trouble, it allowed me to make the transition during the day while production workloads stayed online with no impact to them.

Design Considerations
vNetwork Distributed Switch has certainly saved me time, and its only been in my VMware environment a really short time.  I like several things that the distributed switch allows – such as persistent ports, port statistics and the ability to maintain the distributed switch in one place across all nodes in my cluster.  The downside to this centralized administration is an increased dependence on vCenter Server.

Rich Bramley of VM/ETC posted on the challenges of distributed switches with virtualized vCenter instances — largely the problem of a cyclical relationship that could leave you between “the vDS rock and a hard place,” as he puts it.   He and I drew the same conclusions and ultimately, I have decided to keep a mix of distributed and standard vSwitches within my environment.

Perhaps I am overly cautious, however, I run my service console, VMotion, FT Logging all across VMware Standard Switches.  All VM traffic across the vNetwork Distributed Switch.  The exception to the “all VM traffic” rule will be when  if we introduce a virtualized vCenter instance on our cluster.  The dependance on vCenter for dvSwitches and having a virtualized vCenter assigned to a distributed port group makes for a potential disaster, since the VM cannot access or bind to his port on the distributed vSwitch without vCenter running.  Also of note,  Rich reports that vCenter on a distributed switch is NOT supported by VMware because of the cyclical relationship between the two.


HP provided us with the best practices document for ESX4 connected to an HP EVA array.  There is a major change in ESX4.  For the first time, ESX is ALUA (Asymmetric Logical Unit Access) aware.  (See this post on Yellow Bricks for more detail about ALUA.)  ALUA allows the array and ESX to determine the optimal path — the path to the managing controller’s ports in the EVA’s case — and use those optimal paths until one isn’t available.  This is important because it prevents flip-flopping on the ESX host.

In previous version of ESX, the desired storage setting was fixed path for the EVA.  In our case, we simultaneously presented the ESX3.5 and ESX4 hosts to the same LUNs, meaning some were fixed and some were set to the ESX4 default, which was MRU.  This caused problems.  After initial issues, we backed away and presented one LUN at a time, performed our VMotions and then unpresented the LUN from the old cluster.  This prevented any flapping issues between controllers. Read the rest of this entry »


top