Deploying Stretched vSphere clusters with Site Recovery Manager on SAN Volume Controller

I was parsing my Google Reader feed earlier today and ran across the latest post by Chad Sakac, aka “Virtual Geek” providing an outline of the announcements from Day One of EMC World.   I’m a technologist at heart so I love to read about product updates, new functionality, etc.  I also enjoy to predict (if you can call it that if the predictions are only to yourself) what may come next from a product.

EMC VPLEX is a product I’ve been watching since it debuted both because of the use cases with VMware and the fact that it duplicates some of the functionality of IBM SAN Volume Controller.  EMC announced some VPLEX enhancements today, official press release can be found here and Chad’s post on the topic with the specific VMware relation can be found here.

I prefer to avoid discussing other vendor technology on this blog, my main focus is to provide useful information for VMware related solutions on IBM storage.  However while reading the official EMC Press release a couple of lines really stuck in my teeth….

“EMC is delivering the industry’s only high availability three-site data center protection solution for active/active Hybrid Cloud deployments with RecoverPoint and VPLEX.”

“This new combination delivers the industry’s first solution to combine active-active data centers with third site disaster recovery protection, affording customers the high availability benefits of VPLEX AccessAnywhere™ cache coherency technology and the comprehensive ProtectEverywhere™ of RecoverPoint across all EMC and non-EMC block storage.”

I won’t dissect those lines, I just ask that you read on.

I provided a three part series on this blog around how SAN Volume Controller can be deployed in a “Stretched” configuration, which enables the use of Stretched vSphere Clusters.  Part 1, Part 2, Part 3.

On 10/11/2011 this post  provided information on the SVC/V7000 6.3 code release which included support for longer distance of the SVC Stretched Configuration…up to 300 km!

Finally this post discusses the Site Recovery Manager SRA for SVC/V7000.  Point being SVC is a supported storage system for Site Recovery Manager.

I guess because we have customers who have been deploying SVC in a Stretched configuration (also called Split I/O group) and also replicating with either Metro or Global Mirror to another SVC cluster for years (which combines active-active data center storage with a third site disaster recovery option), I didn’t think to point out it out on this blog.  But I guess since it relevant today I will spell it out.

SAN Volume Controller can be deployed in a Stretched configuration between two data centers up to 300 km.  This provides you with a storage “HA” configuration (see my earlier posts for more details).  Most importantly for VMware, this allows you to have shared storage for hosts at both sites.  The use cases for this are VMotion and HA between sites.  This solution is not for Disaster Recovery!  For this reason the SVC development team has included recommendations in the guidelines for extended split-system configurations (available here) to include replication to a third site using Metro or Global Mirror.   A Stretched SVC is no different than a standard SVC cluster from a software or functionality perspective, so all features like FlashCopy, Replication, Easy-Tier, etc. continue to work the same.  VMware Site Recovery Manager is no different.  Our standard SVC SRM SRA (Storage Replication Adapter) can be used to control VMware data center fail over to a third site.  Even better…the replication/fail over can be to another SVC cluster or Storwize V7000!

So to sum it up…

SVC can be deployed between two data centers in a “Stretched” configuration,  development calls this a Split I/O group system.  This solution allows for VMware VMotion and High Availability between sites.

SVC (Stretched included) provides native replication to another SVC cluster or Storwize V7000.  This solution enables VMware Site Recovery Manager to function.

Stretched SVC can be combined with Metro or Global Mirror to both VMware VMotion/HA between two sites, with replication and SRM to a third site.

I will presenting on this solution at the IBM Technical Edge Conference.  So if you are interested in learning more about this solution please check out session sVC24.

FTC Disclosure: I work for IBM, and endorse the IBM SVC for three-site/stretch cluster configurations.  Any references to EMC, VMware and other companies are for illustrative purposes, based on publicly available information.  I was not paid by any third party to mention their company names or products on this blog, and do not endorse any of their products.

Release – IBM Storage Management Console for VMware vCenter version 3.0.0

A new version of the IBM Storage Management Console for VMware vCenter (3.0.0) has been released.  Enhancements in this version includes:

Support for IBM Storwize V7000 Unified and IBM Scale Out Network Attached Storage (SONAS)

Consistent display of volume names in the vSphere environment

-Any volumes created via the plug-in are now tagged in vSphere panels with the same name as was given when the volume was created.  So if my volume name “Test1″ when I look in the storage management GUI, I also see “Test1″ in my vSphere panels (like the Add Storage Wizard).

Multipath policy for vSphere hosts can now be changed from the plug-in

-This new feature allows for individual LUNs to have the multi-path policy changed which will be reflected across all hosts with access to the LUN, or it can be set for all volumes from a storage system.  An example of this would be setting the default policy on Storwize V7000 to be Round-Robin, and all new LUNs created via the plug-in will be set with this policy.

 

There are other enhancements which you can read about here: http://download4.boulder.ibm.com/sar/CMA/SDA/034vx/0/IBM_MNG_for_VMware_VC_3.0.0_RN.pdf

The release notes for the plug-in are here:

http://download4.boulder.ibm.com/sar/CMA/SDA/034vx/0/IBM_MNG_for_VMware_VC_3.0.0_UG.pdf

The plug-in is available for no cost and works with IBM XIV, Storwize V7000, Storwize V7000 Unified, SONAS, and SAN Volume Controller.

 

Downloading VMware Site Recovery Manager SRA for Storwize V7000

I sometimes forget that everyone may not be completely educated on a product and therefore I overlook some basic and valuable information to communicate.

Case in point…I had a question come up today regarding the VMware SRM SRA for Storwize V7000.  As I covered in my last post, the SRA (Storage Replication Adapter) provides a means for VMware SRM to communicate to the storage array.  The question I received was “How do I download the SRA for Storwize V7000″.

When you browse the VMware SRM support document the Storwize V7000 is listed out as a supported storage system for VMware SRM in its own section (http://www.vmware.com/pdf/srm_storage_partners.pdf)

However when you access the SRM download page (where SRAs are also downloaded), Storwize V7000 is not listed out.  The Storwize V7000 and SAN Volume Controller utilize the same code stack, therefore the storage integrations that IBM and VMware provide work for both products.  In this case the download page only lists the SAN Volume Controller and not Storwize V7000.  But that SRA works for both SVC and Storwize V7000.

 

 

VMware Site Recovery Manager SRA v2.00.11725 for IBM SAN Volume Controller and Storwize V7000

For those of you implementing or planning to implement VMware SRM 5.0 on IBM SVC or Storwize V7000, there is a version of the Storage Replication Adapter (SRA) that is supported.  However be mindful of the support matrix on VMware’s site (http://www.vmware.com/pdf/srm_storage_partners.pdf).  If you look at SVC and Storwize V7000 you will see that this SRA is supported for code levels 6.2.x.  As you may be aware 6.3 code is available for both products.

I assumed backwards compatibility and tried to setup a configuration with two Storwize V7000s running 6.3 code.  I was successful in configuring the array pair, however the SRA was failing on the “discoverLUNs” command.  The exact error in my SRM log was as follows:

–> C:/Program Files (x86)/VMware/VMware vCenter Site Recovery Manager/storage/sra/IBMSVC[ Wed Jan 11 13:00:47.25 discoverLuns Info ] SRA Version 2.00.11725
–> [ Wed Jan 11 13:00:48.459 discoverLuns Error ] discoverLuns::getSVCLuns(): 203
–>
2012-01-11T13:00:48.478-08:00 [02616 error 'SraCommand' opID=3c4a3aa3] discoverDevices’s stderr:
–> java.lang.NullPointerException
–> at discoverLuns.getSVCLuns(discoverLuns.java:267)
–> at discoverLuns.getArrayTypeAndLuns(discoverLuns.java:191)
–> at discoverLuns.main(discoverLuns.java:154)
–>

After spending more time than I cared to getting up close and personal with the inner workings of SRM and the SRA, I decided to get my environment into a supported configuration.  The benefit of lab work is that it is easy to wipe systems and start over…  My Storwize V7000 systems are now running 6.2.0.4 code and my errors are gone.

Please heed the support matrix because as I found out the hard way, backwards compatibility did not exist in this case.  I hope to get more details on this but apparently there was an update to the CIMOM in 6.3.0 which causes problems for the SRA.

IBM SAN Volume Controller and Storwize V7000 VMware Site Recovery Manager 5.0 support

I have been playing with VMware Site Recovery Manager 5.0 in my lab and I ran across a setup problem that I thought I would share.

More information on VMware SRM can be found here:  http://www.vmware.com/products/site-recovery-manager/.  But a very high level description is that SRM, in addition to storage replication, automates the failover of VMs between datacenters.

SRM is a VMware product however it requires a storage vendor developed plug-in, called a Site Replication Adapter (SRA), to fully function.  The SRA provides an interface to the back-end storage, allowing SRM to control failover.

The SRA is developed by IBM however it is available (and should be downloaded) from the VMware website.   This ensures you are getting the correct version of the SRA for the version of SRM being downloaded.  If you happen to need an older version of the SRA, you can download it from this site ftp://ftp.software.ibm.com/storage/ds_open_api/VMWARE/SVC_SRA/.

So onto the issue I experienced…

While trying to setup the Array Managers in SRM I was receiving this error: “Failed to load SRA from ‘C:/Program Files (x86)/VMware/VMware vCenter Site Recovery Manager/storage/sra/IBMSVC’. SRA command ‘queryInfo’ didn’t return a response.”

This was a frustrating problem for me with a simple solution.  The IBM SRA communicates with the back-end storage by utilizing pearl scripts.  The ‘queryInfo’ script was failing simply because my SRM server did not have Pearl installed.  After installing Active Pearl (http://www.activestate.com/activeperl) my environment is working.

The pre-req of Active Pearl is not listed in the guide that accompanies the IBM SRA.

IBM Tivoli updates for enhanced VMware data protection

One of the other items from yesterday’s huge IBM Tuesday announcement list which gets me excited is the enhancements being made to Tivoli Storage Manager and Tivoli FlashCopy manager which increases the functionality of those products with VMware vSphere.

Tivoli Storage Manager for Virtual Environments was released as version 6.2 and it included support for the vStorage APIs for Data Protection.  6.3 was announced yesterday and the major update to that product is the inclusion of a VMware vCenter plug-in which can be used to manage backup jobs.

A new version of Tivoli FlashCopy Manager (3.1) was also announced yesterday.  This version brings support for protection VMware vSphere volumes.

Both Tivoli FlashCopy Manager and TSM for VE can be controlled via the vCenter plug-in, which shows up as the “Tivoli Data Protection for VMware” plug-in in vCenter.  The products can be used separately, or together in a single backup job.  The home-screen of the plug-in looks like this:

TSM for VE uses vStorage APIs such as Changed-Block Tracking to perform very fast and efficient incremental backups of virtual machines, which from those backups you can restore individual files, .VMDKs, or entire virtual machines.  So where does FlashCopy Manager come into play?

FlashCopy Manager, like TSM for VE, interfaces with vCenter to gain awareness of what datastore holds what VMs and before proceeding with a FlashCopy creation makes sure those VMs are prepared for backup.  It then creates a near instantaneous FlashCopy of the physical volume in which the protected VMs reside.  Incremental FlashCopies are very small and are created very quickly, shortening the backup window and the opportunity for performing numerous backups during the day.  If the backup job is configured to also use TSM for VE, the FlashCopy volume is exposed to the TSM for VE server so the protected virtual machines can also be backed up to disk or tape.

This is a screenshot from my test environment.  I am showing the restore points available for the highlighted virtual machine.  I have 7 restore points which are available from FlashCopy backups, and an additional restore point which is backed up to TSM.

These are great updates from the Tivoli family of products!

Big updates to IBM storage virtualization products

It’s been a little over a year since I joined IBM to use my background in server virtualization to work on an unannounced IBM storage product.  The product was the Storwize V7000, and since that time it has seen regular major updates in functionality and application integration (VMware VAAI for example).  Today IBM has made another major announcement to the Storwize V7000 in the form of new 6.3 software code and also a new product offering.

IBM Storwize V7000 Unified

Links to RFA - http://www-01.ibm.com/common/ssi/cgi-bin/ssialias?infotype=AN&subtype=CA&htmlfid=897/ENUS111-180&appname=USN

http://www-01.ibm.com/common/ssi/cgi-bin/ssialias?infotype=AN&subtype=CA&htmlfid=897/ENUS211-433&appname=USN

Simply put the Storwize V7000 Unified takes IBM System X servers running a common IBM NAS software stack, and integrates them with a Storwize V7000 storage system.  A lot of work has been put into making this a true integrated system.  Management of both block and file storage is executed from a single web interface which has been built in the same vein as the standard Storwize V7000 GUI.  Expect a lot more news on this product to begin trickling out.

Storwize V7000/SAN Volume Controller 6.3 code update

http://www-01.ibm.com/common/ssi/cgi-bin/ssialias?infotype=AN&subtype=CA&htmlfid=897/ENUS211-373&appname=USN

The highlights for the 6.3 code update are:

• Extended distance stretched cluster configurations – up to 300km!  I did a series on SVC stretched cluster and VMware data mobility/disaster recovery.  At the time of that post the maximum supported distance for SVC was 10km.  SVC was able to work at that distance with no code changes, it was just a different deployment decision (nodes in the same rack or racks 10km apart).  Feedback received was that 10km was not enough distance for a lot of customers.  After some code changes, and a lot of testing, SVC is now supported to be stretched between datacenters up to 300km.
• Global mirror replication which uses FlashCopy volumes as remote copy sources - This adds more flexibility in Global Mirror configurations.  Lower bandwidth links can be used if the RPO is more flexible.  Replication (Global and Metro Mirror) between SVC and V7000 is now supported.
• Some virtualized storage systems (by SVC and V7000) will now utilize round-robin load balancing -  This brings performance and availability improvements.
• Real-time performance stats within the GUI have been updated - The real-time performance stats will have the ability to display more detailed information that previously available.
• Native LDAP support
• Enhanced storage interop – XIV Gen3, HP 3PAR, Violin Flash Memory Systems, new Fujistu Eternus, Bull StoreWay and TMS RAMSAN versions.