DavsDisorder

I refer here to a recent article penned by Tony Pearson of IBM discussing a recent catastrophic failure of an EMC Symmetrix within the State of Virginia's IT environment. Aside from some cheap inter-vendor point scoring, Tony mentions as one of the 'lessons' from this event; "Lesson 4: This can serve as a wake-up call to consider Cloud Computing as an alternative option". There is a faint tinge of irony here in that this post of Tony's was written in Australia in early September. At the end of that month the IT community (and the broader travelling public) witnessed how a 'cloud' provider can be just as exposed to downtime as your now unfashionable internal IT team. Virgin Blue's ticketing and reservation systems were brought offline due to an as yet unidentified systems failure within the storage data path. Virgin Blue do not own/operate their own ticketing and reservation system, but source such a service from Navitaire (disclosure: Navitaire are an old client of my firm). It took Virgin Blue (and I assume Navitaire) almost 7 days to return this service to normal operation. I wont call these observations 'lessons', rather just 'comments;

1. I agree with Tony (and probably every other seasoned data storage professional)...storage systems fail, that's why we have backup systems. These systems in turn are definitely only as good as their last successful test...if regularly testing is too much of a burden then you ought to at the very least audit the environment according to some baseline.
2. Whilst the person in question at the State of Virginia may be a little ashen faced currently, I can assure you that the "service delivery manager" (as they were called in the hey day of outsourcing), at Virgin Blue for the reservation and ticketing system will be feeling the same churning in the lower part of the stomach...his contact at Navitaire probably likewise. Just because a cloud provider is 'big' or ' branded' or, (inserted alarm bells), 'multi-tenanted', does not mean for one second that they can do better/cheaper job of helping you meet your SLAs for service uptime and/or data recovery.

I advise strong governance of how storage systems are used in medium - large organisations, as well as a strict focus on 'recoverability' in governance of backup systems.

In the former instance remember that 'speeds and feeds' as Tony puts it indicate in my experience the 'bleeding edge' of what a product can reliably do...divide by 2 and set that as your peak load. The more complex the data storage layout on a disk array (fragmented RAID  groups, meta-LUNs, concatenated LUNs, etc, etc), the longer your restore/rebuild will be. Remember that in the never ending race toward better storage performance, there is a necessary compromise around recoverability.

In the latter instance, just 'think' in terms of recovery, not 'did it get backed up'...build backup systems that focus on the process of data restoration (we talk only of data availability here, compute availability is a whole different story). It is far better to have a backup run for 8-10 hours, complete, validate and be easily restorable than a backup that runs in half that time but require multi-step, error prone recovery procedures.

This article probably somes up my disposition towards the issues raised by this election process (note I say issues raised by the process, not issues rasied in the election contention itself), better than anything I have read recently:

Worth subscribing to;

http://www.crikey.com.au/2010/08/02/this-is-all-your-fault/

Whilst doing some research around options for moving our internal mail and file serving/collaboration requirements into a 'cloud' provider, I came across the details of the Telstra T-Suite offering, part of which is the Microsoft BPOS suite. As we are thinking of consolidating telecoms with Telstra also, I thought I'd test how their "implementation" (assuming they are actually hosting it in Australia somewhere) of the suite responded (browser refreshes etc). As a Microsoft partner we can use their BPOS but I am guessing it is hosted in an US or other remote DC so I thought I would try Telstra; unfortunately, whilst trying to secure a trial I got the follow screen:

 
Not an encouraging way to greet prospective SaaS buyers!!!

Advice, especially field tested advice is not risky Happy! But I guess there's sometimes no option but to adopt a bureaucratic approach when positioned as such.

http://www.arnnet.com.au/article/344444/smb_it_supplier_advocate_named/?eid=-4152

"Easter said his first job would be to consult industry bodies and large tech companies to get a sense of the industry. SMB IT companies would need to understand how risky they appeared to Government CIOs"

I would love to get Data Engines to do more government business; it is often challenging work and it feels a little bit like we're using IT as our way to 'give back' in public service!. But...but, but but... $40mil coverage for professional services indemnity is also a regular requirement for tender respondees and that cost is a little too much alongside side a 50 page tender response document for a specification that is often written with a particular solution and/or vendor already in mind.

//TD.

Well, nice to see support for Australian technology firms in this initiative....what a joke! The government should have in place a mandate that Australian based firms (SME or Large) are preferred for some sub-section of the setup and maintenance of what will be a large company funded by government - NBN Co. The whole NBN concept is one of supposed 'nation building', yet when opportunities to stimulate the local technology sector arise like this, we allow NBN Co to line the coffers of 3 well fed international firms!! And....23 million !!! to set up IT infrastructure and services for a company that doesn't have a 1000 employees yet....geez. No, I am not a disgruntled tenderer either...just a business owner in the tech sector a little dismayed at the government's contradictory actions. I wonder whether Malcolm Turnbull would agree in principle?