Conversation with John Engates, Rackspace's CTO

TalkBack 1 of 2:

Next »

• Thread View
• Flat View

Bad Call

John,

This is what I sent Lanham with no replay,

So munch has happened to RS and you have no details.

Please call my cell so I can give you the whole story.

Thanks,
Kelly Galbraith
Data Center Engineer
626-676-1287 Mobile


Lanham,

This is truly a bad situation, I have been trying to
contact you by phone with no luck. I need to send you
this email so you can make your judgment call knowing
all the details.

I feel that the last two outages never needed to
happen do to the fact that I have been trying to get a
SWITCHGEAR maintenance contract signed with Shermco
for the past year!.

I have worked at Countrywide & EarthLink main data
centers in California for the past nine years prior
to coming to Rackspace and know how important it is to
have such contracts. This contract gives all data
centers around the world a piece of mind knowing that
their SWITCHGEAR is working (calibrated Section 7.10.)
correctly on a year to year basses with a
certification sticker on every breaker. Do you know
the last time this was calibrated? I do, it?s on all
of your SWITCHREAR dating for renewal in 2002 telling
me that we have gone SEVEN YEARS!! With no testing. I
as well as Shermco (Mike Moor our rep) have made
countless attempts to get this contract signed with no
support from Neal Parker or Bob Mielke telling me it?s
too disruptive and costly for us right now, the PDU?s
failed because the main breakers feeding them were at
half of their available set point to handle the load.
When Neal had J&N install the PDU?s the breakers could
not handle the additional load which was brought to
his attrition prior to the install by Mike Moor
(Shermco). Please see attachment above.

As for the chillers, again I have made many attempts
to run the test needed to find out why the chillers
will not run on generators. I have had two set up
maintenance windows shot down at the last minute by
Neal Parker and Bob Mielke . If you need to verify any
of this info please do so with my vendors Shermco and
or my engineering crew. We have many issues at DFW and
I would like to be a part of resolving them I just
need to be herd.

Please call me for more details,

Thank You,

Kelly Galbraith
DFW Data Center Engineer
Rackspace Managed Hosting
801 Industrial Blvd. Ste 200
Grapevine, Texas 76051
(888) 850-0211 ext. 6154 Office
(817) 919-4910 Mobile
Kelly.Galbraith@Rackspace.com



From: Lanham Napier [mailto:lnapier@rackspace.com]
Sent: Wednesday, December 05, 2007 10:42 PM
To: allstaff@lists.rackspace.com
Subject: [ALL STAFF] Tough Times Shape Fanatics

Hi Rackers,

We are facing some big challenges lately. Over the
past couple of weeks our critical infrastructure in
the DFW data center has let us and our customers down.
This is frustrating stuff. It drives us all nuts and
creates a ton of pressure on Fanatical Support. The
bad news is we are now in a position where we have to
execute perfectly in order to protect our customers
and business. We have worked together for years to
build a reputation as an incredible service company.
Unfortunately the events of the past couple of weeks
have put this reputation in jeopardy. We have to
execute at a higher level than we ever have before.

For three years the DFW facility has performed
exceptionally well for us. In the last couple of
weeks we have had difficulty starting our chillers
from generator power. It worked for three years, but
the last two weeks we have struggled to get the
chillers going from generator power. These struggles
have led to the heat induced outages we have
experienced. Obviously something changed on us and we
have to figure out what it is.

Our engineering teams are working hard to get DFW back
in fighting shape. Here is what we?re doing:

1. We are making changes to the organizational
leadership at the facility. We are literally doubling
the engineering capability at the facility.
2. We have installed a new chiller plant. This plant
runs independently from our existing chiller plant and
provides us with an additional layer of redundancy at
DFW.
3. With this plant in place we will be able to do some
rigorous testing to diagnose the problem to the
systems. We are going to go through everything in
great detail to be sure we turn over every rock. We
have to do this perfectly and get it right
4. Today we had Intel Data Center Services on site.
Tomorrow Goetting & Associates will be on site. We
are going to tap these and other experts to help us
out with the project. At this point everything we do
needs to be triple checked. We cannot afford a
miscalculation on any of our operations, so we are
going to have to go through every detail multiple
times before we do anything.
5. The experts are going to lead us through an audit
of the facility. We are going to review all of the
systems at the facility to get everything right. We
will come out of this with a hardened asset to support
our growth for a long time and we will get the DC
certified so we can share the results with customers.
6. We are also going to spend lots of time getting our
process and procedures properly documented and in
place at all of our DC?s. A world class experience is
created through the addition of a world class facility
and world class Rackers. We are working on the
facility and we?ll use this opportunity to provide
additional training and capability to our teams in all
of our DC?s.

Rackers we are in a critical phase of our company?s
development. Our competitors are launching missiles
at us every moment. They are trying to capitalize on
our recent operational setbacks. We will hold our
ground by taking care of our customers and supporting
our teams through the upcoming tests and changes. I
have spent lots of time visiting with our customers
over the past couple of weeks. The vast majority of
them want to stay with us, but we have to earn it. We
have to execute perfectly. Please be mindful of this
in everything you do. Let?s focus on quality. Let?s
take the extra step to get everything perfect ? in our
tickets, our conversations and our presentations.
Let?s stand up and show how that when the going gets
tough we get Fanatical. We can do it Rackers. Let me
know how I can support you. I feel good about our
plans for DFW. We are going to get it right.

LN

Posted by: Setup   Posted on: 01/01/08 You are currently: a Guest | Members login | Terms of Use

Alert moderator to an offensive message

Subscribe to this discussion via Email or RSS

SponsoredWhite Papers, Webcasts, and Downloads

• The Impact of Virtualization Software on Operating Environments VMware Today's use of virtualization technology allows IT professionals to ... Download Now
• Three Steps You Need to Know to Stop Data Loss Varonis Sensitive data exposed to misuse or loss... it is the stuff of nightmares ... Download Now
• Five Steps to Determine When to Virtualize YourServers VMware Server virtualization isn't just for big companies. Entry-level ... Download Now