SoftLayer Technologies (tm)
Divider Forums Divider Portal Divider SPacer Divider About Divider KnowledgeLayer Divider InnerLayer
 

Posts Tagged ‘power’

Nnet strikes back

By Shawn Boles on Monday, February 4th, 2008

I’m not going to tell you my name for two reasons: first, I don’t want a million tickets assigned to me asking if I’m crazy. Second, if I am crazy, I don’t want anyone knowing it’s me.

I’m not a writer myself, so I asked Shawn to write this up for me. He’s a programmer, and more important a Trekkie, so he’s likely to understand (and more important, believe) this story. Besides, he’s written a few humorous, slightly preposterous posts for this blog, and that’s very, very important.

Unlucky as I am, I was the first person to notice something strange going on. I’m a datacenter tech for the company (but I’m not going to tell you WHICH datacenter), and my job… well, I’m the power guy. I make rounds in the datacenter, checking breakers and power panels, keep an eye on voltages in the portal, that kind of thing. No power issues at the datacenter? That’s because of me. So, I’m perusing the tickets and keeping an eye on things, like I should.

As I was answering a particularly interesting ticket, I received an IM from a datacenter engineer I hadn’t met yet. That’s not surprising; we’re growing like crazy here, and I don’t always get the “Welcome a new employee” email before I find myself working with the guy or gal. I finished my ticket and opened up the IM window. It was from “Nnet,” and the contents caused me to leap out of my seat:

“The power strips on the new racks (205, 206, 207) are drawing too much current; it will pop the breakers in 52 minutes, 12 seconds.”

I had just CHECKED those racks. I walked down to the server room, muttering about some whippersnapper of a new engineer playing a trick on me. I was going on vacation in a week, and I did NOT want any power issues; I was training another engineer to take the console while I was gone, and if anything happened during testing I would surely be called in. Anyway, I walked into the server room and checked the gauges on the power panel.

And they were drawing almost a full five amps too much. If we had turned on the third rack, the whole aisle would have gone down. That wouldn’t have been too bad; no servers were hooked up. This is exactly why we test the power before we put servers in.

I and the rack crew worked for about an hour rewiring the racks, starting from the third rack. Sure enough, about 52 minutes later, rack 205 shut down. Mentally thanking “Nnet” for finding this (and more importantly, not tinkering with it before letting me know!), we got the racks wired more efficiently (they’re supposed to be on separate breakers, but the electrician labeled the wires wrong), reset the breakers, and had absolutely no issues for the rest of the day.

I got back and thanked “Nnet” for finding that issue. The next day, I got to thinking about how “Nnet” had saved my vacation (I would have spent all week tracing wires to figure out what had happened), and I wanted to invite him or her to lunch. So I IMmed “Nnet” with an invitation. An hour went by with no response, but it’s not too strange to have a datacenter tech away from their desk for a couple hours. So I sent an email to Nnet.

The email bounced back.

Maybe HR hadn’t set up the email yet? So I called them up to see what was up with Nnet’s email address.

That’s when HR told me that nobody with the last name “Net” had been hired (I thought “Net” was a strange name for a tech, but it’s not the strangest last name I’ve ever heard). I called the networking department to ask how I could receive a company IM from somebody who doesn’t work here? They researched it and couldn’t find any incoming links through our firewalls or any of the internal logs. Stranger yet, the Jabber server indeed DOES have an account for “Nnet”, but the engineer who runs the server swears that he never set that up.

We were discussing this back and forth when one of the developers walked by, overhearing our conversation. He laughed, and when we asked why, he told us that he was reading a book about the human brain, and that the brain is made up of million of millions of neurons all interconnected with each other; that these interconnected neurons work together to create intelligence.

Could that be true? Absolutely not. It’s preposterous. Sure, we’ve got tens of thousands of computers around here, dual cores and quad cores running various operating systems and applications, all connected by an incredibly fast private network…

…could it be?

The engineers are all completely sure that one of the datacenter techs must be playing a joke, and they’re currently tracking it down. But I’m not too convinced. “Nnet” knew which power strips were having trouble in a room keycarded to open only for me and a hand full of other techs. And they all swear they didn’t send it.

That’s when I talked to Shawn. He told me that there’s a lot of technically minded people out there who read fantastic science fiction stories and come up with solutions… even knowing that the tech is impossible, they can find a way to solve the problem. So we hatched up this idea to write out a fantastic blog post, an interesting narrative of my predicament.
Then we’d post it to the blog and watch for any discussion on the customer forums. Our customers are really smart, and they like solving problems. Maybe somebody out there has an idea of how we can figure out what’s going on around here.

So here’s the story. A completely fantastic modern day science fiction story about a sentient datacenter.

Preposterous!

…any ideas?

I outsourced it

By Steve Kinman on Friday, February 1st, 2008

Have you ever wanted to tell your CIO that? His response might be, “you outsourced what?? You respond, “it!” With a perplexed look he asks again, “You outsourced what, it?” Again you respond with, “All of it.” His reaction at that point could go either way. In most CIO type heads today, they can’t grasp the savings associated with outsourcing and even the ones that DO understand would then have to go to the CEOs office and inform him or her that all of the company’s valuable data will now be housed in a safe and secure facility off-site on dedicated servers… or “Hosted IT” even. Stop reading and go tell your CEO that right now. I’ll wait…go ahead.

Ok, I see that you are back, are you still employed? We are hiring if you need a new job resumes@softlayer.com

Ok, really, how do you think that conversation would go? I have had that same conversation with ex-bosses and owners of small and medium sized businesses in the past and most of the time they don’t go very well. Granted they were a few years ago so hopefully times are changing.

I have been told a few times, “no, I don’t want to pay $300 per month for a server we don’t own and put my data on it! That is ridiculous, just go buy me a new $3,500 server and we will put it in our local Datacenter, Server Room, Broom Closet, Bathroom, Office Managers office…” well you get my drift.
“But Sir, with this outsourced server we could easily have off-site backups, more processing power, some cool redundancy and it will not annoy everyone in the office with the loud fans and heat generation. And when we have a power outage in the office and everyone goes home for the day, they will be able to work from home because the server will still be online. Oh yeah, and our company website and email will still be functioning as well.”
“Are you insane? Those challenges are so easy to overcome. We will simply add a small air conditioner to the broom closet and buy a big UPS system that will keep the server alive in the event of another power outage, and we can hire a service to come by every morning to pick up tapes and deliver them to an offsite bunker. Instead of a single connection to the internet we can buy two and have redundant connections also.”
“Sir, I am no accountant, but by the time you pay someone to keep up with the depreciation of a new server, buy and install a small A/C unit and UPS unit, pay for a 2nd internet connection that will sit idle and pay a service to DRIVE here daily I really think the outsourced server would be cheaper. Not to mention in the event of data loss we could get the data restored to the server much quicker than waiting on a service to physically bring it to us.” An interesting note here is, I don’t care what kind of offsite data bunker you have, the Monster in Cloverfield IS going to destroy it so think multiple copies of data in multiple cities!
“Well I have made my decision; we will not be outsourcing my very valuable data – Hackers might get it, it is more secure here, so leave my office. Before you go could you please try to get my printer working again, and I am getting this annoying pop-up about spyware and it seems that my ITunes files have lost their license and I used to have a folder called Docs on my desktop with everyone’s salary in it that is missing and my PDA will not sync…(zzzzzzz) — OUTSOURCE IT!

Home | Dedicated | Cloud | Solutions | Services | Facilities | Reseller | Resources | News | About | Specials | Contact | Legal
©2008 SoftLayer Technologies, Inc.
Close
E-mail It
Socialized through Gregarious 42