It was the day of the big secret meeting. All my vice presidents were there except for the unix system administrator. He was a strange man, always wearing that robe, with the long beard and long hair. He considered himself some sort of wizard, and after the conflict last month when we decided to switch all our servers over to SoftLayer, I really didn’t want him involved in the meeting I called today. You see, I called it so I could announce my plan to switch our servers over to Windows. My goal was really to get rid of him; he’s the only one who ever managed to thwart my plans.

Just as I finished that thought, he burst through the door, trailing a long ribbon of old-fashioned printer paper behind him. “How dare you have a systems meeting without me!” he intoned, dropping his stack of papers on the conference table in front of me. A quick glance at the stack tells me that he has printed out operating statistics for every version of Unix and every version of Windows going back to 1985. I didn’t have time for this. Luckily, I always have a back up plan.

Turning away slightly, I quickly activated a program on my Blackberry. You see, yesterday I had written a few custom programs that utilize the SoftLayer API to control a variety of our services. Within moments, a confirmation had appeared on my screen. All of our web traffic had been redirected from our load balanced main servers to our tertiary backup server. In the middle of the work day, that means it was only a matter of minutes before our bandwidth would be exceeded on that server. I allowed the sysadmin to begin his presentation, confident that he would barely get past the 8086 before disaster stuck.

I was right! Within minutes, an email arrived notifying us that we were nearing the bandwidth cap on the hostname last_resort. Panicked, the sysadmin left the meeting. Quickly I summarized my plans to the other VPs, we all voted unanimously for Windows, and I retreated to my office. Shortly after sitting behind my desk, my door burst open. Framed in the light from the hallway, his long shadow washing over me, stood the sysadmin, slowly twirling his staff. “Do you think you can stop me with a simple change to our load balancer? I was configuring load balancers when you were still on dial-up! Now, you will listen, AOL user, and you will see why Unix is your only choice!” Of course, I had a backup plan for just such a situation.

I dove out the window next to my desk, landing nimbly next to my secretary’s bright pink LeBaron. I had made copies of all her keys months ago in order to utilize her unique vehicle for any necessary escapes. I quickly tapped out a text message to Michael in SoftLayer sales. We have a standing agreement that when he receives a message from me containing only the word DAWT, he is to send the best sale at his disposal to my sysadmin. As I drove past the front door of the building I saw him running toward the car. He pulled out his Blackberry in mid-stride and suddenly stopped dead. “Free double RAM AND double hard drives!? IMPOSSIBLE!” he screamed, and I managed to swerve around him and escape. As I drove away, I thought about my secretary. When she first started here, I had convinced her that if her car were ever stolen, the best plan of action would be to change the building security policies so that only my badge could open the doors. I hoped I didn’t need to make use of that plan, but the sysadmin has proved a worthy adversary.

Unbelievable! Even with my masterful backup plan, he was still following me. I saw his battered VW Bus merge into traffic behind me, his vulture-like shadow looming behind the wheel. I sped up until we were both racing down the road, weaving in and out of the other vehicles. Finally we passed a police car, and my next plan sprang into action. I knew that standard procedure was to radio in the vehicles you were pursuing, and I knew my friend Joe was on duty today. Joe knew that if he ever received a radio call about a business man in a pink LeBaron being chased down the highway by a wizard in a VW Bus, he was to call off the police and park a fire truck at a certain intersection. You see, I had hired an actor to pretend to be a corporate Psychiatrist, and learned that the Sysadmin had an irrational fear of fire trucks. Why? Because it always pays to have a backup plan.

I angled toward the intersection and managed to squeeze past the truck just as it pulled up to block the street. I heard the squeal of tires as the sysadmin slammed on his breaks and reversed wildly behind me. Now that I was free, however, I couldn’t return to the office. Luckily I was prepared for just such an eventuality. As I drove to my next location, I quickly used my Blackberry to shut down one of our production web servers. I knew that it would be 20 minutes before the monitoring system would officially declare the server “down,” so I had time.

I made it to my secret office above the video arcade not long after. Before leaving the car I collected the grappling hook and rope from a secret compartment in the door, then went inside. I walked in to the darkened room and immediately noticed something was wrong. My security system wasn’t beeping! The door slammed behind me and the sysadmin boomed out “NO PLAN CAN DEFEAT ME, MORTAL!”

“I’m ALWAYS prepared!” I shot back, and quickly glanced at my watch. It had been 19 minutes and 45 seconds since I shut down my server, the timing was perfect! The sysadmin walked toward me, twirling that staff. Just as he was about to reach me, his blackberry beeped. Pausing to check, he let out a stream of curses and then lunged at me, but I had already rappelled down the side of the building and made my escape.

As soon as I reached the car, my Blackberry alerted me that the server I shut down was back up. How!? The sysadmin must have his own API programs! I cringed as I activated my final backup plan: a program that constantly shut down all our servers. Let’s see him handle that! I took the direct route back to the office, past the still-idling fire truck. I threw Joe a wave, knowing that I’d owe him a big favor for this, and rocketed back to the office. I knew that he would be right behind me, but hopefully with all our servers offline he won’t beat me to my destination. Also, once I made it into the building, the security system wouldn’t allow anyone in behind me. I would be safe!

I raced into the building, looking frantically around for the sysadmin, but he was nowhere to be seen. Finally! I had defeated him! I walked calmly to my office and opened the door, only to see HIM, climbing in through my window. I had forgotten to close it when I escaped this morning! I quickly opened the secret panel in the wall next to the door and put my finger on the red button.

“WAIT!” cried the sysadmin. “We need to put our differences behind us. Our plans have almost destroyed our servers!”

“What do you mean?” I demanded. “They’re fine!”

“No, they’re not,” he said in a sad voice. “You see, I always have a backup plan, and I knew that eventually someone would attempt to power off our machines, so I wrote a script to constantly turn the machines on!”

“B-but…” I stammered, “but I wrote a script to constantly turn them OFF”

“I know” he said, “and the constant power cycling has corrupted our data base. We need to set aside this silly feud and fix it.”

“Don’t worry, dear end user” I proudly proclaimed, “I always have a backup-“

It was right then I realized that in all my planning, I had never actually created any backups.

On May 14th my buddy Shawn wrote “On Site Development.” Aside from the ambiguous title (I originally thought it was an article on web site development, rather than the more appropriate on-site development), there were a number of things that I felt could be expanded upon. I started by simply commenting on his post, but the comment hit half a page and I had to admit to myself that I was, in fact, writing an entire new post.

Updating the computer systems in these restaurants is a question of scale. Sure, it seems cheap to update the software on the 6 computers in a local fast food restaurant. However, a certain “largest fast-food chain in the world” has 31,000+ locations (according to Wikipedia). Now I know how much I would charge to update greasy fast-food computers, and if you multiply that by 31,000, you get a whole lot of dollars. It just doesn’t scale well enough to make it worthwhile. The bottom line is, the companies do cost-benefit analysis on all projects, and the cost of re-doing the messed up orders is apparently less than the cost of patching the software on a quarter million little cash registers and kitchen computers.

It’s the same logic that lead to Coke being sold for 5 cents for more than 60 years, spanning two world wars and the great depression without fluctuating in price. The vast majority of Coca-Cola during that time period was sold from vending machines. These vending machines only accepted nickels, and once a nickel was inserted, a Coke came out. That’s it. Nothing digital, no multi-coin receptacles, just insert nickel…receive Coke. The cost of replacing 100,000 vending machines was far higher than the profits they would get by increasing the price of coke slightly. Only after World War II, when industrialization and the suburb were really taking off, did Coca-Cola start to phase out their existing vending machine line and replace it with machines capable of charging more than 5 cents per bottle.

Of course, we all know how coke machines operate now. Computerized bill changers, many of them hooked up to the internet, allow Coke to charge upwards of $3 for a 20oz beverage on a hot day at a theme park. Coke even attempted (in 2005) to fluctuate the price of Coke based on local weather conditions. People would want a Coke more on a hot summer day, so why not charge more for it? (Because the public backlash was severe to the point where boycotts were suggested the very same day Coke announced their new plan, but that’s another story.)

The fast food problem Shawn mentioned, as well as the vending machine problem, is why so many companies are moving onto the web. Online retail is exploding at a rate that can be described as a “barely controlled Bubble.” To tie back in with my comments on the fast food restaurant, this means that all your customers see the exact same website, written by the exact same piece of code. Want to change the way orders are displayed? Well simply alter the order display page, and every customer in every country from now on will see that new display format.

This doesn’t just apply to retail, however. Many companies are moving towards web-based internal pages. When I got my mortgage, the load officer entered all my information into a web form on their intranet. This is brilliant, because it takes away all the cost of synchronizing the employee computers with the software, it removes the time needed for upgrades, and (most importantly) it means developers don’t have to come into the office at 4am to ensure that upgrades go smoothly before the start of the business day. So any of you business owners out there that have had to deal with the nightmare of upgrading antiquated POS software on dozens, hundreds, or hundreds of thousands of computers, consider making everything a web site. SoftLayer has geographically diverse data centers, so your stores can always log in to a nearby servers to cut down on latency, and we allow for VPN access, distributed databases, and real-time backups, making a web-based solution preferable to even the hard coded local systems that many stores use now.

Recently, as you know, SoftLayer released the new API version 3. We have all been working very hard on it, and we’ve been completely immersed in it for weeks (months, for some of us). This means that, for the developers, we’ve been living and breathing API code for quite some time now. The time came to release the API, and as many of you know, it was a smashing success. However, we were lacking in examples for its use. Sure, we all had examples coming out our ears since the customer portal itself uses the API, but those were written by the same developers that developed the API itself, and therefore were still written from an insider’s perspective.

So a call went out for examples. Many people jumped on the list, offering to write examples in a variety of languages. I thought I would tackle writing an API usage example in Perl. Perl, for those of you unfamiliar, is an infamous programming language. Flexible, confusing, fantastic and horrifying, it is the very embodiment of both “quick and dirty” and “elegance.” It is well loved and well loathed in equal measure by the programming community. Nevertheless, I have some experience with Perl, and I decided to give it a try.

I will attempt to describe my thought process as I developed the small applications (which you should be able to locate shortly in the SLDN documentation wiki) throughout the work day.

9am: “Wow, I really don’t remember as much Perl as I thought. This may be difficult.”
10am: “I need to install SOAP::Lite, that shouldn’t be hard.”
11am: “Where the heck are they hiding SOAP::Lite? There are articles about it everywhere, but I can’t actually find it or get it installed!”
12pm: “Ok, got SOAP::Lite installed, and my first test application works perfectly! Things are going to be ok! Wait…what’s all this about authentication headers?”
1pm: “What have I done to deserve this? Why can’t I pass my user information through to the API?”
2pm: “Aha! Another developer just wandered by and pointed out that I’ve been misspelling ‘authentication’ for 2 hours! Back on track, baby!” (Side note: another “feature” of Perl is how it never complains when you use variables that don’t exist, it just assumes you never meant to type that. Of course, you could tell it to complain, but I forgot about that feature because I haven’t used Perl in 4 years.)
3pm: I finally get example #1 working. It queries the API and shows a list of the hardware on your account.
3:30pm: Example #2 working, this shows the details for a single server, including datacenter and operating system
4pm: Combining examples #1 and #2, the third example shows all hardware on your account, plus the installed OS and datacenter, in a handy grid right on the command line. Success! I put Perl away, hopefully for another 4 years.

The whole experience, though, really gave me an insight into how fantastically awesome the API is. I was looking at it from an outsider’s perspective. I was confused as to how everything worked, I was working with an unfamiliar language, and I was browsing through the API looking for anything that looked “cool and/or useful.” Getting a list of all my account’s hardware to show up in a custom built application that I wrote as if I knew nothing about the API was a great feeling. It showed that not only was the API perfectly suited to the tasks we expected of it, but even a novice developer could, with a little effort, make an API application like mine. Expanding on it to show more and more information, and all the possibilities that it opened up in my mind made me realize how useful this API is that we made. It’s not just something that a small percentage of our customers will be using. It’s something that is truly revolutionary, and that all clients can take advantage of. I’m assuming, of course, that all clients have at least rudimentary skill in at least one programming language, but given the level of success everyone has had with our other offerings, I can assume that assumption is accurate.

If you have been thinking recently “look at all the noise they’ve been making about this ‘API’ nonsense,” I highly recommend dusting off an old programming book and at least looking at it once. Think of all the possibilities, all the custom reports that you can make for yourself, all the data that we have provided right at your fingertips to assemble in any way you wish. We try our best to make the portal useful to every customer, but we know that you can’t please all the people all the time. But with the API, we may do just that. If you’re the kind of customer that is only interested in outbound bandwidth by domain, write an API script that displays just that! If you want to know the current number of connections and CPU temperature of your load balanced servers, get that data and show it! The possibilities are endless, and we’re improving the API all the time.

Here at SoftLayer, all the developers stay on Jabber all day so we’re all accessible. As you know, we’ve recently been doing a major piece of software development: the new API. During development, we all were trying to get things finished as well as continue our day to day operations.

No fewer than 4 times during the last 3 weeks I have had the following conversation with a fellow developer.

Me: Hey, I need you to do something on this API class you wrote.
Other developer: Ok, no problem.
Me: Wait, you didn’t write this, did you?
Other developer: No, I didn’t.

Each time, when faced with a request to understand and modify something that they didn’t write, in addition to their already overwhelming workload, my fellow developers were more than happy to accept the new task.

It just goes to show what kind of environment we all work in here. Everyone is always willing to help, and it’s this attitude that allowed us to develop the API so quickly with such robust features. Each developer was willing to help the others, and that resulted in a tightly integrated product that we’re all very proud of.

The same sort of attitude pervades all of SoftLayer. I have had help on tasks from Networking, Accounting, and Sales since I got here, and each time everyone is more than happy to help out. The end result is, of course, that the customer gets their problems solved faster, and gets higher quality services out it the deal.

But really, fellow developers, if you’re reading this also, it’s acceptable to say “I didn’t write that” when I ask you to change it. I won’t be offended. Half the problem is that we have 5 developers with names starting with J, I just clicked the wrong guy!

Earlier in the week, Shawn Boles wrote a post that included some nostalgic feelings for old-fashioned computers. As I read it, I was thinking of my own experiences with the grandfathers of our desktop computers. I vividly remember getting the floppy disks out of the big dusty box, sliding them into the drive, and listening as the drive began its slow, crunching march towards the data I needed. That’s when it hit me. I can’t hear my computer! Sure, there’s a cooling fan in it, and if I stick my head under my desk I can hear not only the fan, but my coworkers quietly asking each other why my head is under my desk in the middle of the day. Other than that, there’s nothing! No drive heads moving, no crunching, no system beeping, no modem noises, and certainly no death rattles.

That’s right, death rattles. Those of you unfamiliar with the dinosaurs we used to call computers don’t know about the death rattles. The large floppy drives were the worst offenders. Every faulty sector on those disks caused a loud grinding sound that you could FEEL, indicating that the pencil-sized drive head was none too happy with encountering the data equivalent of a marching band at a funeral. Now that we’ve moved away from media that comes in physical contact with the drive head, we’ve moved away from the death rattles. However, hard drives can occasionally produce them, even to this day. When a hard drive goes bad, it emits this high-pitched chirping sound, akin to what you would expect a robotic sparrow to make.

When I was in high school, my best friend’s dad was a programmer. One day we were working on a bad hard drive for a friend and had a brilliant idea. We took a microphone and recorded the sounds the hard drive was making. With some Mp3 editing tools, we added roughly 10 minutes of silence in front of that sound. Then, we made the sound into his dad’s windows startup sound. So upon turning on his computer, he would work for 10 minutes and then hear the DEATH RATTLE. Watching his pasty frame lunge for the power button was hilarious for a good 24 hours after it happened.

Browsing computer message boards (yes, I’m a nerd, that’s why I work here as a developer) I occasionally come across people who are still cursed by dial-up modems, and they invariably ask how they can disable the noises their modem makes when it connects. I want to scream to them “DON’T! You’ll destroy your last audible connection to your machine!” The noises a modem makes are iconic, and a sufficiently trained guru can tell the connection speed simply by listening to the connection noises. From the industrial clanking of the 2400 baud to the sci-fi whooshing of the 56k, each remaining hardware noise is precious.

Precious to humans anyway. When I was younger I saved all my money for weeks to buy a 56k modem. I painstakingly researched (at 24,000 baud) all the different kinds of modems I could purchase, and finally decided on a US Robotics 56k modem. I arrived home from my triumphant shopping trip and promptly tore the cover off the family computer to install my new toy. Once installed, I powered up the computer, installed the drivers, and attempted to connect. Sweet, sweet screeching poured out of the modem, to my delight! However, my dog was also in the room. He had, until this point, tolerated the noises the computer had been making in “his” room. However, how there was a new noise. A new, screechier noise. And it was coming from an OPEN COMPUTER CASE. That’s right, in my excitement, I had left the case cover off just in case I had to tinker some more. My dog got up, walked across the room, and promptly ripped the modem right out of the case with one mighty chomp. He threw it on the ground, chomped it once more and then, satisfied that it would no longer disturb his rest, flopped back down onto his bed in the corner. I still cannot hear a connection sequence without bringing up that memory.

Of course, we don’t have to worry about these things at SoftLayer (dogs OR noises). We developers work on silent machines with no rattles or screeches, while the servers in the data center are attended to by dozens of employees, aware of any slight problem. The noise in the data center is substantial. Each employee wears hearing protection, and even with that the noise of thousands of cooling fans will get to you. Each time the developers have to spend a morning in the data center installing new servers, we spend the rest of the day shouting “WHAT?!” to each other. But computers will continue to get quieter, and our connection to them will continue to be less and less physical. Our customers know this already: none of them have even seen their servers, they rely on our skilled datacenter staff to monitor their hardware for them, day and night. But there are no death rattles, no screeches, just cooling fans.

The last of our noisy connection to our computers is starting to fade. Already we’ve stopped using magnetic media like floppy disks, and the noisy CD and DVD drives of yesteryear are being replaced with silent models. Even the hard drives that caused my friend and I so much joy are being replaced with mechanically simple solid state drives, which run much faster and have no noise at all. Come to think of it, I think I can learn to live in the new silent computing future.

I just got off the phone with my father. Actually, I got off the phone almost 24 hours ago, and I’m just now becoming calm enough to write clearly about it. My father had a problem: he was attempting to use a computer without supervision. Now, my father is a smart man. He has a master’s degree from Harvard, he has “A Brief History of Time” on his bookshelf, and he consistently left clicks when I ask him to right click. The exact nature of the phone conversation is boring an unimportant, except for one thing. My father needed at one point to save a document in MS Word format. Since he has a Mac, he created the document in Pages. He insisted that his efforts had been wasted since (he claimed) Pages was unable to save in MS Word format. I tried to convince him that it could save not only in MS Word format, but roughly 15 others, but he was unrelenting. Finally I got him to check in the Export menu “to humor me,” and lo and behold, that’s where all his Microsoft formats were hiding. Why do people ask geeks for help, then insist that the help provided is incorrect?

I am expecting to spend at least half of my Christmas visit fixing their multiple computers, synchronizing their files, uninstalling the spyware they were tricked into installing, and generally explaining to them that no, the computer cannot just “know what you want.” And at every turn, I expect to hear dissenting opinions and accusations that I am somehow “hurting” or “confusing” the computer by what I’m doing.

My fellow computer geeks all across the country will also be making that periodic tech support pilgrimage. Just talking to the other programmers in the office I’ve discovered quite an arsenal of tools that they will be bringing with them. From special screwdrivers and thumb drives to entire operating systems and (in one case) a whole new computer, we go into the holiday season armed and ready to set ourselves up for future tech support calls.

Some of my more memorable tech support calls have been from relatives, usually helpless in the basic skills necessary to diagnose the problem over the phone. My aunt made one historic call a few years ago. They had just gotten cable internet in their small country town, and after a week or so she was having problems connecting to the internet. So after hearing about the problem I told her I was going to need her to look at the modem. We spent the next few minutes arguing about whether or not she had a modem, and whether or not the problem could have been caused by never having a modem in the first place. After concluding that she did have a modem, and it was still where the technician left it (under the sink, good one technician! bravo sir!), I asked her “what do the lights on the modem look like?” A valid question I thought, and a relatively simple one. I was expecting to hear a short list of the lights’ labels and whether or not the light was lit. What did I get? “Well, they’re about a quarter inch wide and about a sixteenth of an inch…no…make that about three thirty-seconds of an inch tall, they’re spaced about a half an inch apart…why are you laughing!?”

Another fond holiday memory is the argument I got into with my grandmother. She wanted to “get a house page on the wide world web.” I managed to correct her to “world wide web” without offending her, but then the real fun started. She claimed that “the world wide web is better than the internet!” I tried to explain to her that web pages were only a very small subset of the internet, and that the two terms really didn’t describe the same sort of thing. She decided to put it to a vote. Proudly marching into the living room she announced to the 40-so gathered people “raise your hand if you think the internet is better than the world wide web!” They all stared blankly at her for a short time. Sensing victory, she turned to me and screeched “SEE!?” and stormed out.

So this year I will gather my toolkit, my extra networking cables, my CDs with avg antivirus, firefox, spybot, hijackthis, and zone alarm, my copies of windows XP and Mac OSX, two different linux live CDs, my thumb drives, and my overworked laptop, and make the trek down to my parents house. Please, if you are reading this and you didn’t recognize the items in that list, do yourself and the geek in your life a favor: Find out what operating system you run* and go out and buy yourself the “For Dummies” book that corresponds to that operating system. That can be your gift to your geek this year. Show them that you own the book that holds most of your answers, make a promise to them to at least open the book before you pick up the phone, and you will see what it’s like when someone experiences holiday joy.

Plus, you might learn something.

*Look at the top left corner of your screen, if there’s an apple there, proceed to “Apple”. If not, look at the bottom left. If there’s a start menu, proceed to “Windows.” If there’s neither, pick up the phone and call the person who works on your computer and ask them.

Apple: Click the apple, and go down to “About this mac.” There should be an entry on the first screen called “Operating system.” That’s the operating system you have, you’re done.

Windows: Click the start menu button and look at the left side of the start menu. Your operating system may be listed along the left side. If there isn’t, hold down the windows key on your keyboard and press the “Pause” key (you never use it, it’s in the top right). A window should come up that says “system” at the top. Your operating system will be the first item under “system”