?

Log in

No account? Create an account

Previous Entry | Next Entry

Boom Today...

...and not even the ICG blow-up. That was minor compared to the rest of my day.

It's been tech hell week. No laptop Sunday thru Thursday afternoon. Antivirus upgrades on my servers on Thursday evening that took 3 hours instead of an hour and a half. Final tweaks getting taken care of from home at midnight while I watched the rerun of "Rose" (and caught the breast implant comment).

And then the phone rang at 6:50 this morning. Both my mail servers were down, and had been down for a while. I recycled them, but got a call a bit later saying they had crashed two more times.

The morning after I upgraded the antivirus software.

Can you guess that I spent an hour or two trying to diagnose the problem in the new antivirus software?

Wasn't the problem. Just coincidence.

The servers were getting slammed by a dictionary attack phishing scheme. Caught that around 10:00am and put a block on the address.

That got the machines up and stable for my 11:00 meeting that I made it into the office just in time for.

Of course, it still took until 12:30 to work through the whole system to get mail routing and delivery working completely again. Then I could get something to eat. About 6 1/2 hours after I got up. I never go 6 1/2 hours after waking up without eating.

After lunch, it was time to drop stray messages back into the queue. 120,000 phish-mails deleted from the router files that I could recover. Maybe 1000 good messages in the queue. Two more queues I couldn't recover.

Joy. Rapture.

Fortunately I've got management (and most of my customers) who understand that if the downtime has been more than 15 minutes they should stop bothering me.

It's still less time than major outages outside our department.

Comments

( 3 comments — Leave a comment )
kevin_standlee
Mar. 24th, 2006 08:22 pm (UTC)
Parliamentary Nitpick
Not meaning to get too involved in the ICG politics, but where y'all say the President shall be a de facto member of the committee, I think you really mean ex officio. The latter means "by virtue of holding the officw," while the former means "in fact." It is common for either an organization's President or other senior officer -- in SFSFC, it's the Vice President -- to be ex officio a member of all committees (except usually a Nominating Committee if one exists).
bovil
Mar. 24th, 2006 08:28 pm (UTC)
Re: Parliamentary Nitpick
I am sure you are correct. I'll forward this to the acting President and the parliamentarian...
aramintamd
Mar. 25th, 2006 06:14 am (UTC)
How...special...

I was wondering why you were gone most of the day. Now I know.

I continue to be thankful that I'm ignoring any job listing server management as a top priority. Eww...
( 3 comments — Leave a comment )