Rogue SF sysadmin may cost city over $1m The disgruntled sysadmin accused of locking San Francisco out of its IT network may cost the city more than $1m in upgrades, consultants and repairs to undo the damage, according to the City's Department of Technology. Terry Childs, a 43-year-old from the Bay Area city of Pittsburg, is accused of creating a super password for …
Consider that one of the jobs of a sysadm is to keep all data safe from misuse by the unauthorised and the incompetant, and that disclosure of a major password is actually a security breach, a felony in its own right as some might have it.
Sounds like this "rogue" was actually doing his job to me! Too many bosses try to treat their IT staff like shit...
Now the shit is on the other boot!
Just goes to show that so much involved in IT is a scam. They've only spent ~$190k to resolve the issue - but now they want another $800k to spend on "upgrades" and "consultant" (often cousin Tommy), and flat panel monitors for the mid-managers. This is horseshit! There's no need for a password fix to take months to fix once you've got the password. The Admin should be publicly flogged then everyone gets on with their business.
First, if you fire me, I no longer am obliged to tell you anything, you should have thought to ask before you decided to fire me.
Second, If I am still working for you, it is my responsibility to keep others from doing something stupid, therefore I cannot give out the password, lest you do something stupid, and I get fired for letting you have the password.
Finally, in situations like this you should always have two admins, in case something "bad" happens to one of them.
So far the majority of offenses that San Francisco DTIS, city officials, the police, and the District Attorney have alleged against Terry Childs has been exaggerated, standard network engineering practice, or just plain wrong. The remainder is grounds for counseling, reprimand, or at the most discharge. Prison time? Please.
For those who are interested I highly recommend infoworld.com's Paul Venezia and his blog series on this event. I think there is a 90% chance that Terry Childs will win a big settlement from this case.
San Francisco officials at all levels are so far demonstrating worse judgement than even UK local councils, couldn't resist!
Paris, because she is WAY smarter than San Francisco politicians and DTIS managers.
K folks, I'm an IT admin, so I understand the impulse to circle the wagons, so to speak. But this guy is clearly in the wrong ( as well as his managers ). When you are working on a critical system, you document the heck out of it, including passwords. You make sure your boss knows where this documentation is too. It's part of being a "Professional".
I'm not clear on the details of this guy's termination, but I could very easily see him getting prison time. Not for not turning over the passwords, but rather blackmail.
Oh, and Mike? While the admin does have the obligation to protect the network, he does not have the right to refuse a request from his boss. If he were so worried about his job, all he needed to do was get the request in writing, along with his objections, and then fulfill the request. This assuming that he was doing his job wrong to begin with and NOT documenting passwords.
Documenting passwords? Great, let's right them all down on post-its for everyone to see! Any IT manager that approves a system to rely on standalone authentication should be shot. Haven't these people heard of LDAP, RADIUS, etc?
Sadly, the real loser here is the locals in SF supporting this idiotic gov't with their taxes, because they'll have to fit the bill in all cases, regardless of who wins. And ultimately, no one will end up being responsible for the whole situation, even if the admin takes the blame legally. There should be some policitians in SF that bear some of this blame too.
Just because management is perceived to be screwballs, the IT admin should not act the same. Terry Childs may truly have major work performance problems and he may personally be a real as**oe with everyone he works with, and if so, his behavior is definitely inexcusible.
I had worked in government positions for some time, and there are employees who think they are indestructable and treat everyone like crap, while management act like sheep because their hands are tied due to employment rules and laws. These employees do not do anything illegal, so they could not get fired, but they treat every contractor like crap, launch "gernades" in almost every meeting to disrupt teamwork, and generally interfere with any manager's ability to manage. But yet they can still keep their job because there are government employees.
I was wondering how it was going to cost so much until I saw:
'plans to set aside $1m to pay for consultants and upgrades to the network.'
So in other words that would be $999,997:50 for the consultants and $2:50 for a new Patch cable....
I think the picture of the faceless, mindless CON-sultant fits best here
being a professional means you get paid, that is all.
It doesn't mean you get to apply your made up ethics to everyone, it is just a matter of getting payment for something.
Oh and of course you have the right to refuse a request, were are not in Nazi Germany quite yet :)
They have to pay you up to the moment of your dismissal, they owe you not the other away around. If he wishes to refuse the information then that is his right, they pay up to the moment he leaves or they dismiss him.
It is no more complicated than that. What is going on here is an abuse of the legal system, he has not done anything morally wrong that anyone can see at the moment.
If the job required him to document passwords, which would have to have been laid down in a contract, then who is to say they were not documented, and has anyone seen the contract stating that?
I thnk you are just making things up, in business you would be a loose cannon, quite dangerous to work with, but I think in your mind you think your approach is correct - interesting.
"He was eventually convinced to cough up the correct code, when San Francisco Mayor Gavin Newsom visited his jail cell."
'Cough up'... a nice way of putting it. I assume a little spot of waterboarding didn't go astray during the Mayor's little chat with this fellow...
This guy is in NO way criminally liable. Have you done ANY reasearch into this case. The network admin tried THREE TIMES to have HIS superiors come up with a disaster recovery plan. He even wrote one. He WANTED to document everything. Management shot it down because they knew their asses would be in a sling if they had a plan on paper and something went wrong from something they ordered the admin to do.
You do realize they promoted a help desk type person to Security Manager. The first thing this manager said is give me the keys to the kingdom. This Admin was responsible for the Police, Fire, EMS and government networks. How responsible do you think it would be to hand over the keys to such a raw cadet? What do you think this manager would do with the passwords?
When the mayor came and spoke with him, the passwords were freely given.
This is a clear cut case of a good IT worker, probably a little eccentric, who was severly overworked. When the new "Security Manager" showed up and could not get the keys to the kingdom she trumped up a poor performance report.
Anyone who does not see this does NOT work as Systems or Network Administrator. We are treated like the power company. We are only called when something is not working. We are never called and thanked for keeping the systems working trouble free.
"he does not have the right to refuse a request from his boss"
Bollocks! His obligation is to the quality of service provided to the customer, not to the twat in a suit ranking above him! Once you get above the basic "please perform this narrow ranging task" mode of work, and into the "please provide infrastructure solutions" mode, you get into politics, where spin and blame-sharing are rife and the customers interpretation depends on who tells him what and how they are pandered to, NOT necessarily the actual facts.
The admin here is in the difficult position of knowing what is wrong, but not being in the "circle" of those who distrbute information about what was wrong (and who can therefore influence the facts by ommission or outright falsehood). In this case, he has chosen to escalate the issue to the point and place (court of law) where he can reasonably be expected to put forward his side of the story without "taint" from the PHB. A risky proposition, if he is wrong or presents his arguments badly, he can get sent down, but he also has the chance to have his decisions vindicated.
True Story: I was against a junior member of staff implementing a large print solution but was ordered to allow him/her/it to do it anyway, upon veiled threat of my position as Team Leader. It failed, caused outage and I spent ages fixing it.
About 2 months later, an off-hand comment from a user in a meeting about me "not dropping the ball like the last time we changed the print system" lead to me finding out that the boss told the users "I organised the implementation of that change", neglecting to mention he over-rode my professional and technical opinion.
If I ever meet that fat b'stard in a dark alley,....
You guys acting like he was the champion of the system.
Making out he's the hero protecting the computers from the Idiots.
He's an Admin! He greases the wheels and makes sure the system keeps working. Just like the cleaners, and the janitors (except the plumbing is a bit more complicated). These are the systems that enable the real workers to actually do the work.
I'm a developer so I understand the need to overinflate our own importance but in an environment such as the case above the networks are not the reason for the organisations existance. Just a utility.
"While the admin does have the obligation to protect the network, he does not have the right to refuse a request from his boss"
Um, would this be a request from a boss after he was fired? If so, then he's no longer got a boss to refuse requests from.
If you are going to fire someone you get all the information you need to know to continue performing that person's job *before* you sack them, that's rule of letting staff go number 1. Not complain that the worker didn't document things for his replacement after the event. The workers boss is the person responsible for the lack of documentation for not making sure it got done, not the fired worker.
Far too many posts in this comments section are heavily biased towards IT admins who believe they hold their companies togeather... clever you!
IT Admins are payed to keep the network secure and working for the best interests of their employer. Witholding passwords to a system because you have been fired is the IT equivelant of throwing all your toys out of the pram.
Seriously, what happened to intelligent deduction of facts round here!
Good analogy with the Site Staff.
Would you let your PHB fix a leaking gas pipe? Service the heating system? Would you even trust him with your (rather expensive) Mikita 18v drill?
There is a good reason these people tell you to Arkell Vs. Pressdram; You don't know what you're doing, and you could very easily break something vital.
This guy did everything right except not hand over the password to the Police. In the UK, that's an instant 2 years prison. I'm not sure how it works over there. That's the crux of the "blackmail" case.
And I have lost count of the number of times a twerp Developer has come up with a messiah complex asking for complete control / casting vote on all infrastructure changes, rather than expend a little effort to change their program. SAP Basis admins are especiallty good at this. (and yes, I cut code for 8 years before I moved to being a sysadmin).
The battle for best fit of an IT solution goes ever on.
Doesn't mean writing on the back of your notebook or on a post-it under the fecking keyboard, you gonks! You can really tell the cowboys around here, can't you?!
Document all your root and admin passwords, seal them in envelopes, give them to the company security officer and then they are stored in an offsite firesafe. At my site you need two upper managers and security officers approval for a root password, then sign the password approval register which the CTO and his PA hold, it is countersigned by one other authorised person, not the security officer. All the admins have their own admin accounts and they are all audited, the audit records stored in a software vaults which only a handful of people have access to.
That's paranoia at work, but at least I know that I cannot be blamed when some numb-nuts decides to go the IT equivalent of "postal" and hold an entire infrastructure to ransom!
This post has been deleted by its author
I admit, I wasn't expecting such a response. But let's take it point by point, shall we?
1) He should have kept information from his boss
a) No, he shouldn't. He works for the organization, and the organization put someone else in charge. Them are the breaks. I will grant you that his bosses are almost as liable for not making him document things ahead of time. Further, if it really did shake out the way it's claimed ( he was fired and then asked for the passwords ), then there's nothing criminal about that.
2) You shouldn't document passwords
a) Maybe in a small mom'n'pop shop. In the real world, real admins aim to protect the network. That includes from you dropping dead, or more likely, moving on to greener pastures and forgetting to tell someone something before you go.
3) Professionals are anyone who gets paid to do a job
a) A professional is anyone who takes pride in their work and does the job right. I don't grant that title to a vast majority of IT workers out there; they are unreliable ( the afore mentioned ones who don't document passwords ) and untrained. Essentially, kids playing with their toys.
As an IT professional, I know the network doesn't belong to me ( as child's apparently didn't ). I know my purpose there is to increase productivity through the use of IT services and equipment. To that end, I do everything in my power to safeguard that primary purpose.
I do not throw a fit like a child because someone I didn't like got promoted above me. Hell, I don't even care if the only use they have for a keyboard is to drool on it. If they ask me for a piece of information that I failed to document ( which is my mistakes anyway ), then I give it to them. To protect myself, I will get everything in writing, so when the idiot deletes everything on the SAN I have documentation, but that's it.
But let me reiterate; I agree with one premise. If he was let go before the requests were made, it's not criminal for him not to give it up. As an employer, I wouldn't touch him, both because he is being stubborn and because he didn't document, but it's not criminal.
Were I the City of San Fran, I'd fire his handlers too for being so incompetent as to let someone like Childs bring the city to it's knees.
This post has been deleted by its author
A few things to say.
I have gone in and sorted out networks on contract before where the previous admin had "thrown a wobbly". Not to this scale, but with similar stunning moves on the part of the admin.
No way should he have locked out the network and made himself the single point of failure. That smacks of ego and not admin or engineer thinking. For years we have had it drilled into our head about redundancy and load balancing being good and then he goes right off and throws his toys out of the pram and makes himself the weakest link in the whole network (but with the most leverage). Earlier posters had it right, either have a backup admin or a key of some sort stored on a flash drive in a safe off-site. Even better, have both.
Yep, totally agree, get it on record about whose decision it was and who overrode who, by mail, meetings minutes or group discussion.
Except when the slimy sod in charge knows that and is always "taking it offline" (No witnesses) "too busy right now to reply to your email" (Avoidance of evidence of confirmation), follows up later with "I understand you are being inflexible in situations that requires dynamic intervention" (distraction from root cause). Followed even later by "I would hate issues like this to become your legacy here" (Veiled threat to give a bad reference after I left).
Office politics 102 - never announce ahead of time your intention to leave
So, no record on email, no record on paper, and if this is the way they treat the team leader, there is no way my staff can expect fair treatment if they side with me on this issue. And I never knew the Users knew the details until well afterwards, at which point any response I make sounds defensive or self-serving.
Office Politics 103 - get your retaliation in first, either by innuendo before the event (I wouldn't be surprised if....) or afterwards (pity about X doing....)
BTW, the junior involved socialised frequently with the Director of the Company and was hired at his behest.
Office Politics 104 - never go up against the "annointed one" of management, you will always lose.
Will write a whitepaper about all this one day.
Enter the password into the court documents -- Public info. Every joe, tom, dick and sally would have it. You wanted the password, hell, yell it out during a hearing.
Run, run like the wind!!! Change all the passwords on all of the equipment!!!!
Bringing the courts into the mess to get a password was S T U P I D. I guess the need the 'consultants' to hook up a blue Cisco terminal cable and do the old password reset feature. It also sounds like Cisco or whom ever sold them 'upgrades' so 'this didn't happen again.' Time will tell. SF may be converting to Dark Fiber.
The Register 
Biting the hand that feeds IT
Copyright. All rights reserved © 1998–2024
