And the reasons for buying new IT gear are as follows ... Working with small businesses is a master's degree in the real lifespan of IT infrastructure. Companies with real budgets replace everything regularly; refresh cycles are two or three years apart, rarely five. The same doesn't necessarily apply to SMBs, and there are very real lessons to be learned. Perhaps the most important …
I'd be disappointed as a customer to find out you were spending my hard earned money on something vague.
We're not here to spend their money. For one: if you spend more of their money on kit, that's less they've got to spend on your day rate. Secondly: if they get a second opinion that says a piece of kit is 'old but ok' they may turn round and start questioning why you really want to spend their money.
As an industry we get a bad enough rep with built in obsolescence, or just wanting to buy shiny things for our own benefit. Best to tread carefully.
Partly true, but sometimes the customer just doesn't want to hear the truth and someone will always tell them things are okay.
I started a new job at the end 2010 and they told me they had just upgraded their kit and were state of the art...
After a month I was telling my CEO that the 9 year old PCs with XP (not even SP1) and 256MB RAM were not secure and not fast enough and needed to be replaced - they had Symantec corporate AV installed, but it had never been activated to do a scan; I found out why, I turned it on and it took over 2 days to scan the PCs and they were pretty much useless during that time.
It came out the whole network was inundated with viruses, including Conficker. It took me ages to clean up the network and I kept telling him we needed newer PCs or at least memory upgrades.
I got made redundant and then he looked for a sucker to take over. My replacement told him the same thing, but the board sacked the CEO before he could sack my replacement.
Security was even better, their previous, external IT support had reset all passwords to 12345 and all accounts had their email on Exchange OWA available (although nobody in the company knew it was turned on). That was one of the first things I managed to change.
How do you convince me too?
And what are you considering a small business?
I've worked with a few manufacturing companies I consider small... under 100 employees.
Security issues over the years tend to originate from the inside, like walking off with customer lists or CAD files.
I think everyone knows they aren't as secure as they could be.
Right or wrong, new hardware and software to boost security doesn't evert get too high on the priority list.
Insider threats which you can make relevant to management are your key to improving the situation around hardware/software and training. I've done this several times over the past 20 years, If you say "we've got viruses, we're running old operating systems, we really need to fix that" management see it as they would a leaky roof - get it done by winter, but who cares when the sun is shining.
Make it relevant to them "we've got security issues, could lead to our CAD files being copied or leaked" and suddenly they'll take note and start asking questions.
really? Maybe if you are in an ultra high security industry, otherwise generally speaking if it ain't broke no need to fix it. I actively held onto one version of code on my switches until I could no longer get support on it because it eventually became end of life (having performed no major updates on the switches in the previous 3 years).
My current switch code has a couple of known issues with it that are fixed in newer versions - but for now (and for the past year) I'd rather live with those known issues then think what new issues might be introduced by the new code. The features I use on my switches really haven't changed in 11 years now.
No I don't use Cisco, and won't touch Supermicro.
Same goes for other things too, the ONLY reason I upgraded to vsphere 5.5 from 4.1 was because 4.1 was past end of support(by about 6 months). Fortunately I don't have to open vmware cases very often. I have no intention of upgrading to vSphere 6 before 2017. 5.5 does everything I need just fine. I had some early issues with some VMs hanging and having to be killed on the ESXi host themselves (first time I'd ever seen that in vmware myself), but that problem seems to have resolved itself at some point.
My Splunk version has been end of support for probably 2 years(yet we still pay for support). I was burned BADLY by splunk 3->4 upgrade years ago and now am paranoid about future upgrades, the last major splunk issue took more than 1 YEAR to resolve(not related to 3-> upgrade that was probably 2 years of critical issues), and as things stand now it WORKS good and nobody has complained in at least 2 years(since the last issue was resolved). No outstanding issues, no pressure to upgrade, it's on the roadmap of things to do just keeps getting pushed because there are more important things to worry about.
My production load balancer software went end of support in October, there is a newer version and obviously we are under support - the newer version has a critical bug in layer 7 mysql load balancing that is preventing us from upgrading, been working with support on it for 5 months now they still don't know the cause(prior to that a stupid licensing decision the vendor made in versions between what I have and the absolute latest prevented upgrading, the licensing decision was reversed in the latest code). My team used to hate the Java UI in the current load balancer version that we have, then they saw the new all HTML5 UI and they hate that even more they want the Java stuff back (well technically they haven't lost it yet in our production load balancers just the non production ones).
Oh and my latest shiny DL380Gen9 boxes have a critical issue where they can take upwards of 2 hours to boot(and at least 2-3 attempts to get them to POST entirely), my earliest systems had to have their system boards replaced because they kept crashing on POST. HP eventually figured out the cause but does not have a solution for me (their solution is to disable boot from SAN but I boot from SAN so that is not an option). At least they fixed the firmware so the system boards don't become unusable.
http://h20564.www2.hpe.com/hpsc/doc/public/display?docId=emr_na-c04822613&sp4ts.oid=7500984
fortunately I don't have to reboot often, and these systems I don't plan to reboot for at least 6 months.
I used to run a full layer 3 switch at home for a few years 48-port 10/100, though it was obviously very loud(even after I replaced the fans) and drew a lot of power. I used 2 or 3 VLANs on it, ditched it maybe 8 years ago for small netgear switches(the little metal ones). No VLANs, nothing fancy. I do have a separate network zone for less trusted things which is wired into my Soekris OpenBSD firewall, so those things like my PS4 are connected to that zone(along with wifi), and have no access to my internal network.
The more time goes on the more I just want shit to be stable, there are very few new features that I care about anymore. Maybe I'm just getting old.
Indeed. One just like it is still running 24/7 as my personal ftp server. The original 8GB hard drive failed a few years ago (powered down to physically move the machine, refused to boot back up) and was replaced by a couple of 80GB items in software RAID.
My experience is that ordinary consumer-grade kit can last an awful long time if it is left running continuously.
-A.
"powered down to physically move the machine, refused to boot back up"
Been there. That was a no-fun week waiting on a replacement power supply and raid controller card. Can't remember exactly how I unbuggered our situation so we could keep going. Outlook in offline mode from exchange and imap to email provider I think.
I didn't.
But then, having worked somewhere that had an all-Dell policy, I've refused to touch the things with a bargepole ever since.
Horrible, unreliable, slow old bags of shite, with cheap, plastic casings that seem to be made of eggshells and tissue paper and a support service that knows a million ways of saying "fuck off and die".
I've gone from working in an all-Dell shop, to an all-HP one, and really there's not much difference. They both have crappy base level support, but if you pay for it then the support is pretty damn good (I've had to call on Dell's gold level support once before, and they had an engineer on site with spares of practically every part in just over an hour, back up and running maybe 30 mins after that).
As for the hardware, apart from the motherboards, they're using the same CPUs, they buy their memory from the same manufacturers, their harddrives and RAID cards come from the same manufacturers.
Oh, and they both have really shitty websites that are a pain to use.
For me the choice between Dell and HP comes down to "who's kit do you have already?"
My experience is that ordinary consumer-grade kit can last an awful long time if it is left running continuously.
In my previous job at a University, I once had to nurse along an ancient as hell Gateway 2000 Pentium II based PC because it was the only machine in the shop that still had a 16-bit ISA slot. Said slot was populated with an extremely expensive to replace interface board for a scientific instrument.
We had to keep it running 24/7 because if you ever turned it off, it was far more likely to *stay* off than turn back on. But once you convinced it to boot, it ran just fine (Windows 2000 if you were wondering.) Eventually they got a grant to replace the whole thing and I flung the old system with great glee into the dumpster.
We have a windows 98 machine that runs an ancient denford milling machine. Since it takes DXF files it has never been swapped as that would cost more money for a difference licence. Files are moved on floppy disk from a usb floppy on the network to the PC (obviously stand alone). I have deliberately not taken a backup of the 400Mb drive or even bothered to look at the internals as I want it to die. But it wont die. It does spend most of its time shut down and is powered up only when needed (it is a dell machine, one with a map of Ireland as the Dell logo)
worked for as large energy provider once with some Sun servers 13 years out of support with sun but under contract with some third party outfit to replace failed hardware. Talking to an engineer replacing a CPU I asked what the warranty period was on the replacement parts and the answer was if it boots then that is the end of the warranty period. That combined with the power input/heat output of the older kit and the space it took up proved to be useful levers for getting some newer hardware.
The 3 year refresh cycle in big companies really only applies to stuff that is seen to change over that period. We replace servers that often, because we need to be running/testing on the same new kit our customers are. We replace disks because even if they don't die, they fill.
Network equipment will get replaced if it's facing the external world and can't be upgraded to the latest security patches/rules, but internal gear? That gets replaced about as often as the building plumbing does. If there's a blockage, or maybe if there's major rework to move a server or a lab, but otherwise it'll be there until it rots.
One of our customers finally turned off their DEC Alpha mini (2 racks wide job) last year, which meant we could turn ours off as well. That saves us a huge amount on electricity.
Luckily it was very reliably and didn't even lose a single drive in the last 10 years.
You should have seen the hp laserjet printers at my old place of work 15+ years on some of them but only had to be binned because toner was getting harder and harder to locate as well as the drums being rarer than rocking horse teeth (one of them had done over 1 million prints..and still worked petty damn well) oh and you know it's lasted when the brass cogs inside had worn down to nothing.. Just turn it round and off it goes again.
Our Goods In dept is still docking a laserjet 2 and a jetdirect box to print purchase order information from our ancient (soon to be replaced) system.
I swear that printer will never die.
Another team in our firm still uses an ancient Tally Genicom line printer for handling packing notes for orders they send out. They worked for ages, then one of them died during a site move. The printer even had a sign on it, "Do not power off if you want this to work ever again"....i guess they were right.
6 months ago we jettisoned all of our HP Printers... Sad day really. There were some LJ4000's and LJ2000's that had been on premises since before I started working for the organization. Most were still chugging along nicely. Now we have a badge connected printing system, swipe your badge and print. It has dramatically cut down on paper waste...
bought 4 HP2300 when I started here over 13 years ago, 2 are still going, I needed to bin one recently as someone broke the top cover off cleanly and I couldn't find the bits to glue back on. Our HP 4000 is still going, I think the date on the back is 1999 (it is faded). The whole printer is brittle at the back as it sat in a window for years. The Ethernet connector is a bit iffy but it is a daily workhorse, toner is very cheap and still manufactured (PlanIT toner). I don't even buy service kits, I just rough up the rubber when it smooths out.
On the flip side, our M401DN has had a repair under warranty for a feed issue and our CP5525 is always breaking down (don't mention the bad design of transfer belt).
I love the HP4200 and 4300 series printers - for high volume B&W printing they are extremely hard to beat. My old bank had an agreement with a supplier to swap them out for refurbished models every few years as the repair calls crept up, and they'd come back next time round for another five years of constant abuse.
Driver compatibility with everything under the sun, reliable cheap(ish) toners and 40ppm, what's not to like?
I've been at a 50 billion dollar company for 12 years now and we're still runnig the same core switches in my local data center. And I just got rid of a rack of DC powered Rackable Dual CPU Opteron boxes (they were really nice for their day) with 16gb of RAM each. Talk about sweet!
But the myth that people upgrade everything every 2-3 years is just that. I used to be against leasing because it's a horrible way to waste money, but now I do it with a $1 buyout so that good servers are cheap to keep around (dual CPU, quad core, 144Gb of RAM... what's not to love?) but I do so wish we had done leasing on the core switches so we could have done an upgrade to a 10g core earlier on.
And of course the myth that all new servers come with useable OoB (Out of Band management) like ILOM is a crock of shit. I refuse to buy HPs if I can at all help it these days becasue they disable the ILOM once the BIOS is done unless you pay they're extortionate tax. Screw that.
Lately we've been buying the SuperMicro FatTwins with lots of RAM and it's amazing how cool they run, even in a dense 4U of space. The cabling is a bit annoying... but honestly I can stand that.
I honestly think that the average corporate refresh rate is much closer to 5-8 years, if they can get away with it. Esp once you've paid the tax to get a new ERP or othe business critical system installed. Then people become really really really hesitant to upgrade it. And god forbid you've customized it to do things your way. You're just screwed when it comes time to do the upgrade.
Heck, we're now in the position of having to upgrade a bunch of Oracle 6 (yes, they were writting back around 2000....) apps because we're moving from IE9 to IE11 and these apps just will not work no matter what. Whee!!!
So the basic presmise that only SMBs hold onto stuff until it dies is crap crap crap.
And in alot of cases, there's no need to upgrade aggresively. Most people aren't using all the features anyway... so why upgrade?
John
1) When the vacuum tubes can no longer be replaced.
2) When the spiders living inside the cases are replaced by squirrels
3) When people with dust allergies start wheezing whenever they are within 3 rooms of your server racks.
(More seriously, whenever a security or risk management issue is created by the age of your gear.)
4) When you ask the make of printer they are using and they come back with "Adana".
Which reminds me...
Many years ago I had a phone call from a company that wanted an accounts package. I offered to go down and demonstrate something to the guvnor there. "What kind of pc do you have?" I enquired. "Let me have a look... Ah here we are... It says on it Commodore... Commodore Pet."
Two issues here, and it doesn't have anything to do with "convincing" customers - because you can't.
1. Cost, and, 2. If it ain't broke, don't fix it.
Between those two, as long as it's still working, it won't get replaced. I've seen 10+ year old Pre-XT machines still being used, along with a decade worth of legal documents tied to virtually-proprietary format floppy disks storing their data. You guess which disks are still good - not even the customer knows, but you can be sure they'll be screaming blue bloody murder when you can't get the data off when the machine finally breaks.
That. is. bullshit.
I hate it when mfgrs play the game of "guess what hardware/chipset/driver it REALLY is" by pasting the same goddamned model label on everything that comes out of their factory.
Probably just a personal peeve, but it's cost me so much time and hassle that it makes my ears smoke.
There's been so many goddamn times I've installed drivers on N units (where N > 10) then the N+1 machine just won't fucking install, and WON'T TELL ME WHY and eventually it turns out "oh it's slightly different hardware" after a metric assload of detective work, sometimes including taking the thing apart to look at circuit board silkscreening.
I can't remember the last place that did that. True with current kit is less relevant for BAU stuff but would still like to have systems that are still supported.
AC as currently working at a big UK bank which has multiple applications running on 20 year+ RS6000s. Everytime there's a failure you wonder if it will come back up. There's little chance of replacing the disks as they don't come up on ebay that often. However you know the business will scream if it goes down and stays down and the money (Bns per month) stops flowing.
Well, if they're really 20+ years old, then they must be Micro-channel machines, and must be running AIX 5.1 or older. If this is the case, then they've also been out of software support for something like 8 years.
A bank risks it's banking license by using machines out of software support, although I believe that the wording about applying software patches is "Must have all applicable software patches installed". I've heard some people say that they've installed all patches that are available for the level of AIX they're using, so meet the requirement, although I'm not sure whether the auditors would really agree.
I also think that the kernel devs and major Power Linux distros have dropped support for Micro-channel machines.
Germany was using OS/2 in many banks until a couple of years ago.
I worked for a manufacturing company and they had a 1985 Unix ERP system, which they were replacing with NAV in 2010. It had been out of support for years, but it still worked, so they still used it.
For some, no support = no maintenance fees to be paid to the manufacturer or their agents = cheaper to run than investing in a news system... Until it goes tits up.
"How do you convince small businesses to throw away switches that are capable of meeting the functional needs of the business, but you want gone because of vague concerns about security vulnerabilities that haven't yet actually proven to be a problem?"
I think the article answered its own question - just steal them.
"Steal them"
As a netops guy thats worked his way up to net engineer since the late 90's in the F500 space, I still think disappearing your client's equipment even if old/obsolete without their permission is just theft.
Even without experience in the SMB space, it shouldn't take that much brain power to realize that SMB's don't have the capital to engage in such an upgrade cycle.
I can state that even the F500's I work for all have C3560's still in-place.
As a Network Consultant, you should be able to do standard network discovery across their network, and identify what equipment will not support the architecture they have in place.
Part of this may include re-architecting as necessary, and upgrading equipment to support things like trunking, vlan's, etc.
From there you should be able to pull together a proper business justification for new equipment, services, etc.
We all would like to be able to get a client's out-dated mish-mash of equipment upgraded to something modern and coherent.
But the problem is, while that might be what we want, the client is always going to ask if there is any way to keep limping along as they have been.
If you think about it, we do the same thing when the car is in the shop. We get a list of things that NEED to be replace, and there are others that should, or could be replaced.
The problem in the author's story seems to be coming to inherit a customer who was either sold a bill of goods with D-Link or other consumer equipment, or a customer who pushed their original Network consultant to minimize cost as the primary goal.
To some extent, you DO get what you pay for.
While it might seem iffy, eBay has tons of Cisco gear available for pennies on the dollar. I was going to recommend 3560's for general LAN switches, however just checked and it looks like even 3750's are available in abundance down to $125.
This presupposes the client actually needs more advanced equipment.
"This presupposes the client actually needs more advanced equipment."
Many SMB would like to grow, implement new products/services/foo, or just plain spend less time carving wheels from old process rocks. But, digging their rut from the limitations of their equipment, they forget to look up out of the rut just keeping the business going. Been There Done That Have the T-shirt
You'd be surprised how many SMB would spend for the used 3750 at $125 if that would remove just one process rock. For example, this one "sales" guy could not figure out how to use the DOS based quote system to generate new quotes from previous orders. He would delete old orders and manually re-enter them, instead of updating them, to put in new pricing. I started putting manufacturing process information on the orders and process times for cost analysis. And then bozo would delete them.
For the cost of a network switch, some HDD drive space, and a tape box, I backed up the database to my CAD/CAM box. And when bozo weren't looking, I restored those entries he deleted. The owner was quite pleased at the increase in manufacturing turnaround.
And then the bozo deleted the whole database and I walked out the door -- all the data was still on the CAD/CAM box and tape; but they never found it.
How do you convince small businesses to throw away ANY hardware that are still capable of meeting the functional needs of the business?
You can't. Not until it fails, then it's an emergency with money thrown at it until it's fixed, except for the labor side. Unless, of course, that labor is your favorite consultant friend/relative.
I quit dealing with small businesses years ago, They are just too effing cheap. Big companies are hard enough, but SMBs? Fuck 'em.
"Curious thing is, they were the poster boy on your equipment brochure. The stuff that can't be patched or secured any more. Looks like it caught up to them, and they're in forced liquidation.
"And you still have that equipment installed. About time to consider changing it, don't you think?"
"Steal them"
As a netops guy thats worked his way up to net engineer since the late 90's in the F500 space, I still think disappearing your client's equipment even if old/obsolete without their permission is just theft.
Even without experience in the SMB space, it shouldn't take that much brain power to realize that SMB's don't have the capital to engage in such an upgrade cycle.
I can state that even the F500's I work for all have C3560's still in-place.
As a Network Consultant, you should be able to do standard network discovery across their network, and identify what equipment will not support the architecture they have in place.
Part of this may include re-architecting as necessary, and upgrading equipment to support things like trunking, vlan's, etc.
From there you should be able to pull together a proper business justification for new equipment, services, etc.
We all would like to be able to get a client's out-dated mish-mash of equipment upgraded to something modern and coherent.
But the problem is, while that might be what we want, the client is always going to ask if there is any way to keep limping along as they have been.
If you think about it, we do the same thing when the car is in the shop. We get a list of things that NEED to be replace, and there are others that should, or could be replaced.
The problem in the author's story seems to be coming to inherit a customer who was either sold a bill of goods with D-Link or other consumer equipment, or a customer who pushed their original Network consultant to minimize cost as the primary goal.
To some extent, you DO get what you pay for.
While it might seem iffy, eBay has tons of Cisco gear available for pennies on the dollar. I was going to recommend 3560's for general LAN switches, however just checked and it looks like even 3750's are available in abundance down to $125.
This presupposes the client actually needs more advanced equipment.
HA... HAHAHAHAHAAHHHBWAAAAH!
Right now at my VBE (Very Big Enterprise) I can look at my 11-year-old out-of-warranty core switch, my 9-year-old almost out of warranty backup core switch, and tens of switches from a company that no longer exists... this in the HQ for one of the VBE's multibillion-piastre core markets.
Cycles? Beancounters have invaded the cockpit, and they say "run 'em until they burn up"
That sounds familiar, I work for a relatively large civil service organisation, I think most of our switches and infrastructure are over ten years old and of different vintages and manufacturers after years of chronic underinvestment. Going to cause real problems with the brilliant new unified telephony and comms system being rolled out! a 5 year refresh cycle for any of our kit is a total pipedream!
You can but 2nd hand, you can sell, part exchange or ebay the redundant kit etc etc, but you are often dealing with a boss that really has no credible IT experience or knowledge of what needs to be done or indeed why.
One organisation had a senior manager that could not understand why warehouse stock didn't move when it was simply adjusted on the ERP system, so attempting to explain the ins and outs of security or network switch gear was well nigh impossible.
sometimes the organisation is its own worst enemy, regardless of the skill or cost effectiveness of the IT guy (or team). Treating IT installations like the 60 the old diesel truck thats still running is not a viable option, as every time the word upgrade is mentioned it creates images of polished cromium exhausts, and walnut dashbords rather than the necessary new tyres or support for biofuels or emissions limits etc.to those management types.
You can't just nip out for coal for the traction engine any more, even if it does run fine...
Overall the pace of change is exceeding their life experience, modern IT delivery e.g. cloud, virtualisation is simply sorcery, and advice in this context is like the dragon just producing a business case for more gold rather than the fairy tail raid...
Software gets held onto way past due date too.
I've done three separate migrations this year for SMB customers from SBS 2003 to 2011 because they finally decided it was time to shift. Two were virtual, so relatively straightforward, but the physical one was a good challenge to do in the weekend timeframe.
The Register 
Biting the hand that feeds IT
Copyright. All rights reserved © 1998–2024
