Ah, the old days
...where people shared passwords and everyone trusted each other!
BTW: I do like the line about Oracle...
After a hopefully relaxing weekend, we at El Reg want to kick off your week the right way – with a full-scale facepalm. And so we bring you this week's instalment of Who, Me?, where readers share their cock-ups, large and small. This week, meet "Wallace", who wrote in to tell us about a time he forgot himself – almost …
Some lab instruments rely on a local Oracle (or SQL Server) instance. When the machine is provided by the vendor, often you only realise this because the machine is so slow... You'd be surprised how unwilling some of the instrument vendors are to moving this DB onto a proper server in the datacentre!
"You'd be surprised how unwilling some of the instrument vendors are to moving this DB onto a proper server in the datacentre!"
And I can see why.
The instrument sits on the bench here. As the instrument user I control it. What datacentre? Where? What extra cabling is needed to connect it? Who runs the data centre? Who has access?
Unless there's a specific need for an instrument to be connected to a network it should be capable of being used locally; the alternative is to introduce it into the IoT where, as we all know, the S stands for Security.
"You'd be surprised how unwilling some of the instrument vendors are to moving this DB onto a proper server in the datacentre!"
This may be a licensing thing.
In a previous life I had to learn about Oracle licenses. If I recall, there were financial savings to be had if you were able to meet the criteria for ASFU ("application-specific") or ESL ("embedded") instead of FUR ("full-use").
Usually, with embedded installs of SQL/Oracle/whatever database, there are no maintenance routines put in place so the system slowly grinds to a halt as the database(s) fill up, become fragmented, the filesystem fills up, etc, etc. Then you find out that all of the database access is via the system administrator account, with default passwords. This is especially true if you've paid consultants to come in and set it up who then don't have to support the system - throw it in and forget!
Well we had it tough. We used to have to get up out of the shoe box at twelve o'clock at night, and lick the road clean with our tongues. We had half a handful of freezing cold 28.8, worked twenty-four hours a day at the mill for four pence every six years, and when we got home, our Dad would slice us in two with a bread knife.
peasants :-) we were lucky to be round the corner from telewest house in Preston. At that time they used to server from telewest and the local area was a testing grounds for technology. Although the cable TV was still analogue we had 512k back in the late 90's and 1Mb in early 2000. The phone lines were ISDN still. The company wound up and I ended up moving to Penrith - ironically Penrith became BT test town for both ADSL max and fibre. The max meant I had 8Mb around 2002 ish; years before it was adopted elsewhere. The fibre trial didnt work out as well as the rest of the country as we only got 17Mb (which has only recently in the last few years moved up to the usual 40/80 fibre) - it wasnt really any better then everyone else ADSL 2 (but Penrith is all aluminium cable so 2 didnt work too well). The 17Mb was symmetical though and came online just before I moved around 2009- small mercies.
(but Penrith is all aluminium cable so 2 didnt work too well)
Welcome to Cumbria, where most of our lines are still aluminium! A house I once lived in had a street cab 10 yards outside the front door, but my cab was 5km away, the cables ran straight past the new cab but Openreach wouldn't do anything about it, so I had to live with 1mb fibre (at fibre costs) and that was early this year!
We had worse speeds that people up the sides of the fells!
300 baud, you had it lucky, I had to use 110 baud...
Ah bugger it. I mostly used 1200/75 V23 and 1200/1200 V22, but played Essex MUD a lot, and one day the JANET links went down, and the (two) direct dial numbers were only 110 baud. (Usual 300 baud modem but I had to change a jumper on my motherboard! Gosh I must have been addicted..)
We had PSS lines to Essex, one night op played 6 parallel characters over 6 lines to get his "team" through... The bill the company got at the end of the month was horrific!
Luckily he was on good terms with the person in charge of the comms bill, they managed to parcel the costs out over a dozen or so projects, on the promise that it never happened again!
In the late 90's early 2000's I used to work for a "big cyber security" partner and we had access to their network, which was a reasonable 4mbps, due to firewalling we heard ... we could even reach the internet via it but it felt like a 56k modem. Then, one day, it was lightning fast, we checked the internet, and it was lightning fast, too, about 8 times faster than our corporate internet access. It took "big cyber security" months to realize all the data poring through to this partner's network ... I had great trouble, and no authority, keeping the team off p2p - my boss was the hardest abuser ... he literally filled several 4Gb drives with mp3's and mpg's, swapping them out as he went along, his download queues were impressive.
I used to fix those modems:
Prism 2000 (Made by Thorn-EMI 99% of the time it was a failure of the Voltage regulators).
VTX5000 - Spectrum specific.
Voyager 7 or 11 Modems (Also known as Magic Modems & Kirk's Enterprise).
Happy days & much drinking & eating in London during trade shows
Interesting you should comment about knitting and Northern line signal boxes - I believe one of the sigalmen at Park Junction (probably the sparsest box on the Underground for train movements) used to make tapestries in between signalling trains.
There used to be a lot more to the Northern Line than there is now. If you go to East Finchley, you may wonder why are there four platforms? The outer ones go underground to Highgate, the inner ones go to a set of sidings known as Park Junction, but they also continue on, via a short tunnel, to Highgate High Level, and to other parts of north London originally on the Underground map. Though the line is disused, the track removed, you can still legitimately walk sections of it.
I used to work in that short section of tunnel between Park Junction and Highgate High Level on the Holy Grail of Railway Signal Engineering - Block-Jointless Track Circuiting. Notches were cut into the track at measured intervals and wires inserted where we could attach our measuring instruments. Our "office" was on the station, but not the waiting room as the tocal Wiccas/Druids had commandeered that for their nightly rituals.
Ah, the uncompleted conversion of the mainline railway route into the Northern Heights extension, I see!
I love the idea that the remaining track is used for experimenting; Aldwych station is apparently similarly also used for experimenting with new station platform refurbishment ideas, after its closure as a public station.
@Doctor Syntax yes, I think he learnt his lesson.
Someone else didn't. They were caught copying software from all the Macs in the building. On night shift, he'd wander round, turning the Macs on and copy any applications he didn't already have onto floppy disk...
Only on some machines, he left the original icons in other places on the desktop, which made people suspicious.
They pulled him in and confronted him, he admitted it. Immediate dismisal and security went with him, to his flat, and collected all the floppies.
We had 9k6... Seems OK? It was a tertiary institution and there was around 25000 staff and students.
From Monday to Friday, as long as morons weren't sending copies of Windows 95 to their home accounts (which also dates this...), we could generally process a working days e-mail in around 27-28 hours.
We caught up eventually at the weekend.
Not a typo. I hate to spell out my little jokes, but the Manchester computer apparently didn't have a ball with the correct representation of the codes, so "00000b" printed as "/". When he presented progress in London, the audience were completely thrown by this and were, of course, unable to understand the printouts. There is a legend that someone asked if all the slashes were the rain beating on the Manchester windows.
My first modem, I don't know what the rated speed was (somewhere between 300 and 2400), but with an acoustic coupler to the shitty carbon granule handsets and the equally shitty phone lines full of crackle and pop, and predating most error correction niceties, it was usual to see Xmodem continually resending the same block. Think a gigabyte transferred in order to receive a 20K file. Maybe not but that's what it felt like. Oh and no auto resume so if the transfer failed, you began right back at the beginning.
It was pretty normal to pick up the handset and beat the crap out of the desk with it prior to using it. Apparently shaking up the granules inside improved their shitty frequency response so you stood a slim chance that the modem might deign your connection worthy.
Now I live at the end of an insanely long bit of twisted pair and can squeeze ~4 megabit down it in the middle of a
hurricaneMichael Fish says we don't get hurricanes but when a neighbour's roof blows off it's a bloody hurricane... At any rate, you haven't done comms until you've done an acoustic modem.
Ahhh, ISDN. I still have my old Digi DataFire ISDN card that I used to use to dial into the US with for 64K uncontended connectivity directly into our HQ's services, instead of the crawling 33.6K (you try running Lotus Notes over PPTP on 33.6K and 16K international data throttling and tell me it's not anything but crawling) I had for standard Internet before that. The 56.6K Sportster went back into its box until I moved countries.
The phone bill dropped by almost 2/3 when that DataFire arrived and I kicked it into action...
I think my first hosts (of a physical server, once I'd upgraded from a vhost on shared hosting) themselves had about 128k ISDN connectivity. It seemed quite fast back then.
One day my server just vanished from the 'net. Turned out the host had gone bust, and my kit, like theirs, was in limbo at the mercy of liquidators. Until my colleague who knows about such things got in his car and physically rescued it.
Ah, the Good Old Days!
It is not so much the inexperienced sysadmin to blame as the colo provider. 1. Accepting a phone call from a stranger. Phishing was an issue even back then. 2. Not being able to identify the correct system twice. 3. Not having terminal servers. In this case no physical access should have been necessary.
And the software dev. house is to blame too. 1. not securing their system at Colo provider. 2. Apparently no due diligence done when selecting colo. 3. Not showing the new guy the actual systems. 4. Bringing back system in-house in the end
Back in the bad old days when I worked in a data center and often had to ship equipment to customers everything had labels attached on the front and back - Our assigned equipment ID and in large letters "Property of xxxxx - please call our 24/7 network operations xxx-xxx-xxxx before disconnecting, moving or any other issues with this equipment"
Sometimes I would have to visit client sites to troubleshoot connectivity with our equipment and was astounded at the cabinets with stacks of identical comm equipment, ours being the only ones clearly labeled.
Not much better these days, in *Redacted* at another Helldesk we had a Colo location with 2 isles of our equipment.
About 1/3 of the time I went there (Which was a lot because of backup tapes), the damn door was proped open, even though I had to call through and book an appointment and then be let through over the phone outside the door.
Back in the wild old dot com days, I visited my employer at the time's DC in NYC. The previous week, and employee of one dot com was walking past one of his competitors cages, which conveniently enough, the walls did not reach the ceiling. Said employee exploited this by hurling a piece of cardboard over the top with such force it disconnected the competitors fiber uplink :-)
Gosh those days seem so far away now. My biggest mental somersault was in the days when we happily paid £10 for a small sun box as a gateway/firewall, and another £10k per annum for "management", and then in the late 90s finding that a throwaway box running linux was just as, if not more, capable, and that maybe, just maybe, we'd been taken for a ride for some years. No wonder we all developed such cynicism. I remember so desperately trying to see the flaw in the reality when concluding that there really wasn't some special secret security sauce in the expensive version.
The Sun (or otherBigName) with the expensive contract would be for users needing that reliable very-high uptime. For the rest of us, Linux or *BSD on commodity hardware has made more sense since about the mid-90s.
The difficulty back then was that the choice was between an expensive package like yours and something slapdash like the host in the story. It's only really this century we've seen the rise of cheaper hosts who also make it their business to know their arse from their elbow.
Place I worked at used ISDN lines to host application servers for their customers. Our new server admin decided to get some brownie points by setting up network monitoring.
They only made two mistakes.
One was to set the ping size to 64kb rather than 64bytes.
The second was to ping test a router at the other end of an ISDN call. An international ISDN call from the UK to the Republic of Ireland.
No one noticed until the first phone bill arrived.
Best I've managed is to firewall all traffic (turns out to need to configure a whitelist before blocking all traffic on specific ports... who'd've guessed?). Embarrassing phone call later and a quick talking the tech through how to kill the rule and all was right with the world.
Nowadays I've my server at home warming the attic so it's a much shorter walk.
On a Cisco router (note: I haven't logged into a Cisco CLI in over a decade, so beware of fading memory and outdated knowledge):
#rel in 5
Yes. Its funny how quick one learns that one, especially if it involves driving down to Telehouse with a laptop to fix an ACL on a router that didn't have remote console at the time.
one of my co-workers likes to rant about Juniper (whenever the topic comes up). I don't use either Cisco or Juniper myself(Extreme networks customer for ~19 years now). Anyway his rant mainly revolved around this feature you mention. Which I had heard touted by another friend a long time ago. Had to do with the JunOS software not giving errors when the configuration was incorrect/syntax errors or whatever. It would go along like everything is fine, shit wouldn't be working but gave no indication as to why. He evaluated Juniper at his previous company (they were a Cisco shop) and it didn't get past the eval stage because of that I think, drove them crazy.
commit confirm <minutes-default-10>
Cisco box config always feels so antiquated compared to Junper.
I used Cisco kit for years before ending hands-on on any Juniper kit so perhaps bit more comfortable with it and quirks of IOS. Having said that, I use both and I'd say they both have their pros and cons.
Juniper may have the edge in more value for money if you look at what you need to pay for the kit to get the throughput you want.
Ugh... don't even get me started. Network jiggery pokery was never something I liked doing and other than turning it off and back on again, no-one at work wants to touch our dev network. Not even the actual people employed to poke such networking kit.
That said turns out L2 network hubs with SPF ports aren't that expensive on fleabay (one of these and a SAN array is on my christmas/birthday list for the 'home' server)
Let's re-read the story. The OP said that their ISDN 128k line wasn't good enough to host their website - hence the server being in the co-lo. 256k isn't likely to cut it either unless the website was entirely text based with no dancing baby GIF files or they were getting very few hits per-day.
I managed to run ip route delete default on a linux firewall box I was SSHing to over the internet.
Hadn't saved any config changes, so I rang the site guy and asked for a reboot please. Alll good so far.
Machine stops replying to ping, starts pinging, so I'm sitting there pressing up-arrow, enter, (SSH connect refused) up-arrow, enter (refused) over and over.
SSH starts, I press up-arrow, enter, get on the box with an SSH key so no password, then up-arrow, enter.
What was the last command? `ip route delete default `
That became the definition of Muppetty.
I do remember one time I had to take down Server A and to make sure everything was correct I took the opportunity to turn off the UPS and then go around the back and disconnect Server A's UPS from the mains socket.
Beep beep beep went Server B's UPS. Glad I caught that before a real problem happenned, I swapped the plugs to the correct sockets and nobody knew that had happenned.
Biting the hand that feeds IT © 1998–2019