back to article OK, team, we've got the big demo tomorrow and we're feeling confident. Let's reboot the servers

After a long, hard week, what better way to start Friday than with a dose of On Call, El Reg's weekly column for tech traumas, mishaps and eureka moments. This time, we meet "Robin", who tells us of a time he was consulting for a group building a web-based billing system in the early 2000s. In some cases, he said, the …

  1. Robin

    Another Robin in the 'Hood

    When I saw the submitter's name I panicked for a split second and wondered if I'd previously submitted a story whilst N-sheets to the wind :-)

    1. Anonymous Coward
      Anonymous Coward

      Re: Another Robin in the 'Hood

      I remember we had a sales rep go to demo our in-house developed to a customer. The licensing was always a bone of contention from a support perspective as it was made needlessly complicated to defeat those pesky software pirates. It usually resulting in calls from legitimate customers complaining their software had shut down with licensing issues.

      Anyway, when the rep started the demo, he was presented with the embarassing "Trial License Expired" error. At this point, the potential customer saved the day by producing a keygen he found on the Internet for the software and the demonstration proceeded.

      Surprisingly, the visit turned into a sale but I'm not sure if that was down to guilt on behalf of the customer or something else. Needless to say, he ordered the cheapest license available .... ;)

      1. Anonymous Coward
        Anonymous Coward

        Re: Another Robin in the 'Hood

        Tell us, you didn't trust the sales rep to give him a full, not expiring licenses for demos?

        1. Anonymous Coward
          Anonymous Coward

          Re: Another Robin in the 'Hood

          No - they didn't. That was a decision well above my pay-grade unfortunately.

          1. A.P. Veening Silver badge

            Re: Another Robin in the 'Hood

            There should have been a specific demo-license, one where everything is rolled back when logging off.

            1. BinkyTheMagicPaperclip Silver badge

              Re: Another Robin in the 'Hood

              In an ideal world you're right. In reality there's often insufficient developer resource/justification to do that.

        2. Anonymous Coward
          Anonymous Coward

          Re: Another Robin in the 'Hood

          "Tell us, you didn't trust the sales rep to give him a full, not expiring licenses for demos?"

          A number of vendors don't seem to have a concept of a "non-expiring licence" - they are so paranoid about anyone getting their software for free, it always expires, in-spite of the pain it may cause.

          In my experience, such licences are best addressed with a debugger and an hour or two of patience...

          1. Andrew Moore

            Re: Another Robin in the 'Hood

            One of the products I look after has a perpetual license on it. Which has to be renewed every year.

            1. Anonymous Coward
              Anonymous Coward

              Re: Another Robin in the 'Hood

              Imagine what happens when the licence they send you on renewal doesn't match the product you're renewing... Happened to my company last week: software accepted the new licence without any issue, but for some reason started to throw various seeminlgy legitimate data errors... After 3 days of 'fixing' stuff, someone realised that these issues started when the new licence was applied, had a closer look, asked the supplier 'Dafuq did you give us?', and fixed everything once applied the correct licence. Just on time: it was Friday!

      2. JimboSmith Silver badge

        Re: Another Robin in the 'Hood

        In the distant past circa 2000 I worked for a company who had just upgraded to a whole new version of a piece of software. As someone who used the software and the most junior after the first renewal I was the one who had to phone up every 45 days to get a new license key. It wasn't just one key though as each brand using it required one. So I'd call up and speak to their UK office and go through the procedure. This involved giving them the code that the software would generate and getting one back to type in. Sadly the code I had was about 20 digits and there were four of them. The first time I did it I saw why I'd been given the job. A new code was being generated every minute or so. If didn't get the code entered before that minute was up my machine generated a new one. At that point you started again which was nice.

        Now whilst it was possible to do this it just wasn't very easy. I bought a small kitchen timer so that I could make sure I was in time. Spelling things out NATO phonetically helped with the accuracy but not the speed. I said that they needed to increase the time allowed for key entry and the lady I used to talk to when doing it agreed. However she said that there was also an option to do it over the web. It wasn't automatic and required someone to manually do it every time. I said we didn't have those machines connected to the net and so that wouldn't work. I asked if anyone did and she said not in the UK. The next version had a two minute window for you to enter the code. They'd allegedly bought in the licensing module and therefore getting the timing changed had taken a while. Their previous system had been developed in house and was hackable. After a while I looked for a keygen online because I was sick of it. Sadly I never found one and I do not miss those calls.

        1. Charles Calthrop

          Re: Another Robin in the 'Hood

          In the distant past circa 2000

          Cheers for that mate

      3. Prst. V.Jeltz Silver badge

        Re: Another Robin in the 'Hood

        but what does that have to do with Robin's comment that you replied to?

      4. royprime

        Re: Another Robin in the 'Hood

        Yes, had to do that myself away on site as the trial licence had expired and no one would give me the nod to purchase it.

        Grabbed a licence from the web and just hoped that no one checked the about / registered tab as it was registered to a user name along the lines of idontpayfor**k@gotohell.com (or something very similar).

  2. LDS Silver badge
    Devil

    Big project involving *banks*, and no money for development licenses?

    The company and the banks really deserved that....

    Most software has specific development licenses available - usually it means you have to register and pay an annual subscription - but you get full product licenses to be used for development only - plus support, patches, etc with which you usually need to test your software with - something you usually don't get with trials.

    1. big_D Silver badge
      Facepalm

      Re: Big project involving *banks*, and no money for development licenses?

      We are talking about banks here, how do you think they make money? By actually paying for licenses they "don't" need*?

      * if the trial license is free, you don't need to pay for a development license, that would be throwing money away!

      1. LDS Silver badge
        Facepalm

        "By actually paying for licenses they "don't" need*?"

        The same banks that usually throw away a lot of money in many silly ways?

        Which licenses they didn't need? The one that actually made the software working, as the demo didn't work when the license expired? I guess it was the usual case of "we'll address the licenses issue later..."

        A trial licenses *is not* a development license - the name tells what it is for. Development licenses are usually exactly alike the full ones, so you can develop and test your software *properly* without any trial/demo restrictions and lack of support and fixes.

        If you're mean and prefer "free trials" to save some money, you can end to lose a lot because you can't develop properly.

        If you win a bid because you're going to save on development expenses using such tricks, there are big risk you become a liability when the software is deployed.

    2. Trixr Bronze badge

      Re: Big project involving *banks*, and no money for development licenses?

      Back in that day, no, most enterprise software did not have development licensing. Very rarely, we managed to negotiate a short licence term, like 6 months. Maybe for two products where we were one of a few potential customers in the country.

      Otherwise, it was an evaluation copy (generally still complete with incredibly annoying licence activation) until it expired or someone paid. There ended up being much screwing around with system dates if some idiot restarted the dev system after a licence expired (and this was genuine dev stuff - no intent to run production functions on "dev" systems).

      Frankly, I'm a little surprised they didn't h@X0r the system date during the demo in the OP.

  3. Nick Kew Silver badge
    Pint

    Golly, is it Monday?

    ... or is it just me to whom this looks like a "Who, Me?"

    Now the million dollar question: how many would-be embarrassments did this inability to log on save them? Not that you can answer a counterfactual.

    1. Prst. V.Jeltz Silver badge

      Re: Golly, is it Monday?

      yep , its monday :(

  4. defiler Silver badge

    Big demo. Should we test?

    Nah - we'll wing it on the day!

    The number of times I've seen this. It seems like everyone thinks that everything should just work every time, without giving it a wee check beforehand.

    Got a video file in your keynote presentation? It worked fine on computer A - we'll never need to test that the codecs are in computer B... For example.

    1. ArrZarr Silver badge
      Unhappy

      Re: Big demo. Should we test?

      At one point, I was giving training to a client on how to follow a new process for reporting using a supplier's site.

      The process itself was simple - pull this report from here and paste into this Excel.

      The training session was made more entertaining by the supplier's site going down two minutes in.

      1. Aladdin Sane Silver badge

        Re: Big demo. Should we test?

        Producing reports by pasting into excel always makes me twitch.

        1. veti Silver badge

          Re: Big demo. Should we test?

          Isn't that what Excel is for?

        2. ArrZarr Silver badge

          Re: Big demo. Should we test?

          In my defense, the pasting into excel bit was because this was a backup process for when the automatic importing failed.

      2. JulieM Silver badge

        Re: Big demo. Should we test?

        No, no, no! The proper way to create reports is by running a script on a Cron job; which pulls the required figures directly from the database, does a little light munging and then spits out a .csv file with the wanted information on it; already attached to an e-mail.

        Best part about this is you only have to write the script once; and you now have back the amount of time that the tie-wearers thought it would have taken you to create the report by hand each time.

        1. baud Bronze badge

          Re: Big demo. Should we test?

          Or create your script but tell no-one about it: then use the time allocated for the report creation to do something productive instead

          1. Aladdin Sane Silver badge

            Re: Big demo. Should we test?

            Like The Times crossword?

        2. Anonymous Coward
          Anonymous Coward

          Re: Big demo. Should we test?

          And make sure the script runs from cron in your home directory so when you leave and your home directory gets deleted your work goes with it.....

    2. GlenP Silver badge

      Re: Big demo. Should we test?

      Got a video file in your keynote presentation? It worked fine on computer A - we'll never need to test that the codecs are in computer B...

      Which is why I've always insisted presentations are made from a single computer and provided at least 24 hours in advance. Of course the more senior the person the less chance of this actually happening.

    3. big_D Silver badge

      Re: Big demo. Should we test?

      Been there and seen that many times as well - and averted it a few times myself by, you know, double checking before the meeting that everything still works.

      I've seen the meeting room PC go through patch installation, delaying a telco by 30 minutes a few times...

      No matter how hard I banged it into the support staff, they never thought to go round after "patch Tuesday" and ensure that all the meeting room kit was ready to go.

      If it was my meeting, I'd always go in half an hour early, if the room was free, and ensure that everything was working. It saved my bacon a few times.

      1. ma1010 Silver badge
        Alert

        Re: Big demo. Should we test?

        I'd always go in half an hour early, if the room was free, and ensure that everything was working. It saved my bacon a few times.

        THIS ^^^^^

        Last Saturday, I was doing a big presentation for 30+ people. I went into the room early and fired everything up -- and discovered that the ceiling-mounted projector was all buggered up (my slides were unreadable). Paranoid that I am, I had my own projector in my car, and, being early, I had time to run out, grab it, rearrange the room a bit, set my projector up on a table and get everything going in time. The attendees didn't notice any problems, which is exactly how it should be.

        1. I Am Spartacus
          Holmes

          Re: Big demo. Should we test?

          Like me going in to do a demo for a very large trading company, only to find that the only projectors they had were circa the turn of the millennium and only supported a 640x480 VGA display. Their desired configuration on the desktop was two 22" DVI connected monitors. It didn't go well.

          Or the time we went to demo the software to another oil company. This was a web-based document management software. But they would not provide us with a PC or laptop, refused to let us connect to the wifi, and the building was Faraday shielded to block cellphone signals. That demo didn;t go at all!

    4. Anonymous Coward
      Anonymous Coward

      Re: Big demo. Should we test?

      "... test that the codecs are in computer B..."

      I recently attended some training given by an outside contractor. It was heavily based on a DVD video. Turns out the laptop they brought, while it had a functioning DVD drive, didn't have playback software...

      1. John 110

        Re: Big demo. Should we test?

        Because people can't believe that they (or should that be THEY) would include a DVD drive without the ability to play DVDs. (neither can I)

        1. Anonymous Coward
          Anonymous Coward

          Re: Big demo. Should we test?

          I can never find the software when I need it, i have an external BluRay drive for the odd occasion I need one, but I can never find the install CD's to play DVD's or blurays!

      2. JimboSmith Silver badge

        Re: Big demo. Should we test?

        After a merger the head of health and safety is doing the rounds of the regional offices of the other firm. He checked and was told that there were presentation facilities at all the offices. He turned up at the first one and found that it was a struggle even to get into the building. Staff eventually let him into the building after checking who he was with head office. The staff directed him to the lunch/meeting room and he sat there watched by the staff as he attempted to do his presentation. After the laptop had booted and one of them had logged in for him he attempted to insert his CDRom. The drive will not open by pushing the button and he assumed that it was broken. So he tried eject on windows explorer which didn't work either. Taking a paperclip he managed to open the drawer and insert the disc. It would't show up in explorer though which was annoying. Apologising he tried a USB stick which wouldn't show up in explorer either. The staff have sat there in silence and when he asked if they had another machine the answer was yes. Won't work either though he's told, it's just as locked down.

        He said he felt like he was the person who had driven off down a very long but dead end road out of a village watched by the local yokels who hadn't tried to stop him. He asked when they were going to tell him that particular fact if at all if he hadn't asked. They said that they enjoyed watching people trying and to see how far they went before asking. One of them took pity on him and loaned him their personal laptop so he could do the presentation. He got back to the office the next day and sent round an email telling people to book out one of our laptops if visiting.

      3. Robert Carnegie Silver badge

        DVD, no player?

        Download VLC, free.

        Main problem, making sure you don't get one of many bootleg downloads with nasty things added. But this isn't very difficult.

    5. JimboSmith Silver badge

      Re: Big demo. Should we test?

      Saw a presentation where on one PowerPoint slide the video they wanted to show was just a link to a YouTube file. When it was being shown during one training session i attended the link was clicked and internet explorer loaded. A YouTube page appeared and the video not available graphic. Apparently Warner Brothers had objected to their music and or video being used. Much hilarity all round from the trainees.

    6. Doctor Syntax Silver badge

      Re: Big demo. Should we test?

      Then there was the situation where the meeting room had cabled up ceiling-mounted projector, bluetooth keyboard and mouse so the PC could be locked in a cupboard because nobody needed to access it. Not even for things like switching it on....

    7. bombastic bob Silver badge
      WTF?

      Re: Big demo. Should we test?

      A demo is an integral part of the development process. A functionality "proof of concept" demo should be a part of the initial planning phase of any 'waterfall' project, even if it's just dummied up for the dog and pony show.

      not sure what the article meant by referring to 'that' as a 'a classic waterfall affair', when there's apparently NOTHING to demo progress along the way, for 12 months even!

      In just about every project I've worked on, especially my own, there's a demo that must be done early on as a "proof of concept", and some occasional functionality tests to make sure integration is working properly. And in one specific case, a team of 3 (minus me) went off to do something that was initially called 'agile' but was later deemed to be 'cluster-@#$%', and spent a YEAR on something I'd dummied up and demo'd in about 3 weeks... [and my version was used in several subsequent potential customer demos to show what the company was working on].

      The 3 man team version was a re-write of what I'd done, with my version as a 'guide' of sorts, and the specs kept changing. They _NEVER_ implemented the functionality in the 3-man version until after one member was lost from the layoff cycle and I was brought in to help finish the thing. That project was always chasing the mayfly of "features" without FIRST getting the core to work.

      In any case, what was described in the article is NOT a 'waterfall' process. A 'waterfall' process would focus on the big stuff up front as part of the overall design spec. And, a _PROPERLY_ done waterfall process would have a "dog and pony show ready" demo of the features as a proof of concept. I'd actually use that to test the system along the way, from time to time, swapping in 'real features' for dummy ones as needed to test things. [this differs from 'test driven development' in that I'd just make sure it all fits and works, rather than doing 'unit tests' all of the time and wasting effort re-testing trivial things over and over]

      Typically my bosses/managers/customers would want to see this kind of demo from time to time to make sure I was getting work done. It makes them happy to see something, to see "progress", and they usually gave feedback which helps me to make them happy. Then you can tell them 'yes' 'no' 'it will be expensive' or 'it will take too long' and discuss stuff without having to move the target a whole hell of a lot.

      Anyway, if the project went on for a year without any kind of dummied-up demo to at LEAST keep the client from asking too many questions, that's not 'waterfall'. that's more like 'poorly managed'. And from what I understand, 'FRagile' projects are well known for 'poorly managed' more often than not.

      I would say that 'Agile' should look like 'waterfall' the way I described it, with only occasional 'scrum' meetings and more frequent one-on-one's with the project manager. But just like your average cluster-blank isn't Agile, it isn't waterfall, either.

  5. pavel.petrman Bronze badge

    Infrastructure

    Infrastructure was tested; features, like firewalls, failure, redundant databases, servers, were all tested

    Well they thought of everything a development project might need, I'd give them that! (Or is there one comma too many?)

    1. Bronek Kozicki Silver badge

      Re: Infrastructure

      One might have hoped that principles of chaos engineering were applied, to ensure soft fail and quick recovery. One would be wrong.

    2. Paul Shirley

      Re: Infrastructure

      I've worked on many game codebases with built in "crash now" commands. Sometimes failure really is an option!

      More worrying is the times you try desperately hard to cause a crash but fail...

      1. pavel.petrman Bronze badge

        Re: Infrastructure

        Re "crash now" - this is actually what got us into this wonderful world of reasonably functioning microcomputers used for everything what was once domain of mainframes. Perhaps the term "kernel panic" rings a bell - yes, that was it! Instead of trying to recover from any possible failure state, like Multics would do, Unix just drew the line and was built to give up. That made Unix lightweight and simple enough for it to be deployable on a small scale computer, and the rest is history, erm, I mean present state of the art :)

        1. JulieM Silver badge

          Re: Infrastructure

          Well, the idea behind that was: If you're in a hole, stop digging. Continuing to use a faulty piece of equipment can make the damage worse and damage other things. If Unix gets into a situation from which it can't recover, it stops with a kernel panic message precisely in order to avoid making anything any worse than it already is. Overwriting the wrong location with the wrong data, for example.

          There is an error message which I put into some of my scripts, for use in serious situations: "Your problem, meatsack!"

        2. Doctor Syntax Silver badge

          Re: Infrastructure

          "recover from any possible failure state, like Multics would do"

          Multics is able to order up a replacement for the failed disk drive, open the box, hunt up a spare drive cable, plug the drive in, find and do up the fixing screws and then mount the drive, recover the data and carry on?

          1. ibmalone Silver badge

            Re: Infrastructure

            Maybe Plan 9 can?

  6. PM from Hell
    Facepalm

    Exec demo's - test and freeze and don't restart the bl**dy infrastructure

    Over 30 years of managing projects I have lost count of the number of times I have had to reschedule demo's because there has been a 'minor' change implemented outside my control.

    I've also been caught out by 'trial' licences in the past and I won't agree to use them for development platforms, also if I can't schedule a server reboot a couple of weeks in advance of the demo then it gets embargoed. I do work with the infrastructure teams to make sure that my dev and test environments are patched and software is updated in line with production as that avoids issues at cutover, but the period before a major demo changes are frozen. Similarly if I don't have a static demo environment then there is a config freeze on the environment being used for a week before the demo.

    Even with these rules in place I was still embarrassed last year as a very senior dev decided to make a 'minor config change to the ruby config which won't affect the UI' so he didn't even bother telling me. Guess what, it completely broke the links between screens and he had to type in the full url to navigate between them. Needless to say the demo was abandoned after 5 minutes and I had to spend the next 2 weeks apologising to numerous managers who had heard that we would not be ready to go live because the system was completely broken. The demo 2 weeks later was perfect but we'd already lost the confidence of the operational business by then and it never fully recovered.

  7. sandman

    The other side of the coin

    In one company I worked for, I was on a small panel that evaluated any software we were thinking of buying. The scenario was always the same, the super-confident salesperson would spout about the ease of use, the huge productivity gains to be had and the miraculous ROI. Then the poor bloody techie would attempt to demonstrate the miracle software... To be fair, after much sweating and naked panic, something would usually appear on screen, at least temporarily. It was fun, in a perverse way, to watch the salespersons eyes as the prospect of a juicy commission slinked away.

    1. big_D Silver badge

      Re: The other side of the coin

      One of the best sales pitches I saw was a mainframe supplier.

      The rep turned up with a massive machine, it was duly installed and he handed us a tape with source code. We should load it onto our existing system (a VAX) and on the new mainframe, compile it on both (using the optimization on the VAX) and to call him back in a week or so, when the mainframe was finished.

      An hour later, when he got back to the office, there was a message that he should call us...

      The test software was running fine on the mainframe, but the VAX was already finished!

      It turns out that he had been too optimistic. The compiler checked the code:

      1.) Input into the program: none.

      2) Processing : check

      3) Output from the program: none.

      Optimization = processing is redundant, optimize it out of the executable. The program loaded into memory and quit immediately.

      The mainframe, on the other hand, was busily building a multi-million point multi-dimensional array and filling it with randon numbers...

    2. Anonymous Coward
      Anonymous Coward

      Re: The other side of the coin

      A friend of mine must have been a serial killer in a past life, because his current job has him doing both sales and implementation.

      He has to sell software based on what he's been told it *should* do, and then implement it based on what the customer understood of what he told them. Anything and everything that goes wrong looks like his fault.

  8. SPiT

    Another failed demo

    My favourite failed demo was many, many, many years ago when system builds took a long time. Our manager had been given 6 weeks notice of having to give a demo but didn't bother to tell anyone. When he did at 3:00 in the afternoon we were 2 hours into a complete system rebuild to incorporate some core system changes which we expected to break things all over the place.

  9. chivo243 Silver badge
    Trollface

    chortle

    is all I can do when people fail like this. I've lost count of the number of presentations etc that didn't "Start" correctly because the users didn't test before they went live. When the dust settles, and the presentations\demos are finally moving forward, I usually leave a little gem like: "Before your next presentation\demo, give me a call and we'll make sure it works before you go live."

    What I really fee bad about is outsiders that roll up and find they haven't brought any accessories, like a charger!

    1. Nick Kew Silver badge

      Re: chortle

      Like the time I had a conference presentation to give, only to find that my laptop was incompatible with the conference's OHP? I actually tried very hard to test that in advance (and fix as necessary), but was thwarted by the perfectly legitimate claims of earlier speakers from the moment the room was opened.

      After that I took to uploading material to the 'net[1] ahead of time for maximal alternative access methods.

      [1] The term "cloud" wasn't yet in use. Can't remember how widely-available wifi was at the time.

      1. Glen 1 Bronze badge

        Re: chortle

        OHP?

        This is an OHP:

        https://en.m.wikipedia.org/wiki/Overhead_projector

        I'd be very surprised if *anyone's* laptop was compatible.

        1. Robert Carnegie Silver badge

          Re: chortle

          Ahem. https://en.wikipedia.org/wiki/Overhead_projector#LCD_overhead_displays

          1. Anonymous Coward
            Anonymous Coward

            Re: chortle

            Never seen one of those, but it looks like the sort of Heath Robinson boffinery I admire

          2. Solmyr ibn Wali Barad

            Re: chortle

            Thinkpad 755CV had a detachable LCD cover, so its screen could be placed on the OHP. Neat.

            http://www.thinkwiki.org/wiki/Category:755CV

        2. Anonymous Coward
          Anonymous Coward

          Re: chortle

          OHP?

          Was asked to fix on of those last year.....

          I made some polite excuses and left.

    2. Anonymous Coward
      Anonymous Coward

      Re: chortle

      I don't - I would say it's lack of planning on their part not to bring their own charger. Also, we use HP laptops, what makes you think I would have (say) a Dell PSU just laying around??

  10. JulieM Silver badge

    Why?!

    Why do people continue to put up with these shenanigans from software vendors?

    When I use a piece of software, I expect the right to ENJOY the use of it without interference (so no "time-restricted trials" or whatever have you). I expect the right to STUDY its operation (for the exercise of which, the annotated Source Code is highly desirable). I expect the right to SHARE it with my neighbours, under the same terms as I acquired it (including not doing anything that might restrict someone else's exercised of their rights); and I expect the right to ADAPT it the better to suit my individual requirements (for which the Source Code is also highly desirable). I also expect the right to delegate any of these rights to anyone I trust, without anyone else's say-so -- and the reciprocal right to assist others in the exercise of their rights, without anyone save the local tax authorities demanding a cut of any money I may earn from doing so.

    For almost everything you might have to do with a computer nowadays, there is software available which fully respects the above. And I would rather do whatever I have to do by hand, than have anything to do with any piece of software which does not respect these fundamental rights.

    1. Adrian 4 Silver badge

      Re: Why?!

      Because in a corporate environment, it's traditional to expect another right. The right to BLAME someone else. You don't get that with OSS : you're the first-line support. Of course, being able to do your own support is a feature, not a bug. Wanting someone else to shout at is a corporate bug.

      1. Kubla Cant Silver badge

        Re: Why?!

        You don't get that with OSS : you're the first-line support.

        Most of the time, yes.

        But don't forget the Red Hat paradox: you can make a lot of money selling something that's free. Despite the massive contribution of unpaid developers, large FOSS projects often need some source of income, and paid support is often the solution.

        1. Anonymous Coward
          Anonymous Coward

          "the Red Hat paradox: you can make a lot of money selling something that's free."

          Just RedHat ended up swallowed by IBM - so maybe it's business model wasn't so sound.

          Anyway, RedHat didn't sell support. It sells peace of mind - ensuring long-term availability of software which as soon as upgraded breaks everything around - at the cost of maintaining very old versions. So customers can keep on using their software which wouldn't work with anything released later.

          For example, in the past days I had the "pleasure" of trying to understand why Rabbit-MQ under CentOS 7 didn't work with TLS 1.0+ enabled. It turned out the version of Erlang/Rabbit it installs is so old it has several TLS issues. Sure, install the latest release and go - but that's not what RedHat sells.

          Anyway RedHat is one of the few successful companies in the FOSS space - other had to change their licenses because selling support only lead them nowhere, especially those who actually build software from scratch, not only mostly repackage software developed elsewhere - usually as a sub-product of another business - which requires far less resources.

          Not surprisingly also, RedHat has other business - i.e. Ansible - where yo pay dear money to get the usable versions.

          1. Anonymous Crowbar

            Re: "the Red Hat paradox: you can make a lot of money selling something that's free."

            Eh Ansible is free. Tower is the paid for stuff, and TBH in my last job, we had it, and just dumped it as it just wasnt very useful. To us anyway.

        2. Doctor Syntax Silver badge

          Re: Why?!

          "But don't forget the Red Hat paradox... Despite the massive contribution of unpaid developers, large FOSS projects often need some source of income, and paid support is often the solution."

          No paradox. What you're not realising is that by and large is somebody paying the developers and that somebody is often Red Hat. In other cases it's often people with some other financial interest such as Intel who expect to sell the processors the S/W will run on.

          The actual principle behind the funding is that a group of businesses who have something to gain from a project find it more effective to get together, do their bit at their own expense and share it rather than each of them try do the whole thing themselves with huge and pointless duplication of effort.

          1. Anonymous Coward
            Anonymous Coward

            "No paradox"

            Are you sure? Actually, RedHat was the only one to become really profitable - many other distro that tried the same path failed. RedHat focused on a "narrow" type of users - those for whom even a five-year LTS release it's really a very short time - they need ten-fifteen years or more.

            It's, after all, a kind of lock-in - once you're running some expensive applications on RHEL, you don't really have much other options - as there's a good chance they won't work on another distro (but CentOS - but you have no commercial support).

            Also, there's an inherent risk when software becomes a by-product of other needs - it becomes far less versatile, as those driving the product are mostly interested in their needs only, as they don't plan to sell it to a wider audience. While commercial software needs to appeal to the larger user base to be really profitable.

            That's why, for example, Linux server gets a lot of attention, while the desktop part doesn't. And this is the most successful FOSS project, and it's still versatile enough.

            Other FOSS projects are far less versatile, as they are often designed and paid for to fit narrow needs, and cash/code will come often just to support specific needs, not an overall development. It's no surprise that often the most generic code comes from universities or the like, where developers may have some more freedom - just, usually they'll move on, and the code will suffer.

            1. Doctor Syntax Silver badge

              Re: "No paradox"

              "Are you sure? Actually, RedHat was the only one to become really profitable - many other distro that tried the same path failed"

              There's usually an annual report with a break-down by the main individual submitters and the associated companies where they can be identified. For 2017 "none" and "unknown" together amount to 12.3% whicn means that 7/8ths of Linux kernel development is by developers sponsored by identifiable companies or "consultants" (3.3%) which I take to mean contractors paid for out of funds contributed by the Linux Foundation (itself sponsored by big companies).

              Intel comes top at 13.1%. Red Hat is the next at 7.2%. AMD and other semiconductor manufacturers add at least another 10.2. SUSE and Canonical add 4% on behalf of other distros. Google adds 3%, whether this is on behalf of Android or their data centre operations isn't stated but Facebook also add 0.9% which I take to be data cantre oriented. IBM is 4.1%

              So, yes, the Linux kernel is mostly if not entirely built by contributions from a whole spread of businesses who have interests in having their products run on it or having their products at least in part based on it. The notion of it being built by a load of teenagers in their bedrooms is a complete fiction no matter how many people want you to believe that.

      2. Anonymous Coward
        Anonymous Coward

        "Of course, being able to do your own support is a feature, not a bug. "

        Classical mindset of a nerd who believes the world begins and ends with computer and software. Most business are not IT business and have no will to become ones. Nor most developers are willingly to work for companies that are not IT business, and thereby will never have the resources for proper software development. More so if you're an SMB and you really have no way to be your "first line of support".

        Most companies need software that 1) [mostly] work 2) have professional support when needed.

        It's not a matter of blaming someone else, it's just like you need someone else to fix things you don't have any idea to work on yourself, and their critical enough.

        1. Doctor Syntax Silver badge

          Re: "Of course, being able to do your own support is a feature, not a bug. "

          More so if you're an SMB and you really have no way to be your "first line of support".

          Most companies need software that 1) [mostly] work 2) have professional support when needed.

          I have worked in environments where we had support from H/W & S/W for servers from HP & IBM in big iron territory and from industry-specific package vendors but commodity H/W & S/W generally means 1st line support from a local dealer or a freelancer; how many SMBs are able to get first line support from Microsoft?

          Why should you think a local dealer or freelancer can support Windows more professionally than they could support Linux if they chose to do so? And, as has been said elsewhere, you can get support contracts from Red Hat, SUSE and Cannonical for their distros.

    2. Kevin Fairhurst

      Re: Why?!

      And if the software doesn't exist that does what you want, you have the right to write it yourself :)

    3. Anonymous Coward
      Anonymous Coward

      "I expect the right to..."

      Sure. Just, you'll have to pay enough, and for some software, it could run into the millions or billions... if you want to buy the whole product.

      And,sorry, those are "fundamental rights" only in Stallman's head.... incidentally, someone who never had to write code and sell it to live. It's far easier to think that way when you're paid regardless of what your produce.

    4. BinkyTheMagicPaperclip Silver badge

      Re: Why?!

      Bulllllllshit.

      Companies exist to make money, and ideally to satisfy the customer. Open source is not necessarily incompatible with making money, but often the pricing and license has to be carefully set.

      There are plenty of areas where closed source is the only reasonable option, and an open source option either does not exist, or is substantially inferior to any closed source product.

      Time is also money. People do not always have time to fanny around cobbling bits of software together.

      1. JulieM Silver badge

        Re: Why?!

        So, you think Open Source software isn't quite ready for your needs? Well, that's a valid point; it was written with some level of unconscious bias that if not the ultimate user then at least the person installing and configuring it knows what they are doing. The solution, rather than paying for proprietary software over which you have no control, is instead to spend that money employing your own local programmer to turn some existing Open Source ingredients that are almost right for you into something that's absolutely right for you. Local people with money to spend tend to shop in local stores, drink in local pubs, eat in local restaurants and take their families to visit local tourist attractions Most of the money that you pay a local programmer winds up remaining in the local economy.

        If you buy proprietary software, by contrast, not only are you enriching billionaires overseas; but you are also entirely dependent upon the whims and caprices of the vendor. If it breaks, and they won't fix it, you're SOL. If Open Source software breaks, at least you get to keep all the pieces; and then any sufficiently-competent person can put it right. (And usually with the beneficial side-effect of putting it right for everybody in the end.)

    5. Doctor Syntax Silver badge

      Re: Why?!

      When I use a piece of software, I expect the right to ENJOY the use of it without interference (so no "time-restricted trials" or whatever have you). I expect the right to STUDY its operation etc.

      Even amongst FOSS S/W users (and I include myself) you are exceptional and even more so amongst software users in general. Most of us (almost all, actually) don't include studying software as one of its uses.

      Most users have no understanding of any computer language nor should they need to. Very few will have fluency of all the languages in the stacks in a typical Linux distro and I doubt anyone at all has sufficient understanding of the minutiae of the all the sub-systems to avoid inflicting damage if they stray outside their competence* - hence the frequent advice not to try writing your own cryptography.

      * Remember the great Debian random number fiasco.

      1. doublelayer Silver badge

        Re: Why?!

        Those rights are nice, but not fundamental. The analogous rights are rarely available. For example, if I buy a table, I am given a table. Nothing more. I have the right to use the table as I wish, and to disassemble, modify, or destroy my table. If I rented the table, I may not have all those rights. I don't have the universal right for the table manufacturer to give me their engineering plans for the table, although it might be useful if I was modifying the table. That is a thing I have to figure out on my own if the company doesn't want to get those for me.

        The same is true of software. I have a problem with people who sell me a license to use their software in its shipped, compiled, form but implement that software in a way at odds with their license (for example, saying the license will work perpetually but requiring it to be renewed, or saying the license will function on airgapped machines but requiring an online check). I have a problem if companies sell the software as something it is not. I do not have a problem if they refuse to let me see their source code. Sometimes, this is enough for me to choose not to buy their software, as there are major benefits to having the source, but other times, this is not as important.

        Either way, I do not have the right to look at their source if they have not agreed to give it to me, any more than I have the right to look at your email if you would rather I didn't.

    6. Trixr Bronze badge

      Re: Why?!

      Oh boy. You realise the Open Software movement is relatively new? It's great, of course, but back in ye olde days, you were clobbered with very onerous licencing requirements and cost, or you wrote it yourself.

      There were always homebrew clubs and the like, but that's basically writing it yourself - you needed to understand what you're running.

      While I work in IT support, I'm not a dev and never have been. It's like saying you need to be an automotive engineer to maintain a car. The engineer would be bored and under-utilised, and getting the basic stuff fixed would be prohibitively expensive. Also, with complex systems like modern cars or enterprise IT environments, you never have one person with all the expertise required. (Don't get me started on devs doing server administration, database management or identity management - it works if everyone's an admin!)

      Given the fact we're in a capitalist society, I don't begrudge people making money off their labour. As for "exercising rights" for whatever it is you think is a "right" in terms of software, it's not any taxation authority limiting your ability to exercise your "rights". They're skimming a little bit off the money you were charged to enter into a contractual relationship with the licence-holder. Blame the people who create the ridiculous licencing regimes for software they sell.

  11. Chairman of the Bored Silver badge

    Defense in depth

    My paranoia and age are growing in tandem. When money is on the line I try to establish four defenses: 1) design freeze for a demo milestone. The demo environment is walled off from both the development and production servers 2) test/demonstration readiness review meeting during which we look back over the numerous ways we've screwed up in the past and discuss what specifically the new design does to avoid a repeat. 3) prepare a video in case all hell breaks loose on the live demo, install and test video on the demo-specific laptop. 4) Full dress rehearsal of the demo system in front of a panel of abusive curmudgeons from unrelated projects.

    Once the demo environment is up, one keeps it away from the developers. At all costs.

    Still screw up sometimes but at least the screwups are now fairly unique.

  12. Maelstorm

    Back when I still worked for the local phone company, the machine had something called a secure feature upgrade. You had to enter in a code that represented a feature and a screen appeared that was filled with a bunch of hex digits. Then you have to read off that hex to someone at the vendor who then read back a response code that you had to input. If all was well, the feature would activate. The ability to block your caller ID for one call was an upgrade that cost USD $30,000 for each switch in the network...we had almost 1,000 switches in the state alone. And this was in 1995.

    1. Antron Argaiv Silver badge
      Happy

      The ability to block your caller ID for one call was an upgrade that cost USD $30,000 for each switch in the network...

      ...and now, thanks to The Internet, we have caller ID spoofing and video conferencing (not necessarily at the same time or by the same people) for FREE!

      Isn't Technology wonderful?

      1. Anonymous Coward
        Anonymous Coward

        "not necessarily at the same time or by the same people"

        And I, for one, am glad. Having "Visa-Mastercard" videocalling me to try to get me to give them my credit card number would be even worse.

      2. Lilolefrostback

        About that time, I was working for a major telephony equipment manufacturer. Yeah, it sound like a lot of money, especially when multiplied by the number of installs. However, it's very likely that the features paid for themselves within a month of installation.

        A co-worker and I spent 6 months, so one-geek-year, working on a feature for 800 service. We charged the telco $100K per switch for that feature. We were talking to the telco rep a few months later and he mentioned that, on most switches, the feature paid for itself in a week. Needless to say, we were not encouraged.

        Before you stomp on the equipment manufacturers, consider how much the telcos make.

    2. Anonymous Coward
      Anonymous Coward

      "The ability to block your caller ID for one call was an upgrade that cost USD $30,000"

      Well, it shouldn't have changed much since there are companies that still tries to sell you the caller ID service for a few euro a month....

  13. Anonymous Coward
    Anonymous Coward

    Canonical example of failed demo:

    Bill Gates demonstrating Windows 98. "That must be why we're not shipping Windows 98 yet." "Absolutely. Absolutely."

    https://www.youtube.com/watch?v=IW7Rqwwth84

    1. bombastic bob Silver badge
      Devil

      Re: Canonical example of failed demo:

      I'm pretty sure I was in the audience for that one. San Diego 1997 PDC?

      plug & play, multimedia, web, ActiveX, Windows '98, and NT 4.0 - except for ActiveX, all good

  14. Doctor Syntax Silver badge

    I can't help feeling that the solution in this case would be the same as last weeks expired certificate article. Reset the system date to be inside the period, boot and then set the date to current.

    1. Anonymous Coward
      Anonymous Coward

      "Reset the system date to be inside the period"

      A particularly bright bit of software will, once it detects it's out of date, record that it's expired and refuse to work, even once the system date is "ok" - it would assume (usually correctly) that the user is attempting to bypass the expiration restriction. A REALLY bright bit of software will compare the current system date to the stored date - if the current date is earlier than the stored one, the user is doing something hinky. Otherwise, store the current date for next time.

      1. Brenda McViking
        FAIL

        And how do you then account for me, the idiotic luser, setting the system date 20 years into the future and then back to the current time? Not exactly your business as to why I've done that, but because I have, your software is now broken and you're in breach of contract.

        And that's why you never make assumptions about your paying customers trying to "pirate" your software - you can start worrying about shit like that once you've cleared all your zero days, you have a 6 figure bug bounty which hasn't paid out in the last quarter and the last luser support ticket was dated 6 months ago.

        1. Killfalcon Silver badge

          "And how do you then account for me, the idiotic luser, setting the system date 20 years into the future and then back to the current time? Not exactly your business as to why I've done that, but because I have, your software is now broken and you're in breach of contract."

          That's not how 'breach of contract' works, though. Contract law is largely focused on not taking up any court time if at all possible,, and strongly encourages the parties to take reasonable steps to put things right.

          In this case, where you've done something and the software has stopped working - you contact the vendor, and they should then fix it for you. This is pretty much the same story as home licenses tied to a limited number of activations - they'll almost always reset that count if you just call them up.

          It's not "breach of contract" if you're not attempting to take up the remedies already in place.

          1. Brenda McViking
            Black Helicopters

            Sorry, if I sign a licence contract stating that use of software is licenced in perpetuity, and then I get an error stating licence expired, due to guess what - the licence being time limited when I purchased one that wasn't allowed to be time limited, then they're in breach of contract. End of. This is exactly how breach of contract works - it's written in a contract what is expected and how things behave. When they don't behave exactly as written, it's a breach.

            Sueing over it and wasting court time without seeking appropriate remedies - the thing that you mention is "strongly encouraged" (a shade of grey, this is black versus white), that's another matter entirely. This situation happened to me, though we didn't bother to release the lawyers on this occasion, we merely didn't order £7-figures worth of licencing from that vendor after our handful of testing licences failed to conform to requirements. The vendor would not have been able to fix it, for reasons I won't go in to. See icon for details.

            1. Killfalcon Silver badge

              Agreed - perpetual licences are a distinct case, and they sure as hell should not expire!

      2. Doctor Syntax Silver badge

        A particularly bright bit of software will, once it detects it's out of date, record that it's expired and refuse to work, even once the system date is "ok"

        In this case and last weeks the S/W in question wasn't at all bright enough to do that. It checked the date at start-up only.

  15. Adrian Jones

    In my first job, in the mid 90s we were told there was going to be a photoshoot to for a brochure demonstrating all that the company had to offer.

    This included someone using our software. Not the tried and tested DOS version but the whizzy new Windows version. Except there was barely an object model built, let alone a user interface.

    So we drew one. In Paint. And displayed it full screen.

    We must have done a reasonable job, because the boss was quite surprised when he tried clicking on it. And it made it into the brochure.

  16. IT's getting kinda boring

    'Rattle' through a demo...

    I remember the US office shipping a server to the UK in order to demo a new product. Had a nice shiny shipping case. That turned up dented.

    Their machine rattled when I took it out to set it up.

    After re-seating a couple of cards that had been flying around the case, attempting to boot it up resulted in disk errors - until I led it on its side.

    How it made it through their demo I don't know.

  17. IT's getting kinda boring

    When a demo isn't a demo....

    How about the time the network team tried to demo auto link fail-over. Only someone was already routing live traffic over it, and the failover 'demo' failed to 'failover'.

    Queue lots of shouting as the production service fell over.

  18. Smoking Man

    No trial licenses necessary.

    At least for those products I used to work with for years and decades.

    Name of the company developing and selling those products: Oracle.

    You could (and can) basically install and use all of their products (at least around the RDBMS thingy), with all named features.

    No need to apply a license key.

    And still they're making _some_ money with that type of business, don't they?

  19. Jason Hindle

    A principle we seem to have lost recently...

    It used to be that whenever a software team handed over a build, the test team would first check all the components survived a simple reboot. Amazing what we used to find.

  20. Anonymous Coward
    Anonymous Coward

    They should have use MSDOS

    My company for years sold complex data collection system that that ran on MSDOS but presented a GUI with no indication that the underlying operating system was MSDOS - they never bought a single MSDOS license for resale, just copied the boot disk and sold the kit. Never had any problems.

  21. Antonio Malcolm

    Well it can help if you're running linux servers. If there's a sys bug you can get more help in the newsgroups. But for custom apps, well like the article was saying, it can get very rough.

    Antonio Malcolm

POST COMMENT House rules

Not a member of The Register? Create a new account here.

  • Enter your comment

  • Add an icon

Anonymous cowards cannot choose their icon

Biting the hand that feeds IT © 1998–2019