back to article Config file wipe blunder caused deadly Airbus A400M crash – claim

A dodgy software installation that deleted vital files caused last month's Airbus 400M transport plane crash in which four people died, it is claimed. On May 9, a test flight of the A400M, intended to replace the aging Hercules as a mainstay of NATO's air mobility fleet, crashed in Spain, killing four of the six crew. …

Page:

        1. Alan Edwards

          Re: The investigation should center on...

          " I'm don't recall a single simple hardware failure leading to a crash of an Airbus plane"

          Kind of a combination of both, but a frozen up attitude vane (measures the angle of the plane relative to the airflow) led to the anti-stall system not noticing the plane had stalled and doing nothing about it.

          Daft thing is they were testing the anti-stall and had deliberately stalled it. I can't remember why they couldn't recover it - possibly just ran out of altitude trying to make the anti-stall do it's thing.

    1. Anonymous Coward
      Anonymous Coward

      Re: The investigation should center on...

      "THAT was a pretty stupid idea."

      Stop blaming. Some poor f*cker is enduring sleepless nights because they unintentionally did something wrong, the most important thing now is to find out what went wrong, learn from it, and prevent recurrence.

      Maybe you've never made a mistake, in which case lucky you, because I've made plenty.

      1. Johan Bastiaansen

        Re: The investigation should center on...

        I agree, some poor f*cker will be blamed.

        Accidents like this are usually the result of a chain reaction of mistakes. The biggest mistake was to design an unstable system, that could not be overridden manually.

        And it's very unlikely that the people who made that decision, will be held responsible. It is much more likely that some of them will be sitting in the evaluation committee.

        1. Anonymous Coward
          Anonymous Coward

          Re: The investigation should center on...

          Accidents like this are usually the result of a chain reaction of mistakes.

          Heh.

          - The plane builder have some pressure on the delays, push the pressure down to its suppliers.

          - The suppliers write as fast as possible the system specifications, which include oh so precise sentences like "The software shall be robust", and hire IT consulting subcontractors to do the dirty work (from software specification to software validation). As those subcontractors are paid by the hour, the timings are reduced again.

          - As the slave-driversIT consulting firms needs to make a profit, they assign on this project 10 newbies out of school for one project manager experienced on the domain (he took a plane twice).

          - Strangely, the supplier is not satisfied by the outputs from the consulting subcontractor and require more and more rework.

          - As the deadline is getting closer, the review procedures are getting sloppier.

          - A SW is out for testing, and due to the length of the previous phase the V&V team have half the time they would need to properly review and test the SW.

          - BONUS : If the V&V team belong to the same consulting company that made the specification/design/coding part, then it is asked to be more lenient by the bosses - hey, they need to make a profit!

          - The supplier, under strong pressure from the plane builder, accept this mostly-working software, quickly perform some half assed HW/SW integration tests in the best cases and send the whole package to production.

          You guessed it, this message is from a disgruntled senior critical embedded SW V&V engineer...

          1. Johan Bastiaansen

            Re: The investigation should center on...

            And that's exactly why we shouldn't trust a system so complex and we should always give a highly trained pilot the last word.

            1. Anonymous Coward
              Anonymous Coward

              Re: The investigation should center on...

              "And that's exactly why we shouldn't trust a system so complex and we should always give a highly trained pilot the last word."

              This would be fine if so many military aircraft were not dynamically unstable. Until human pilots have the same ability to process avionic data as birds - many of which are also unstable in flight - the human pilot simply cannot have the last word, or it will be very literally the last word.

      2. icesenshi

        Re: The investigation should center on...

        Yes, let's not lay blame when someone dies or something goes wrong. Just say 'oops it happens', right?

      3. Anonymous Coward
        Anonymous Coward

        @Ledswinger - Re: The investigation should center on...

        Well I guess that poor developer (a synonym for f*cker here) will also have to endure a trial in front of a judge because if I remember correctly, French authorities will treat this as a criminal investigation for manslaughter.

        It is about time that programmers get off their high horses and stop being so smug. You can't solve anything in this universe with algorithms and a few lines of code.

        Before someone tries to down-vote me, I should respectfully suggest he should do some research on criminal negligence. Dura lex...

        1. Vincent Ballard
          FAIL

          Re: @Ledswinger - The investigation should center on...

          What do French authorities have to do with it?

        2. This post has been deleted by its author

        3. Jim99

          Re: @Ledswinger - The investigation should center on...

          Actually, the aviation industry has a pretty mature attitude to this: the priority is learning lessons, not assiging blame. The Civil Aviation Authority's "Mandatory Occurrence Reporting Scheme" is a document many industries (e.g. banks, food manufacturers) could learn from.

  1. Anonymous Coward
    Anonymous Coward

    The A400M isn't meant to replace the "aging" hercules. It sits at a point about half way between the herc and the C17 Globemaster, and is meant to fulfil a different role to both.

    1. Anonymous Coward
      Anonymous Coward

      "It sits at a point about half way between the herc and the C17 Globemaster, and is meant to fulfil a different role to both."

      Which is unfortunate given that it is vastly more expensive than both. Another shit-headed, misbegotten euro-project.

      1. Yag

        Which is unfortunate given that it is vastly more expensive than both. Another shit-headed, misbegotten euro-project.

        Apple and oranges...

        Both C130 and C17 are old planes, their developpement costs have been repaid.

        Furthermore, try to land a C17 on the kind of very rugged terrain the Hercs and A400M can handle, hilarity will ensue. Sure, it can handle short and unpaved runways, but not the random pasture the other can use.

        Comparision between the A400M and the C130 are more accurate, but if you look closely at the characteristics...

        The price is 150M for the A400M, 100M$ for the C130J, but the A400M can carry up to 37t (20t for the C130J). As you need a single A400M instead of two C130J, this seems a far better deal. The A400M is also a bit faster with a similar range...

        So no, it is not that clear cut as you seems to think.

      2. Anonymous Coward
        Anonymous Coward

        To some extent yes, but there is also the desire to keep some good engineering people in employment and maintain the ability to build this sort of thing again when needed in the not too distant future.

    2. x 7

      "and is meant to fulfil a different role to both."

      Which is why its being purchased to replace the RAFs fleet of Herks

      1. Richard 81

        God bless the MOD.

  2. Anonymous Coward
    Anonymous Coward

    Assume human stupidity with computers...

    ... and plan accordingly

  3. Mystic Megabyte

    on error

    I know nothing about these kind of systems but even cars have a "get you home" mode when a sensor fails. Surely that would only be a couple of extra lines of code.

    1. David Roberts

      Re: on error

      Motor vehicles have a "limp mode" for problems where it is safe to run the engine in a reduced power mode.

      There are some sensors which are judged to be critical to safe operation, and loss of data can result in an immediate engine shut down.

      Having had an engine shut down because of a wiring fault between sensors and ECU I am painfully aware of this.

  4. Wanting more

    I'm glad the code I work on will never kill anyone. At worse it'll lose a bit of money / reputation for the company, but that's it. Can't be nice for the guys who worked on this.

    1. asdf

      >At worse it'll lose a bit of money / reputation for the company

      Those knights guys whose screw up with new trading software cost their company over a million dollars a minute (total tab over 400 million) probably didn't feel very good for a few weeks after that. Better than killing someone I suppose but that was probably not the opinion of the owners of the company at the time.

    2. Roland6 Silver badge

      Re: I'm glad the code I work on will never kill anyone.

      I'm glad that the fail safe systems I had designed and coded back in the 80's were replaced at the end of their life in the late 90's having not caused any one's death.

      As for the systems I worked on subsequently, they were (in comparison) straight-forward commercial stuff.

      1. werdsmith Silver badge

        Re: I'm glad the code I work on will never kill anyone.

        It wouldn't be just the coder. There are testers etc.

  5. This post has been deleted by its author

  6. I. Aproveofitspendingonspecificprojects

    magnitude 7 or larger earthquakes tend to occur the same week as a large aircraft crash. I'm not sure how to make use of this information in a fail#safe check,

    But it is a fact.

    1. Red Bren

      > magnitude 7 or larger earthquakes tend to occur the same week as a large aircraft crash

      It's God punishing us for our lack of air worthiness

    2. petur

      http://xkcd.com/552/

  7. razorfishsl

    You would think that the first thing any critical system would do is validate any config files with a checksum, BEFORE allowing the software to continue booting....

  8. x 7

    the surprising thing is that theres not a safe-default fall-back setting

  9. Paul Renault

    Shouldn't such a life-important software..

    ..check to make sure the config files are 1) there and 2) OK?

    1. Mark 65

      Re: Shouldn't such a life-important software..

      That would be a change to the initial spec and cause the project to greatly overrun in terms of time and cost. /sarc

  10. lsces

    Seems to be the norm today ... assuming impossible things can't happen.

    How much would it have cost on top of £13 million to have a buffer on the back of the roller coaster set? I'm sure a lot more examples can be found in this current 'cost cutting' marketplace?

    1. Anonymous Coward
      Anonymous Coward

      @Isces - Re: Seems to be the norm today ... assuming impossible things can't happen.

      Are you thinking what I'm thinking ? Like self-driving cars or autonomous killer drones ?

  11. Forty Two

    Sad and Avoidable.

    First, my sympathy and condolences to the families of those lost. Very sad.

    I agree with several above, this should not have happened or happened on a simulator.

    I call them dependencies. Every function, subroutine, class, whatever I write checks that anything needed external to the code block is both available and as much as possible valid. Input parameters/dependencies are checked for existence and sanity. Only then is processing allow to continue on to the primary function of the code block. Output/results, as much as possible, are checked for sanity as well. Much of this "overhead" can be kept in its own routine and knowing what is needed to be checked should already be in the initial design documentation before you write the first line. Perhaps too much bloat and time for systems that need to be super responsive but with todays CPU speeds, multiple cores, huge amounts of memory available dirt cheap I cannot imagine something managed by humans not having the spare 100mS needed to make sure of avoiding, wait for it old-timers.... Garbage in, Garbage out.

    But I don't write code for airplanes. However I have often told a project manager they will get the code when it is ready and not before. This wins me few promotions or friends in management.

  12. DryBones
    Flame

    There's the problem...

    "According to safety documentation, the pilots would only get a warning from the ECUs when the aircraft is 400 feet (120 metres) off the ground."

    Why does this not show up while it's still on the bloody tarmac?

    1. Anonymous Coward
      Anonymous Coward

      Re: There's the problem...

      Warnings are often inhibited during take-off to avoid the crew being expected to take action when they should be getting airborne and ensuring that they stay above V2, minimum safety speed. Once this is achieved they can they deal with the emergency/warnings and decide on their next course of action.

      In this case the initial problem was that the 3 engines would not respond to a reduction in power demand, in the process of trying to arrest the rapid ascent/acceleration flight idle was selected and this then allowed the protection mechanism to engage and refuse to provide more power when commanded. By the time it became clear that the engines couldn't deliver more power there was not enough time to shut down and feather the 3 that were broken while having enough altitude and manoeuvre capability to reach the runway. The power lines were not likely to be easily visible from the air, once they committed to a forced landing any attempt to miss them would probably have resulted in an even heavier arrival and more structural damage.

      1. Anonymous Coward
        Anonymous Coward

        Re: "the 3 engines would not respond to a reduction... (etc)"

        Is this definitive? Do you have a quotable source?

  13. Anonymous South African Coward Bronze badge

    No fallback fault-tolerant option available?

    Wut?

  14. Anonymous Coward
    Anonymous Coward

    Airbus' track record on what happens when someone reports a technical risk

    Since hearing the story of Airbus engineer Joe Mangan, I've always been relieved when finding I'm flying on some other brand of aircraft (although of course it might just be that they cover things up even better).

    So if someone had spotted the fault, would they have dared report it?

    1. anothercynic Silver badge

      Re: Airbus' track record on what happens when someone reports a technical risk

      a) Mangan was not an Airbus engineer. He was an engineer for an Airbus supplier.

      b) Whistleblowers are often threatened with jail. It's not specific to either Airbus or his direct employer.

      c) Boeing has done the same with engineers at their Washington plants who blew the whistle, so by that standard, flying Boeing won't be safe either.

      d) Same sh**, different company. Airliners are still the safest mode of transport per passenger transported.

  15. Tanuki
    Devil

    Windows has detected new hardware.

    Do you wish to configure "Airbus A400M" now [Y/N]?

  16. Chris Evans

    Who makes the engine?

    On reading the article my first thought was Who makes the engine?

    A. Europrop International a consortium "Europe's four leading engine manufacturers: Industria de Turbo Propulsores (ITP), MTU Aero Engines, Rolls-Royce and Snecma Moteurs, the four partners of EPI."

    So I wonder if a 'Committee' was involved or a translation problem!

  17. Dave 42

    Funny how Software – being “invisible” isn’t treated in the same way as stuff we can see – even by SW engineers. The config files are as critical to the aircraft as are the propellers. We can see the propellers “Oh, 3 props are missing, maybe we shouldn’t try a take off”. Are the config files there? Who knows? Let’s just go for it!

    It’s the duty of the SW designers to make the critical items visible to the pilots – are the files there?, are they the ones you expect? – OK, next item on pre-flight checklist.

  18. rpark

    ...I think their new number one priority is solvency.

  19. Anonymous Coward
    Anonymous Coward

    FADEC

    I'm surprised no-one has mentioned so far that this is the TP400 engine's second set of FADEC software. The A400M was intended and contracted to be certificated to civil standards, not just military, and for that to happen the software writing process had to be fully documented to EASA standards. The prototype A400M was getting close to initial testing when it dawned on someone that the TP400's FADEC hadn't been documented in this way, and was therefore uncertifiable. It had to be written again from scratch, following EASA procedures. That is probably the main reason the project is so far behind schedule. Yes, the A400M is 'overweight' and can't fulfil some of its initial contracted performance but this is not the show-stopper the FADEC debacle was.

    It's funny Wikipedia makes no mention of this. There were plenty of reports about it in the pro aviation press at the time.

  20. Dylan Fahey

    The computer games I play daily, have more safeguards than that jet. My game won't start without configuration files, and CRC checks on files.

    Maybe EA should take over development (/joke).

Page:

POST COMMENT House rules

Not a member of The Register? Create a new account here.

  • Enter your comment

  • Add an icon

Anonymous cowards cannot choose their icon