If it aint a Boeing I'm not going
Investigators probing last Thursday's Heathrow Boeing 777 crash may be able to glean useful information from six previous engine failures on the type, one of which could prove highly significant in pinpointing the cause of the incident. The Air Accidents Investigation Branch (AAIB) has apparently ruled out bird strike and fuel …
If it aint a Boeing I'm not going
Over 25-odd years in the embedded industry, I've seen/heard this before in the context of other planes, car braking systems etc. This, apart from being a tight fist, is why I still drive a 1984 Toyota which has almost no electronics (and none of it programmable or critical).
The silly part of this is that no development teams are any better/worse than the others. By dodging the devil you know, you just end up flying with the devil you don't know.
The people developing embedded systems typically have no special training on designing robust systems. Most of them are either electrical engineers (with no clue about software architecture) or recycled desktop programmers and come from the "all software has bugs"/"did you try a reboot" mindset
As embedded electronics complexity has increased, it becomes ever more difficult to verify a whole system. The larger microcontrollers of today will run more code (more code == harder to verify) and are easier to write code for (easier coding == sloppier coding). Even if individual subcomponents work to specification, they don't always work reliably as a system. More complex systems mean more corner cases with less chance of finding all failure modes.
In this picture, the throttle levers DO NOT retain full command authority in the traditional sense.
As John Freas points out, and as others have already mentioned, on any plane with a FADEC (which is pretty much any modern airliner or military aircraft, among others), the "full authority" (FA) is the "digital engine control" (DEC). Hence FADEC. The throttle lever angle is one of a number of inputs to the FADEC computer, and the computer has full control over the fuel flow rate. The aircraft has no reversionary mode which cuts the computer out of the loop. Turn off the computer's power, and lose control. Fortunately they're not your ordinary computer, and, whether you knew it or not, whether you like it or not, they've been in control for at least ten years now. And not a virus or a Windows Update to be seen.
Just to add what I know about the systems:
The AIMS and FADECs are only linked at the throttle levers, AIMS drives servos and resolvers determine the angle selected for the FADEC thrust setting.
The FADECs on the B777, R-R engines are NOT programmed in ADA - most of the rest of the plane is, true, but not those particular items.
The teams that developed the software for the FADEC are VERY experienced professionals who have had safety-critical training. Not that we can't make mistakes just the same, just not very likely.
(anyone spot the 'we' there - oh, what a give away!!!)
I am not second guessing the AAIB, I will leave that to the PPRUNE massive, who seem to have attracted even more speculators than a gold-rush. And, I do post there too, trying to keep the arguements to the facts, though it's an uphill struggle.
at which point, pass my anorak and show me the door
Is this plane ever likely to flight again?
Whatever about all of the fear about fly-by-wire systems, I think they've proven themselves to be reasonably safe. How the carbon-fibre fuselage of the Boeing Dreamliner will cope with regualr freezing/thawing is what my major fear will be once these planes go into service.
Don't we regularly see chunks of the existing carbon fibre tails break off?
Well it is some years since I wrote avionics software. And that was A310s, which were pre-Ada. And I did fly in them, too - trusting, over-confident, arrogant, or what? And it's also some years since I wrote Ada. But nothing in the AAIB's update suggests an Autothrottle problem, so the Reg's headline is rubbish: you should let Lewis Page do these bits. Software in FADECs is still in the frame. But it looks more like a Mysterious External Influence on the electronic signals sent to the engines.
Notice that AAIB reports a five-second delay between one engine spooling down and the other following suit. So which went down first, Port or Starboard? and which road route was Gordon Brown's motorcade following? You see how this works: aircraft is on finals from east; motorcade crosses under flightpath at an angle; malign electromagnetic influence reaches first one engine and a few seconds later the other... I think a bit of sketch-work with Google Maps and a stopwatch is called for here...
But then, I failed to persuade BAE against Windows-for-Warships (TM), didn't I, so what would an old avionics programmer know, anyhow.
Isn't that what these discussions are here for?
Mine's the ski jacket... yes, on Sunday... well, ski charters actually perk up and passengers grin when there's a bit of unscheduled excitement.
FADECs have been around a lot longer than fly-by-wire airliners. They manage engines efficiently, saving fuel (=money) and prolonging life (=money). An engine control computer local to the nacelle is overall more reliable than a complex monitoring and control system linking cockpit and engine.
A further FADEC benefit is that they irritate control freaks. The FA (Full Authority) part of the acronym refers to the computer having authority to shut down the engine in case of emergency without asking the crew first. If there's a manual over-ride, it's not a FADEC. Think about that the next time you're on a 767, my FBW-hating friends :-)
Like world+dog I have my pet theory as to what went wrong, but until we get the final AAIB report I'm going back to "who shot JFK?".
"I didn't know the throttles on a fly-by-wire aircraft weren't motorised, like the faders on a high-end mixing desk. I guess there's no real reason for them to be."
I would have thought that there is every reason to motorise them. If a pilot does have to take manual control and the levers had been left in some random position then either the switch would involve a rapid change in engine output (or at least a rapid change in the instructions to the engine - jet engines don't change power output as rapidly as a car engine). I suppose a pilot could estimate the right new position or it would also be possible to motorise the throttles so they move to the right place only when you switch to manual, but what's the point apart from maybe reducing a bit of wear-and-tear? Much better to motorise the throttle levers and that also has the great advantage of giving an immediate indication of the autothrottle working. It also means that a manual override be affected simply by grabbing the throttles and overriding a clutch mechanism (or some such equivalent). In reality I would have thought the whole throttle assembly and its ergonomics would be very carefully thought out to allow for rapid manual override.
While the difference in windspeed at altitude and at ground level may be important that's not the main reason why aircraft need increased power before landing.
At slow speeds at landing configuration airplanes operate behind power curve - the slower they fly the more power is needed. This is because to generate enough lift at slow speed you must increase the angle of attack and high angle of attack generates high induced drag. You need more thrust to overcome the increasing drag.
They're not? So what are they programmed in then, given that it's presumably something acceptable to Rolls, Boeing, FAA and CAA and airlines and ... for safety critical software (which probably rules out Visual Basic and a few others)? Inquiring minds may want to know.
Or are you drawing a distinction between "designed in" and "programmed in"? Some folks might think it a bit devious to say a system wasn't "programmed in" Ada if it actually needed an Ada compiler as part of the process to get from code to PROM (I have no idea if that is the case here).
Pointers to definitive papers etc very welcome.
Don't forget not to cross the picket line on Monday:
The FADECs on the B777 are the last generation to use a modular assembler language developed specially for engine systems by Lucas Aerospace. It has no assocation with ADA, no need for any ADA complier, it has its own unique interpreter. I am not sure that there are definitive papers on this but I will try to find something that may satisfy.
The design was done using Yourdon methodology with Teamwork and a lot of sweat and tears.
The picket line doesn't apply to me but I will get a Lucas pension in 2012 - should be 67p a year!
Don't know why everyone is so up in arms about FBW control systems - Concorde was the first commercial airliner to have full fly by wire and including electronic engine control (though not a glass cockpit), and she would have retired after 34 years with a perfect safety record had it not been for FOD on that fateful Paris runway.
From a pilot's perspective, when you are behind the power curve as Vladimir pointed out; you are controlling altitude with power and airspeed with pitch. You're descending with one power setting and want to level out (even with the ground only a few feet above)? Increase power. For the same reason, a touch of power right before touchdown makes you touch down more softly. They don't usually do this in the airliners because the pilots are beyond professionals and can usually round out at the right height that simple deceleration and a good flare will you get you down softly, but it can happen; especially in smaller and private planes. Flying non-pilots always seem to forget that the pilots in front have literally thousands of hours of experience and recieve special training for this specific make and model of aircraft. They know what to do, what they can do, what the plane can do, and how to execute a safe flight. The safety and professionalism in aviation (training of personnel, planning, design, maintenance, etc.) make it second in quality, technology and safety only to space flight. As with anything else humans do, mistakes can happen, but it drives me insane when people complain about aviation safety and pilot errors. You spend thousands of dollars and invest years of your life studying and practicing, perfecting your skills and then we'll see what you have to say. And thats all I am going to say.
Now, where did I leave my coat....?
James and Vladimir - Thanks for your reminders of basic aerodynamics. I never seem to be able to get everything into any of my posts, so I apologize!
Given that, I don't think I will back off my contention that jet jockeys never like it when their fans drop below thrust rpm's. Flameouts occur at low rpm's, regardless of airspeed, wind shear/inertia complications, computers, what-have-you, and are even more likely in wind shear conditions. Being able to totally trust the computers to spool up the engines after approach zero-thrust rpm, in my reading and hangar-flying experience, is rare. Then, after beginning to spool up, to inexplicably drop back to zero thrust... well, another couple of sceptics are born, if they live long enough.
I'm not saying that lack of pilot over-ride caused the 777 engines to abort throttle-up inpupt instructions and flame out or drop back to zero thrust - I am saying that all pilots, fly-by-wire or with total computer control of engine speed, land with their hand on the throttle levers. We all learned that in basic flight training and it is so ingrained in us we probably would still keep our hand on the throttle even after the wings fall off, jet turbine or recip prop.
I continue to believe all aircraft should have pilot over-ride (computer by-pass) in their basic control systems, no matter how wonderful computers control the aircraft. 99.9% ain't good enough. In this forum several have played with the idea of external electronic incursion into the on-board computer system. If this is even only remotely possible, which I tend to see as VERY remote, a manual over-ride system seems desireable and even dictated.
Where did I ever imply these airline pilots are not qualified? Anon Coward seems to think I'm attacking pilots, or airliners, or programmers, or God knows who else!
This is an opinion forum, and, being a pilot, I think I have as much a right to an opinion as most of the other intelligent posters on this thread.
Of course little tin SEL's are not the same as turbine airliners; of course low-time (relative to the airline jet jockeys) white-knucklers cannot even approach the skills and training of jet pilots; of course guys who program and plan the computer control systems of complicated aircraft work long and hard anticipating every eventuality; of course... well, enough.
I'd still feel better back in my passenger seat if the guys up front could over-ride computer control systems if they had to.
And that's all I'm going to say.
Now, where did I leave my sunglasses? :^)
There have been several comments thus far citing the presence of Mysterious Electromagnetic Interference Coming From an Indistinct Black Box of Unknown Origin and Attached to a Car, and as one of the maligned ground crew (avionics), it really makes me smile.
Most of these systems are fairly well shielded, as is the wiring that is associated with them. The idea that some random radio or another causing these things to fail (and also causing the redundant systems to go as well) is something that I don't think is within the realm of that which is reasonable.
Something that ought to be established here is that there are tests that these systems go through to ensure they will stand up to electromagnetic interference (EMI) far beyond what they would see in a lifetime of normal operating conditions. I've been working with aircraft electronics, avionics, and related systems for about 15 years now and have seen one EMI induced failure.
That specific incident was caused when the aircraft I was working with was hit by the beam from a U.S. Navy phased-array radar at approximately 0.5 miles from the antenna. It ought to be understood that this particular radar isn't something that you could easily vehicle mount, as the antennas are designed for shipboard use and have output power in excess of 4 MW. (I don't care how big your alternator is, or what kind of capacitors you've got wired up to the stereo, you aren't lugging one of these around under your jacket.)
Furthermore, the specific failure was limited to a piece of equipment that dealt with the reception of RF at that particular band. (Melted receiver module, lots of smoke, otherwise harmless.) What I would like the tinfoil hat set to note here is that the FBW and automatic flight control systems in the plane worked fine before, during, and after the event. What I would also like them to note is that a common thing to do with this particular receiver is to wrap it in silvered barrier paper (read: plasticized aluminum foil) to prevent this sort of thing from happening. (For those of you in the tinfoil hat set that may be a little on the slow side: I just said that wrapping your heads in tinfoil really does work. However, fine copper mesh works much better. You should also wrap it around your entire body, not to forget the bottoms of your feet, because the signals can still get under the hat and into your brains.)
I would be positively amazed to find that the mishap investigation cites EMI-induced total FADEC failure as the sole primary causal factor for the event. Again, the probability of the entire system, as well as the redundancies, failing simultaneously is too high.
Hang on... So if I understand you correctly, it is perfectly safe to use cellphones or other personal electronics at all times in a flight? I don't seem to remember any of these devices being available in the MW range. That means... oh my god... I've been lied to... Quick! Hand me some of that copper mesh stuff...
BTW, love the way that the yanks (I'm guessing) managed to find a way to blame the non-US engines rather than the US plane. It kinda reminds me of the (rather sick imo) relief when the as-it-happens reporters of the Rockaway crash discovered that the aircraft involved was European rather than a Boeing as first thought - they went all pensive again when it was announced that the engines were possibly to blame and they were GE.
The FADECs are under investigation, not fried, still functional, as far as I know.
...LUCOL (or is it Lucol), which as you say "has no assocation with ADA, no need for any ADA complier".
It isn't interpreted though (but I can see where that confusion may arise). Nor is it compiled. It's "translated", by a simple process which is too complicated to explain here.
The language itself is remarkably simple (which is good). The implementation is also simple (which is good). It's designed for speed of execution and simplicity of development and runtime environment (which is good). Ada arguably does not have these characteristics, and arguably that is not good, but Ada was/is trendy and has taken over the safety-critical world whereas Lucol has largely gone the same way as Coral.
Because the origins of Lucol in the early 1980s predate the Interweb (and predate usable embedded-Ada setups), and because Ada had out-trendified Lucol by the time the Web caught on, there are very few web-accessible bits documenting Lucol in any detail. Googling for Lucol and Dolman (surname of man in charge of development team) finds pointers to some documents (books, papers, proceedings, etc) which those in the trade may be able to access, if they're that interested.
Or in your particular case, e.g. if you want to clarify the "not interpreted" thing, there may still be a few people around who were there when the first Lucol stuff flew. They may not be around much longer, hopefully their Lucas pensions will amount to more than 67p.
I can't remember where I heard it, but (apocryphal, I'm sure)
Passenger to pilot: What happens if one engine fails?
Pilot to passenger: The other engine takes you straight to the scene of the accident...
</coat> (not even bothering to take it off....)
Quite right LUCOL it is, and it is translated not interpreted and its overhead in timing and memory is far lower than compiled languages. The typical box it runs in is a hansom cab compared to a PC being an F1 car. And ADA (I can do that too) is complicated, was trendy and could have been excellent, but only a sub-set is really safe enouigh to fly with (IMHO, of course).
Personally, I don't need to G**GLE for LUCOL or Dolman, though I did try last night just out of interest, but as AC reports above you tend to get results that point you at restricted sites. Should I want information I can read my own notes (should have done that first, would have got over the interpreted/translated error) or have a beer or two with the guys who really wrote LUCOL that I am still in touch with.
I was around when it first flew, I still am and I intend to be around for quite a while longer thanks, but if Goodrich get their way my 67p might appear over-generous!
"They're not? So what are they programmed in then, given that it's presumably something acceptable to Rolls, Boeing, FAA and CAA and airlines and ... for safety critical software (which probably rules out Visual Basic and a few others)? Inquiring minds may want to know."
ADA *is* often used in aero software but most is still written in 16 or 32-bit assembler.
And no: there is no operating system as such. The hardware *is* the operating system.
"They are VERY firm in the life-jacket demonstration. :)"
Off-Topic I know - but as the world's expert seem to be here - how many safety demonstrations have there been, and how many times have they been actually used successfully?
I've flow many times from London to Scotland, would pilots ever aim at Lake Windermere?
The following story has just emerged about the Boeing 777 with Rolls-Royce engines. It MAY be a total coincidence but its timing is bad news for Rolls-Royce and its operation up in Derby.
Following the Boeing 777 accident at Heathrow last week, the American Federal Aviation Administration (FAA) has just released an Airworthiness Directive (AD) with the following title:-
[This AD has been issued] to prevent internal engine damage due to ice accumulation and shedding, which could cause a shutdown of both engines, and result in a forced landing of the airplane.
This AD becomes effective February 27, 2008.
Note that at this time the UK AAIB (Air Accidents Investigation Branch) has not commented on the finding.
Thanks for the info Robin, others
This thread reminds me of a Yes minister quote;
"There nothing worse than accurate uninformed speculation" (-:
I'm aware that there is lots of software out there still in assembler. I would be surprised (sometimes I am surprised) if much of it is recent and non-trivial, especially if it is safety critical. If you have publishable (especially documented) examples, folk might be interested. (I'm not sure what classes as non-trivial but it could probably be argued that anything a 16bit (64k address) micro can do is relatively trivial)
In other news, Aviation Week is mentioning investigations into possible fuel contamination:
"so for the plane to lose height it would need both engines to fail at the same time"
I suddenly remembered that Darwin Award where some dudes that sounded like Beavis & Butthead took an RJ-200 up to 41,000 feet (the design ceiling for the aircraft) and managed to stall/kill both engines. The black box recordings by themselves are priceless, you'd think someone was transcribing "Beavis & Butthead on a Plane" or something like that. Ironically, they overrode the automatic stall prevention system, and it was their doing that ended up killing them; while the "usual" fly-by-wire incidents are because of the system overriding the pilot and screwing up everything.
Can manufacturers spend a fortune ensuring their billions of cars have EMC (Engine Management Computers) which are not susceptable to electromagnetic interference from any source, right ? Right. So Land Rover Freelanders from the first five years of production frequently had to be towed away when parked near any Italian police station, because the Italian police use radios which locked the Freelander's Immobiliser. The manufacturer screwed up. So unlikely on aircraft systems ?
Biting the hand that feeds IT © 1998–2017