Powerless over London: The crash of British Airways flight 38
On the 17th of January 2008, a British Airways Boeing 777 was on final approach to London Heathrow when the pilots faced the unthinkable: less than a minute from landing, both engines suddenly rolled back and refused to accelerate. With only seconds to act, the pilots worked together to extend their glide just far enough to avoid a series of potentially deadly obstacles, before crashing down in the grass more than 300 meters short of runway 27 Left. Although the landing gear collapsed, the fuel tanks ruptured, and the engines were torn apart, the wide body jet slid to a stop astride the runway threshold, mostly in one piece. To the astonishment of onlookers and the flight crew alike, all 152 passengers and crew swiftly evacuated into the arms of the first responders, essentially unharmed — the most serious injury was a broken leg.
The cause of the crash turned out to be at once simple and incredibly unlikely, a combination of many obscure environmental factors and seemingly insignificant operational conditions that created the circumstances for the accident to occur. British Airways flight 38 was an outlier case among hundreds of thousands of flights, falling so many standard deviations away from the mean as to push into unexplored territory, despite appearing utterly normal. Even so, the mechanism that caused the dual engine rollback was fundamentally random, difficult to reproduce. Would it ever happen again? Could anything be done if it did? Although the risk was eliminated with a marvelously simple fix, the crash also stands as a stark reminder of the dangers of “unknown unknowns,” edge cases at the margins of possibility, waiting for some unlucky soul to stumble into them.
◊◊◊
Since its formation in 1974, British Airways has maintained a world class safety record, with only a single fatal accident — a 1976 mid-air collision over Yugoslavia in which a BA flight was simply in the wrong place at the wrong time. The same is also true of some of the British flag carrier’s small handful of famous non-fatal accidents, including a Boeing 747 that flew through a cloud of volcanic ash in 1982 — and, of course, a routine Boeing 777 flight from Beijing to London in 2008, that went awry at the last possible second.
British Airways flight 38 from Beijing Capital International Airport to London Heathrow on the 17th of January that year was lightly loaded, with 136 passengers and 16 crew aboard the four-class, 236-seat wide body airliner. The Boeing 777 (pronounced “triple seven”) scheduled to operate the flight was an ordinary example of what has become Boeing’s most popular wide body model, with over 1,700 delivered and more rolling off the assembly line every year. Much like British Airways, the 777 has a good safety record, and in fact by January 2008 the type had been in service for 13 years without a single hull loss accident. In keeping with that standard, the incoming crew reviewed the technical documents and observed that the 7-year-old 777, registered G-YMMM, had no outstanding defects.
That crew consisted of three experienced pilots, led by 43-year-old Captain Peter Burkill, who had accumulated 12,700 flying hours over a 20-year career. He was joined by two first officers, 41-year-old John Coward and 35-year-old Conor Magenis, who had about 9,000 and 5,000 hours respectively. The three pilots would rotate through the two flight deck positions over the course of the 10-and-a-half-hour flight, ensuring that each received at least a few hours to rest during the cruise phase. However, all three pilots would be present for the takeoff and landing.
Before the flight, a thorough review of the paperwork supplied by the airline indicated that other than some clouds over London, the only significant weather phenomenon along their route across China, Russia, and Scandinavia would be some unusually low temperatures at their planned cruising altitude. Airline pilots must keep in mind that there is a minimum air temperature in which they can fly before jet fuel begins to freeze, which is obviously undesirable. Like all modern airliners, the 777 is equipped with fuel temperature gauges to help the crew determine when the fuel is getting too cold. Jet A-1 type fuel, which is used in most of the world, normally begins to freeze at around -44˚C, and the 777 will generate a low fuel temperature warning at -41. However, the outside temperature required to cool fuel to -41˚C is considerably lower.
According to the weather reports, a patch of freezing air over Siberia presented a risk at higher flight levels, as a result of which their flight plan called for a cruising altitude of 9,600 meters (31,500 ft) after passing the China-Mongolia border, which is much lower than the 777’s normal cruise altitude. (China and Russia use meters instead of feet for aircraft altitudes.) In general, jets are most efficient when flown at the highest practical altitude, so the crew were aware that they should climb higher than this if temperatures permitted.
At 10:09 a.m. local time (2:09 UTC), flight 38 departed Beijing and climbed uneventfully toward its cruise altitude. As they ascended, the air temperatures remained slightly warmer than expected, and the crew received clearance to climb to 10,600 meters (34,800 feet) instead of the planned 9,600 meters. The pilots regularly monitored the fuel temperatures and observed nothing out of the ordinary.
Several hours passed, and the crewmembers rotated through their scheduled rest breaks, using the opportunity to take naps or chat with passengers. West of the Ural Mountains, the outside temperature fell to a crisp -74˚C, but the fuel temperature remained well above freezing. It’s unknown whether anyone remarked on the miracle of aircraft climate control.
After crossing the Urals, the temperature began to rise again, and while passing north of Moscow the flight was able to climb to 38,000 feet. The pilots elected to climb using the autopilot’s vertical speed mode with a very sedate climb rate in order to avoid a large acceleration that would disturb the passengers. Around this time, the fuel temperature bottomed out at a perfectly comfortable -34˚C, then began to rise. Before long, the temperature had increased enough to allow a further climb to 40,000 feet over Scandinavia.
All continued as normal as the flight drew near its destination shortly after noon local time, beginning its descent with Captain Burkill and senior First Officer Coward at the controls. With the engines at low power, the descent progressed steadily, interrupted only by a small holding pattern between 11,000 and 9,000 feet for air traffic control sequencing purposes. The crew were subsequently cleared to perform an instrument landing system (ILS) approach to runway 27 Left at Heathrow, and the autopilot successfully locked on to the localizer and glideslope, aligning the airplane laterally with the runway and vertically with the descent path. Minutes later, the flight broke through the clouds at 3,000 feet over central London, and their destination hove into view.
As the flight descended through 1,000 feet, the pilots declared the approach to be stable. The aircraft was on course, on speed, and configured for landing with its gear down and the flaps extended to 30 degrees for low speed flight. Air traffic control had just granted landing clearance, the runway was in sight, and it seemed that no obstacles remained.
At 800 feet, Captain Burkill handed over control to First Officer Coward so that he could gain manual landing experience — an important form of practice for even experienced 777 crewmembers, given that long haul pilots might get to make only one or two landings a week, sometimes less.
Acknowledging the handover, Coward began assessing the situation ahead of disconnecting the autopilot. As he did so, the autothrottle automatically commanded an increase in thrust in order to maintain the approach speed of 135 knots through some light turbulence while in the high-drag landing configuration. Four separate commands for higher thrust were issued over the course of about 40 seconds, which was neither abnormal nor unexpected, and none of the crewmembers commented on it.
However, as the fourth acceleration command was issued, the engines initially responded normally, only for the right engine to abruptly spool back down. The autothrottle responded by attempting to increase thrust on the right engine even more, but there was no response. Within seconds, the right engine had rolled back to a power setting somewhere just above flight idle, well below the value required during final approach.
Glancing down, First Officer Coward noticed that the thrust levers had “staggered” — one was positioned farther forward than the other. But before he could identify the reason, the left engine rolled back as well, seven seconds after the right.
Descending through 500 feet, as yet unaware of any problem, Captain Burkill called out “stable,” reaffirming that they were on course and on speed.
But Coward was growing uneasy. “Just,” he said. “What’s happened to the speed? …Yeah but look, what’s going on?” Noticing that engine power was lower than it should be despite the autothrottle’s efforts, he pushed both thrust levers forward, but there was no response from either engine. Suddenly alarmed, he said (according to Burkill’s recollection), “Pete, I can’t get power on the engines; it’s not giving me power.”*
*Note: The Air Accidents Investigation Branch did not publish a cockpit voice recorder transcript of flight 38. Some lines were included in the official report, while other lines come from Burkill’s book, published two years after the accident. However, the wording in the book doesn’t always match the wording in the report; for example, where the report states Coward said “Just,” Burkill recalls, “Well, sort of.” This indicates that Burkill was not working off of an official transcript when he wrote his memoir. Therefore, lines from Burkill’s book, marked with an asterisk, should not be taken as a word-for-word transcription of what was actually said.
There had been no warnings from the aircraft, no fault messages, no caution lights. Everything on the instruments was normal, the plane was on course, the engines were running. There was no indication of any malfunction. And yet, somehow, even with the thrust levers pushed all the way forward, both engines continued to spin merrily along at just above flight idle power. Confused and alarmed, Coward moved the thrust levers rapidly back and forth in an attempt to elicit some response, but nothing happened. It was like the levers weren’t even connected to the engines.
There was no training for this scenario, there was no procedure in the quick reference handbook, there wasn’t even any precedent that the crew was aware of. If both engines had failed, the response would be to initiate the restart procedure, which they had both memorized. But what were they supposed to do if the engines were still running, all indications were normal, and yet power refused to come?
Still struggling to take stock of the situation, Captain Burkill asked, “What? What do you mean?”*
“It looks like we’ve got double engine failure,”* said the relief First Officer Magenis, who was observing from the jump seat.
At this point, they had only about 30 seconds before the airplane would inevitably strike the ground. Up until then, the aircraft had been on course to land in the touchdown zone on runway 27L, but without sufficient forward thrust, it wouldn’t reach that far — they would come up some unknown distance short. It crossed Captain Burkill’s mind that they might, in fact, be done for. Only seconds earlier, all had seemed normal.
Due to the sudden appearance of an emergency, Coward had not yet disconnected the autopilot, which continued to maintain the glideslope as best it could. Without sufficient engine thrust to overcome the drag from the extended flaps and landing gear, their speed was decreasing. Since higher speed means more lift, less speed meant that the aircraft’s desire was to descend faster. In order to avoid this, the autopilot began pitching up, compensating for the loss of lift by increasing the angle of attack. But higher angles of attack induce more drag, because more of the aircraft is facing into the oncoming airflow, causing speed to reduce even further. The autopilot would then have to raise the angle of attack further still, creating a potentially deadly feedback loop. If the crew didn’t intervene within seconds, then at a speed of 104 knots the angle of attack would reach the critical point, and the aircraft would stall and fall from the sky.
As their airspeed fell through 115 knots, an “airspeed low” warning sounded, and Coward began pushing the nose down to prevent the imminent stall. However, if he pushed the nose down too much, then they would descend into the ground too early, causing them to strike obstacles. But if he didn’t push down enough, they would stall and strike the obstacles anyway. Both pilots could clearly see what lay ahead of them: an industrial park, the Hatton Cross tube station, a petrol station, a four-lane highway, a row of trees, the airport perimeter fence, and then finally the instrument landing system antennas, all of which would have to be avoided to reach the open ground of Heathrow Airport. It was not immediately clear whether a path between these two encroaching dangers, their own Scylla and Charybdis, even existed.
Grasping for any solution, Captain Burkill checked the status of the fuel pumps, turned on crossfeed between the fuel tanks, and attempted to put the engine starter switches in “start” mode. Nothing worked. “Airspeed low!”* he called out.
From his perspective it appeared that they weren’t going to make it. The only way to reduce their descent rate was to reduce drag — but how? There wasn’t enough time to retract the landing gear, and they’d need it for touchdown anyway. What about the flaps? Burkill made a quick mental calculation, recalling the tables that had been drilled into his head during training. Changing the flap setting from 30 degrees to 25 degrees would reduce drag, but it would also reduce lift, increasing their stall speed from 104 knots to 106 knots. With mere seconds to act before it would be too late to make any difference, he decided that the reduced drag would outweigh the reduced lift, and he pulled the flaps back to 25.
Moments later, flight 38 cleared the tube station, the petrol station, and the houses, streaking low over the A30 trunk road, the trees, and the airport fence. In the control tower, controllers observed that the 777 was lower than usual with its nose especially high, which the supervisor interpreted as a sign that they were going around. Without waiting for a call from the flight, he sat down to log the missed approach — only for events to go suddenly sideways.
In the final moments, the stick shaker activated, warning of an impending stall, but First Officer Coward kept the nose as high as possible, riding the stick shaker threshold to extract maximum performance from the airplane. At the same time, Captain Burkill flipped the auxiliary power unit starter switch, hoping to provide backup power once they were on the ground, but it immediately became clear that this action was futile. The plane just barely cleared the ILS antennas; First Officer Coward pulled back to flare for touchdown; and at the last second Burkill keyed his mic and said, “Mayday, mayday, Speedbird…” For a brief moment, in the midst of the action, he couldn’t recall their flight number. “…Speedbird nine five, nine five,” he said, inadvertently using the callsign for his next scheduled flight. It didn’t matter — events were already out of his hands.
At 12:42 and 9 seconds, British Airways flight 38 touched down in the muddy grass 330 meters short of runway 27L, still descending rapidly. The landing gear dug into the soft earth, where it was wrenched upward and aft, sending the airplane crashing down onto its belly. Sliding across the grass at 100 knots, the right main gear separated from the aircraft, breaching the right main fuel tank before breaking into two sections. A large portion of the gear flew backward, glanced off the rear cargo bay, and bounced upward into the horizontal stabilizer, tearing a chunk out of the leading edge. Another piece of the landing gear, with a truck beam and two wheels attached, slammed against the side of the passenger cabin, penetrating the wall between seat rows 29 and 30 before flying back in the slipstream. At the same time, the left main gear punched straight up through the wing; the nose gear folded back; and both engines struck a pair of concrete covers placed over ILS wiring inspection pits. The aircraft then slewed hard to the right, lurched up onto the start of the paved runway undershoot area, and finally ground to a halt.
◊◊◊
In the cockpit, the pilots felt an impact so enormous that Burkill assumed the tail section had broken off the plane, presumably taking passengers with it. In the cabin, however, the impact apparently was mild enough that some passengers didn’t even realize they had crashed until they looked outside and saw the damage to the aircraft. The air traffic controllers, who witnessed the accident from start to finish, were perhaps most alarmed — in fact, the crash button was pushed before the plane even hit the ground, and firefighters were on their way within seconds. Controllers didn’t initially realize that the airplane was upright and intact until the dust cleared.
On board, Captain Burkill, with adrenaline pumping, attempted to make an evacuation announcement, but accidentally transmitted it to air traffic control instead. Without the slightest pause, the controller informed him that he was speaking on the air, at which point Burkill switched over to the public address system and barked, “This is the captain — this is an emergency, evacuate, evacuate!”*
Although some flight attendants found the concurrent evacuation alarm difficult to hear, the evacuation got underway smoothly. There was no fire, and all but one of the doors were usable, allowing the 136 passengers to exit in well under the expected 90 seconds. By the time the last passengers were leaving, fire trucks had already arrived and began to spray down the plane to prevent the ignition of leaking fuel. Only later would anyone learn that not only was fuel gushing from the tanks, but that the passenger oxygen supply had been disrupted when the main landing gear deflected off the cargo hold, causing an oxygen leak as well. Incredibly, neither of these accelerants ignited.
◊◊◊
News that a British Airways Boeing 777 had crash-landed at Heathrow spread quickly, as did the equally remarkable news that all 152 passengers and crew had survived. Thirty-four passengers and 12 of the 13 cabin crew reported minor injuries, mostly strained necks and backs, while an Australian passenger who broke his leg when the landing gear penetrated the cabin was the only serious injury.
The combination of the occupants’ miraculous survival, the unexpectedness of a crash at British Airways, and the fact that it was the first loss of a Boeing 777 made the accident an immediate media sensation, not just in Britain, but around the world. However, for many of those involved, the story had only just begun. The fact remained that no one, not even the pilots, could explain why the flight lost power in both engines just seconds from landing. And with hundreds of 777s flying around the world, there was considerable pressure on Britain’s Air Accidents Investigation Branch (AAIB) to find the answer as quickly as possible. In the end, this turned out to be easier said than done.
There are relatively few events that can cause a simultaneous loss of power in both engines on a twin-engine aircraft. This short list includes fuel exhaustion, fuel contamination, bird strike, and ice ingestion, along with a few other more remote possibilities. However, some of these could be ruled out right away. An examination of the engines showed no evidence of a bird strike, and in fact not only was there no pre-impact damage, the engines appeared to have been running normally, albeit at low power, right up until the moment of the crash. Rumors of electromagnetic interference led investigators to check whether powerful signals could induce the spar valves to close, cutting off fuel, but they were unable to produce such an effect. Fuel exhaustion was also seemingly out of the question, as an inspection of the fuel tanks confirmed that there was plenty of fuel on board, and some of it was certainly reaching the engines, since fuel was found there as well. Furthermore, detailed laboratory analysis of fuel recovered from the engines showed no evidence of contamination by water, microbiological material, or any other foreign substance.
The flight data recorder (FDR) readout confirmed and expanded upon these findings, revealing that the autothrottle had commanded an increase in thrust, causing more fuel to be supplied to the engines, before fuel flow dropped off abruptly. During the descent, fuel flow was around 4,000 to 5,000 pounds per hour (pph), which increased to 12,000 pph just before the incident. Then, after four cyclic increases, the actual fuel flow diverged from the commanded fuel flow, dropping to 5,000 pph on the left engine and 6,000 pph on the right engine even as the autothrottle continued advancing the thrust levers. More detailed data from the Quick Access Recorder (QAR), a non-crash-protected maintenance diagnostic recorder, confirmed that both engines’ fuel metering valves, which determine the fuel flow rate, moved to the fully open position in order to supply as much fuel as possible, but the flow rate didn’t change.
This startling discovery led investigators to conclude that a blockage somewhere in the fuel system was the only explanation for the simultaneous rollback. Otherwise, with plenty of fuel in the tanks and the metering valves wide open, fuel flow should have increased and the engines should have accelerated.
In order to find that blockage, they would need to conduct a complete teardown of G-YMMM’s complicated fuel distribution system.
The fuel system on the 777 consists of hundreds of diverse components, but for the purposes of this story, a basic description will suffice.
The 777 has three main fuel tanks: one in each wing, and one in the center fuselage. The wing tanks are considered primary, and normally the center tank is emptied first, leaving the wing tanks in use through the final approach phase. The tanks are equipped with boost pumps to propel fuel into the distribution system via a manifold that connects all three tanks and both engines, with configurable valves that determine which fuel source feeds which engine. When the wing tanks are in use, fuel will flow separately into the left and right portions of the manifold, and thence toward the engines. As the fuel nears each engine, it is further accelerated by a low-pressure (LP) engine-driven pump, which propels the fuel through the fuel-oil heat exchanger, or FOHE. This device allows the cold fuel and hot engine oil to come into contact with a mutual surface, simultaneously helping raise the fuel temperature in advance of combustion and ensuring that the lubricating oil doesn’t overheat. From there, the fuel passes through the LP filter, the high-pressure (HP) engine-driven pump, the fuel metering valve, the flow meter, and the HP filter, before finally reaching the engine.
The flight data clearly indicated that the blockage was upstream of the fuel metering valve, because otherwise the fuel flow could not have been so low with the valve fully open. Furthermore, the teardown of the system revealed an intriguing clue proving that the blockage was also upstream of the HP pump. The HP pumps from both engines on G-YMMM displayed a unique type of damage called “cavitation.” Cavitation damage is observed when the pressure in a fluid-filled system drops below the vapor pressure of the fluid — remember high school physics? — leading to the formation of bubbles. (This is the same phenomenon that causes water to boil in a vacuum without raising the temperature.) When these bubbles reach an area of higher pressure, they collapse violently, creating a shockwave that blasts small pieces off of the surrounding machinery.
If the HP pump was operating at high power to pump as much fuel as possible, but not enough fuel was coming, then the result would be a lower pressure on the inlet side as the pump sucked fuel out of the system faster than it was coming in. In the resulting low-pressure environment, the fuel would have effectively started to boil, much like the example of water in a vacuum, forming the bubbles that caused the observed cavitation damage. Therefore, the blockage must have been upstream of the HP pump.
At the other end of the system, inspections of both main fuel tanks turned up some interesting items left behind when the aircraft was manufactured, including a red plastic scraper in the right tank; a piece of fabric or paper in the center tank; and a piece of black plastic tape, a piece of brown paper, and a piece of yellow plastic in the left tank. Investigators considered the possibility that these objects had somehow become lodged in the boost pumps, preventing them from delivering fuel to the system, but the recorded flight data indicated that the boost pumps were operating normally during the accident sequence. Furthermore, no “low boost pump pressure” faults were recorded. This hypothesis was thus ruled out.
The restriction therefore must have been somewhere between the boost pumps and the HP pumps. One location in this range, the LP filter, was ruled out because the filter had a bypass channel that would open in case it became clogged. That left the LP pumps and the FOHEs as the only remaining locations. However, no physical evidence of any blockage in any of these components, or any other components, was found.
The fact there was nothing physically present, even as all evidence pointed to a blockage during the flight, led investigators to conclude that the only plausible source of the restriction was something that was already gone when they got there — that is to say, ice.
◊◊◊
The flight data made it clear that the fuel itself never froze. The lowest recorded fuel temperature was -34˚C, well above the freezing point of Jet A-1, and at the time of the engine rollback the fuel temperature had increased to a comparatively balmy -22˚C. However, what isn’t immediately obvious is that all jet fuel also contains a little bit of water, usually on the order of about 30 parts per million (ppm). The water concentration in the tested fuel from G-YMMM was 35–40 ppm, which was well within the normal range, and the fuel was therefore considered uncontaminated. But if that water froze into ice, and then somehow all broke free in one fell swoop, then it could conceivably have blocked the fuel lines, even at an original water concentration that was considered harmless. The fact that water in fuel sometimes freezes inside the fuel system was well known, but there was no real precedent for this ice causing any disruption to system operations when water concentrations were normal. So could this really be the answer?
To find out, investigators needed to know the conditions under which ice could form; how much ice was needed to restrict fuel flow, and where; whether this could result in similar indications to the accident flight; and whether the requisite conditions actually existed during flight 38.
Determining the conditions under which ice could form in the fuel system, and how much, promised to be both difficult and expensive. The only way to identify those conditions would be to reproduce them, but performing test flights wouldn’t work because the conditions were too difficult to control. Instead, the AAIB spent enormous amounts of time and money recreating the entire 777 fuel system in a warehouse, with individually climate-controlled sections designed to mimic the operating environment that exists in flight. Various means of injecting water into the system and measuring ice buildup were also added, along with a fully functional fuel-oil heat exchanger equipped with actual hot oil. Wherever possible, the components used were those salvaged from G-YMMM.
Initial results showed that ice in the fuel usually doesn’t form at temperatures above -5˚C, and at temperatures below -20˚C most of the ice crystals pass harmlessly through the system. But between these two temperatures, ice crystals in the fuel can stick to components of the fuel system, and to each other, enabling ice to build up over time. Real world experience showed that ice in the fuel tank itself tended to adhere very tightly to the tank floor, where it was almost impossible to remove, but within the fuel lines ice buildups tended to be soft and easily dislodged.
In order to simulate the sudden dislodging of a large quantity of this soft ice, investigators injected various amounts of water into the cold fuel directly at the boost pumps. This water quickly froze as it traveled down the fuel lines, forming clumps of ice crystals. These crystals proved able to pass through the LP pump, but the same could not be said of the fuel-oil heat exchanger.
The FOHE consisted of a set of around 1,000 small-diameter fuel tubes that passed through a hot oil-filled chamber. This design was intended to maximize the surface area over which the fuel and oil were exposed to one another, ensuring more efficient transfer of heat. At the upstream end, the fuel inlet chamber and the oil chamber shared a common wall, called the end plate. Any ice that touched the end plate would melt due to the hot oil on the other side. However, the openings of the tiny fuel tubes rose about 4 millimeters above the surface of the end plate, as shown in the above diagram.
Investigators observed that when the clumps of ice crystals in the fuel reached the FOHE, they tended to adhere to the openings of the tubes, which were just far enough away from the hot end plate to prevent the crystals from melting. In some tests, these crystals noticeably restricted fuel flow, without restricting it so much as to prevent the engines from operating above flight idle — just like what happened on flight 38. However, it was observed that if fuel flow was reduced below the rate at which it was restricted, then, paradoxically, the blockage would clear. This turned out to be because a lower power setting resulted in less fuel flowing through the FOHE, while the amount of hot oil stayed the same, causing the temperature in the FOHE inlet chamber to increase and melt the ice.
Overall, the results of the test varied widely with a high degree of randomness, but the AAIB was nevertheless able to demonstrate that as little as 25 mL of water, when released all at once in a frozen state, could block the FOHE and restrict fuel flow. Meanwhile, testing on a full engine setup confirmed that a restriction at the FOHE would produce similar readings to those recorded on the accident flight, assuming that the blockage was introduced just before the first of the four autothrottle-commanded accelerations, described previously. The fuel already in the system downstream of the FOHE was sufficient to allow the four accelerations to take place before the effects of the blockage reached the engine, causing a rollback. The fuel metering valve would then open fully in an attempt to supply the commanded fuel flow; the HP pump would cavitate; and engine power would decrease markedly, exactly as occurred on flight 38.
A restriction at the FOHE was also consistent with a decrease in oil pressure that was recorded in the right engine during the rollback. This decreased oil pressure was caused by decreased oil viscosity, which was in turn caused by higher oil temperatures due to the reduced quantity of cold fuel flowing through the exchanger. A similar drop in oil pressure was not recorded for the left engine, however, because this data was only recorded at frequent intervals by the QAR, which was not designed to provide post-crash data and included a 45-second buffer before the data was saved. Because the left engine rolled back less than 45 seconds before the crash, the QAR didn’t capture its rollback or the effects thereof. However, investigators felt they had sufficient grounds to assume that the mechanism behind the rollback of the left engine was the same as that of the right.
Still, another question remained: could the amount of ice required to block the FOHE have actually formed under real operating conditions?
Testing on the full-scale rig showed that when fuel flow through the system was high, ice tended not to stick to the pipes, preventing any sort of buildup. However, when fuel flow was low — as would be expected during cruise and descent — buildups could occur, but only when the fuel temperature was within the sticky range.
On flight 38, the temperature of the fuel recorded by the FDR on takeoff from Beijing was -2˚C. The temperature then steadily fell to -34˚C over the course of about 7 hours. When this rate of temperature change was simulated on the test rig, ice built up in the fuel lines during the first three hours when the temperature was within the sticky range, and although ice stopped building up once the temperature fell much below -20, the ice that had previously built up remained in place. When the tests were stopped, this ice was collected and was found to contain up to 90 mL of water, much more than was required to block the FOHE when released all at once.
In the final phase of their full-scale testing, investigators attempted to trigger this sudden release by allowing ice to build up in the lines for 6 hours with a steady, low fuel flow of 6,000 pph and steadily decreasing temperature, before abruptly increasing fuel flow above 12,000 pph. In several of these tests, the ice was swept out of the lines and lodged on the face of the FOHE, causing a restriction in fuel flow that varied greatly in magnitude due to the seemingly random character of the original ice buildups. However, in one test the fuel flow became restricted to 6,000 pph, as observed on flight 38, and afterwards ice with a water content of 55 mL was recovered from the face of the FOHE.
In this line of work, there is no such thing as a smoking gun. However, the tests proved that ice in the fuel lines could have caused the accident, and sometimes that’s the closest anyone is going to get.
◊◊◊
Having gathered all of the above data, investigators concluded that after flight 38 took off from Beijing, the normal traces of water in its fuel froze and began to accumulate in the fuel lines after power was reduced at the top of the climb. Fuel flow remained low throughout the cruise phase and even during the climbs to 38,000 and 40,000 feet, because the pilots used a sedate rate of climb for reasons of passenger comfort. Power was subsequently reduced even further for the descent. Although no new ice had formed for several hours, the existing ice remained in place because fuel flow remained low and because the fuel temperature remained well below freezing, even as flight 38 came in for landing at London, where the outside air temperature was +10˚C.
When the autothrottle increased thrust during final approach to maintain the approach speed amid minor turbulence, fuel flow reached its highest value since the climb out of Beijing, dislodging the ice that had accumulated earlier in the flight. This ice then made its way down to the FOHEs, where it adhered to the faces of the exchangers, restricting fuel flow to between 5,000 and 6,000 pph. Consequently, power rolled back to a value below that required to maintain the glidepath to the runway. The pilots received no engine failure warnings because the engines had not failed, and they didn’t receive “low fuel pressure” warnings either because fuel pressure is measured at the boost pumps, upstream of the restriction. It was the perfect storm — unpredictable, insidious, striking at the worst possible time.
Investigators praised both pilots for their handling of the emergency, noting that they made several decisions that contributed to the safe outcome. First Officer Coward did an excellent job extracting near-maximum performance from the airplane by riding the hairy edge of a stall all the way down. Captain Burkill, meanwhile, made the difficult but correct decision to leave his first officer at the controls during the emergency so he could make tactical decisions. Captains sometimes feel tempted to take the controls when lives are on the line, but more often than not this kind of last-minute handover leads to confusion and disarray.
Boeing test pilots also examined Burkill’s decision to retract the flaps from 30 degrees to 25, and found that it might have made a significant difference to the outcome. In their simulations, the greater drag induced by flaps 30 resulted in touchdown at least 50 meters earlier than the airplane’s actual touchdown point. Since the actual touchdown point was only 40 meters in front of the ILS antennas — an array of metal poles arranged on a concrete platform — this meant that the airplane almost certainly would have struck the antennas had Burkill not retracted the flaps. Investigators declined to speculate about what would have happened in such a scenario, except that it “would probably have led to more substantial structural damage to the aircraft.” At the very least, Burkill saved many passengers from more serious injuries, if not death, by clearing the antennas with one split second decision.
For his part, Burkill has pointed out that this figure of 50 meters was achieved by Boeing test pilots, and that he and Coward were only regular line pilots. In his recollection, it felt as though the aircraft was headed toward the commercial area next to the A30 trunk road, and that’s where he thinks it would have come down had he not retracted the flaps. It did not seem to him in that moment that they were on track to achieve the result that the Boeing test pilots did. But then again, none of the test pilots managed to get farther than Burkill and Coward did with flaps 25, so maybe they aren’t giving themselves enough credit. Either way, however, the actions of both pilots directly ensured that the outcome was as benign as possible.
In the immediate aftermath, First Officer Coward was hailed as a hero, which he undoubtedly was. Unfortunately, however, Captain Burkill’s experience after stepping off the airplane was not nearly as sunny. In his memoir, “Thirty Seconds to Impact,” published in 2010, Burkill and his wife Maria painted a vivid picture of how events, for a time, turned sharply against him. According to his memoir, British Airways barred him from interacting with the press, but also failed to release any other information about him, or what had happened, as a result of which the notoriously unscrupulous British tabloid news industry started publishing unfounded speculation instead. Various newspapers with wide circulation but trashy reputations published racy photos and allegations that he had frozen at the controls, that First Officer Coward had to save the plane all by himself, and that he hadn’t made a mayday call or a brace call. It’s unclear where these rumors came from, and only the last one was true — Burkill says he didn’t have time to call “brace.” The other allegations were patently false, but in the absence of verifiable facts, they were widely believed, including by other aviation professionals. Being the subject of salacious rumors, without any means of defending himself, caused Burkill and his wife to suffer serious emotional damage. This damage was further reinforced on his first flight back in the cockpit, when a cabin crew member informed him that British Airways flight attendants were told, during the course of a formal training session based on the crash, that Burkill “froze” and did nothing to save the plane!
As a result of this maddening encounter, Burkill voluntarily withdrew from flying, taking a severe pay cut, until the promised publication of an internal report that he was told would clear his name. This report ultimately never materialized, and Burkill spent months on leave; he fell behind on mortgage payments and almost lost his house. False rumors spread that cabin crews were scared to fly with him. Although he did eventually return to flying, British Airways’ failure to correct the record damaged his trust in the company, and in 2010 he applied for “voluntary redundancy” and was laid off. He left BA with a fire hose salute, but he ultimately returned to the company two years later, and his LinkedIn page suggests that he’s still flying for British Airways today.
Fortunately, time has changed Burkill’s public perception to a considerable degree. The final report on the crash exonerated him completely, and even before then, he received several awards for outstanding airmanship. Media coverage generated by his book and the release of the report largely corrected the record, reversing those same papers’ earlier mistakes. Still, Burkill rejects the notion that he is a hero — as he puts it, he was just doing his job, and it was his rigorous training that allowed him to make the split second decision to retract the flaps.
I also think Burkill’s story holds some valuable lessons about the way we treat one another. During his ordeal, he was told over and over again to “just ignore the papers,” to just ignore what people were saying about him, as though it didn’t matter. But the truth is that it does matter. I’m not nearly as famous as Peter Burkill, but I am well-known enough to be talked about in circumstances outside my control, watching my work and decisions get picked over by total strangers, as is anyone who publishes anything — it comes with the job. And yet, I know that being misunderstood or having my motivations questioned is painful, even when it happens only occasionally. I can only imagine what Burkill experienced, because he was involved in something that actually mattered, and the questioning of his competence was pervasive — everybody was talking about him, much of that discussion was negative, and much of it was provably false. It was a lonely ordeal, and one that caused him real, tangible harm. His story has made me re-examine the way I’ve talked about strangers, the way anyone talks about strangers. And the lesson I want to extract from it — cliché as it may be — is that while we, as aviation enthusiasts, deeply enjoy speculation, that speculation needs to be based on hard data, and verifiable facts released by entities directly involved in the fact-finding mission. When facts are scarce, baseless speculation feeds upon itself, circling through discussion boards and news analysis segments until it starts to take on its own recursive credibility, because after all, “everybody is saying it.” Everybody was saying that Burkill froze up, but no one could actually say where that idea came from. It wasn’t based on anything he said, or anything his first officer said, or anything investigators said, or any kind of flight data. And yet, credible people repeated it as though it were fact.
Sometimes, I think a lot of us roll our eyes when told not to speculate about “pilot error” until proof of an error is provided — after all, when a plane crashes, it’s more likely than not that somebody screwed up. But there is a line to be drawn, and what was said about Burkill crossed it. Reading his story has made me think about that line a lot harder.
◊◊◊
Leaving that fourth-wall-breaking tangent aside, it’s worth thinking a little bit deeper about why the accident happened at all.
Analyzing the crash from a holistic perspective, investigators raised an excellent question: if the 777 could operate for 13 years without an incident like flight 38, then how rare exactly are the conditions that triggered it? Was there something about flight 38 that set it apart from the crowd, that explained why ice blocked its fuel lines and not the fuel lines of countless previous flights? To find out, investigators used data mining techniques to examine detailed QAR data from over 175,000 Boeing 777 flights, retained by various airlines for quality control purposes. What they discovered was that while flight 38 had a very low minimum fuel temperature, it wasn’t uniquely low, and in fact lower fuel temperatures make ice less sticky anyway. But what it did have was a unique confluence of low cruise fuel flow, high fuel flow during approach, and low fuel temperature during approach. Flight 38 wasn’t the most extreme example in each of these categories taken individually, but if the categories were combined into a single parameter, then flight 38 was an extreme outlier.
During the investigation, the AAIB identified only one other case of a known or suspected engine rollback on a 777 due to fuel system icing. This occurred on board Delta Air Lines flight 18 from Shanghai to Atlanta on November 26th, 2008, when the right engine rolled back during a slight power increase while in cruise at 39,000 feet. Fuel flow to the right engine was restricted for 23 minutes, until the crew reduced power to idle, at which point the engine roared back to life — exactly as the AAIB had observed in their full scale fuel system tests with a blocked FOHE. The HP pump from the Delta flight’s right engine was also found with cavitation damage even worse than that from G-YMMM, which was expected given the length of time that the blockage persisted. Working with the AAIB, the United States NTSB determined that the cause of the Delta incident was most likely the same as that of flight 38. However, anecdotally, some in the aviation industry harbor doubts that the mechanism was identical, mainly because the profile of Delta flight 18 was not an outlier like flight 38 was, making a connection between the two potentially fraught. I personally see no reason not to believe that there was a connection, but the fact that flight 18 didn’t have a unique profile nevertheless raises unanswered questions.
Regardless of the exact conditions that might lead to the fuel icing phenomenon, the problem was solved, as far as anyone knows, with a design update developed by engine manufacturer Rolls Royce in 2009. The solution was almost painfully simple: Rolls Royce simply removed the 4 millimeters of relief between the inlets of the FOHE fuel tubes and the face of the oil chamber. This ensured that the face of the FOHE was hot enough to melt any ice that came into contact with it regardless of fuel flow rate or fuel temperature. The Federal Aviation Administration subsequently issued an airworthiness directive making this fix mandatory, and it was incorporated into all Boeing 777s with Rolls Royce Trent 800 series engines by January 2011. In the 13 years since, there has not been a similar incident.
In their final report, the AAIB also suggested a number of other measures that might help prevent similar incidents more broadly, including the expanded use of fuel system icing inhibitors, or FSIIs, which prevent ice from forming in jet fuel by adhering to water molecules and lowering their freezing temperature. FSIIs have been in use by the US military ever since a B-52 bomber crash in 1958 that was attributed to fuel system icing. However, the additives never made much headway in the civilian aviation industry for a number of reasons, including a lack of problems with fuel system icing over many decades of experience. Also, at the aircraft design level, FSIIs were considered to be an alternative to FOHEs as a means of excluding ice from the fuel, rather than a means of preventing ice-induced restrictions in the system. In fact, until the crash of flight 38, there had been almost no research into how ice builds up in aircraft fuel systems. Certification rules called for manufacturers to prove that their fuel systems could withstand concentrations of water up to 200 ppm, five to six times the normal value, with no impact on operations at a variety of temperatures. Boeing had gone even farther, testing the 777’s fuel system with a water concentration of 600 ppm with no adverse effects. But no one had ever considered what would happen if this water were to accumulate as ice and then release all at once. If they had, then maybe the need for FSIIs would have been obvious. Sometimes these things seem so much simpler in hindsight.
◊◊◊
In the end, these simple fixes were made possible by the findings of an AAIB investigation that ultimately cost many millions of dollars and years of labor. Someone once asked me: was that expense really worth it, just to remove four millimeters from a few pipes, to prevent a freak accident made possible by a truly calamitous alignment of the stars that might very well never happen again? Can we really argue that that great expense was ever likely to be recouped, whether in dollars saved, or in lives? And if not, then why was the investigation doggedly continued even after the AAIB realized that ice in the fuel, broadly speaking, was the likely cause? Was it just to make people feel better about flying, because we would all bay for blood if it looked like the authorities had done nothing? I don’t even remember who asked me this, or where, or why, but it has stuck with me for a long time. In some ways, this person was right — the crash of flight 38 was caused by what essentially amounted to a freak alignment of the stars, a force some might call fate, which I would prefer to describe as entropy, the randomness inherent in the universe. But I disagree that a solution was sought only out of a need to placate the masses. For one, investigating all avenues is the only reason we know that the “randomness inherent in the universe” was the cause of the crash in the first place. And perhaps more important is that as humans, we have an overpowering desire to know, to understand, to discover all that is hidden. We are curious creatures who abhor mysteries, and we will shell out large amounts of money to make them go away, even if the only material reward is an answer.
_______________________________________________________________
Don’t forget to listen to Controlled Pod Into Terrain, my podcast (with slides!), where I discuss aerospace disasters with my cohosts Ariadne and J! Check out our channel here, and listen to our latest episode, about an airline composed entirely of pure, distilled crime. A bonus episode for Patreon subscribers is also out now. Alternatively, download audio-only versions via RSS.com, or look us up on Spotify!
_______________________________________________________________
Join the discussion of this article on Reddit
Support me on Patreon (Note: I do not earn money from views on Medium!)
Visit r/admiralcloudberg to read and discuss over 260 similar articles
_______________________________________________________________
Note: this accident was previously featured in episode 67 of the plane crash series on December 15th, 2018, prior to the series’ arrival on Medium. This article is written without reference to and supersedes the original.
