Remarks attributed to Elon Musk in which he discussed a possible cause of SpaceX’s Falcon 9 launch pad firexplanomaly leaked out to the public last week after his his presentation before officials at the National Reconnaissance Office (NRO).
“We are close to figuring it out. It might have been formation of solid oxygen in the carbon over-wrap of one of the bottles in the upper stage tanks. If it was liquid it would have been squeezed out but under pressure it could have ignited with the carbon. This is the leading theory right now, but it is subject to confirmation,” Musk is reported to have said.
Musk was reportedly referring to carbon composite overwrapped pressure vessels (COPVs) that hold helium inside the Falcon 9’s second stage liquid oxygen (LOX) tank.
The remarks were briefly posted on Reddit before they and the entire thread were deleted. SpaceNews reports that a SpaceX spokesman “declined to confirm or deny that the statements, published anonymously online, conformed exactly to what Musk said.”
The Reddit post also attributed the following remarks to Musk concerning the prospect of sabotage of the rocket.
“The other thing we discovered is that we can exactly replicate what happened on the launch pad if someone shoots the rocket. We don’t think that is likely this time around, but we are definitely going to have to take precautions against that in the future. We looked at who would want to blow up a SpaceX rocket. That turned out to be a long list. I think it is unlikely this time, but it is something we need to recognize as a real possibility in the future.”
A long list, indeed. As SpaceNews points out, it’s not clear whether Musk volunteered the information about possible sabotage or whether he was asked to comment upon it by a member of the audience.
A source with contacts at Cape Canaveral tells Parabolic Arc that people there believe SpaceX is spending a bit too much focusing on a sabotage angle they don’t see as very credible. But, since SpaceX is in charge of investigating its own failure, the company is going to proceed as it sees fit.
What’s really interesting is that SpaceNews asked several experts in the field to comment on the report anonymously and summarized the results in its story. The experts found the theory reportedly offered by Musk to be “perfectly credible.” The story also raises the following issues.
SpaceX’s Use of COPVs. The composite helium bottles are not new, but the way SpaceX is using them is. And that might go to the core of the problem.
“It has been used at least since the 1980s on several launchers and spacecraft. But to my knowledge they have never been used in such a cryogenic, unsteady environment. In a cryogenic environment you have thermal gradients that are not always oriented in the same way from the interior to the exterior of a structure, or the exterior toward the interior.
“The difficulty is reproducing the thermal demands applied to the wound exterior of the bottle against the interior side. If wound or wrapped layers slide even a little bit, you get shear de-lamination. This can cause a catastrophic result.”
Did a Burst Helium Bottle Bring Down Another Falcon 9? A SpaceX investigation into the explosion that destroyed a Falcon 9 rocket in flight last year blamed a defective strut provided by a supplier that broke under pressure, causing a helium bottle to shoot up to the top of the second stage LOX tank. The company says a strut was not involved in the launch pad accident last month.
But, is that what actually happened last year? SpaceNews experts raised serious doubts.
“The question is whether the strut was really the origin of the June 2015 failure. The struts were of below-specified quality, but it was not demonstrated that this was the root cause of the failure.
“It remains possible that a helium bottle burst in June 2015 and that, in parallel, the struts supporting it were of poor quality….
“But there is always the risk of correcting things that are not the real cause, such as correcting a badly built strut, which might not have prevented the June 2015 failure…”
Parabolic Arc has heard reports for about a year that experts didn’t buy SpaceX’s conclusion. Sources believe the accident was more complicated than a defective strut breaking under flight pressure.
A separate investigation by NASA’s Launch Services Program (LSP) did not find a probable cause for the accident. An audit by the NASA Office of Inspector General released last June summarized the program’s findings as followed.
“LSP did not identify a single probable cause for the launch failure, instead listing several ‘credible causes.’ In addition to the material defects in the strut assembly SpaceX found during its testing, LSP pointed to manufacturing damage or improper installation of the assembly into the rocket as possible initiators of the failure. LSP also highlighted improper material selection and such practices as individuals standing on flight hardware during the assembly process, as possible contributing factors.
The SpaceX investigation board included 11 company employees, including its chairman, and a long FAA representative. The final report was signed by the 11 SpaceX employees only.
The Falcon 9 that failed in June 2015 was carrying a Dragon supply ship to the International Space Station for NASA’s Commercial Resupply Services-1 (CRS-1) program. Eight months earlier, an Orbital Sciences (now Orbital ATK) Antares rocket carrying a Cygnus supply ship to the station for the same program exploded shortly after liftoff.
The NASA IG audit criticized NASA’s practice of allowing SpaceX and Orbital ATK to conduct their own investigations in the failures.
“NASA’s official policy for investigations requires all official Mishap Investigation Boards to be independent. NTSB and USAF have similar requirements. In contrast with these best practices, the CRS-1 contract and FAA license requires SpaceX to conduct its own investigation but does not require company investigation boards to screen for conflicts of interest or maintain independence. FAA officials stated NASA can implement additional independence requirements for contractor-led investigations through its contracts as long as they do not conflict with FAA regulations….
“We acknowledge SpaceX’s investigation was transparent and the observers from FAA, ISS, LSP, NTSB, and USAF had access to the investigation’s data and analysis. However, an investigation led by the employee responsible for the SPX-7 launch and run by the contractor responsible for the failure raises questions about inherent conflicts of interest.”
Why Did SpaceX Bring Helium Bottle Production In-house? The helium bottles used by SpaceX during its early years were supplied by Cimarron Composites of Huntsville, Ala. However, SpaceX brought production of these bottles in house in 2014.
At the time, there was a general belief that SpaceX made the change due to its preference for vertical integration. The company felt it had learned everything it needed to learn from Cimarron and could proceed on its own.
However, SpaceX ended up delaying multiple launches due to helium leaks in 2014. Instead of launching 11 Falcon 9’s as planned that year, the company achieved only six flights. So, maybe SpaceX hadn’t learned everything about how to manufacture these bottles.
The SpaceNews article suggests another reason for bringing COPV production in house.
“In 2014 Cimarron Composites suffered an explosion which caused extensive damage. My understanding is that after this event, SpaceX decided to bring in-house the production of these vessels.”
Space — and COPVs — are Hard. Not only is manufacturing the composite helium bottles challenges, but producing them consistently for the “cryogenic, unsteady environment” SpaceX uses them in creates additional risks.
Fiber- or tape-, or strip-wound “overwrapped” technology is not something you can learn in a day or two. But SpaceX did it. They have many good technicians and engineers and equipment.
But this kind of multilayer, resin-coated winding is never totally reproducible. The big problem in serial production of this kind of manufacturing is to be able to identify the drift and defects, during quality control inspections, in an operating mode that is representative of the intended use — with margin, as necessary. That is not easy for severe cryogenic uses.
In cryogenic geyser cycling environments, such as SpaceX’s rapid filling of a LOX tank at 66 Kelvin, you have violent variations of temperature between the interior and exterior. The thermal gradient is deeply heterogeneous from one location to another.
Can SpaceX Begin Flying Again in November?
SpaceX President Gwynne Shotwell have expressed confidence Falcon 9 can be returned to service in November and fly a couple of missions by the end of the year. She has suggested the the September accident was caused by operational issues rather than a flaw in the COPV design.
The verdict from SpaceNews’ experts on such a rapid return to flight is a definite maybe.
“There are lots of ways they could proceed, depending on how acceptable risk is assessed. For example, they could launch a mission that does not need the rocket’s full performance, and so they can pressurize the helium tank to a much lower extent. But it’s difficult to move forward without demonstrating a root cause and corrective actions.”