IG Report: NASA’s SOFIA Not Meeting Expectations

by Douglas Messier
Managing Editor

NASA’s flying Stratospheric Observatory for Infrared Astronomy (SOFIA) has struggled to meet its scientific expectations due to a lengthy development delay and a series of technical, operational and managerial challenges, according to a new audit from the agency’s Office of Inspector General (IG).

“Although responsible for several first-of-its-kind discoveries, SOFIA’s 13-year development delay reduced the Program’s ability to produce impactful science in a cost-effective manner, particularly when compared to the cost of and science produced by other infrared observatories that launched in the interim,” the report said.

“Further, SOFIA has not fully utilized its unique capabilities to serve as an instrument test bed due to high instrument development costs, or to fly anytime anywhere because of a lack of instrument scheduling flexibility, the amount of time necessary to switch out instruments, and the prioritization of observations with greater scientific significance,” the document added.

SOFIA is a flying telescope that uses a modified Boeing 747. The observatory is a joint program with the German Aerospace Center (DLR) and is operated by NASA’s Armstrong Flight Research Center in cooperation with the Universities Space Research Association (USRA) and German SOFIA Institute.

Despite attempts by the Trump and Obama administrations to cancel SOFIA due to poor results, Congress has repeatedly continued to fund the program.

The IG report made nine recommendations for improving SOFIA’s scientific output. These included implementing quantifiable research hour requirements, establishing a set of scientific metrics, and conducting 8-hour flights five times per week when SOFIA operates from its base in California between May and November.

NASA concurred with all nine recommendations. “We consider management’s comments responsive; therefore, the recommendations are resolved and will be closed upon completion and verification of the proposed corrective actions,” the audit stated.

The report’s Results in Brief section is reproduced below.

NASA’s Management of the Stratospheric Observatory for Infrared Astronomy Program (SOFIA)

Office of Inspector General
September 14, 2020
Report No. IG-20-022
Full Report

Results in Brief

Why We Performed this Audit

The Stratospheric Observatory for Infrared Astronomy (SOFIA) is an airborne observatory housing a 106-inch telescope mounted onboard a Boeing 747SP that makes observations from between 38,000 and 45,000 feet, putting it above 99 percent of water vapor that interferes with ground-based infrared observations.

The aircraft is operated out of NASA’s Armstrong Flight Research Center in Palmdale, California, and is currently equipped with six interchangeable instruments that can be upgraded or replaced. SOFIA is used by astronomers to study astronomical objects and phenomena including star birth and death, formation of new solar systems, identification of complex molecules in space, nebulas and galaxies, and transient events like eclipses.

The SOFIA Program is a partnership between NASA—within the Science Mission Directorate’s (SMD) Astrophysics portfolio—and the German Aerospace Center and operates in cooperation with the Universities Space Research Association (USRA) and German SOFIA Institute. NASA began development of SOFIA in 1991 and the Program achieved full operational capability in 2014, 13 years behind schedule with development costs over $1 billion (more than 300 percent over the initial cost estimate).

Because SOFIA has experienced ongoing operational and technical challenges and has not met science output expectations, NASA has questioned whether its $83 million in annual operating costs could be put to better use. Likewise, the President’s Budget Request has proposed terminating SOFIA on multiple occasions including most recently for fiscal year 2021. However, in the past Congress has continued to fund the Program.

In this audit, we evaluated NASA’s management of the SOFIA Program’s cost and technical performance to ensure maximum scientific return. Specifically, we assessed the Program’s

(1) delivery of reliable flight opportunities;

(2) efficiency of research flights in maximizing the quantity and quality of data obtained from observations;

(3) ability to meet established data processing time commitments and provide quality data;

(4) expected technical capabilities, including development of next-generation instruments; and

(5) overall operational effectiveness in maximizing science output.

To complete this work, we interviewed SMD and SOFIA Program management; observed an operational flight; identified factors affecting Program scientific productivity; surveyed the science community; and reviewed operational requirements and goals, NASA’s contract with USRA, SOFIA science instrument development, and NASA reports and studies on the SOFIA Program.

What We Found

Although responsible for several first-of-its-kind discoveries, SOFIA’s 13-year development delay reduced the Program’s ability to produce impactful science in a cost-effective manner, particularly when compared to the cost of and science produced by other infrared observatories that launched in the interim.

Further, SOFIA has not fully utilized its unique capabilities to serve as an instrument test bed due to high instrument development costs, or to fly anytime anywhere because of a lack of instrument scheduling flexibility, the amount of time necessary to switch out instruments, and the prioritization of observations with greater scientific significance.

SOFIA also continues to experience operational and technical challenges related to flight operations, observation completion, data processing, USRA’s award fees, and instrument development. While SOFIA was designed to fly 960 research (or observation) hours annually, it has yet to achieve this number of hours or dispatch the expected number of scheduled flights, resulting in much less science output than expected.

While SOFIA typically has flown a 10-hour flight profile four times per week, an alternate schedule of 8-hour flights five times per week out of Palmdale from May to November appears to achieve more time in the stratosphere and therefore higher quality data.

Further, although the Program’s scientific productivity has improved in recent years, the resulting publications and citations are significantly below what is expected for a mission of SOFIA’s budget and lower than comparable observatories. SOFIA’s limited science output can be attributed to multiple factors including incomplete observation programs and issues with the timeliness and quality of data produced during an observation.

In addition, performance ratings and award fees for USRA—which was awarded a contract in 2017 to manage SOFIA Program science operations—have been based on subjective judgments and were not adequately tied to primary science productivity metrics (publication and citation rates) that are standard across the astronomical community.

The Program has also not developed any new instruments, only implemented one instrument upgrade, and SMD recently canceled the only new instrument in development due to cost concerns and technical challenges. Finally, SOFIA has not retired non-productive instruments in a timely manner.

The lack of clear and achievable performance expectations and lack of concurrence between SMD and SOFIA management on science output goals including publication and citation metrics has reduced productivity and threatens the Program’s future viability.

The Program is unlikely to achieve the community’s expectation of 150 publications per year by 2022, or the Program’s goal of 100 annual publications, as it only produced 33 publications in 2019 and the actions proposed to meet this goal fall short of the transformational changes required to address current operational and technical challenges.

Further, the proposed actions are unlikely to mitigate SOFIA’s lack of competitiveness because of the Program’s poor efficiency on a science-per-dollar basis when compared to other observatories.

What We Recommended

To improve the Program’s productivity, we recommended the Associate Administrator for SMD direct SOFIA Program management to:

(1) implement quantifiable research hour requirements

(2) establish a requirement to maximize research hours in the stratosphere that considers implementing an 8-hour, five times per week flight profile whenever SOFIA operates from Palmdale between May and November;

(3) establish observation program completion metrics, based on science value prioritization, to increase the probability observations result in publications;

(4) develop and implement a process to increase the timeliness of delivering data to researchers;

(5) establish science metrics, such as publications and citations per year, as criteria for the performance evaluation of the USRA contract award fee;

(6) establish a requirement for new technology implementation; and

(7) develop a process for tracking science instrument productivity and cost to maintain and establish an evaluation program to determine when an instrument should be retired.

We also recommended the Associate Administrator for SMD and SOFIA Program management coordinate to:

(8) reassess SOFIA’s strategy and mission to identify and consider implementing alternative operational approaches and models to maximize SOFIA’s capabilities within the Astrophysics portfolio and return on investment and

(9) develop consensus between SMD and Program management and implement quantifiable operational and science output requirements.

We provided a draft of this report to NASA management, who concurred with all of our recommendations. We consider management’s comments responsive; therefore, the recommendations are resolved and will be closed upon completion and verification of the proposed corrective actions.