NASA Technology Means No More Flying Blind

Synthetic Vision Systems Technology

  Imagine a world where pilots see clear skies all the time. It's not some weather fantasyland, but a revolutionary cockpit display technology called Synthetic Vision. NASA is developing it to make flying safer.

 NASA aeronautics researchers from Langley Research Center in Hampton, Va., tested the Synthetic Vision Systems technology this summer. They

HUD and panel display - Eagle Vail Tests

 tested the system aboard a Gulfstream GV business jet in air space around Reno, Nev., and NASA's Wallops Flight Facility, Wallops Island, Va.

 The system gives pilots a clear electronic 3-D perspective of what's outside, no matter what the weather or time of day. It combines Global Positioning System satellite signals with an onboard photo-realistic database to paint a terrain picture for the crew.

 During the flight tests, NASA evaluated an integrated version of the technology. It included a bird's eye view of topography, voice-recognition system and advanced sensors. Also included was Database Integrity Monitoring Equipment that ensures accuracy by using sensors to compare the real world to generated pictures. Added to this was a Runway Incursion Prevention System, which included an airport moving map. It also had software that can predict possible encroaching runway traffic, while alerting the crew.

 NASA will use the results of the flight test to advance the development of technology to help reduce fatal aircraft accident rates. Synthetic Vision Systems could help eliminate the world's deadliest aviation accidents, called Controlled Flight Into Terrain (CFIT). In CFIT accidents, a normally functioning aircraft crashes, because the pilot wasn't aware the plane was headed in the wrong direction, due to bad weather or a combination of factors.

 "NASA has already tested the individual technologies of Synthetic Vision and Runway Incursion Prevention Systems onboard a NASA 757 jet aircraft," said Randy Bailey, Synthetic Vision principal investigator. "We were excited to see it fly as an integrated system on the Gulfstream. We were particularly excited to be partnered with Gulfstream, which has been an industry innovator in aviation technology," he said.

 Other flight test partners included the U.S. Air Force Research Laboratory, Wright-Patterson Air Force Base; Rockwell Collins, Cedar Rapids, Iowa; Jeppesen, Englewood, Colo.; Rannoch Corp., Alexandria, Va.; The Boeing Company, Huntington Beach, Calif.; RTI International, Research Triangle Park, N.C.; and Ohio University, Athens, Ohio.

 Seventeen pilots selected from the Federal Aviation Administration (FAA), U.S. Air Force, the Joint Aviation Authority, aerospace industry and major airlines flew the GV more than 67 hours in 22 flights to collect data. Gulfstream provided command pilots.

 During the flight evaluations, the test pilots' windshield was often intentionally covered or flights were conducted at night. The process simulated low visibility conditions, so the pilot would have to rely on computer-generated instrument displays. The information included a head-down display mounted in the instrument panel and a head-up display to superimpose terrain and guidance information onto a screen in front of the pilot's eyes.

Runway Incursion Protection

A number of airline pilots have already flown components of the Synthetic Vision System in simulators and a NASA 757 research jet. "I think it's awesome," said United Airlines 767 Captain Rick Shay of the technology. "To explain the difference in the situational awareness that you gain, it's just a complete leap from the technology that's there today," he added.

 The NASA Aviation Safety and Security Program is part of NASA's Aeronautics Research Mission Directorate. It is also a partnership with the FAA, aircraft manufacturers, airlines and the U.S. Department of Homeland Security. The program's goal is to help reduce the fatal aircraft accident rate and protect air travelers and the public from security threats.

 For information about NASA's Aviation Safety and Security Program, on the Web visit:

For information about NASA's Aeronautics Research Mission Directorate, visit:

Link to ESVS movie on this page   and here  (includes published research papers and other documents)


Helmet-Mounted Displays

Fiber-Optic Helmet-Mounted Displays (FOHMD)

Enhanced and Synthetic Vision System (ESVS)

Enhanced and Synthetic Vision System (ESVS)

Computer simulation of the enhanced and synthetic vision concept
Computer simulation of the enhanced and synthetic vision concept

The ESVS is a system that provides an augmented visual scene to the pilot. This augmented visual scene allows the accomplishment of Search and Rescue (SAR) mission task elements by providing the visual cues that the pilot would normally use during the execution of these tasks in visual flight conditions and that are obscured or otherwise unavailable from a direct view of the outside world due to the weather or the night environment.

The image presented by the ESVS is a composite of four distinct images produced on the aircraft:

  • a synthetic (or computer generated) image based on a pre-stored, digital data base
  • a series of non-database objects determined by the output of a range finding sensor which scans the area around the helicopter
  • an enhanced image based on the fusion of a variety of electro-optical sensor outputs (InfraRed, Low Light Level TV, etc.)
  • a symbology set which provides aircraft and navigation information.

These four images are combined into a single, coherent image and presented to the pilot on a Helmet Mounted Display (HMD).

National Research Council-Conseil national de recherches Canada  line
Date Modified:  2003-10-30 Top of page

02:00 AM Mar. 27, 2003 PT

Nature accomplished earlier this week what Iraq's Republican Guard could not: Blinding sandstorms paralyzed the American air campaign, grounding helicopters and cutting bombing runs by as much as 85 percent in some areas.

But an Air Force program in the works may enable pilots to plow through just about any foe -- even an Iraqi sandstorm.

The solution is an onboard computer that digitally renders the pilots' surroundings when they can't rely on the real one to guide them. It's called "synthetic vision," and its backers are promising that the system will let pilots see in nasty weather, just like night-vision goggles let troopers roam around in the dark.

"There's no doubt in my mind that there's a tremendous military benefit to this technology," said Thomas Schnell, director of the University of Iowa's Operator Performance Laboratory, an experimental flight center. "As long as you could receive a GPS signal, you could fly using synthetic vision -- no matter what the weather."

Sandstorms included, he added.

Both the Air Force and NASA are trying out versions of synthetic vision. In recent tests, the Air Force's 412th Flight Squadron flew a modified C-135 transport plane -- nicknamed the Speckled Trout -- using the system.

NASA has committed $100 million over five years to develop synthetic vision as part of a larger effort to improve aviation safety. Its first full-fledged demonstration of the system will come in late May or early June, at the Wallops Island test flight center. The second will be in June or July, at the airport in Reno, Nevada.

With these tests pending, synthetic vision clearly isn't about to be deployed in Iraq. Too bad -- the U.S. military could have used it over the last few days.

"You would be able to use synthetic vision systems to effectively see through sandstorms," said Russ Parrish, chief scientist of NASA's synthetic vision project, "if you had a high enough resolution image of the area."

That assumes, of course, the dust doesn't choke your engine.

A few of the Army's Humvees have been able to creep through sandstorms by relying on satellite-based locators. Night-vision systems have boosted visibility to about 10 feet when sand kept soldiers from seeing their hands in front of their faces. And infrared sensors have provided images a bit further out. But these makeshift answers have only been marginally helpful.

"The particles that block your normal vision would block these sensors, too -- just on a different wavelength," said James Ratches, chief scientist of the U.S. Army's CECOM Night Vision and Electronic Sensors Directorate.

Synthetic vision avoids that problem by having a picture of the world stored on the plane. In 2000, the Space Shuttle Endeavour took detailed 3-D radar images of more than 80 percent of the earth's terrain. The Defense Department's National Imaging and Mapping Agency is now translating this data into digital, topographical maps. These shuttle maps could then be checked against public satellite images; double-checked against data in the Global Positioning System or Internal Navigation System; and triple-checked by radar, infrared or millimeter wave sensors.


A single digital representation of what's on the ground and what's in the air would be assembled from these sources. It'd look extremely familiar to players of flight simulation games.

Pilots could see these images in one of a number of ways -- on a tablet PC, on a screen built into the cockpit's dashboard, through a pilot's helmet, or on a piece of glass or plastic mounted near the windshield.

But synthetic vision is still years away from widespread commercial or military use. Integrating all the maps and the sensor data -- and doing it in real time -- is going to take a significant amount of work.

Technology is not the biggest hurdle for synthetic vision, advocates said. It's the Federal Aviation Administration.

"The FAA makes things go very slowly," said Garth Smith, founder of MetaVR, a maker of visualization systems, mostly for the military. "The amount of regulation that's involved just to test (synthetic vision) is staggering."

Right now, the FAA won't allow planes with synthetic vision systems to operate in any worse weather than the planes without them. So the big commercial aviation companies have little incentive to invest in the program. Calls to the FAA were not returned immediately.

Military development of synthetic vision would be further along, said one source familiar with the project's development, "but all the money is being dropped" on unmanned vehicles, like the Predator spy drones.

But Space Imaging, the public satellite concern, believes that a homeland security market exists for synthetic vision. The company is pushing to adapt the system to helicopters, so they can monitor areas when the weather sours.

More Research Papers here


to Hot off the Press Menu