INFLUENCE OF OPTICAL AND GRAVITO-INERTIAL CUES TO HEIGHT PERCEPTION DURING SUPERVISORY CONTROL

HAL Id: hal-03195979

https://hal.archives-ouvertes.fr/hal-03195979

Submitted on 12 Apr 2021

HAL is a multi-disciplinary open access archive for the deposit and dissemination of sci- entific research documents, whether they are pub- lished or not. The documents may come from teaching and research institutions in France or abroad, or from public or private research centers.

L’archive ouverte pluridisciplinaire HAL, est destinée au dépôt et à la diffusion de documents scientifiques de niveau recherche, publiés ou non, émanant des établissements d’enseignement et de recherche français ou étrangers, des laboratoires publics ou privés.

INFLUENCE OF OPTICAL AND

GRAVITO-INERTIAL CUES TO HEIGHT

PERCEPTION DURING SUPERVISORY CONTROL

Martine Godfroy-Cooper, J. D. Miller, Jean-Christophe Sarrazin, François Denquin, E. Bachelder

To cite this version:

Martine Godfroy-Cooper, J. D. Miller, Jean-Christophe Sarrazin, François Denquin, E. Bachelder. IN-

FLUENCE OF OPTICAL AND GRAVITO-INERTIAL CUES TO HEIGHT PERCEPTION DUR-

ING SUPERVISORY CONTROL. Vertical Flight Society’s - 76th Annual forum & Technology Dis-

play, Oct 2020, VIRGINIA BEACH (virtual), United States. �hal-03195979�

INFLUENCE OF OPTICAL AND GRAVITO-INERTIAL CUES TO HEIGHT PERCEPTION DURING SUPERVISORY CONTROL

Dr. M. Godfroy-Cooper Research Scientist SJSURF/US Army ADD AvMC Aviation Moffett Field, 94035 CA

J. D. Miller Research Engineer SJSURF/US Army ADD AvMC Aviation Moffett Field, 94035 CA

Dr. J.C. Sarrazin Research Scientist Ingénierie Cognitive et

Neurosciences Appliquées/DTIS/ONERA

Salon de Provence, France F. Denquin Ingénierie Cognitive et

Neurosciences Appliquées/DTIS/ONERA

Salon de Provence, France

Dr. E. Bachelder Research Engineer SJSURF/US Army ADD AvMC Aviation Moffett Field, 94035 CA

ABSTRACT

Future vertical lift (FVL) missions will be characterized by increased agility, degraded visual environments (DVE) and optionally piloted vehicles (OPVs). Increased agility will induce more frequent variations of linear and angular accelerations, while DVE will reduce the structure and quality of the out-the-window (OTW) scene (i.e. optical flow). As helicopters become faster and more agile, pilots are expected to navigate at low altitudes while traveling at high speeds. In nap of the earth (NOE) flights, the perception of self-position and orientation provided by visual, vestibular, and proprioceptive cues can vary from moment to moment due to visibility conditions and body alignment as a response to gravitoinertial forces and internally/externally induced perturbations. As a result, erroneous perceptions of the self and the environment can arise, leading ultimately to spatial disorientation (SD). In OPV conditions, the use of different autopilot modes implies a modification of pilot role from active pilot to systems supervisor. This shift in paradigm, where pilotage is not the primary task, and where feedback from the controls is no more available, is not without consequences. Of importance is the evidence that space perception and its geometric properties can be strongly modulated by the active or passive nature of the displacement in space. An experiment was conducted using the vertical motion simulator (VMS) at the NASA Ames Research Center that examined the contributions of gravitoinertial cueing and visual cueing in a task where the pilot was not in control of the aircraft but was asked to perform altitude monitoring in a simulated UH-60 Black Hawk helicopter with a simulated autopilot (AP) mode. Within the altitude monitoring task, the global optical density (OD), flow rate and visual level of detail (LOD) were manipulated by the introduction of an 18ft vertical drift, upward or downward that simulates a vertical wind shift. Seven pilots were tested in two visual meteorological conditions, good visual environment (GVE) and degraded visual environment (DVE) and two gravitoinertial conditions, where platform motion was either ON or OFF. The results showed that both the good quality of the visual environment and the presence of gravitoinertial cues improved altitude awareness and reduced detection/ reaction times.

The improvement of the tracking performance in the visuo-vestibular setting as compared to a visual only setting when the visual cues were poor indicated some level of multisensory integration. Task-dependent limitations of a popular aeronautics metric called DIMSS-PM (Dynamic Interface Modeling and Simulation System Product Metric) and its sub-components were shown, and recommendations for OPV operations were formulated.

INTRODUCTION

This study was a joint effort between the US Army ADD AvMC Aviation and ONERA in the context of a US/ French Rotorcraft Project Agreement (RPA).

Presented at the Vertical Flight Society’s 76th Annual Forum

& Technology Display, October 6–8, 2020, Virtual. Copyright © 2020 by the Vertical Flight Society. All rights reserved. This is a work of the U.S. Government and is not subject to

The capability to govern self-motion in rich and changing environments is one of human’s most important perceptual–

motor skills. Self-motion, whether walking, driving, or flying requires trajectory control while avoiding collisions with obstacles. The perception of self-motion, which includes direction or heading perception and speed, relies on the integration of multiple sensory cues, mostly visual and

copyright protection in the U.S. DISTRIBUTION A. Approved for

public release: distribution unlimited.

vestibular. Optical patterns specify the position and velocity of distant objects (Refs 1, 2, 3), while variations in self - motion impact the gravito-inertial field (Refs. 4, 5, 6). Visual motion sensors are tuned to velocity rather than acceleration, and the frequency response of visual motion perception approximates a first-order low pass filter (Ref. 7). Meanwhile, vestibular, and proprioceptive motion sensors are specifically tuned to acceleration (transient movements) and have high- pass filter characteristics (Ref. 8). Under natural conditions, it is always the case that information from several sensory modalities is concurrently available. In the case of self- motion, visual proprioceptive and proprioceptive-vestibular interactions are often casually related. Whereas our perception of position and orientation provided by visual, vestibular and proprioceptive cues is relatively constant and veridical while on the ground, it can vary from moment to moment in flight due to visibility conditions, body alignment as a response to gravitoinertial forces and internally/externally induced perturbations. As a result, erroneous perceptions of the self and the environment can arise, leading ultimately to spatial disorientation (SD). For example, erroneous visual perception of distance often occurs during poor visual conditions such as night, whiteout, or brownout. Meanwhile, erroneous perception of motion caused by extreme velocities (too fast or too slow) can result in misinterpretation of directional cues. This is exemplified in the climbing/descending illusion in which a pilot that is accelerating or decelerating can experience the illusion that the aircraft is climbing or diving due to the resultant force being perceived as the force of gravity (Ref. 9). As a result, an inexperienced pilot may attempt to make a correction by pitching the aircraft upward, or worse, downward toward the ground.

The effects of translational and rotational accelerations on the detection of motion and direction while resting immobile, upright, or supine, have been studied extensively in the literature, but little is known when motor control is involved.

Most studies of perceived translation have involved the horizontal plane, however, rectilinear vertical acceleration, an inertial stimulation that remains parallel to gravity and alters only the magnitude of background force, has received little attention. In a height control task that considered visual cueing aspects as well as motion, Johnson (Ref. 10) investigated how the displayed visual level of detail (LOD) changes as one gets closer or further away from an object. The results showed that changing the visual LOD to maintain constant global optical density (OD) as the altitude changed, like that of the real world, improved altitude awareness.

Separately, adding platform motion improved speed regulation and altitude perception.

To our knowledge, the perception of altitude in low-level flight for a passive observer, i.e. when the pilot is not actively flying, was never investigated. In the context of future vertical lift (FVL) and optionally piloted vehicles (OPVs), the use of different autopilot modes will imply a modification of the pilot’s role from active pilot to systems supervisor, e.g. air

mission commander (AMC). This shift in paradigm, where pilotage is not the primary task, and where feedback from the controls is not available, is not without consequences. Indeed, it has been demonstrated that space perception and its geometric properties can be strongly modulated by the active or passive nature of the displacement in this space or this environment (e.g., Refs. 11, 12). While the question of being an active vs. a passive operator has regarding the perception of ego motion in specific gravitational and visual conditions has been largely unexplored, it can be investigated using the theoretical frame of “agency”. Agency refers to one’s ability to control his/her actions and, through them, events in the external world (Refs. 13, 14). In the context of automation and human-computer interaction, the question of agency, i.e. the perception of the level of control that we have on these systems, is central (Ref. 15). Because automation can fail, and because the AMC must maintain a holistic situation awareness (HSA), it is critical to understand how visual and gravitoinertial cues contribute to the perception of self- motion when the pilot is not an active agent, and when attention may be divided between tasks. In the case of low- level flight such as nap of the earth (NOE) missions, the perception of height is critical as flying too high can lead to aircraft detection by the enemy’s radars, and flying too low can lead to controlled flight into terrain (CFIT) or collision with an obstacle. Optimal perception of height relies on the synergistic contribution of multiple senses, mostly the visual and the vestibular systems, the role of each and their interactions detailed in the next sections.

Optical information and the visual system

The human visual system is composed of two complementary sub-systems, the ambient visual system which enables orientation relative to the global environment, and the focal visual system, allowing orientation relative to an object (Refs.

1, 16, 17, 18). Gibson (Ref. 1) has shown first that the direction of self-motion can be derived from the motion pattern of texture points in the visual field. He showed that for an observer in rectilinear motion, the “optical flowfield” or

“streamer” pattern seems to expand from a focal point that indicates the direction of motion. The optical flow generated at the pilot’s observation point contains crucial information for controlling self-motion (Refs. 1, 19). One of the most important components of the optical flow field structure is motion parallax, which informs about relative distance (Ref.

1) and egocentric distance (Ref. 20), and strongly depends on the ground texture.

Visual cues for altitude perception in Flight

A two-dimensional (2D) texture on a flat terrain surface

provides two types of cues for NOE flight: 1) the depression

angle, which is the visual angle formed by the horizon and a

terrain edge that is oriented perpendicularly to the direction of

motion and 2) the optical splay angle, which is the visual

angle formed by the motion path and a terrain edge oriented

parallel to the direction of motion at the convergence point on

the horizon (Ref. 21). However, it is unclear how to define splay and depression angles when the terrain consists of randomly placed three-dimensional (3D) objects and how they contribute to the perception of altitude when other cues, such as visual occlusion cues are more salient. On the simulated terrain shown in Figure 1 extracted from the video of an experimental trial, there are no clear terrain edges oriented perpendicularly or parallel to the path of travel.

Optical splay angle and depression angle have been proposed as quasi-independent sources of information about speed and altitude (Refs. 22, 23, 24). Both splay angle and depression angle are components of an expansion of texture that is associated with the approach to a surface or time to contact, tau (Refs. 25, 26).

Another visual cue that could be used for altitude maintenance is the change in the optical flow rate (ground rush) which is a measure related to the angular speed (splay angle) of terrain elements (trees, fields, etc.) as one moves through a visual scene. The rate of change of size is proportional to the rate of change of altitude (Ref. 23). Similarly, optical edge rate (also referred to as texture rate) is the rate at which texture elements pass a reference point that is fixed relative to the observer.

The rate at which depression angle changes is affected both by altitude and forward speed. Texture rate yields a good estimation of ground speed when the spacing between these elements remains relatively constant. Thus, changes in flow rate can veridically signal altitude deviation when texture rate is constant. Although edge rate does not change with altitude, it can affect the perception of self-motion (Ref. 27), which may interact with perceived texture density and optical flow rate. Evidence for interaction among visual cues to altitude has been reported by Flach et al. (Ref. 21).

Global optical density (OD) (perceived ground texture density) has been defined by Owen and Warren (Ref. 22) as

“the number of ground elements required to span one eyeheight distance”. For a constant texture size, changes in altitude will result in proportional changes in optical density (Ref. 28). Assuming that actual texture density is constant, the perceived texture density will increase as altitude increases, as seen in Figure 1. In the case of movement over a textured surface or a scene containing 3D objects, the most salient perceptual cue is the motion gradient formed by differential movements of the texture elements or the 3D objects (Ref.

29). Patterson et al. suggested that motion gradient is a visual cue that can be used for altitude maintenance (Ref. 30).

Motion perspective, as conceptualized by Gibson (Ref. 1), is another cue that may be relevant to altitude control in NOE.

Motion perspective refers to the relative movements of objects that occurs when the observer moves and is a consequence of the fact that nearer objects move faster across the retina than do farther objects. There is some evidence that vertical motion-perspective cues may be used for altitude control (Ref. 30). Optical flow is the logical extension of motion perspective (parallax) to all points in a scene (see Figure 7).

Figure 1. The same scene extracted from an experimental video trial in GVE at 27 ft above ground

level (AGL) on the left and 63 ft AGL on the right.

Because the actual texture density is constant, the perceived texture density increases as altitude increases.

The role of visual occlusion as a cue to altitude maintenance in low-altitude flight has been rarely investigated. Leung and Malik (Ref. 31) showed that the amount of visual occlusion present in a scene made up of 3D objects oriented perpendicularly to the ground is related to the product of object height, object density and object radius. In a simulated altitude maintenance task, Gray et al. (Ref. 32) showed that participants were using changes in the magnitude of visual occlusion (i.e. changes in the amount of visible ground surface between trees) as a visual cue.

Vision for Perception and Vision for Action

It has been proposed (Refs. 26, 33, 34) that the visual perception of objects and the visual control of action relies on relatively different neural pathways, respectively the ventral stream and the dorsal stream. These two streams differ in the metrics of interest (Ref. 33). Vision for perception relies on the relative size of objects (relative metric) and vision for action on their physical size (absolute metric) (Ref. 34).

Visual illusions

Even during natural flight in a 3D environment, and despite

providing the most important information to maintain spatial

orientation with respect to the terrestrial frame of reference,

visual orientation can be biased. The importance of vision to

self-motion perception is clear when considering the effects

of degraded or disrupted vision during self-motion. For

instance, pilots flying in a Degraded Visual Environment

(DVE) that deprives them of visual Earth-based orientation

cues could be prone to orientation errors if they do not rely on

instruments. One example is the illusion of self-motion (e.g.,

vection, Refs. 35, 36, 37) occurring (but not only) when pilots

misinterpret peripherical visual stimulation, often due to

surrounding objects moving at different speeds or in a

different direction, or to rotating light (e.g. reflection of

aircraft's rotors or ground lights). Another illustration is the

illusion of height perception (Refs. 37, 38), that occurs during

flight over featureless terrain where few visual cues are

available. This can give an illusion of lack of movement since

the normal passage of visual details is missing (poor optical

structure). It can also give the pilots a false sense of their

height above ground, and lead to controlled flight into terrain.

In most cases, these misperceptions are benign, short in time, and easily compensated for thanks to the contribution of the vestibular system (Ref. 39) and visuo-vestibular interactions (Ref. 40).

Gravitoinertial (GI) information and the vestibular system

The vestibular system is the most influential of the non-visual senses for the detection of information about passive and active, linear, and angular self-accelerations (Ref. 41). The vestibular system generates information for the three axes of head translation (transverse, longitudinal and sagittal) and the rotation, and provides spatial orientation in relation to the vertical gravity. Located at the level of the inner ear, the vestibular system consists of five distinct organs: the three semi-circular channels (sensitive to angular accelerations;

head rotations) and the two otolithic organs (saccule, utricule) which are sensitive to linear accelerations and gravity.

Beyond this anatomical aspect, it is important to note that vestibular integration has the distinction of being intrinsically multisensory (Ref. 42). There is no primary vestibular cortex per se, and there is more of a network of vestibular areas interconnected with the parieto-vestibulo-insular cortex (PVIC) (Ref. 43). Thus, the vestibular sensory dimension is essential to a set of processes essential for movement perception such as vision stabilization (vestibulo-ocular reflex, VOR), balance maintenance and head orientation estimation. In addition to providing consistency with visual, proprioceptive, and auditory inputs, the vestibular system allows self-motion to be discriminated from an external movement. Also, the weight of one sensory information compared to another depends on environmental constraints (gravito-inertial, optical) and the nature of the task (intentional control, automation, etc.).

Visuo-vestibular interactions

The relation between optical changes (detected by the visual system) and inertial changes (detected by the vestibular system) during self-motion has been widely investigated (Refs. 44, 45) and studies have shown the importance of spatiotemporally coherent visuo-vestibular cues for successful control of self-motion (Refs. 5, 46). However, discrepancies (e.g., non-coherence or noise) in the ambient arrays can lead to an erroneous sense of height, orientation, or speed, with dramatic consequences such as loss of control.

Decreasing altitude in a helicopter generates both optical (e.g., variations in the flow structure) and inertial (variations in the GI structure) changes. In a nominal situation (e.g., no wind, good weather, daylight) variations in optical and GI structures are continuously congruent. But in more challenging situations, such as when landing in a desert, the sand lifted by the rotors often creates a condition in which ground textural cues are absent and the horizon is indistinguishable (Ref. 47). This sudden interruption of visual stimulation without affecting vestibular stimulation creates

unnatural covariations between the two senses, and in this context, pilots are often unable to efficiently control their altitude and self-motion (Ref. 48). These observations are theoretically grounded into two theoretical approaches, the sensory integration approach, and the ecological approach.

According to the sensory integration approach, the various cues sampled by our senses are combined to produce an integrated percept allowing us to successfully interact with our environment. Because of the variability of sensory cue reliability (due to environmental variations, or errors in sensory detection), this theoretical framework proposes that cue integration depends on probabilistic inferences (Ref. 49).

One version of this approach is the sensory weighted approach, which proposes that each sensory cue is weighted based on this reliability, and that weight depends on integration patterns derived from the Bayesian probability theory (Refs. 6, 50, 51, 52).

In the ecological approach, the interaction with the environment is directly specified in the covariations of the flow structures detected by the various senses. The intermodal theory of perception (Ref. 53) proposes that variations in the optical structure reaching the eyes of the pilot and variations in the gravitoinertial structure stimulating their vestibular system are simultaneously specified in a higher-order structure called the Global Array (GA). The GA is a structure that extends across multiple forms of ambient energy. Higher- order invariants existing in the GA have been demonstrated to be responsible for the perception and control of reaching (Ref. 20), but to our knowledge it remains to be discovered in the context of NOE flight.

Nevertheless, and in the case of ego motion, visual information is physiologically dominant, and it is now established that the vestibular system plays a key role with this sensory dimension for the coding of information of the environment and the subject with respect to it (Ref. 45, 54).

In 2010, Fetsch and his colleagues (ref. 44) explored visual- vestibular integration by introducing disparities in vestibular inputs (moving platform) and visual inputs (optical flow).

They demonstrated a weighting of visuo-vestibular sensory inputs according to their reliability (sensory motion vectors are combined by additive or weighted) where vestibular information is attenuated when the visual information is of high relevance for body movement coding summation (see also modality appropriateness hypothesis for vision and audition, Ref. 54, 55). Thus, in GVE conditions, the visual information available in the external environment can be sufficient for the pilot to characterize his own movement and attitudes with respect to the terrestrial reference. On the other hand, in DVE conditions, when visual information can be very limited (e. g. entry into a cloud layer, night or brownout), the acquisition of information about the terrestrial reference is hindered and even the most experienced pilot may be unable to properly assess (consciously or not) the attitudes of his aircraft. Furthermore, the reliability (signal-to-noise ratio) of these cues can vary rapidly and unpredictably, because of environment changes or because of sensory encoding error.

If, from an evolutionary point of view, the vestibular system

is completely adapted to the earth's motion, it does not follow

that it is well-adapted to the aeronautical environment and

may constitute a major physiological component of the SD (Ref. 56). SD is therefore due to the functional inability of the vestibular system to inform the operator about his/her own motion when visual information is insufficient, given certain kinetic condition. Furthermore, visual-vestibular integration is a multisensoriality topic that is relevant to numerous aeronautical domains.

The present study

Testing the visual (optical) and vestibular (gravitoinertial) contributions (and their interactions) in a successful performance during NOE flight requires manipulating the perceptual environment of the pilot. Due to the high-risk level and the difficulty in controlling environmental variables (e.g., wind or luminosity), this manipulation is difficult in real life.

Motion-based flight simulators are widely used to provide an experimentally controlled environment. They can recreate, to a certain extent, natural flying scenarios with (limited) inertial and optical changes. The NASA Ames Research Center vertical motion simulator (VMS) provided the ideal platform to evaluate the contribution of optical and gravito-inertial information, as well as their interactions, on the perception of height. Two conditions were tested. In the first condition, pilots were asked to passively report their perceived height above the ground while moving in a simulated NOE flight in AP mode.

In the second condition the pilots had to actively regulate their height as if they were in a real NOE situation. These two conditions require different mechanisms. The first relies on the integration of relative cues during visual perception decoupled from action, while the second involves a continuous control-oriented action-perception task. Each condition was flown at 35 knots and 55 knots, in GVE and DVE, and with and without cabin motion.

Here, we present the results of the passive condition flown at 55 knots, where the pilots had to report their perceived height above the ground while moving in simulated NOE in AP mode. Assuming transitivity between passive and active observer perceptive mechanisms, the hypothesis was that pilots would produce a better performance in the presence of congruent visual and gravitoinertial stimulation rather than during visual stimulation alone. It was also posited that when the visual information was compromised in DVE or when flying higher, the contribution of the gravitoinertial cues would be more heavily weighted.

Methods Participants

A total of seven male pilots from the US Army (one research instructor pilot, three experimental test pilots, two research pilots and one instructor pilot) aged 27 to 57 (mean 37.5 years) participated in the experiment. Flight hours varied between 560 hours and 7300 hours (mean 2736 hours) and simulator experience between 100 hours and 1000 hours (mean 365 hours). All had flown NVG / DVE conditions (40 to 1500 hours, mean 765 hours).

Figure 2. NASA Ames Vertical Motion Simulator.

Figure 3. R Cab cockpit Field of view.

Figure 4. R Cab cockpit emulating a utility class UH-

60 sized helicopter.

The Simulator

The experiment was carried out on the NASA Ames Research Center VMS, an uncoupled six-degree-of-freedom (three translational and three rotational) motion simulator (Figures 2, 3, 4). The distinctive feature of the VMS is its unequaled large amplitude, high fidelity motion capability. It was equipped with a R-cabin emulating a utility class UH-60 sized helicopter, with an out the window (OTW) field of view representative of that class of vehicle. Two gravitoinertial conditions were tested, one with cabin motion (the gravitoinertial profile is the double derivative of the terrain profile) and one without cabin motion (the visual environment only is optically in motion).

The Visual Display

The out-the-window visual scene was generated by a Rockwell-Collins EPX-5000 image generation (IG) system providing a high-resolution and complex visual environment at update rates exceeding 60Hz. The visual scene was presented on only the top three windows (the chin window was not used to prevent the ground from being viewed and used to assess the altitude). The horizontal field of view spans +-78 degrees and the vertical field of view spans -16 to +12 degrees for the upper three windows, as shown in Figure 3.

The virtual Environment

The virtual visual environment characteristics, terrain profile, flight and Drift parameters are summarized in Table 1 and Figures 5, 6, 7 and 8. The virtual environment started as a flat ground surface that was traversed in 9 seconds followed by a short ascent (1.8 sec) that ended Phase 1. This phase was followed by a 21.8 sec period repeated three times, forming a succession of hills. The total length of a trial was 90 sec.

Figure 5: Terrain profile for phase 1 (distance travelled at 55 knots).

Figure 6: Terrain profile for phase 2, 1 block, repeated 3 times (distance travelled at 55 knots).

Figure 7. Example of out-the-window optical fields as a function of terrain profile. Top to Bottom: GVE and

DVE. Left to Right: 27ft AGL and 63 ft AGL. Optical vector fields (red arrows) are superimposed on the simulation image to represent the differences in visual

cues available in the different configurations.

Table 1. Visual environment, terrain profile, flight, and drift parameters.

Parameters Tree height (all same

color) 18 𝑓𝑡. (+/- 1 ft.)

Tree canopy diameter 15 𝑓𝑡. (+/- 1ft.)

Tree density 193/square mile

Height (altitude) initialization (Pilot eye- level)

45𝑓𝑡. (2,5 𝑡𝑟𝑒𝑒𝑠) 𝐴𝐺𝐿 ± 𝑟𝑎𝑛𝑑𝑜𝑚 ∗ 4.5 with 0 ≤

𝑟𝑎𝑛𝑑𝑜𝑚 ≥ 1 Height (altitude) above

tree 27𝑡.

Phase 1: Plateau altitude 78.7𝑓𝑡 Phase 1: Plateau length 843.9 𝑓𝑡. (9.1 𝑠𝑒𝑐) Phase 1: Ascent length/

angle of attack 168.78 𝑓𝑡. ( 𝑠𝑒𝑐), 25°

Phase 2: Plateau low 0 𝑓𝑡.

Phase 2: Plateau high 157.4 𝑓𝑡.

Phase 2: Plateau length 675.12 𝑓𝑡. (10 𝑠𝑒𝑐) Phase 2: Ascent/

Descent length/ angle of attack

168.78 𝑓𝑡. ( 𝑠𝑒𝑐), 25°

Drift magnitude and Direction, randomly presented 10 to 55 seconds after the beginning of phase 2)

±18 𝑓𝑡. (+/- 1 ft.), randomized positive slope, negative slope, plateau low

Initial speed

(Independent Variable) 55 𝑘𝑛𝑜𝑡𝑠

Block Trial length 6413.67 ft, 90 seconds

Because heading was maintained constant (rectilinear motion), and no pitch was involved, the optical flow was generated by a strictly forward translation. Therefore, variations in the optical flow filed were only induced by the terrain variations, the meteorological conditions, and the flight level.

Previous research has shown that performance in simulated NOE tasks (altitude maintenance) is related to variations in both global object density, object height an object radius (Refs. 28, 32). To control for these effects, the terrain was populated with 193 identical trees/ square mile randomly distributed (Figure 5) to maintain the same density gradient (number of threes per unit solid visual angle) throughout the entire trial. The threes height and canopy diameter were maintained constant to prevent differences in the magnitude of visual occlusion (refer to Table 1 for details). A patched texture was layered over the profile. A mountainous background surrounded the environment.

Two conditions were contrasted: vision with no fog (Good Visual Environment, GVE) vs. degraded visual environment with fog (DVE, ¼ mile visibility). For each trial and each pilot, the initial flight level (pilot’s eye level) was set to a randomized value of 45 ft AGL +/- (4.5 ft * random), with 0<=random<=1; min=40.51, max=49.44.

The Simulation Scenario A Pseudo Autopilot

Because an auto pilot model was not available at the time of the experiment, a “pseudo” autopilot (AP) trajectory was simulated using the recording of an experienced pilot who was instructed to fly above the terrain at a constant 45 ft height and constant speed of 55 knots (see Figure 8, top). Figure 8 (middle), illustrates the altitude AGL and the speed during the recorded flight. It can be seen, as expected, that the pseudo AP trajectory was affected by the terrain profile. Indeed, height estimation over flat textured terrain is relatively easy and height perceptual resolution are roughly inversely related.

Estimating height above sloped terrain during ascent is more challenging as it requires: 1) Distance estimation from the nearest visible point on the slope; 2) Estimation of the slope associated with the reference point established in (1); 3) Recollection of the terrain contour that is no longer in view when the gradient of that terrain changes from level to sloped;

4) Mental projection of distance along the overrun (unseen) terrain contour based on perceived groundspeed. Height estimation when established in a descent is also challenging, requiring: 1) Recollection of the terrain contour that is no longer in view when the gradient of that terrain changes from level to sloped; 2) Mental projection of horizontal distance flown beyond the crest of the hill until the descent begins; 3) Estimation of descent rate based on distant line of sight (LOS) cues groundspeed (see Figure 8 bottom); 4) Integrating #2 and

#3. Because the slope during the descent is never in view, height can only be inferred via #4 –and prone to inaccurate height perception.

Figure 8. Recorded flight (pseudo autopilot, AP) profile without drift for an entire trial. Top: Terrain profile and pseudo Ap profile, Middle: AP altitude AGL (ft) and ground speed (GS, knots), and Bottom, AP Line of Sight (LOS). Note the variations in height induced by

the nature of the terrain.

Figure 9. Top. 18 ft Downward drift temporal profile over 5 sec. Middle: RADAR altitude over terrain after the Downward drift (AP for reference). Bottom: Height

AGL after the Downward drift (AP for reference).

The Drift

A +/- 18 ft vertical drift (+/- 1.8 ft * random, with 0<=random<=1) representing ≅ 40% of the initial altitude, was introduced randomly 10 to 55 seconds after the beginning of phase 2. It can be seen from Figure 10 (top) which depicts the temporal drift profile (downward in this example) that it takes approximately 5 secs to be completed (50% of the magnitude of the drift is achieved after 2.5 sec). In Figure 9, middle and bottom, one can see an example of a downward drift introduced at the beginning of a high plateau. RADAR altitude is the simply the sum of the initial AP height and the 18ft drift. The amplitude, direction and time of the drift were unknown to the pilot. The altitude remained then constant until the end of the trial.

Experimental Matrix

The experiment followed a full-factorial repeated-measures design (all the pilots experienced all the conditions) with two within subject factors, Visibility (GVE, DVE) and Cabin Motion (ON, OFF), giving rise to 4 experimental blocks:

Visual [GVE, DVE] * Gravitoinertial [ON, OFF]. Each block contained 5 trials, divided randomly between two drift directions, Upward vs. Downward. The experiment consisted in 20 trials per participant, each block (Visibility * Gravitoinertial) counterbalanced between participants. Each trial lasted on average 2 minutes (90 seconds trial + reconfiguration).

Table 2. Experimental design: Two within subject factors, Visibility (GVE, DVE) and Cabin Motion (ON,

OFF)

Supervisory Control (Perception) condition

GVE DVE

Gravitoinertial ON 5 5

Gravitoinertial OFF 5 5

Procedure

Participants were passive observers of a pseudo automated NOE flight. They had no control over the simulated motion.

They were instructed that they were observing a pre-recorded flight with inherent small variations in height, and that an 18ft vertical drift, upward or downward simulating a vertical wind shift would be introduced during the Phase 2 of the flight.

Their task was to report their perceived height above the ground using a cursor on a vertical tape, from 0ft to 120 ft, displayed on the HMD window, controlled by the collective position (see Figure 10). The tape was located 45 degrees and 5.25 inches from the center of the FOV (lower right quadrant).

At the beginning of the trial, the aircraft position was set at a random height (45 ft +/- 4.5 ft) unknown to the pilot. The cursor’s initial position was also set at a random position (45 ft +/- 4.5 ft), uncorrelated to the aircraft’s initial altitude.

Figure 10. Participant in situ using a cursor on a vertical tape displayed on the head mounted display (HMD) window to report his perceived altitude. The

cursor was controlled by the collective position.

The speed was nearly constant throughout the trial (55 knots), as shown in Figure 9. Pilots did not have access to instruments or OTW cues from the lower side window.

Data Analysis

Profile segmentation and Pre- and Post-Drift Cycles

Because the terrain profile had an impact on AP (see Figure 11) and also because the drift could occur within any of each four segments of a profile period, it was necessary to isolate the effects of the terrain to evaluate the true effects of the drift.

Figure 11. Pre-Drift (C0) and Post-drift Cycles (C1 and C2). To minimize the effects of terrain, the time window analysis was set to 5 sec after the beginning of a

cycle. Here, an Upward Drift was introduced 42.5 sec after the beginning of the trial. The dashed line indicates

the AP altitude without the drift for reference.

A two-step process was applied. First, a pre-drift region

(referred to as Cycle 0) and two post-drift regions (referred to

as Cycle 1 and Cycle 2) were determined based on the time of

the drift, and the period of the terrain. For example, as

illustrated in Figure 11, if the drift occurred during a descent,

it would end at the same descending terrain location one

terrain period later. The second cycle starting approximately

21.8 sec after the drift (see Figure 6) would follow the same

rules. Similarly, the pre-cycle would start at the exact same

locus, approximately 21.8 sec before the drift. To further

minimize the effects of terrain, the time window analysis was

set to 5 sec after the beginning of a cycle as seen in Figures

11 and 12.

Figure 12. Detail of the 5sec time windows for pre- (Cycle 0) and post-Upward drift (Cycle 1 and Cycle 2).

Top: RADAR altitude, pseudo AP altitude (dashed line) and Terrain. Note the 18ft difference between RADAR altitude and AP at the end of Cycle 1 and during Cycle 2.

Bottom: RADAR altitude, pseudo AP altitude (dashed line) and tracking height. In this example, during the

first 5 sec of Cycle 1, the tracking altitude is below RADAR altitude and virtually superimposed on the pseudo AP trajectory, suggesting that the pilots did not

perceive the drift at that time. At the end of the 5sec Cycle 2 window, the distance between the tracked trajectory and the actual trajectory decreases, indicating

a possible detection of the drift.

Quantitative Measures of Performance Tracking Error

Mean Biased Error (MBE) expresses the quality of the tracking error and was selected to provide an estimate of the quality of the response, i.e. an overshoot or an undershoot of the RADAR altitude. Mean Absolute Error (MAE) is the unsigned (unbiased) tracking error and provides an estimate of the magnitude of the differences between the RADAR altitude and the tracking height. Measures of variability was expressed by the Standard Deviation (SD, 𝜎). The variance (𝜎 ) was then used to produce an estimation of the redundancy gain, RG. All the metrics were calculated using a 150 msec moving average window.

Response Times (RTs)

Because pilots were not asked to report when they first perceived the perturbation drift, a methodology was designed to infer the perception/reaction time based on the differences between AP altitude (without drift), RADAR altitude (with drift) and tracking height. One way to assess whether the participant detected the perturbation is to compare the error between the tracking height and both the RADAR altitude and

the AP altitude (RADAR altitude minus 18ft drift). One can see in Figure 13 an illustration of the method. After the beginning of the drift, AP and RADAR curves start to separate to reach a maximum of ≅ 18ft after 5 sec (Figure 13, Top). The tracking error relative to RADAR altitude and AP altitude was computed and is plotted in Figure 13, Center.

Figure 13, Bottom, shows the mean absolute tracking error in reference to AP altitude and RADAR altitude.

Figure 13. Illustration of the methodology used to determine indirectly the detection of the drifts. In this example, a Downward drift is introduced 54 3 sec after

the beginning of the trial. Top: RADAR altitude, tracking altitude and AP altitude (RADAR minus Drift).

Center: Biased (signed) tracking error in relation to RADAR altitude and AP altitude. Bottom: Absolute tracking error in relation to RADAR altitude and AP

altitude. Response Time (here 11.5 sec after the beginning of the drift) is determined by the crossover

between the RADAR and AP curves.

One can clearly see the crossover between the two curves:

before the crossover, tracking error is lower in relation to AP than to RADALT, indicating that the pilot did not detect/ or react to the drift. After the crossover, the tracking error is lower in relation to RADALT than in relation with AP, indicating a detection of the drift. It is likely that the RTs were identified a few seconds after the actual detection, at the maxima of the curve before the crossover. In some instances, for Upward drifts, the tracking height varied little after the introduction of the drift, while the tracking height curve remained largely superimposed over the AP curve. In these cases, a way to determine the RTs was to look at the direction of the error locally, i.e. the sign of the error around the reference curve (RADALT or AP). A change preceded and followed by a sustained constant sign was interpreted as a perturbation detection.

Lastly, we provided a measure of multisensory integration (MSI) by calculating the redundancy gain (RG, Charbonneau et al., 2013, Godfroy-Cooper et al., 2015), assuming that the visual cues (𝑉) are, in general, more effective than the gravitoinertial cues (𝑉𝐺):

𝑅𝐺 = 𝜎

𝜎 ∗ 100

where 𝜎 represents the variance (𝑆𝐷 ) of the estimate.

Specifically, this measure relates the magnitude of the response to the multisensory stimulus to that evoked by the more effective of the two modality-specific stimulus components. According to the principle of inverse effectiveness (IE), the reliability of the best sensory estimate and RG are inversely correlated, i.e., the less reliable single stimulus is associated to maximal RG when adding another stimulus. All the metrics were calculated using a 150 msec moving average window.

DIMSS

The Dynamic Interface Modeling and Simulation System Product Metric (DIMSS PM, Refs. 57, 58) is a pilot workload metric that was used to evaluate helicopter shipboard launch and recovery. The metric is composed of the product of the number of control reversals and the standard deviation of control deflections in a moving three-second window (control motion is filtered at 3.3 Hz). Here, stick activity directly correlates with the tracking activity. It was calculated on a sliding window [𝑊𝑛; 𝑊𝑛 + 1] as follow:

𝐷𝐼𝑀𝑆𝑆[𝑤 ; 𝑤 ] = (𝑁𝑏 + 𝑁𝑏 ) ∗ 𝑆𝑡𝑑(𝑠𝑡𝑖𝑐𝑘[𝑤 ; 𝑤 ]) where (𝑁𝑏 + 𝑁𝑏 ) is the number of control reversals with the stick multiplied by the standard deviation ( 𝑆𝑡𝑑 ) of the stick value; 𝑁𝑏 and 𝑁𝑏 being respectively the number of local maxima and the number of local minima on the stick value curve. The higher the value, the less stable the joystick control. A 0 value means that pilots did not move the joystick.

In order to obtain a more qualitative comprehension of the pilots’ behavior, we also investigated the two components of the DIMSS variable, i.e., the standard deviation of the collective stick value (𝑆𝑇𝐷 𝐶𝑂𝐿) and the number of control reversals (𝑁𝐶𝑅), which is an estimate of the frequency of the stick motion. We calculated mean and standard deviation values for these two variables in each condition, on a specific temporal window.

Univariate and repeated-measures analyses of variance (ANOVAs) (SPSS) were used to test for effects of Cycle (within-subject-factor) and Visibility, Motion and Drift Direction (between-subjects variables). Means (𝜇) Standard Errors (𝑆𝐸) and/or Standard Deviations (𝑆𝐷) were computed for the pre- (C0) and the two post-drift cycles (C1 and C2), described previously in section “Profile segmentation and Pre- and Post-Drift Cycles”. Post-hoc Bonferroni test were performed for multiple comparisons. All the effects described here were statistically significant at p < 0.05 or better. Outliers (7.9 % of the data) were removed using Mahalanobis distance procedure for multivariate data (3 variables). The threshold value of 0.001 was suggested by Tabachnick & Fidell (2007), who state that a very conservative probability estimate for outlier identification is appropriate for the Mahalanobis Distance.

RESULTS

Accuracy

Figure 14 summarizes the results for MBE and MAE for the four conditions, (GVE, DVE) * (Cabin Motion ON, cabin Motion OFF) as a function of the Drift Direction, UP vs.

DOWN, and the trial Cycle, Pre- (C0) or post-Drift (C1, C2).

Three-way repeated measures mixed-design Analysis of Variance (ANOVA) were performed for MBE and MAE.

For MBE, there was a significant effect of Cycle (𝐹

= 16.69, 𝑝 < .0001), a significant effect of Drift Direction (𝐹

= 23, 𝑝 < .0001), and a significant effect of interaction between Cycle and Drift Direction (𝐹

= 90.30, 𝑝 < .0001). There was no effect of Motion or Visibility.

For MAE, there was no overall effect of Cycle, but a significant effect of Drift Direction (𝐹

= 6.08, 𝑝 = .01), and a significant effect of interaction between Cycle and Drift Direction (𝐹

= 27.14, 𝑝 < .0001). There was no effect of Visibility or Motion, but there was a significant effect of interaction between Drift Direction, Visibility and Motion (𝐹

= 12.93, 𝑝 < .0001).

Effect of Drift Direction (Upward vs. Downward)

Figures 15 and 16 show the evolution of the mean biased error

(MBE) and the mean absolute error (MAE) before (C0) and

after the drift (C1 and C2) as a function of its direction,

Upward vs. Downward.

Figure 14: Top: MBE and Bottom: MAE for the four experimental conditions (Visual * Gravitoinertial) as a function of Drift Direction for the pre-(C0) and post-drift

cycles (C1, C2).

Figure 15: Mean Biased tracking Error for Pre- (C0) and Post-Drift (C1, C2) as a function of the Drift Direction, Upward vs. Downward. Before the drift, pilots underestimate their altitude by on average 10 ft.

Five sec after an 18 ft Downward drift, the perceived altitude is almost veridical (+.74 ft). After an 18 ft Upward drift, pilot’s underestimation increased by more

than 6 ft.

Figure 16: Mean Absolute Error for Pre- (C0) and Post-Drift (C1, C2) as a function of the Drift Direction,

Upward vs. Downward.

Tracking accuracy was strongly affected by the disturbance, in a way that was qualitatively (MBE) and quantitatively (MAE) different as a function of the drift direction. The tracking error significantly increased after an Upward drift while it significantly decreased after a Downward drift.

Before the drift (C0), pilots underestimated the radar altitude by on average 9.52 ft (SD=13.24 ft). After an 18 ft Upward drift, the underestimation increased, and pilots reported flying on average 11.78 ft lower during C1 and 14.68 ft lower during C2, differences that were both statistically significant (C0, C1: 𝑡 = 3.36, 𝑝 < .0001; C1, C2: = 2.97, 𝑝 = .001).

Conversely within 5 seconds after an 18 ft Downward drift, the estimate of the tracking height is almost veridical and the tracking error is almost null (.74 ft) (C0, C1: 𝑡 = −11.53, 𝑝 <

.0001). One cycle later, the tracking altitude is slightly overestimated (1.06 ft) (C1, C2: = −.27, 𝑝 = 1).

In terms of absolute error (MAE), performance decreased by 17.7% between Co and C1 (C0, C1: 𝑡 = −2.01, 𝑝 = .01) and by an additional 15.7% between C1 and C2 (C1, C2: 𝑡 =

−2.20, 𝑝 = .01) for Upward drifts, for a total of 35%. Note that the magnitude of the differences between C0 and C1 on one hand and C1 and C2 on the other hand is very similar, suggesting that only a partial detection of the drift occurred during C1. For Downward drifts MAE decreases by 31%

between C0 and C1 and by an additional 21.1% between C1 and C2., for a total of 45%. Note here that 68% of the total performance enhancement occurred within the 5 first seconds after the drift, which suggest an early detection of the drift.

Effect of Visibility and Motion

Because there was a significant effect of interaction between Drift Direction, Visibility and Motion, the effects of Visibility and Motion were assessed separately for the Upward and Downward drifts directions.

Upward Drifts

For MBE, there was a significant effect of Cycle (𝐹

=

29.65, 𝑝 < .0001) as well as a significant effect of interaction

between Visibility and Motion (𝐹

= 4.38, 𝑝 = .04).

Figure 17 Top: MBE and Bottom: MAE in GVE (left) and in DVE (right) for the pre- (C0) and post-drift

cycles (C1, C2) as a function of the gravitoinertial condition, Cabin Motion On, Cabin Motion OFF.

Figure 18: Top: MBE and Bottom MAE in GVE (left) and in DVE (right) for the pre- (C0) and post-drift

cycles (C1, C2) as a function of the gravitoinertial condition, Cabin Motion On, Cabin Motion OFF.

For MAE, there was a significant effect of Cycle (𝐹

= 15.72, 𝑝 < .0001) as well as a significant effect of interaction between Visibility and Motion (𝐹

= 5.54, 𝑝 = .02). In GVE, the underestimation of altitude was not different between Gravitoinertial conditions, i.e. cabin motion ON or OFF. There was no effect of interaction with Cycle. In DVE, the effect of Motion was significant, and pilots were reporting flying higher (less undershoot, as seen in Figure 17) with cabin motion than without cabin motion (ON: 𝜇 =

−9.68, 𝑆𝐸 = 2.44, OFF: 𝜇 = −17.36, 𝑆𝐸 = 2.79, 𝐹

= 4.27, 𝑝 = .04), although still underestimating their actual altitude.

For MAE, the difference between cabin motion ON and cabin motion OFF didn’t reach significance (ON: 𝜇 = 12.48, 𝑆𝐸 = 1.87, OFF: 𝜇 = 17.69, 𝑆𝐸 = 2.14 𝐹

= 3.33, 𝑝 = .07).

One can also see that pilots reported flying closer to the ground in DVE than in GVE when the cabin motion was OFF (MBE: GVE: 𝜇 = −9.07, 𝑆𝐸 = 2.20, DVE: 𝜇 =

−17.36, 𝑆𝐸 = 2.59 𝐹

= 6.03, 𝑝 = .02), which is associated with a larger tracking error in reference to the RADAR altitude (MAE: GVE: 𝜇 = 10.70, 𝑆𝐸 = 1.94, DVE:

𝜇 = 17.69, 𝑆𝐸 = 2.28 𝐹

= 5.41, 𝑝 = .2). When the cabin motion was ON, there was no significant difference in tracking error in GVE and DVE, suggesting that gravitoinertial cues mitigated the effects of the degradation of the visual cues.

Downward Drifts

In GVE, for MBE there was no significant effect of Motion and no interaction between Cycle and Motion. MAE was significantly lower when motion cues were present, as seen in Figure 18 (ON: 𝜇 = 9.07, 𝑆𝐸 = 1.39, OFF: 𝜇 = 13.69, 𝑆𝐸 = 1.43, 𝐹

= 5.30, 𝑝 = .02). In DVE, MBE and MAE were not affected by motion.

Precision (Variability)

A multivariate repeated-measures ANOVA showed a marginally significant effect of Cycle (𝐹

= 2.75, 𝑝 = .06), and a significant effect of interaction between Drift Direction, Motion and Visibility (𝐹

= 8.09, 𝑝 = .005).

Upward Drifts.

For Upward drifts (Figure 20), the effect of Cabin Motion,

ON vs. OFF was significantly different for GVE and DVE

conditions (Cabin Motion * Visibility: 𝐹

= 8.80, 𝑝 =

.004). In GVE, SD was higher in the presence of

gravitoinertial cues (ON: 𝜇 = 6.93, 𝑆𝐸 = .66, OFF: 𝜇 =

4.96, 𝑆𝐸 = .66, 𝐹

= 4.14, 𝑝 = .05). The effect of Cycle

was not significant and there was no effect of interaction with

motion. In DVE, The effect of motion was the opposite of that

observed in GVE, and STD was lower when motion cues were

available (ON: 𝜇 = 5.09, 𝑆𝐸 = .31, OFF: 𝜇 = 7.18, 𝑆𝐸 =

.72, 𝐹

= 4.77, 𝑝 = .03). The effect of Cycle did not quite

reach significance, and there was no effect of interaction with

Motion. When Cabin Motion was OFF, STD was

significantly larger in DVE than in GVE (𝐹

= 5.21, 𝑝 =

.03).

Figure 19: STD for the four experimental conditions (Visual * Gravitoinertial) as a function of Drift Direction

for the pre-(C0) and post-drift cycles (C1, C2).

Figure 20: Upward Drifts. Mean Standard Deviation (Std) for Pre- (C0) and Post-Drift (C1, C2) as a function

of the Cabin Motion, ON vs. OFF.

Figure 21: Downward Drifts. Mean Standard Deviation (Std) for Pre- (C0) and Post-Drift (C1, C2) as a

function of the Cabin Motion, ON vs. OFF.

When Cabin Motion was ON, there was no significant difference in STD in GVE and DVE, suggesting that gravitoinertial cues mitigates the effects of DVE (𝐹

= 3.67, 𝑝 = .06).

Downward Drifts

In GVE, no significant effect of motion no effect of cycle and no effect of interactions. In DVE, no significant effect of motion no effect of cycle and no effect of interactions (Figure 21). When cabin motion was OFF, STD was higher in GVE than DVE (GVE: 𝜇 = 8.66, 𝑆𝐸 = .90, DVE: 𝜇 = 6.17, 𝑆𝐸 = .80 𝐹

= 4.25, 𝑝 = .04), an effect that is the opposite of that observed for Upward drifts. When cabin motion was ON, there was no difference in STD as a function of the visibility condition, GVE vs. DVE.

Redundancy Gain

As described in detail in the Data Analysis section, the variance of the estimates, here the variance when only visual cues were available (unisensory condition, 𝜎 ), and the variance when visual cues where presented synergistically with gravitoinertial cues (multisensory condition, 𝜎 ) can be used to compute the redundancy gain (RG), assuming that, under GVE, visual cues are more heavily weighted than vestibular cues. The measure provides the opportunity to test the principle of inverse effectiveness (IE) according to which the reliability of the best sensory estimate (here vision) and RG are inversely correlated, i.e., the less reliable single stimulus (here gravitoinertial) is associated to maximal RG when adding another stimulus. In other words, In GVE, and after Downward drifts, where visual cues are optimal, the RG should be low, and conversely, when visual cues are degraded, in DVE and after Upward drifts, the RG should be the highest.

Table 3. Percentage of Bimodal Visual- Gravitoinertial Estimate Enhancement as a Function of

Visibility and Drift Direction.

GVE DVE

UP 33.3% 78.6% 𝑥 = 5.41, 𝑝

= .02

DOWN 66.7% 64.3% 𝑥 = .01, 𝑝

= .89 𝑥 = 2.66, 𝑝

= .1 𝑥 = .7, 𝑝

= .4

Table 4. Percentage of Bimodal Visual-Gravitoinertial Estimate Enhancement as a Function of Visibility and

Drift Direction for C1 and C2.

GVE DVE

UP DOWN UP DOWN

C1 50.0% 66.7% 85.7% 71.4%

C2 16.7%

66.7% 71.4%

57.1%

When the variance in the bimodal visual-gravitoinertial

condition was lower than that in the unimodal visual

condition, it was categorized as performance enhancement.

Conversely, when the variance in the bimodal visual- gravitoinertial condition was higher than that in the unimodal visual condition, it was categorized as performance Depression. Table 2 summarizes the percentages of performance enhancement as a function of Visibility condition, GVE vs. DVE and Direction of the Drift, Upward vs. Downward. A percentage <50% represents a depression, and >=50% an enhancement. One can see that Enhancement was maximum for Upward drifts in DVE, a difference that was significant with the distribution of the estimates in GVE.

A decomposition of the performance as a function of the Post- Drift Cycles shows that performance enhancement was overall higher during C1 than during C2 and maximum (85.7%) for Upward drifts in DVE (see Table 3). This result is in agreement with the principle of IE. However, we did not observe a lower RGs when visual conditions were optimal, i.e. in GVE for Downward drifts.

Response Times (RTs)

A multivariate ANOVA showed a significant effect of Drift Direction (𝐹

= 6.88, 𝑝 = .01), and a significant effect of interaction between Drift Direction and Motion (𝐹

= 6.12, 𝑝 = .01).

Figure 22: RTs for the four experimental conditions (Visual * Gravitoinertial) as a function of Drift Direction.

One can see that the contribution of the gravitoinertial cues was the strongest in DVE and when the visual level

of detail was low (Upward drift condition).

RTs were significantly longer for Upward drifts than for Downward drifts, as seen in Figure 22 (Downward: 𝜇 =6.17 sec, SE=.77; Upward: 𝜇 = 9.05 𝑠𝑒𝑐, SE= .77, 𝐹

= 6.88, 𝑝 = .01). There was no overall effect of Visibility or Motion, but there was a significant effect of interaction between drift direction and Motion (𝐹

= 6.12, 𝑝 = .01). For Depression drifts, RTS were statistically equivalent with or without cabin motion (ON: 𝜇 =6.89 sec, SD=6.85; OFF:

𝜇 =5.54 sec, SD=5.13; 𝐹

= .82 𝑝 = .36). For an Elevation drift, RTs were shorter with Motion ON than with Motion OFF (ON: 𝜇 =7.06 sec, SD=4.88; OFF: 𝜇 =10.57 sec, SD=7.88; 𝐹

= 4.66, 𝑝 = .03). In GVE, there was no effect of Motion or drift direction. In DVE, conversely, there

was a significant effect of Drift direction (𝐹

= 5.89, 𝑝 = .01), and a significant effect of interaction between Drift direction and Motion (𝐹

= 4.05, 𝑝 = .04). One can see from Figure 22 that in conditions where Visibility was degraded and visual cues low (Upward drifts), the presence of motion cues reduced the detection / reaction time to the drift by an average of 6.61 seconds, a difference that can be critical in NOE flight (DVE, Upward drift: OFF: 𝜇 =13.95 sec, SD=9.3; ON: 𝜇 =7.34 sec, SD=5.65; 𝐹

= 5.81, 𝑝 = .02).

These results are in line with the response enhancement observed previously for the variance: gravitoinertial cues play a stronger role in DVE and when the quality of the visual cues is low, i.e. in Upward drifts conditions.

DIMSS

A repeated measures ANOVA showed a significant effect of Cycle (𝐹

= 89.29, 𝑝 < .0001), as well as a significant effect of interaction with Drift Direction (𝐹

= 3.99, 𝑝 = .02). There was no effect of Motion or Visibility.

Figure 23: DIMSS PM for the four experimental conditions (Visual * Gravitoinertial) as a function of Drift Direction for the pre-(C0) and post-drift cycles (C1,

C2).

Figure 24: DIMSS for Pre- (C0) and Post-Drift (C1, C2) as a function of the Drift Direction, Upward vs.

Downward.

DIMSS significantly decreased after the disturbance, as seen in Figure 24, both between C0 and C1 (𝐹

= 42.81, 𝑝 <

.0001) and C1 and C2 (𝐹

= 97.35, 𝑝 < .0001). DIMSS decreased essentially between C0 and C1 for Depression drifts (70%) while it decreased essentially between C1 and C2 for the Elevation drifts (58%).

Number of Control Reversals

A repeated measures ANOVA showed a significant effect of Cycle (𝐹

= 68.31, 𝑝 < .0001). Effects of Drift direction, Visibility and Motion were not significant. There was no effect of interaction.

Figure 25: Number of Control Reversals (NCR) for the four experimental conditions (Visual * Gravitoinertial) as a function of Drift Direction (Upward

vs. Downward) for the Pre-(C0) and Post-Drift cycles (C1, C2).

The Number of Control Reversals significantly decreased after the perturbation, C0, C1 (𝐹

= 193.36, 𝑝 < .0001) and C1, C2 (𝐹

= 114.27, 𝑝 < .0001). Although the difference was not significant, one can see from Figure 25 that the difference in NCR between C1 and C2 was greater for Upward than for Downward drifts.

Standard Deviation of the Collective

A repeated measures ANOVA showed a significant effect of Cycle (𝐹

= 14.47, 𝑝 < .0001), and a significant effect of interaction between cycle and Drift direction (𝐹

= 6.53, 𝑝 = .002). There was no effect of Visibility of Motion.

For Upward drifts, the standard deviation of the collective (STD COL) was not significantly different between C0 and C1, but decreased between C1 and C2 (C0: 𝜇 =6.89, SD=5.10; C1: 𝜇 =6.26, SD=4.92; C2: 𝜇 =5.34, SD=5.01; C0, C1: 𝑡 = .63, 𝑝 = .22; C1, C2: 𝑡 = .92, 𝑝 = .03; C0, C2: 𝑡 = 1.55, 𝑝 = .001). Conversely, for Downward drifts, SDT COL increased significantly between C0 to C1 and then, decreased significantly between C1 and C2 (C0: 𝜇 =7.06, SD=4.79; C1: 𝜇 =8.49 SD=4.61; C2: 𝜇 =6.26, SD=5.10; C0, C1: 𝑡 = −1.42, 𝑝 = .005; C1, C2: 𝑡 = 2.24, 𝑝 < .0001).

Figure 26: Standard deviation of the collective (STD COL) for the four experimental conditions (Visual *

Gravitoinertial) as a function of the Drift Direction (Upward vs. Downward) for the Pre-(C0) and Post-Drift

cycles (C1, C2).

Figure 27: Standard deviation of the collective (STD COL) for the Pre- (C0) and Post-Drift cycles (C1, C2) as a function of the Drift Direction, Upward vs. Downward.

Standard Deviation of the Collective Rate

A repeated measures ANOVA showed no overall effect of Cycle, no overall effect of Motion, Visibility or Drift Direction, but a significant effect of interaction between Cycle and Drift Direction (𝐹

= 7.87, 𝑝 < .0001).

It can be seen from Figure 29 that the Drift direction had a significant effect on the magnitude of the standard deviation of the collective rate (STD COL rate).

For Downward drifts, STD COL rate was not statistically different before and after the perturbation.

For Upward drifts, STD COL rate decreased significantly

between pre- drift (C0) and the first five seconds after the

drift, during C1 (C0: 𝜇 =4.41, SD=3.25; C1: 𝜇 =3.83,

SD=2.69; 𝐹

= 14.09, 𝑝 < .0001). However, there was no

significant difference between C1 and C2 (C1: 𝜇 =3.63,

SD=2.31, 𝐹

= 1.25, 𝑝 = .26).

Figure 28: STD COL rate for the four experimental conditions (Visual * Gravitoinertial) as a function of Drift Direction for the pre-(C0) and post-drift cycles (C1,

C2).

Figure 29: STD COL rate for Pre- (C0) and Post- Drift (C1, C2) as a function of the Drift Direction,

Upward vs. Downward.

Table 5. Correlations between Control Metrics.

NCR STD COL STD COL rate

C0 DIMSS .37** .778* .83**

NCR .26 -.11

STD COL .36

C1 DIMSS .14 .88** .72**

NCR -.26** -.18*

STD COL .78**

C2 DIMSS .28** .73** .77**

NCR .44* .39**

STD COL .46*

Table 4 show the correlations between the four metrics describing the collective activity during the altitude tracking task. One can see that sign of the correlation between NCR and STD COL changed between C0 and C1 and then between C1 and C2, i.e. NCR decreased while STD COL increased just after the drift, a trade-off that expresses the change in perception and cognitive uncertainty.

The complexity of the results for the number of reversals, linked to the levels of interactions, raises the question of the interindividual differences in control style between pilots, although the task was not pilotage. In a recent study, Bachelder et al. (Ref. 52) showed in a lateral station keeping task that pilots used different pulse techniques when controlling the acceleration and jerk commands on a gamepad joystick, including pulse modulation and pulse with amplitude modulation. Although the task was not a pilotage task, all the participants were experienced pilots and one cannot exclude the hypothesis of a transfer of control style between active flying and altitude tracking, in particular because the collective stick was used to perform the task.

Figure 30 shows the differences in the number of control reversals (NCR) and amplitude of the filtered collective between the time histories of the seven pilots in a GVE, Cabin Motion OFF configuration.

Figure 30: Filtered collective time history for the seven pilots. Peaks and troughs define the moments of reversals. Trade-off between Amplitude and Frequency.

From top to bottom: high frequency low amplitude to

low frequency, high amplitude.