Change Detection Using Synthetic Aperture Sonar: Preliminary Results from the Larvik Trial

HAL Id: hal-00645827

https://hal-ensta-bretagne.archives-ouvertes.fr/hal-00645827

Submitted on 28 Nov 2011

HAL is a multi-disciplinary open access archive for the deposit and dissemination of sci- entific research documents, whether they are pub- lished or not. The documents may come from teaching and research institutions in France or abroad, or from public or private research centers.

L’archive ouverte pluridisciplinaire HAL, est destinée au dépôt et à la diffusion de documents scientifiques de niveau recherche, publiés ou non, émanant des établissements d’enseignement et de recherche français ou étrangers, des laboratoires publics ou privés.

Change Detection Using Synthetic Aperture Sonar:

Preliminary Results from the Larvik Trial

Øivind Midtgaard, Roy Hansen, Torstein Sæbø, Vincent Myers, John Dubberley, Isabelle Quidu

To cite this version:

Øivind Midtgaard, Roy Hansen, Torstein Sæbø, Vincent Myers, John Dubberley, et al.. Change

Detection Using Synthetic Aperture Sonar: Preliminary Results from the Larvik Trial. OCEANS

2011 MTS/IEEE conference, Sep 2011, Kona, United States. �hal-00645827�

Change Detection Using Synthetic Aperture Sonar:

Preliminary Results from the Larvik Trial

Øivind Midtgaard, Roy Edgar Hansen, Torstein Olsmo Sæbø Norwegian Defence Research Establishment (FFI)

P O Box 25, NO-2027 Kjeller, Norway Oivind.Midtgaard@ffi.no

Vincent Myers

Defence R&D Canada, 9 Grove Street Halifax, Nova Scotia B2Y3Z7, Canada

John R Dubberley

Naval Research Laboratory, Code 7441 Building 1005 Stennis Space Center, MS 39529 USA

Isabelle Quidu

ENSTA Bretagne, E3I2 laboratory, 2 rue Franc¸ois Verny, 29806 Brest Cedex 9, France

Abstract— In April of 2011, FFI led a sea trial near Larvik, Norway on FFIs research vessel the H.U. Sverdrup II with participation by representatives from Canada, United States, and France. One objective of the sea trial was to acquire a data set suitable for examining incoherent and coherent change detection and automated target recognition (ATR) algorithms applied to Synthetic Aperture Sonar (SAS) imagery. The end goal is to produce an automated tool for detecting recently placed objects on the seafloor. To test these algorithms two areas were chosen, one with a comparatively benign seafloor and one with a boulder strewn complex seafloor. Each area was surveyed before and after deployment of objects. The survey time intervals varied from two days to eight days. In this paper we present the trial and show examples of SAS images and change detection of the images.

I. I NTRODUCTION

Change detection is the process by which objects of interest are detected by comparing current data with historical data. In the context of route survey with unmanned systems, this means repeatedly surveying ports and important transit routes with high frequency imaging sonar, geographically colocating two images, and comparing the results to find changes or objects of interest which are present in the new data.

Currently, the process of change detection is operator and time intensive and prone to errors. To remedy this, some methods for automating the processing have been proposed, however the field as applied to the underwater environment remains in its infancy. The Norwegian Defence Research Establishment (FFI), Defence Research and Development Canada (DRDC), Naval Research Laboratory (NRL), US, and Ecole Nationale Sup´erieure de Techniques Avanc´ees (ENSTA) ´ Bretagne, France, have initiated a collaboration including a combined sea trial and workshop in 2011, arranged in Norway and Canada, respectively. The overall goal is to significantly progress automated change detection technology for autonomous underwater vehicles (AUVs) equipped with synthetic aperture sonars (SAS).

Change detection has been studied for sidescan sonar in [1]

and SAS in [2], [3]. In [4] and [5] the spatial decorrelation as a function of baseline (displacement of tracks) is discussed.

The temporal decorrelation due to actual changes of the

Fig. 1. The HUGIN HUS AUV during launch from H U Sverdrup II April 11, 2011.

seafloor is considered in [6]. In [7], a technique for coherent coregistration is described. Change detection is a well known topic in synthetic aperture radar (SAR) [8]–[10].

In April 2011, we conducted a trial in an area outside

Larvik, Norway, using FFI’s research vessel HU Sverdrup

II and HUGIN 1000-HUS AUV equipped with a HISAS

1030 interferometric SAS (Fig 1). Objects were placed in two

different areas: One with a fairly benign seafloor, and one

with complex seafloor. Both areas were in shallow water, with

difficult ocean environment conditions. The vehicle conducted

three missions in both areas in different days, with up to 8

days of separation between first and last mission in the same

area. In the fairly benign area, 7 objects were deployed, and

in the complex area, 8 objects were deployed. Missions were

conducted with and without objects. In this paper, we present

the trial, and show example SAS images from both areas. We

show that incoherent image based change detection makes it

substantially easier to detect objects in an area of very high

clutter.

Fig. 2. The HISAS 1030 interferometric SAS mounted on a HUGIN autonomous underwater vehicle.

II. S YSTEM D ESCRIPTION

The experiment was conducted using the HUGIN 1000- HUS AUV [11] owned by FFI as the sensor carrier. The vehicle was operated from the research vessel HU Sverdrup II.

In change detection, the navigation accuracy is critical since sonar images from repeated missions are to be compared in order to detect changes. The HUGIN AUV uses a Doppler Ve- locity Logger (DVL) aided Inertial Navigation System (INS) in combination with post processing of the navigation data using optimal smoothing [12], [13]. GPS fixes were taken regularly during the missions. Even though the navigation system is accurate, it still is a challenge to coregister the images from repeated passes for change detection. The expected navigation error (drift) is up to a few meters per hour, which means that sonar data driven techniques must be applied to coregister the SAS images from the repeated passes [7].

A. HISAS 1030

The primary payload for the change detection data acqui- sition was a HISAS 1030, which is a wideband widebeam interferometric SAS [14], [15]. The sonar contains two along- track receiver arrays of 1.2 m length with 32 elements in each array, and a vertical baseline approximately 30 cm which equals 20 wavelengths. Fig. 2 shows the sonar mounted on a vehicle. The specifications for the sonar during the change detection missions are summarized in Table I. The vehicle is equipped with two complete sonars, one on each side.

B. TileCam Optical Camera

The TileCam optical camera has been specially designed for the HUGIN 1000 AUV through a collaborative development effort between Norsk Elektro Optikk (NEO), FFI and Kongs- berg Maritime. The system consists of a high-resolution, high- sensitivity, digital still image camera and a synchronized strobe

TABLE I

S

HISAS 1030

SAS

L

.

Center frequency 100 kHz

Wavelength 1.5 cm

Bandwidth 30 kHz

Along-track resolution 3.4 cm Cross-track resolution 3.2 cm Maximum range @ 2 m/s 200 m Area coverage rate 2 km

/h

TABLE II

L

Mission date Area Targets Avg wind speed

2011 04 05 1 Yes 9.0 m/s

2011 04 10 1 No 9.8 m/s

2011 04 13 1 No 3.0 m/s

2011 04 08 2 No 3.0 m/s

2011 04 10 2 Yes 5.4 m/s

2011 04 13 2 Yes 5.4 m/s

with multiple light emitting diodes (LEDs). This light source is ideal for AUVs, due to the LEDs high energy efficiency and the use of pulsed, rather than continuous, lighting. Also, the small LEDs can be built into a narrow panel fitted into the vehicle hull. The camera and strobe are mounted with maximum separation distance on the AUV, thus reducing the loss of image contrast due to backscattered light from particles in the water (Fig. 3). The prototype system is mounted on HUGIN 1000-HUS and was used to document the object deployment and the seafloor conditions during the change detection trial.

III. E XPERIMENT D ESCRIPTION

The trial was conducted in the archipelago outside Larvik, Norway, April 1-14 2011. During the first trial days, the HUGIN AUV mapped and imaged the seafloor at potential sites to collect the data that would be used as a basis for the selection of exact locations for object deployment. Two different seafloor conditions were sought, representing fairly benign and difficult change detection environments. The water depth was also a critical parameter, as the divers deploying the targets had a 20 m depth limit.

The Larvik Change Detection trial consisted of 6 missions in two different areas. An overview of the mission dates and target deployment is shown in Table II. The average wind speed is retrieved from the detailed weather statistics at www.met.no. Every mission was approximately 4 hours in duration, and the plan was identical in the three repeated missions in each area.

A. Area 1 - Smooth Seafloor

The targets were deployed by the Norwegian Coast Guard vessel KV Nornen at 19-20 m water depth on April 4th. The

Fig. 3. The prototype TileCam optical camera and multiple LEDs panel on

HUGIN AUV.

Fig. 4. HUGIN mission plan for the first mission in Area 1. The depth contour interval is 1 m (blue lines).

first mission was planned to be conducted the day before target deployment, but had to be postponed as the trial area had not yet been sufficiently mapped. The first Area 1 change detection mission was thus performed with targets, while the second and third missions were conducted without targets (Table II). This reverse mission order did not affect the scientific outcome of the trial, except that the seafloor in the two reference missions revealed some small scars from the removed targets.

The mission plan had to reflect that Area 1 was surrounded by nearby islands in three directions (Fig. 4). The green lines indicate the planned route for the vehicle, and the red markers show the target positions. Note that the mission lines are such that each target will be seen from at least eight look directions.

The seafloor had a small, even slope and consisted of soft, fine- grained deposits with some scattered, various sized rocks.

B. Area 2 - Complex Seafloor

The first change detection mission in Area 2 was performed without targets on April 8th (see Fig. 5). Although less confined than Area 1, Area 2 had rough bathymetry with steep rock-faces, and the AUV performed three emergency collision avoidance maneuvers in the northern part of the mission area. The mission was successfully executed and the vehicle reference depth for these survey lines was reduced on subsequent missions.

The seven targets were then recovered from Area 1 and deployed at 18-20 m depth in Area 2 by KV Nornen on 8th - 9th April. Two missions were performed with targets (see

Fig. 5. HUGIN mission plan for the first mission in Area 2. The depth contour interval is 2 m (blue lines).

Table II).

The seafloor contained variable density of clutter (rocks) and several different sediment types, yielding highly complex sonar images. This is evident in Fig. 10 showing highly variable backscatter from the seafloor inside the target box.

The effective sonar range was significantly smaller than in Area 1, due to severe multipath dominantly caused by the rock walls of the fjord with contributions by bedrock outcroppings.

C. Ocean Environment

The exercise was conducted inside the confines of a fjord during the spring run-off leading to a surface fresh water layer and higher turbidity due to particulate matter contained in the run-off. These conditions caused the sound speed profile to exhibit a relatively uncommon upward refracting profile.

Given the shallow water depths (20 meters or less), this profile provided challenges due to the higher than normal surface reflection implied. As can be seen in Fig. 7, alge lowered the optical transparency of the sea water. The exposed rock walls of the fjord and bedrock intruding above the seafloor created hard reverberators limiting the coherence range during the trial. Tidal currents were strong enough to occasionally crab the vehicle, and heavy weather affected the sound speed profile as well as the surface reflection.

D. Objects

The following 8 objects were deployed during the trial:

1) Cylinder, length 2.1 m, 53 cm diameter

Fig. 6. Optical images of the water bag (left) and the two cubes (right) before deployment.

Fig. 7. Diver taken optical images of the water bag (upper left), the smooth cube (upper right), the glider (lower left) and the anchor for the glider (lower right). The pictures are from Area 1.

2) Small cylinder, 80 cm length, 10 cm diameter

3) Truncated cone with ground diameter of 1 m, with acoustic transponder on top

4) Truncated cone with ground diameter of 1 m 5) Concrete cube 40x40x40 cm (rough surface) 6) Concrete cube 40x40x40 cm (smooth surface) 7) Slocum Glider, diameter 21 cm, length 150 cm 8) Water bag of size 91x60x7.5 cm

Object number 4 was only deployed in Area 2. In this paper, we show results from parts of the seafloor containing the four last objects. Fig. 6 shows images of the water bag and the two cubes before deployment. The objects were deployed by divers from the Norwegian Coast Guard ship KV Nornen. Fig. 7 shows optical images taken by the divers during deployment in Area 1. There is a thin rope between the objects which the divers used to navigate between them.

In Fig. 8 we show optical images taken by the prototype TileCam optical camera on the vehicle (see section II-B).

These images are from Area 2 and seen from above. Note that the water bag is adjacent a large rock. The water filled bag was hold onto the seafloor by 15 kg of gravel inside and a large shackle on top of it. The vehicle altitude is 3.25 m for the smooth cube, 5.8 m for the rough cube, 5 m for the water bag and 6 m for the glider.

Fig. 8. TileCam optical images taken by HUGIN HUS of the four targets after deployment in area 2. Upper left: water bag; Upper right: glider; Lower left: smooth cube; Lower right: rough cube.

IV. SAS R ESULTS

In this section we show example SAS images from the targets in the two different areas. All images are made using the backprojection algorithm onto an assumed or estimated ground plane [16, pages 117-119]. Fig. 9 and Fig. 10 shows SAS images of 75 m by 100 m in size (along-track and range) of four targets deployed in Area 1 and Area 2. The vehicle track is along the x-axis, and sonar range is along the y axis.

The SAS image from Area 2 is substantially more cluttered than the SAS image from Area 1. The dark area at short range in Fig. 10 is due to the reduced illumination underneath the vehicle (the blind zone of the two-sided sonar). Since both areas are fairly shallow, the SAS images might suffer from multipath pollution at far range [17]. For the image shown in Fig. 9, the single pass interferometric coherence is high over the whole image. In the image shown in Fig. 10, the single pass interferometric coherence drops below 0.66 at approximately 80 m range.

Fig. 11 shows a zoom in on the rough cube in Area 2. The

target is clearly visible despite being only 40 cm wide. The

smooth cube in Area 2, shown in Fig. 12 is placed in an area

with high clutter density. It is almost impossible to spot the

target unless one knows exactly what to look for. The glider

in Area 2 (shown in Fig. 13) is placed in the area with soft

sediment and low backscatter. Note that the wings of the glider

are not visible. The small object 3 m to the right of the glider

is probably the anchor of the glider. The deployment site for

the water bag in Area 2 is shown in Fig. 14. The upper and

lower images show the site before and after deployment of the

water bag. We see that the water bag is partially covered by

the shadow of a large rock, and therefore difficult to detect.

Fig. 9. SAS image of Area 1. The image size is 75 m by 100 m and the dynamic range is 50 dB.

The bright spot on the water bag is the shackle that was put on top to keep it on the seafloor (see Fig. 8).

V. C HANGE D ETECTION

Change detection is the process by which newly collected survey data is compared with historical (baseline) data in order to find targets of interest that were not present in the baseline survey. As the baseline survey is considered ”clean”, objects present in both surveys can be dismissed as benign, and only new objects need to be investigated further. It is often the only viable method of detecting targets in areas of high clutter density, or detecting targets of arbitrary sizes and shapes. Automated change detection methods fall into three general categories:

•

ATR-based change detection is when an automatic target recognition (ATR) method is applied separately to the two data sets. The resulting detections are then associated [18]

between surveys, and objects detected in the new data without a corresponding detection in the historical data are called changes.

•

Incoherent image-based is when the two images are compared pixel-by-pixel in an incoherent (without phase information) way [1], [19]. The images first need to be coregistered in some way before this can be done.

Fig. 10. SAS image of Area 2. The image size is 75 m by 100 m and the dynamic range is 40 dB.

The resulting ”difference” image can be used to detect changes.

•

Coherent image-based also performs a pixel-wise com- parison of the two images. Now the comparison is done coherently, using both the amplitude and phase of the images. Again, the images need to be coregistered to a degree of precision such that the phase coherence of the two images is preserved. Drops in the coherence between two images, or differences in phase, can be used to detect changes.

The last two methods have less requirements on a priori information on the size and shape of the targets of interest.

In fact, the coherent change detection method can potentially

detect targets that are smaller than the resolution of the

sonar. As mentioned, this performance comes at the cost of

needing to precisely coregister the two images [7]. This also

places stringent limits on the repeat navigation, since the

look direction of the sonar onto the imaging scene cannot

change significantly between the two surveys [4], [20]. In the

case of coherent change detection, this requirement is such

that in order to maintain coherence, the seabed itself cannot

have changed significantly between the two looks. Depending

on the environment such as currents and marine life, the

temporal baselines can be prohibitively short [6]. The ATR-

Fig. 11. SAS image of the rough cube in Area 2. The dynamic range is 40 dB.

Fig. 12. SAS image of the smooth cube in Area 2. The dynamic range is 40 dB.

based change detection is less sensitive to these requirements.

However, in addition to a robust object association method, a target detection algorithm is needed. This requires some explicit or learned model of the target class, thus bounding from above the performance of the change detection method to that of the detection method used.

A. Preliminary results

Fig. 15 presents preliminary results from incoherent image based change detection. The upper plot shows a 50 m by 80 m SAS image from Area 2 on April 8th before target deployment. The centre plot shows the same seafloor area on April 10th after deployment of four targets (two concrete cubes, a glider and a water bag). The targets are however difficult to detect, due to the complex and cluttered seafloor.

Anisotropic diffusion filtering has been applied on both images

Fig. 13. SAS image of the glider in Area 2. The dynamic range is 40 dB.

Fig. 14. SAS image of the water bag in Area 2. The dynamic range is 30

dB. Upper: before deployment. Lower: after deployment.

to reduce speckle [21]. The two images are slightly offset, due to navigation inaccuracies. To map the first image onto the pixel coordinates of the second image, interest points (landmarks) are extracted from the images using the SURF algorithm [22]. Matching interest points can then be used to estimate the parameters for affine transformation (translation, rotation, scale) between the images. The difference image shown in the lower plot was created by pixel-wise subtraction of the coregistered despeckled logarithmic intensity images.

The calculation of difference images is closely related to the methods described for change detection in SAR [8]–[10].

Seafloor variations present in both the images have now been filtered out, making the four targets easy to detect against an almost homogeneous background. Even the small anchor above the glider is discernable as a small highlight.

VI. S UMMARY

This paper has introduced a data set collected by the HUGIN AUV operating HISAS in a shallow water area near Larvik, Norway. Two seafloor areas were chosen for investigating change detection: one a fairly benign environment for de- tecting objects and the other a very challenging area for detecting objects. Preliminary investigation has shown the data set contains images suitable for incoherent change detection.

The optical camera has shown promise as an adjunct to the acoustical change detection process for identifying objects of interest. The long term goal of this effort is to create automated systems for detecting objects recently placed on the seafloor.

Methods based on coherent and incoherent change detection will be explored. Though a full analysis of this data set has not been completed, the preliminary results show the current equipment can accomplish change detection in this challenging environment.

A CKNOWLEDGEMENTS

The authors would like to thank the HUGIN operators, FFI scientists and engineers participating in the Larvik 2011 sea trial. The authors also thank Tony Lyons and Derek Olson from Applied Research Laboratory, Pennsylvania State University, USA. Finally the authors would like to thank the crews of H.

U. Sverdrup II and KV Nornen.

R EFERENCES

[1] V. Myers, A. Fortin, and P. Simard, “An automated method for change detection in areas of high clutter density,” in International Conference on Underwater Acoustic Measurements (UAM) Conference Proceedings, May 2009.

[2] M. Gendron, M. Lohrenz, and J. Dubberley, “Automated Change De- tection Using Synthetic Aperture Sonar Imagery,” in OCEANS 2009 MTS/IEEE, Biloxi, MS, USA, October 2009.

[3] D. D. Sternlicht, J. K. Harbaugh, and M. A. Nelson, “Experiments in Coherent Change Detection for Synthetic Aperture Sonar,” in OCEANS 2009 MTS/IEEE, Biloxi, MS, USA, October 2009.

[4] S. A. V. Synnes, H. J. Callow, T. O. Sæbø, and R. E. Hansen,

“Multipass coherence processing on synthetic aperture sonar data,” in Proceedings of European Conference on Underwater Acoustics (ECUA 2010), Istanbul, Turkey, July 2010.

[5] D. D. Sternlicht and T. G-Michael, “Change Detection by Image Correlation for Synthetic Aperture Sonar,” in Proceedings of Synthetic Aperture Sonar and Radar 2010, Lerici, Italy, September 2010.

[6] T. P. Lyons and D. C. Brown, “Temporal Variability of Seafloor Rough- ness and its Impact on Coherent Change Detection,” in Proceedings of Synthetic Aperture Sonar and Radar 2010, Lerici, Italy, September 2010.

[7] T. O. Sæbø, R. E. Hansen, H. J. Callow, and S. A. Synnes, “Coreg- istration of synthetic aperture sonar images from repeated passes,” in Proceedings of Underwater Acoustic Measurements 2011, Kos, Greece, June 2011.

[8] E. J. M. Rignot and J. J. van Zyl, “Change Detection Techniques for ERS-1 SAR Data,” IEEE Trans. Geosci. Remote Sensing., vol. 31, no. 4, pp. 896–906, July 1993.

[9] A. Singh, “Digital change detection techniques using remotely-sensed data,” Int. J. Remote Sensing, vol. 10, no. 6, pp. 989–1003, 1989.

[10] Y. Bazi, L. Bruzzone, and F. Melgani, “An Unsupervised Approach Based on the Generalized Gaussian Model to Automatic Change De- tection in Multitemporal SAR Images,” IEEE Trans. Geosci. Remote Sensing., vol. 43, no. 4, pp. 874–887, April 2005.

[11] P. E. Hagen, N. J. Størkersen, and K. Vestg˚ard, “HUGIN-use of UUV technology in marine applications,” in Proceedings of the OCEANS ’99 MTS/IEEE, vol. 2, Seattle, WA, USA, September 1999, pp. 967–972.

[12] B. Jalving, K. Gade, K. Svartveit, A. Willumsen, and R. Sørhagen,

“Dvl velocity aiding in the hugin 1000 integrated inertial navigation system,” in Proceedings from ADCPs in Action, June 2004. [Online].

Available: www.navlab.net

[13] K. Gade, “NavLab, a generic simulation and post-processing tool for navigation,” European Journal of Navigation, vol. 2, no. 4, pp. 51–59, November 2004. [Online]. Available: www.navlab.net

[14] P. E. Hagen, T. G. Fossum, and R. E. Hansen, “HISAS 1030: The next generation mine hunting sonar for AUVs,” in UDT Pacific 2008 Conference Proceedings, Sydney, Australia, November 2008.

[15] T. G. Fossum, P. E. Hagen, B. Langli, and R. E. Hansen, “HISAS 1030:

High resolution synthetic aperture sonar with bathymetric capabilities,”

in Shallow survey, Portsmouth, NH, USA, October 2008.

[16] D. H. Johnson and D. E. Dudgeon, Array signal processing: Concepts and Techniques, ser. Signal processing series. Englewood Cliffs, NJ, USA: Prentice Hall, 1993.

[17] S. A. Synnes, R. E. Hansen, and T. O. Sæbø, “Assessment of shallow wa- ter performance using interferometric sonar coherence,” in Proceedings of Underwater Acoustic Measurements 2009, Nafplion, Greece, June 2009.

[18] E. Coiras, J. Groen, D. Williams, B. Evans, and M. Pinto, “Automatic change detection for the monitoring of cluttered underwater areas,”

in Proceedings of the 1st International Conference & Exhibition on Waterside Security (WSS), August 2008.

[19] V. Myers and S. Daniel, “Techniques de d´etection de changements assist´e par ordinateur appliqu´ees aux images sonar,” in Monitoring Quantitatif de lEnvironnement Sous-Marin (MOQESM) conference pro- ceedings, June 2010.

[20] R. F. Hanssen, Radar Interferometry: Data Interpretation and Error Analysis. Kluwer Academic Publishers, 2001.

[21] P. Perona and J. Malik, “Scale-space and edge detection using ansotropic diffusion,” IEEE Transactions on Pattern Analysis and Machine Intelli- gence, vol. 12, no. 7, pp. 629–639, July 1990.

[22] H. Bay, T. Tuytelaars, and L. Van Gool, “Surf: Speeded up robust

features,” in European Conference on Computer Vision, vol. 1, 2006,

pp. 404–417.

Fig. 15. SAS image before deployment of targets (upper), after deployment of targets (middle), and difference image (lower). The dynamic range is 45 dB

in all three images. Note that the deployed target (the changes) are clearly visible in the difference image although the area is highly cluttered.