An adaptive statistical method for 4D-fluorescence image sequence denoising with spatio-temporal discontinuities preserving

Texte intégral

(1)An adaptive statistical method for 4D-fluorescence image sequence denoising with spatio-temporal discontinuities preserving Jérôme Boulanger, Charles Kervrann, Patrick Bouthemy. To cite this version: Jérôme Boulanger, Charles Kervrann, Patrick Bouthemy. An adaptive statistical method for 4Dfluorescence image sequence denoising with spatio-temporal discontinuities preserving. Biophotonics for life sciences and medicine, FontisMedia, 2006. �hal-02814481�. HAL Id: hal-02814481 https://hal.inrae.fr/hal-02814481 Submitted on 6 Jun 2020. HAL is a multi-disciplinary open access archive for the deposit and dissemination of scientific research documents, whether they are published or not. The documents may come from teaching and research institutions in France or abroad, or from public or private research centers.. L’archive ouverte pluridisciplinaire HAL, est destinée au dépôt et à la diffusion de documents scientifiques de niveau recherche, publiés ou non, émanant des établissements d’enseignement et de recherche français ou étrangers, des laboratoires publics ou privés..

(2) AN ADAPTIVE STATISTICAL METHOD FOR DENOISING 4D FLUORESCENCE IMAGE SEQUENCES WITH PRESERVATION OF SPATIO-TEMPORAL DISCONTINUITIES J. Boulanger, C. Kervrann. P. Bouthemy. IRISA - INRIA / INRA MIA Campus Universitaire de Beaulieu 35042 Rennes Cedex - France. IRISA - INRIA Campus Universitaire de Beaulieu 35042 Rennes Cedex - France. ABSTRACT We present a spatio-temporal filtering method for significantly increasing the signal-to-noise ratio in noisy fluorescence microscopic image sequences where small particles have to be tracked from frame to frame. Image sequences restoration is achieved using a spatio-temporal adaptive window approach with an appropriate on-line window geometry specification. We have applied this method to noisy synthetic and real 3D image sequences where a large number of small fluorescently labelled vesicles are moving in regions close to the Golgi apparatus. The SNR is shown to be drastically improved and the enhanced vesicles can be segmented. This novel approach can be further used for biological studies where the dynamic of small objects of interest has to be analyzed in molecular and sub-cellular bio-imaging. 1. INTRODUCTION Fluorescence microscopy is now able to routinely produce 4D or 5D digital data sets. Both confocal microscopy and deconvolution microscopy aim at improving the spatial resolution of the image in lateral xy directions as well as in depth z. High resolution images are then relatively easy to obtain. These technologies also allow the acquisition of temporal series in eventually different wavelength (L) domains. The recorded data sets have then four (3D + t) or five (3D + t + L) dimensions and are crucial for dynamic understanding at sub-cellular or molecular levels. This explains their widespread in biomedicine and biophysics. Despite the increasing resolution of images, SNR is often low due to the photon-limited nature of the images. Denoising is then an important processing step for visualization purposes and pattern recognition. Filtering especially takes an important place when dynamics have to be estimated. In several applications in biology, acquired sequences represent small fluorescently tagged particles with highly variable velocities. Moreover, trajectories are spatio-temporal line-like structures that smoothing algorithms must preserve.. However, particle motion cannot be reliably computed without a denoising pre-processing step. In practice, microscopy image sequences are usually processed frame by frame using spatial filters. This is due to the lack of tools for 4D data even if spatio-temporal filtering, investigated for a decade, could to be more appropriate. In video processing, previous commonly-used methods use temporal or spatio-temporal information, and can use motion compensation [2]. Most of spatio-temporal filters rely on an extension of well-known principles developed for 2D still image denoising, such as anisotropic diffusion [5, 6] and wavelet shrinkage [7]. In this paper we also address the sequence denoising problem and use the framework described for still images in [8]. Unlike anisotropic diffusion [1] and bilateral filtering [9], it is a data-driven estimation approach and a pointwise scale selection is obtained by locally estimating the optimal spatio-temporal filtering window. The properties of such an estimator are well known in statistics for adaptive smoothing [10, 11, 3]. Additionally, it requires no explicit a priori knowledge on the unknown image and provides a confidence level attached to the each pixel. The proposed method is not based on energy minimization [4] and its complexity is easily controlled. Furthermore, no motion estimation is required which is known to be problematic when images are heavily corrupted by noise. Our approach has been developped in order to denoise biological images sequences involved in the analysis of fluorescently tagged proteins moving around the Golgi apparatus and participating in the intra-cellular traffic. These proteins are embedded into vesicles whose movement is supposed to depend on a microtubules network. 2. METHOD DESCRIPTION 2.1. Model specification We consider a statistical image model described by the following equation: Yi = u(xi ) + ξi where xi ∈ Ω ⊂ Rd with d = 3 for 2D image sequences or d = 4 for 3D. 0-7803-9134-9/05/$20.00 ©2005 IEEE Authorized licensed use limited to: UR Rennes. Downloaded on March 05,2010 at 12:47:42 EST from IEEE Xplore. Restrictions apply..

(3) volume sequences. The image function ui = u(xi ) has to be recovered from the data Yi corrupted by an independent zero-mean random noise ξi with variance τi2 . We consider that u is a locally piecewise constant function in a neighborhood of a point ξi . However, the size and the shape of this neighborhood are unknown and must be estimated. Once characteristics of such a neighborhood of xi are determined, the regression function can be estimated by optimizing a local maximum likelihood (ML) criterion. The designed method addresses these issues and estimate u in an iterative way. The main idea in our approach is to define a sequence of increasing spatio-temporal neighborhoods (Wi,n )n∈[0:N ] at each point xi such that Wi,n ⊂ Wi,n+1 and N indexes the larger window. The initialization of the window sequence is performed by setting an initial window Wi,0 at point xi . As an example, this first window may contain the 26 nearest neighbors in space domain. Then, we can compute an initial 2 as: estimate of u(xi ), ûi,0 , and its associated variance σ̂i,0 ûi,0 =. 1 |Wi,0 |. X. 2 = Yj and σ̂i,0. xj ∈Wi,0. 1 |Wi,0 |2. X. τ̂j2. xj ∈Wi,0. (1) where τ̂j2 is an empirical estimate of the local noise variance τj2 (see Section 3). This initialization step provides the first estimates of the two sequences of estimates (ûi,n )n∈[0:N ] 2 and (σ̂i,n )n∈[0:N ] . The next step consists in considering a larger window Wi,1 with Wi,0 ⊂ Wi,1 . Then, at iteration 2 n = 1, we calculate a new estimate ûi,1 and σ̂i,1 over Wi,1 . At iteration n, we consider weighted averages and get: X X 2 2 2 τ̂j , (2) ωij = ωij Yj and σ̂i,n ûi,n = j∈Wi,n. j∈Wi,n. and the weights ωij are defined as a function of the contrast between the estimate ûi,n−1 at point xi and the estimates ûj,n−1 at points xj belonging to the neighborhood Wi,n . 2.2. Estimation step Considering its robustness and smoothing properties, we have chosen the influence function of the Huber M-estimator defined as 1 if |x| ≤ 1 f (x) = (3) 1 else |x| but other influence functions are possible [1]. At iteration n, the weights are calculated as follows: ωij =. f (δij ) X f (δij ). (4). xj ∈Wi,n. ûi,n−1 − ûj,n−1 . λσ̂i,n−1. 2.3. Window geometry The second idea governing the definition of the sequence of neighborhoods (Wi,n )n∈[0:N ] is to combine the spatial neighborhood with the temporal neighborhood. This can be achieved not only by choosing a general shape/geometry for the neighborhoods but also by the way they are increased through iterations. Because the data are selected by weights, we can use a simple anisotropic hyper-cubic spatio-temporal volume for the geometry. This neighborhood can be easily parametrized respectively by its spatio-temporal extents. Note that, in confocal microscopy, dz = 3dx = 3dy and this spatial anisotropy of 1/3 along the z dimension can be taken into account by scaling the depth dimension accordingly. As to the time dimension, we can prefer to separate the past and the future by introducing two temporal extents for each spatio-temporal neighborhood, i.e. t− to denote the extent in the direction of the past and t+ to denote the extent in the direction of the future. We can even apply a causal temporal filtering by setting t+ = 0. 2.4. Local stopping rules Two pointwise rules are used to stop the estimation process. The first one allows us to estimate the optimal window for the estimation of ui . The criterion to measure the closeness of the estimator û to the unknown function u is the local L2 risk. It can be decomposed in two terms: squared bias and variance. The selected window will be the window which achieves an optimal compromise between these two terms. Letting E(.) denote the mathematical expectation, the local L2 risk can be written as: 2. 2. (6). 2 . E [ûi,n − ui ] ≈ 2σ̂i,n. (7). 2 E [ûi,n − ui ] = [Bias (ûi,n )] + σ̂i,n. and the optimal solution corresponds to : 2. Considering the sequence of growing windows (Wi,n )n∈[0:N ] and the sequence of estimates (ûi,n )n∈[0:N ] of variance 2 )n∈[0:N ] , the ideal window is the last one satisfying the (σ̂i,n following pointwise statistical rule [10, 11, 8]: ∀n0 < n,. where the normalized contrast δij is defined by: δij =. Therefore, the weights decide which points in the spatiotemporal neighborhood should contribute to the estimation. This decision is made under the hypothesis that the contrast is Gaussian distributed. The parameter λ controls the probability of false alarm to assign a point to the region Wi,n . In our experiments, we will take λ = 3 which corresponds to a probability of type I of 0.0027.. (5). |ûi,n − ûi,n0 | < η σ̂i,n0 .. (8). This local statistical rule (8) means that, for each point xi , while the estimates ûi,n are sufficiently close to each other. Authorized licensed use limited to: UR Rennes. Downloaded on March 05,2010 at 12:47:42 EST from IEEE Xplore. Restrictions apply..

(4) then we decide to continue the estimation process. The factor η can be chosen in the range [2, 4] in order to adapt the decision rule [11, 3, 8]. We also introduce a second rule which consists in locally stopping the estimation if the variance is small enough. This upper bound is set considering that we are dealing with digital integer data. Typically, if σi,n < 1/3, we stop the estimation for the point xi . This simple rule enables to save computing time for uniform regions where larger windows will be accepted with a high probability.. 3. IMPLEMENTATION As noticed in subsection 2.1, an estimation of the noise variance is required. For denoising purposes, we assume that the noise variance is constant over the whole volume sequence i.e. τi2 = τ 2 and that it can be robustly estimated by calculating pseudo-residuals εi defined √ at point xi in a similar way than in [13] by : εi = 1/ 72 (8Yi − ∆Yi ) where ∆Y is the discrete Laplacian of Y calculated over a 4D 8neighborhood. Given the residuals εi , we can then robustly estimate the noise variance τ by: τ = 1.4826 Medi (| εi − Medj |εj | |) .. (9). A pointwise estimation of the variance is also proposed in [8]. We have also seen in sub-sections 2.2 and 2.3 that λ and η can be well √ calibrated using statistical arguments and set to 3 and 2 2 in experiments. The maximum number of iterations N is set to 8 which is a good compromise between computation cost and accuracy.. 4. EXPERIMENTS Our method has been tested on both synthetic and experimental 2D and 3D image sequences using fast 3D microscopy deconvolution[14]. The method is applied to fluorescently tagged proteins moving around the Golgi apparatus and participating to the intra-cellular traffic. Figure 1 contains the temporal intensity profiles extracted for three labeled pixels. They correspond to three typical classes of profiles observed in those sequences. Figure 3 shows slices of the 2D + t volume, that facilitate a qualitative evaluation of the filtering process. Thus, we can see that discontinuities are well preserved and in the same time, that the sequence is strongly smoothed. In addition, simulations in table 1 demonstrate that P SN R = 20log10 (255/rms) where rms denotes the quadratic error, is drastically improved. Finally, figures 4 (a) et (b) show the interest of a denoising processing for visualization purposes.. τ 15 20. P SN Rnoisy 24.41 22.00. P SN Rdenoised 35.07 33.88. IP SN R 10.65 11.88. rms 4.45 5.12. Table 1. Filtering performance for two noise levels. The test sequence was obtained adding a white Gaussian noise of variance τ 2 to pre-filtered data in order deal with √ realistic dynamics and photometry. N = 8, λ = 3, η = 2 2. 5. CONCLUSION We have presented a original adaptive and efficient method for denoising 2D +t or 3D +t image sequences. Numerous experiments demonstrate its ability to smooth images while preserving discontinuities in time as well as in space. Extensions of this method is to reduce the computational load, and to exploit spatio-temporal features (window size) given by the algorithm. Such a denoising algorithm is a first step that will allow us to extract information about the lifetime kinetics of specific Rab proteins. This novel approach does not require motion compensation and can be further used for biological studies where dynamics have to be analyzed. 6. ACKNOWLEDGMENTS The authors would like to thank the ACI-IMPBio (MODYNCELL5D project) for its support and Curie Institute for providing image sequences. 7. REFERENCES [1] M.J. Black, G. Sapiro, D.H. Marimont, and D. Heeger, “Robust anisotropic diffusion”, IEEE Trans. Image Process., vol. 7, no. 3, pp. 421–432, 1998. [2] J.C. Brailean, R.P. Kleihorst, S. Efstratiadis, A.K. Katsaggelos, and R.L. Lagendijk, “Noise reduction filters for dynamic image sequences: a review,” Proc. of the IEEE, vol. 83, no. 9, pp. 1272–1291, 1995. [3] V. Katkovnik, K. Egiazarian, and J. Astola, “Adaptive window size image denoising based on intersection of confidence intervals (ICI) rule”. J. Math. Imag. Vis., vol. 16, no. 3, pp. 223–235, 2002. [4] L. Rudin, S. Osher, and E. Fatemi, “Nonlinear total variation based noise removal algorithms,” Phys. D, vol. 60, no. 1-4, pp. 259–268, 1992. [5] Suk Ho Lee and Moon Gi Kang, “Spatio-temporal video filtering algorithm based on 3-D anisotropic diffusion equation,” in Proc. of IEEE Int. Conf. Image Process. (ICIP’98), Chicago, 1998, vol. 3(2), pp. 447–450. [6] D. Uttenweiler, C. Weber, B. Jähne, R.H. Fink, and H. Scharr, “Spatiotemporal anisotropic diffusion filtering to improve signal to noise ratios and object restoration in fluorescence microscopic image sequences,” J. Biomedical Optics, vol. 8, no. 1, pp. 40–47, 2003. [7] N. Rajpoot, Z. Yao, and R. Wilson, “Adaptive wavelet restoration of noisy video sequences,” in Proc. of IEEE Int. Conf. Image Process. (ICIP’04), Singapor, 2004.. Authorized licensed use limited to: UR Rennes. Downloaded on March 05,2010 at 12:47:42 EST from IEEE Xplore. Restrictions apply..

(5) (a) original frame 58. (b) denoised frame 58. Fig. 2. Three frames of a real 2D image sequence denoised with our method.. (c) temporal signals associated to the 3 marked pixels Fig. 1. Three pixels have been selected in frame number 58 extracted from a 2D + t volume. [8] C. Kervrann, “An adaptive window approach for image smoothing and structures preserving,” in Proc. of Eur. Conf. Comp. Vis. (ECCV’04), Prague, Czech Republic, 2004, pp. 132–144. [9] Danny Barash, “A fundamental relationship between bilateral filtering, adaptive smoothing, and the nonlinear diffusion equation,” IEEE Trans. Patt. Anal. Mach. Intell., vol. 24, no. 6, pp. 844–847, 2002. [10] O. Lepski, “Asymptotically minimax adaptive estimation 1: upper bounds,” SIAM J. Th. Probab. and Appl., vol. 36, no. 4, pp. 654–659, 1991. [11] J. Polzehl and V.G. Spokoiny, “Adaptive weights smoothing with applications to image restoration,” J. Royal Statist. Soc.: Series B (Stat. Meth.), vol. 62, no. 2, pp. 335–354, 2000. [12] L. Stankovic, “Performance analysis of the adaptive algorithm for bias-to-variance tradeoff,” IEEE Trans. on Signal Processing, vol. 52, no. 5, pp. 1228–1234, 2004. [13] T. Gasser, L. Sroka, and C. Jennen Steinmetz, “Residual variance and residual pattern in nonlinear regression,” Biometrika, pp. 625–633, 1986. [14] J.-B. Sibarita, H. Magnin, and J.R. De Mey, “Ultra-fast 4D microscopy and high throughput distributed deconvolution,” in Proc. of IEEE Int. Symp. on Biomedical Imaging (ISBI’02): Macro to Nano, Washington, 2002, pp. 769–772.. Fig. 3. Noisy and denoised data (see figure 2) viewed as a 3D = 2D + t sliced volume. Temporal discontinuities are well preserved on slice column = 23.. (a) noisy. (b) denoised. Fig. 4. Volume rendering of a cropped region of a 3D frame extracted form a 4D volume. Figure (a) shows raw data while figure (b) shows the restoration results using our 4D spatio-temporal filtering method.. Authorized licensed use limited to: UR Rennes. Downloaded on March 05,2010 at 12:47:42 EST from IEEE Xplore. Restrictions apply..

(6)