HAL Id: hal-00691249
https://hal.archives-ouvertes.fr/hal-00691249
Submitted on 25 Apr 2012
HAL is a multi-disciplinary open access
archive for the deposit and dissemination of
sci-entific research documents, whether they are
pub-lished or not. The documents may come from
teaching and research institutions in France or
abroad, or from public or private research centers.
L’archive ouverte pluridisciplinaire HAL, est
destinée au dépôt et à la diffusion de documents
scientifiques de niveau recherche, publiés ou non,
émanant des établissements d’enseignement et de
recherche français ou étrangers, des laboratoires
publics ou privés.
Blind deconvolution of 3D data in wide field fluorescence
microscopy
Ferréol Soulez, Loïc Denis, Yves Tourneur, Éric Thiébaut
To cite this version:
Ferréol Soulez, Loïc Denis, Yves Tourneur, Éric Thiébaut. Blind deconvolution of 3D data in wide
field fluorescence microscopy. International Symposium on Biomedical Imaging, May 2012, Barcelone,
Spain. pp.CDROM. �hal-00691249�
BLIND DECONVOLUTION OF 3D DATA IN WIDE FIELD FLUORESCENCE MICROSCOPY
Ferr´eol Soulez
1, Lo¨ıc Denis
2,3, Yves Tourneur
1, ´
Eric Thi´ebaut
21Centre Commun de Quantimetrie –
Universit´e Lyon 1 – 8 avenue Rockefeller, 69373 Lyon cedex 08, France
ferreol.soulez@univ-lyon1.fr
2Centre de Recherche Astrophysique de
Lyon – CNRS-UMR 5574 – Universit´e Lyon 1 – ENS Lyon – avenue Charles Andr´e, 69561 Saint-Genis Laval cedex,
France.
3Laboratoire Hubert Curien – CNRS
UMR 5516 – Universit´e Jean Monnet – 18 rue B. Lauras, 42000 St- ´Etienne,
France
ABSTRACT
In this paper we propose a blind deconvolution algorithm for wide field fluorescence microscopy. The 3D PSF is modeled after a parametrized pupil function. The PSF parameters are estimated jointly with the object in a maximum a posteriori framework. We illustrate the performances of our algorithm on experimental data and show significant resolution improvement notably along the depth. Quantitative measurements on images of calibration beads demonstrate the benefits of blind deconvolution both in terms of contrast and resolution compared to non-blind deconvolution using a theoretical PSF.
Index Terms— blind deconvolution, wide field fluorescence mi-croscopy, image restoration, inverses problems
1. INTRODUCTION
Using fluorescent dyes to identify specific cellular structures, the wide field fluorescence microscopy (WFFM) is a widely spread imaging modality in biology. It consists on imaging at its emission wavelength a cellular structure marked by fluorescent dye excited by uniform illumination. On the resulting 2D image, structures are more or less defocalized according to their distance to the focal plane. Moving this focal plane through the sample produces a 3D representation of the object. WFFM however suffers from a very coarse axial resolution compared to the lateral resolution. This blur-ring of the data y can be modeled by a 3D convolution between a point spread function (PSF) h and the observed 3D object x:
y= h ∗ x + n , (1) where n accounts for the noise. The resolution of the setup is given by the shape of the 3D PSF. To improve depth resolution, one can either change the imaging setup (e.g., using confocal or two-photon microscopy) or enhance WFFM 3D data with a deconvolution algo-rithm ([1, 2] and references therein). Most deconvolution algoalgo-rithms use the theoretical diffraction-limited PSF or a more realistic exper-imental PSF measured using calibration beads. Blind deconvolution algorithms bypass the problem of PSF calibration by simultaneously estimating the PSF and the object. Few blind deconvolution algo-rithms have been proposed for WFFM, the most notable being para-metric blind deconvolution (PBD)[3], AIDA[4, 5] and the method proposed by Kenig[6] where the PSF is constrained to lie in a (pre-viously learned) subspace.
The proposed method is closely related to PBD[3] method as we model a PSF as a function of a parametrized pupil function.
This work has been supported by project MiTiV funded by the French National Research Agency (ANR DEFI 09-EMER-008-01).
The main differences are (i) the use of a much more robust MAP framework instead of maximum likelihood and (ii) the parametriza-tion of the PSF that enforces its normalizaparametriza-tion and can model non-symmetric PSF.
2. MAP FRAMEWORK
In blind deconvolution both object x and PSF h are derived from the same measurements y. We consider maximum a posteriori estimates {x+, h+} obtained by minimizing cost function f (x, h):
{x+, h+} = arg min
x∈X,h∈H
J (x, h) , (2) J (x, h) = Jdata(x, h) + µ Jprior(x) , (3)
withJdata(x, h) the neg-loglikelihood, Jprior(x) a regularizing prior
andµ a hyper-parameter. Prior knowledge on the PSF is implicitly enforced through PSF parametrization. RegularizationJprioris thus
applied only to object x.
2.1. Data-fitting term
Under Gaussian noise assumption, the neg-loglikelihood reads: Jdata(x, h) = (y − h ∗ x)⊤· C−1noise· (y − h ∗ x) , (4)
where Cnoiseis the noise covariance matrix and is diagonal if noise
is uncorrelated: Jdata(x, h) = X k wk(h ∗ x)k− yk 2 , (5)
wherewkis the inverse of the noise variance at pixelk. This model
can cope with non-stationary noise and can be used to express con-fidence on measurements on each pixel of the data. Thus it can deal with unmeasured pixels (due to saturation, borders. . . ) by setting wk = 0 for such pixels. Furthermore, except for very low detector
noise, this formulation can account for mixed Poisson + Gaussian noise by approximating it as a non-stationary uncorrelated Gaussian noise [5]: wk def = γ max(yk, 0) + σ2k −1 ifykis measured, 0 otherwise, (6) whereγ is the quantization factor of the detector and σ2
kis the
vari-ance of other approximately Gaussian noises (e.g., read-out noise) at pixelk.
2.2. Object regularization term
As most observed objects are smooth with few sharp structures (e.g., edges and spikes), we use as regularizing prior a hyperbolic version of the classical 3D total variation[7]:
Jprior(x) =
X
k
q
k∇xkk22+ ǫ2. (7)
Parameterǫ > 0 ensures differentiability of Jpriorat 0. Whenǫ value
is close to the quantization level, this function smooths out non-significant differences between adjacent pixels. In addition, since only intensities are measured, we constrain object x to be positive valued.
3. PSF PARAMETRIZATION
An abundant literature discusses PSF modeling for fluorescence mi-croscopy. In the present work, we chose a monochromatic scalar model that defines the 3D PSFh from pupil function p. This pupil function is the in-focus point source wavefront phase and modulus at the exit pupil of the objective. To reduce the number of degrees of freedom of our model, the pupil function is parametrized by a limited number of modes.
3D PSF h(x, y, z) is defined as the squared magnitude of complex-valued amplitude PSF ¯h(x, y, z): h(x, y, z) = ¯h(x, y, z) 2 . (8)
Complex-valued PSF¯h is, in turn, defined as the 3D Fourier trans-form of the complex-valued amplitude optical transfer function (OTF). This OTF is non-zero only on a spherical cap of radiusni/λ
limited by the aperture angle. It can thus be fully described by a 2D function: pupil functionp(κx, κy) which is non-zero on the disk
defined bypκ2
1+ κ22 ≤ NA/λ. 3D PSF h(x, y, z) is then related
to pupil functionp through:
h(x, y, z) = Z Z p(κx, κy) exp (2 i π z d(κx, κy)) . . . exp (2 i π (κxx + κyy)) dκxdκy 2 , (9)
withd(κx, κy) = p(ni/λ)2− (κx+ κy)2the defocus[8],ni the
refractive index of the immersion medium andλ the emission wave-length of the fluorophore. The defocus and the 2D complex pupil functionp(κx, κy) describe the properties of the optical system.
As noted by Hanser[9], the support ofp(κx, κy) is a disk thus
Zernike polynomialsZn provide a suitable basis to express both
modulusρ(κx, κy) and phase φ(κx, κy) of pupil function p:
ρ(κx, κy) = X nβnZn(κx, κy) , (10) φ(κx, κy) = X nαnZn(κx, κy) . (11)
As phase piston has no effect on the resulting PSF we setα0 =
0. Similarly, we cancel the tip-tilt parameters (α1 = α2 = 0) to
keep the PSF laterally centered (this fixes the position degeneracy common with blind deconvolution problems). PSF normalization (i.e., energy conservation) can be expressed as anℓ1norm constraint
khk1= 1, or equivalently as an ℓ2constraint on the complex-valued
PSFk¯hk2 = 1. This latter ℓ2constraint can further be written as an
ℓ2constraint in Zernike orthonormal basis:kβk2 = 1. By selecting
only purely radial Zernike polynomials, one can easily constrain the axial symmetry of the PSF. This relatively simple PSF model offers a flexible parametrization that requires only the knowledge of the wavelengthλ, the numerical aperture NA and the refractive index of the immersion mediumni.
4. BLIND DECONVOLUTION ALGORITHM To solve the blind deconvolution, we have to find the optimal param-eters{x+, α+, β+} that minimizes the cost function J (x, h(α, β)).
A simple way to do this is to use continuous optimization techniques. Due to non-linearity of this cost function and as these parameters have different physical meanings, the simultaneous estimation of parameters of both the object x and the PSFh(α, β) is known to very badly conditioned. This slows down convergence of optimiza-tion algorithm. For that reason, as it is classically done[10], we use an alternating minimization scheme to minimize the criterion J (x, h(α, β)). The algorithm is thus:
1. t = 0, define the initial PSF h(0)as the theoretical diffraction
limited PSF of the microscope i.e., α(0)= 0, β(0)= 0. 2. t = t + 1, estimation of the optimal non-negative object x(t)
according to given PSF h(t−1): x(t)≈ arg min x≥0 Jdata(x, h(t−1)) + µ Jprior(x) . (12)
3. Improvement of the phase parameters α(t) given the other model parameters that minimize:
α(t)= arg min
α
Jdata
x(t), h(α, β(t−1)). (13)
4. Improvement of the modulus parameters β(t)given the other model parameters that minimize:
β(t)≈ arg min kβk2 2=1 Jdata x(t), h(α(t), β), (14)
5. Go to step 2 until convergence.
Each step involves minimization of a criterion with respect to a large number of variables. To that end, we used VMLM-B algorithm [11] which can account for bound constraints on the parameters to en-force object positivity. This algorithm has proved its effectiveness for image reconstruction and only requires the computation of the cost function and its gradient. The memory requirement is a few times the size of the problem. In each step, we do not exactly solve the minimization defined in Eq. (12), Eq. (13) and Eq. (14) but only execute few iterations (about 10) of each inner optimization.
5. RESULTS
We processed the dataset1used by Griffa et al. in [14, 12] for
com-parative study of available deconvolution softwares. That study only considers “non-blind” deconvolution with theoretical PSF and we used it to evaluate the advantages of using a blind deconvolution technique compared to state of the art available “non blind” decon-volution methods.
1the dataset can be found at http://bigwww.epfl.ch/
data Hyugens AutoDeblur Deconvolution proposed method parameters Lab non-blind blind transversal FWHM (innm) 2867 2709 2709 2664 2736 2783 axial FWHM 4760 4000 4640 4160 3054 2977 Relative contrast 18 % 53 % 78 % 68 % 84 % 88 %
Table 1. Performance of 3 state-of-the-art deconvolution methods as reported by Griffa [12] compared to the proposed method (both blind and non-blind). Hyugens and AutoDeblur are commercial softwares and Deconvolution Lab is an imageJ plugin implementing [13].
5.1. Calibration bead
The first dataset is an observation of an InSpeck green fluorescent bead with a known diameter (2.5µm) on a Olympus Cell R micro-scope with a×63, 1.4 NA oil objective. The data cube is composed of256 × 256 × 128 voxels of size 64.5 × 64.5 × 160 nm3. During the first step of the algorithm, the solution of Eq. (12) is a “non-blind” deconvolution with a theoretical PSF (no aberration). On this solution, plotted in Fig. 1(b) and Fig. 1(e), the transversal resolution improvement is quite important but, in the axial section, we can no-tice artifacts caused by PSF mismatches. The result of our blind de-convolution algorithm (hyper-parameter set toµ = 10−3) is shown
on Fig. 1(c) and Fig. 1(f). These figures illustrate the resolution im-provement especially in the axial section with almost no artifact.
0 5 10 15
0 5 10 15
(a) XY section of origi-nal data 0 5 10 15 0 5 10 15 (b) XY section of de-convolved data 0 5 10 15 0 5 10 15 (c) XY section of blindly deconvolved data 0 5 10 15 0 5 10 15 20 (d) XZ section of original data 0 5 10 15 0 5 10 15 20 (e) XZ section of deconvolved data 0 5 10 15 0 5 10 15 20 (f) XY section of blindly deconvolved data
Fig. 1. Sections of observed and restored images of an InSpeck green fluorescent bead of diameter 2.5 µm. Axis are in µm
Both theoretical and estimated PSF are shown on Fig. 2. The es-timated PSF is representative of wide field fluorescence microscopy. Although the in-focus axial sections of both PSF are similar, there are strong differences between axial sections. Unlike the theoretical PSF, the estimated PSF is not symmetric with respect to the plane (z = 0). This is well explained by a mismatch between the refrac-tive indices of the immersion oil and the mounting medium.
To perform quantitative evaluation, Griffa et al. [12] proposed three criteria: the size of the bead given by the lateral and the ax-ial full width at half maximum (FWHM) and the relative contrast between the center of the bead and the maximum of its radial pro-file. We present in Tab. 1, the values of these criteria given by [12]
together with the values obtained with our method. We can notice that in its non-blind form (i.e. step 2 only), the proposed 3D-TV is competitive with the state of the art. As it tends to smooth the object, TV regularization may lead to an over-estimation of the diameter of the bead (its nominal width is2500 nm). However the axial FWHM and the relative contrast shows the improvements given by our blind deconvolution method.
5.2. C Elegans embryo
In addition to this quantitative evaluation on a calibration bead, we processeded the biological sample used by Griffa et al. in their study. It is an observation of a C. Elegans embryo containing DAPI, FITC and CY3 stainings with the same microscope and objective than in previous section. The data cube is composed of 672 × 712 × 104 voxels of size 64.5 × 64.5 × 200 nm3in three spectral channels. The DAPI (blue477nm) stains chromosomes in the nu-clei, the FITC (green542nm) the microtubule filaments and CY3 (red654nm) some point wise protein aggregates. We processed each channel from this dataset individually using the same hyper-parameter for each channel (µ = 2 × 10−5). For each channel, the algorithm takes about 4 hours to converge using an Intel i7-975 CPU. The result of the blind deconvolution (Fig. 3) presents a sig-nificantly improved contrast without the typical haze of fluorescence wide field microscopy: the red spots are much sharper and brighter in the restoration. Furthermore, the resolution is clearly improved especially along the depth axis: the out of focus nuclei encircled in red is almost invisible in the blind deconvolution. However, as it can be seen on axial sections, the deconvolution becomes less effective as one looks deeper in the sample. This can be explained by the use of a single stationary PSF for the whole observation. Because of re-fractive index mismatch, it is well known that the real PSF may vary axially and even laterally.
6. CONCLUSION
We presented a new method for blind deconvolution of WFFM data. It uses a PSF parametrization by means of decomposition of the pupil on Zernike basis. We used a continuous optimization method to iteratively estimate both PSF parameters and the object. Except from some parameters of the setup that are generally known (wave-length, numerical aperture and refractive index of the immersion medium), it requires only tuning one hyper-parameter. Results show clear improvements of both resolution and contrast.
Possible directions for future research include extension of the PSF model to more sophisticated models (e.g. vectorial model with pixel integration), extending this method to confocal and two photon microscopy, and introducing space-variant PSF blind-deconvolution[15].
−1.5 −1.0 −0.5 0.0 0.5 1.0 1.5 −1.5 −1.0 −0.5 0.0 0.5 1.0 1.5 (a) XY section −3 −2 −1 0 1 2 3 −4 −2 0 2 4 (b) XZ section −1.5 −1.0 −0.5 0.0 0.5 1.0 1.5 −1.5 −1.0 −0.5 0.0 0.5 1.0 1.5 (c) XY section −3 −2 −1 0 1 2 3 −4 −2 0 2 4 (d) XZ section −3 −2 −1 0 1 2 3 −4 −2 0 2 4 0 0.0001 0.0002 0.0003 0.0004 0.0005 0.0006 0.0007 (e) YZ section
Fig. 2. Sections of theoretical ((a) and (b)) and estimated PSF ((c), (d) and (e)). Axis are in µm.
(a) Observation
(b) Blind deconvolution result
Fig. 3. C. Elegans embryo observation and the blind deconvolution result. Each panels are cuts in the 3D volume along the yellow lines.
7. REFERENCES
[1] J.B. Sibarita, “Deconvolution microscopy,” Microscopy Techniques, pp. 1288–1291, 2005.
[2] P. Sarder and A. Nehorai, “Deconvolution methods for 3-d fluorescence microscopy images,” IEEE Signal Process. Mag., vol. 23, no. 3, pp. 32–45, 2006.
[3] J. Markham and J-A. Conchello, “Parametric blind deconvolution: a robust method for the simultaneous estimation of image and blur,” J.
Opt. Soc. Am. A, vol. 16, no. 10, pp. 2377–2391, 1999.
[4] E.F.Y. Hom, F Marchis, T.K. Lee, S. Haase, D.A. Agard, and J.W. Se-dat, “Aida: an adaptive image deconvolution algorithm with application to multi-frame and three-dimensional data,” J. Opt. Soc. Am. A, vol. 24, no. 6, pp. 1580–1600, 2007.
[5] L. Mugnier, T. Fusco, and J-M Conan, “Mistral: a myopic
edge-preserving image restoration method, with application to astronomical adaptive-optics-corrected long-exposure images,” JOSA A, vol. 21, no. 10, pp. 1841–1854, 2004.
[6] T. Kenig, Z. Kam, and A. Feuer, “Blind image deconvolution using ma-chine learning for three-dimensional microscopy,” IEEE Trans. Pattern
Anal. Mach. Intell., pp. 2191–2204, 2010.
[7] Rudin L.I., Osher S. and Fatemi E., “Nonlinear total variation based noise removal algorithms,” Physica D: Nonlinear Phenomena, vol. 60, no. 1-4, pp. 259–268, 1992.
[8] P.A. Stokseth, “Properties of a defocused optical system,” JOSA, vol. 59, no. 10, pp. 1314–1321, 1969.
[9] BM Hanser, MGL Gustafsson, DA Agard, and JW Sedat, “Phase-retrieved pupil functions in wide-field fluorescence microscopy,”
Jour-nal of microscopy, vol. 216, no. 1, pp. 32–48, 2004.
[10] J. Besag, “On the statistical analysis of dirty pictures,” Journal of the
Royal Statistical Society. Series B (Methodological), vol. 48, no. 3, pp. 259–302, 1986.
[11] E. Thi´ebaut, “Optimization issues in blind deconvolution algorithms,” in Astronomical Data Analysis II., Jean-Luc Starck, Ed., dec 2002, vol. 4847, pp. 174–183.
[12] A. Griffa, N. Garin, and D. Sage, “Comparison of deconvolution soft-ware in 3d microscopy. a user point of view part 2.,” G.I.T. Imaging &
Microscopy, vol. 1, pp. 41–43, 2010.
[13] C. Vonesch and M. Unser, “A fast thresholded Landweber algorithm for wavelet-regularized multidimensional deconvolution,” IEEE Trans.
Image Process., vol. 17, no. 4, pp. 539–549, April 2008.
[14] A. Griffa, N. Garin, and D. Sage, “Comparison of deconvolution soft-ware in 3d microscopy. a user point of view part 1.,” G.I.T. Imaging &
Microscopy, vol. 1, pp. 43–45, 2010.
[15] L. Denis, E. Thi´ebaut, and F. Soulez, “Fast model of space-variant blurring and its application to deconvolution in astronomy,” in 18th
IEEE Int. Conf. on Image Proc., Bruxelles, France, Sept. 2011, pp. 2873–2876.
