Image quality measures
The application of privacy protection methods can alter the image quality of the recovered images (when the inverse process of a privacy filter is applied in order to recover the original image). Therefore, we need to assess the quality of recovered images.
6http://www.deepdetect.com/
Chapter 2.Visual privacy protection: Related work and Background 27 Two metrics have been selected to measure the visual quality of images, thepeak signal-to-noise ratio, (PSNR) and thestructural similarity(SSIM). The performance of both, SSIM and PSNR is correlated with human judgments. They are commonly used to evaluate the quality of image reconstruction.
The community of that domain (privacy protection preserving surveillance) uses additional metrics, the luminance similarity score(LSS) and theedge similarity score(ESS) [96]. We can also use all these metrics to assess the amount of degradation that a privacy protection method induces.
• The Peak Signal-to-Noise Ratio (PSNR)measures how much the signal has been corrupted (i.e., the level of degradation of a signal) and is commonly used to evaluate the quality of reconstruction.
The higher is the PSNR the better is the quality of the reconstruction.
P SN R(I0, I) = 10log10( 2552 whereIandI0 are, respectively, the reference and the alternated images, with a size ofm∗n.
• The Structural Similarity (SSIM)measures the similarity between two images. The closer the SSIM is to the value one, the greater the similarity iss. Such objective metric has been designed to be more consistent with the characteristics of the human vision system, with respect to traditional metrics such as PSNR [97].
SSIM(Iw0, Iw) = (2µIµI0 +C1)(2σII0+C2)
(µ2I+µ2I0+C1)(σI2+σI20+C2), (2.9) whereIw andIw0 are two windows, respectively, in the reference and the alternated images,µI
andµI0 are the average ofIwandIw0,σ2I andσI20 are the variance ofIwandIw0 andC1andC2 are two constants to avoid instability.
• The Edge Similarity Score (ESS)measures the degree of resemblance of the edge and contour information between two images.
• The Luminance Similarity Score (LSS) measures the dissimilarity in luminance between two images.
2.5.5 Robustness against attacks
In this section, we assume that attackers know the regions of interest and the steps of the target privacy protection method.
Brute Force Attack (BFA)
A BFA consists of an attacker trying many combinations with the hope of possibly guessing correctly. In this thesis, instead of evaluating the security of the key (assuming that it is secure enough), we apply the brute force attack to the coefficients because they are encrypted using permutations. The attacker
Chapter 2.Visual privacy protection: Related work and Background 28 systematically checks all possible coefficient combinations knowing the algorithm until the correct one is found.
Replacement Attack (RA)
RA [53] implies to set all encrypted values to zero while keeping the unencrypted values. For image and video compression, it consists in extrapolating the encrypted data by error compensation from the previous block using the prediction modes or the motion vectors that are, both, available to the attacker.
Thus, this attack confirms that we cannot recover all encrypted blocks by predicting from previous ones.
Parrot Attack (PA)
PA consists to train a model (e.g., an identity recognizer) and test it on images on which the same privacy protection has been applied. Indeed, with the success of deep learning algorithms, it has been proven that if we train an identity recognizer with the degradation applied on images (i.e., the privacy protection method), the identity may be recognized [98].
Chapter 3
Objective VS Subjective evaluation of gender recognition with privacy
protection filters
3.1 Introduction
Deep learning-based algorithms have become increasingly efficient in recognition and detection tasks, especially when they are trained on large-scale datasets. Such recent success has led to a speculation that deep learning methods are comparable to or even outperform human visual system in its ability to detect and recognize objects and their features. In this Chapter, we focus on the specific task of gender recognition in images when they have been processed by privacy protection filters (e.g., blurring, mask-ing and pixelization) applied at different strengths. Assummask-ing a privacy protection scenario, we compare the performance of a deep learning algorithm with a subjective evaluation obtained via crowdsourcing to understand how privacy protection filters affect both machine and human vision.
3.2 Objective VS Subjective gender evaluation
This section provides a detailed description of a subjective and objective evaluation of the gender recog-nition task when applying privacy protection filters. We have selected a Convolutional Neural Networks (CNN) for the objective evaluation, and we did a Crowdsourcing for the subjective one. We use the PETA collection of databases (a complete description of this database is done in the section2.5.3).
29
Chapter 3.Objective VS Subjective evaluation of gender recognition with privacy protection filters 30
3.2.1 CNN-based gender recognition
Following the work [92], we employ a convolutional neural network for the objective evaluation. In par-ticular, we adopt an architecture proposed by Krizhevsky et al. [68], often denoted as AlexNet. This architecture is presented in Figure3.1. It consists of five convolutional layers and three fully connected layers. We only fine-tune the model to recognize genders of pedestrians. Therefore, we train practically the same architecture on PETA dataset. The only difference is that in the last fully connected layer we use 2 neurons instead of 1000, since we only have two target classes (men and women).