Multimedia Forensics

Digital sources are more and more frequently used to make important decisions. This is especially clear in the forensic field, where images and videos can be used, for example, to describe the scene of a crime or to define responsibilities in road accidents. The problem is that many softwares exist, nowadays, which allow one to easily modify photos leaving little or no sign of manipulation. Therefore, it is important to devise tools that help to decide on the authenticity of an image or a video, as testified by the growing attention in the image forgery detection field.



SpoC: Spoofing Camera Fingerprints

spocThanks to the fast progress in synthetic media generation, creating realistic false images has become very easy. Such images can be used to wrap rich fake news with enhanced credibility, spawning a new wave of high-impact, high-risk misinformation campaigns. Therefore, there is a fast-growing interest in reliable detectors of manipulated media. The most powerful detectors, to date, rely on the subtle traces left by any device on all images acquired by it. In particular, due to proprietary in-camera processes, like demosaicing or compression, each camera model leaves trademark traces that can be exploited for forensic analyses. The absence or distortion of such traces in the target image is a strong hint of manipulation. In this paper, we challenge such detectors to gain better insight into their vulnerabilities. This is an important study in order to build better forgery detectors able to face malicious attacks. Our proposal consists of a GAN-based approach that injects camera traces into synthetic images. Given a GANgenerated image, we insert the traces of a specific camera model into it and deceive state-of-the-art detectors into believing the image was acquired by that model. Likewise, we deceive independent detectors of synthetic GAN images into believing the image is real. Experiments prove the effectiveness of the proposed method in a wide array of conditions. Moreover, no prior information on the attacked detectors is needed, but only sample images from the target camera.

D. Cozzolino, J. Thies, A. Rössler, M. Nießner, L. Verdoliva. "SpoC: Spoofing Camera Fingerprints". arXiv preprint arXiv:1911.12069, 2019.


Deepfake Detection


The rapid progress in video manipulation has now come to a point where it raises significant concerns for the implications towards society. In particular, face manipulations are of special interest because faces play a central role in human communication, as the face of a person can emphasize a message or it can even convey a message in its own right. A manipulated video could potentially be used to help the spreading false information or fake news. Therefore, it is important to develop tools that help to detect its authenticity.

To help the research in this field, in collaboration with the TUM (Technical University Munich), we have made a dataset of facial forgeries, called FaceForensics++. The dataset will enable researchers to train deep-learning-based approaches in a supervised fashion. The dataset contains manipulations created with four state-of-the-art methods, namely, Face2Face, FaceSwap, DeepFakes, and NeuralTextures. We also examined the realism of state-of-the-art image manipulations, and how difficult it is to detect them, either automatically or by humans.


Read more: Deepfake Detection

Noiseprint: A CNN-Based Camera Model Fingerprint

noiseprintForensic analyses of digital images and videos rely heavily on the traces of in-camera and out-camera processes left on the acquired images. Such traces represent a sort of camera fingerprint. If one is able to recover them, by suppressing the high-level scene content and other disturbances, a number of forensic tasks can be easily accomplished. We propose a method to extract a camera model fingerprint, called noiseprint, where the scene content is largely suppressed and model-related artifacts are enhanced. The noiseprints can be used for a large variety of forensic tasks including image and video forgery localization. Experiments on several datasets widespread in the forensic community show noiseprint-based methods to provide state-of-the-art performance.

To guarantee reproducible research, the code for Noiseprint extraction is available. To download click here.



Read more: Noiseprint: A CNN-Based Camera Model Fingerprint

Blind PRNU-Based Image Clustering for Source Identification

blindclusteringWe address the problem of clustering a set of images, according to their source device, in the absence of any prior information. Image similarity is computed based on noise residuals, regarded as single-image estimates of the camera's photo-response non-uniformity (PRNU) pattern. First, residuals are grouped by correlation clustering, and several alternative data partitions are computed as a function of a running decision boundary. Then, these partitions are processed jointly to extract a single, more reliable, consensus clustering and, with it, more reliable PRNU estimates. Finally, both clustering and PRNU estimates are progressively refined by merging pairs of the same-PRNU clusters, selected on the basis of a maximum-likelihood ratio statistic. Extensive experiments prove the proposed method to outperform the current state of the art both on pristine images and compressed images downloaded from social networks. A remarkable feature of the method is that it does not require the user to set any parameter, nor to provide a training set to estimate them. Moreover, through a suitable choice of basic tools, and efficient implementation, complexity remains always quite limited. The software is available online for the interested researchers.

To download the code click here

F. Marra, G. Poggi, C. Sansone and L. Verdoliva: “Blind PRNU-Based Image Clustering for Source Identification, IEEE Trans. on Information Forensics and Security

Autoencoder with Recurrent Neural Networks for Video forgery detection

video autoencoderVideo content can be acquired with off-the-shelf hardware, and is thus increasingly used to record events. With the growing role of video data for communicating to a large audience, it is desirable to have a set of tools ensuring the authenticity of video content. However, until now, only few methods exist to forensically analyze videos. In this work, we propose a method for statistically comparing two video sequences. Per sequence, intra- and inter-frame residuals are computed. Optical flow is used to compensate for motion artifacts on inter-frame residuals. We use one sequence to build a statistical model, and compare it to the second sequence. From a forensic perspective, the proposed method enables two applications. First, manipulations can be accurately localized if both sequences are subsequences of the same video. Second, source cameras can be distinguished if both sequences stem from different videos. The proposed method is evaluated on collected smartphone data and greenscreen splices. Further, it is quantitatively compared to both a recent PRNU-based approach and a feature-based technique. The splicing video dataset is available online for the interested researchers.

To download the dataset click here

D. D'Avino, D. Cozzolino, G. Poggi, L. Verdoliva: “Autoencoder with Recurrent Neural Networks for Image Forgery DetectionIS&T Electronic Imaging: Media Watermarking, Security and Forensics

A PatchMatch-based Dense-field Algorithm for Video Copy-Move Detection and Localization

cfmd videoWe propose a new algorithm for the reliable detection and localization of video copy-move forgeries. Discovering well crafted video copy-moves may be very difficult, especially when some uniform background is copied to occlude foreground objects. To reliably detect both additive and occlusive copymoves we use a dense-field approach, with invariant features that guarantee robustness to several post-processing operations. To limit complexity, a suitable video-oriented version of PatchMatch is used, with a multiresolution search strategy, and a focus on volumes of interest. Performance assessment relies on a new dataset, designed ad hoc, with realistic copy-moves and a wide variety of challenging situations. Experimental results show the proposed method to detect and localize video copy-moves with good accuracy even in adverse conditions. The software and dataset are available online for the interested researchers.

To download the code click here
To download the dataset click here

L. D'Amiano, D. Cozzolino, G. Poggi and L. Verdoliva: “A PatchMatch-based Dense-field Algorithm for Video Copy-Move Detection and LocalizationIEEE Transactions on Circuits and Systems for Video Technology

Splicebuster: A new blind image splicing detector

splicebusterWe propose a new feature-based algorithm to detect image splicings without any prior information. Local features are computed from the co-occurrence of image residuals and used to extract synthetic feature parameters. Splicing and host images are assumed to be characterized by different parameters. These are learned by the image itself through the expectation-maximization algorithm together with the segmentation in genuine and spliced parts. A supervised version of the algorithm is also proposed. Preliminary results on a wide range of test images are very encouraging, showing that a limited-size, but meaningful, learning set may be sufficient for reliable splicing localization.

To download the code, click here.

D. Cozzolino, G. Poggi, L. Verdoliva: “Splicebuster: A new blind image splicing detector”, IEEE International Workshop on Information Forensics and Security (WIFS) 2015



Efficient dense-field copy-move forgery detection

cmfdWe propose a new algorithm for the accurate detection and localization of copy-move forgeries, based on rotation-invariant features computed densely on the image. Dense-field techniques proposed in the literature guarantee a superior performance w.r.t. their keypoint-based counterparts, at the price of a much higher processing time, mostly due to the feature matching phase. To overcome this limitation, we resort here to a fast approximate nearest-neighbor search algorithm, PatchMatch, especially suited for the computation of dense fields over images. We adapt the matching algorithm to deal efficiently with invariant features, so as to achieve higher robustness w.r.t. rotations and scale changes. Moreover, leveraging on the smoothness of the output field, we implement a simplified and reliable post-processing procedure. The experimental analysis, conducted on databases available online, proves the proposed technique to be at least as accurate, generally more robust, and typically much faster, than state-of-the-art dense-field references. All experiments are fully reproducible, with all software and data available online for the interested researchers.

To download the code click here
To download the dataset click here

D. Cozzolino, G. Poggi, L. Verdoliva: “Efficient dense-field copy-move forgery detection”, IEEE Trans. on Information Forensics and Security, 2015 in press

A Bayesian-MRF Approach for PRNU-based Image Forgery Detection

Graphics editing programs of the last generation provide ever more powerful tools which allow to retouch digital images leaving little or no traces of tampering. The reliable detection of image forgeries requires, therefore, a battery of complementary tools that exploit different image properties. Techniques based on the photo-response non-uniformity (PRNU) noise are among the most valuable such tools, since they do not detect the inserted object but rather the absence of the camera PRNU, a sort of camera fingerprint, dealing successfully with forgeries that elude most other detection strategies. In this work we propose a new approach to detect image forgeries using sensor pattern noise. Casting the problem in terms of Bayesian estimation, we use a suitable Markov random field prior to model the strong spatial dependencies of the source, and take decisions jointly on the whole image rather than individually for each pixel. Modern convex optimization techniques are then adopted to achieve a globally optimal solution and PRNU estimation is improved by resorting to nonlocal denoising. Largescale experiments on simulated and real forgeries show that the proposed technique largely improves upon the current state of the art, and that it can be applied with success to a wide range of practical situations. The software is available online for the interested researchers.

To download the code click here

G. Chierchia, G. Poggi, C. Sansone, L. Verdoliva: “A Bayesian-MRF Approach for PRNU-based Image Forgery DetectionIEEE Trans. on Information Forensics and Security