Noiseprint: A CNN-Based Camera Model Fingerprint

noiseprintForensic analyses of digital images and videos rely heavily on the traces of in-camera and out-camera processes left on the acquired images. Such traces represent a sort of camera fingerprint. If one is able to recover them, by suppressing the high-level scene content and other disturbances, a number of forensic tasks can be easily accomplished. We propose a method to extract a camera model fingerprint, called noiseprint, where the scene content is largely suppressed and model-related artifacts are enhanced. The noiseprints can be used for a large variety of forensic tasks including image and video forgery localization. Experiments on several datasets widespread in the forensic community show noiseprint-based methods to provide state-of-the-art performance.

To guarantee reproducible research, the code for Noiseprint extraction is available. To download click here.

.

 

 

In our work "Noiseprint: A CNN-Based Camera Model Fingerprint", we propose a new camera model fingerprint extracted by a suitable trained Convolution Neural Network. The goal of the training is to remove the semantic content and, at the same time, to emphasize all the camera-related artifacts, which contribute to the digital history of an image. The training conceptual schema, shown in the below figure. is inspired by Siamese networks.

np learning

The training dataset which includes pristine images or videos from many different camera models. During training, we only need to know whether the extracted patches come from the same camera and position or not. Once the training is over, the network is freezed and can be used to extract the noiseprint from an image captured by any camera model, both inside and outside the training set. In the following, some extracted noiseprints are shown for different images.

np example

It can also happen that image manipulations leave traces very evident in the noiseprint, such to allow easy localization even by direct inspection. Allway, a heatmap then be obtained directly from noiseprint through a clustering operation, see the paper for more details.

np results

Our noiseprint-based technique works on different types of manipulations including inpainting and seam carving. The two below images are circulated on the Web. Does civil rights activist Emma Gonzalez rip a target or a copy of the American constitution?

np emma

The technique based on noiseprint leaves little doubt, the image on the right is counterfeit. 

The noiseprint can be also used in a supervised setting, assuming a reference noiseprint is available. The strategy follows the classic pipeline used with PRNU-based methods. A set of pristine images, taken by the same camera of the image under analysis, are averaged to obtain a clean reference. This reference is then compared in sliding window modality with the residual of the image under test, using the Euclidean distance as a measure of similarity. For more detail, see our paper "Camera-based Image Forgery Localization using Convolutional Neural Networks". In the following, some examples:

np reference

 

In the paper "Extracting camera-based fingerprints for video forensics", we extend the noisprint method to video data. The network is trained on pristine videos with a Siamese strategy, minimizing distances between same-model patches, and maximizing distances between unrelated patches. Experiments show that methods based on noiseprints perform well in major forensic tasks, such as camera model identification and video forgery localization. For more detail, see the paper. In the following, an examples of the noiseprint extracted from a deekfake video of the youtube channel Ctrl Shift Face:

np video

np video heatmap