Project Overview

Data Collection & Cleaning

We can gather data re- lated to a specific place, ob ject or event that were shared or posted in vari- ous places (e.g., social net- works, CCTVs and body cams) or that were seized in an operation.

Data Organization & X-coherence

The main task consists of tying together different information pieces in a spatial and temporal coherent form (X-coherence). Space-time-coherent pieces can be exploited to have an overall idea of the event, place or object as a whole, as well as of specific points within the event’s timeline.

Content Understanding & Inference

With X-coherent pieces, we can (i) look for specific clues (authorship, related topics); (ii) mine possible suspects, objects or places of interest through biometrics and machine learning techniques; (iii) counter fake propaganda and media repurposing through provenance analysis methods; and (iv) assess sensitive content through diverse filters (e.g., violence), everything underpinned by open-set recognition techniques.

Motivational Video

This video presents the main ideas behind the DéjàVu project.

X-Coherence

Feature-space-time coherence, i.e., space-time coherence in physical terms (position in time), that is where and when something happened.

How to connect different information for the same event be it in the physical world or online
How to gather time information
How to synchronize the different pieces in time
How to organize the different pieces (shortly before, during, and shortly after) with respect to the event

Content Understanding & Inference

Crowd Data Analysis

Given any of these events, it would be extremely important to find associated information pieces in space and time both in the pool of apprehended materials and in possible collected social media information.

Mining points of interest

Leveraging the feature- space-time coherent information pieces from previous stages to find possible persons, objects or places involved with the event and, ultimately, propose some candidate suspects for further inves- tigation.

Provenace Analysis

Forensics analysts are interested not only in determining if a digital object is fake or real but also in pinpointing who created it, what happened, when and how (genealogy) an asset was created.

Sensitive Media Analysis

Given data collected and processed with the previous steps of the research, we will focus on deciding whether or not an image or video stream presents sensitive content.

News

Spotlight