Unsupervised Motion Segmentation for Neuromorphic Aerial Surveillance

Sami Arja*, Alexandre Marcireau, Saeed Afshar, Bharath Ramesh, Gregory Cohen
Western Sydney University
*s.elarja@westernsydney.edu.au

TL;DR: How can we detect any moving target from an airborne platform using only motion cues? We develop a network that detect any salient object regardless its shape and appearance and estimate its motion without the need of any human labelling, relying purely on self-supervised features from Vision Transformers (ViT) and optical flow.

Moving SUV low oblique
Airplane takeoff
Big cars low oblique

Summary of the paper

Frame-based vision sensors on aerial platforms face bandwidth-latency challenges. On the contrary, event cameras offer promise with their high temporal resolution and low power needs. Existing methods for event-based motion segmentation often require manual labeling or scene-specific parameter tuning. Our approach addresses these issues by employing self-supervised transformers on event data and optical flow, eliminating the need for human annotations and reducing parameter tuning. We evaluate our method extensively on diverse datasets using an HD event camera on a dynamic aerial platform in urban settings, demonstrating superior performance compared to existing methods in handling various motion types and multiple moving objects

Image ALT TEXT
Output of the network at every processing step
GIF ALT TEXT
A Prophesee Gen4 camera (1280x720) is mounted on a small airplane pointing downward
Event Camera on Drone

Comparison with EMSGC method

Ours
EMSGC
Ours
EMSGC
Ours
EMSGC
Ours
EMSGC
Ours
EMSGC
Ours
EMSGC
Ours
EMSGC
Ours
EMSGC
Ours
EMSGC

Results on public dataset (EED, EV-IMO, EV-IMO2, DistSurf, HKUST-EMS)

Input
Output
Event Camera on Drone

BibTeX


@misc{arja_unsupervised_2024,
  title = {Unsupervised Motion Segmentation for Neuromorphic Aerial Surveillance},
  url = {http://arxiv.org/abs/2405.15209},
  publisher = {arXiv},
  author = {Arja, Sami and Marcireau, Alexandre and Afshar, Saeed and Ramesh, Bharath and Cohen, Gregory},
  month = may,
  year = {2024},
}