Top content

Bio-Inspired Architecture for Deriving 3D Models from Video Sequences

Supplementary Material

Abstract The reconstruction of complex 3D objects from video sequences captured by surveillance, smartphone, and other cameras is a common technique in Hollywood blockbusters and TV series. Unfortunately, the automatic or interactive 3D object reconstruction from this kind of videos is not yet possible in the real world. Enabling computers to recognize the actual 3D shape of objects from complex video sequences, we developed a bio-inspired processing architecture, motivated by findings in the area of human object recognition. By utilizing viewpoint-specific object recognition, changes in position and size of the object of interest in video sequences can be eliminated to a great extent. The result is a representation, comprised of multiple pictures showing 2D projections of the object of interest (OOI) from different viewpoints. Based on this representation, a 3D point cloud (PC) from the object can be obtained. After a detailed description of our architecture and its similarities to the human view-combination scheme, we demonstrate its potency by reconstructing several OOI from complex video sequences. Because some processing modules of the architecture cannot yet be fully automatized, we introduced interactive modules instead. Thus the prototypical implementation of our approach could be realized. Based on the resulting PC, we evaluate our architecture and consider more analogies between human and computer vision, which improve image-based 3D reconstruction.

Resulting 3D Models

(a)
(b)
(c)

Resulting point clouds of the (a) red car, (b) table, and (c) desk.

Download

Reference