Institute of Cognitive Science

Research Group Computer Vision


Navigation und Suche der Universität Osnabrück


Main content

Top content

3D Dino

Interactive 3D Modelling

3D reconstruction and modeling techniques based on computer vision show a significant improvement in recent decades. Despite the great variety, a majority of these techniques depend on specific photographic collections or video footage. For example, most are designed for large data collections, overlapping photos, captures from turntables or photos with lots of detectable features such as edges. If the input, however, does not fit the particular specification, most techniques can no longer create reasonable 3D reconstructions. The purpose of this project is the development of a software components that overcome these drawbacks in cooperation with the user's real world knowledge. Upon this basis, this project should lead to an interactive 3D reconstruction and modeling software

  • to achieved accurate, reliable, non-monolithic, functioning and CAD-ready models
  • in cooperation with an non-expert user
  • from video footage
  • in a few simple user interactions, only.

3D Interaction The the main drive of this project is the multitude of computer vision based techniques, that reconstruct and model 3D objects or scenes from photographs or video footage captured in 2D (monocular) or 3D (stereo). Many of these techniques and approaches reconstruct objects or scenes with automatic algorithms from image sequences. But would an architect or an engineer call the resulting 3D reconstruction a CAD-model? Is a non-expert user able to apply these tools? Can the user apply the method without special hardware like a rotating plate, laser, stereo camera or even a main frame computer? Are users able to reconstruct models of real world objects in such a way that they will be able to translate them back to real world replicas? The answer to these questions is mostly – No – but why?
Therefore, this project reviews automatic and interactive 3D reconstruction and modeling techniques, compare them and and analyze their weaknesses. Under these perspectives, it should be discussed how an interactive computer vision application could be employed to overcome existing weaknesses. Finally, an application should be developed for the interactive creation of non-monolithic, functioning 3D models — models that architects or engineers would call a model, and CAD-ready models that can be applied for simulation tasks, reverse engineering, replication and many more purposes.

Project Pages

Eye Tracking Data in Multimedia Containers for Instantaneous Visualizations - ETVIS'16
Interactive feature growing - EPIC@ECCV'16
iSeg - semi-automatic ground truth annotation tool - K-CAP'15; ICPRAM'16
Evaluation of multi-view 3D reconstruction software - CAIP'15

Project publications

 2017

[1] J. Schöning & G. Heidemann.
Ventral Stream-Inspired Process for Deriving 3D Models from Video Sequences.
In New Trends in Image Analysis and Processing -- ICIAP 2017 Workshops 2017. Springer International Publishing [InPress].
| BibTeX
[2] J. Schöning, X. Jiang, C. Menon & G. Heidemann.
Content-Aware 3D Reconstruction with Gaze Data.
In International Conference on Cybernetics (CYBCONF) 2017. IEEE.
| PDF | DOI | BibTeX
[3] J. Schöning, T. Behrens, P. Faion, P. Kheiri, G. Heidemann & U. Krumnack.
Structure from Motion by Artificial Neural Networks.
In Scandinavian Conference on Image Analysis (SCIA), pages: 146-158, ISBN: 978-3-319-59126-1, 2017. Springer International Publishing.
| DOI | URL | BibTeX
[4] J. Schöning, P. Faion, G. Heidemann & U. Krumnack.
Providing Video Annotations in Multimedia Containers for Visualization and Research.
In IEEE Winter Conference on Applications of Computer Vision (WACV) 2017. IEEE.
| PDF | DOI | URL | BibTeX
[5] J. Schöning, A.L. Gert, A. Açik, T.C. Kietzmann, G. Heidemann & P. König.
Exploratory Multimodal Data Analysis with Standard Multimedia Player --- Multimedia Containers: a Feasible Solution to make Multimodal Research Data Accessible to the Broad Audience.
In Proceedings of the 12th Joint Conference on Computer Vision, Imagingand Computer Graphics Theory and Applications (VISAPP), pages: 272-279, ISBN: 978-989-758-225-7, 2017. SCITEPRESS.
| PDF | DOI | URL | BibTeX

 2016

[1] J. Schöning & G. Heidemann.
Bio-Inspired Architecture for Deriving 3D Models from Video Sequences.
Computer Vision -- ACCV 2016 Workshop, pages: 62-76, ISBN: 978-3-319-54427-4, 2016. Springer Nature.
| PDF | DOI | URL | BibTeX
[2] J. Schöning, P. Faion, G. Heidemann & U. Krumnack.
Eye Tracking Data in Multimedia Containers for Instantaneous Visualizations.
In IEEE VIS Workshop on Eye Tracking and Visualization (ETVIS), pages: 74-78, 2016. IEEE.
| PDF | DOI | URL | BibTeX
[3] J. Schöning & G. Heidemann.
Image Based Spare Parts Reconstruction for Repairing Vital Infrastructure after Disasters.
In Global Humanitarian Technology Conference (GHTC), pages: 225-232, 2016. IEEE.
| PDF | DOI | BibTeX
[4] J. Schöning, P. Faion & G. Heidemann.
Interactive Feature Growing for Accurate Object Detection in Megapixel Images.
Computer Vision – ECCV 2016 Workshops, 9913 : 546-556, ISBN: 978-3-319-46604-0, 2016. Springer Nature.
| PDF | DOI | URL | BibTeX
[5] J. Schöning & G. Heidemann.
Taxonomy of 3D Sensors - A Survey of State-of-the-Art Consumer 3D-Reconstruction Sensors and Their Field of Applications.
In Proceedings of the 11th Joint Conference on Computer Vision, Imaging and Computer Graphics Theory and Applications (VISAPP 2016), 3 : 194-199, ISBN: 978-989-758-175-5, 2016. SCITEPRESS.
| PDF | DOI | BibTeX
[6] J. Schöning, P. Faion & G. Heidemann.
Pixel-wise Ground Truth Annotation in Videos - An Semi-automaticApproach for Pixel-wise and Semantic Object Annotation.
In Proceedings of the 5th International Conference on Pattern Recognition Applications and Methods (ICPRAM-2016), pages: 690-697, ISBN: 978-989-758-173-1, 2016. SCITEPRESS.
| PDF | DOI | BibTeX

 2015

[1] J. Schöning, P. Faion & G. Heidemann.
Semi-automatic Ground Truth Annotation in Videos: An InteractiveTool for Polygon-based Object Annotation and Segmentation.
In Proceedings of the 8th International Conference on Knowledge Capture, pages: 17:1-17:4, ISBN: 978-1-4503-3849-3, 2015. ACM, New York.
| DOI | BibTeX
[2] J. Schöning.
Interactive 3D Reconstruction: New Opportunities for Getting CAD-ready Models.
In 2015 Imperial College Computing Student Workshop (ICCSW 2015), 49 : 54-61, ISBN: 978-3-95977-000-2, 2015. Schloss Dagstuhl--Leibniz-Zentrum fuer Informatik.
| PDF | DOI | URL | BibTeX
[3] J. Schöning & G. Heidemann.
Evaluation of Multi-view 3D Reconstruction Software.
In Computer Analysis of Images and Patterns, 9257 : 450-461, ISBN: 978-3-319-23116-7, 2015. Springer International Publishing.
| PDF | DOI | BibTeX
[4] J. Schöning & G. Heidemann.
Interactive 3D Modeling - A Survey-based Perspective on Interactive 3D Reconstruction.
In Proceedings of the 4th International Conference on Pattern Recognition Applications and Methods (ICPRAM-2015), 2 : 289-294, ISBN: 978-989-758-077-2, 2015. SCITEPRESS.
| PDF | DOI | BibTeX