LUT

RTMOSAIC

Real-time mosaicing and 3-D reconstruction

Reconstruction from Middlebury Reindeer set
(Click on the image for the large version.)

Goals

The aim of the project is to develop real-time or near real-time methods for mosaicking and later for three-dimensional reconstruction or structure-from-motion. These methods should work on commodity computer hardware.

The basis for real-time mosaicing is the Master's thesis work by Pekka Paalanen. This work produced a system that is capable of tracking restricted camera motion from 0.8 megapixel video imagery at 30 frames per second. Much of the real-time performance is due to taking advantage of graphics acceleration hardware.

The core of the doctoral thesis work is an online 3D-reconstruction pipeline, fed from a single hand-held camera. Camera ego-motion is tracked using an extended Kalman filter visual simultaneous localization and mapping (EKF-VSLAM). Selected frames from the video are processed by multiview stereo (MVS) to produce point clouds with color and normals.

News

30.6.2011
Slamrecon code and Legos data set released.
19.5.2010 3DPVT
Poster (PDF, 2.1 MB) presentation on Narrow baseline GLSL multiview stereo in 3DPVT'10, Paris, France.
14.5.2010
Updated project introduction.
12.5.2010 GPUStereo code
Multiview stereo implementation for GPUs released.
18.10.2008 ECCV workshop
Presentation of the paper Towards Monocular On-Line 3D Reconstruction.
26.9.2008 Researchers' Night
A truly on-line working application is shown to visitors live.
27.6.2008
Some videos added.
5.3.2008
First rough working version of 3D reconstruction.
16.1.2008
Work for real-time 3D reconstruction application begins.
26.9.2007
Research report "Image Based Quantitative Mosaic Evaluation with Artificial Video". Research focus moves on to 3D reconstruction.
20.-21.9.2007
EPSRC Summer School: Vision, Video and Graphics in Bath, UK.
30.8.2007
Presentation and paper in Finsig'07.
22.11.2006
The project is starting based on the Master's thesis by Pekka Paalanen.

Contact information

Lappeenranta University of Technology
Pekka Paalanen E-mail WWW Graduate school researcher
Joni Kämäräinen E-mail WWW Post-doc researcher

This project is part of the research in the Machine Vision and Pattern Recognition Research Group, MVPR.

Download

Source code

  1. Monocular on-line 3D reconstruction (Paalanen's doctoral thesis), a.k.a slamrecon or recon, source code package: slamrecon-20110630-configured.tar.gz, 1.2 MB. Please, refer to the thesis before trying to use this. Note, that it contains a copy of Itlabcpp, and its dependencies apply.
  2. GPU multiview stereo in Itlabcpp, module GLAlgorithms. Released in package 2010-05-12. See also the 3DPVT poster (PDF, 2.1 MB).

Data sets

  1. Legos video sequence from Paalanen's doctoral thesis: legos-20110630.tar.gz, 50 MB.
  2. Image sets from the SCIA2009 paper "Image Based Quantitative Mosaic Evaluation with Artificial Video":

Videos

  1. Please, see the videos page.

Links to other resources

See Links page.

Publications

Do not hesitate to contact authors (e-mail, etc.) in order to retrieve copies or reprints of the following publications.

  1. Articles in international compilation works and in international scientific conference proceedings with referee practice
    1. Pekka Paalanen, Joni-Kristian Kämäräinen, Lasse Lensu, Ville Kyrki, Heikki Kälviäinen, FINSIG`07, Real-Time On-Screen Mosaic Rendering, (2007)
    2. Pekka Paalanen, Ville Kyrki, Joni-Kristian Kämäräinen, ECCV 2008 Workshop: Vision in Action: Efficient strategies for cognitive agents in complex environments, Towards Monocular On-Line 3D Reconstruction, (2008) pdf (1.06 MB)
    3. Pekka Paalanen, Joni-Kristian Kämäräinen, Heikki Kälviäinen, Proc. of the Scandinavian Conf. on Image Analysis (SCIA2009), Image Based Quantitative Mosaic Evaluation with Artificial Video, (2009) pp. 470-479,
    4. Pekka Paalanen, Joni-Kristian Kämäräinen, 3DPVT, Narrow Baseline GLSL Multiview Stereo, (2010) pdf (1.06 MB)
  2. Scientific monographs
    1. Pekka Paalanen, Master´s thesis, Real-time Imaging and Mosaicking of Planar Surfaces, (2006) pdf (11.05 MB)
  3. Other Scientific Publications
    1. Pekka Paalanen, Joni-Kristian Kämäräinen, Heikki Kälviäinen, Image Based Quantitative Mosaic Evaluation with Artificial Video, (2007) pdf (4.05 MB)
    2. Joni-Kristian Kämäräinen, Pekka Paalanen, Experimental Study on Fast 2D Homography Estimation from a Few Point Corr espondences, (2009) No. 111 pdf (0.37 MB)