Model-based Optical Flow: Layers, Learning, and Geometry

DSpace Repository

Show simple item record

dc.contributor.advisor Black, Michael (Prof.) Wulff, Jonas 2018-04-23T07:21:47Z 2018-04-23T07:21:47Z 2017-12-01
dc.identifier.other 50753011X de_DE
dc.identifier.uri de_DE
dc.description.abstract The estimation of motion in video sequences establishes temporal correspondences between pixels and surfaces and allows reasoning about a scene using multiple frames. Despite being a focus of research for over three decades, computing motion, or optical flow, remains challenging due to a number of difficulties, including the treatment of motion discontinuities and occluded regions, and the integration of information from more than two frames. One reason for these issues is that most optical flow algorithms only reason about the motion of pixels on the image plane, while not taking the image formation pipeline or the 3D structure of the world into account. One approach to address this uses layered models, which represent the occlusion structure of a scene and provide an approximation to the geometry. The goal of this dissertation is to show ways to inject additional knowledge about the scene into layered methods, making them more robust, faster, and more accurate. First, this thesis demonstrates the modeling power of layers using the example of motion blur in videos, which is caused by fast motion relative to the exposure time of the camera. Layers segment the scene into regions that move coherently while preserving their occlusion relationships. The motion of each layer therefore directly determines its motion blur. At the same time, the layered model captures complex blur overlap effects at motion discontinuities. Using layers, we can thus formulate a generative model for blurred video sequences, and use this model to simultaneously deblur a video and compute accurate optical flow for highly dynamic scenes containing motion blur. Next, we consider the representation of the motion within layers. Since, in a layered model, important motion discontinuities are captured by the segmentation into layers, the flow within each layer varies smoothly and can be approximated using a low dimensional subspace. We show how this subspace can be learned from training data using principal component analysis (PCA), and that flow estimation using this subspace is computationally efficient. The combination of the layered model and the low-dimensional subspace gives the best of both worlds, sharp motion discontinuities from the layers and computational efficiency from the subspace. Lastly, we show how layered methods can be dramatically improved using simple semantics. Instead of treating all layers equally, a semantic segmentation divides the scene into its static parts and moving objects. Static parts of the scene constitute a large majority of what is shown in typical video sequences; yet, in such regions optical flow is fully constrained by the depth structure of the scene and the camera motion. After segmenting out moving objects, we consider only static regions, and explicitly reason about the structure of the scene and the camera motion, yielding much better optical flow estimates. Furthermore, computing the structure of the scene allows to better combine information from multiple frames, resulting in high accuracies even in occluded regions. For moving regions, we compute the flow using a generic optical flow method, and combine it with the flow computed for the static regions to obtain a full optical flow field. By combining layered models of the scene with reasoning about the dynamic behavior of the real, three-dimensional world, the methods presented herein push the envelope of optical flow computation in terms of robustness, speed, and accuracy, giving state-of-the-art results on benchmarks and pointing to important future research directions for the estimation of motion in natural scenes. en
dc.language.iso en de_DE
dc.publisher Universität Tübingen de_DE
dc.rights ubt-podok de_DE
dc.rights.uri de_DE
dc.rights.uri en
dc.rights.uri en
dc.subject.classification Optischer Fluss , Bildverarbeitung , Maschinelles Sehen , Bewegungsunschärfe , Bildsegmentierung de_DE
dc.subject.ddc 004 de_DE
dc.subject.other Optical Flow en
dc.subject.other Computer Vision en
dc.subject.other Bildebenen de_DE
dc.subject.other Image Processing en
dc.subject.other Motion Blur en
dc.subject.other Video Analyse de_DE
dc.subject.other Bewegungsschaetzung de_DE
dc.subject.other Image Segmentation en
dc.subject.other Video Segmentation en
dc.subject.other Motion Estimation en
dc.subject.other Video Analysis en
dc.subject.other Layers en
dc.subject.other Principal Component Analysis en
dc.subject.other Geometric Reconstruction en
dc.subject.other Scene Understanding en
dc.subject.other Scene Reconstruction en
dc.title Model-based Optical Flow: Layers, Learning, and Geometry en
dc.type Dissertation de_DE
dcterms.dateAccepted 2018-04-13
utue.publikation.fachbereich Informatik de_DE
utue.publikation.fakultaet 7 Mathematisch-Naturwissenschaftliche Fakultät de_DE


This item appears in the following Collection(s)

Show simple item record