Spatio-Temporal Frequency Analysis for Removing Rain and Snow from Videos

Peter Barnum (1) , Takeo Kanade (1) , Srinivasa G. Narasimhan (1)

1 The Robotics Institute

Capturing good videos outdoors can be challenging due to harsh lighting, unpredictable scene changes, and most relevant to this work, dynamic weather. Particulate weather, such as rain and snow, creates complex flickering effects that are irritating to people and confusing to vision algorithms. Although each raindrop or snowflake only affects a small number of pixels, collections of them have predictable global spatio-temporal properties. In this paper, we formulate a model of these global dynamic weather frequencies. To begin, we derive a physical model of raindrops and snowflakes that is used to determine the general shape and brightness of a single streak. This streak model is combined with the statistical properties of rain and snow, to determine how they effect the spatio-temporal frequencies of an image sequence. Once detected, these frequencies can then be suppressed. At a small scale, many things appear the same as rain and snow, but by treating them as global phenomena, we achieve better performance than with just a local analysis. We show the effectiveness of removal on a variety of complex video sequences.

Domaines: Vision par ordinateur et reconnaissance de formes [cs.CV]; Synthèse d'image et réalité virtuelle [cs.GR]; Traitement des images [eess.IV]

Peter Barnum, Takeo Kanade, Srinivasa G. Narasimhan. Spatio-Temporal Frequency Analysis for Removing Rain and Snow from Videos. Proceedings of the First International Workshop on Photometric Analysis For Computer Vision - PACV 2007, Oct 2007, Rio de Janeiro, Brazil. 8 p. ⟨inria-00264716⟩