Background extraction from videos produced in laboratory experiments

Abstract

In laboratory cages or arenas, animals are tracked to study their behavior, movement and activity. But before animals can be tracked, they first have to be detected separate from the background. In order to do this, the background without the animal (e.g. ' empty' or reference background) needs to be available. This background is sometimes easy to obtain, i.e. when the animal is hidden or absent. Other times, it has to be extracted from sequences of images while the animal is present. This involves estimating the background in the places where the animal is located over time.

There are different methods or algorithms to extract the background. One of these is the background modeling component of the so-called ' Background Subtraction' methods, which are commonly used to extract moving objects from video sequences. The objective of this study was to find the algorithm among a set of different algorithms that best provides the empty background for videos produced in laboratory experiments. To achieve this, the requirements of a good algorithm were identified and different algorithms were studied. An evaluation method to derive the best algorithm was devised. The evaluation was then performed and the best algorithm was identified. The chosen algorithm was thereafter tested to see how it performed using videos showing different situations in the laboratory.

The identified requirements of a good algorithm were high speed, good quality of the extracted background and applicability to different situations in the laboratory. A total of 24 algorithms were examined. The identified measures used to evaluate the algorithms were (1) Speed or the time it takes to run the algorithm, (2) Minimum Thresholded Difference, which is calculated as the lowest ' thresholded' difference between the extracted and reference background images, taken over all the frames in the video, and (3) the Frequency by which this lowest difference value was calculated in the whole video. The algorithms were run with 2 sample videos and the values for the 3 measures were obtained. The multi-criteria evaluation (MCE) procedure was then used with the extracted values to rank the different algorithms. The best algorithm found was the LBMixtureOfGaussians. This algorithm was run with 37 videos showing different situations in the laboratory, and empty backgrounds were extracted in most of the cases. Only in videos where the animal hardly moved or in multiple arenas where more than one animal was present, were empty backgrounds not obtained.

There were some results which were not as expected. For example, the algorithms FrameDifference, StaticFrameDifference, and AdaptiveBackgroundLearning, which always produced backgrounds with animal or traces of animal, got relatively high scores in the MCE analysis. Reasons for this behavior were sought and recommendations were made. It was also suggested to optimize the extraction of the background.