Experimental Evaluation of Four Intermediate Filters to Improve the Motion Field Estimation

Abstract

For the last 40 years, optical flow (OF) estimation has been challenging the computer vision community. OF is the apparent motion of the pixels in two consecutive images. Variational models are the most used technique to estimate OF, stating an OF estimation error energy model. The minimization of this energy performs an iterative coarse-to-fine approach. In each iteration, the OF is processed with an intermediate filter to eliminate outliers. In the literature, there are studies related to the number of warpings, the number of considered scales, and their effect on the optical flow estimation. However, in the literature, there is no work that studies in depth the effect of different intermediate filters on the optical flow estimation performance. That fact motivates us to compare the effect of different filters in optical flow estimation. Taking an OF model, which is robust to large displacements and illumination changes, we tested it with four different filters: Bilateral filter (BF), Median filter (MF), Weighted median filter (WMF), and a new filter called a Balanced median filter (BMF). This BMF is a weighted sum of the Median filter and the Bilateral filter (BF) using an adaptive weight map that balances the optical flow estimation error due to the median filter or the Bilateral median filter. We assessed these three filters in the state-of-the-art MPI-Sintel training set. That contains realistic synthetic scenes presenting blur, fog, illumination changes, and sequences with significant displacement. Our experimental evaluation shows that the Balanced median filter performs better than the other two filters. When we validated our results on the MPI-Sintel benchmark webpage, we observed that our model performs better than classical methods such as Horn-Schunck, TVL1, and LDOF. Besides, it performs similar to current optical flow estimation methods such as GeoFlow, CPNFlow, and FLOW_UFO.

For the last 40 years, optical flow (OF) estimation has been challenging the computer vision community. OF is the apparent motion of the pixels in two consecutive images. Variational models are the most used technique to estimate OF, stating an OF estimation error energy model. The minimization of this energy performs an iterative coarse-to-fine approach. In each iteration, the OF is processed with an intermediate filter to eliminate outliers. In the literature, there are studies related to the number of warpings, the number of considered scales, and their effect on the optical flow estimation. However, in the literature, there is no work that studies in depth the effect of different intermediate filters on the optical flow estimation performance. That fact motivates us to compare the effect of different filters in optical flow estimation. Taking an OF model, which is robust to large displacements and illumination changes, we tested it with four different filters: Bilateral filter (BF), Median filter (MF), Weighted median filter (WMF), and a new filter called a Balanced median filter (BMF). This BMF is a weighted sum of the Median filter and the Bilateral filter (BF) using an adaptive weight map that balances the optical flow estimation error due to the median filter or the Bilateral median filter. We assessed these three filters in the state-of-the-art MPI-Sintel training set. That contains realistic synthetic scenes presenting blur, fog, illumination changes, and sequences with significant displacement. Our experimental evaluation shows that the Balanced median filter performs better than the other two filters. When we validated our results on the MPI-Sintel benchmark webpage, we observed that our model performs better than classical methods such as Horn-Schunck, TVL1, and LDOF. Besides, it performs similar to current optical flow estimation methods such as GeoFlow, CPNFlow, and FLOW_UFO.