Joint decoding of independently encoded compressive multi-view video streams

We design a video coding and decoding framework for multi-view video systems based on compressed sensing imaging principles. Specifically, we focus on joint decoding of independently encoded compressively-sampled multi-view video streams. We first propose a novel distributed coding/decoding architecture designed to leverage inter-view correlation through joint decoding of the received compressively-sampled frames. At the encoder side, we select one view (referred to as K-view) as a reference for the other views (referred to as CS-views). The video frames of the CS-view are encoded and transmitted at a lower measurement rate than those of the selected K-view. At the decoder side, we generate side information to decode the CS-views as follows. First, each K-view frame is down-sampled and reconstructed, and then compared with the initially reconstructed CS-view frame to obtain an estimate of the inter-view motion vector. The original CS-view measurements are then fused with the generated side image to reconstruct the CS-view frame through a newly designed algorithm that operates in the measurement domain. We also propose a blind video quality estimation method that can be used within the proposed framework to design channel-adaptive rate control algorithms for quality-assured multi-view video streaming. We extensively evaluate the proposed scheme using real multi-view video traces. Results indicate that up to 1.6 dB improvement in terms of PSNR can be achieved by the proposed scheme compared with traditional independent decoding of CS frames.