Spatial Signal Processing in Optical Display-Camera Communication

Display-Camera Communication (DCC) is a special form of Visible Light Communication in which digital displays and cameras are utilized for optical free-space data transmission. Data is typically represented by 2D barcodes that can be reconstructed and decoded from camera recordings of the display. In DCC, synchronization issues arise in the 2D space domain of the image signal. On the one hand, optical projection including perspective distortion leads to an unknown position of the display in the camera image. The position varies frame by frame due to camera motion during handheld recording (frame-level spatial asynchrony). On the other hand, lens distortion results in nonlinear deviation of the symbol positions in the received signal (symbol-level spatial asynchrony). To address the frame-level spatial asynchrony, a Frame Position Recovery method is presented in this thesis to localize the display and compensate for camera motion. This method allows reliable detection of the display position as well as the rotation of the display in the camera images, while minimizing the visual impact on the perceived video quality in case of imperceptible DCC. To cope with the symbol-level spatial asynchrony, a novel Symbol Position Recovery approach is presented. Unlike most pattern-based linear techniques in the literature, the proposed method estimates the symbol positions based on the data pattern and compensates for lens distortion using a nonlinear model, which allows accurate detection of sampling points even for extremely small data blocks. Comprehensive evaluation by simulations of the optical channel as well as experiments with different hardware combinations demonstrate excellent accuracy and reliability of the developed methods compared to state-of-the-art techniques.