Due to the ubiquity of smartphones, research on data communications over display-camera channels has received increasing attention in recent years. Such communication systems are characterized by the use of cameras as receivers and electronic displays as transmitters. Common 2D barcodes can be viewed as examples of such communication systems which involve channel coding, modulation, channel estimation, demodulation and channel decoding. Recently introduced systems, such as VCode, PixNet, LightSync, etc., exploit a series of time-varying 2D patterns to transmit a larger amount of data. However, most of these communication systems have not been designed to provide meaningful visual information to human eyes and they typically sacrifice visual appearance in order to guarantee machine-reada...[ Read more ]

Due to the ubiquity of smartphones, research on data communications over display-camera channels has received increasing attention in recent years. Such communication systems are characterized by the use of cameras as receivers and electronic displays as transmitters. Common 2D barcodes can be viewed as examples of such communication systems which involve channel coding, modulation, channel estimation, demodulation and channel decoding. Recently introduced systems, such as VCode, PixNet, LightSync, etc., exploit a series of time-varying 2D patterns to transmit a larger amount of data. However, most of these communication systems have not been designed to provide meaningful visual information to human eyes and they typically sacrifice visual appearance in order to guarantee machine-readability. Though electronic displays were originally designed for showing visual information such as images and videos, few prior works have considered simultaneous image (or video) displaying and data transmission.

Motivated by mobile advertising and related applications, in this thesis, we propose novel designs of display-camera communication systems, which can effectively combine high quality image (or video) displaying and data transmission. Our novel image based communication system over display-camera channels, called MarkCode, can display images with such an impressive visual quality that human eyes cannot easily differentiate the encoded images from the original ones. This is accomplished by formulating the waveform modulation design as a quadratic programming convex optimization problem. MarkCode is inspired by image watermarking, which is a mature technology that enables users to hide secret data in image files for later retrieval. However, the latter cannot recover the hidden data from a camera captured image which is typically corrupted by complicated distortions, such as geometric distortion, inter-symbol interference, motion blurs, etc. We address these challenges by introducing our imperceptible waveform modulation scheme and online supervised pattern recognition based demodulation scheme. In other words, MarkCode can be viewed as an advanced communication system designed for transmission of imperceptible image watermarks over display-camera channels. We also extend MarkCode to the so-called VMarkCode, which is a novel display-camera communication system designed for transmitting imperceptible video water-marks. By viewing VMarkCode as a random network coding system, we developed subspace coding based inter-frame encoding and decoding methods to allow complete data reconstruction based on randomly retrieved video frames. Finally, experimental results are presented to demonstrate that our MarkCode and VMarkCode systems can display high visual quality images and videos, respectively, without sacrificing decodability in some practical situations.