Digital images and videos are making their way into our living rooms, laptops, hand-held devices, and cell phones. High-resolution High Definition (HD) digital video broadcasts, as well as lower resolution streaming video over wireless networks are here to stay. The popularity of video-centric websites such as Youtube and Facebook are good examples of how this communication medium has rapidly advanced and impacted our lives. Indeed, acronyms like JPEG, MPEG, and H.264, once the parlance of engineers only, have become a part of our common vocabulary.
Given the phenomenal rate at which image and video content is being generated and distributed, a critical task is the evaluation of the perceptual quality of the content. For example, video content providers need to evaluate encoding parameters, while network service provider need perceptual quality scores to decide load balancing. Subjectively evaluating the quality of the content is an extremely difficult task due to the time and cost involved. Indeed, the only reliable subjective test involves using large numbers of human test subjects, under controlled psychometric experimental conditions, to evaluate the images and/or videos, resulting in statistically meaningful Mean Opinion Scores (MOS). This approach is of course impractical in most situations.
The ideal substitutes for human subjectivity are objective quality assessment algorithms whose scores have been shown to correlate highly with human subjectivity. Perfect correlation is, of course, impossible, since human subjects vary in their judgment too much. There are many challenges in the design of such algorithms for image and video quality assessment. In this article we discuss the challenges involved and present some state-of-the-art image and video quality assessment algorithms.
Generally, image and video quality assessment algorithms are classified into three groups -- full-reference (FR), reduced-reference (RR), and no-reference (NR) algorithms. As their names suggest, the groups correspond to the amount of information available about the original, presumed pristine reference signal. The design of true no-reference algorithms is extremely challenging and little progress has been made. Reduced reference algorithms are somewhat easier and are interesting, but are generally specific to an application. In this article, we limit our discussion to full-reference image quality assessment (IQA) and video quality assessment (VQA) algorithms, where much progress has been made.
Image quality assessment
The primary goal is to produce automatic image and video ratings that correlate well with MOS. A natural approach is to try to mimic the human visual system (HVS), but the HVS itself is still poorly understood. Several of these types of FR algorithms have been proposed, notably, the just noticeable difference (JND) metric. Another approach is to treat IQA as the evaluation of an image communication system. This approach expresses the test signal as the reference signal distorted by an imperfect communication channel, and uses properties of both the source and receiver in its design. The communication system model has resulted in the emergence of two popular IQA algorithms called SSIM and VIF. First however, let's discuss a very popular -- but flawed -- measure of quality -- the mean squared error (MSE).
Mean squared error (MSE)
The MSE and the related peak signal to noise ratio (PSNR) are popularly used to assess image quality. Given two vectors x = {xi|i =1, , N} and y = {yi|i =1, , N}, then
 ,
while
 ,
where L is the image dynamic range (typically [0, 255]).
Of course, the MSE is easy to compute and implement in software and hardware, and has not suffered from any competition until recently. Moreover, it is easy to use in analysis and often gives closed form solutions to optimization problems.
Next: Problems with MSE (photo examples), Structural SIMilarity (SSIM) index
|
