Abstract #

Online experiments have become popular and it is useful to test how data collected online compare to data measured in the laboratory. Here we compared perceptual thresholds of the perceived quality of computer-generated images (CGI) in a large-sample (N=174) measured online, and a smaller-sample (N=71) obtained in a laboratory-controlled study. Stimuli were three sets of CGIs picturing different scenes of the interior of an apartment. The algorithm used for generating the images reduces the amount of visual noise when the computation time increases so that each successive image within a set has less noise than the previous one. The last generated image of each set (i.e. with the less visual noise) was used as a reference image (RI) and compared to other images of the set. Each comparison image (CI), was cut randomly and the missing part was replaced by its corresponding part from the RI. On each trial observers had unlimited time to report whether they see a single or a composite image. We used a QUEST+ Bayesian adaptive method, which minimizes the expected Shannon entropy, to choose the next CI after each trial. Perceptual thresholds were expressed relatively to the size of the image set. Although online participants reached a stable threshold in fewer trials than laboratory participants (90 vs 156 trials on average, respectively), the equivalence test (TOST) revealed a significant similarity (p<0.05) between the online and laboratory perceptual thresholds. Online measurements also replicated the effects of the scene (THRESHOLDscene1=0.38, MADscene1=0.12; THRESHOLDscene2=0.35, MADscene2=0.09; THRESHOLDscene3=0.42, MADscene3 =0.08) we observed in the laboratory (THRESHOLDscene1 = 0.39, MADscene1 =0.11; THRESHOLDscene2 = 0.38, MADscene2 =0.09; THRESHOLDscene3 = 0.41, MADscene3 = 0.11). Overall, this study shows consistent data collected online and in the laboratory. Despite strong differences in experimental conditions, online measurement of perceived image quality could accurately substitute the laboratory measurements.

URI #

• DOI: https://doi.org/10.1167/jov.21.9.1959

Citation #

@article{myrodia2021comparison,
  title={Comparison of threshold measurements in laboratory and online studies using a Quest+ algorithm},
  author={Myrodia, Vasiliki and Buisine, Jerome and Madelain, Laurent},
  journal={Journal of Vision},
  volume={21},
  number={9},
  pages={1959--1959},
  year={2021},
  publisher={The Association for Research in Vision and Ophthalmology}
}