Alexandre Chapiro

This paper continues our research into perceptual video difference metrics. ColorVideoVDP is the first fully spatio-temporal color-aware metric. It models display geometry and photometry, produces an output in interpretable just-objectionable-difference units, and runs fast. Code is available on GitHub, and extensive documentation and comparisons with other methods in the literature can be seen on the project page.

In this work we explored several perceptual algorithms (PEAs) for power optimization of displays (PODs). Different methods may introduce alterations of different types, and save more or less power depending on the display technology. We compare like-to-like in a user study, obtaining interpretable results for perceptually subtle breakpoints on how much display power can be saved across a wide range of technologies: LCD, OLED, with or without eye tracking, and more.

We followed up on our work unifying contrast sensitivity datasets from the literature by integrating color. This extends our data-driven CSF model to now cover color, area, spatiotemporal frequency, luminance and eccentricity. Our model performs better than any existing work, and importantly covers critical perceptual parameters necessary for display engineering and computer graphics tasks. Data, code, and additional information available on our project page.

This short paper and presentation given at SID Display Week 2024 in San Jose gives a quick overview of different aspects of image/video difference metrics, and some of the philosophy behind our work on VDP metrics, in particular ColorVideoVDP.

We created a perceptual framework that determines the optimal parameters for a tone mapper in real time (>1ms per frame on Quest2). We use this system and the Starburst HDR VR prototype to demonstrate that content shown with an optimized version of the Photographic TMO is preferred to heuristics or unoptimized versions even when display luminance is reduced tenfold.

In this work, we examined tattoos through the lens of computational fabrication. To build our model, we created an automatic tattoo robot, and processes to generate synthetic skins to experiment on. We created a framework to predict and modify tattoo color for different skin tones, which we hope will lead to better tattoo quality for everyone. This work also has medical and robotics applications!

We study the perception of geometric distortions. We create a novel demographically-balanced dataset of human faces, and find the perceived magnitudes of several relevant distortions through a large-scale subjective study.

We used the Starburst HDR VR 20,000 nits prototype display to run a study measuring user preferences for realism when immersed in natural scenes. We found that user preference extends beyond what is available in VR today, and changes significantly between indoor and outdoor scenes.

We unified contrast sensitivity datasets from the literature, which allowed us to create the most comprehensive and precise CSF model to date. This new model can be used to improve many applications in visual computing, such as metrics. Data, code, and additional information available on our project page.

We created a new foveated spatiotemporal metric, following the VDP line of work. This metric is fast, easy to use and has been carefully calibrated on several large datasets.

We created and validated a perceptual model of judder (non-smooth motion) based on its principal influencing components.

We studied the influence of screen size and distance on perceived brightness for screens as large as cinema and as small as mobile phones, an issue that affects artistic intent and appearance matching.

We defined the production and distribution challenges facing the content creation industry in the current HDR landscape and proposed Continuous Dynamic Range video as a solution.

We use non-photorealistic shading as an alternative 3D cue, augmenting the feeling of depth in stereoscopic images.

We created an immersive gaming system out of a legacy console.

This work received wide media attention and was presented at the Ludicious game festival in Zurich.

We developed a pipeline to segment and analyze video. Our technique could be applied for smarter editing.

We measured perceptual aspects of autostereo content and created a depth re-mapping algorithm that tries to optimize content so that the most important regions have a fuller sense of depth while staying within the limits of the technology.

We conducted perceptual experiments to quantify cardboarding - an artifact that occurs when not enough depth is given to a stereoscopic 3D image region.

We created a capture and processing pipeline to generate HDR video on a mobile phone by taking sequential multiple exposures.

We improved our edge detector by using eigenvalues.

We detected high frequencies using an orientation tensor and multiresolution.