• Platinum Pass Platinum Pass
  • Full Conference Pass Full Conference Pass
  • Full Conference One-Day Pass Full Conference One-Day Pass

Date: Tuesday, November 19th
Time: 4:15pm - 4:36pm
Venue: Plaza Meeting Room P2


Speaker(s):

Abstract: Over the past century, as display evolved, people have demanded more realistic and immersive experiences in theaters. Here, we present a tomographic projector for a volumetric display system that accommodates large audiences while providing a uniform experience. The tomographic projector combines high-speed digital micromirror and three spatial light modulators to refresh projection images at 7200 Hz. With synchronization of the tomographic projector and wearable focus-tunable eyepieces, the presented system can reconstruct 60 focal planes for volumetric representation right in front of audiences. We demonstrate proof of concept of the proposed system by implementing a miniaturized theater environment. Experimentally, we show that this system has wide expressible depth range with focus cues from 25 cm to optical infinity with sufficient tolerance while preserving high resolution and contrast. We also confirm that the proposed system provides uniform experience in a wide range of viewing zone through simulation and experiment. Additionally, the tomographic projector has capability to equalize vergence state that varies in conventional stereoscopic 3D theater according to viewing position as well as interpupillary distance. This study is concluded with thorough discussion about tomographic projectors in terms of challenges and research issues.

Speaker(s) Bio:

Date: Tuesday, November 19th
Time: 4:36pm - 4:57pm
Venue: Plaza Meeting Room P2


Speaker(s):

Abstract: Designing a fully integrated 360 degree video camera supporting 6DoF head motion parallax requires overcoming many technical hurdles, including camera placement, optical design, sensor resolution, system calibration, real-time video capture, depth reconstruction, and real-time novel view synthesis. While there is a large body of work describing various system components, such as multi-view depth estimation, our paper is the first to describe a complete, reproducible system that considers the challenges arising when designing, building, and deploying a full end-to-end 6DoF video camera and playback environment. Our system includes a computational imaging software pipeline supporting on-line marker-less calibration, high-quality reconstruction, and real-time streaming and rendering. Most of our exposition is based on a professional 16-camera configuration, which will be commercially available to film producers. However, our software pipeline is generic and can handle a variety of camera geometries and configurations. The entire calibration and reconstruction software pipeline along with example datasets will be open sourced to encourage follow-up research in high-quality 6DoF video reconstruction and rendering.

Speaker(s) Bio:

Date: Tuesday, November 19th
Time: 4:57pm - 5:18pm
Venue: Plaza Meeting Room P2


Speaker(s):

Abstract: We present "The Relightables", a volumetric capture system for photorealistic and high quality relightable full-body performance capture. While significant progress has been made on volumetric capture systems, focusing on 3D geometric reconstruction with high resolution textures, much less work has been done to recover photometric properties needed for relighting. Results from such systems lack high-frequency detail and the subject's shading is prebaked into the texture. In contrast, a large body of work has addressed relightable acquisition for image-based approaches, which photograph the subject under a set of basis lighting conditions and recombine the images to show the subject as they would appear in a target lighting environment. However, to date, these approaches have not been adapted for use in the context of a high-resolution volumetric capture system. Our method combines this ability to realistically relight humans for arbitrary environments, with the benefits of free-viewpoint volumetric capture and new levels of geometric accuracy for dynamic performances. Our subjects are recorded inside a custom geodesic sphere outfitted with 331 custom color LED lights, an array of high-resolution cameras, and a set of custom high-resolution depth sensors. Our system innovates in multiple areas: First, we designed a novel active depth sensor to capture 12.4 MP depth maps, which we describe in detail. Second, we show how to design a hybrid geometric and machine learning reconstruction pipeline to process the high resolution input and output a volumetric video. Third, we generate temporally consistent reflectance maps for dynamic performers by leveraging the information contained in two alternating color gradient illumination images acquired at 60Hz. Multiple experiments, comparisons, and applications show that The Relightables significantly improves upon the level of realism in placing volumetrically captured human performances into arbitrary CG scenes.

Speaker(s) Bio:

Date: Tuesday, November 19th
Time: 5:18pm - 5:39pm
Venue: Plaza Meeting Room P2


Speaker(s):

Abstract: Understanding the endpoint distribution of pointing selection tasks can reveal the underlying patterns on how users tend to acquire a target, which is one of the most essential and pervasive tasks in interactive systems. It could further aid designers to create new graphical user interfaces and interaction techniques that are optimized for accuracy, efficiency, and ease of use. Previous research has explored the modeling of endpoint distribution outside of virtual reality (VR) systems that have shown to be useful in predicting selection accuracy and guide the design of new interactive techniques. This work aims at developing an endpoint distribution of selection tasks for VR systems which has resulted in EDModel, a novel model that can be used to predict endpoint distribution of pointing selection tasks in VR environments. The development of EDModel is based on two users studies that have explored how factors such as target size, movement amplitude, and target depth affect the endpoint distribution. The model is built from the collected data and its generalizability is subsequently tested in complex scenarios with more relaxed conditions. Three applications of EDModel inspired by previous research are evaluated to show the broad applicability and usefulness of the model: correcting the bias in Fitts's law, predicting selection accuracy, and enhancing pointing selection techniques. Overall, EDModel can achieve high prediction accuracy and can be adapted to different types of applications in VR.

Speaker(s) Bio:

Date: Tuesday, November 19th
Time: 5:39pm - 6:00pm
Venue: Plaza Meeting Room P2


Speaker(s):

Abstract: Typical camera optics consist of a system of individual elements that are designed to compensate for the aberrations of a single lens. Recent computational cameras shift some of this correction task from the optics to post-capture processing, reducing the imaging optics to only a few optical elements. However, these systems only achieve reasonable image quality by limiting the field of view (FOV) to a few degrees -- effectively ignoring severe off-axis aberrations with blur sizes of multiple hundred pixels. In this paper, we propose a lens design and learned reconstruction architecture that lift this limitation and provide an order of magnitude increase in field of view using only a single thin-plate lens element. Specifically, we design a lens to produce spatially shift-invariant point spread functions, over the full FOV, that are tailored to the proposed reconstruction architecture. We achieve this with a mixture PSF, consisting of a peak and and a low-pass component, which provides residual contrast instead of a small spot size as in traditional lens designs. To perform the reconstruction, we train a deep network on captured data from a display lab setup, eliminating the need for manual acquisition of training data in the field. We assess the proposed method in simulation and experimentally with a prototype camera system. We compare our system against existing single-element designs, including an aspherical lens and a pinhole, and we compare against a complex multi-element lens,} validating high-quality large field-of-view (i.e. 53◦) imaging performance using only a single thin-plate element.

Speaker(s) Bio:

Back