• Platinum Pass Platinum Pass
  • Full Conference Pass Full Conference Pass
  • Full Conference One-Day Pass Full Conference One-Day Pass
  • Basic Conference Pass Basic Conference Pass
  • Student One-Day Pass Student One-Day Pass
  • Experience Pass Experience Pass
  • Exhibitor Pass Exhibitor Pass

Date: Tuesday, November 19th
Time: 2:15pm - 4:00pm
Venue: Plaza Meeting Room P4

[Curated] An Integrated 6DoF Video Camera and System Design

Presenter(s):

Description: Designing a fully integrated 360◦ video camera supporting 6DoF head motion parallax requires overcoming many technical hurdles, including camera placement, optical design, sensor resolution, system calibration, real-time video capture, depth reconstruction, and real-time novel view synthesis. While there is a large body of work describing various system components, such as multi-view depth estimation, our demo is the first to show a complete, reproducible system that considers the challenges arising when designing, building, and deploying a full end-to-end 6DoF video camera and playback environment. Our system includes a computational imaging software pipeline supporting online markerless calibration, high-quality reconstruction, and real-time streaming and rendering. Most of our exposition is based on a professional 16-camera configuration, which will be commercially available to film producers. However, our software pipeline is generic and can handle a variety of camera geometries and configurations. The entire calibration and reconstruction software pipeline along with example datasets is open sourced to encourage follow-up research in high-quality 6DoF video reconstruction and rendering.

Bio of Speaker(s):

TwinCam Go: Proposal of Vehicle-Ride Sensation Sharing with Stereoscopic 3D Visual Perception and Vibro-Vestibular Feedback for Immersive Remote Collaboration

Presenter(s): Vibol Yem, Tokyo Metropolitan University, Japan
Reon Nashiki, Tokyo Metropolitan University, Japan
Tsubasa Morita, Tokyo Metropolitan University, Japan
Fumiya Miyashita, Tokyo Metropolitan University, Japan
Tomohiro Amemiya, The University of Tokyo, Japan
Yasushi Ikei, Tokyo Metropolitan University, Japan

Description: We proposed and developed a prototype of vehicle-ride sensation sharing system that enables a rider to remotely collaborate with a driver and can receive both 3D visual perception and vibro-vestibular sensation. A remote rider can collaborate with driver via voice communication and perceive motion sensation via wheelchair’s movement.

Bio of Speaker(s):
Vibol Yem received a PhD in engineering from University of Tsukuba in 2015. He is currently an assistant professor in the Department of Computer Science, Graduate School of Systems Design, Tokyo Metropolitan University. His research interests are Human Interface, Tactile/Haptic Device, VR/AR, Wearable, Robotics. He is a member of ACM and IEEE.

Reon Nashiki is a master's degree student at Tokyo Metropolitan University. He obtained the bachelor degree in mechatronics system engineering from Tokyo Metropolitan in 2017. His research interests include development of telexistence system, virtual reality and ultra reality.

Tsubasa Morita is a master's degree student at Tokyo Metropolitan University. He obtained the bachelor degree in mechanical system engineering from Tokyo Metropolitan in 2019. His research interests include development of telepresence system, virtual reality and ultra reality.

Miyashita Fumiya is a master's degree student at Tokyo Metropolitan University. He obtained the bachelor degree in mechanical system engineering from Tokyo Metropolitan in 2017. His research interests include investigating sensation of activeness, virtual reality and ultra reality.

Tomohiro Amemiya is an associate professor in the Graduate School of Information Science and Technology, the University of Tokyo, Japan. He received B.S. and M.S. degrees in mechano-informatics from the University of Tokyo, in 2002 and 2004. From 2004 to 2019, he was a research scientist at NTT Communication Science Laboratories, Japan. He was also an honorary research associate at the Institute of Cognitive Neuroscience, University College London, UK in 2014-2015, concurrently. He received a Ph.D. in Information Science and Technology from Osaka University, Japan, in 2008. His research interests include haptic perception, tactile neural systems, VR, and assistive technologies.

Yasushi Ikei graduated from the Graduate School of Engineering, The University of Tokyo with a PhD degree in the industrial mechanical engineering in 1988. He joined Tokyo Metropolitan Institute of Technology in 1992. He is currently Professor at the department of computer science, Tokyo Metropolitan University. His research interests are in the virtual reality, ultra reality, telepresence, multisensory display, and cognitive engineering. He received paper awards and contribution award from the Virtual Reality Society of Japan (VRSJ). He gained innovative technologies awards from METI Japan and DCAJ in 2012,2017,2018. He is former vice president and fellow member of the VRSJ.

Three-dimensional Interaction Technique Using an Acoustically Manipulated Balloon

Presenter(s): Takuro Furumoto, The University of Tokyo, Japan
Takaaki Kamigaki, The University of Tokyo, Japan
Mitsuru Ito, The University of Tokyo, Japan
Masahiro Fujiwara, The University of Tokyo, Japan
Yasutoshi Makino, The University of Tokyo, Japan
Hiroyuki Shinoda, The University of Tokyo, Japan

Description: We use an acoustically manipulated balloon as a visual and tangible interface for the representation of a mid-air virtual object in a full-body AR/MR environment. The user can manipulate the physical balloon by manipulating the corresponding virtual object, and conversely, can manipulate the virtual object by physically manipulating the balloon.

Bio of Speaker(s):
Takuro Furumoto is a Ph.D student at the Graduate School of Frontier Sciences, the University of Tokyo, Japan. He recieved his B.E. and M.E. degrees from the University of Tokyo in 2014 and 2017, respectively. His research interests include acoustic manipulation and its applications. He is a member of ACM, IEEE, and VRSJ.

Takaaki Kamigaki is a project researcher at the Graduate School of Frontier Sciences, the University of Tokyo, Japan. He recieved the B.E., M.E. degrees from Kumamoto University, and Ph.D degree in Science from the University of Tokyo. His research interests include ultrasound-based mid-air haptics.

Mitsuru Ito is a Ph.D. student in the Graduate School of Frontier Sciences, the University of Tokyo, Japan. He received his M.S degree in Frontier Sciences, the University of Tokyo in 2015. His research interests include midair haptic interfaces and aireal displays.

Masahiro Fujiwara is a project assistant professor in the Graduate School of Frontier Sciences, the University of Tokyo, Japan. He received the BS degree in Engineering, the MS degree and the PhD degree in Information Science and Technology from the University of Tokyo, in 2010, 2012, and 2015, respectively. His research interests include information physics, haptics, non-contact sensing and application systems related to them. He is a member of IEEE.

Yasutoshi Makino received his Ph.D. in Information Science and Technology from the Univ. of Tokyo in 2007. He worked as a researcher for two years in the Univ. of Tokyo and an assistant professor at Keio University from 2009 to 2013. From 2013 he moved to the Univ. of Tokyo as a lecturer, and he is an associate professor in the Department of Complexity Science and Engineering in the University of Tokyo from 2017. His research interest includes haptic interactive systems.

Hiroyuki Shinoda is a professor in the Graduate School of Frontier Sciences, University of Tokyo, Japan. He received his Ph.D. degree in engineering from the University of Tokyo in 1995. After a period at UC Berkeley as a visiting scholar in 1999, he was an associate professor in the University of Tokyo from 2000 to 2012. His research interests include information physics, haptics, midair haptics, two-dimensional communication, and application systems related to them. He is a member of IEEJ, RSJ, JSME, IEEE and ACM.

Biofeedback Interactive VR System Using Biological Information Measurement HMD

Presenter(s): Masahiro Inazawa, The University of Tokyo, Japan
Xianyin Hu, The University of Tokyo, Japan
Yuki Ban, The University of Tokyo, Japan

Description: We proposed biofeedback interactive VR system. In this system, VR experiences are interactively changed according to user's biological information or emotion estimated by it. To realize this system, we developed the respiration and pulse wave measurement device that can be easily attached to various HMDs.

Bio of Speaker(s):
Masahiro Inazawa received the M.S. degree in Human environment from the University of Tokyo, Tokyo, Japan, in 2017. His current research interests focus on stress sensing and biological information sensing during virtual reality.

Xianyin Hu received her bachelor degree of computer science in digital media technology from Hunan University, China in 2017. Her undergraduate research with Dr. Gao focused on applications of deep learning in graphics. After graduation, She spent 1 year in University of Tokyo as a research student. She is currently a student in the Graduate School of Frontier Sciences at the University of Tokyo. Her current research interests focus on biological signals processing.

Yuki Ban received the M.S. degree and the Ph.D. degree in Information Science and Technology from the University of Tokyo, Tokyo, Japan, in 2013 and 2016, respectively. He was a researcher at Xcoo inc. research from 2016 to 2017. He is currently an Assistant Professor of Graduate School of Frontier Sciences at the University of Tokyo. His current research interests focus on modifying spatial perception using the visuo-haptic interaction.

Polyvision: 4D Space Manipulation through Multiple Projections

Presenter(s): Keigo Matsumoto, The University of Tokyo, Japan
Nami Ogawa, The University of Tokyo, Japan
Hiroyuki Inou, Kyoto University, Japan
Shizuo Kaji, Kyushu University, Japan
Yutaka Ishii, Kyushu University, Japan
Michitaka Hirose, The University of Tokyo, Japan

Description: 4D space has always been a source of imagination and intellectual activities for human beings. By our novel visualization technique based on multiple 3D projections created in VR, Polyvision equips us with the sense of 4D letting us explore high dimensional data and mathematical objects in a totally new way.

Bio of Speaker(s):
Keigo MATSUMOTO is currently pursuing his Ph.D. in Information Science and Technology at the University of Tokyo in Japan. His research fields are mainly virtual reality (VR) and spatial perception. Based on perception theory, especially cross-modal effects, he tries to manipulate spatial perception and expand VR space. At the same time, he is developing creative demonstrations that offer fascinating opportunities to recognize how relative and uncertain our senses are. His awards include Japan Media Arts Festival Excellence Award and the President's Award of The University of Tokyo.

Nami OGAWA is currently pursuing her Ph.D. in Engineering under the supervision of Prof. Michitaka HIROSE at the University of Tokyo in Japan. Her research fields are mainly virtual reality (VR) and perceptual psychology. Based on psychological theory, especially well-known illusions, she tries to develop ingenious displays and artworks that evoke artificial novel sensations and experiences. At the same time, she utilizes VR as an experimental tool to investigate human perception. She is interested in body ownership and cross-modal perception.

Hiroyuki INOU is an Associate Professor of Mathematics at Kyoto University. He received a Ph.D. from the Department of Mathematics at Kyoto University in 2001. His research field is complex dynamics. He has been developing visualization programs of fractal objects which appear in complex dynamics in order to understand their complicated structures not only in complex dimension one such as Julia sets and the Mandelbrot set, but also in complex dimension two (i.e. in real dimension four) using virtual reality systems.

Shizuo KAJI is an Associate Professor of the Institute of Mathematics for Industry at Kyushu University. He received a Ph.D. from the Department of Mathematics at Kyoto University in 2007. His research interest is in topology and its applications to sciences.

Yutaka ISHII is a Professor of Mathematics at Kyushu University. He received his Ph.D. from The University of Tokyo in 1998. His research interest includes fractal sets arising from dynamical systems in four-dimensional space and their visualization using virtual reality systems.

Prof. Hirose Michitaka focuses on developing a high-level user interface that unites human and computer, called Cybernetic Interface. Starting with virtual reality technology, he seeks to research and develop such interfaces. Some of the research themes that he is working on are image-based rendering technology, augmented reality technology, multimodal and crossmodal interfaces, and technology concerning lifelog and Big data. He is also interested in the contents that are made from such technology, and social implementation of those technologies. Some projects that he is working on are the Digital Museum project, the Digital Public Art project, and the Senior Cloud project.

PinpointFly: An Egocentric Position-pointing Drone Interface using Mobile AR

Presenter(s): Linfeng Chen, Tohoku University, Japan
Akiyuki Ebi, Tohoku University, Japan
Kazuki Takashima, Tohoku University, Japan
Kazuyuki Fujita, Tohoku University, Japan
Yoshifumi Kitamura, Tohoku University, Japan

Description: We proposed PinpointFly, a novel AR-based egocentric drone manipulation interface that increases spatial perception and manipulation accuracy by overlaying a cast shadow on the ground. We designed and implemented a proof-of-concept prototype using a motion tracking system, see-through AR techniques and a programmable drone.

Bio of Speaker(s):
Linfeng Chen is a graduate student in ICD lab, Tohoku University. He received his bachelor from Jilin University in China. His research interests are drone, user interface and augmented reality.

Akiyuki Ebi is a master student in ICD lab. He received his bachelor from National Institute of Technology, Toyama College in Japan. His research interests are drone, user interface.

Dr. Takashima received his Ph.D from the Graduate School of Information Science and Technology at Osaka University in 2008. He then worked as an assistant professor at Osaka University, and joined Tohoku University's Research Institute of Electrical Communication as an assistant professor in 2011. Dr. Takashima was promoted to the rank of associate professor at Tohoku University in 2018.

Dr. Kazuyuki Fujita is an Assistant Professor at Tohoku University. He received his Ph.D. in Information Science and Technology from Osaka University in 2013. His research interests are spatial user interface and virtual reality.

Dr. Yoshifumi Kitamura has been Professor in the Research Institute of Electrical Communication, Tohoku University, since 2010. His research interests include interactive content design, human computer interactions, 3D user interfaces, virtual reality, and related fields. He serves in positions such as the Japan Liaison of IFIP TC-13, Japan Liaison and Chair of ACM SIGCHI Asian Development Committee, Chair of Japan ACM SIGCHI Chapter, Steering Committee Chair of ACM VRST, and Conference Chair of ACM SIGGRAPH Asia 2015, General Chair of CHI 2021. His formal education was obtained at Osaka University, B.Sc. (1985); M.Sc. (1987); and Ph.D. (1996).

AlteredWind: Manipulating Perceived Direction of the Wind by Cross-Modal presentation of Visual, Audio and Wind Stimuli

Presenter(s): Kenichi Ito, The University of Tokyo, Japan
Yuki Ban, The University of Tokyo, Japan
Shin'ichi Warisawa, The University of Tokyo, Japan

Description: We developed AlteredWind, a multisensory wind display system that manipulates users' perception of wind by the integration of visual, audio, and wind stimuli. AlteredWind present images of flowing particles and three-dimensional (3D) sounds of wind together with the wind to induce the visuo-haptic and audio-haptic cross-modal effect.

Bio of Speaker(s):
Kenichi Ito received his bachelor of engineering degree from the Faculty of Engineering, the University of Tokyo in 2019. He is a master course student of the Graduate School of Frontier Sciences, the University of Tokyo now. His research interest is improving presence and immersion of virtual reality by presenting the feelings of wind.

Yuki Ban received the M.S. degree and the Ph.D. degree in Information Science and Technology from the University of Tokyo, Tokyo, Japan, in 2013 and 2016, respectively. He was a researcher at Xcoo inc. research from 2016 to 2017. He is currently an Assistant Professor of Graduate School of Frontier Sciences at the University of Tokyo. His current research interests focus on modifying spatial perception using the visuo-haptic interaction.

Prof. Warisawa is a professor of Department of Human and Engineered Environmental Studies at the University of Tokyo. From April of 1994 to March of 2000, he served as an Assistant Professor of Tokyo Institute of Technology, from April of 2000 has been working at the University of Tokyo. He was a visiting researcher at Massachusetts Institute of Technology from 2010 to 2011, and a visiting professor at Université Jean Monnet in 2016. His current research focuses on wearable/ambient human health monitoring. Research cores are nano/micro sensing devices fabrication and sensing information technology application for human well-being.

360Drops: Mixed Reality Remote Collaboration using 360 Panorama in 3D Scene

Presenter(s): Theophilus Teo, University of South Australia, Australia
Gun A. Lee, University of South Australia, Australia
Mark Billinghurst, University of South Australia, Australia
Matt Adcock, CSIRO's Data61, Australia

Description: Mixed Reality Remote Collaboration that enables a remote user to create multiple 360 panoramas in a 3D scene that can be accessed and interacted in different ways to communicate with a local user. This research is intended to investigate enchantment in terms of performance and technique for remote collaboration.

Bio of Speaker(s):
Theophilus Teo is a Ph.D. candidate with particular interests in Augmented Reality safety and awareness and remote collaboration. Before he commenced his Ph.D study, he graduated with a first-class Honours from Bachelor of Software Engineering in University of South Australia. On year 2016, he participated in an ICT (Information and Communications Technology) project titled “VR for Big Data analytics” sponsored by CSIRO. Currently, he is doing his research degree under the supervision of Prof. Mark Billinghurst and Dr. Gun Lee in Empathic Computing Lab at University of South Australia.

Dr. Gun Lee is a Senior Research Fellow at the University of South Australia (UniSA) investigating interaction and visualization methods for sharing virtual experiences in Augmented Reality (AR), Mixed Reality (MR), and immersive 3D environments. Recently, using AR and wearable interfaces to improve remote collaborative experience has been one of his main research themes. Extending this research into sharing richer communication cues and scaling up to a larger group of participants are the next steps he is working on.

Mark Billinghurst is Professor of Human Computer Interaction at the University of South Australia in Adelaide, Australia. He earned a PhD in 2002 from the University of Washington and researches innovative computer interfaces that explore how virtual and real worlds can be merged, publishing over 300 papers in topics such as wearable computing, Augmented Reality and mobile interfaces. He was Director of the HIT Lab NZ at the University of Canterbury and he has previously worked at British Telecom, Nokia, Google and the MIT Media Laboratory.

Dr Matt Adcock is a Research Engineer, Experimental Scientist and Director of the CSIRO Data61 Immersive Environments Lab. He uses Interactive Computer Graphics, Computational Imaging and User Experience Innovation to build systems that enable new connections between the digital and physical worlds. Dr Adcock has published over 50 research papers and holds degrees from the Australian National University (ANU), the University of South Australia (UniSA) and the Massachusetts Institute of Technology (MIT). He is also an adjunct researcher at ANU.

Enhancing Suspension Activities in Virtual Reality with Body-Scale Kinesthetic Force Feedbacks

Presenter(s): Yuan-Syun Ye, National Chiao Tung University, Institute of Multimedia Engineering, Taiwan
Hsin-Yu Chen, National Chiao Tung University, Taiwan
Liwei Chan, National Chiao Tung University, Taiwan

Description: This work presents a suspension kit that can suggest a range of body postures and thus enables various exercise styles of users in virtual reality. Users immersed in an exercise experience perceive active kinesthetic force feedback produced by the kit via suspending their weight with arm exertion.

Bio of Speaker(s):
Yuan-Syun Ye is a graduate student at National Chiao Tung University Institute of Multimedia Engineering Multimedia Engineering Institute in Taiwan. His research field is on Human-Computer Interaction and his research topic is to design haptic feedback sport for virtual reality.

Hsin-Yu Chen is a graduate student at National Chiao Tung University Institute of Multimedia Engineering Multimedia Engineering Institute in Taiwan. His research field is on Human-Computer Interaction and his research topic is to design haptic feedback sport for virtual reality.

Liwei Chan is an Assistant Professor of National Chiao Tung University in Taiwan. After receiving his Ph.D. from National Taiwan University, he had been an Assistant Professor in Keio Media Design, Keio University. His main research area is Human-Computer Interaction.

Back