Architecture of Integrated Machine Learning in Low Latency Mobile VR Graphics Pipeline

Speaker(s): Haomiao Jiang, Facebook, United States of America
Rohit Rao Padebettu, Facebook, United States of America
Kazuki Sakamoto, Facebook, United States of America
Behnam Bastani, Facebook, United States of America

Haomiao is a software engineer at Facebook, developing future graphics system and pipeline for mobile VR headset. Haomiao got his PhD degree in Electrical Engineering at Stanford University in 2016. His research interest includes machine learning for graphics and camera image processing pipeline.

Rohit Rao Padebettu is a software engineer in Oculus at Facebook, developing rendering algorithms for next generation VR/AR headsets. His research interests include foveated rendering and reconstruction, ray tracing and neural network accelerated rendering on embedded devices. He received his masters degree in computer science from Stony Brook university.

Kazuki is an experienced software engineer at Facebook in a wide variety of domains, including Low-level programming in UNIX kernel and device drivers, Real-Time programming with real-time OS and Game consoles, User Interface programming with Mobile devices and games. He has deep knowledge of mobile OSs such as Android and iOS.

Dr. Behnam Bastani runs immersive graphics system at AR/VR division of Facebook where his team develops a range of high fidelity graphics systems for untethered VR devices. His team focuses on graphics rendering, optical system, image processing and human visual perception.

Description: We propose a framework to execute machine learning algorithms in the real time mobile VR graphics pipeline to improve efficiency and image quality.

Automatic Generation of Chinese Vector Fonts via Deep Layout Inferring

Speaker(s): Yichen Gao, Peking University, China
Zhouhui Lian, Peking University, China
Yingmin Tang, Peking University, China
Jianguo Xiao, Peking University, China

Yichen Gao received the bachelor degree in software engineering from Dalian University of technology, China, in 2017. He is currently a master candidate at the Wangxuan Institute of Computer Technology, Peking University, China. His main research interests include computer vision and computer graphics.

Zhouhui Lian received the Ph.D. degree from Beihang University, China, in 2011. He worked as a guest researcher at NIST, Gaithersburg, USA, from 2009 to 2011. He is currently an associate professor at the Institute of Computer Science and Technology, Peking University, China. His main research interests include computer graphics, pattern recognition and computer vision.

Yingmin Tang is a senior engineer engaged in the research and development of typesetting software and word processing in Wangxuan Institute of Computer Technology, Peking University. His main research interests include text and graphic image information processing and Chinese information internationalization.

Jianguo Xiao received the master degree from Peking University, China, in 1989. He is currently a professor at the Institute of Computer Science and Technology, Peking University, China. His main research interests include computer graphics, image and video processing, web information processing and text mining.

Description: This paper proposes a data-driven system which integrates the CG-based and deep learning-based methods to generate complete Chinese vector fonts from a small number of designed glyphs.

Enhancing Piecewise Planar Scene Modeling from a Single Image via Multi-View Regularization

Speaker(s): Weijie Xi, University of Science and Technology of China, China
Siyu Hu, University of Science and Technology of China, China
Zhiwei Xiong, University of Science and Technology of China, China
Xuejin Chen, University of Science and Technology of China, China

Weijie Xi is a master candidate in Department of Electronic Engineering and Information Science, University of Science and Technology of China. His research interests focus on geometry in computer vision. Weijie Xi obtained his B.S. from Chongqing University in 2018. He started his master in University of Science and Technology of China in 2018.

Siyu Hu is a Ph.D. candidate in Department of Electronic Engineering and Information Science, University of Science and Technology of China. His research interests focus on capturing, generation, processing and meshing of 3D point cloud. Siyu obtained his B.S. from Hunan University in 2013. He started his Ph.D. in University of Science and Technology of China in 2013.

Zhiwei Xiong received his B.S. and Ph.D. degrees in Electronic Engineering from USTC in 2006 and 2011, respectively. After working at Microsoft Research Asia (MSRA) as a Researcher for five years, he returned USTC as a Professor in 2016. His research interests include computational photography, low-level vision, and biomedical image analysis.

Xuejin Chen is an associate professor of the University of Science and Technology of China. She received the BSc degree in 2003 and the PhD degree in 2008 from the University of Science and Technology of China (USTC). She conducted research as a postdoctoral scholar in the Computer Graphics Lab at Yale University from 2008 to 2010. She visited Stanford University from Feb. to Aug. 2017. Her research interests include 3D modeling, geometry processing, sketch-based content generation, and scene understanding.

Description: We propose a method to address the single-image piece-wise planar 3D reconstruction problem. Our key innovation is to enhance a single-view planar reconstruction network by multi-view regularization during training.

Unpaired Sketch-to-Line Translation via Synthesis of Sketches

Speaker(s): Gayoung Lee, NAVER WEBTOON Corp., NAVER Corporation, South Korea
Dohyun Kim, Chung-Ang University, South Korea
Youngjoon Yoo, Clova AI Research, NAVER Corp., South Korea
Dongyoon Han, Clova AI Research, NAVER Corp., South Korea
Jung-Woo Ha, Clova AI Research, NAVER Corp., South Korea
Jaehyuk Chang, NAVER WEBTOON Corp., South Korea

Gayoung Lee recieved her B.S.(2014) in Computer Science and M.S.(2016) in Electrical Engineering from Korea Advanced Institute of Science and Technology(KAIST). From 2017, she worked at NAVER WEBTOONS Corp. as a researcher. From 2019, she joined NAVER CLOVA OCR team. Her research interests lie on applying machine learning in comics including image to image translation, object detection and image-text understanding.

Dohyun Kim received his B.S. degree in computer science and engineering from Chung-Ang University, Seoul, Korea, in 2018. He is currently working toward M.S. degree in computer science and engineering at Chung-Ang Universitym Seoul Korea. His current interests are in computer vision, image restoration, and image translation.

YoungJoon Yoo received the B.E. degree in electrical engineering and computer science from Seoul National University in 2011, and Ph.D. degrees in electrical and computer engineering from Seoul National University in 2017. In 2017, he has been with the graduate school of convergence science and technology, Seoul National University, where he was an post-doctorial researcher. Since Dec. 2017, he has been working as a research scientist in CLOVA AI Research., NAVER.

Dongyoon Han (S’15) received the B.S., M.S., and Ph.D. degrees in electrical engineering from the Korea Advanced Institute of Science and Technology (KAIST) , Daejeon, South Korea, in 2011, 2013, and 2018, respectively. He is currently research scientist at Research Scientist at CLOVA AI Research, NAVER Corp. His research interests include machine learning and computer vision, especially deep learning-based object classification, detection, and segmentation and unsupervised learning methods. ​

Jung-Woo Ha works as Research Head of Clova AI, NAVER. Also, he serves as an advisory member for several departments of Korean government. He got his BS and PhD from Seoul National University. His research interests contain deep learning-based computer vision, NLP, audio signal modeling, and recommendation. ​

Jaehyuk Chang is a research manager at W Research, NAVER WEBTOON Corp., Korea., where webtoon, the unique digital comic in Korea, is very famous. He received the B.S. (2000) in electrical engineering from KAIST, Daejeon, Korea., and he has been working over 10 years continuously as a software engineer at NAVER, Korea. His current interests are deep learning, computer vision, generative models and etc.

Description: In this paper, we propose a training scheme that requires only unpaired sketch and line images for learning sketch simplification using both rule-based algorithm and deep learning approach.