avatar

Fangzhou Hong 洪方舟

Ph.D. Student in MMLab@NTU

About Me

Fangzhou Hong is currently a second-year Ph.D. student in the School of Computer Science and Engineering at Nanyang Technological University (MMLab@NTU), supervised by Prof. Ziwei Liu. Previously, he received B.Eng. degree in Software Engineering from Tsinghua University in 2020. His research interests lie on the computer vision and deep learning. Particularly, he is interested in 3D representation learning and its intersection with computer graphics.

News
[2022-05]

One paper (AvatarCLIP) accepted to SIGGRAPH 2022 (journal track).

[2022-03]

One paper (HCMoCo) accepted to CVPR 2022 for Oral presentation.

[2021-09]

One paper (Garment4D) accepted to NeurIPS 2021.

[2021-09]

I am awarded Google PhD Fellowship 2021 (Machine Perception).

[2021-07]

One paper (extended Cylinder3D) accepted by TPAMI.

[2021-03]

Two papers (DS-Net and Cylinder3D) accepted to CVPR 2021.

[2021-01]

Start my journey in MMLab@NTU!

Publications
avatarclip.png

AvatarCLIP: Zero-Shot Text-Driven Generation and Animation of 3D Avatars

Fangzhou Hong*, Mingyuan Zhang*, Liang Pan, Zhongang Cai, Lei Yang, Ziwei Liu

ACM Transactions on Graphics (SIGGRAPH), 2022

AvatarCLIP empowers layman users to customize a 3D avatar with the desired shape and texture, and drive the avatar with the described motions using solely natural languages.

hcmoco.png

Versatile Multi-Modal Pre-Training for Human-Centric Perception

Fangzhou Hong, Liang Pan, Zhongang Cai, Ziwei Liu

IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2022 (Oral)

The first to leverage the multi-modal nature of human data (e.g. RGB, depth, 2D key-points) for effective human-centric representation learning.

garment4d.png

Garment4D: Garment Reconstruction from Point Cloud Sequences

Fangzhou Hong, Liang Pan, Zhongang Cai, Ziwei Liu

35th Conference on Neural Information Processing Systems (NeurIPS), 2021

The first attempt at separable and interpretable garment reconstruction from point cloud sequences, especially challenging loose garments.

dsnet.png

LiDAR-based Panoptic Segmentation via Dynamic Shifting Network

Fangzhou Hong, Hui Zhou, Xinge Zhu, Hongsheng Li, Ziwei Liu

IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2021

Rank 1st in the public leaderboard of SemanticKITTI panoptic segmentation (2020-11-16); A learnable clustering module is designed to adapt kernel functions to complex point distributions.

cylinder3d.png

Cylindrical and Asymmetrical 3D Convolution Networks for LiDAR Segmentation

Xinge Zhu*, Hui Zhou*, Tai Wang, Fangzhou Hong, Yuexin Ma, Wei Li, Hongsheng Li, Dahua Lin

IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2021 (Oral)

Rank 1st in the public leaderboard of SemanticKITTI semantic segmentation (2020-11-16); Cylindrical 3D convolution is designed to explore the 3D geometric pattern of LiDAR point clouds.

cylinder3d_extend.png

Cylindrical and Asymmetrical 3D Convolution Networks for LiDAR-based Perception

Xinge Zhu, Hui Zhou, Tai Wang, Fangzhou Hong, Wei Li, Yuexin Ma, Hongsheng Li, Ruigang Yang, Dahua Lin

IEEE Transactions on Pattern Analysis and Machine Intelligence (TPAMI), 2021

Journal Extension of the CVPR21 version; Extend the cylindrical convolution to more general LiDAR-based perception tasks.

lrcnet.png

LRC-Net: Learning Discriminative Features on Point Clouds by Encoding Local Region Contexts

Xinhai Liu, Zhizhong Han, Fangzhou Hong, Yu-Shen Liu, Matthias Zwicker

Computer Aided Geometric Design, 2020, 79: 101859. (SCI, 2017 Impact factor: 1.421, CCF B)

To learn discriminative features on point clouds by encoding the fine-grained contexts inside and among local regions simultaneously.

Technical Report
humman.png

HuMMan: Multi-Modal 4D Human Dataset for Versatile Sensing and Modeling

Zhongang Cai*, Daxuan Ren*, Ailing Zeng*, Zhengyu Lin*, Tao Yu*, Wenjia Wang*, Xiangyu Fan, Yang Gao, Yifan Yu, Liang Pan, Fangzhou Hong, Mingyuan Zhang, Chen Change Loy, Lei Yang, Ziwei Liu

arXiv Preprint, 2022

A large-scale multi-modal (color images, point clouds, keypoints, SMPL parameters, and textured meshes) 4D human dataset with 1000 human subjects, 400k sequences and 60M frames.

4D-DS-Net.png

LiDAR-based 4D Panoptic Segmentation via Dynamic Shifting Network

Fangzhou Hong, Hui Zhou, Xinge Zhu, Hongsheng Li, Ziwei Liu

arXiv Preprint, 2022

Extension of the CVPR21 Version; Extend DS-Net to 4D panoptic LiDAR segmentation by the temporally unified instance clustering on aligned LiDAR frames.

Education & Experiences
Nanyang Technological University
Singapore
Jan. 2021 - Present
Ph.D. Student in MMLab@NTU
The Chinese University of Hong Kong
Hong Kong, China
Jul. 2020 - Dec. 2020
Research Assistant in MMLab
SenseTime Group Limited
Beijing, China
Feb. 2019 - Dec. 2019
Research Intern
Tsinghua University
Beijing, China
Aug. 2016 - Jun. 2020
Bachelor Degree in Software Engineering
High GPA 3.93/4.0. Ranking 1/84.
Awards & Scholarships

Google PhD Fellowship 2021

2021

Outstanding Undergraduate Thesis of Tsinghua University

2020

Outstanding Graduate of Tsinghua University

2020

Outstanding Graduate of Beijing

2020

Outstanding Graduate of School of Software, Tsinghua University

2020

ICBC Scholarship (Top 3%)

2019

Hua Wei Scholarship (Top 1%)

2018

Tung OOCL Scholarship (Top 5%)

2017