avatar

Fangzhou Hong 洪方舟

Ph.D. Student in MMLab@NTU

About Me

Fangzhou Hong is currently a third-year Ph.D. student in the School of Computer Science and Engineering at Nanyang Technological University (MMLab@NTU), supervised by Prof. Ziwei Liu. Previously, he received B.Eng. degree in Software Engineering from Tsinghua University in 2020. His research interests lie on the computer vision and deep learning. Particularly, he is interested in 3D representation learning and its intersection with computer graphics.

News
[2023-01]

One paper (EVA3D) accepted to ICLR 2023 as Spotlight.

[2022-07]

One paper (HuMMan) accepted to ECCV 2022 for Oral presentation.

[2022-05]

One paper (AvatarCLIP) accepted to SIGGRAPH 2022 (journal track).

[2022-03]

One paper (HCMoCo) accepted to CVPR 2022 for Oral presentation.

[2021-09]

One paper (Garment4D) accepted to NeurIPS 2021.

[2021-09]

I am awarded Google PhD Fellowship 2021 (Machine Perception).

[2021-07]

One paper (extended Cylinder3D) accepted by TPAMI.

[2021-03]

Two papers (DS-Net and Cylinder3D) accepted to CVPR 2021.

[2021-01]

Start my journey in MMLab@NTU!

Publications
EVA3D.gif

EVA3D: Compositional 3D Human Generation from 2D Image Collections

International Conference on Learning Representations (ICLR), 2023 (Spotlight)

EVA3D is a high-quality unconditional 3D human generative model that only requires 2D image collections for training.

humman.png

HuMMan: Multi-Modal 4D Human Dataset for Versatile Sensing and Modeling

Zhongang Cai*, Daxuan Ren*, Ailing Zeng*, Zhengyu Lin*, Tao Yu*, Wenjia Wang*, Xiangyu Fan, Yang Gao, Yifan Yu, Liang Pan, Fangzhou Hong, Mingyuan Zhang, Chen Change Loy, Lei Yang, Ziwei Liu

European Conference on Computer Vision (ECCV), 2022 (Oral)

A large-scale multi-modal (color images, point clouds, keypoints, SMPL parameters, and textured meshes) 4D human dataset with 1000 human subjects, 400k sequences and 60M frames.

avatarclip.png

AvatarCLIP: Zero-Shot Text-Driven Generation and Animation of 3D Avatars

Fangzhou Hong*, Mingyuan Zhang*, Liang Pan, Zhongang Cai, Lei Yang, Ziwei Liu

ACM Transactions on Graphics (SIGGRAPH), 2022

AvatarCLIP empowers layman users to customize a 3D avatar with the desired shape and texture, and drive the avatar with the described motions using solely natural languages.

hcmoco.png

Versatile Multi-Modal Pre-Training for Human-Centric Perception

Fangzhou Hong, Liang Pan, Zhongang Cai, Ziwei Liu

IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2022 (Oral)

The first to leverage the multi-modal nature of human data (e.g. RGB, depth, 2D key-points) for effective human-centric representation learning.

garment4d.png

Garment4D: Garment Reconstruction from Point Cloud Sequences

Fangzhou Hong, Liang Pan, Zhongang Cai, Ziwei Liu

35th Conference on Neural Information Processing Systems (NeurIPS), 2021

The first attempt at separable and interpretable garment reconstruction from point cloud sequences, especially challenging loose garments.

dsnet.png

LiDAR-based Panoptic Segmentation via Dynamic Shifting Network

Fangzhou Hong, Hui Zhou, Xinge Zhu, Hongsheng Li, Ziwei Liu

IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2021

Rank 1st in the public leaderboard of SemanticKITTI panoptic segmentation (2020-11-16); A learnable clustering module is designed to adapt kernel functions to complex point distributions.

cylinder3d.png

Cylindrical and Asymmetrical 3D Convolution Networks for LiDAR Segmentation / LiDAR-based Perception

Xinge Zhu*, Hui Zhou*, Tai Wang, Fangzhou Hong, Yuexin Ma, Wei Li, Hongsheng Li, Dahua Lin

IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2021 (Oral) IEEE Transactions on Pattern Analysis and Machine Intelligence (TPAMI), 2021

Rank 1st in the public leaderboard of SemanticKITTI semantic segmentation (2020-11-16); Cylindrical 3D convolution is designed to explore the 3D geometric pattern of LiDAR point clouds. Further extend the cylindrical convolution to more general LiDAR-based perception tasks.

lrcnet.png

LRC-Net: Learning Discriminative Features on Point Clouds by Encoding Local Region Contexts

Xinhai Liu, Zhizhong Han, Fangzhou Hong, Yu-Shen Liu, Matthias Zwicker

Computer Aided Geometric Design, 2020, 79: 101859. (SCI, 2017 Impact factor: 1.421, CCF B)

To learn discriminative features on point clouds by encoding the fine-grained contexts inside and among local regions simultaneously.

Technical Reports
SHERF.png

SHERF: Generalizable Human NeRF from a Single Image

Shoukang Hu*, Fangzhou Hong*, Liang Pan, Haiyi Mei, Lei Yang, Ziwei Liu

arXiv Preprint, 2023

Reconstruct human NeRF from a single image in one forward pass!

ReMoDiffuse.gif

ReMoDiffuse: Retrieval-Augmented Motion Diffusion Model

arXiv Preprint, 2023

ReMoDiffuse is a diffusion-model-based motion generation framework that integrates a retrieval mechanism to refine the denoising process, which enhances the generalizability and diversity.

MotionDiffusion.png

MotionDiffuse: Text-Driven Human Motion Generation with Diffusion Model

Mingyuan Zhang*, Zhongang Cai*, Liang Pan, Fangzhou Hong, Xinying Guo, Lei Yang, Ziwei Liu

arXiv Preprint, 2022

The first diffusion-model-based text-driven motion generation framework with probabilistic mapping, realistic synthesis and multi-level manipulation ability.

4D-DS-Net.png

LiDAR-based 4D Panoptic Segmentation via Dynamic Shifting Network

Fangzhou Hong, Hui Zhou, Xinge Zhu, Hongsheng Li, Ziwei Liu

arXiv Preprint, 2022

Extension of the CVPR21 Version; Extend DS-Net to 4D panoptic LiDAR segmentation by the temporally unified instance clustering on aligned LiDAR frames.

Education & Experiences
Nanyang Technological University
Singapore
Jan. 2021 - Present
Ph.D. Student in MMLab@NTU
Tsinghua University
Beijing, China
Aug. 2016 - Jun. 2020
Bachelor Degree in Software Engineering
High GPA 3.93/4.0. Ranking 1/84.
Awards & Scholarships

Google PhD Fellowship 2021

2021

Outstanding Undergraduate Thesis of Tsinghua University

2020

Outstanding Graduate of Tsinghua University

2020

Outstanding Graduate of Beijing

2020

Outstanding Graduate of School of Software, Tsinghua University

2020

ICBC Scholarship (Top 3%)

2019

Hua Wei Scholarship (Top 1%)

2018

Tung OOCL Scholarship (Top 5%)

2017
Invited Talks [Youtube]
Academic Services

Conference Reviewer: CVPR’21/23, ICCV’23, NeurIPS’22, ICML’23, SIGGRAPH’23, AAAI’21/23

Journal Reviewer: TPAMI, IJCV, TCSVT, JABES, PR

Teaching
[2022]

NTU CE/CZ1115 Introduction to Data Science and Artificial Intelligence (Teaching Assistant)

[2022]

NTU CE2003 Digital System Design (Teaching Assistant)

[2021]

NTU CE/CZ1115 Introduction to Data Science and Artificial Intelligence (Teaching Assistant)

[2021]

NTU SC1013 Physics for Computing (Teaching Assistant)