Research
My research interests are efficient neural rendering technology, including: 4D Reconstruction, 3D Generation etc.
* Equal contribution.
|
|
Dynamic 2D Gaussians: Geometrically accurate radiance fields for dynamic objects
Shuai Zhang*,
Guanjun Wu*,
Xinggang Wang,
Bin Feng,
Wenyu Liu
ArXiv, 2024
[
Paper] [
Code]
we propose a novel representation that can reconstruct accurate meshes from sparse image input, named Dynamic 2D Gaussians (D-2DGS).
We adopt 2D Gaussians for basic geometry representation and use sparse-controlled points to capture 2D Gaussian's deformation. By extracting the object mask from the rendered high-quality image and masking the rendered depth map, a high-quality dynamic mesh sequence of the object can be extracted.
|
|
GaussianDreamerPro: Text to Manipulable 3D Gaussians with Highly Enhanced Quality
Taoran Yi,
Jiemin Fang,
Zanwei Zhou,
Junjie Wang,
Guanjun Wu,
Lingxi Xie,
Xiaopeng Zhang,
Wenyu Liu,
Xinggang Wang,
Qi Tian
ArXiv, 2024
[
Paper] [
Page] [
Code]
Aiming at highly enhancing the generation quality, we propose a novel framework named GaussianDreamerPro. The main idea is to bind Gaussians to reasonable geometry, which evolves over the whole generation process. Along different stages of our framework, both the geometry and appearance can be enriched progressively. The final output asset is constructed with 3D Gaussians bound to mesh, which shows significantly enhanced details and quality compared with previous methods. Notably, the generated asset can also be seamlessly integrated into downstream manipulation pipelines, e.g. animation, composition, and simulation etc., greatly promoting its potential in wide applications.
|
|
4D Gaussian Splatting for Real-Time Dynamic Scene Rendering
Guanjun Wu*,
Taoran Yi*,
Jiemin Fang,
Lingxi Xie,
Xiaopeng Zhang,
Wei Wei,
Wenyu Liu,
Qi Tian,
Xinggang Wang
CVPR, 2024
[
Paper] [
Page] [
Code]
Ranked 2nd in CVPR paper influence, 2000+ GitHub stars.
In 4D-GS, a novel explicit representation containing both 3D Gaussians and 4D neural voxels is proposed. A decomposed neural voxel encoding algorithm inspired by HexPlane is proposed to efficiently build Gaussian features from
4D neural voxels and then a lightweight MLP is applied to predict Gaussian deformations at novel timestamps. Our 4D-GS method achieves real-time rendering under high resolutions, 82 FPS at an 800 X 800 resolution on an
RTX 3090 GPU while maintaining comparable or better quality than previous state-of-the-art methods.
|
|
GaussianDreamer: Fast Generation from Text to 3D Gaussian Splatting with Point Cloud Priors
Taoran Yi,
Jiemin Fang,
Junjie Wang,
Guanjun Wu,
Lingxi Xie,
Xiaopeng Zhang,
Wenyu Liu,
Qi Tian,
Xinggang Wang
CVPR, 2024
[
Paper] [
Page] [
Code]
A fast 3D generation framework, named as GaussianDreamer, is proposed, where the 3D diffusion model provides point cloud priors for initialization and the 2D diffusion model enriches the geometry and appearance. Our GaussianDreamer
can generate a high-quality 3D instance within 25 minutes on one GPU, much faster than previous methods, while the generated instances can be directly rendered in real time.
|
|
Fast High Dynamic Range Radiance Fields for Dynamic Scenes
Guanjun Wu*,
Taoran Yi*,
Jiemin Fang,
Wenyu Liu,
Xinggang Wang
3DV, 2024
[
Paper] [
Page] [
Code] [
Data]
we propose a dynamic HDR NeRF framework, named as HDR-HexPlane, which can learn 3D scenes from dynamic 2D images captured with various exposures.With the proposed model, high-quality novel-view images at any time point can
be rendered with any desired exposure. We further construct a dataset containing multiple dynamic scenes captured with diverse exposures for evaluation.
|
Academic Services
Conference Reviewers
- ACM SIGGRAPH Asia
- Neural Information Processing Systems (NeurIPS)
- International Conference on Learning Representations (ICLR)
- International Conference on Robotics and Automation (ICRA)
Journal Reviewers
- International Journal of Computer Vision (IJCV)
- IEEE Transactions on Circuits and Systems for Video Technology (TCSVT)
|
Internships
- 10/2024 - Present Huawei, ShangHai
- 11/2022 - 3/2023 SenseTime, ShangHai
- 3/2022 - 10/2022 ByteDance, ShangHai
|
Misc
I really like playing games, including but not limited to:
- RPG/ARPG: Black Myth Wukong, Elden Ring, The Witcher 3: Wild Hunt, The Legend of Zelda, Cyberpunk 2077, Baldur's Gate3, Total War, Mount & Blade, Monster Hunter, Dark Soul, Assassin's Creed, Final Fantasy, etc.
- MMO RPG: World of Warcrafts.
- RTS/MOBA: LOL, Age of Empires 4, StarCrafts, etc.
- FPS: BattleField, etc.
- SIM: Civilization, FrostPunk, etc
|
Template is adapted from
Here
Last updated: Oct. 2024
|
|