DCARE: Dynamic Clothed Avatar Reconstruction with Normal-Aware Gaussian Splatting from Monocular Videos
Fangzheng Tian, Sungchan Kim
Abstract
Reconstructing dynamic clothed avatars from monocular video using 3D Gaussian Splatting (3DGS) remains challenging due to complex clothing deformations and appearance inconsistencies. We present DCARE, an integrated framework that addresses these challenges by directly leveraging surface curvature, derived from normal maps, to guide Gaussian density control. At the core of our approach is a curvature-guided Adaptive Density Control (ADC) mechanism that dynamically allocates Gaussian density based on local geometric complexity. This design fundamentally differs from previous normal-based 3DGS methods, which use normals primarily for regularization losses rather than direct density management. DCARE further employs a synergistic deformation model that combines adaptively determined virtual bones for handling large movements with normal-guided nonrigid refinement for capturing fine grained details. Appearance consistency is enhanced using a VQ-VAE learned global codebook, providing a robust prior decoupled from geometry. DCARE achieves state-of-the-art results in challenging dynamic clothing reconstruction, excelling particularly in complex scenarios such as loose garments and semi-transparent fine details, significantly enhancing geometric accuracy and appearance fidelity.
Demo video
- The demo video is best viewed in 1080p.
Part I. Comprehensive Evaluations on UBC-Fashion, 4D-Dress, and DNA-Rendering datasets
- Drag the slider between left and right to compare.
- It is highly recommended that you click on the images to enlarge them and clearly see the performance advantage of our method. Press the Esc key to return.
- If you encounter a synchronization problem between two images, please reload the web page.
Dataset: UBC-Fashion Sequence: 91+bCFG1jOS
Dataset: UBC-Fashion Sequence: 91Ile3zLhMS
Dataset: UBC-Fashion Sequence: 91QFEra7jDS
Dataset: 4D-DRESS ; Sequence: 00122_outer_take_09_0004
Dataset: 4D-DRESS ; Sequence: 00129_inner_take_01_0004
Dataset: 4D-DRESS ; Sequence: 00148_outer_take_12_0004
Dataset: 4D-DRESS ; Sequence: 00152_outer_take_10_0004
Dataset: DNA-Rendering Sequence: 4K4D-0013_03_26
Dataset: DNA-Rendering Sequence: 4K4D-0019_08_26
Dataset: DNA-Rendering Sequence: Part1-0102_02_26
Dataset: DNA-Rendering Sequence: Part1-0128_04_14
Part Ⅱ: Rendering Results
Dataset: 4D-DRESS Sequence: 00122_outer_take_10 Click on the image to enlarge and view details.
Dataset: 4D-DRESS Sequence: 00122_outer_take_12 Click on the image to enlarge and view details.
Dataset: 4D-DRESS Sequence: 00129_inner_take_21 Click on the image to enlarge and view details.
Dataset: 4D-DRESS Sequence: 00129_inner_take_22 Click on the image to enlarge and view details.
Dataset: 4D-DRESS Sequence: 00148_outer_take_13 Click on the image to enlarge and view details.
Dataset: 4D-DRESS Sequence: 00148_outer_take_16 Click on the image to enlarge and view details.
Dataset: 4D-DRESS Sequence: 00151_inner_take_06 Click on the image to enlarge and view details.
Dataset: 4D-DRESS Sequence: 00151_inner_take_08 Click on the image to enlarge and view details.
Dataset: 4D-DRESS Sequence: 00190_outer_take_13 Click on the image to enlarge and view details.
Dataset: 4D-DRESS Sequence: 00190_outer_take_14 Click on the image to enlarge and view details.
Dataset: DNA-Rendering Sequence: 0012_11 Click on the image to enlarge and view details.
Dataset: DNA-Rendering Sequence: 0013_09 Click on the image to enlarge and view details.
Dataset: DNA-Rendering Sequence: 0013_11 Click on the image to enlarge and view details.
Dataset: DNA-Rendering Sequence: 0113_06 Click on the image to enlarge and view details.
Part Ⅲ: Applications
Free View Rendering
Avatar Animation in Novel Poses
Reconstruction from In-the-wild Videos
Video Credit: https://www.youtube.com/shorts/dNMmuHWynRA
Video Credit: https://www.youtube.com/shorts/opHSDbGn4mE
Video Credit: https://www.youtube.com/shorts/kGTKT8dSL-k
Video Credit: https://www.youtube.com/shorts/SUFOcqfBv1g
Video Credit: https://www.youtube.com/shorts/-g4-blRQcKA
Rendering Animals
References
[1] GART -- Lei J, Wang Y, Pavlakos G, et al. Gart: Gaussian articulated template models[C]//Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 2024: 19876-19887.
[2] GaussianAvatar -- Hu L, Zhang H, Zhang Y, et al. Gaussianavatar: Towards realistic human avatar modeling from a single video via animatable 3d gaussians[C]//Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 2024: 634-644.