This project introduces a novel approach to 3D human reconstruction using 3D Gaussian splatting techniques. By leveraging advanced splatting methods, we enhance the precision and detail in the reconstruction of clothed human figures from monocular video sources. Our method combines the strengths of implicit and explicit geometric representations, ensuring high fidelity and temporal coherence across the reconstructed sequences. The use of 3D Gaussian splatting allows for a more nuanced and detailed capture of the human form, making it a significant advancement in the field of 3D reconstruction.
Our method represents the human body in canonical space with gaussians parameterized by 3D gaussian centers x, rotations q, scales S, opacity αo, colors SH, skinning weight w and its associated parent triangle ix. It takes body pose (θt) of tth frame as input and applies forward linear blend skinning to transform v' to posed space vp. We compute gaussian center x from the posed space vertices vp of ix. The normal of parent triangle ix is encoded to SH^n and rasterized to obtain the normal map I^n. Then, we apply photometric loss and normal map loss to recover both geometry and color. The GT normal map (Ī^n) is obtained from monocular RGB image (In) using pix2pixHD [Wang et al., 2018] network.
Our method accurately reconstructs surface details like shirt collar and cloths wrinkles.
View Synthesis and 3D Mesh Reconstruction on challenging UGB-Fashion dataset.
Our method produces high fidelity mesh even capturing subtle facial details like hair, ear in 15 seconds of computational budget.
We can feed the SMPL input pose to our iHuman Template Model to synthesize novel poses. Though trained on only A-pose, our method stably renders new image under complex poses.
@inproceedings{3DHumanReconstruction2024,
author = {Authors},
title = {3D Human Reconstruction using 3D Gaussian Splatting},
booktitle = {Conference},
year = {2024}
}