Skip to content
forked from xg-chu/GAGAvatar

[NeurIPS 2024] Generalizable and Animatable Gaussian Head Avatar

License

Notifications You must be signed in to change notification settings

MrNeRF/GAGAvatar

 
 

Repository files navigation

     

Xuangeng Chu1Tatsuya Harada1,2
1The University of Tokyo, 2RIKEN AIP

🤩 NeurIPS 2024 🤩

drawing
GAGAvatar reconstructs controllable 3D head avatars from single images.
GAGAvatar achieves one-shot 3DGS-based head reconstruction and ⚡️real-time⚡️ reenactment.
🔥 More results can be found in our Project Page. 🔥

Installation

Build environment

conda env create -f environment.yml
conda activate GAGAvatar

Install the 3DGS renderer

What’s the difference between this version and the original 3DGS?
  • We changed the number of channels so that 3D Gaussians carry 32-dim features.
  • We changed the package name to avoid conflict with the original Gaussian splash.
git clone --recurse-submodules [email protected]:xg-chu/diff-gaussian-rasterization.git
pip install ./diff-gaussian-rasterization
rm -rf ./diff-gaussian-rasterization

Prepare resources

Prepare resources with bash ./build_resources.sh.

Also prepare resources for GAGAvatar_track using:

cd core/libs/GAGAvatar_track
bash ./build_resources.sh

Quick Start Guide

Driven by another image:

# This will track the images online, which is slow.
python inference.py -d ./demos/examples/2.jpg -i ./demos/examples/1.jpg

Driven by a tracked video:

python inference.py -d ./demos/drivers/obama -i ./demos/examples/1.jpg

Driven by a tracked image_lmdb:

python inference.py -d ./demos/drivers/vfhq_demo -i ./demos/examples/1.jpg

To test the inference speed, refer to the speed_test() function in inference.py.

To test your own images online, refer to lines 52-55 in inference.py.

To test your own driving sequences (videos/images), refer to GAGAvatar_track and demo sequences to build your own tracking sequence.

Citation

If you find our work useful in your research, please consider citing:

@inproceedings{
    chu2024gagavatar,
    title={Generalizable and Animatable Gaussian Head Avatar},
    author={Xuangeng Chu and Tatsuya Harada},
    booktitle={The Thirty-eighth Annual Conference on Neural Information Processing Systems},
    year={2024},
    url={https://openreview.net/forum?id=gVM2AZ5xA6}
}

Acknowledgements

Some part of our work is built based on FLAME, StyleMatte, EMICA and VGGHead. The GAGAvatar Logo is designed by Caihong Ning. We also thank the following projects for sharing their great work.

About

[NeurIPS 2024] Generalizable and Animatable Gaussian Head Avatar

Resources

License

Stars

Watchers

Forks

Releases

No releases published

Packages

No packages published

Languages

  • Python 99.4%
  • Shell 0.6%