# ai_choreographer

**Repository Path**: mirrors/ai_choreographer

## Basic Information

- **Project Name**: ai_choreographer
- **Description**: 这个包包含 AI Choreographer 的模型实现和训练基础设施，包括 FACT 模型实现
- **Primary Language**: Python
- **License**: Apache-2.0
- **Default Branch**: main
- **Homepage**: https://www.oschina.net/p/ai_choreographer
- **GVP Project**: No

## Statistics

- **Stars**: 12
- **Forks**: 2
- **Created**: 2021-09-14
- **Last Updated**: 2025-09-20

## Categories & Tags

**Categories**: machine-learning

**Tags**: None

## README

# AI Choreographer: Music Conditioned 3D Dance Generation with AIST++ [ICCV-2021].

## Overview

This package contains the model implementation and training infrastructure of
our AI Choreographer. 

## Get started

#### Pull the code
```
git clone https://github.com/liruilong940607/mint --recursive
```
Note here `--recursive` is important as it will automatically clone the submodule ([orbit](https://github.com/tensorflow/models/tree/master/orbit)) as well.

#### Install dependencies
```
conda create -n mint python=3.7
conda activate mint
conda install protobuf numpy
pip install tensorflow absl-py tensorflow-datasets librosa

sudo apt-get install libopenexr-dev
pip install --upgrade OpenEXR
pip install tensorflow-graphics tensorflow-graphics-gpu

git clone https://github.com/arogozhnikov/einops /tmp/einops
cd /tmp/einops/ && pip install . -U

git clone https://github.com/google/aistplusplus_api /tmp/aistplusplus_api
cd /tmp/aistplusplus_api && pip install -r requirements.txt && pip install . -U
```
Note if you meet environment conflicts about numpy, you can try with `pip install numpy==1.20`. 

#### Get the data
See the [website](https://google.github.io/aistplusplus_dataset/)

#### Get the checkpoint
Download from google drive [here](https://drive.google.com/drive/folders/17GHwKRZbQfyC9-7oEpzCG8pp_rAI0cOm?usp=sharing), and put them to the folder `./checkpoints/`

#### Run the code

1. complie protocols
```
protoc ./mint/protos/*.proto
```

2. preprocess dataset into tfrecord
```
python tools/preprocessing.py \
    --anno_dir="/mnt/data/aist_plusplus_final/" \
    --audio_dir="/mnt/data/AIST/music/" \
    --split=train
python tools/preprocessing.py \
    --anno_dir="/mnt/data/aist_plusplus_final/" \
    --audio_dir="/mnt/data/AIST/music/" \
    --split=testval
```

3. run training
```
python trainer.py --config_path ./configs/fact_v5_deeper_t10_cm12.config --model_dir ./checkpoints
```
Note you might want to change the `batch_size` in the config file if you meet OUT-OF-MEMORY issue.

4. run testing and evaluation
```
# caching the generated motions (seed included) to `./outputs`
python evaluator.py --config_path ./configs/fact_v5_deeper_t10_cm12.config --model_dir ./checkpoints
# calculate FIDs
python tools/calculate_scores.py
```


## Citation

```bibtex
@inproceedings{li2021dance,
  title={AI Choreographer: Music Conditioned 3D Dance Generation with AIST++},
  author={Ruilong Li and Shan Yang and David A. Ross and Angjoo Kanazawa},
  booktitle = {The IEEE International Conference on Computer Vision (ICCV)},
  year = {2021}
}
```