Chang-E: A High-Quality Motion Capture Dataset of Chinese Classical Dunhuang Dance

Zeyu Wang1,2, Chengan He4, Zhe Yan1, Jiashun Wang5, Yingke Wang6, Junhua Liu7, Angela Shen8, Mengying Zeng3, Holly Rushmeier4, Huazhe Xu9, Borou Yu3, Chenchen Lu3, Eugene Y. Wang3
1HKUST(GZ), 2HKUST, 3Harvard University, 4Yale University, 5Carnegie Mellon University, 6Stanford University, 7CUHK(SZ), 8Northeastern University, 9Tsinghua University,
Paper Cave Dance Project

The generation process of the Chang-E dataset and its applications. We captured dance artists performing Dunhuang dance using Motion Capture technology, fitted the 3D motion data into SMPL format, and further created reinterpretations and visual effects based on the dataset. The mural image is from Mogao Cave 112 in Dunhuang. The stage drama image is from Silk Road, Flower, and Rain.

Abstract

Derived from the mural drawings in the UNESCO-listed Mogao Caves, Dunhuang dance has unique cultural value but faces challenges of digitization and preservation.

In this paper, we introduce the first open comprehensive motion capture dataset of Dunhuang dance, Chang-E, including full-body movements documented across eight categories, totaling 40 minutes of professional dance. This dataset contains three formats: skeleton data acquired from motion capture, body mesh generated from skeleton using machine learning, and multiview videos recorded on site. The dataset supports various creative applications for Dunhuang dance culture, as demonstrated by an immersive new media exhibition.

Through the curation process, we applied motion inbetweening algorithms to concatenate different dance sequences for choreography. Also, these reinterpreted dance sequences are synchronized with music using retiming techniques, augmenting the rhythms and harmony between the music and dance performance. Furthermore, we applied visual effects on the regenerated motion sequences of digital dancers, achieving artistic and appealing visual results echoing Buddhist discourses of meditation and bodily cognition. The Chang-E dataset enables digital preservation and creative reimagination of Dunhuang dance, offering not only high-quality data but also an interdisciplinary collaboration framework for future graphics and cultural heritage research.

Video

Dataset Preview

To access the complete dataset, please fill out the License Agreement and send it to zyan698 AT connect.hkust-gz.edu.cn with cc to zeyuwang AT hkust-gz.edu.cn.

Flying Apsaras

Flying Apsaras

Duration: 6 min 32 s

Performer(s): One female

Lotus Steps

Lotus Steps

Duration: 3 min 51 s

Performer(s): One female

Thirty-Six Postures

Thirty-Six Postures

Duration: 5 min 29 s

Performer(s): Two females

Thirty-Six Postures

Thirty-Six Postures

Duration: 4 min 31 s

Performer(s): Two males

Revelation Meditation

Revelation Meditation

Duration: 3 min 24 s

Performer(s): One female & one male seperately

Sogdian Whirl

Sogdian Whirl

Duration: 1 min 18 s

Performer(s): One male

Playing the Pipa Behind the Back

Playing the Pipa Behind the Back

Duration: 2 min 08 s

Performer(s): One male

Lei Gong Drum

Lei Gong Drum

Duration: 4 min 51 s

Performer(s): Two males

BibTeX


        @article{Wang2024Chang-E,
          author = {Wang, Zeyu and He, Chengan and Yan, Zhe and Wang, Jiashun and Wang, Yingke and Liu, Junhua and Shen, Angela and Zeng, Mengying and Rushmeier, Holly and Xu, Huazhe and Yu, Borou and Lu, Chenchen and Wang, Eugene Y.},
          title = {Chang-E: A High-Quality Motion Capture Dataset of Chinese Classical Dunhuang Dance},
          year = {2024},
          publisher = {Association for Computing Machinery},
          address = {New York, NY, USA},
          issn = {1556-4673},
          url = {https://doi.org/10.1145/3709000},
          journal = {J. Comput. Cult. Herit.},
          month = dec,
          }