The Dunhuang Research Institute has been engaged in rescue, restoration, and conservation work on the cultural relics of the Dunhuang Grottoes. Due to the diverse and complex nature of mural diseases and the lack of automated annotation tools, their identification and segmentation require a significant amount of manpower.
“The Mogao Grottoes have over 700 caves, but we have only surveyed and documented a little over 200 of them, and we have been working on it for over 20 years. Digital technology is advancing rapidly. For example, when we started around the year 2000, the pixel resolution of digital cameras was very low. We faced great challenges during that time, but now we are accelerating the pace."
—— Zhao Shengliang, Director of Dunhuang Research Institute, told reporters from the "Daily Economic News.”
Reports indicate that the preserved murals in the Mogao Grottoes cover an area of over 45,000 square meters, with more than half of them affected by alkali salt disease. Furthermore, in recent years, climate conditions in the Dunhuang region have changed, exacerbating the problem of water leakage into the caves. Coupled with the increasing number of tourists visiting Dunhuang, the excessive influx of visitors has further strained the condition of the murals.
It is understood that, for example, a survey of mural diseases on an area of 10 square meters would require a minimum of approximately 96 hours of work from capturing images, processing and exporting them, to CAD drawing. The multitude of disease types, often occurring in combination, further adds to the difficulty of the annotation work.
In fact, Tencent and the Dunhuang Research Institute reached their first strategic cooperation agreement as early as 2017. In this upgraded collaboration, over the next three years, both parties will enhance cultural exchanges with overseas Dunhuang cultural heritage collecting institutions and promote the digital presentation and dissemination of cave manuscripts. As early as the late 1980s, Fan Jinshi, the former Deputy Director of the Dunhuang Research Institute, proposed the idea of using computer technology to store and reproduce Dunhuang cave art. After more than 30 years of efforts, the "Digital Dunhuang" project is gradually achieving the "permanent preservation and sustainable utilization" of Dunhuang cave cultural relics.
<aside> 🔥 "Nowadays, digitization is a very important aspect of cultural heritage preservation. Through digital acquisition, we can store the information. Even if a cave is damaged due to unforeseen circumstances, if we have digital resources, we can replicate it completely," said Zhao Shengliang.
</aside>
However, Zhao Shengliang revealed to reporters that the work of digital acquisition is actually very challenging.
"The Mogao Grottoes have over 700 caves, and we have only completed the work for a little over 200 caves. We have been working on this for over 20 years. Of course, digital technology has been advancing. For example, when we started around the year 2000, the pixel resolution of digital cameras was very low, and the work was very difficult. But now, we are accelerating the pace. It is estimated that it will become faster in the future," Zhao Shengliang stated. "In the initial phase, we need to collect the digital data, and in the later phase, we need to stitch and perfect it to create a complete cave. In terms of post-production, we hope to utilize Tencent's artificial intelligence technologies to improve this work."
While digitizing the existing cave manuscripts, the Dunhuang Research Institute and Tencent will also collaborate on the digitalization of overseas cave manuscripts.
"Friends who are familiar with Dunhuang history know that due to historical reasons, although the caves are still in Dunhuang, many cave manuscripts were lost to foreign countries for various reasons before the liberation. Using new technology to digitize the cave manuscripts allows all researchers, enthusiasts, and viewers to see them," said Cheng Wu.
According to Cheng Wu, starting from 2021, Tencent and the Dunhuang Research Institute will initiate dialogues with museums along the Silk Road, beginning with a cross-temporal dialogue with the Guimet Museum in France. Based on this foundation, they will carry out digital exhibitions of dispersed cave manuscripts. Over the next three years, the dialogues and exhibitions will expand to museums in more countries and regions.
Yu Qiuyu once said, "Dunhuang is not a thousand-year-old specimen that has 'died,' but a thousand-year-old life that has 'lived.' It is built up by generations of people with their lives." Indeed, the breathtaking murals, clay sculptures, and other artistic works in the Mogao Grottoes are the result of the dedication of craftsmen from generation to generation. Cultural heritage preservation is a race against time, and with the assistance of technology, the life of art can be made eternal.
Dunhuang Research Institute disclosed on the 19th that it signed a new three-year strategic cooperation agreement with Tencent. The two parties will establish a joint working group and introduce AI disease identification technology, immersive remote consultation technology, and more to provide "medical care" for Dunhuang murals and enable remote cultural heritage consultations, thereby assisting in the protection and restoration of Dunhuang murals.
Both parties will strengthen cultural exchanges with overseas Dunhuang cultural heritage collecting institutions, promote the digital presentation and dissemination of cave manuscripts, and further contribute to the joint construction of the "Belt and Road" in Dunhuang cultural research. They will also expand new cultural and creative collaborations based on existing achievements, creating a 30-episode animated series based on Dunhuang murals to tell the story of Dunhuang.
Zhao Shengliang, Director of Dunhuang Research Institute, stated, "The Mogao Grottoes are the world's largest, longest-lasting, most content-rich, and best-preserved Buddhist cave art site. Based on the digitalization of Dunhuang cultural heritage, the two parties have launched open online cultural products such as 'Virtual Dunhuang.' With the help of Tencent's internet platform, cutting-edge technology, and innovation capabilities, culture will further integrate with technology."
The study and restoration of mural diseases have always been key challenges in cultural heritage preservation. Due to the diverse and complex nature of mural diseases and the lack of automated annotation tools, their identification and segmentation require a significant amount of manpower.
To address this challenge, the Dunhuang Research Institute will introduce Tencent Multimedia Lab's AI disease identification technology. By training the AI with relevant data on Dunhuang mural diseases through deep learning, the AI will learn to "diagnose" the murals, enabling automated and efficient segmentation and identification of mural diseases.
Furthermore, Tencent Multimedia Lab will develop an immersive panoramic remote consultation system that achieves 4K ultra-high-definition 360-degree immersive display, allowing experts from different regions to clearly observe the onsite conditions, facilitating barrier-free remote cultural heritage consultations.
<aside> ⚡ There still exists challenges
</aside>
Liu Xiaolan, Assistant General Manager of the Marketing and Public Relations Department at Tencent, revealed to Observer Net that for AI, the amount of data available for Dunhuang cultural relics is extremely small, and there are very few common characteristics, making it challenging for AI to learn. The diagnosis of severely damaged murals poses even greater difficulty for AI. To address this, the technical team has had multiple conversations with archaeological institutes, conservation institutes, and art institutes. After half a year of fine-tuning, the AI has finally acquired the ability to diagnose mural diseases.
The Dunhuang Research Institute will introduce AI disease identification technology, enabling AI to "diagnose" murals through deep learning with relevant data on Dunhuang mural diseases. This will facilitate automated and efficient segmentation and identification of mural diseases. In addition, Tencent has developed an immersive remote consultation system that enables a 4K ultra-high-definition 360-degree immersive display, making it convenient for experts to conduct remote cultural heritage consultations.
Since their cooperation began in 2017, Tencent and the Dunhuang Research Institute have successively launched projects such as "Digital Donors," "King of Glory" flying skin, "Ancient Music Resonance" concert on QQ Music, "Dunhuang Poetry Scarves," "Virtual Dunhuang" mini-program, and Dunhuang animated series. Among them, the "Virtual Dunhuang" mini-program launched during the pandemic in 2020 has garnered over 37 million visits to date.
Cheng Wu, Vice President of Tencent, CEO of Yuewen Group, and CEO of Tencent Pictures, stated that they will venture into the deep water zone of cultural heritage preservation and explore the use of digital technology to assist in solving challenges related to mural protection and restoration. They aim to jointly contribute to the digital repatriation of Dunhuang cultural heritage lost overseas. Preserving the brilliance of historical culture in the present and keeping it alive in people's hearts is the most meaningful form of protection and inheritance.
<aside> 💡 Report from China Daily News
To address this challenge, the Dunhuang Research Institute has partnered with Tencent to establish a joint working group. They have introduced Tencent Multimedia Lab's AI disease identification technology, which utilizes deep learning with relevant data on Dunhuang mural diseases. This enables the AI to "diagnose" the murals, achieving automated and efficient segmentation and identification of mural diseases.
</aside>