Neural Network Model Restores Damaged Historical Artifacts From Photos

Researchers have developed a novel neural network model that uses old photos to reconstruct damaged cultural heritage objects as restored 3D virtual reality (VR) images using old photos.

Reliefs – wall-mounted sculptures that remain attached to a flat base  – are found at historical sites but are subject to damage and deterioration over time.

Modern 3D scanning and photogrammetry methods can digitally preserve their current form, but cannot restore the original appearance of these carvings before damage. 

Restoration using traditional methods requires intensive, highly-skilled labor, It is also costly and can affect the integrity of the original object. 

A team from Japan and China has developed a neural network model that can reconstruct these reliefs as 3D digital images from old photographs that show them in an undamaged state. 

Reliefs are ideal candidates for this process as they have a shallow depth and can be viewed from the front of sides, meaning 2D photographs capture most of the information required for 3D digital reconstruction algorithms.

The research was led by Professor Satoshi Tanaka from the College of Information Science and Engineering at Ritsumeikan University, Japan and Jiao Pan, from the University of Science and Technology Beijing, China.

Related:How AI Supports Global Biodiversity Efforts, IBM Insights

“Previously, we proposed a 3D reconstruction method for old reliefs based on monocular depth estimation from photos,” said Tanaka.

“Although we achieved 95% reconstruction accuracy, finer details such as human faces and decorations were still missing. This was due to the high compression of depth values in 2D relief images, making it difficult to extract depth variations along edges. Our new method tackles this by enhancing depth estimation, particularly along soft edges, using a novel edge-detection approach.” 

The relief scanning process. Credit: Ritsumeikan University,

The researchers used the model to reconstruct wall reliefs at Borobudur Temple, a UNESCO World Heritage Site in Indonesia. They are covered by stone walls due to reinforcement work carried out during the Dutch colonial period and cannot be viewed. 

“Our multi-task neural network successfully reconstructed these hidden sections of Borobudur’s ground-level reliefs from surviving old photographs,” said Tanaka. “ Through computer visualization and virtual reality, our research now allows virtual exploration of these unseen treasures.”

The technology holds the potential for digitally preserving and sharing cultural objects, not only for archeologists but also for immersive virtual experiences through VR and metaverse technologies, preserving global heritage for future generations.

Related:AI-Powered Smart Glasses Launched to Assist Dementia Patients