Indiana Jones may know all about digging up antiquities, but once discovered and restored, ancient artworks are typically housed in museums. Some are on public view, but many are not. That’s because students, scientists, and historians from all over the world need access to them in order to study, analyze, theorize, and restore them. And often, the only way to see and experience them had been with an expensive on-site visit.
Now, with advances in photogrammetry (using photography to survey, measure, and map distances between objects), computing power, and software, academics have an easier way to record, share, and study ancient 3D objects.
The Virtual World Heritage Laboratory (VWHL) is at the forefront of this initiative. Established in 2009 by Bernie Frischer when he was a professor at the University of Virginia, the lab moved with him in 2013 to its current home at Indiana University.
The mission of the laboratory is to track new 3D technologies and experiment with ways they can be applied to support the work of scholars in the traditional fields of cultural heritage (anthropology, archaeology, architectural history, art history, Egyptology, and so forth).
Through its Digital Sculpture Project (www.digitalsculpture.org), the VWHL is pioneering new solutions and applications in this important but neglected area of the digital humanities. It is devoted to studying ways in which 3D digital technologies can be applied to the capture, representation, and interpretation of sculpture from all periods and cultures.
“Up to this point, 3D technologies have been used in productive ways to represent geometrically simple artifacts such as pottery, or larger-scale structures like buildings and entire cities,” says Frischer. “But with some notable exceptions, sculpture has been neglected by digital humanists.”
The Digital Sculpture Project, however, fills this gap by focusing on 3D data capture and documentation; digital restoration; digital tools for processing and analysis of digitized sculpture, including colorization; and analysis of earlier forms of sculptural reproduction, particularly the cast.
Digitizing Antiquity
In the summer of 2016, the VWHL – now hosted by Indiana University’s School of Informatics and Computing, in collaboration with partners at the Politecnico di Milano and the University of Florence – started 3D digitization of the complete collection of Greek and Roman sculpture in the Uffizi, Pitti Palace, and Boboli Gardens on behalf of the Gallerie degli Uffizi in Florence, Italy.
Largely assembled by the Medici Family from the 15th to the 18th centuries, the statues include works of exceptional interest to students of Greek and Roman art.
Totaling some 1,250 pieces, the Uffizi houses the third largest collection of ancient Greek and Roman sculpture owned by an Italian state museum. Today, it’s best known for its impressive collection of paintings, but from the 16th to 18th centuries, the Uffizi was primarily famous for its ancient sculpture, which the Medici purchased in Rome and transferred to Florence. In that period, the Uffizi’s art collection was even called “The Statue Gallery” (“Galleria delle Statue”).
The main goal of the VWHL project is to revive interest in the museum’s sculpture collection by making it freely available through interactive 3D models on the Internet.
“Our project aims to raise the profile of the ancient Greek and Roman sculpture in the Uffizi,” Frischer says. “So many modern visitors walk past the statues with very little appreciation of what they are seeing, as they make the long trek from one amazing gallery of paintings to another. But in its first three centuries, the Uffizi’s reputation rested on its sculpture.”
As Frischer points out, things shifted in the 19th century when, thanks to the Romantics, Gothic painters like Giotto, and Renaissance masters like Raphael and Michelangelo, started to outshine the ancient sculptors. “We hope that through our project, the statues will once again receive the attention they deserve,” he adds.
The challenge of the Uffizi project is its scale: No one has ever tried to digitize such a large collection of sculpture, though advancements in technology have made the process much quicker. Ten years ago, Frischer could only digitize one sculpture a year; now he and his small team of doctoral students can complete eight in a single day.
“This project was mainly possible because of two key breakthroughs in software that have occurred in the past five years: the arrival of robust, reliable photogrammetry and the implementation of WebGL by most Web browsers,” Frischer points out.
Photogrammetry means that students can capture the 3D data of a statue by using digital cameras instead of dedicated devices such as laser and structured light scanners. Cameras are much less expensive than dedicated scanners, and the data collection can be done in orders of magnitude faster by digital photography as compared to scanning.
The team uses a Nikon D810 and Sony a6300 for photography, with Agisoft PhotoScan and Capturing Reality’s RealityCapture for processing the photogrammetric data. Then they put the final touches onto the 3D model with Pixologic’s ZBrush.
Modeling the Models
3D digital modeling often encounters a barrier when confronted with the kind of complex geometry that characterizes most sculpture, but photogrammetric software largely automates the task of converting the raw data into a 3D model. With fast video cards, the photogrammetric software the teams use can process the raw data much faster compared to the lengthy workflow of processing scan data. Finally, thanks to WebGL, they then publish the final product – interactive 3D models of the statues as elements of Web pages – which makes studying the pieces easier and more straightforward.
The average life-size statue requires approximately 500 photographs; the model obtained from that contains one point per pixel. Since the Nikon D810 takes photos that are 36 megapixels, this means that the average raw model has 500x36,000,000 points, or 18 billion data points.
The point cloud (the 18 billion points) has to be made into a mesh (that is, the points are joined to form triangles). The triangles furnish the surface onto which textures can be generated so that the surface appearance of the 3D model is quite close to the original statue.
Typically, the textured 3D model has much more data than is needed for visualization on a Web page, so the final step in the process entails decimation of the model to about one million to two million triangles.
“We keep the full, undecimated model to be used when a scholar or museum conservator needs very detailed information about the statue, like when the person wants to be able to see tool marks, scratches, or nicks on the surface,” says Frischer.
At 500 36-megapixel photographs per object, and with about 1,250 objects consisting of Greek and Roman statues, inscriptions, sarcophagi, altars, and reliefs, there is a massive amount of data to process.
The Challenge of Sculpture
Many of the statues are displayed against walls. This causes a problem for data capture since the 3D models must contain all sides, including the back. In the project’s first year (May through June 2016), Frischer’s team – comprising four doctoral students as well as Cristiana Barandoni, the project coordinator based in Florence – concentrated on sculpture exhibited in open areas.
In the second through fifth years, the remaining statues will be moved away from the walls by at least two meters so they can be easily photographed from all angles. This will be time-consuming and costly – one cubic meter of marble weighs 2.7 tons.
The team’s partner at the Politecnico di Milano is currently experimenting with lowering a high-quality, albeit very small camera with a wide-angle lens, behind the statues that are situated only a few inches from the walls. The initial results are promising, but experimentation with this approach is still in progress.
Another difficulty the team faces is access; they can only work in the Uffizi on Mondays, when it is closed to the public, and they can only afford to be present in Italy six weeks in the summer. This slows down the pace of the work and explains why the entire project will take five years to complete.
To mitigate this problem, they have involved Professor Gabriele Guidi and his researchers in reverse engineering at the Politecnico di Milano, which is a short 90-minute bullet train ride away from Florence.
“Professor Guidi’s group can work in the Uffizi many more Mondays than can our team at Indiana University,” says Frischer. “We also trained some post-doctoral students of Professor Paolo Liverani at the University of Florence who have taken charge of digitizing the fragments in the storerooms. They can access them five days a week.”
Technicians edit the images, and the final result is then optimized and uploaded to a Web service, which then generates an embedded code that is added to the project webpage.
Hauling Hardware
The laboratory is based in Bloomington, Indiana, but the project itself is in Florence, a continent away. To this end, another challenge the group faced was getting the computing speed required to process these huge images on-site.
“We needed the computer power of a tower but didn’t want the hassle, expense, or inconvenience of finding a special suitcase for it or dealing with the customs process,” says Frischer. “And there was no way we could get enough processing or video memory in something so small as a laptop. The only solution available was the powerful but still portable Shuttle SZ170R8.”
The Shuttle’s small form factor provides them with all the key components they need to do their work, especially the possibility of using the latest video cards, without all the bulk and weight that a tower entails. The Shuttle gave the group mobility without having to sacrifice computing power.
The team also works with Indiana University’s Information Technology Services (UITS) to port the photogrammetric software to a high-performance computing platform, which allows them to speed up processing time and handle more objects at the same time. The data is collected in Italy and FTP’d to Indiana University for backup and storage.
The entire collection is expected to be processed, uploaded, and available on Indiana University’s website by 2020.
What’s Next?
While there is still a great deal of work to be done, Frischer is optimistic: Through this project at one of the world’s most visited and respected museums, the word will quickly get out that they have now entered what he likes to call “The Golden Age of 3D” when it comes to digitization of three-dimensional objects such as statues, archaeological sites, and buildings.
Compared to just five or six years ago, costs have decreased dramatically, processing time has been greatly reduced, and the results can be easily embedded on a Web page or used as an asset on the new VR and AR devices.
“The future is bright. The interactive 3D model of a work of art or architecture will serve the same purpose in the 21st century as did the 35mm slide in the 20th,” Fischer says. ¡
Nancy Napurski is a technology writer and has worked in CG for many years.