Artists digitally reconstruct the Villa Tugendhat, a 1930s modern architectural icon that's now a Unesco World Cultural Heritage site.
When it was constructed in 1930, the villa of Greta and Fritz Tugendhat in Brno, Czech Republic, was already considered quite special. Designed by Architect Ludwig Mies van der Rohe, the building was a pioneering monument to modern architecture in Europe. However, the splendor was short lived, as the home fell into the hands of the Germans during World War II and, later, the Russians. After various repairs and transformations, restoration began in 2010, and two years later, the site opened to the public.
Today, the freestanding, three-story villa is a must-see, and its unique design and construction earned it designation as a UNESCO World Cultural Heritage site. Due to its historical, cultural, and architectural significance, there is often a two-month wait for tickets to tour the villa. However, those wanting to enjoy the structure without a wait or without traveling to the Czech Republic now can do so via a digital re-creation that enables users to interactively explore the upper and main floors of the structure, which are presented in amazing detail.
The real-time, immersive visualization was built by InterMoca (a Czech company founded by Robert Greenyer and Ian Childers), in conjunction with the City of Brno Museum, using NewTek's LightWave 3D and WTools3D's LWCAD software and running within the Unity game engine.
The reflective surfaces within the villa required the artists to devise creative techniques for the visualization.
The visualization is free of charge for the upper floor, with a nominally priced Pro version providing access to additional rooms and the lower living floor that offers views of the garden (http://intermoca.com/store/index.php). The intention is to acquire additional revenue from advertising, sales of the Pro edition, and sponsorship that will help support the monument and fund future development of the visualization to incorporate far more interactivity and access points to related information.
Virtual Rebuild
Greenyer and Childers moved from the UK to the Czech Republic several years ago, then relocated temporarily to India to set up a facility there that would focus on digital media and design work. Greenyer suggested that the studio's artists re-create Villa Tugendhat as a disciplinary initiative between jobs.
"I have a passion for architecture and thought this would be a good exercise for the team," says Greenyer. Using a book about the villa as a guide, the artists eventually finished a visualization that was a fairly close representation of the Villa Tugendhat as depicted in the book. "It looked good but still had a game-like quality because of the game-engine baking solution we had to use," he adds.
Greenyer visited the manager of the villa's museum just six days after completion of a two-year, $8.8 million restoration that returned the villa to its original splendor. "This was a significantly different building than the one we had built in India," says Greenyer.
So, it was back to the drawing board.
However, creating an architectural visualization to increase skill sets for real-time presentations is one thing, and doing a precise re-creation is another, and one that requires exacting detail.
Over the span of two years - about the same amount of time it took to rebuild the actual villa to its original state - the digital artists worked to reconstruct the virtual villa. Unlike the practical laborers, though, the CG crew began from scratch, with three to four artists working on the project whenever time allowed.
Because this would be a real-time application intended to run on 1gb, 2012 smart devices, a lot of time was spent developing processes that would result in a high-quality model with a very small footprint. The workflow, in short, required the team to obtain proper source information on-site, and then model, texture, and pre-light the objects, baking the light maps, separating the high- and low-frequency details, determining when to bake the lighting into the texture maps, and then building the interactivity within the Unity game engine, where the shaders, textures, compressions, and so forth would be applied.
Picture Perfect
First, the team had to obtain resources and reference information inside the villa. Instead of using scanning equipment, Greenyer took a large number of photos and video clips using a Canon 7D and a GoPro. While using the 7D, he obtained small Raw files since they had more color depth. "The advantage of using a camera supported by Photoshop's Raw importer is that it can account for the imperfections in the camera and lens, including removing lens distortion," he explains.
Meanwhile, Greenyer obtained principal measurements, such as dimensions of key objects and room heights, using a tape measure. He also would place a business card (that was a known size) in each picture to provide a scale reference.
In instances when the objects were not photographed square on, Photoshop distortion tools were used to "unwarp" the images with the reference scale applied. The group also used the Camera Raw lens adjustment tools in Photoshop, which compensated for lens distortions from the Canon 7D camera. The images were then imported into LightWave Modeler and scaled and positioned so they lined up with a scale model of the business card for accurate sizing. "This meant good dimensional accuracy could be obtained alongside quality textures," says Greenyer.
In addition to size reference, the photos were used for textures in the visualization. However, because of the restrictions in the game engine, all the texturing had to be achieved with simple layers and two UVs only in order for the piece to export from LightWave, through the FBX Exporter, and into Unity. This meant using a diffuse unlit color map (which could be a tile repeating pattern on the first UV) and a color light map. According to Greenyer, in instances where there wasn't any high-frequency data and the texture was not reused on the color UV, then the light was baked into the texture.
For the Fritz bed in the visualization, which contains a rich wood frame with a plaid covering, the artists used a high-resolution pattern for the fabric that was tiled onto one UV map, along with a second low-resolution color, diffuse, modifier map. So, UV1 was the diffuse, unlit color, high-frequency repeating tile-map texture on a color layer, while UV2 was a baked color map on a luminosity layer.
"With textures in a game engine, you are ideally looking for repeating elements that do not look like they are repeating, so you have high-frequency repeating textures like patterns and grains, and low-frequency detail like color casting," explains Greenyer. "I developed a technique whereby we baked the color of objects around the target object that was colored mid-gray. By doing so, we could capture the bounce light and the direct light with all the contrast into a color light map that would adjust the high-frequency, repeating-texture component colors, such as the bed fabric. It is richly detailed and to scale, but when you look closely, you have the changes you would expect from radiosity."
As Greenyer points out, the project required a good amount of trial and error when it came to techniques. "Everything we needed to develop the visualization, I developed from scratch, including the lighting technique. There was no documented way to create reflection maps in the way it had to be done here, so I created cubic reflection maps for the chrome and other reflective surfaces," he says.
For the highly reflective surfaces, such as chrome stools and door handles, Unity-compatible reflection cube maps were initially made from specialized LightWave cameras, rendered in LightWave using Liberty3D UberCam's Skybox, and then processed by Photoshop scripts and exported to Unity, where the artists applied the textures.
Although the Unity engine presented certain roadblocks, the decision to go with Unity as opposed to another option, such as the Unreal Engine, was a financial one. There are no costs to start out, with licensing purchased at the time the project is published. "We chose Unity because we did not know if this would ever go live, since we were still negotiating [with the museum] while we were building the visualization," says Greenyer. "Also, we didn't know what platforms we would be offering this on." Presently, the visualization runs on PC, Mac, iOS, and Android platforms.
Digital Building Blocks
According to Greenyer, the linear color space workflow in LightWave helped the artists maintain the same color throughout every part of the pipeline, with predictable results. Moreover, GLSL representation in the LightWave viewport ensured that when the artists matched the lighting in the application to the photo source, the final results seen in LightWave would look the same as they would in Unity.
Restrictions of the game engine required the artists to use simple layers for the various textures.
"Another important thing was the surface baking tools. The custom cameras enabled us to create the reflection maps and adjust the individual textures in HDRI to match up everything," Greenyer explains. "It was about getting fast realism without resorting to full-lighting solutions, which were not practical here."
Also, FBX interchange proved invaluable, enabling the artists to transfer assets easily into Unity.
A lot of heavy lifting was done using LWCAD, which allowed for easy and accurate rail extrusion and mesh cleanup. "It really helped speed the modeling process, making it easier to obtain accurate models," Greenyer adds.
Pilgway's 3D-Coat was another key tool when it came to the all-important UVs, as well as for painting the base color map and baked light map cleanup. In addition, the program's retopology tools helped generate more optimized meshes from higher-fidelity models in order to run better within the Unity engine.
While older PCs were used for the modeling, newer PCs with high-end Nvidia graphics cards, including a GeForce GTX Titan, were employed for the texturing within 3D-Coat. A RAID 5 server was used for source storage and content development.
Another important tool was the UberCam, used for simultaneously creating the reflection-map images necessary for Unity and for LightWave.
Instances, however, do not copy over to Unity as instances; instead, they present themselves as mesh clones, which are helpful but less memory-efficient. So, the team used MeshBrush, an inexpensive Unity plug-in, to place instances in Unity directly. Later, they used another Unity plug-in, called Cubemapper (from Spreadcamp), to rapidly create cube-map reflection images for ceramic scene elements.
Running in Real time
Despite the impressive detail in the images, all the models were highly optimized to run in real time. This was made easier, in part, by the villa architect's use of functionalist architecture, as well as the pre-Ikea boxy furniture within, as the villa's design is linear, with straight lines.
For efficiency, the artists used occlusion-culling methods, including those within Unity Pro. Sometimes, though, Childers had to devise special programming to reduce the number of models rendered, without compromising real-time reflections on the desktop versions. Also, prior to final UV creation and baking, the team would assess whether there were any polygons that would remain hidden; if so, they deleted them from the mesh to provide additional texture-area space and real-time performance.
Indeed, the InterMoca group is well versed in generating assets for running in real time. However, there are some big differences that are specific to this kind of visualization. "Our other work involving the creation of game assets and scenery did not require precision when it came to placement or the overall geometry. Also, game models often just have to look the part or look cool. With architectural visualization, the aim is to make the imagery look as nice as possible. In both instances, many textures can be repeated and simulated with libraries or off-the-shelf textures," says Greenyer. "With Villa Tugendhat, the goal was to create a replica of an existing building; flexibility is only afforded by the limitations of the delivery platforms. While the building prides itself in having few materials (wood, glass, and so forth), the grains in the wood are unique for each item of furniture. The real skill was establishing a way to texture the assets so they would render extremely fast on low-specification mobile platforms."
The mobile platforms presented the biggest challenge: The release versions had to run on OpenGL ES2, 1gb RAM smart devices. As a result, the artists had to be constantly aware of the polygon count, texture size, and rendering challenges - far more restrictive than desktop machines.
"The real skill was establishing a way to texture the assets so they would render extremely fast on low-specification mobile platforms."
One More Floor to Go
The next major development of the villa visualization is the technical level, which contains pipes and complex machinery, such as boilers and motors. For this section, Greenyer is using a highly accurate mobile 3D scanning system with real-time mesh reconstruction for model references and, later, optimized meshes. The group also hopes to make a version of the visualization for the Oculus Rift or Galaxy Note 4-based headset, to make the experience even more immersive.
Meanwhile, Greenyer has already been scouting another monument that he would love to re-create virtually: the Antoni Gaudí house in Barcelona - a structure that is almost fundamentally opposite in design to the Villa Tugendhat in that "it has almost no straight lines," he says. In fact, InterMoca would like to develop accurate, real-time visualizations for other protected buildings and UNESCO monuments, eventually using that business model as the foundation for the company.