The Whole Look

By Jonathan Stone & Doug Smythe

In July 2017, Lucasfilm launched an open standard and codebase called MaterialX, the third open-source project in its history. MaterialX is used to represent rich material and look-development content in computer graphics. In contrast with previous approaches, it focuses on describing artist-facing, networked materials in an application-independent way, with a well-defined, customizable library of connectable nodes and robust handling of data types and color spaces.

MaterialX was authored as a collaboration between Doug Smythe, a veteran CG supervisor at Industrial Light & Magic (ILM), and Jonathan Stone, a senior software engineer in the Lucasfilm Advanced Development Group (ADG), with generous assistance from Lucasfilm leadership, developers, and artists, as well as significant contributions from Autodesk, Pixar, Sony Pictures Imageworks, and Foundry.

The MaterialX project has greatly evolved in its journey from ideation, through production, to open-source release, and in this article, we will discuss this history as well as provide some thoughts on the road map ahead.

STAR WARS: THE FORCE AWAKENS WAS ILM’S FIRST SHOW TO LEVERAGE MATERIALX AS A CANONICAL MATERIAL FORMAT.

In Production at Lucasfilm

In preparation for a new generation of Star Wars films, Lucasfilm launched an internal project called the Unified Assets Initiative in April 2013, as a collaboration between ILM and the newly formed ADG. The goal of this project was to standardize the production of digital assets for sharing between different tools, facilities, and media platforms. Existing open standards such as Alembic, OpenSubdiv, OpenEXR, and OpenColorIO were recognized early on as key elements of this project, allowing geometric assets, textures, and colors to be defined in a portable, application-independent fashion.

For look development, however, there was no existing standard that could represent the rich, networked materials that artists were authoring in tools such as Foundry's Mari, Allegorithmic's Substance Designer, and Foundry's Katana, and these assets were greatly restricted in their usage and compatibility over time. As a potential path forward, the Unified Asset team was drawn to an early, internal specification named MaterialX, which had been proposed by Smythe in the previous year.

This technical proposal seemed like an ideal starting point for a new, unified material standard, and Stone began working with Smythe to extend the specification to cover real-time as well as off-line rendering. Work began immediately on a C++ and Python codebase for common functionality such as material import, material export, and the editing and traversal of material graphs.

The first show to leverage this new codebase was Transformers: Age of Extinction, for which MaterialX was integrated with Lucasfilm's look-development framework in Mari. This gave artists the ability to transfer material content between individual material layers and also between geometries. These new features proved popular and gave Lucasfilm a sense of how experienced artists could use fine-grained material transfer functionality in new and creative ways - as well as give them valuable experience with the new codebase in a complex production environment.

By the time Star Wars: The Force Awakens entered pre-production in late 2013, Lucasfilm was ready to test MaterialX as a canonical material format for the entire show, from the initial prototyping of material libraries to the end-of-show archiving process. For the first time, libraries of MaterialX presets based on Lucasfilm's new unified shader were authored in Katana and used for painting in Mari, with completed material assets being archived as networked MaterialX files in the Star Wars franchise digital backlot. This general approach has been used on all subsequent ILM shows, with support for additional tools and pathways being developed in the years that followed.

In parallel, Lucasfilm began leveraging MaterialX as a vehicle for transferring material content from the ILM CG visual effects pipeline to the real-time technologies powering interactive experiences such as Star Wars: Trials on Tatooine.

This transmedia sharing of materials took a significant step forward on Rogue One: A Star Wars Story, for which three shots of K-2SO were rendered in a custom build of Epic's Unreal Engine 4. Rogue One production leveraged MaterialX for the transfer of materials between the film and real-time pipelines. A similar material transfer approach was used by ILMxLAB and The VOID on the real-time immersive experience Star Wars: Secrets of the Empire. MaterialX was also used for production of the Millennium Falcon attraction for Star Wars: Galaxy's Edge at Disney Parks.

IN ROGUE ONE, K-2SO WAS RENDERED IN A CUSTOM REAL-TIME ENGINE, WITH MATERIALX USED TO TRANSFER MATERIALS BETWEEN FILM AND REAL-TIME PIPELINES.

The Path to Open Source

Lucasfilm's previous open-source projects, OpenEXR (2000) and Alembic (2011, with Sony Pictures Imageworks), had been important steps forward for production computer graphics, and consideration for an eventual open-source release was factored into even the earliest days of MaterialX development at ILM and ADG. Only by making MaterialX an open standard and codebase would it be able to evolve along with the larger industry, allowing developers to embed support for MaterialX in new tools and pipelines as they are built, and to contribute improvements and extensions as they are needed.

In addition to taking advantage of their access to original OpenEXR and Alembic leadership within Lucasfilm (including Florian Kainz and Rob Bredow), the MaterialX team sought out discussions with industry leaders at other companies, to broaden the applicability of the standard and forge connections with other initiatives.

In early 2015, the team consulted with Larry Gritz at Sony Pictures Imageworks about using Open Shading Language (OSL) as the reference language for MaterialX nodes, providing a well-defined, open language to describe their behavior. This idea took root at both studios, and the following year, Gritz brought together a small team of OSL developers (including Adam Martinez and Derek Haase) to write the canonical shading definitions for MaterialX, which were ultimately released as part of its open-source codebase.

During the same year, developers at Lucasfilm and Autodesk began discussing their respective goals for material standards, and recognized connections between MaterialX and an Autodesk initiative named Abstract Material Graphs (AMG). The two projects were soon merged, and developers at Lucasfilm and Autodesk began to collaborate on MaterialX as a common standard.

This partnership had a profound effect on MaterialX, with Autodesk developers (including Niklas Harrysson and Bernard Kwok) contributing key ideas that would improve its design, as well as features that would be incorporated into the open-source release.

A SCREEN FROM THE MILLENNIUM FALCON ATTRACTION, FOR WHICH MATERIALX IS A CORE ELEMENT OF ITS TRANSMEDIA ASSET PIPELINE.

The Road Map Ahead

The original open-source release of MaterialX was developed for Version 1.35 of the MaterialX specification. Lucasfilm later finalized an improved Version 1.36, with publication and code updates prior to SIGGRAPH 2018.

In addition, compelling projects at external companies continue to strongly shape the evolution of MaterialX, as it has over the past year.

Autodesk is extending the MaterialX codebase with support for shader code generators and BxDF shader primitives, in addition to integrating MaterialX as a core feature of its tool suite (including Maya and 3ds Max). The addition of shader code generators has enormous potential for MaterialX, opening the door to stand-alone MaterialX viewers that could provide an application-independent reference for implementations. BxDF shader primitives are a compelling path forward for the renderer-agnostic expression of shading models, potentially eliminating the need for explicit BxDF implementations in each target environment.

Pixar is also involved, extending its open-source Unified Scene Description (USD) with a new UsdMatX schema, in order to provide direct support within USD for MaterialX node libraries and documents. This development opens the door to USD geometric hierarchies and MaterialX networks coexisting within a single framework, with composition arcs combining them in flexible ways. Lucasfilm and Pixar have closely collaborated to align the two standards for maximum compatibility.

Additional details about MaterialX can be found at www.materialx.org, including links to its full specification, open-source codebase on GitHub, and discussion forum on Google Groups. n