In the original three Matrix films, as well as the recent fourth, a handful of visual effects studios generated the real world of the Matrix and beyond, and the virtual “real world” using state-of-the-art visual effects. However, in addition to those 3D elements, there is a plethora of digital work that may not leap off the screen but is equally vital to the storytelling. It involves the user interfaces (UI) that appear across screens of all types throughout the film.
Here, CGW Chief Editor Karen Moltenbrey talks to Sam Jones, director and motion graphic supervisor, and Gordon Spurs, creative director/3D artist-animator and motion designer, with a focus on fantasy/fictional user interfaces, design, and holograms, both from Studio C, which was tasked with generating the UI graphics in
The Matrix Resurrections.
Tell me about the work you did for The Matrix 4.
Jones: We were asked to create all the screen graphics for on-set and post delivery, across all 20-plus sets. From the UI for the new ship Mnemosyne and a holographic scanning device in the cockpit, to the farm and captain’s quarters screens on the Io sets. Also, Neo’s desktop screens in Deus Machina, the game company he works for, and the screens seen on the theater set where the extraction takes place. There were other screens as well, but those were the highlights.
Spurs: Initially, we worked as part of the art department, with Production Designers Peter Walpole and Hugh Bateup, and with direct feedback from Lana [Wachowski]. We spoke with Production VFX Supervisor Dan Glass a few times in pre-production, and the aim was for us to give him as little as possible to do in post. I think when the project is this big, there is an expectation that you’ll be replacing all the screens because they won’t be what the director wanted, whereas in this case, Lana loved them, so I would say 90 percent of the work was in-camera.
We then picked up where we left off the following March in post, working closely with Dan to finish the key opening sequence in Thomas Anderson’s office as well as some other actor interactions and a hologram for the Mnemosyne.
Did this include the "signature" green code shots?
Jones: We didn't create the signature green waterfall code, as this was an already established asset; however, we were involved with other like-for-like re-creations for the modal sequence on Thomas Anderson’s desk screens on the Deus Machina set. When you first see the fake Trinity, she is on her laptop as per 1999, and we re-created those screens frame by frame, as well as the screens in Thomas Anderson's apartment with the blinking cursor and the ‘Wake up, Neo…’ dialog and the search for Morpheus.
It was one of the first things we thought about, and we ended up with a few options for Lana, including a version that fleshed out the blinking cursor with more content. We knew that we had to work with two screens, so we drew on the original content, updating it with new news clippings that reference what's happened in the interim.
What kind of direction were you given?
Jones: Directions varied with the set and world, so for the Mnemosyne screens, we were asked to create an advanced version of the Matrix 60 years in the future [from the end of
Revolutions] while keeping the ‘feel’ of
The
Matrix, which in itself is tricky, as you are trying to drag motion graphics from the ’90s into the future while keeping true to the original trilogy. The art department gave us some references, including two binders that had screen grabs of each time a graphic had appeared in the previous three films. That was our constant point of reference for how the information was organized and layered. We had some discussions with Production Designers Peter Walpole and Hugh Bateup, and made suggestions about what we felt were the best ways to use graphics to support a scene or a story beat in a way that made sense in the Matrix world, and we were given the freedom to follow our instincts.
For the Deus Machina (DM) sets, which were ‘inside’ the Matrix, the premise was that the visuals were largely contemporaneous with what we consider state-of-the-art UI and OS now, so we researched game design companies, game engines, etc., so that the DM screens felt consistent with an innovative game company. The exception to the corporate game company styling were the graphics for the modal terminal, which we designed to reflect the content of Neo’s original screens from the first film, where he searches for Morpheus and the Matrix.
Spurs: Aside from the bible of graphics from the original trilogy, we were pretty much entrusted to bring the reference to the production designers ourselves and sell them on our own vision of the screens.
For Thomas Anderson’s desktop screens on the Deus Machina set, we wanted our graphics to reflect and support the shift in Thomas Anderson’s work environment. Gone is the dingy claustrophobic office cubicle of the corporate software company from The Matrix, and now we have the modern trappings of a bright agency/game studio.
What served as the inspiration for the work?
Jones: As fans of the franchise, we felt it was important to maintain some cohesion with previous films, so we kept true to the iconic green code that is central to the visual language of the Matrix and referenced some other UI and typographic elements so that this world would feel cohesive. Also, we know that fans will appreciate that continuity and will look for it.
But this story is set in Neo’s future, so to evolve the visual language to reflect that leap, we looked at how our own technology has evolved, and in the same way that our own GUIs have become more refined, we applied that to the Matrix world.
Our thinking was always, how would this setup have evolved over the 60 years that have passed in that universe? We were constantly considering how far the designs could be pushed before they became something that didn’t fit the Matrix’s style and had to be pulled back and reassessed.
The final look, which is more structured, sharper, and more optical, comes from what we called a ‘Synthient infusion,’ where technology is informed by the Synthients, friendly machine creatures, from within the Matrix world. The rebels now have these Synthient creatures as part of their crew that look more organic and animalistic than robotic, like stingrays, etc., so the idea was that the technology and UI had evolved to reflect the interaction and influence of these new robotic or synthetic hybrids.
Spurs: The first film captured a zeitgeist with the audio and the visuals alike, and when getting my head into this job, I immersed myself in the whole thing. Listening to Massive Attack, focusing on the design as a whole, not just what was delivered for the screen graphics.
We started looking at a lot of obsolete analog and digital media, like minidiscs, CDs and Datadiscs which definitely had an influence on the pure abstract graphic design of the Mnemosyne main deck screens. In the first film, Neo keeps his hacks on minidiscs in the hollowed-out Simulacra and Simulation book, and Tank loads training into the computers via a fictitious drive and disc, which back then seemed mad futuristic. It’s the last great sci-fi film before the full shift to digital downloads and media, and you want to keep the analog romance of that in the ether, even if very subtly.
To achieve the right UI for the Deus Machina Games set and Thomas Anderson’s screens, we researched game developers and current Silicon Valley sensibility for the corporate visual language, and developed the look of a game engine/dev software [image], a DM Games website, and a modern OS that could house Neo’s modal experiment and Matrix coding where he is subconsciously ‘growing programs.’ We looked at a lot of custom Linux software mods and more retro-dated OS designs for the early 2000s for the modal terminal to capture the cyberpunk spirit of the first (film), then tried to bring some of that sensibility to a Silicon Valley update. We also had a lot of fun adding little easter eggs to the OS icons, like the flip phone from the original film.
Please detail the creation process.
Jones: In terms of time, we worked at various points over two years, totaling about 14 months. We had initial preproduction conversations with production designers Peter Walpole and Hugh Bateup and flew to San Francisco to meet with Peter in December 2019, then to Berlin to meet Hugh.
We started production immediately for the February 2020 shoot in San Francisco, concepting, designing, and delivering the Deus Machina game screens before the planned move to Berlin, which was obviously pushed due to COVID. We then picked things up in July 2020 in Berlin, spending a lot of time on set delivering the Mnemosyne and Io assets, and shot through November 2020. For post, we started in March 2021, working closely with Dan Glass, and delivered our final assets at the end of July.
Spurs: There is a disproportionate amount of excitement when starting a job like
The Matrix, particularly what it meant to the team growing up. Reading the script and re-watching the original trilogy, listening to the sound track was amazing. I was 15 when the first film came out, and it obviously blew me away with the philosophy, action, and pioneering VFX.
One of the main challenges with the process for
The Matrix Resurrections was it being split between the US and Germany. When the project began, Lana and Peter Walpole were on-site in San Francisco shooting the first location set pieces, while Hugh Bateup and the art department were readying the sets in Berlin. So we were dealing with two different teams with varying levels of input on several different sets regarding the brief. Sometimes the important thing for all involved was structure, script, and progressing the story, other times it was purely about the aesthetic of the design. We had to be flexible and responsive to deal with the various voices and time zones until we all arrived in Berlin for the homestretch. Then we were eventually brought on in post as a VFX vendor to work with Dan Glass to augment the graphics we had started 18 months earlier.
What tools did you use?
Spurs: Our main tools were (Maxon) Cinema 4D, (Adobe) After Effects, (Maxon) Redshift, (Insydium) X-Particles, (Adobe) Illustrator, (Adobe) Photoshop, Unity, Blender, (Adobe) Substance Painter, and various plug-ins for those suites. We often touch on (Robert McNeel & Associates) Rhino, and (Trimble) SketchUp for the tweaking and conversion of CAD elements supplied by the art department.
Usually we design the graphic elements first in Illustrator and comp them in After Effects/Photoshop with a 3D render so you can see the major component parts and how they sit together. Cinema 4D and Redshift do most of the heavy lifting for elements like ship schematics or 3D scans. Hardware-wise, we have a few Renderbox machines to do multi-GPU rendering, as iteration in film has to be fast. We were dealing with accelerated deadlines due to COVID disruption so some sets were brought forward by weeks, and our pipeline was crucial to meet that demand.
We also have a Unity programmer who builds screens that need any interactivity, which was essential on this job.
Did you have to devise any new techniques or create new tools for the work on this film?
Spurs: There were several moments where we were on the edge of what was possible technically for playback. For example the medical screens for each character who was jacked into the Matrix… we had an interactive Unity app installed on the tablets so the cast could cycle through 3D organs and vitals. This also featured a live feed of the characters sitting in the ‘dentist chairs’ that was streamed to ensure performance could be captured in-camera without a need for post.
For the Thomas Anderson office scene on the Deus Machina set, we needed to design an interactive sequence, in addition to developing the look of a game engine/dev software [image], a DM Games website, and a modern OS that could house Neo’s modal experiment and Matrix coding where he is subconsciously ’growing programs.’
One of the things I’m especially proud of is the level of support Studio C provided to the delivery of the modal sequence. We designed and built a Unity program across multiple monitors that served as an interactive guide for Keanu’s performance on the day of the shoot. We then came back in post and helped Dan Glass with 3D look-dev for the modal environment and tightened up timings for the final edit with existing elements. This sequence had evolved and mutated quite a bit by the time we were finishing it in post as Lana was giving feedback on the final edit and seeing how it worked for herself. They actually adjusted the edit to allow more time for our elements to tell the story, which is unusual in my experience, graphics often being an afterthought.
What was the most challenging aspect of the work?
Spurs: In addition to getting the modal sequence right, it was redesigning the graphics for the iconic operator station on the Mnemosyne. To give the UI and graphic elements an evolved appearance suggestive of a story set 60 years in the future, we built on the imagery established by concept designers and storyboard artists Geof Darrow and Steve Skroce in The Matrix trilogy. Elements central to the era of the first film, like texture backgrounds in an OS (circuit boards and so on), bulky gradients or classic sci-fi fonts synonymous with the Matrix, are both a treat to play around with but also present a challenge when updating the visuals for a new audience and not having them feel dated.
Jones: For me, the Mnemosyne main deck and the Thomas Anderson opening sequence were the most challenging.
The main deck because it’s the iconic set that Lana wasn’t going to see until the night before shooting, which thankfully she loved.
And Thomas Anderson because of the complexity of the scene and the actor having an interactive graphic to play with. His desk/workstation features screens of games he’s working on plus a modal window of matrix code, a side project that Thomas is working on. We have a very short window of time to get across an incredibly complicated bit of storytelling with the modal experiment having a breech and the subsequent purging when Anderson returns.
Because Deus Machina is a game dev company, we researched real-world companies and how they work, as we needed to create a realistic piece of software that would convincingly look like a game engine, without ripping off any existing engines. During the opening sequence, Thomas Anderson opens an older, more bespoke, piece of software for his original Matrix modal experiment, which allowed us to have a bit more fun with it and include some easter eggs, such as small icons referencing the blue pill/red pill and the Nokia 8810 from the original film, alongside the classic dripping Matrix code. There is even a rendering of Neo’s head, as if he is building that asset to load into his experiment.
What, if any, technical difficulties did you face?
Spurs: The big one, I think, was generating realistic background screens for Deus Machina Games set. That office had around 50 screens, which we knew would be deep background, but having worked with Lana on Sense8, we also knew the Steadicam could be wandering into every area of the set and getting really close to the work. We were generating assets like 3D models, Lidar scans, custom mocap characters, environments, and anything that looked plausible for Thomas Anderson’s new game Binary. It was a lot to pull out of thin air considering they weren’t hero screens; thankfully some of the assets made their way into Keanu modal scene.
How many shots did your work appear in?
Jones: We delivered over 100 graphic animations for on-set playback and then again for post delivery, plus a couple of VFX elements, including the holographic scanning device in the Mnemosyne’s cockpit. Our work was featured across about 20 sets altogether, so we’re really happy with the result!
What makes this work so unique from what we see in other films and series?
Jones: I think
The Matrix world gives you an opportunity to make very unique interfaces and designs that don't have to make sense to the audience, with abstract ideas that aren't tethered to having to look functional.
Spurs: For me, the original
Matrix screens are sci-fi folklore, and it’s some of the dated characteristics of these designs that make them so distinct. The fonts, the photographic background textures, big chunks of binary code, pastel gradients… none of those elements are something you’d use for a formulaic modern sci-fi film. I think balancing these bold retro visuals from the first film and leaning on shape elements from the second two films, while bringing 3D elements with an optical edge into the mix, we arrived at a place that’s quite abstract and unique, with a lot of depth and texture.
How is the work different from that of other projects you have done?
Jones: This has been Studio C’s biggest project to date, so the scale, number of sets, and different worlds to reflect really pushed us. And working with a lot of creative freedom on such an iconic franchise was a lot of pressure. And of course, the pandemic made the experience of travel and working on set very different and challenging. It was also our first experience of creating and delivering assets for post, which is obviously a much different workflow to production content.
What role/impact does this type of work provide in a film?
Jones: Screen graphics, be they UI or content, help tell a story. We constantly hear about the impact of virtual production techniques and sets made in Unreal, but tangible props are important, giving actors something to perform against and to. They help make that world feel integrated and believable not just for the audience, but for the actors on set, too. The complexities and benefits of designing the modal sequence and med screens for the operator chairs on the Mnemosyne really proves the value of supporting actors with tangible, physical props that serve the performance. For me, there is nothing more satisfying than walking into a fully lit set with all the graphics working and getting the shots in-camera.
Is there anything else you want to add?
Jones: We feel that this film has really established our ability to take on a big project and deliver. It’s quite unusual to be the sole provider of motion graphics and holograms on a project this huge, so we are very grateful for that opportunity.
Founded in 2018 as a high-end motion graphics studio, Studio C is the creative offering of Compuhire, the UK’s leading playback company. The growing team offers bespoke graphic content for on-set and post delivery, and VFX across film and TV productions. Credits include Venom 2, Godzilla vs. Kong, The Suicide Squad, Fast & Furious 9, Black Widow, WandaVision, The Midnight Sky,
and more.