Renderfarms Past, Present, and Future

Rendering is still doing what it was doing 10 years ago: growing. Breakthroughs occur, expectations develop, and companies respond to meet them. But it is really through points of disruption that we have seen the biggest gains being made. For rendering, these have been: the introduction of the multi-core processor, the huge decline in the cost of both content creation and rendering software, and the rise of commercial render managers.

Richard Lewis

The advantages of the multi-core processor become abundantly clear when you stack up old costs against new ones. For instance, while working with Square Pictures on its Final Fantasy: The Spirits Within feature film (2001), I helped deploy a system that utilized more than 1,000 single-core Pentium 2 and 3 CPUs and several hundred MIPS processors to render, what was at the time, the single most complex CG movie ever made. That renderfarm was also enormously expensive, occupied an entire floor of an office building, and needed tens of thousands of dollars per month in electricity and cooling.

It is also a dinosaur by today's standards, as CPU technology has advanced to the point where, at a fraction of the cost, a small studio can attain rendering power equal to or greater than the kind Square had, thanks to things like high-density processors, memory density, and advancements in blade server technology.

The trend of better, faster, cheaper has extended to software as well, which has driven down the costs associated with creating content. I remember selling Alias PowerAnimator when the suite (modeling, animation, and rendering) sold for $45,000. If you add in another $50,000 for a tricked-out SGI Octane (which some companies did), you would quickly find yourself spending nearly $100,000 for every artist in the animation department. Enter the desktops of today, which are far more powerful and can be acquired with a suite of high-level software for 10x less than those historical prices. This has expanded not only the reach of rendered content, but the opportunities for professionals who can create it in a very real, and very tangible way, as well.

Images from NASA’s Conceptual Image Lab

Free the System

The problem is these jumps in technical capabilities haven't always addressed pipeline efficiency at the same level we now see via commercial render managers. Even their rise was pre-dated by a period of disorganized R&D. When we started our company 10 years ago, there were just a few supercomputing apps, some open-source grid systems floating around, a few commercial render managers written and supported by individuals, and a lot of custom code. Studios were struggling with vendors that didn't understand digital media applications, and R&D departments were building and supporting custom render managers at great expense.

Today there are finally a few viable render managers for sale, and they are supported by teams of software engineers who come from digital media and understand both the apps and the render pipeline. Rendering has never been so accessible to so many, and this is due to the lower cost and increased support provided by commercial render management software companies and their products.

Images from NASA’s Conceptual Image Lab

Look to Efficiency

At its best, an efficient render pipeline will minimize the ever-expanding render time and allow for more cycles of the creative iterative loop. By "creative iterative loop," I mean the entire design process from start to finish, including all the successive revisions that force companies to re-render each new step in order to get to their final product.

Being that this loop is the foundation of a higher-quality product, most companies and artists want that leeway. Or if quality is not the issue, then an efficient render pipeline allows an organization to take on more work with the same resources, leading to more profit. And profits, especially in the visual effects industry, are hard to come by these days. This makes rendering efficiency very important; or, as I like to say, "business critical."

The new ArtistView GUI in Cube!

Right now, we are at a point in rendering's history where the value of renderfarm management can be easily seen and felt by system admins. In the past, this wasn't the case-but over the last decade, we've seen several key improvements.

• More studio insight into actual utilization. This is a simple education issue. Most studios don't really know how much of their computing assets are truly being used each day/week/month or per project. A common thing we hear from our customers in the beginning is, "We need more render nodes!" But most studios just need to use what they have more often and more effectively. There are a lot of things you can do before you start to actually need more hardware and software, and if your utilization rates are not over 90 percent, you don't need more computers.
• Increased middleware integration with render management. It's only been recently that we've begun to see production tracking, asset management, preview/playback, and other pipeline-critical systems being integrated with render management. Most of these apps were traditionally developed in-house. Today, they are services offered by third-party software vendors like ourselves, Shotgun Software, Tweak Software, and so on, so companies can utilize their entire pipeline from the get-go, in the most efficient way possible.
• Better and more training for admins, TDs, and artists. In 1986, I started an AutoCAD user group. I had to. There was only a Command Reference Guide at the time-no books, no classes at a community college, no private CAD or animation schools. We had to meet and tell each other what we had figured out on our own about the software. I feel like render pipelines are still at that stage now. Try searching Amazon.com for books on render pipeline engineering, or customizing your render pipeline, or anything about render pipelines, and you get nothing. PipelineFX was the first render management vendor to offer certification training in a render management application for this very reason-the knowledge needed to be spread. Through these classes, more than 250 technical professionals around the world have been certified. This is a start, but there are still thousands of organizations that depend on rendering every day and don't employ staff that have any specialized training in the subject.

The Future Is Cloudy

The search for new ideas around the pipeline has also become a big piece of the rendering landscape, in part because the industries that rendering touches have become more organized and more aware of how important rendering is to their day-to-day work processes.

The biggest trend we are seeing right now is cloud rendering-considered by many to be the next panacea. To be honest, the hype around the cloud reminds me of the hype around that "storage as a utility" idea which was touted as a paradigm shift around the year 2000. Remember when we were all going to plug Ethernet into the wall and get Tier 1 data storage at high-performance speeds for super-cheap prices? Never happened. All those companies went out of business.

The Internet, and the cloud, have not changed the rules of economics. There's price, performance, and quality. You only get two. If the cloud is fast (performance) and truly scalable with all the software available for rent (quality), then it's not going to be cheap; or if it's cheap, it won't have fast performance and high quality. For the most part, cloud rendering will be successful in 911 situations-times when you need 200 more machines for two months and you can't afford to buy them or find the time to set them up even if you could afford them. In that case, it's worth it to pay more (sometimes a lot more) to get the work done on time.

The cloud can also work for the occasional user of rendering, but if CG is your business, you are going to need in-house rendering capability so your capacity can be something you count on and use over and over again.

Western X discovered the advantages of using renderfarm management while outputting these furred animals.

Offline Rendering Is Here to Stay

The enthusiasm shown for cloud rendering is really just indicative of how people approach technology, in general, nowadays. They want new, they want meaner and leaner, and they want to believe there is a silver bullet out there. In terms of rendering, the CG equivalent of this is the idea that we are always progressing toward perfectly photorealistic real-time rendering. But even though we've been hearing about the rise of real-time rendering and the fall of CPU-based offline rendering for many years, this will never happen. And the reason is simple: You can do more in offline rendering than you can inside a graphics card.

Real-time rendering certainly has its place, but imagine if you could have an accurate 3D model of the city your film is taking place in that is accurate down to the door knob, both inside and out. Then imagine if you had a model at that level of detail of the whole country, or the whole planet, or the whole solar system. Imagine how useful this would be, and how much you could accomplish with so vast a dataset. That's where we're headed. But it will never fit on a graphics card, or any array of graphics cards. And it will never be processed competitively in real time. If you tried to switch to only real-time rendering and your competitors still use offline CPU-based rendering, you will lose.

In the years ahead, the winners will be the ones that find a balance between online/offline rendering and data. The future is all about data and understanding what's really going on with your pipeline so you can make informed financial and resource-based decisions.

Rendering something great is only part of the equation now. In the early days, there was so little competition that we could pardon ourselves for not paying as much attention to the bottom line. It was all expensive and inefficient, so it was OK to write a bit of it off. Today, that doesn't make sense. We've evolved, our technology has evolved, and we no longer can afford to suffer with inefficient pipelines. This makes things like dashboards, reporting, charting, and mobile access tools valuable, as they help a studio understand what's happening and where they need to make adjustments within their ecosystem.

So, for the smart companies, rendering can now be another avenue to cost leadership, rather than a loss center. That should excite companies that want to do more, and should open us all up to more possibilities and more profits in the next 10 years.