BTS #31, dealing with problem renders (Patreon)

Hey y'all! This one's quite dry. I hope it's interesting none the less, but the technical gory details of making these animations work is uh, less sexy. Still, this is what y'all pay me to do, so I should tell you about it!

Rendering is, generally, the bane of a lot of 3D artists. The longer you do it, the more kinds of problems you see, and what's great about it is that there's always something new! Just when you think you've seen it all, the computer comes up with some new fresh hell for you to grapple with! Here are some of my favorite examples:

Here, I never figured out what was happening, but it appeared that the vertices in Starlight Glimmer's hair were getting their locations... randomized? It was flickery and inconsistent, and extremely ugly, which is the worst kind of error.

Here, the optix denoiser started introducing these massive black 1024x1024 sqaures of black pixels into the image - again, randomly flickering. This one I was actually able to solve by just switching to a different denoiser, but I find it very funny that on some frames it looks like she's being censored:

So how to deal with problems like this? well, first an ounce of prevention is worth a pound of cure! So, try to avoid scenes that are going to stress the hardware. That means keeping poly counts under control, not making the renderer load gigabytes of texture files, avoiding extreme and weird dependency chains, and minimizing complex ray paths in the environment. 

There are two ways to do that: scope control, and optimization. Scope control is simple: just don't try and depict things that are hard to render. But what if you want to do something difficult? Well, now you need to optimize. It isn't fun, and there aren't shortcuts, you just have to go through your scene and identify things that are causing problems, then try and simplify them without sacrificing too much of the final image. Downsampling or re-using textures, decimating geometry, even baking animation can all be massive gains. The guiding principle of optimization is this: it's an information game. You need to find those big 80-95% moves, and it's easy to spend hours optimizing things that aren't actually going to make much of a difference. This is why we gotta do a lot of test renders.

Test renders are the other major principle that solves a lot of these problems before they come up, and frankly, I'm terrible at doing them. almost every time something goes wrong with an animation, I could have done a smaller, faster render first and not wasted so much time rendering a broken scene. If you test early and often, you can save yourself a lot of pain later on. Similarly, when you have a problem, test rendering is the only way to know if your optimizations are having any effect at all.

Of course, optimizing is a slow process, especially with iteration after iteration of test renders to wait for. If you're on a tight schedule, it's good to have a "nuclear option" where you can skip the time-consuming process of solving the problem entirely, and just side-step everything. Wondering why I never figured out that starlight problem earlier, but still got the scene rendered? This is what I did, instead of working it out. In fact, it's good to have multiple such options. My go-to is this:

The current Soarin-Celestia footstool scene from the February poll is kicking my ass. The combination of outfits and environment is completely overloading the VRAM on any of my rendering machines, so it's just crashing before it even starts. The solution? don't render the whole scene at the same time. Split it up into parts, so blender only has to load smaller chunks into memory at once. Then, after the fact, put them back together.

This sounds really hard. And it can be! But it doesn't have to be. The important thing is to find a simple, reliable and quick way to do it that doesn't rely on extensive setup, unreliable compositing steps or manual work. Thankfully, blender has such a system! No, I'm not talking about the composite nodes, although they can also be used. I'm talking about restriction toggles!

These are extremely powerful, and very easy to use. Really, the toggles are the best argument for why you should organize your scene into collections. It's like zero-setup almost automatic compositing.

The first, the checkmark, is simple but powerful - it just disables the collection. Everything it contains doesn't get loaded into memory, so the computer doesn't have to think about it at all. This is great for heavy, static backgrounds that don't contribute to the scene lighting. 

The little arrow is "Indirect Only" - it's like disabling the collection, but it still contributes to lighting everything else. Of course, this way blender still has to load some stuff into memory, but it's not nearly as bad. 

Finally, there's the square with a circle: "holdout". This is like "indirect  only", but it cuts a transparent hole out of the image exactly where the collection's contents is. This is vitally important for getting objects that are being rendered in separate passes to correctly occlude one another, and is how we can just simply layer each pass over eachother after rendering without doing any complex masking or keying, it's all done for us by the holdout restriction. This is a pretty good illustration of the effects of a holdout:

Of course, this can also go wrong. Since we're not using z-information to recombine the image, we need to think through what gets set to indirect, full, disabled and holdout for each layer, or we get issues like this. Oh dear.

And with that, we can render scenes that are trying to physically manifest snakes in our homes! Sometimes. No solution is universal, but this is a pretty good overview of the main techniques I tend to use. As always with these behind the scenes writeups, I've tried to condense a a single topic out of a complex, interconnected web of stuff, I hope it's relatively readable! 

If nothing else, consider this a sneak peek at the Celestia animation, which should be going up next weekend!