Distributed rendering without textures on slaves?

[Ragazzaccio]

Hi everyone. I work for a small architectural design studio doing visualization work. We currently are using backburner in order to send our scenes to our in-house render farm. We are getting ready to move into our new offices very soon and have been looking into setting up our farm to take advantage of Vray distributed rendering as an alternative to backburner. I am aware that there are some drawbacks in using Vray DR such as having to have all your maps in your scene on each slave as is on your local workstation.

 

At our studio we work with thousands of textures as well as create new ones all the time, and therefore would be a real hassle to have to copy every new texture we make to each slave. We were a very busy team and honestly we just do not have the time for this. To all those who use backburner you are aware that having to copy your textures to the slaves is not an issue since backburner brings them over to your farm for you temporarily. Basically I want to know if theres any painless way to bypass this step. If this is unavoidable would there be an easy way to auto-sync your texture maps to the render slaves instead of doing this manually? Any insight on this would be greatly appreciated. We would like to get this to work with as little pain as possible. Thanks all.

 

Sal

Edited May 3, 2011 by Ragazzaccio

[Tommy L]

DR works the same as BB in this regard. The file being rendered references its files from whatever location you specify. If you tell it to look at a local drive you will have to copy the files to each machine (bad idea). But if you make sure you are selecting files from a server in max then the DR nodes will also seek the files on the network.

DR and BB work well for different things. DR is better for stills. BB is good for stills also using the strip rendering method. BB excells at animations.

Hope this helps,

Tom.

[Ragazzaccio]

Thanks so much for the quick reply Tom! We mostly do stills at our studio which is the reason we have been looking into DR. It seems to be more efficient for what we do here. We occasionally do animations but very rarely. The clients we deal with usually want to have things done very quickly. My team team consists of 3 visualization artist (myself included) and the 3 of us have to constantly create texture maps of real samples from its manufacturers. We deal with a ridiculous amount of clients who all request something different and therefore usually can't go through a job without adding new textures to our library. We have a render farm of around 10 machines so far and have plans to add more once we move into our new offices. I'm trying to find out what the best approach to set this up is. Would it help if the 3 of us pulled all our maps from a server instead of locally, perhaps one of the slaves or main slave? Then perhaps tell DR to pull those maps from the same location we are pulling our textures from when we bulld our scene instead of having to copy your maps on all of the other slaves? I'm just a little bit confused about how this works and looking for the simplest way to make this work. Thanks again and I appreciate your reply.

 

Sal

[Dave Buchhofer]

Would it help if the 3 of us pulled all our maps from a server instead of locally, perhaps one of the slaves or main slave?

Sal

 

exactly, set up the textures with network paths \\server\path\to\texture.png etc.

 

alternately for huge textures (IE: for HDR's that dont change often) you can set up a synced folder on the local machine drives using something like microsoft synctoy, that way you're loading that 150mb texture thats used in a ton of jobs locally instead of over the network every frame, and leave the normal size texutes to the server.

 

one thing to keep in mind, is that you'll want to have an actual server OS machine as your file server if you do expand the farm much further, as the windows connection limit can start to cause strange looking errors if you have too many connections to a simple workstation os (xp/w7/etc..)

[Ragazzaccio]

Thank you Dave for the info. The only thing I was worried about was that it may slow my scene locally while I'm building it due to it fetching textures across the network, especially if 3 of us are doing this. Have you run into any issues by doing this? As for HDRs we rarely use them since we mostly do interior scenes. We have a library of them but we use them to do renders of equipment only so therefore our HDR library rarely ever gets updated.

 

We were looking into a source control program such as perforce in order to sync all our textures and scenes to our server from our local machines, then have Vray read those textures for DR off the server. It was a good idea in theory but we are afraid that it will not work due to the fact that our render servers will have different user names than our local machines (of course). If this is the case it will not be able to find the textures unless there is an easy way to convert the texture paths. Any luck using source control while using DR?

[erickdt]

FWIW: I'd recommend you stick with backburner instead of DR within VRay. In my experience it's much more stable and predictable (VRay DR has a tendency to drop proxies regardless of whether or not they have proper VNC paths). Also, you're already experienced with it. If it aint broke right? As Tommy pointed out you can use backburner for strip rendering which will basically have the same result as DR for still images.

 

Regarding network rendering in the general sense it is always a good idea to keep all of your working files somewhere on a network where all machines can see them as opposed to working locally off your C drive. I suggest you get in the habit of doing this from the outset of a project as it will make your life a lot easier come final render time...

 

E

[Ragazzaccio]

Thank you Erick. The reason we were looking to switch over is because we run into problems occasionally doing strip rendering. Sometimes the render comes out with all these strip lines in it which becomes impossible to take out in post work. This doesn't happen all the time but its definitely an occasional annoyance. Since we do mostly stills and were thinking its better to use the power of all the servers to do one frame very quickly without any worries. It will also help us while we are building our lighting and doing test renders. Doing this with backburner is not possible. We would still use backburner when we need to, but we are looking for another available tool to make our lives a little bit easier.

[erickdt]

Hmm. That's interesting. I wouldn't be suprised if those stripe artifacts had something to do with all of the map files etc. not truly being on a network drive accessible to all machines. I could be wrong though. TBH I only use strip rendering when I have to do extremely large images. In those instances I haven't had any artifacts like what you describe. I typically have to do a lot of different views per project so BB works great for getting that done as quickly as possible. In terms of preview rendering I have a series of preview presets that I use when setting up my files which all result in fairly speedy preview render times (~1-2 minutes) noisy as though they might be.

 

E

[Ragazzaccio]

We always do extremely large images Erick because we work for big cooperations which usually need them high res in order to print them to presentation boards and show our proposal. Our final renders are always 2550x1650. It usually takes an entire night to render them out. Sometimes we need to make a quick change which can require only a render region. In this case we can benefit much more from DR because you cannot render to strips while rendering a region using BB. Theres many things we can use DR for in the studio. Backburner isnt bad by any means, but in certain cases its not that efficient nor practical for us.

[Dave Buchhofer]

the hdr thing was simply an example for instances where theres "large things being used often";

 

Over the network can be mildly slow when first opening the material editor, after that, it works just as normal.

[Tommy L]

We always do extremely large images Erick because we work for big cooperations which usually need them high res in order to print them to presentation boards and show our proposal. Our final renders are always 2550x1650. It usually takes an entire night to render them out. Sometimes we need to make a quick change which can require only a render region. In this case we can benefit much more from DR because you cannot render to strips while rendering a region using BB. Theres many things we can use DR for in the studio. Backburner isnt bad by any means, but in certain cases its not that efficient nor practical for us.

 

You shouldnt have ANY issues with strip, its foolproof....

DR will help. Its more push button (once its properly set up) than strips, but can be a little hinky on occasion.

I would consider purchasing a dedicated server. I bought mine for around $1000. I can send you the spec if you like. At the very least you should keep shared assets on there such as model, material and texture libraries. I work on my own, so Im not sure about the impact of three of you working off one drive. Should be a problem if its justr libraries though.

 

You shouldnt consider 2550x1650 extremely large. Its actually fairly small in print context and only slightly larger than 1080p.

 

BTW: a cheap switch can be a bottleneck. Make sure its a Gig switch.

[Dave Buchhofer]

. I wouldn't be suprised if those stripe artifacts had something to do with all of the map files etc. not truly being on a network drive accessible to all machines. I could be wrong though.

 

Yea, we would run into that issue also, though i can't recall what the fix was, as it was quite a few years back, i do remember there being a fix if you do a bit more searching.

 

another strips problem is that pesky one where Backburner couldn't combine strips beyond a certain image size. vray + vray image buffer can save to whatever image size you wish without being stuck with the huge frame buffer in memory (at 48bit x image size it can add up to a decent chunk of ram).

 

Anyway, its worth taking the time to figure out your issues.

[Tommy L]

Yea, we would run into that issue also, though i can't recall what the fix was, as it was quite a few years back, i do remember there being a fix if you do a bit more searching.

 

another strips problem is that pesky one where Backburner couldn't combine strips beyond a certain image size. vray + vray image buffer can save to whatever image size you wish without being stuck with the huge frame buffer in memory (at 48bit x image size it can add up to a decent chunk of ram).

 

Anyway, its worth taking the time to figure out your issues.

 

I strip rendered a 80" @ 300dpi image for corona (the sea) once. Thats 24k pixels across by around 10k high....with displacement!

[Dave Buchhofer]

I guess i should add the caveat that i haven't had to deal with strips since the 32bit os days... the strips always worked, but at the time, something over i think it was 16k refused to combine

[Ragazzaccio]

Dave Ive had this problem too and totally forgot to mention it. Its happened to me a few times and I had to wind up stitching the strips together in Photoshop which was a huge pain and waste of time.

 

Another issue I forgot to mention is that when I render to strips, I cannot render my render passes because BB will not stitch your passes together, and therefore every strip overwrites the last strip of your render passes. I'm a big fan of doing post work so this is another reason why I wanted to research DR.

[Ragazzaccio]

Is there any insight on how Vray RT (CPU) works with DR?