In-house renderfarms

[Nils Norgren]

Hello CGA,

 

There has been much discussion lately about render farm services as well as building or buying rendering machines. We are refreshing our farm, but also working with a new IT service provider who is pushing us to look at various cloud providers for on-demand computing. We have also discussed doing a total cost of ownership analysis of the farm, not sure where to start with that...

 

That being said, I have a few questions for the group:

 

How many of you are using dedicated, in-house render farms?

 

In your opinion, what are the benefits of dedicated render power, downsides?

 

What are the breakdown between uses, between animation rendering (frames) and distributed rendering?

 

Of course feel free to add as much or as little detail to your response, I don't want anyone to reveal competitive secrets.

 

Thanks,

 

Nils Norgren

[John Dollus]

we use an internal renderfarm here so I'll add in what i can:

 

Downside:

maintenance overhead expense for routine IT support

initial cost and high rate of depreciation

if they aren't rendering, you are not using them to generate revenue

 

Benefits:

Immediate access

flexibility with the configuration (switch between distributed pool and frame cruncher quickly)

reliability (sorta depends on your IT support though)

 

use is increasingly closer to 50/50 regarding animation and distributed rendering use

 

for cloud computing and renderfarm services, reliability and access are factors. you simply can't control all of the points between you and the provider to guarantee there won't be outages or performance issues. As far as the financials go, the more you use an internal farm, the cheaper it becomes but the exact opposite is true of a external computing resource. Contracts have to be renegotiated and the prices rarely decrease. There's also the customization factor to consider. if you need a unique utility, script, plugin etc, you can deploy that on your internal farm with no problem but external may restrict you and limit technologies or techniques you could implement otherwise.

[Nils Norgren]

Thanks John, hope things are well! Is the render farm you are talking about up in RI? is it hard to work with it remotely? We have a Small Farm in NYC, but the big farm is in Boston, they(NYC folks) submit to the Boston farm when needed.

 

For whats it's worth, we recently switched to Deadline from Backburner, and so far things are much better. One big cost with the Render farm is the power and cooling, Deadline has a feature where it will hibernated slaves when the farm is idle, this should impact our bottom line directly.

 

-Nils

[John Dollus]

I'm in Dallas now with a design firm. Our farm is all internal and supported by IT staff so it does simplify things. That said, I've never really had much trouble working with a remote farm setup. As long as the pipe is big enough, it works pretty well.

I'm a huge fan of deadline and have started using it at home for my own measly 2 system setup. The hibernation feature would have been great to have in FL. With everything running all of the time, my room temps were always uncomfortably high in the afternoons

[neil poppleton]

In house render farm. I always get a headache thinking about external render farm providers....how do you upload all mpas, size of files, proxies, maps, do all plugins work, how do i get 6000 tgas and passes back quick.....starting to get that headache again..

[Scott Dombrowski]

We just brought a small internal render farm online. 2 nodes, each is a dual hex-core Xeon. I usually don't have enough schedule time to do animations, so I use them mainly for DBR. Sometimes I'll fire up Backburner so I can send a final render down and continue working on my machine.

 

Downside:

Cost. You pay for these up front, so the cash-flow is totally front loaded, not pay as you go as it would be for an online render service. The up front cost is not as big of a deal for a larger firm, but it's probably an issue for a small shop.

 

Upside:

Speed. It's like cramming another 24 cores in my desktop. There's some small overhead for sending the scene data to the nodes, but it's on our dedicated gigabit network so that is minimized. It makes the test renders go so much faster. Plus, no wait to download finished files from an online service. My deadlines are tight and minutes count sometimes.

 

Over the long run, with as much use as the farm is currently getting, my ROI will be in under a year. This is based on increased speed in my daily test-rendering workflow plus now being able to use my machine while the farm is doing a final render. Plus, they are available to the engineers for other computational crunching (traffic modeling, structural analysis, etc.) when I'm not using them.

[Nils Norgren]

In house render farm. I always get a headache thinking about external render farm providers....how do you upload all mpas, size of files, proxies, maps, do all plugins work, how do i get 6000 tgas and passes back quick.....starting to get that headache again..

 

Thanks Neil, i feel the same way about outsourcing, we have done it successfully, but having the resources at fingertips seems so much better, how big is your farm?

 

-Nils

[CGRendering]

Hey Guys,

this week client called and says that they need animation to be done by Monday first thing.

So decided to use render farm. What render farm is the best what web site? Is anyone had experience?

thanks

Slava

[Tommy L]

I think there's a huge divide in the discussion about studios vs freelancers in this topic. Lone freelancers have a different operating model to studios but here's my take: (Im a lone freelancer who occasionally collaborates)

GOOD:I bought a farm 5 years ago, 7 of the cheapest machines money could buy. They were the best investment Ive ever made along with Vray. Ive upgraded them so they are still fairly current. My total investment would be around $10k, so 2k a year. They enable a style of workflow thats just impossible on one or two machines. Testing renders at close to final quality with DR and I have never had to send out to a commercial farm. Its also an interesting feature of the room and clients often say 'wow, you have so many computers...'

BAD:The bad would be the maintenance and network issues. Sayng that, maintenance has only been replacing a few power supplies and network issues have been a thing of the past since having all machines on W7. Upgrade was tough, pretty much replacing everything. Network issues nearly drove me crazy. But thats part of why its a different discussion for freelancer vs studio. I have to do everything myself.

 

Seriously best thing about having power? It gives you the edge over someone who doesnt.

[Nils Norgren]

here is a quick pic of our refreshed render farm, -Nils

[ATTACH=CONFIG]47169[/ATTACH]

[marius e]

wow, what on earth do you do with that!

[Nils Norgren]

wow, what on earth do you do with that!

 

The main farm (at this time) entirely dedicated to Max/Vray rendering through Deadline.

[John Dollus]

looks similar to ours here. In the summer, it's nice to go in that room to experience the plenum cooling

[odouble]

That's a ton of power in that farm.

We have a render farm of (4) dual quad xeons and (5) dual 6 core Nehalems for 4 guys.

The benefits for us are instant/on demeand use. It really shines when we have to make last minute changes on huge animation scenes. this can happen on 2 or more concurrent projects in different stages. It's hard to imagine that we could push all our necessary assets back and forth easily to a cloud service in crunch time.

I really can't think of any downsides to having a farm at my fingertips.

[Frosty]

wow, what on earth do you do with that!

 

You do know who Nils is don't you?

http://www.neoscape.com/

 

Nils,

I too feel there is a huge benefit to an in-house farm. When I freelanced, I had a 10 node system (2003-2006). It was awesome.

From a design perspective it allowed me to do things that were simply impossible for others in my M.Arch program at the time. Specifically, designing with soft-body dynamics. And I second Tom's comment. Clients are VERY impressed when they see "the wall" that will be crunching on their project.

 

Right now I am in the process of setting up a farm in our office that will utilize all of the i7s after hours. I'm close and the IT guy is very supportive and enthusiastic about it.

[marius e]

haha, thats sick. Strange, its like having a sports car, you just wanna go drive...

 

Damn nice nils.

[egmehl]

Another idea to consider is using Amazon ec2, or the equivalent from Microsoft (and I think rackspace has a similar thing).

 

Good: Far less expensive than the render-farm specific services (per hour of use at least). You are fully in control of plugins, etc. on the nodes. You can save a lot of last-minute upload time by doing periodic syncs of your active projects to S3 (storage service). Then you'd only have to upload the assets that have changed since the last sync when you are ready to render. There are solutions for mounting the S3 storage like a regular drive for Windows & Linux which is convenient.

 

Bad: you will have to do all your own IT work to figure out how to get it going with your software, render manager, etc. You might have to get more licenses if you are using software that wants a license for render nodes. There's still an upload to start / download when finished bottleneck.

 

There's a 100 node limit, but the site says you can request that to be raised.

 

At the least, it's something more to plug in to a spreadsheet and see how it compares.