Rack Mount Render Farm... Help!

[Brian Cassil]

I think I've finally convinced "those who write checks" that having a dedicated render farm makes sense. I will be using it primarily for distributed rendering with FR stage-1. I've got a few questions:

 

1. Where to go? Price and customer service are most important here. I'm also considering having this custom built.

2. 64 amd's or dual core xeon's? I'd like to pick one and stick with it over the long haul since it's my understanding that when doing bucket rendering the two do not mix well.

3. I know I don't need video cards for these guys, but what about large/fast hard drives? I suppose this will improve read/write speeds which would come in handy in some scenarios.

4. Is there a big performance boost in using gigabit ethernet connection?

 

All your help is really appreciated. I'm looking to get something set up pretty soon.

[ihabkal]

I say the short useful info.

 

1- check http://www.cgarchitect.com/vb/showthread.php?t=10468 Jeff already asked the same question

2- Dual cpu dual core Opterons.

3- 35 GB SCSI hard drives

4-yes especially with Mental Ray, which turns some image formats into large RAW files.

[Devin Johnston]

1. Where to go? Price and customer service are most important here. I'm also considering having this custom built.

 

I had the same problem you've got, and I'm also using Final Render. We decided to go with boxx systems and they are all AMD's. We picked boxx because they test all their systems for compatability with Max or Viz, and they have great customer service. I've had them for over a year now and I've never had any problems and when it's time to upgrade I'll be getting some more from them.

2. 64 amd's or dual core xeon's? I'd like to pick one and stick with it over the long haul since it's my understanding that when doing bucket rendering the two do not mix well.

 

What ever you decide to get definatley get the dual core systems and I personally would go with AMD 64's. I think Final Render is going to be releasing Stage 2 some time this year and I think it's a 64 bit program. Once Windows comes out you'll really be able to do some fast rendering!

3. I know I don't need video cards for these guys, but what about large/fast hard drives? I suppose this will improve read/write speeds which would come in handy in some scenarios.

 

All my boxes have the basic on board video cards which is all they need but they do have 10,000 RPM SATA drives which really make a difference especially when there are a lot of maps to load.

4. Is there a big performance boost in using gigabit ethernet connection?

You have to get the gigabit connection especially when doing DR, if you don't you'll have problems with dropped buckets.

[Brian Cassil]

Thanks for the info guys. Here's the conclusions I've come to so far:

 

 

1. Where to go? Price and customer service are most important here. I'm also considering having this custom built.

 

 

I think I will be having the local place our office gets most of its hardware from build these. They don't have much experience doing these types of machines but they have been very good to work with and they are just down the street from us. Plus it will cost less, which means I can get more nodes.

 

2. 64 amd's or dual core xeon's? I'd like to pick one and stick with it over the long haul since it's my understanding that when doing bucket rendering the two do not mix well.

 

I wasn't aware that there were dual core opterons, but that sounds like the best option hands down. Maybe someone could direct me to what models are the dual core procs because I'm a little confused looking at AMD's website.

 

3. I know I don't need video cards for these guys, but what about large/fast hard drives? I suppose this will improve read/write speeds which would come in handy in some scenarios.

 

Ok, fast is what I need but not large. I believe the best price/performance would be SATA 150 drives (maybe 10,000 rpm if I can find a good deal on them) in a raid-0 configuration.

 

4. Is there a big performance boost in using gigabit ethernet connection?

 

Ok, sounds like a definete YES on this one. After talking with our office IT guy I found out we don't have any gigabit switches here, and that to get a good one is going to cost around 2k. We may try to fly with a cheap one ($200) for now.

 

Also, one more question: Is 2U better than 1U? I've heard some mention that there are overheating problems with 1U systems.

[Chad Warner]

We've been running 1u servers for about 3 years now, the only time they overheat is when a fan goes out. It's really all about making sure your heatsink/fan works with the case you buy.

 

Most server motherboards come with gigabit on board anyways, so I would get gigabit and have it for the future if you don't want to upgrade your switches at the moment.

 

-Chad

[ihabkal]

about the dual core opterons, they have models 265, 270 and 275, that are like 244, 246 and 248 respectively speed wise but with two cores instead of one so twice faster. speeds 1.8, 2.0 and 2.2 GHZ respectively. for cost get the 270 the system with 2 cpus will cost around $500 cheaper than the 275. for best available cpus get 275. wait 2 more months until windows 64Bit is easier to get and install and the drivers are available to make it easier on you so you wouldn't need to upgrade next year.

Opterons are faster in rendering with 3dsmax than XEONS.

[Brian Cassil]

I finally found the model numbers for the dual cores, as well as the info on clock speeds and their single core counterparts... just as you posted ihabkal. I also see that they are not available for purchase just yet... maybe in another month I hear. I always hesitate buying something so new on the market because you are really paying a premium price that way, and the price on these dual core cpu's is steep. I'm now thinking that I will start by getting just 1 dual opteron 248 system with 4 gigs of ram so I can at least stop rendering on my workstation, then in a few months when the price of the dual core drops get 1 or 2 of those to add into the render farm.

[ihabkal]

good idea. Xicomputer.com is selling the same configuration of dual core opterons $1500 cheaper than their competitors Boxxtech.com

around $5000 for a regular non rackmount system with 4GB ram and a Quaddro graphics card. I have been drooling over these systems since I heard dual opterons exist...I want one too...

[luckytohaveher]

Three pieces of advice:

 

1.) Make sure they are all the same... I mean exactly the same.

 

2.) SuperCache II from Superspeed:

http://www.superspeed.com/servers/supercache.php

It is absolutely positively required for the rendering manager and exceedingly recommended for the slaves.

 

3.) Dual-Gigabit Ethernet with bridged connections. For routing, you can join 8 port boxes together to keep the cost down.

 

Remember any rendering combination is an ecosystem. Any one weakness will totally waste the dual cores you are all worked up about.

[Brian Cassil]

1.) Make sure they are all the same... I mean exactly the same.

 

So your saying that not only do they need to be the same brand of proc... but also the same model, and the same type of ram, and the same whatever else? I've never heard of anyone having problems with distributed rendering on different models of the same proc brand. Maybe you could show me some threads where people have stated this as being a problem.

 

2.) SuperCache II from Superspeed:

http://www.superspeed.com/servers/supercache.php

It is absolutely positively required for the rendering manager and exceedingly recommended for the slaves.

 

In the past, for network rendering, I've always run the manager from my workstation and never had a problem. That's what I had planned on doing with DR as well. In order to use SuperCache II, it looks like I need to be running a server OS. Adding these sorts of things is exactly what will kill the practicality of even getting this ordered because all the bean counters see are $$$$. I've got to believe that this can be done without SuperCache and managing from my workstation.

 

3.) Dual-Gigabit Ethernet with bridged connections. For routing, you can join 8 port boxes together to keep the cost down.

 

I'll have to run that stuff past the IT guy here because I'm not sure what "bridged" connections are. But again, sounds like more $$$ which is bad for me.

 

Remember any rendering combination is an ecosystem. Any one weakness will totally waste the dual cores you are all worked up about.

 

Maybe you could explain this one a little better, because I don't understand why. If FR is rendering a scene and one system renders buckets twice as fast as another, won't it wind up rendering twice as many buckets and speed up the overall process?

[ihabkal]

"bridged connections"

many motherboards come with 2 ports for ethernet. Bridged connections is linking those together to act as one faster connection. done by windows xp settings. sometimes there are comptability issues.