New studios, Maya Suite Premium, and Render Farm questions

[danielmoore]

Hello Everyone,

I am working for a small 3D Studio that just started up. I am in the projects Audio Design team, however I have also been tasked with the technical aspects of the company. I am not as experienced as most should be for tech work, but I do know my way around decently enough. Currently they have not received their workstations, render nodes, or server. It has been appointed to myself to figure out the most reliable and cost effective way to get the studio in working order. I already have the quote for Autodesk’s Maya Suite Premium, so now I am really just down to the hardware aspect. I would like to share a break down with you of what I have as my idea thus far from my research. I haven’t ever had experience with moderate technical infrastructures so I am just in need of a little help understanding how this will work.

 

 

Our project is a fully animated motion picture. For the motion picture, they will be modeling/textures/animating everything as realistic as they can without going overboard. They are wishing to achieve the best render capabilities they can on our budget. The budget they gave me is $20,000. They want all components to be new and reliable. Also if it helps, our project is for 3 years.

 

 

Their idea is to have a small/moderate render farm to achieve this. What I have thus far is 6 workstations (Maya suite will be installed on), 10 render nodes, and 1 file server. Specifications for each of these are listed below:

 

 

Workstations: (One will also be used as the Queue Supervisor via Qube! and will have a KVM

switch hooked to it to monitor the nodes)

- Windows 7 Professional 64 bit

- Intel I7 3930k (4.2ghz)

- Asus X79 Maximus Gene Motherboard

- 32Gb DDR3 1600

- Nvidia Quatro K2000

- WD Red intellipower 1Tb HDD

 

 

For 6 = $14,292

 

 

Nodes:

- Windows 7 Home Premium 64 bit

- Intel Pentium G2020 (2.9Ghz)

- MSI H61 Motherboard

- 4Gb DDR3 1333

- Intel HD Graphics

- Samsung Spinpoint F3 250Gb HDD

 

 

For 10 = $3,340

 

 

File Server:

- Windows Server 2008 Standard

- Intel Xeon E3-1220 (3.1Ghz)

- Supermicro 1155 C202 Motherboard

- 8Gb DDR3 1333 UDIMM ECC

- Nuvoton WPCM450RA0BX

- 3ware 9650SE-12ML SGL PCI Express x8 SATA II (3.0Gb/s) Controller Card

- 8, WD Red intellipower 1Tb HDD’s (2 in raid 1 for OS, 6 in raid 6 for file storage)

 

 

For 1 = $2,240

 

 

Total = $19,872

 

 

This is all for the core of the studio. As far as our audio design and office workstations, those are already good to go. Also networking is pretty much covered as well as far as hardware, but I do have a few questions about running Autodesk on the local network.

 

 

Questions:

1.) I know that the Maya Suite/rendering software (like Mental Ray) needs to be on the render nodes. My question is whether or not it has to be the full license or just a stand alone non activated instance? Can’t find a clear answer on this.

 

 

2.) Is it ok, to use one of the workstations as the Queue Supervisor if it will be utilized as a slave anyway for quicker renders? Also will a VNC work just as good as a KVM?

3.) Will the studio require a network license for Autodesk if all the work is done at the studio on the workstations and sent to our local file server at the studio.

 

 

4.) Will the above hardware configurations work for our purposes of producing a semi-realistic animated motion picture?

 

 

I know this is asking a bit much, however I am really in need of some help with this. Any help is greatly appreciated.

 

 

Thank You & Best Regards,

Daniel Moore

Lead Audio Designer

Edited May 7, 2013 by danielmoore

[Dimitris Tolios]

The render nodes are seriously underpowered for what you are planning undertaking.

 

I don't know if you are preoccupied against AMD, but pretty much all new vichera based FX CPUs, will embarass the G2020. I get it, it is an ultra cheap chip, but that says all you need to know about it at the same time: the G2020 is a very basic dual core, capable of serving basic office applications, internet browsing and light HTPC tasks.

 

For $600 +/- 10% you can build FX 8350 based, 8-core machines @ 4GHz that rival 3770K performance and clearly out-perform i5s - as all-out rendering CPUs, but also for other multithreaded apps.

That price includes Win 7 home and ofc 16GB of RAM (the max for home edition windows). 4GB will struggle even with relatively low resolution textures and frame outputs.

 

The nodes will be an important part of you project. I would even prefer to downgrade a few 3930K workstastions to 3770K than "force" those low specs to the nodes if I was running out of budget. For most modelling tasks the 3770K is just as fast, if not faster, as a 3930K.

 

Even using fast i5 CPUs for the workstations with lots of RAM and those decent GPUs, and "demoting" the 3930K to render nodes using a basic $15 GPU for them to boot would be a more justifiable strategy than the G2020s. Workstations won't be using more than 1-2 cores outside of rendering and the i5 will be just as fast if not faster (Ivy > Sandy) than the 3930K during most of the content creation process, while the 3930Ks will make short work of rendering tasks where 6C/12T actually does matter.

[Scott Schroeder]

Mental Ray through Maya doesn't need a full license if it is just for a render slave.

[Francisco Penaloza]

Dimitris is right, your PC are very small for the task that you guys are planing actually, and I don't want to sound insulting in any mode, this task should not be yours, keeping those computers working and rendering will be much of a task as creating the movie it self, you need to talk with your management and re-think your strategy, you need to hire a IT professional, at least keep it in call, he will know exactly what type of PC you need for each function, for instance you don't need 2x1Tb Raid HDD to run windows server, the Intel Pentium G2020 will be the big disaster for your project they won't render on a decent time, your work station will be running in circles while your "render nodes" try to load your scene, Windows Home is not useful or practical for any major networking set up, and the list goes on.

If you are short in budget you guys should look on paid render farm, such REBUSFARM or The Ranch and any similar, maybe you can call them and get a better deal. sometimes pay this guys is more efficient than build and maintain a smaller render farm. Honestly reconsider your strategy, before you end up working 24x7 for 3 month and sacrifice quality for what ever can be rendered.

[Scott Schroeder]

+1 to everything that has been said.

 

You have really low end render farm boxes. The reality is you should put your power in your render farm as that's where you'll need it and keep the medium-high machines to the user boxes. It sounds counter productive, but it's not. There is no point maintaining a daytime render farm if you can't render on it and all you do is wait to get the higher end user boxes on at night.

 

If you are doing an animation that spans 3 years production time, you presumably are doing something that is going to be fairly long. Let's just say you have a target of 30 minutes of animation. That's 54,000 frames, minimum, you need to render. If each frame takes 20 minutes to render, that is 18,000 hours or 75 days of pure render time, that takes into account 10 machines working full time around the clock.

 

You may want to rethink your strategy here. Online render farms can be cost savers if you use them right. Especially if this is your first project and you aren't exactly sure if there is going to be a follow up.

[danielmoore]

Thank You all for the replies. I completely understand where you guys are coming from. I was under the impression before reading your replies, that just simply more cores = faster times. Francisco, I do not take what you said insulting by any means. The information you provided is exactly what is needed, no matter the method of delivery . I had a meeting with my project lead this morning about using online render farms. After checking out the sites, I believe it would make what we need to achieve much more obtainable than what they are wanting me to do. However the decision is his to make. He seems very keen on having his own render farm, which I can understand to a point. With our budget though, he may have to choose a less demanding project for the first run.

 

I have a great deal of confidence in our small team to be able to create a semi-realistic animation motion picture. The motion picture they have drawn up is ~1hr 27mins by their sequencing of the film and acting it out from start to finish. They have done several very realistic shorts that are very impressive. However, on their current home based machines, we are talking render times exceeding 10+ days for something 4-5 minutes long lol.

 

I brought my project lead to the understanding that we are in need of an IT guy if he plans to follow through with a farm. I am apart of the project to do what I do best (music/audio). Sure I can build a computer, but I am no guru in the IT department. I do have to say though, what you guys have said really makes me want to understand this concept better. I have taken a new approach just in case the project lead still wants to build a small farm, I've changed things up quite a bit due to your responses and I just wanna see if this makes more since.

 

Workstations:

- Windows 7 Professional 64-bit

- Intel i7 3770 (3.4Ghz)

* Did not go with the "k" model due to the fact that we aren't going to oc and still have to remember

the budget.

- Asus P8Z77-V Motherboard

- 32Gb DDR3 1600

- Nvidia Quadro K2000

- WD Red intellipower 1Tb HDD

- Asus ProArt Series Monitor

 

Going with 6 still

 

Render Nodes:

- Windows 7 Professional 64-bit

- Intel i7 3770 (3.4Ghz)

- Asus P8Z77-V Motherboard

- 8Gb DDR3 1600 (Will 8 be enough?)

- Intel HD Graphics

- WD Blue 320GB HDD

 

Going with 10 still

 

NAS Box:

- OS (Not decided yet)

- 4, 3tb WD Red HDD's (Raid 10)

- 3ware 9650SE-4LPML Raid Controller

 

Node Supervisor/Queue Manager:

- Windows Server 2012 (Lead said he planned on purchasing '12)

- Intel Xeon E3-1230 V2 (3.3Ghz)

- Supermicro 1155 C202 Motherboard

- 32Gb DDR3 1333 UDIMM ECC

- Nuvoton WPCM450RA0BX

- 1Tb WD Red HDD (decided to only due 1 drive because this system is just going to mainly manage the

nodes and help with renders)

 

Total Cost of these systems not including the windows licenses = $19,446

 

This would give us if I am not mistaking, 68 physical processors and 68 logical processors that would be rendering. That sounds like nice horse power, but nothing compared to the online farms unfortunately.

 

I don't mean to be man of 20 questions here, but I do have a couple more if you guys don't mind helping me out just a bit.

 

I am under the impression that it is best to keep the same architecture for all of your systems (AMD or Intel based). Lets just say I had Intel workstations and AMD nodes, I've heard that this can cause some rendering headache.Is there truth in this statement?

 

Reason I ask is because of dtolios's comment. AMD seems to keep up for a fraction of the cost (performance per dollar). 1 AMD 8 core vs. 1 Intel i7, the intel seems to win. However you can almost buy 2 AMD's for the same price of 1 intel. So would it be more conceivable to go AMD for the nodes? I mean, how would 10, i7's compare to 15 or 16 AMD 8325's or 8350's?

 

Also, it looks as though we are gonna have an extra copy or two of win7 Pro. Would that do fine for the NAS Box or would FreeNAS be better?

 

That is as much as I will burden you guys with until I find out what the lead wants to do. We have a meeting this weekend where the funds will go out one way or the other. Just hope it's a good decision.

 

Thank You all again for the assist,

Daniel Moore

Lead Audio Designer

Edited May 7, 2013 by danielmoore

[Nic H]

Im not a technical person but you are going to want to about the same RAM on your render nodes as your workstations.

 

If you end up unknowingly using the RAM on the workstation scene files at 16gb or so to render (not unreasonable usage for a heavyish 1080p scene tbh) then send to nodes with only 8gb they will crawl and barely render...

 

i think you are somewhat underpowered for what you are trying to achieve. what about all the draft versions and amendments? unless you have a super tight and seamlessly executed schedule its going to be difficult at some point. I would assume at least around 20mins for a high quality frame - some much less and some perhaps more. Im speaking from turning around 3-4 minutes of architectural animation around in a couple of weeks

[joelmcwilliam]

Try to put 16 gb ram in your rendernodes en monitor to see if you need more.

 

Because you are not overclocking you should be fine with a B75 or a H77 mobo if you go for intel cpu's ( for example ASRock B75 Pro3-M). Z77 only needed if overclocking.

 

You might even look at Q77 mobo if you find the features the Q77 chipset has to offer useful.

 

For your workstation you might want a faster HDD:

 

http://reybasti.hubpages.com/hub/Top-5-Best-2TB-Hard-Drive-For-2013

 

For maya i believe an AMD GPU will be better.

 

http://www.pugetsystems.com/labs/articles/AutoDesk-Maya-2013-GPU-Acceleration-166/

 

http://www.cgchannel.com/2011/10/review-professional-gpus-nvidia-vs-amd-2011/

 

http://www.polycount.com/forum/showthread.php?t=92072

 

If you care about the electricity bill go for intel, if not go for AMD. In the case of rendering there is no real difference between the intel i7 3770 and the fx 8350.

 

http://www.xbitlabs.com/articles/cpu/display/fx-8350-8320-6300-4300_7.html#sect0

 

http://www.behardware.com/articles/880-7/amd-fx-8350-review-is-amd-back.html

 

(You even might consider buying a ssd just to experiment and see if it has benefits for you)

 

The stock intel cooler can be loud under load. If that annoys you, you might consider buying a aftermarket cooler:

 

Gelid Solutions Tranquillo Rev.2

 

For your workstation as well as your rendernodes this PSU will be sufficient:

 

Seasonic G-Serie 360Watt

 

I see no cases.

[Devin Johnston]

Daniel I think your under estimating how long it will take to render out an 87 minute movie. Assuming you can get your render times down to 20 minutes per frame per machine you're looking at a total render time of 135 days. That's 4 months of full time 24 hour a day rendering where no one can touch their computers or make any changes. I'm not saying it can't be done but there would be no margin for error, I think it's much more realistic to expect to rent time on a large farm when it comes time to render out the final product.

[Scott Schroeder]

Your team can create all the cool sequences they want. If it takes 6 months to render them, then all of that effort went down the toilet. For an 87 minute movie you are going to need way more than 10 machines for the day and 16 nighttime machines.

 

Listen the to the render math that is being given to you. We aren't pulling these number out of our arses. The rule of thumb should be that your farm boxes are at least equal, though they should be higher powered in reality, than your user boxes. Especially in the case of RAM as nicnic points out. Render farms should never be penny pinched, it will bite you every time.

 

Just curious, but why do they have you, the audio guy, doing this work? Should they take one of the actual CG creating team members and have their input on this?

[Francisco Penaloza]

Hi, your PCs are getting better, but not there yet, to your questions, there is not problem whatsoever mixing AMD and Intel, and shoot if you have you can trow some Mac's on the pile too, some people here will suggest mini mac's but that's other problem

You mentioned that some people in your team have experience creating CG animation, I am guessing most of you are just coming out of school or work for a little for some one else and now try to make it alone?

 

That's cool but as mentioned before one thing is modeling, other one is managing a rendering farm, is not that hard, but so many thing can go wrong and if you are guessing it will take forever to debug, where I work (Arch Firm) from time to time beside our small farm we use architect's PC to handle our animations, we get a total of 30 extra decent machines, between 16Gb and 32Gb Ram, and yes we do get 1 hour a frames in some animations full HD, so do your math, plus you not only have the rendering time out of Maya or 3d Max you'll also have the rendering time out of After Effects, or Nuke or what ever you use to comp.

Out of your computer list, I am missing who is gonna be managing your textures and rendered frames? the NAS station is cool for small project but when there is many PC reading and writing the small cpu of those boxes won't be able to handle. In a nut shell you need, Workstation, server for textures and meshes proxies and frames storage, ideally this can be done in other server, then a render manager and render nodes, one render node can perfectly be a render manager too. Your Servers will work better if they have several small disk instead few large ones, and all this under a good Gigabit network environment.

If this animation is a hobby experiment or just to try yourself, a la Blender, with whatever you get it will be fine, now if some one else is paying you and waiting some result in a timely manner, cutting cheap in this component will make your project fail and after that nobody will hire you again, at leas under the same company name.

 

So you mentioned your the "sound" guy there, what software you use to compose? I am a musician my self too and from time to time I do the music for some of our animations too

[danielmoore]

Hey Guys,

 

Thank you all for the information. It has helped us greatly in deciding our way forward with a commercial development as an indie developer. Our project lead and I have been discussing a great deal on how we are going to achieve what we have set out to do. We have decided to just get our workstations and team ready to start our motion picture development (long story short). The team lead has yet to decide on which farm the company will be using, however he will be working with a third party to render the motion picture.

 

The reason why I was asked to look into farming is partly due to my ability to accurately communicate my understanding of the information obtained here in a timely, organized fasion. Everyone on this small team is having to partake on a few adventures out of their comfort zone like I have here. Someone else could have asked, but then I would not have learned what I have , which I thank you all for.

 

As for what software/hardware I use to compose (Even though I am the Lead Audio Designer on the team. I do composition mainly, I have an audio engineer who works with me), thats quite a broad question. I am guessing Francisco Penaloza, you are reffering to my DAW which is Cubase 7. I use many virtual instruments to write music along with Cubase.

 

Our team has put a lot of time into the production thus far. Our lead pulled some strings apparently to release some budget pressure and things are looking a bit more "real" to say the least. Thanks again guys for the assisst, hopefully i can return the favor soon.

 

Thank You,

Daniel Moore

Lead Audio Designer

Edited May 18, 2013 by danielmoore

[Fooch]

Sounds like a wiser choice.

 

We've got a lot more muscle gear wise here and thats only for our 2-3 min films constantly. A full feature will be a killer render wise and i would be looking at say 4-5x times more farm gear, easily...

 

To save cost for your server, why dont you get a linux fileserver built up. Windows server is good but you need to pay heaps for extra machines to get on it.. which can get big quickly when u have a massive farm connecting to it.

Get a server person to look at it too. Trust me. I've been there starting a system up and it pays to have good heads telling you what to do.

 

plus , good luck. this sounds like a fun startup. enjoy the growth years.