Render Node Configuration in Light of New V-ray Licensing Scheme

[braddewald]

I'm wondering if anybody has any thoughts on whether Chaos Group's new licensing system (pay-per-node) impacts what hardware you would choose if you were to set up a new render node. In the past, it seemed you should do whatever would get you the most cores/aggregate GHz for your money. But now since you're being asked to put down an additional $250 for a node license, doesn't it make more sense to go for the higher end processors to get as much oomph from the node as you can?

[Scott Schroeder]

We ended up purchasing the bundles, which tended to cover our render farm needs. For the oh-my-god last minute animation needs, we'll rent out a few extra vray licenses (which are rolled in to the last minute cost to the client).

 

I will say this. It made us look at our old render farm and really decide if having that particular box on there was worth it or not. In the end, I believe our render farm has been made much more lean and efficient with this new pricing scheme. Despite only having 1/3 of the boxes licensed that we had in the past, we have yet to hit a render wall.

 

So in a long winded answer. Yes, I would agree that you go with fewer more powerful machines rather than multiple "good enough" machines.

[braddewald]

Thanks for the response. I'm glad that it's not just me thinking that way.

 

I see what you mean about making your farm more lean. I have a couple nodes that were really expensive but are now pretty outdated and every time I get add another node, it makes no sense to just ditch them because I could always use the extra buckets. But with the new licensing, it may make sense to finally retire them, rather than essentially paying $250 to keep each one going.

[Dimitris Tolios]

You will have to factor in the licensing scheme, and your upgrade pattern / schedule, but it is not as complicated after all.

 

http://www.chaosgroup.com/en/2/vray_licensing.html

 

VRay is not a program that upgrades on an annual cycle.

Getting 3.0 now, you are probably good for 2-3 years - even more - it's not like 4.0 will make 3.0 irrelevant.

 

You get 2 nodes with the workstation license. That's it. Then you need $250~350 per node, technically for the life of the node. So it is not a $250 per year per node or w/e you have to factor in...

 

If you were already going for either a quad core i7, 6 core i7 or a decent Xeon in 1P configurations, it is hard to get anything significantly faster for those $250~350. You might squeeze a few extra MHz here, getting a faster s1150/s1155 CPU or a couple extra cores in a slow Xeon, but that's it.

 

It won't cover getting a s2011 6-core over a s1150 quad, it won't come close covering the difference for fast s2011 E5s over slower E5 or E3s.

 

Yes, it is an extra cost, but economy of scale should kick in only if we are talking lots and lots of machines and just makes "expensive" 2P solutions appear more desirable for large studios/individuals that would had this option open.

For individuals looking for an affordable solution, counting their beans here and there, this just pushes costs up with no real way out of it.

Eventually you will end up spending more money.

[braddewald]

Dimitris,

 

I took your advice and built a few nodes with FX-8350's. Do you not think it's smarter now to go with a more expensive/powerful processor since you're going to be adding on a $250 per machine cost for the node license?

[Dimitris Tolios]

Well...kinda hard to explain it to myself too, so I went to my lil spreadsheet....disclaimer - there are lots of "assumptions", not to make my logic forcefully "work", but to keep things simple.

 

Apples to apples, my $500 & $600 nodes, would cost ~$1,010 should you wish to go for a 4930K ($580) and the cheapest X79 board (Asrock Extreme 4 $200).

So the break down would be roughly something like that:

 

 

Assuming that Vray - at least version 2~2.4 - had 8350 and 3770K around the same performance (with the FX on the top), it is kinda fair to assume the slightly lower clocked Haswell E3 will be roughly the same. Maybe it is a tad faster or not, but that's the assumption I wish to get granted for the sake of the example and be "generous" with the 8350.

 

The 4930K node, is assumed to be 1.5x the speed of the other two. Something also pretty generous in favor of the hex (it is not that faster), but keeps the math simple to follow. I've also rounded the cost down to $1,000, assuming you can get dirt cheap GPUs (not factored in, but required) etc.

 

So, a 8350 is a "performance unit", and we assume that when rendering, a 4930K is 1.5x performance units.

 

Lets see a scenario with someone adding 8350 nodes to his existing workstation.

The first 2x licenses are free with his original workstation licence for Vray 3.0. The rest are calculated based on the scale given in the URL above (i.e. licenses get gradually cheaper the more you buy, first 4 are $350, next 5 are $280 etc.

 

 

Lets do the same math with the 4930K node.

 

 

I won't repeat it for the E3, but up to 5 nodes or so, the E3 holds on to its "value" better. i.e. you get more performance per $ spend, despite software costs being higher than that in the 4930K scenario. After that, the 4930K gets the upper hand by a few $/performance unit, but the overhead for each node on hardware level is so higher, that technically the 2 options remain within 5% of output per $ spend, with the 4930K taking this small lead after you start spending serious money and deploying nodes in the 2-digits.

 

Still, given our assumption has even the slightest gravity, the 8350 holds its own just as well - of course using more nodes, adding some redundancy but also making the node farm a tad harder to manage and deploy.

 

In the end, the 4930K might be the better choice from the perspective of getting something that is more versatile as it can be used as both a workstation CPU - where it vastly outperforms the 8350 due to single thread performance efficiency (it is easily more than 1.5x faster), saving space and micromanagement time etc, but there is still no clear answer, as the FX's value for the "render node" niche is good enough.

Edited July 8, 2014 by dtolios

[RyderSK]

wow, excellent table write-up by Dimitris ! Now that's some patience I wish I had :- )

 

 

On realistic level, if someone is counting beans, he won't be able to afford (or pay for electricity,esp. the FX) even 4-6 nodes, these people struggle to buy good workstations.

On other side, if someone has enough money to buy more nodes, he really ought to be smarter than to buy 10+ low-performance loud boxes, and pay for someone to manage that, put it in separate room and whatnot.

I will probably never understand this want.

[numerobis]

I had done a comparison of single 4930K nodes vs dual Xeon boxes a few days ago... it is based on the Maxwell benchmark Benchwell (BM=benchmark score) (http://www.maxwellrender.com/index.php/benchwell/page/17) and prices are in Euro, but you can compare the price-performance ratio for the hardware+OS (without vray licences)...

 

 

If you compare 3 4930K systems to one dual 10 core sys (E5-2680 v2) it's almost the same price and price-performance ratio (incl. 32GB RAM 1600MHz, 120GB Samsung 840 EVO SSD, Thermalright True Spirit 140, video card, Seasonic PSU, case, Win 7 Pro x64). And the ratio is nearly the same for all mid range Xeon systems, the 12 core system is clearly less cost-effective.

When you look at 3 overclocked 4930K (@4,4GHz) you get a 33% better ratio compared to the 2680 V2 10-core sys.

 

The advantage of only one box compared to single nodes is that you need only one piece of every component and you can easily increase the RAM to 64GB instead. Comparing 64GB systems would result in better values for the dual CPU systems. And administration is much easier.

The only problem is the limited per core performance if you compare them to a high clocked i7.

 

Concerning the v-ray licences... are we speaking about upgrade cost for nodes or new licenses? If you have a v2 license the upgrade price for 1 workstation + 10 nodes is $970, which is $88 per node if you include the workstation into calculation. I think, this is high compared to the free nodes before, but still ok. But above these 10 nodes this new licensing really becomes unfair when you have to buy new full node licenses as far as i understood...

Edited July 8, 2014 by numerobis

[RyderSK]

You guys have some patience heh ! :- ) It's so nice to read these charts.

 

4930k do become more price effective when overclocked, but is it realistic to run 1000 euro node with its components overclocked in its limits 24/7 and assume that will be stable or even reachable in all cases ? It's very optimistic scenario, one that will also draw nice amount of electricity.

 

The Vray upgrade path is very fair, much better than vanilla V3+ 10 nodes...

[Dimitris Tolios]

You guys have some patience heh ! :- ) It's so nice to read these charts.

 

4930k do become more price effective when overclocked, but is it realistic to run 1000 euro node with its components overclocked in its limits 24/7 and assume that will be stable or even reachable in all cases ? It's very optimistic scenario, one that will also draw nice amount of electricity.

 

The Vray upgrade path is very fair, much better than vanilla V3+ 10 nodes...

 

Well, your logic on overclocking & wattage (on an above post) stands on its ground as you are deploying more and more nodes, which are working 24/7 etc. Yes, the 8350 is a powerhog, yes, the 4930K overclocked is much much worse, but supossingly it is working to save you time (time is money) or to directly make you money. Even in Europe, where electricity cost is the highest (I believe), you should be making more money than your electricity bill, or valuing the experience your experimentation with a couple of nodes gives you more than your electricity bill increase.

 

It is the reason you never realistically talk about fuel and tire costs in supercar reviews - when you "realistically" looking into getting one, you can afford it nomatter what it is. That, or you are not realistic, cause you darn should know that anything with 500HP or more, burns through expensive rubber as if it was cotton candy, and gulps gas faster than a German drinks beer in Oktoberfest (perhaps before 9am)!

 

Not all users with the need for a couple of nodes will have to run them 24/7, and of course very few will end up deploying $6K, $10K or more expensive farms, with more than 32GB of RAM for each node etc.

 

The numbers above are for "illustration" purposes. FX and E3 nodes are valuable for those starting slow, getting a node or two to speed up their workflow. Up to $3~3.5K or so total for the farm, it's ok, you get 3-4 nodes and you pump away. More than that, and the 2P multicores (not 1P 49xx that much) start getting more and more attractive, but whoever is talking this kind of transition, is turning from small time solo crook, to a small studio. Right Juraj?

[RyderSK]

I agree :- )

 

When I mention electricity cost, it's not in fashion of running these, but only as potentional price offset compared to initial buying of the nodes.

Confusingly articulated, over the moral lifetime span of these nodes (3-4 years) will the additional electricity cost not offset the higher prices of less more expensive nodes ?

 

I didn't do any exact math, just my hypothesis on this.

[braddewald]

So is the consensus that the difference is negligible? I currently have 7 nodes:

 

Node 1: Xeon E5520

Node 2: Xeon E5520

Node 3: Dual E5-2620's

Node 4: FX 8350

Node 5: FX 8350

Node 6: FX 8350

Node 7: FX 8350

 

I want to at least add three more nodes, since I've already purchased 10 licenses. My thoughts were to go with higher-end processors, since to get the same power from lower-end processors I would need more of them, which would put me over the 10 licenses I have now and I would need to start adding $$$ for a license to the cost of each new node. Does that better explain my dilemma? I'd really appreciate any advice.

[Dimitris Tolios]

Take those numbers lightly, but that's the 1st thing I've found with most CPUs included:

 

 

 

Disclaimer - Again, take these with a grain of salt. Its based on data that I did not do any real research on - just fast data mining from a fast google search - and some personal assumptions / scenarios that "made sense" for the sake of this argument.

 

 

 

Source: Electricity cost

 

So, what I see from the above figures, is that for a couple of nodes and what I really consider a realistic usage scenario, consumption doesn't really matter. Idling consumption between platforms is very very close - simply because the CPU is not the only part consuming.

 

I believe that for small time crooks / small companies that work mostly on stills, a 700~1000Hr/Year usage scenario is realistic. You work 5D weeks / 40Hr week average (some days you have all-nighters, others, you don't work as hard, nodes don't need to be working - even idling). You use your farm during the day for test renders and final renders once a while.

 

In this scenario, the s2011 platform is slightly less efficient power-wise vs. a s1155/1150 but only by a tad. The 8350 will burn through more power, but since idling consumption is not that much, the differences are not enough to recuperate the extra overhead for setting up the Intel platform through electricity bills. We are talking $200 a year, in the most expensive countries, with US closer to $70 a year - or $11-12 per node.

 

Maybe it is not realistic for you, so I've dropped in that "heavy load" scenario, where the farm is standby 24/7 all year, and works 8Hr a day, every working day. Averaging some week-long continuous tasks, with slow days and busy ones, I think this is above your average workload - unless you are a big studio, doing lots of projects and even renting your farm to others.

 

In EU, we see a significant "rise" in running costs using FX CPUs. But in the US, RU and China, you will still spread recuperating costs over 2+ years.

 

We also see that the multi-core s2011 platform becomes the most efficient of the bunch in this scenario, and of course should we plug in numbers for 2P 10C/12C or even 8C Xeons, we would get vastly better numbers.

 

But all this is relevant to farms - not couple of nodes.

Edited July 8, 2014 by dtolios

[RyderSK]

Impressive, thanks.

 

I still don't completely share the conclusion though. Your "heavy" scenario is basically 8hours a day "full" rendering. That's not any extreme and I would consider to be rather average and much more norm. Even with the small farm solution (6x4770/8350,4x3930) you can save up to 100-120 euros a year per node, which over 3-4 year span adds up. It's nothing extreme, but it does somehow offset what is saved with cheapest solution.

 

My objections to "Ikea Helmer" and other bullshit farms (let's be honest, the second guy who did it here basically admitted publicly he did not even know he needs licences, he simply cloned single licence pack he had, and by 90perc. that was pirated too), is only the upfront cost seems smaller and thus look attractive and in eyes of many, like a clear winner. But it was never so clearcut, but no one cared anyway. It was quickly became the "people's choice".

If all the little things (direct costs:Licences/Electricity, indirect: managability, noise/heat, redudancy for future use,..) start to add up together, it does became disadvantage that seems completely ignored by most.

 

Interesting the electricity is so cheap in US. Did not know that.

[braddewald]

So is the consensus that the difference is negligible? I currently have 7 nodes:

 

Node 1: Xeon E5520

Node 2: Xeon E5520

Node 3: Dual E5-2620's

Node 4: FX 8350

Node 5: FX 8350

Node 6: FX 8350

Node 7: FX 8350

 

I want to at least add three more nodes, since I've already purchased 10 licenses. My thoughts were to go with higher-end processors, since to get the same power from lower-end processors I would need more of them, which would put me over the 10 licenses I have now and I would need to start adding $$$ for a license to the cost of each new node. Does that better explain my dilemma? I'd really appreciate any advice.

[Dimitris Tolios]

Impressive, thanks.

 

I still don't completely share the conclusion though. Your "heavy" scenario is basically 8hours a day "full" rendering. That's not any extreme and I would consider to be rather average and much more norm. Even with the small farm solution (6x4770/8350,4x3930) you can save up to 100-120 euros a year per node, which over 3-4 year span adds up. It's nothing extreme, but it does somehow offset what is saved with cheapest solution.

 

My objections to "Ikea Helmer" and other bullshit farms (let's be honest, the second guy who did it here basically admitted publicly he did not even know he needs licences, he simply cloned single licence pack he had, and by 90perc. that was pirated too), is only the upfront cost seems smaller and thus look attractive and in eyes of many, like a clear winner. But it was never so clearcut, but no one cared anyway. It was quickly became the "people's choice".

If all the little things (direct costs:Licences/Electricity, indirect: managability, noise/heat, redudancy for future use,..) start to add up together, it does became disadvantage that seems completely ignored by most.

 

Interesting the electricity is so cheap in US. Did not know that.

 

If you add 1 node, effectively"doubling" your rendering power with just $500-600, its a big thing.

That's realistically happening with the 2x "free" node licenses included in the initial investment for VRay 3.0 (or 2.0 for other packages). For a student, starting professional or low-wage professional in countries other than NA/EU, this is not a small deal.

 

Sure, as you keep adding nodes, your actual cost per node climbs above $900 due to licensing, but again, this is still cheaper than what a notably more powerful machine would cost - i.e. either 1P or 2P s2011.

 

Electricity costs are an important factor, but we should not just "run" that farm for the sake of it.

8Hr a day at full tilt, EVERY DAY, is a lot. You must be lucky to have that much work, most people don't.

 

Also, do not forget that solutions like the "Helmer" are not just cost efficient, but also fun. People like building things themselves.

Of course the first people that started those, didn't have a 2-node limitation, or weren't' working with Vray or even Widnows to begin with, so licensing software scenarios are task-specific. Or depending on your ethics - but I won't elaborate on that.

 

Sure, you can get better and faster things, but initial investment is usually much higher, so I do believe there is room for all those solutions. Even the not so elegant ones.

 

So is the consensus that the difference is negligible? I currently have 7 nodes:

 

Node 1: Xeon E5520

Node 2: Xeon E5520

Node 3: Dual E5-2620's

Node 4: FX 8350

Node 5: FX 8350

Node 6: FX 8350

Node 7: FX 8350

 

I want to at least add three more nodes, since I've already purchased 10 licenses. My thoughts were to go with higher-end processors, since to get the same power from lower-end processors I would need more of them, which would put me over the 10 licenses I have now and I would need to start adding $$$ for a license to the cost of each new node. Does that better explain my dilemma? I'd really appreciate any advice.

 

I don't know about the "consensus", but my take on it is that kwh savings alone are not enough to outweigh the initial investment cost, unless you are really putting the nodes to work a lot every day, and you live in countries where electricity is very expensive.

 

When you have lots of work, this farm is making you a good profit, if times are slow, you can power it down and spend nothing over the initial investment - something you cannot do for the ramped up cost for getting better hardware.

 

When you are battling against "a given number" of licenses, the faster node will always "win" as far as speed goes. 10 licenses are "enough" and I definitely see room for improvement, even with your existing hardware.

 

Since s2011 platform's value really seems to be peaking after 4-5 nodes, I could easily see you depleting your available licenses adding 4930K or s2011-3 upcoming i7s and eventually retiring your old Xeons, replacing them with more s2011/s2011 in the future - if money is a limiting factor.

 

If you think you could utilize those extra licenses "right now" and you are expecting return of investment within the next year or two, then stop thinking 1P i7s, and switch gears to build 1-2 2P 8C/10C Xeon nodes that will make the best use of those remaining licenses, and perhaps give you a reason to retire your older hardware all-together to release some Vray and Win licenses.

Edited July 8, 2014 by dtolios

[RyderSK]

I think it's worth to compile your messages here into sticky post. Might help lot of users decide using some unbiased outlook.

 

Regarding:

 

but we should not just "run" that farm for the shake of it.[/Quote]

 

I think it's rather this case :- ) With enough render power, life is easier, you can manage so much more tests, and those tests are in good quality and high resolution. The farm then basically never stops, there is always some use for it. I am very non-linear wysiwyg person, there is just so much I can go without running for preview. Vray3/Corona progressive modes made this process also much more enjoyable than used to be the norm.

[Dimitris Tolios]

So wasteful =)

 

But again, 8Hr a day - every day - is way above average.

A SoHo studio farm will be idling most of the time, and idling "mileage" performance is nearly identical.

 

As for the sticky post, maybe I will compile the thing again, perhaps with 2P numbers (do you have a kill-a-watt to contribute?) and make it more coherent. Then post it in my blog too, as it doesn't see much action lately.

 

Sidenote: used market is going nuts with GTX Titans being out of stock...sold my year old Titan (I) 6GB for $400 more than what I've paid for it (used)...that's full retail.

Think I will be downgrading to a 780 3GB (still have my waterblock and it is the only fully compatible). Used, that's less than $400.

I did not end up using more than 3GB of VRam doing anything but toying around in games and just pushing VRay RT for the sake of it (like you do with your nodes )

 

Right now I'm rocking a 750Ti.

[RyderSK]

Interesting, that made me check local e-shops, but everything seems in order here. Opened(unused) Titans (vanilla) start at 600 euros even (Gainward...always much cheaper than other models, not sure why that is), New about 750 euros, 900 euros for new TitanBlack. Is that a US a thing ? Or maybe these cards are just completely unaffordable in economy like in my country, so they just idle in stocks here.

[lateralustool]

I've compared

 

Render Node A = 4 core 4790K 4,0 GHz (perm. unit: 10144 (each core 2536)) and

Render Node B = 6 core 4930K 3,4 GHz (perm. unit: 12012 (each core 2002)).

 

Price of 4790K 395$ and

price of 4930K 637$ (in Poland).

 

Total cost with soft of RN 4790K: 1341$

Total cost with soft of RN 4930K: 1788$

 

For 6 render nodes i get: 4790K (24 cores total) = 60864 perf units Price: 8051$ || 4930K (36 cores total) = 72072 perf units Price: 10727$

For 5 render nodes i get: 4930K (30 cores total) = 60060 perf units Price: 8940$ || 4790K (20 cores total) = 50720 perf units Price: 6709$

For 4 render nodes i get: 4930K (24 cores total) = 48048 perf units Price: 7151$ || 4790K (16 cores total) = 40576 perf units Price: 5367$

 

What do You think? Did I made a mistake with this 4790K = perf unit 10144 vs 4930K = perf unit 12012. Or Cost of one render node is too high?

 

From this info it looks like it is better to buy 6x 4790K RN than 5x 4930K RN. (same performance, less price) (But probably more electricity)

Edited August 16, 2014 by lateralustool

[DanGrover]

This licensing cost is actually one of the big things stopping us moving to VR3. Almost all of our rendering is done in the cloud, and we have, on occasion, had 400 render nodes cooking away at our frames. That'd cost us just shy of £50k.