Jump to content

Cheap 16core Xeon Node - Pros/Cons?


Recommended Posts

  • 3 weeks later...
  • Replies 64
  • Created
  • Last Reply

Top Posters In This Topic

Well I've taken the plunge and ordered myself a couple of E5 2670 v2s off eBay £65 each. I was holding back because of the cost of the motherboards but a few new cheap ones have popped up in Germany £250, which I though was worth the gamble. I plan to be using this a my main workstation eventually as I'm in the process of building 6x 6700k systems for my render farm today. So I'll report back with some comparative speed tests.

Link to comment
Share on other sites

I'm in the process of building 6x 6700k systems for my render farm today.

So you're using the two low clocked 8-core Xeons (i think you mean 2670 v1?) for your workstation and 6 higher clocked quad cores for the farm? Why?!? I think for the money spend on the 6700K you could build maybe 5-6 E5-2670 systems, each of them more than twice as fast as the 6700k sys.

Edited by numerobis
Link to comment
Share on other sites

So you're using the two low clocked 8-core Xeons (i think you mean 2760 v1?) for your workstation and 6 higher clocked quad cores for the farm? Why?!? I think for the money spend on the 6700K you could build maybe 5-6 E5-2760 systems, each of them more than twice as fast as the 6700k sys.

 

No, I mean V2.. http://ark.intel.com/products/75275/Intel-Xeon-Processor-E5-2670-v2-25M-Cache-2_50-GHz

 

I had considered the idea of using these chips for my whole rig, but there are significant cost savings to be had for me using the 6700Ks. I already have all my DDR3 ram, so I'm re-using this with the 6700s, with the XEONs I would have to upgrade all 6 of my PSUs, which would add further cost, I have 2U rack cases, so I would also potentially have cooling issues using dual xeons as well. Additionally, the chips and specifically the motherboards aren't readily available from reliable sources, so there is quite a gamble there, which I'm not prepared to take over my whole render farm. I'm thinking the 6700 will run cooler too than the Xeons and use less power, which is a concern when you have 6 of them next door to your bedroom running 24/7.

 

I'm using this for my workstation as a bit of fun really.

Link to comment
Share on other sites

I thought you meant the 2670 V1 because you're posting in the 2670 V1 thread...

 

So what is your current hardware if you plan to upgrade the nodes to a quad core? What PSU do you have?

I mean, "less power", "cooler" and cheaper is relative... you could build only 3 Xeon systems instead of 6 6700K systems which should still be faster overall. The 2670 v1 is by far the cheapest option for render nodes at the moment - especially if you're upgrading an old system where you can re-use some components. You could even try to re-use your RAM. I recently built 4 2670 v1 systems myself and re-used 2x 16GB DDR3 of my i7 2700K nodes in one sys. (The other machines got new 8x8GB DDR3 reg from ebay for ~150€/sys).

 

And why are you chosing a Haswell Xeon for the WS? Is it used hardware?

Edited by numerobis
Link to comment
Share on other sites

So I'm most of the way into this build now and discovered a few pitfalls. The main one being that finding a case that fits an SSI EEB motherbo has been both difficult and expensive. To be fair I had some very specific restrictions on the width of the tower as I have to fit the case into an area in my desk, but none the less I ended up using a Silverstone TJ09B, which was about £150.

 

The other thing was the PSU which Jens used would not work with my Mobo without an extension lead. The CPUs individual power connections are quite far apart on the ASrock EP2C602 so I had to get an extension lead. I'm told the EVGA 750 gold PSU has individual outputs for multi CPUs so if I built again I'd go for one of those instead.

 

So CPU £130 / Mobo £250 / PSU £80 / Case £150 / SSDs £160 / RAM £307 / £80 fans it's not been a cheap build!

Link to comment
Share on other sites

SSI EEB fits into pretty much every e-atx case. Few hole don't line-up, but that has never been issue for me, neither when mounting memory modules or gigantic cards (titan-x).

Temjin is really good case though, just needlessly more expensive compare to something like Fractal ( that aluminium.. )

 

Also, seems you bought rather expensive memory... I was able to source 64GB of used ECC DDR3 memory for around 120 +/- euros, 3 times cheaper that that. Your quoted price is adequate for new DDR4 ECC.

 

If this kind of build (without gpu) can't be sourced under +/- 1100 euros, I would not bother, and go straight into v3/v4 adventure.

Link to comment
Share on other sites

So I'm most of the way into this build now and discovered a few pitfalls. The main one being that finding a case that fits an SSI EEB motherbo has been both difficult and expensive. To be fair I had some very specific restrictions on the width of the tower as I have to fit the case into an area in my desk, but none the less I ended up using a Silverstone TJ09B, which was about £150.

 

The other thing was the PSU which Jens used would not work with my Mobo without an extension lead. The CPUs individual power connections are quite far apart on the ASrock EP2C602 so I had to get an extension lead. I'm told the EVGA 750 gold PSU has individual outputs for multi CPUs so if I built again I'd go for one of those instead.

 

So CPU £130 / Mobo £250 / PSU £80 / Case £150 / SSDs £160 / RAM £307 / £80 fans it's not been a cheap build!

 

What brand of memory are you using? Did you go with the G.Skill that was mentioned in the "32-thread xeon monster" article? I've gone through two ASRock mother boards now and have received RMA #'s on both after lengthy trouble shooting with ASRock support. Even thought the MB's website page at ASRock says it will use the non-ECC type, ASRock support agrees with me either the RAM just doesn't work, for whatever reason, or I am the most unlucky person int he world and have received two different models of perfectly compatible MB's that have defective RAM and/ or CPU sockets.

 

I'm very tempted to just buy some used sticks as Juraj suggests with his own build just to have on hand when I get the third MB.

Link to comment
Share on other sites

  • 2 weeks later...
  • 2 months later...

Something funny for me to tell...

 

I build myself a similar configuration. At the beginning I was wondering about temps which were rising to 96°C after a few minutes. I did not go further than that and stopped the test render.

Well it turns out the cpu fans work a lot better if you remove the cardboard that has been put in there to protect them during transport.....

 

Bye, Jan

Link to comment
Share on other sites

Something funny for me to tell...

 

I build myself a similar configuration. At the beginning I was wondering about temps which were rising to 96°C after a few minutes. I did not go further than that and stopped the test render.

Well it turns out the cpu fans work a lot better if you remove the cardboard that has been put in there to protect them during transport.....

 

Bye, Jan

 

oh dear... thanks for the laugh though Jan! I hope you'll enjoy the new node now ;)

Link to comment
Share on other sites

  • 3 months later...

Hey Jens,

Super-useful thread and even more useful blog post. I'm planning to follow your example at building my node. I'm an architect basicly and visualiser on the side but this part of my business is taking over for better or worse, and I thought raising my game by setting up a slave unit.

A couple of questions related to your process..

a) You mention using network paths to avoid problems with the node finding your files. Would you recommend using an external NAS on your switch for this reason? Would that slow things down you think?

b) You mention your Homegroup runs on windows 10. In your opinion having Win 7 ultimate on one or both machines (workstation and node) would be recommended or not?

 

Again thanks for sharing the knowledge!:)

 

Alex

Link to comment
Share on other sites

a) You mention using network paths to avoid problems with the node finding your files. Would you recommend using an external NAS on your switch for this reason? Would that slow things down you think?

b) You mention your Homegroup runs on windows 10. In your opinion having Win 7 ultimate on one or both machines (workstation and node) would be recommended or not?

 

Hi Alex, glad you found it useful :)

 

a) I'm slowly moving my stuff over on my NAS, but still have my main stuff on the workstation's network mapped drive. I haven't seen any slow downs with having stuff on the NAS and I don't think you will run into any problems there when it's just you accessing the NAS. Without knowing your NAS/server type, things might start to slow down, if you expand and have many connections trying to fetch data from your NAS at the same time. For now you should be good.

- As a bonus info Corona Renderer has an option in the coming 1.6 version that will send missing assets to the node (unless they are on an available network drive).

 

b) I can't answer this for sure, but sooner or later you'll be forced to update to win10 anyways. So I would just go with win10 on both :)

Link to comment
Share on other sites

  • 4 years later...

Create an account or sign in to comment

You need to be a member in order to leave a comment

Create an account

Sign up for a new account in our community. It's easy!

Register a new account

Sign in

Already have an account? Sign in here.

Sign In Now
 Share


×
×
  • Create New...