Help - First Build with GPU rendering in mind

[lidijag]

Hi,

 

I'm relatively inexperienced, but managed to make a list of components. At first I wanted to buy two computers and in time create my own little render farm, but decided to invest in an overall good machine which I can use properly straight away and after a while get a few more graphics cards for GPU-based rendering. I will be using 3ds Max, Zbrush, RealFlow, After Effects. It would be great if I could get a few opinions on it, in terms of major component compatibility. Money is not really an issue, but I wouldn't want to throw it away on stuff I don't really need.

 

CPU: i-4790k

GC: MSI GeForce GTX 970 Gaming 4G, 4GB GDDR5,

RAM: 16GB HyperX FURY Black 1866MHz DDR3 Non-ECC CL10 DIMM,

SSD: Kingston SSD 240GB Hyper X 3K Series,

Case: Chieftec BH-01B-U3-OP Miditower Case Bravo,

PSU: Corsair CP-9020051-EU VS Series 650 Watt,

2 Monitors: ASUS VS239H-P 23-Inch Full-HD

and I need a motherboard that can support 3 or 4 graphics cards.

 

Apologies for any mistakes in formatting this thread (or blatant display of ignorance on my part). Thanks!

[Nikolaos M]

Since money is not an issue, let me write down some thoughts. Choosing a Z97 platform with a 4790K would be more than enough for your basic workflow since it's the fastest cpu in terms of per core performance right now. But, if you plan to put 3 or 4 gpus on a Z97 motherboard for rendering, you must keep in mind that you will have to choose one with plx chips onboard, because 4790K has only 16 pcie lanes natively. This means that you'll have to pick a pretty expensive Z97 mobo in order to have at least x8 (PCIe 3.0) lanes assigned to each gpu and not physical ones, but artificially multiplied with the plx chip. In your place I would consider buying a X99 motherboard with a 5930K in order to have 40 pcie lanes at hand exclusively out of the cpu. THat would cost ~300€ more in total compared to a Z97 build.

 

Something else. There is no way a 650W psu, and especially the Corsair VS650 (VS=Value Series), could handle a multi gpu setup like the one above. At full load, each gpu (gtx970) would aprox. pull 150-160W and you must add the consumption of the cpu to estimate the total consumption. I would choose at least a 1000W psu (1200W would be ideal, in my opinion). I know this choice might look exaggerated, but I always want a big headroom in my psu's wattage. Keeping the consumption somewhere between 50-60% of the total wattage of the psu, means less heat and less noise.

 

One more thing is the case selection. Imo, pick a full tower case with a good airflow. 4 gpus running side by side is an extreme setup and needs space and good cooling to run properly.

 

As for the monitors, are you sure you need 2x23" and not 1x27" (or bigger)?

 

And last, the gpus. As other members have already mentioned in other threads, gpu rendering for complex models need the biggest VRAM available. You should probably wait for the 8gb versions of gtx 970/980. 4gb could narrow your choices in the future.

 

I also made a part-picking from geizhals.de, based in a 1150 platform. Take a look if you want:

 

1 x SanDisk Extreme PRO 240GB, SATA 6Gb/s (SDSSDXPS-240G-G25)

1 x Intel Core i7-4790K, 4x 4.00GHz, boxed (BX80646I74790K)

1 x G.Skill TridentX DIMM Kit 16GB, DDR3-2400, CL10-12-12-31 (F3-2400C10D-16GTX)

1 x ASUS Z97-WS (90SB04E0-M0EAY0)

1 x Noctua NH-U14S

1 x Phanteks Enthoo Primo weiß mit Sichtfenster (PH-ES813P_WT)

1 x Super Flower Leadex Gold 1300W ATX 2.3 (SF-1300F14MG)

Edited April 2, 2015 by nikolaosm

[lidijag]

Thanks, Nikolaos. I'll definitely check out the X99 motherboard 5930K combo. Better PSU sounds logical. I can't wait for the 8GB GTX, so I'll probably go with the 4GB one for now. I'm used to working with 2 monitors. Super thankful!

[Dimitris Tolios]

Just my 2cent: even if you were going the s2011-3 way, I would not care for the 40 lanes the 5930K offers (over say the 28 lanes the 5820K offers). It is a gimmick "penalty", and almost irrelevant to GPGPU performance, as the rate at which data get in and out the GPUs is more than enough in any case.

 

Even with 4-way SLI & gaming and top-top of the line GPUs, where there is definitely a more active exchange of data back and forth with the CPU, you barely see a difference between 5820K and 5930K. Spend the price difference on something more meaningful.

 

I won't elaborate on the s1150, but the idea is pretty much the same: the PCI 3.0 protocol is very very fast, so it is hard to saturate it, and when you do, it is just a small part of what determines the performance as experienced on the user's end. For GPGPU, you might experience differences in the range of a second or fractions of when loading the assets for the 1st time to the GPUs, initiating a rendering session. After that, each GPU exchanges data slowly with the CPU/App as it is mostly doing its own thing.

[Nikolaos M]

Does 5820K support quad sli? I thought it only supported 3-way. I know that this is irrelevant with gpu rendering, of course, but I'd like to know in any case. I guess that 4 gpus for rendering purposes would work at x8/x8/x8/x4 with a 5820K, is that right, Dimitris?

[Dimitris Tolios]

The CPU doesn't "support" SLI.

Motherboards and Cards do.

 

And cards care for an X amount of bandwidth, depending on what they are doing. Some of this bandwidth is for CPU GPU(s) communication, and some is for GPU GPU communicating to each other (that is one of the things the SLI bridge was adding to the mix, as when SLI was first introduced, PCIe protocal was not as "broad", while now we can have bridge-less SLI/CF without killing the CPU GPU communication performance.

 

The chipset/mobo needs to be "aware" of those GPUGPU direct links, and should allow for enough bandwidth to facilitate them.

Should those be facilitated through the CPU provided lanes, or a PLX chip etc, doesn't really matter much for them. As long as there is enough bandwidth, things will work fine.

 

And that is exactly the point: only massively fast GPUs in Tri or quad SLI/CF can really saturate PCIe 3.0 - and that is occasionally, and on the high-end of things. E.g. you are using Quad-SLI, with GTX Titan or 980s, and you are running a benchmark 1080/1440p, which makes it really a CPU bound scenario. You would hit - say - 120 fps with 28 lanes, but you do 130 fps with 40 lanes.

 

It is a similar scenario where you would see more fps running DDR4 2666 vs. DDR4 2133, i.e. the usual suspects that bottleneck the graphic workflow (GPU / API) are not limiting you, so it is barely a scenario of the CPU pre-drawing as many frames as possible and feeding them to all those GPUs, which spit out the "resulting" frames faster than they can get the next.

 

If the task is more complex, e.g. we are taking 3K / 4K resolutions, in most scenarios the GPUs become the bottleneck as those cannot draw the final frame faster than what the CPU/Main Ram combo draws the "Schematics" of the following frames to be displayed, but then we still might have a PCIe lane issue as the coordination between the GPUs now becomes the pressing issue. But provided you have enough bandwidth, even through a PLX, you could see a 4790K CPU for example out-performing a s2011-3 chip, given all were running @ stock speeds, as was the case even with the 3770K vs. a 3930K.

 

For GPGPU, none of these works this way, as what happens in a nutshell, is each GPU loads up all the assets from the get-go as the active-shade window launches, and each card then starts resolving the rendered images progressively and to a large extend "independently".

There is no GPU GPU communication, and the calls from the CPU to each of the GPUs for the results on a pixel here and there are not capable of saturating the 16 or 28 lanes the cheaper CPUs have.

Edited April 3, 2015 by dtolios

[lidijag]

Thanks, guys!

[RyderSK]

Money is not really an issue.

 

 

I need a motherboard that can support 3 or 4 graphics cards.

 

I can't wait for the 8GB GTX, so I'll probably go with the 4GB one for now

 

 

What are you going to do with such GPU(s) with all their speed, if your scene won't fit ?

 

GPU renderers became quite efficient in managing their limited memory compared to CPU which have almost unlimited capacity.

They store textures on system memory (Redshift), or compress it (Octane), they can render out of core ("memory cycle", Redshift),etc..

 

But in the end, 4GB is going to be 4GB. You can't fit any serious scene inside it, and the moment you will need it, your GPUs will be useless.

 

It's ok if you plan to use your 970/4GB as placeholder, and later for display only (or sell it) if you intent to buy 980/8GBs when it comes (Q2/early Q3 probably). Also, if you plan to use such array of GPUs, 650W is not going to cut it. These GPUs reach 280W in full draw each (factory overclocked versions even more)

With such load, you're looking at 1000+ W , Gold/Platinum grade, and not cheapest bronze option on the market.

 

 

In all honestly, if I were to build GPU rig today for serious commercial visualization, I would set aside roughly 3500 euros, and start with 2x Titan-X, with their 12GB of Vram, something far more crucial then speed. Otherwise I wouldn't bother at all.

[lidijag]

Yes, I'd use the 970 as a placeholder until I can buy 980/8GB. The thing is, I am not nearly good or knowledgeable enough to fully utilize such an expensive machine at the moment. There is no point in buying a 3500e worth of equipment to have it just sit there. I do need a CPU/Mobo setup that won't create bottlenecks when, in a year or two, I decide to switch over to GPU rendering. Right now I am wondering about 5820k and it's low clock speed compared to 4790k. I understand that 3D rendering and video editing are very dependent on the number of cores and use hyper-threading. The price difference between the two is not that big.

[RyderSK]

Ignore baseclock and compare turboclock for both single and all-cores, alternatively, they both overclock fairly similarly (into high 4.4+ Ghz).

 

Knowledge has nothing with utilization of hardware, either you need, or don't need performance. You might not even need GPU rendering at all, and it's far more important to sort those priorities before planning or buying hardware. You originally planned Quad-GPU setup, but if I am right, you don't even use currently any GPU renderer extensively, correct ?

 

As Dimitris outlined, neither platform (1150 vs 2011) creates any bottleneck. Their difference is in your planned utilization of CPU performance.

5820K is slightly more future-proof solution, and generally better deal if you can spare the additional budget that 2011 board and DDR4 cost brings.

 

1) Set priorities

2) Set budget

3) Plan PC parts.

 

Not oppositely.