Local vs Network I/O

[JimTour]

Hello folks - I have a simple, perhaps deceptively simple, question for you. As you work with CAD, 3dsMax, Mental Ray, etc., what is your normal way of working with files? Assuming that most of you are working in a networked environment, do you open a project file on a networked disk, work with it and save it back to that disk? Or do you copy the networked file to a local disk, work with it, then save it back to the network disk?

 

I am a sort of part/part/part-time "sysadmin" for an art/architecture studio and we recently ran up against the 10-connection limit in XP Pro. This shook the purse strings loose and we are having a huge debate on how to best upgrade our LAN. The debate has taken on the form of Local Disk Advocates vs. Network-Centric Advocates. Not being an artist/architect myself, it would help tremendously to get a few outside, qualified opinions. I would have thought that folks using such intensive apps as 3dsMax, etc., would insist on working locally and bringing all the power of their desktop machines to bear.. or does opening a file on a network disk not entail the performance hit one would at first think? Perhaps it's not a question of local bus i/o vs. "network i/o"?

 

Thanks for any replies,

Jim Tour

[mbowers]

It all depends on your office work flow. If you have people who manage their own projects and fly solo, then perhaps it is a better solution for your network head-ache. Keep a good habit of loading the most current to the network folder at the end of each day. If your office is more team-collaborated, then a single file on the server would be the best bet.

 

We use a server based filing system. Project files stay in one spot, they don't get copied to a local hard drive. This happens for a number of reasons:

 

1) Multiple versions of a file can exist in many different places, resulting in incorrect information

 

2) If the "seed" file is moved or copied, it will lose it's place as reference for other files (thinking about linked drawings...autocad, microstation, max/viz)

 

3) Having one file does increase the risk of running corrupt but we (and you should too) have a robust backup system in place and that keeps our IT manager headache-free.

 

If your running REVIT, you only work with one file, but you assign privileges so multiple team members can work on one project.

[alfienoakes]

Client & server everytime time once you get over 10 nodes.

 

Many reasons for it, not least backups, file integrity, simplicity, printing, security and fault tolerance to name just a few.

 

The only time you may have I/O issues, is when you save back to the network, but most networks run at least 100mbs now, or should do, so should not really be an issue.

 

Peer to peer is fine for home or small office. Copying files between local and networked drives etc, WILL end up in tears due to one of the above.

[JimTour]

Thank you mbowers for good insights and advice... Our backup has been pretty shoddy and we will certainly move to correct that...

 

Both of you (mbowers and alfienoakes) sure are enamored of working straight off network disk-- but I've read so much advising the opposite, especially for intensive apps like 3dsMax and Maya. As a random example, how about the following from Adobe:

Saving or opening files across a network or off of removable media involves many variables. Because of this, some problems (for example, damaged files, denied access, or slow performance) occur more frequently when you work from a network than they would if you worked off a local hard disk. For example, Adobe Photoshop may return one of the following errors:

-- "Could not complete this operation because this file is in an unknown format."

-- "This document has been damaged by disk error. The most likely cause of this error is a defective disk drive, SCSI, or SCSI termination. Pixels may be damaged. Open anyway?" (When you click OK, Photoshop may not open the file.)

However, problems opening or saving Photoshop files across a network or off of removable media may not happen immediately and may be intermittent. Adobe Technical Support only supports using Photoshop on a local hard disk because of the difficulty to re-create or accurately identify network- and peripheral-configuration problems.

 

That's at:

http://kb.adobe.com/selfservice/viewContent.do?externalId=322391&sliceId=2

 

And it really doesn't matter to you that your super fast disk subsystem is just sitting there while you load up from network? On huge files in particular, don't you find that chunks are being thrown into and out of RAM, causing wait states while data is read from remote disk, a wait that would not occur if it was local RAM talking with local disk?... Also, aren't you always using file level access and never using block level access with your arrangements, despite block level being significantly faster? ( see http://www.answers.com/topic/block-level?cat=technology ).

 

Could it be that the type of work you do is not "heavy" enough to see these speed differences? Actually, our Studio is probably on the "lighter" end also, compared to shops doing full animation, week-long renders, etc.,. So maybe "network I/O" would suit us fine... Thx-- JT

[BrianKitts]

For matt's first reason of multiple files in multiple places, working locally is absolutlely out of the question in an office atmosphere. You are just setting yourself up for trouble. Someone doesn't put a file back when done or someone pulls a file to work on when someone else is editing it. you end up with multiple files in multiple places, next thing you know you end up under a deadline, with updates in multiple files in multiple locations you will pull your hair out trying to get it all worked out then you'll most likely shoot your network admin for setting the system up like that and learn to NEVER work like that again.

 

 

In my five years of rendering, I've never had a problem with saving large scenes and files over a network. That is with the exception of a server going down which is actually less likely to happen then a rendering workstation going down. You local machine grinds away all day crunching your ram, and well crashes happen occassionally..... a server sits idle half the time till you start saving files, it runs less processes, and is in the end less prone to faults.

 

 

Do it right, invest in a gigabit network and all will be well.

[alfienoakes]

Do it right, invest in a gigabit network and all will be well.

 

Amen...

 

As Brian says, I have NEVER seen any of those problems you have listed, and thats in about 15 years of working on a network.. Not to say those problems dont though.. just unlikely that with a well configured network, you probably wont have to worry about them.

[mikedeerf]

I agree with Andy and Brian. Every office I've worked at has used a central server for all project files. One of those offices had a sort of hybrid setup - when you wanted to work on a file, they had a custom-written DOS program to "check out" the file to your local disk. When you were done working, you had to check it back in. Anyone else trying to access that file could see that you had it checked out - just like a library catalog system. Anyway, it was a pain in the butt sometimes, but it's the ONLY way I would ever consider working from a local drive.

 

Last month we had a guy go on vacation. A client requested a set of construction specs he was working on. 3 of us spent a good chunk of a day searching the network for the files... long story short, he had been working from his hard drive. Under "My Documents" - where only his login name or an administrator can access them. Our primary network admin was also on vacation. So we had to track down a secondary person with administrative rights to login to search for the files we needed.

 

And that's just one instance. I've seen lots of others (separate file versions, file-overwriting battles, etc.) but Matt, Brian and Andy already covered those.

 

As for heavy workload, I've used AutoCAD R14-2008, including Land Desktop, Architectural Desktop, Civil 3D, and Map 3D; On the graphics side: Vue 5 Infinite, Viz 2008, and Photoshop 6. Company-wide people run Microstation, structural analysis software, hydrology stuff, etc.

We keep all of our files on central servers and I've never had a problem. Sometimes a really huge (200MB or so) Photoshop file will take a while to load, but once it's loaded, Photoshop uses the scratch file on the local hard disk; it's not noticeably slower than working on it at home saved on the local disk.

 

Just my 2 cents. There is one caveat: don't skimp on the server or on a backup solution. We backup the entire server every night. Week-end backup tapes get labelled and saved for 5 years or so; daily tapes overwrite each other (i.e. we have 4 tapes labelled M-Th; when Monday runs, it overwrites the previous Monday) - these are for short-term (oops, I just deleted a file) purposes.

A central server is all your eggs in one basket. It IS a better solution IMHO, but make sure it's a good basket.

 

Best of Luck

[JimTour]

Alright then, I must bow to the voices of experience here, though it goes deeply against the grain. That little voice inside gets big and shouts, "They are putting collaboration/backups over performance! Nothing should be over performance!" But I re-read what you're saying (for instance, alfienoakes saying he's never had a problem working from network disk in 15 years) and I think, "Well you just can't argue with that"... I guess part of the answer is that our high-performance desktops *are* at work with a file loaded over the network. Our 4 or 5 gigs of RAM are in there doing big time buffering/chunking and, as mikedeerf mentioned, Photoshop and many other apps are using local disk for scratch files... Even so, some important operations certainly are happening slower due to this "network i/o" vs "local bus i/o" (like the initial file read, additional file reads for huge files that can't be swallowed whole, file writes-- and all this with other files open and other progs running) . But obviously this never caused enough of a performance hit to bother you guys at all

 

So-- thanks again all (u2 BKittsARC), and happy networking. -- JT