Sharing workflow experiences to save my mental health/employment

[Eliot Blenkarne]

Hi all,

I'm hoping to get a sense of clarity about how other studios operate a reliable-ish workflow in 2021, from a software + hardware standpoint.

The reason I ask is due to the obscene amount of time myself and my team are wasting on software crashes/freezes, meaning jobs are dragging on, we're losing money and working extremely long days to make up for it. I estimate 50% of my standard working day is wasted re-opening files or waiting for max to respond. I'm in a western country with decent pay figures, so the $$$ will be mounting quickly. I cannot imagine that this is the standard way to work, or that dedicated viz companies operate like this.

Our software package for modelling/rendering is 3dsMax 2021 + Corona 6 (soon to be 7). We use the typical array of plugins; Itoo package, Sini etc. We use Project Manager by K Studio for managing assets in our libraries. Our individual machines are dual xeon/256gb RAM/dual SSD/3090 weapons that are likely way overpowered. 

We are under ongoing pressure due to the arrival of Enscape in our studio, and to be frank it feels very justified given how painfully inflexible the workflow seems to be becoming from our side of the coin. I imagine there are a few variables in play here, and I certainly do not profess to being a technical power user, but I am very prepared to learn and listen to those with far more wisdom and experience than I. So to get the ball rolling, I might suggest some areas to focus on;
 

1. Have others had these same issues, and if you've started to fix them, how did you do so?
2. Any common knowledge in the viz world of crap plugins or the like going around that might cause this?
3. We are a large-ish architecture firm (350+ employees), so we are dependent on everything feeding through an established network - are there others in a similar situation who have found a specific way to work more efficiently?
4. Any other suggestions around workflow - it seems to make very little difference as to how well a file is managed in terms of layers/xrefs, but I am open to hearing anything at this point. As basic as it can possible be through to more advanced, gimmie anything! I want to go to our IT department somewhat prepared to get things moving if that is what is needed.
5. We have one "old head" who runs our team - the rest of us are 2 - 10 years in the game, so I'm wondering if there is a knowledge issue?

Thanks all.

[Jose Negrete]

Hi Eliot,

We have a mix of software use - 3DS Max + Corona/Vray, Maya, Twinmotion, Lumion, Blender 3D, UE, Adobe suite, etc. Similar systems except we have 3990x threadripper for CPUs. I think the Xeon's single core speed is low enough to cause a bottleneck with most Max processes being single threaded. Additionally, I would check what type of network speed you have. If you're loading several xref from network on a slow network that could be a headache. We upgraded to gigabit network and that improved read/write speeds.

Even w/ the faster network and workstations. When using max, I break up scene to xref files then when those get referenced in set them to not visible, ignore animation, etc. 

[Scott Schroeder]

We do stadium size projects and have very little issue with Max outside of rare random crashes. Almost 100% of what you are describing is user driven or issues with network speed. We're on MSI laptops, so there is a lot less power than what you have and as I mentioned, almost zero max issues outside of random ones that you can never fully get rid of. I typically get less than 5 max crashes for an entire work week, even on one of our NBA arenas with 15,000+ seats and people, fully kitted out etc.

I've helped other studios in the firm chase down Max issues and they are usually related to

• Too much in the file for no real reason. Every bolt is turbosmoothed 15 times. The stronger the machine you develop your work on, the more you tend to throw at it but eventually you still hit the same bottlenecks. Using 8k+ textures that are 200mb+ just because.
• Rendering 8k, final quality preview images just to see a texture (ie wasting time watching buckets)
• Lousy file organization and management such as leaving everything on to navigate the scene and not having the ability to quickly turn off areas of the project that are not in the scene, be in using layers or selection sets or something else.
• IT not understanding that more cores is NOT better. You can have 10,000 cores but if each individual core is weak then you are going to have a rough time. You need a decent amount of cores but the most important factor is the speed of the individual core.
• Thinking that Max and rendering is a magical process.
• Using arch or BIM models as-is. Revit geometry can kill your scene if you are not careful. If you are using Revit geometry, do a test. Create a simple scene in Revit and compare that to the exact same scene created in Max. You may see a pretty large difference. A lot of what kills our Revit models is just how in the Revit to max process the verts turn into a jumbled mess of dots. This not only leads to longer load times, but also longer render times. You may not have the capacity to rebuild everything, but know what Revit geometry always needs to be rebuilt. In our case, it's always structure and railings that are explicitly rebuilt in Max with RailClone.
• Leaving garbage in the file. You open the file and you still get 100 missing texture warnings or missing plugin warnings.

[Eliot Blenkarne]

This is fantastic feedback, thank you guys so much. I had been campaigning for threadripper CPUs but got drowned out by the more-core group, so I will certainly be raising this with them. With that said, they're still 3.4GHz, so hardly "slow" I'd have thought. Otherwise, in my case at least, most of the points above are usually addressed, so I'll have a chat to IT about network speeds.

Scott - your candor is appreciated, hoping you could drill down on the "magical process" aspect?

[Scott Schroeder]

Sure thing. Often times, rendering is seen as something that wouldn't be out of place in Hogwarts. We see this quite often when troubleshooting Enscape/Lumion crashes and it turns out, the scene is too complicated for that software to handle and the response is almost always, "But I passed the benchmarks!" Which is true, the machine can do well at those software, but the user also needs to know when to limit complexity. This is no different for some max users or even IT. Sure, you have a monster computer to work on but you can always max it out. It's just that your max level and my max level are millions of polygons apart but we can both just as easily crash Max with a too complicated scene.

Out of curiously, do you have any sort of active anti-virus working on your machines or network? I've seen that cause major slowdown issues as it scans files over and over again when you load scenes. It's part of the never ending battle between good network security and slowing down daily processes. Also in regards to the network, what kind of servers are they running, those could be playing into the longer load times along with an internal network that needs to have the speeds upgraded. Think about your average Max scene with all of the textures, proxies, etc that need to be pulled over the network to load in your file. Doing that can certainly clog your network pretty quickly, especially if your render farm (if you have one) is on the same network. I've been known to drag the studio's network down a few notches if I load up a stadium full of animated proxies during the day and have that get loaded on the render farm at the same time, so I need to save that rendering for the night when no one is in the studio. Though the move to BIM360 has freed up a ton of network traffic that Revit used to use. 

I know there is a lot more in the CPU cores discussion in terms of CPU architecture and speeds. Most of what I've read about Xeon's is what Jose referred to in that they tend to get a little wonky when they are running single thread processes, which is where CPU's like the i9 or Ryzen start to pull ahead of the Xeon class of chip.

At the end of the day, certain files in Max will take a bit to open but if you have to wait more than 5 minutes for it to open/save then there is a bottleneck somewhere. Do you use "Compress on Save" in max? I've seen arguments for and against using that, but I've always been on the side of using it just to keep the file size smaller on the network.

Lastly, as I typed my first message about Max not crashing too often. Max crashed when I went back to work, just to show me who the boss was.

Edited November 2, 2021 by VelvetElvis

[Jose Negrete]

1 hour ago, VelvetElvis said:

Lastly, as I typed my first message about Max not crashing too often. Max crashed when I went back to work, just to show me who the boss was.

?
 

 

[neil poppleton]

wow seems a lot of issues. seems a long time since we have max crashing on us. we are max based using vray render engine and of course photoshop post work.

not sure what the secret is but keeping very clean organised files is a must. we use very little plugins as they continually update and change and cause issues, so unless its a must have plugin, we tend not to use.

with 350 employees the opportunity to pull in the power of the top 10% machines would be ideal.

from personal experience, learning from crashes and gaining such experience is probably the way out of the situation, getting a good understanding of why something crashes seems vital to going forward, or get hold of an experienced consultant or employee to work with

[Morne Erasmus]

1) Make sure your network is fast enough to handle all users and their large files. There's a big difference in 300 users using autocad vs 20 users using max.

2) Make sure your file server(s) can handle the traffic, and have the network bandwidth, processing power and ram to deal with it all

3) If your file server is in the cloud, make sure you have a VERY FAST broadband connection(s).

4) Use max automation, standardization  and central max management. In othe words setup startup scripts from a central location to adjust user settings across the company. This can include typical settings, scripts, menu items etc. That way everybody is using same settings. I've setup our scripts to reset and re-install itself each time max is launched (takes 2 seconds). That way if a user messes up his settings, it will be magically fixed next time he starts up max.

5) Make use of either/or/combination of: XREFS, Live links, Layers, Selection sets, Groups (try and use groups as a last resort). And then show/hide/disable things not needed. If you've added trees/plants, those are typically not needed to be seen constantly while you texture your building.

6) If your working from home in the cloud, avoid thousands of small files. Those typicaly kills your sync. For example if you're using Google Workspace as your file server, dont attempt Unreal or Unity projects direct from your Google Drive. Instead, copy Unreal/Unity projects locally and copy it back. This could also be problematic as you may forget. Instead, setup a version control like GIT or Perforce and that will sort Unreal/Unity files for you more efficiently.

7) Make sure your hardware is cooled efficiently. Overheating can quickly become the cause of major instabilities/crashes

 

Edited November 20, 2021 by Morne Erasmus

[Francisco Penaloza]

Hi everyone, a lot of great information here. not too much to add,  but just reiterate that as mentioned before I also believe that user input plays a big part on all this mess.

I am oltimes on this industry, when I started our machines could barely do anything we want them to do (3Gb trick was just magic LOL) nowadays since technology and hardware has advanced so much, we all got lazy and just rely on the super power machine to take care of everything. This apply to REVIT, Rhino, and any 3D app, lazy modeling create problems.

Regarding network we supposed to have top of the line Gigabit connection, but even with that, time ago we decided to create our own server, totally separated from the Architects and designers. As @VelvetElvis mentioned every time we do an animation we suck up as much bandwidth as the network give us. Using our own server, have some issues but overall it really improved our transferring and network processing time. Also Backburner doesn't fail as much as before.

We proxy the hell out of everything, using parametric tools such Raiclone and Forrest also help us to maintain the scene flexible. Regarding software updates, we wait until Digital Practice force us to move to a new version, I don't trust any brand new software. We also rely a lot on scripts and plugins so if it ain't broke, we don't change it.

I am trying to move away from Adobe circus, have very bad experience with their latest updates, been using Davinci resolve and Affinity Photo and man the performance difference is just phenomenal.

Most of our designers and Architect uses Enscape or Lumion for their renderings and presentations, with that when we receive their files all meshes and textures are just a jungle. I have a few actions on Photoshop to resize and prep textures. Also use optimize modifier inside 3D Max a lot. or just re model or replace with my own version of their models.

Regarding CPUs yes some Xeon has many cores but the standard speed is low, also there are some version of CPU that not all the cores Turbo.  Our IT group didn't care of any of these detail because nobody knew or have experience about it, we created a good partnership with them, this way every problem we have they know what we mean and don't take us just like a spoiled user

I have settled for a 6 core CPU that everything turbo to 4Ghz as a workstation, but for render nodes we got dual 12 cores CPUs with RTX cards so everything render goes to these nodes or cloud rendering when need it.