Fastest texture format for high res textures.

[Crazy Homeless Guy]

I have a 20,000 pixel wide painted site map that I am using. I think is slowing my render times quite a bit. Does anyone know what file format is best for speed rendering when the image is that size?

 

Right now it is saved as a jpeg, and registering about 186 megs. I am wondering if a tiled EXR map might render faster? Anyone else have experience with this?

 

I have never used one as a texture, so I am not sure if that is the direction I should be looking in or not. I am open to all suggestions that are more insightful than simply saying don't use a 20,000 pixel wide texture map.

 

Thanks in advance.

[Matt McDonald]

Kind of piecing things together here but:

 

I've always sort of understood that JPGs still get opened up as 32bit images and that there is no memory saving to be had from the JPG format...take that I bit further and I would guess that the image not only has to be opened but then converted. So maybe there's some speed benefits there.

 

On the other hand your 186mb JPG will be a much larger file (in terms of space on disk) and the transfer times for a huge TIF might slow it down more.

 

I assume this is not just a displacment/bump sort of map. You can't convert it to grayscale?

 

I've seen plugins for Mental Ray over at scriptspot or maxplugins that will take an image and convert it over to Mental Ray's internal image file format. There might be something similar for V-Ray.

 

I hope this (at least) gives you some ideas. I'll be curious to hear what you find out.

[Crazy Homeless Guy]

I think you are correct in how things are translated, and I think uncompressing a jpeg is a bit more memory intensive than using a tiff, but I am not sure that it is a significant enough hit to slow down render time or not. I am using Xeon processors that range from 2.3ghz to 3ghz to render this image, so they should be able to handle that task fairly quickly.

 

I mainly used jpeg to reduce the time needed for file transfer. I am just not 100% comfortabel as to whether or not the image is slowing down the overall speed of the rendering. I need to do more extensive testing to have a definete answer on that.

 

On the Chaos forum they talk about using a tiled EXR file for large textures because they can be loaded and unloaded dynamically from memory.

 

As I am typing this though I am not sure that a tiled EXR would solve my question because memory consumption is not a problem. All machines have 8-12 gb of RAM, and I think I am running about 4.5 to 5gb while rendering this file.

 

According to Vray estimations I should get a 6000 pixel wide image out in about 2.5 hours using 24 cores. Which isn't horrible, but I really thought I could get it under 2 hours.

 

The texture size came into question because it seemed like the buckets where it was caculating the texture, and proxies on top of the texture were taking longer than I would expect them to. Maybe it is just in my head though.

[Crazy Homeless Guy]

Edit: The Tiled EXR for loading and off loading textures may only work for Maya. The bitmap pager was suggested for Max. But again, memory consumption is not my worry, just what processes fastest.

[Matt McDonald]

If my render time weren't what I expected, I'd probably suspect the map as well.

 

Can you break the map up so that stuff that is farther away is lower res? (50 ppi vs. 72ppi - or something like that)?

[Crazy Homeless Guy]

Not a bad idea.

 

I might try rendering the map in two passes. A high res, and low res. The site is likely not to change, so I could keep re-using the high res pass, and simply re-render the scene with a low res map when needed, and comp them in post.

[Crazy Homeless Guy]

I did a test where I reduced the texture from 20,000 pixels to 2,500 pixels, and then rendered a patch. The patch with the full res took 9:08. The patch with the reduced res took 7:20 minutes.

[Matt McDonald]

That's a start. Right? 9 min to 7 min is 2:45 to 2:15 or something.

[Tommy Burns]

I did a test where I reduced the texture from 20,000 pixels to 2,500 pixels, and then rendered a patch. The patch with the full res took 9:08. The patch with the reduced res took 7:20 minutes.

 

Seems good now render it without the texture to see what you will really save? If you know what I mean?

[Koper]

that ratio doesn't seem right??

 

what if you cut up the highrezz into 20 smaller chunks and have them as proxies in the scene so that at rendering time vray only loads up the chunks which are needed.

 

but yes, i will also render without the texture to determine what the texture actually does with the scene

[Crazy Homeless Guy]

Sometimes the texture is displayed smaller than the 6000 pixels wide when we are viewing from an aerial shot, so then the 20,000 is a waste. ...But at other times I am sitting almost directly over the texture, so it will be even less than a 1 to 1 as it is. I wanted to use the texture without swapping out to streamline the work flow, and reduce the chances of user errors during the process.

 

Right now I am doing a series of 6 stills, with no animation, but that may change next week.

Edited February 12, 2010 by Crazy Homeless Guy

[erickdt]

I know you said that the site plan was painted but is there any way it could be imported and somehow converted to geometry?

[Crazy Homeless Guy]

All solid objects such as sidewalks, roads, and benches are geometry.

 

The painted part is gradations and nuances in the grass and marsh areas. I did it this way to add a greater level of tactile appearance, and visual depth. I might have been able to use vertex mapping, and blend textures on the fly, but I am not sure I would have had as much artistic control as what I would by simply painting a map in Photoshop.

[erickdt]

I was more thinking that if the painted site map was created in Illustrator that you could import those vectors into max and extrude them, tecturing them individually (with procedural maps where possible).

[Dave Buchhofer]

no time to read, but the key is Divide and conquer, break it up into 9x9 (or 3x3 or XxX) squares and map the terrain accordingly.. speeds will jump tremendously. after 4096+ textures things get slow due to the amount of ram uncompressed textures take up (in memory space, its uncompressed unless saved as a .DDS, where you can 'pre' set the memory compression type and save to disk in the same format it would be in memory (DXT1, DXT5, RGBA, etc.., but the file type is a pain to work with.)

 

what format a file is on disk doesn't have any indication of how much memory it uses in the system ram..

 

gotta run! can try and explain later if that doesn't clear up

 

all the weird crap needed to know to do interactive 3d until recently! heh!