Colour management when using back to beauty 32bit EXR

[Tom Bussey]

I'm trying out a 32bit back-to-beauty compositing workflow and photoshop is saying my EXRs are in the RGB Built-in colourspace. Is this just a way of saying it's effectively no colour profile/ linear image?

 

What should my process be to ultimately use this in an 8bit aRGB working space? I'm inserting it in this space by placing the 32bit comp in the 8bit sRGB PSD as a linked smart object.

 

Also the photo I'm using (in the 8bit file) seems to have the ProPhoto profile assigned, presumably because it was taken in RAW. I know this probably means that I should be working in Adobe RGB, but I don't have a wide Gamut monitor, nor does anyone else at my company.

 

Does any of this even matter if I've got an sRGB working space assigned in colour policies? Am I right in thinking that photoshop is translating the native profiles of the source imagery to the working space? As it happens the image looks identical in the 32bit and 8bit files.

 

And yes I need to read Jeff's article again.

 

TIA.

Edited May 30, 2019 by busseynova

[James Vella]

your EXRs are in sRGB linear space if you are rendering from 3dsmax. (this is a color space, similar to rec709).

 

If you want to be ultimately in 8bit you can either place it as a smart object or just double click on the layer to make sure its unlocked, select Image > Mode > 8 Bit > Don't Merge. This will keep the colors as they are in the EXR.

 

The photo you are using if its using pro photo was probably exported with this color profile from Lightroom or such. It doesnt matter if you are converting this to sRGB as its a lower colorspace, as long as it looks as expected.

Just use Image > Assign Profile > sRGB if you intend all your images to be in this color space and you should have no issues.

edit: Red text is incorrect, use Image > Convert to Profile

 

As a side note, if you are rendering to EXR I would advise using a 16bit sRGB color space for post production, you will have more depth for your 3D elements and if you really need extra blacks/whites (zdepth for example) then keep this as a 32bit smart object.

Edited May 31, 2019 by redvella

[Tom Bussey]

Thanks for the reply, I'll check my Lightroom export settings.

 

Will back-to-beauty compositing work correctly with 16bit? My research seemed to suggest it would be slightly off, and photoshop opens them as 32bit anyway, although I suppose the EXRs themselves will be slightly smaller on disc.

[James Vella]

you have very limited options working in 32bit mode in photoshop, nuke would be a better option if you want it 1:1. However I still prefer photoshop for its simplicity/layer workflow. That being said, working in a 16bit photoshop file will give you most of the flexibility you need (again you can just keep what you need in 32bit mode such as zdepth layers and light selects and have them as a smart object within the 16bit working file so you can readjust the blacks/whites when you need separately).

 

When you export your EXR you can export them as a Half Float (16bit/channel) and photoshop will read this as a 32bit file (and its a very compressed file, smaller than a png). Theres a huge amount of data to work with, you rarely ever need Full Floating (32bit/channel) EXR files, you can do a test for yourself and check the difference.

 

edit: instead of nuke you could also try after effects, it can also do quite a lot in 32bit mode these days.

Edited May 31, 2019 by redvella

[James Vella]

Also a few notes regarding your initial questions:

 

I'm trying out a 32bit back-to-beauty compositing workflow and photoshop is saying my EXRs are in the RGB Built-in colorspace. Is this just a way of saying it's effectively no color profile/ linear image?

 

What should my process be to ultimately use this in an 8bit aRGB working space? I'm inserting it in this space by placing the 32bit comp in the 8bit sRGB PSD as a linked smart object.

 

You can place a linear sRGB render into an 8bit photoshop file as a smart object if you wish. You can always go down to a lower color space just not up eg. Pro Photo > Adobe RGB > sRGB. You can technically but you have already clipped the color range by doing so (in the 8bit working document, not in the smart object) - just something to be aware of.

 

Also the photo I'm using (in the 8bit file) seems to have the ProPhoto profile assigned, presumably because it was taken in RAW. I know this probably means that I should be working in Adobe RGB, but I don't have a wide Gamut monitor, nor does anyone else at my company.

 

You dont need to work in Adobe RGB space, sRGB is just fine for 95% of archviz work. There is more to this but really for simplicity all web content uses sRGB and lots of print labs now use sRGB printers - with exception of few. Ive used EIZO monitors calibrated for 100% Adobe and you really only get marginal amount of more blue/green. I would recommend if anything having a nicely (hardware) calibrated sRGB montior. Another thing if you are using 3dsmax is that its not color managed, you simply cannot render to another color space other than sRGB, so for now it remains easiest to stay in sRGB. Once HDR Monitors become mainstream and if your 3D software is color managed you can relook into this topic.

 

Also another note on RAW files, they do not have a color profile assigned to them - Lightroom assigns ProPhoto by default because thats the largest color space available for it currently (and contains apparently more colors than the eye can see). It does this so that you can always bring it down to another color space later such as Adobe/sRGB.

 

 

Does any of this even matter if I've got an sRGB working space assigned in color policies? Am I right in thinking that photoshop is translating the native profiles of the source imagery to the working space? As it happens the image looks identical in the 32bit and 8bit files.

 

It doesnt matter if you intend on bringing everything to sRGB working space. Whatever the image was before (say you are bringing in a ProPhoto/Adobe RGB backplate, and you bring it into a sRGB working file, it will do the conversion for you and should look exactly the same - since you cannot see the additional colors on an sRGB monitor anyway). If you DO see a difference when doing the conversion you have done it incorrectly, refer to my first reply (I have corrected a mistake I made in red).

 

All this by the way has nothing to do with your 32bit file, just to clarify so I dont confuse you I was just suggesting working in a 16bit photoshop file so that you can composite your renders in a similar fashion that the VFB does in 3D.

 

If you want to go down the rabbit hole I have provided some references below for additional reading if it takes your interest - these specific pages helped me learn a bit more about color management (although theres lots to learn).

 

Color Management

https://www.fxguide.com/featured/the-art-of-digital-color/

 

Color Managed Workflow

https://www.adobe.com/digitalimag/pdfs/color_managed_raw_workflow.pdf

 

Color Modes & Color Models

https://www.designersinsights.com/designer-resources/understanding-color-models/

 

Understanding Color Space in Compositing

https://www.designersinsights.com/designer-resources/understanding-color-models/

 

Xrite Color Management

https://xritephoto.com/color-management-guide

 

A Guide to ACES

https://acescentral.com/t/aces-primer-glossary-and-quick-start-guides-get-up-and-running-with-aces/1433

Edited May 31, 2019 by redvella

[Tom Bussey]

Thanks for all the info. I was doing a lot of that already, but it's good to have it confirmed as this is all fairly new to me.

 

I did a test with 16bit vs 32bit EXRs and you're right, that there's no perceivable difference, so that'll save me a lot of space over time. However when I try to comp the files in a 16bit PSD the result is definitely off. Not sure if this is what you were suggesting, but it would have given me the flexibility of 16bit (specifically magic wand for selecting masks). As far as I know there is no colour keying in photoshop, but then it's not really true compositing software. Yes, Nuke would be a better option, and I do want to find an alternative eventually, but then you're going to lose the more painterly retouching tools (AFAIK). Would have been nice to keep it in one programme. I did try comping in AE but found it a bit clunky, and I don't think you could insert this in a PSD as a smart object anyway, so it'd add another stage of import/export.

 

That makes sense about Prophoto. I realised I'm using the cloud-based one rather than classic, so I think my export options are limited, but probably makes sense to to any convertion in photoshop as I'll sometimes want to use photos in the 32bit comp.

[James Vella]

However when I try to comp the files in a 16bit PSD the result is definitely off.

 

Can you show an example of the comparison? PS/VFB?

 

Not sure if this is what you were suggesting, but it would have given me the flexibility of 16bit (specifically magic wand for selecting masks). As far as I know there is no colour keying in photoshop, but then it's not really true compositing software.

 

You are correct - no color keying in Photoshop. You can select by RGB using the Channels tab next to the Layer tab if you are using a multimatte pass (red, green, blue). Using the wand for IDs that arent pure 255, 0, 0 like a wirecolor will always be a difficult job in Photoshop.

 

Yes, Nuke would be a better option, and I do want to find an alternative eventually, but then you're going to lose the more painterly retouching tools (AFAIK). Would have been nice to keep it in one programme. I did try comping in AE but found it a bit clunky, and I don't think you could insert this in a PSD as a smart object anyway, so it'd add another stage of import/export.

 

I hear Affinity does a good job - especially for 32bit EXRs and keeps a similar pipeline to Photoshop. There is plenty of workarounds in Photoshop working in 16bit but I would have to have a better understanding of exactly what you are trying to achieve step by step. Im curious as to why you are doing a back-to-beauty & then doing post work on top? If you can show the reason behind the method there may be a better solution - could you instead render the final and do the painterly retouching on top (adding/removing refraction/reflection/lighting passes etc where needed)?

 

That makes sense about Prophoto. I realised I'm using the cloud-based one rather than classic, so I think my export options are limited, but probably makes sense to to any convertion in photoshop as I'll sometimes want to use photos in the 32bit comp.

 

Yeah I wouldnt be too concerned about it, do a small test bringing it into your working file as a backplate (I assume is what its for), and if it looks like it does in lightroom you are good to go.

 

edit: You may find this a good solution to your masks, I have not used it but it looks promising.

https://www.exr-io.com/exr-io-2-00/

Edited June 1, 2019 by redvella

[Tom Bussey]

You are correct - no color keying in Photoshop. You can select by RGB using the Channels tab next to the Layer tab if you are using a multimatte pass

 

Thanks, I don't know why I didn't think of this!

 

Im curious as to why you are doing a back-to-beauty & then doing post work on top? If you can show the reason behind the method there may be a better solution - could you instead render the final and do the painterly retouching on top

 

I'm not 100% percent that it's totally necessary, but it seems like a quick and manageable way of getting the result I'm after. Basically everything in or on the building goes in the 32bit PSD, everything outside goes in the 8bit PSD. I'm rendering out a simple model with virtually no interior features - just wall, and just reflections of the HDRI in the windows, then adding photographic elements on top of these, masked to the approriate areas and adjusted with an exposure layer for strength. It seems to work well for getting the balance right between reflection / refraction without blowing anything out.

 

Once that's sorted I can use the 8bit file to mask the render into the photo of the street the building is on, and add in some additional people in front of the building

 

 

 

Can you show an example of the comparison? PS/VFB?

 

I haven't got it in the VFB not, but here's a screengrab of the comp vs. beauty pass in a 16bit PSD

 

 

Edited June 3, 2019 by busseynova

[James Vella]

I think I can understand, maybe... You are showing me your RGB_color pass - this is what you see in the VFB correct? If so then you cannot overlay the reflection/GI etc passes on top of this to get a back to beauty as you are already at final beauty stage before you have started layering the passes (reflection/refraction etc have already been applied). The rgb_color (beauty pass) should appear totally flat, without reflections or gi or anything - refer to this in the vray help https://docs.chaosgroup.com/display/VRAY3MAX/Beauty+%7C+RGB_Color

 

In essence, it seems like a very complicated setup - I would maybe re look at your workflow here. Would it not be easier to put the backplate into the render itself and adjust the render in VFB to match the elements - and give yourself some wriggle room to lower/increase each element slightly in post? You can then add people after and color match them to the render. Or did I miss the point?

Edited June 3, 2019 by redvella

[Tom Bussey]

overlay the reflection/GI etc passes on top of this to get a back to beauty as you are already at final beauty stage before you have started layering the passes

 

That's not what I'm doing, in the first image you're seeing Refr. + Refl. +G.I + Lighting + Spec; in the second one you're seeing just the Beauty pass (in normal mode in photoshop - no additions) Which is what I see in the VFB and what I want to achieve. 32bit PSD gives this result.

 

he rgb_color (beauty pass) should appear totally flat, without reflections or gi or anything

 

This is not what I understand from that document, sounds likewhat you're describing is the diffuse filter pass?

 

VRayLighting + VRayGlobalIllumination + VRayReflection + VRayRefraction + VRaySpecular + VRaySSS2 + VRaySelfIlumination + VRayCaustics + VRayAtmosphere + VRayBackground = RGB_Color (Beauty)

 

Would it not be easier to put the backplate into the render itself and adjust the render in VFB to match the elements

 

Not sure what you mean by the the second part, but in my experience, it's not more work. Embeding the relections in the 3D scenes means getting them balanced with the refraction first time, and If I want to add a tree into the reflections, or change a shop interior for instance, then it means a re-render, not to mention the time spend acquiring, placing and adjusting those 3D elements to get the same visual complexity as a photo, and the added scene size. Also can't always get flat photographed back-plates on a specific site, so it's a lot easier to compensate for perspective doing them in post.

 

Oh, also makes inserting 2D people behind glass a doddle

Edited June 3, 2019 by busseynova

[James Vella]

Ok that makes much more sense.

 

...in the second one you're seeing just the Beauty pass (in normal mode in photoshop - no additions) Which is what I see in the VFB and what I want to achieve. 32bit PSD gives this result.

 

So let me know where things go astray, when you open the beauty pass in photoshop, it opens as a 32bit file, it looks correct. What happens from this point for it to not be correct?

[Francisco Penaloza]

Ok that makes much more sense.

 

 

 

So let me know where things go astray, when you open the beauty pass in photoshop, it opens as a 32bit file, it looks correct. What happens from this point for it to not be correct?

 

Just be aware that when you load a 32Bits image in photoshop, this software adds a gamma curve to it. You see the image gamma corrected. If you open the same image in Fusion, Nuke or any other compositor, that image will display gamma 1.0 (all these if you saved the image at 32 Bits with gamma 1.0 to start of course)

 

Now 32Bits workflow in Photoshop is very limited, the only way around is to convert those images to smart objects, then switch Photoshop to 16 Bits and take it from there. Still, there are some limitation to this workflow but it gives you more tolerance shifting colors than traditional compressed 8-bit images.

 

If you need to have full control at 32Bits gamma 1.0 then you need to use a more complex tool such Nuke, Fusion, Nantron or so.

After effect also could do the same but in my experience it slowdowns with large size images.

 

Having said all that, using this workflow just to put people behind reflection it seems a little overkill. It is satisfying to work at full Float but at the end, most people never see the difference, besides you'll save your image to 8 bits JPG or similar so all that extra info is lost.

 

If you are working in a production environment when your file will go to a color editor or VFX artist then it makes more seance to keep at 32Bits gamma1.0 so you can flex colors as need it.

I usually save at 32Bits EXR then save as smart layer, then work the files at 16 Bits, if it is an animation and lighting is not changing then I save it at 16Bits with the burn already fixed in VRay, this way we save file size, we save multilayer EXR frames.

[Francisco Penaloza]

I'm trying out a 32bit back-to-beauty compositing workflow and photoshop is saying my EXRs are in the RGB Built-in colourspace. Is this just a way of saying it's effectively no colour profile/ linear image?

 

What should my process be to ultimately use this in an 8bit aRGB working space? I'm inserting it in this space by placing the 32bit comp in the 8bit sRGB PSD as a linked smart object.

 

Also the photo I'm using (in the 8bit file) seems to have the ProPhoto profile assigned, presumably because it was taken in RAW. I know this probably means that I should be working in Adobe RGB, but I don't have a wide Gamut monitor, nor does anyone else at my company.

 

Does any of this even matter if I've got an sRGB working space assigned in colour policies? Am I right in thinking that photoshop is translating the native profiles of the source imagery to the working space? As it happens the image looks identical in the 32bit and 8bit files.

 

And yes I need to read Jeff's article again.

 

TIA.

 

To answer your questions specifically,

Photoshop has a color profile already by default, you can change that but you need to check your monitor capabilities to know that you can see the correct colors. When working with 32Bits image, Photoshop will apply a Gamma correction to that file to display correctly to Gamma 2.2.

I would not recommend changing your Photoshop color profile because as you mentioned not many people will have Calibrated monitors or specific color profiles. Most of the planet work with the default. If you are working on films or related business then Color profiles are important so your files are compatibles with other people in the team, but at the end everything is compressed to the standard output. Well that last part is changing now because we can have TVs with wige gamut so eventually, we all will be able to see more colors

[Tom Bussey]

I have been trying to convince my employer to buy a colour calibration tool for a hwile, but it's low priority for them, they'll only notice when something looks different on their monitor to mine, and wonder why! I may give up, buy my own and sneak around in the night profile people's displays.

 

Thanks for the clarification though.

 

Wide Gamut TVs you say? Didn't know that was a thing.

[James Vella]

Wide Gamut TVs you say? Didn't know that was a thing.

 

Yes - HDR TVs/Monitor (I mentioned earlier). Also Samsung Note devices (and maybe a couple others) are using HDR screens now with 10 bit color depth. However for now we should be able to get a few more years out of sRGB workflow and I think ACES is overkill as we dont really remaster our older projects. It will be interesting when the ACES workflow becomes much more simple, Im looking forward to really pushing 3D lighting in post (if its even necessary by then with this GPU rendering catching up - Im seeing 10 minute renders from Fstorm that would take me 4 hours and blown away by the quality).

Edited June 5, 2019 by redvella