White outline of objects (halo) after combining Vray render elements in PS

[paulzet]

So after I stack Vray elements into photoshop, there is always a white outline or halo visible when the object borders a reflective or refractive element.

 

This is the RGB render:

http://imgur.com/q2wtRI3

 

This is the composite with RawGI, Diffuse, Refraction, Reflection, and Dirtmap

http://imgur.com/zJsQBCD

 

I have read other threads with people use the defringe and remove mattes in PS, but none of these work for me since these halos don't occur around transparency.

 

Any help would be great!

[Corey Beaulieu]

I don't know how you are compositing these passes in PS, but they are not how the RGB is created so I would expect differences.

 

GI + Lighting + Refraction + Reflection +Specular + Self Illumination + Atmosphere + Background = RGB. Raw is not necessary nor are the filters unless you want to introduce something not in the 3D Scene, and Atmosphere and Background are usually not included, but would need to be if you use those features.

 

Your comp should be in 32-Bit and every (+) should indicate the Add/Linear Dodge blend mode.

 

Dirt is not something that should be included as it is not baked into the final render and if memory serves, this may be the cause of your halos as something is wrong with the pre-multiply. Well, it isn't wrong, the pre-multiply is against white on the Dirt Pass, but RGB or Black information in the rest of the passes. I believe even on Multiply where the white information should be removed, the dirt will halo in 32-Bit.

 

The only remaining caveat to this method is that you need to make sure you don't use "All Channels" for any material, not even glass or water. All channels will put that material information into all passes and then they will become over bright in the final comp.

 

I always save to an EXR format and use the defaults. There is little if any difference between the comp stack and the RGB.

[Nicolai Bongard]

Regarding the compositing of passes with exr files; On Grant Warwicks twitch channel there are some free streams in the archives, and in one of the episodes where they are working on a car there are some good tips and tricks for working with separate passes as exr`s in photoshop, you may want to check those out.