Unable to cast shadows from hidden object through glass

[Casey Hawley]

In Max 2013 using Vray 2.4, I have an interior office scene with windows in the background. I have clear refractive vray glass with affect shadows on color+alpha. Beyond the glass..outside..I have a billboard tree using an alpha for opacity and it's positioned as such in order to cast shadows into the space. This set up works great.

 

The problem is, I don't actually want the tree to be visible to the camera so that I can comp a nice looking background in later with a tree that looks better.

 

Hide it, right? So.. object properties: visible to camera off usually does the trick. However, the tree still shows! So.. object properties: visible to reflection/refraction off... Yes! It's gone! BUT.. I no longer get the effect of the shadows it was casting through the windows. I'm really baffled why it still appears when it is hidden from the camera. Does not compute!

 

I would rather not delete the glass because it catches nice reflections of the interior. I'm stumped. Is there any way to do this? I'm not used to using render elements, but maybe that's the way to go?

 

TIA

[Corey Beaulieu]

Have you tied using the VRay Properties menu to create a matte object?

 

Tick on Matte Object, Affect Shadows, and contribute alpha as -1. It will show in the render, but not the Alpha.

 

Here is a tutiorial using the MtlWrapper which is the same thing as Vray Properties, but a little more malleable: http://www.onoff.ch/2007/01/12/shadow-matte-with-v-ray/

[Casey Hawley]

I ended up faking it via careful patchwork and masking in post because of my deadline. The matte object route sounds like it will work. I've used them in the past, but only for ground objects to catch shadows in the alpha. Anyway, I will give it a go for my own education. Thanks, Corey.

[Zdravko Barisic]

The tree was visible to render due to this parametar

visible to reflection/refraction off

as it is behind the refractive glass mat...if there is no glass, it would not be visible, only shadow casting.

Best Regards!

[Casey Hawley]

I understand that, Zdravko, thank you. What I don't understand and take issue with is if 'Visible To Camera' is 'Off', then it should logically not render, whether the object is behind a refractive surface or not. Apparently there is something behind the scenes in the algorithms about rendering refraction that makes my simple setup ineffective.

[Casey Hawley]

Have you tied using the VRay Properties menu to create a matte object?...

 

Ok, I did some testing making the tree object a matte surface.. and still no dice. Even as a matte object with contribute alpha -1, and shadows on, it shows in the alpha. I also tried the VrayMtlWrapper method with no luck.

 

Obviously, removing the glass makes things work as I intend, but then I lose the nice reflections from the interior objects & lights.

 

So, after all of this, here is the only way I can figure to get what I want:

1. Use render elements to get both (a) my beauty pass, and (b) my reflections from the interior, with tree showing and casting shadows.

2. Next, remove the prop tree and glass, turn off all surface properties and GI and render again, but just the windows region to get a clean alpha.

3. In post, use the clean alpha to remove the prop tree from the windows (thus the interior reflections too), and composite back in the reflection pass, once again using the alpha to mask everything but the windows (so as not to get a doubling effect elsewhere).

4. Composite new tree and backdrop.

 

A little extra work, but really not so bad now that I have it straight in my head!