Jump to content

VRay render with alpha contains a white matte/outline


ryandmonk
 Share

Recommended Posts

I'm trying to render a very simple animation with an alpha channel so I can do video composting to it. I'm using 3ds max 2010 SP1 with VRay Adv 1.50.SP4a. The issue that's going on is that there's a 1-2 pixel white matte/outline the geometry edges because of the alpha channel. I'm rendering as a targa sequence with a split alpha.

 

Other areas I've played around with is the the material window, where I made sure the "Affect channels" is set to "Color+Alpha", but overall I'm trying to figure out how to make the alpha channel more precise without the couple pixel edge.

Link to comment
Share on other sites

i had VRaySky as my environment background. I disabled it, and made the background color gray. The outline is still there and visible if I make the background color to a black, white, etc.. but the gray hides it well.

Edited by ryandmonk
Link to comment
Share on other sites

the AA filter is "Area" with a size of 1.5 and sub-pixel mapping is turned on. i also previously tried turning on clamp output with the same effect.

 

the adaptive subdivision image sampler setter is: min. rate of -1, max. rate of 2. clr thresh is at 0.1.

 

other people have also mentioned several times about making sure sub-pixel mapping is turned on, but i'm thinking this cause is from something else.

 

if i switch to another renderer, such as mental ray, the alpha channel is dead on, so i know its something with vray...

Link to comment
Share on other sites

  • 6 months later...
  • 1 year later...

I don't know what causes it but if you're using after effects to comp the video you can use the Remove Color Matting effect to minimize the halo around your objects. I'd really like to know what causes it as it makes working with layers very difficult, have you posted your issue on the Vray forum yet?

Link to comment
Share on other sites

This is how I get around the issue:

Before rendering, I try to match the background color with whatever I'll comp it with... If I'm not sure then I go with black.

Save the alpha channel as well.

In PS, apply defringe to the alpha map (most cases) and mask all the rendered layers with it.

In AE, use the same alpha map (most cases with color matting) set to "stencil alpha", thus masking all the rendered layers.

All this may be dumb solutions... but who knows? Works for me.

Worth mentioning that higher resolution renders and AA settings will incrementally help a lot.

Link to comment
Share on other sites

There's a thread on the Vray forum that's talking about a similar issue, the consensus is that Vray seems to be getting worse in dealing with render channels. There's obviously a problem and it's been there for a while, I don't know why it's being ignored seeing as how it's pretty important.

Link to comment
Share on other sites

Although it's not a solution to the problem (that ball is in Chaos Groups court) you can manually adjust clipping masks in Photoshop by right clicking on your layer mask. There all sorts of settings you can tweak to reduce the ammount of fringe.

 

E

Link to comment
Share on other sites

I always got this problem when using vraysun+sky in exterior scene. I think the problem is glossiness from material that reflect vraysun's light is cause of this outline effect.

The solution to solve this problem that I always use is putting something (such as trees or buildings) behind the scene to block vraysun's reflection from creating glossiness to the edge of object.

Link to comment
Share on other sites

  • 2 weeks later...

I've pretty much managed to contain this problem by using backgrounds that roughly match the background used in the composition and using unmultiplied TGA file.

 

Also when doing videos, tools like after effects have advanced tools that allow you to trim off these kind of things without doing too much damage to the render.

Link to comment
Share on other sites

  • 1 month later...

OKay , found another solution.

 

 

check "dont affect colors" , clamp, subpixel mapping on. now u can use hi intensity light as usual...

 

the downfall is you wont be able to use reinhard or anyother mapping except linear.:)

Link to comment
Share on other sites

  • 5 months later...
  • 2 weeks later...

Hello!

I had the same issue when rendering grass, I think I may have found a solution. I remade the alpha channel pic many times and nothing changed. This happened only for grass and not for leaves on trees. The difference between the two : leaves have black all around them while grass touched 3 sides of the alpha. So i cut a very small black outline around the alpha for definition and it worked. I think however small its allowed without compromising continuity will work fine.

Link to comment
Share on other sites

Create an account or sign in to comment

You need to be a member in order to leave a comment

Create an account

Sign up for a new account in our community. It's easy!

Register a new account

Sign in

Already have an account? Sign in here.

Sign In Now
 Share

×
×
  • Create New...