Home Technical Talk

Is it possible to use planes with alpha to bake on to other planes?

polycounter lvl 4
Offline / Send Message
arnov polycounter lvl 4
I'm using Blender btw. I made some leaves that I baked onto planes with alpha, then I put them all together to make a single branch and now I want to bake it into another plane, but I cant seem to save alpha that I initially made for leaves? Whenever I bake it just fills empty spaces with black color, no transparency.

Pretty much what this post described 

Replies

  • Mark Dygert
    Options
    Offline / Send Message
    I mostly use 3dsmax and speed tree to assemble my foliage. In max I do a high quality render from a specific camera set up in the scene to render geo-to-opacity. I try not to use opacity in the high poly because it never works in any of the rendering engines I've used.

    Opacity to Opacity rendering is one of the more computationally complex processes that rendering engines have to preform, so most render engines don't do opacity to opacity that well, if they even attempt it at all, most don't.

    The few that do, usually only do it through high quality viewport/camera rendering and aren't set up to bake textures using cages, which is meant to be a quick fast process. So you end up stuck using the long rendering process but only on things like like planes (hair/fur rendered to a plane) and not complex models that require cages.

    Personally I would tightly trim around your high poly leaf cards so they no longer use opacity, or create several leaf models that don't use transparency and scatter them on the surface of a branch using a particle system, then bake from the geometry to opacity. But even "geometry to opacity" will be difficult to do with some rendering engines.

    Substance Designer is the only one that I know of that will bake Geo to Opacity, which is a pain but it works reliably. All of the rest (xNormal, 3dsmax, Maya and Blender) at best do a "missed ray" color when a ray hits the void and that can be used as a hacky opacity mask. But that's pretty crappy and doesn't give you much control over the edges of the transparency. It also has trouble rendering cards behind cards and will usually be a huge pain in the ass, more trouble than it's worth.

    Geometry to Opacity, sort of works but it's dicey with most render engines.

    Opacity to Opacity, is a unicorn process that you would think saves a bunch of time but isn't supported by the vast majority of baking options.





  • arnov
    Options
    Offline / Send Message
    arnov polycounter lvl 4
  • Eric Chadwick
    Options
    Offline / Send Message
    Unless I'm misunderstanding the problem, I'm able to do this just fine in 3ds Max. 

    Not by baking, but by simply rendering a top/camera view of opacity mapped meshes, saving out to a format with alpha. TGA or TIFF, usually.

    http://wiki.polycount.com/wiki/Texture_Baking#Transparency
  • m4dcow
    Options
    Offline / Send Message
    m4dcow interpolator
    Unless I'm misunderstanding the problem, I'm able to do this just fine in 3ds Max. 

    Not by baking, but by simply rendering a top/camera view of opacity mapped meshes, saving out to a format with alpha. TGA or TIFF, usually.

    http://wiki.polycount.com/wiki/Texture_Baking#Transparency
    Getting transparency is fairly straight forward this way.

    But doing the same with normals(texture & geometry based), AO, translucency is a nightmare.

    Back in the day I remember seeing someone do the normals with a special light setup and camera render. I was able to achieve the same in mental Ray but using different render layer overrides, but it wasnt a very efficient workflow.

    Last year I tried to tackle it again, this time using Arnold and AOVs, I made some decent progress but got stumped again and gave up, can't remember the thread but it should be somewhere in my replies.

    Ultimately I found it quicker to just make the leaves high poly cutouts not using alpha, and bake that out normally instead if fiddling around.
  • poopipe
    Options
    Offline / Send Message
    poopipe grand marshal polycounter
    I was also involved in that thread.

    It was possible in max with mental ray but since that's gone away we're all a bit buggered. 

    It's worth pointing out that it's in no way a technically impossible problem - especially in a software renderer - so I fail to see why nobody's implemented it
  • gnoop
    Options
    Offline / Send Message
    gnoop sublime tool
    poopipe said:
    It was possible in max with mental ray but since that's gone away we're all a bit buggered.
    In what version of Mental ray it was?  Perhaps I still have it . :p      
    I tried it there for years with zero success.  

    Octane GPU render  "baking" camera can do it  in stand alone version but it's useless anyway since it cast the rays in one direction only and usually you want it in both directions, backward of low poly surface too.   
    Also couldn't bake tangent space normals, worlds only,  so an extra pain in the a.. with converting them  in something like SD.  
     
    Definitely not for leaves/foliage. Few years ago I tied to persuade them to make the baker more of use  but seems nobody cared.   They did it just to bake surface shaders+lighting in a single image. 

    For some  reason it's  almost impossible to explain software developers the purpose of decent baking.
  • m4dcow
    Options
    Offline / Send Message
    m4dcow interpolator
    gnoop said:
    Also couldn't bake tangent space normals, 
    There is a direction from which world space normals and camera space (from an orthographic camera) would be the same so that's an easy fix.
  • gnoop
    Options
    Offline / Send Message
    gnoop sublime tool
    m4dcow said:
    gnoop said:
    Also couldn't bake tangent space normals, 
    There is a direction from which world space normals and camera space (from an orthographic camera) would be the same so that's an easy fix.
    Yeah, but often you need to bake on a low poly envelope.  For example a couple layers of  camouflage mesh and a canvas tent beneath, all as a single layer of low poly surface. I mean with that mesh as alpha textures on hi poly.     

    Octane baker uses the low poly object polygons as ortho projection ray casters along surface normal . if there is too much of a surface  bend in between neighboring polygons it does artifacts.   So not very useful baker anyway, especially for hard surface things. 
    Still the alpha textures are not a problem for it. Could bake even shadows  those alphas cast on things
  • Noren
    Options
    Offline / Send Message
    Noren polycounter lvl 19
    With the limitation of using 1 bit alpha / cutout shaders, you could do it at least up to Mental Ray in Max 2010 (whatever version that was) but it didn't seem to work in  2012 according to this thread: https://polycount.com/discussion/91116/3dsmax-rtt-how-to-bake-using-alpha-thru-dif-nrm-etc  (never tried it myself in 2012 or higher, though).
    Vray does still work (via RTT and refraction) in Max. (I think there is a  version of Vray for Blender?)
    Normals don't make sense for non opaque / soft alpha surfaces anyway, as you can't simply mix the normals of two surfaces and get the same effect as seeing the two surfaces through each other.

    Generally any renderer who can do a cutout of sorts should theoretically have the potential to bake at least 1 bit alpha.
  • thomasp
    Options
    Offline / Send Message
    thomasp hero character
    I'd love to have this sort of functionality in a baker. Would be perfect to e.g. bake hair planes from a hairstyle down onto a head texture to conceal the scalp. Right now it's all manual labor i the 3d painter for me. Somebody should probably get the Marmoset guys onto the case ASAP. Paging @EarthQuake Ahem.



  • Eric Chadwick
    Options
    Offline / Send Message
    V-Ray can't do it right, at least not in Max's RTT.



    This is the same map in Opacity (inverted) and Refraction, IOR set to 1.
  • Noren
    Options
    Offline / Send Message
    Noren polycounter lvl 19
    What happens if you remove the map in opacity? Have you "affect color + alpha" checked in the refraction part? I'll give it it a try with a similar scene like yours a bit later.
  • Noren
    Options
    Offline / Send Message
    Noren polycounter lvl 19
    So the alpha for anything but the complete map seems indeed to be messed up in Vray, so I guess that's what I must have used. 
    But that at least works, so turning all textures white (edit: Brainfart. Not even necessary, as the alpha itself is ok. I'm using a background object set to matte, matte for refl/refr and alpha contribution = -1 in the background) for baking the alpha and using a world normal texture (in my case three world space falloff maps blended additively with a composite map, after the bake converted to tangent space with handplane) these can be rendered via the Vray complete map. Still a hassle, of course.
    I used the diffuse as a bump map on the HP as a rough example that normal maps / bumps of the HP get baked, too, that way.



  • gnoop
    Options
    Offline / Send Message
    gnoop sublime tool
    Thanks Noren  for sharing .    I also coped to do it decade ago or so when played with vray demo  but  couldn't recreate it after.       Thought It was just in my dreams .   Thanks for proving it's possible. 

    ps. Just tried,  same approach is kind of working with arch materials in Max2008 mental ray  + mr sun and exposure control  but makes so many artifacts   it's still hardly usable.
  • Noren
    Options
    Offline / Send Message
    Noren polycounter lvl 19
    The newest version of Arnold/MaxtoA (needs to be updated manually) bakes alpha and diffuse just fine via RTT in Max 2019. No normals, though, so those would have to be rendered via diffuse and object space for the time being.
  • gnoop
    Options
    Offline / Send Message
    gnoop sublime tool
    Noren said:
    The newest version of Arnold/MaxtoA (needs to be updated manually) bakes alpha and diffuse just fine via RTT in Max 2019. No normals, though, so those would have to be rendered via diffuse and object space for the time being.
    How did you do it ?  
    Have just downloaded and installed Arnold 2.3.37 for MAx2019 . Besides baking surface shading  It bakes few weird artifacts and nothing more.
    From what I saw  I am not even sure Arnold is fully  capable of baking.      That Arnold have always seemed to me even bigger pain in the a..  than Mental Ray ever was .    As always  Autodesk finds a way how to make  crazy twisted  what should be simple as 2x2 and self-obvious.
  • Noren
    Options
    Offline / Send Message
    Noren polycounter lvl 19
    Make sure you exclude the target mesh from rendering and shadow casting via the Arnold properties modifier. I used the Arnold Standard Surface Mat for my tests. Other than that I'd have to take a look at your scene.
    Arnold isn't bad, in my opinion, and actually quite straight forward (so far I prefer it over Mental Ray, to be honest). It  just does many things a bit different, doesn't offer much options in regards to optimization and is overall still poorly integrated (not gonna lie, quite disappointed after I skipped 2018 to avoid just that). Also the axing of free render nodes makes it obviously a bad option in that regard.
  • gnoop
    Options
    Offline / Send Message
    gnoop sublime tool
    Noren said:
    Make sure you exclude the target mesh from rendering and shadow casting via the Arnold properties modifier. I used the Arnold Standard Surface Mat for my tests.
    Thanks Noren, I think I figured it out.   Seems regularly Arnold only bakes surface shading with Max RTT  and for making hi-res to low res baking it works only with low res always hovering over and fully encompassing hi-res one. Should  never intersect it.  Basically bakes only to what usually a cage object. In one ray direction  along normals only  ignoring whatever you set with Projection modifier altogether.  
     in a word pretty similar to Octane with very same limitations

  • Noren
    Options
    Offline / Send Message
    Noren polycounter lvl 19
    I'm afraid you are correct. Bummer.
Sign In or Register to comment.