Extreme pixelation after Xnormal bake

Leofwin

Hey guys!

I'm currently creating hair cards for a character of mine and encountered a problem while baking the Alpha and other maps in Xnormal - the bake becomes extremely pixelated. No matter if I use Antialiasing or not. I'm using 4x AA right now and if I zoom in I can see that the pixelated edges are a tiny bit blurred which I assume is the AA - but that can't be what it's supposed to be. No matter which size I bake, nothing helps. Even 8k becomes pixelated - not as much as lower bakes but if I scale that down afterwards it gets worse again. I know that good results for hair maps can be achieved with bakes in Xnormal but I have no idea what I'm doing wrong. I also tried scaling everything up by 100 in the baker but it did nothing for the result. I should mention, though, that I use Xnormal 2015 and can't update at the moment.

Any help is appreciated! :] 

Leofwin

This is how the bake looks, by the way. Even the 8k bake doesn't look much better. There are fewer pixelation-induced gaps where there should be hair but the pixelation itself doesn't really decrease. What am I doing wrong..? 

poopipe

You'll get better results if you bake hair from hair data than if it's converted to geometry - most hair rendering setups use tricks to avoid exactly what you're getting here.  You also don't have to deal with massive high res meshes that take ages to export and render. 

Assuming you're using maya/Xgen... 
Arnold does a great job of softening off the edges etc. And can generate direction maps for anisotropic specular etc. 
In my own testing it was a thousand times quicker and looked far better than exporting meshes and baking in substance. 

Leofwin

Yup, you're right, I'm using Xgen and Maya. But how do I bake the hair data without converting it into meshes before? I'm trying to create a game ready mesh and waa following the texture workflow Andrew Giovannini posted on 80lvl and youtuve. Is there a better one :?

A propos Substance - I baked several times with dofferent bakers now and all create this strong aliasing except for Substance Painter. However, the maps baked there can't be used for different reasons..

Maybe the hairs are too thin? 

poopipe

bake the hair clumps directly to a plane using Arnold's render to texture

DustyShinigami

Sorry to revive an old thread, but I’m currently experimenting with baking hair via Arnold too. I do find the bake results to be much better compared to xNormal, but I’m a little bit stuck with a few things and have a few questions.

It’s great that Arnold allows you to bake the raw data for the hair, rather than turning it into geometry first, but should that only be done via Render to Texture? Can it be done just from setting up a camera and light and using the regular renderer? I imagine with that, that materials would have to be assigned though, correct?

I’ve been using the regular Arnold renderer and assigning materials to the hair clumps that have been turned into geometry in order to get various baked maps. So far, I’ve managed to get a Normal/flow map, an Alpha, an ID map, gradients/root maps, although I’ve been a bit stuck with the height maps. xNormal’s Height Map option gives you a Height Tone Mapper after the bake, so you can adjust sliders to get various results, which is great for compositing different variations in Photoshop. However, how can something like that be done with the Arnold renderer? And the Render to Texture option only gives you the option to bake a Displacement map.

It would make sense just to do that bake with xNormal instead if I want that level of control, but I’ve found that the renders don’t match up if I combine Arnold renders with xNormal’s. 😕 For instance, I’ll have the top orthographic view set up, along with adjusting it with the Render Gate, and set the desired resolution, though it seems xNormal’s ‘view’ is set a specific way…? Unless this can be changed…? Even then, I have no idea how I’d get it to match exactly. But once I bring all baked maps into Photoshop from Arnold and xNormal, I discover that the Arnold renders are a bit smaller or zoomed out…?

At the moment it just feels like I should only bake with one method or the other. That I can’t combine. If xNormal - the bakes aren’t as clean, but I have the option to keep their render sizes consistent and have the option to get various Height map results. But with Arnold - the baked results are faster and cleaner, but I have no idea how to get various Height map results like with xNormal.

Any suggestions/workarounds I could do? Thanks.

poopipe

you dont need to use RTT - i found it convenient because it allowed me to bake to a known area without having to argue with cameras. just simple renders should work fine. I seem to remember it being possible to dump each relevant channel out to it's own AOV but my memory of this is very fuzzy now.

Arnold will allow you to plug some maths in to the shader to mess with the depth/height map - it's probably more practical to just piss around with it in photoshop / designer after the fact. if you export everything as 32bit exr you won't need to worry about precision