Normal map to vertex normals?

throttlekitty

Is it possible to apply pixels from a normal map as vertex normals of a model?

I'm trying to make a cosmetic for Dota2, and don't have access to Valve's high poly mesh. In my case, I need for my model to meet up with theirs, but can't come up with any decent results for the normal map where the two meet. Covering the seam between the two models isn't an option, it's a hairstyle. (rather, will be, but this demonstrates what I'm about here)

What I'd like to do is use a high poly interim that covers the seam area. From there, I could use Photoshop to fudge a blend between that and my actual high poly normal map.

throttlekitty

Er, to clarify, what I'd like to do is this:
* Valve's low poly + normal map ((normals)) >> High poly approximation mesh
* High Poly Approximation mesh ((bake normals)) >> Low Poly Mesh

throttlekitty

Aha, so via the Paint Vertex Color Tool, we can apply a texture map as vertex colors, which can then be queried with "polyColorPerVertex -q -r -g -b", and written as normals via "polyNormalPerVertex -xyz".

I'll try to write a script to write the colors to normals in the morning.

throttlekitty

I need some help here. How would I properly increment for each vertex? this script sets the same value for the entire selection.

global proc tkColorToNorm () {

    string $verts[] = `ls -fl -sl`;
    
    for ($i = 0; $i < `size $verts`; $i++){
        float $color[] = `polyColorPerVertex -q -rgb`;
        polyNormalPerVertex -x $color[0] -y $color[1] -z $color[2];
        }
}

And looking again, I think there's some kind of math that needs to be done on the normal map before RGB (0:1) values can be used as vertex normals, but I don't know what. But looking at values for normals in the component editor shows that normals don't add up to 1 as it is. I'm really confused at this point.

Kurt Russell Fan Club

Normal maps need to be scaled from 0..1 to -1..1 so usually you multiply by two and subtract one.

Aside from that I'm not really sure what you're trying to do - my best guess is you want to eliminate a seam between two meshes and rather than using a normal map you're planning to do it by adjusting the vertex normals using the normal map (??)

I think more pictures of what you've got and what your problem is could help

throttlekitty

Sure, here's another image. I was a bit frazzled after spending the day on the problem. I scrapped my Zbrush sculpt, I went about it all wrong in that case, so it's not pictured here.




So, Valve's asset has Phantom Assassin with a cosmonaut hood, and I'll be making a cornrow hairstyle up into a braided ponytail model as a cosmetic. So I need a scalp model first, then sculpt/paint the braids and feed strands as a high poly model. Then it's baking to lowpoly as usual.

Left is the original, Middle is my repaint of the color map. Right represents the topology that I need to keep, and have blend maps between their model and my final lowpoly (along the darkened strip).

The problem is that I don't have access to their high poly model, so it's a game of avoiding a nasty seam in the normal map, and be able to change UV layout.

Maybe I'm overthinking my problem here and just need a better sculpt?

Ace-Angel

I'm not sure I understand the complex issue here, you have about 300 polies to play around with for the head.

Now I assume, you're going to be deleting the helmet and not use it, right? Meaning PA will have her head exposed, right?

So why not use that 300 polies you have access to, to overlay some strands of poly on the head and bake everything on them instead of the head itself?

Also, I MIGHT be mistaken, but I think Valve allows through FBX importation of your Normal information, so why not use a Normal Modifier, and 'bend' the edges of your new strands, so they look like they're seamless in the head? You don't need to touch the head at that point.

If you're worried about adding 'subtle' details to the head itself to make like the Corncrow's are blending into the head, I suggest testing and see if the head will even support those details, since 256 resolution might now allow such a thing. So a couple of strands of polies could do all the work.

Also, remember that a 'flat normal' color (127,127,255?) could be used to help and avoid transition issues, if the seams don't required any detail on them.

There are plenty of small simple tricks you can use to get what you want, use them.

throttlekitty

The head on the left, with the cowl thing is what I have to work with, there is no bare head under what you see. Before I can think about adding hair, I need to have a working scalp that meets up at the front of the forehead. Her actual helmet looks like this.

Using a 'flat' normal would be grand, but won't work for blending a seam, sadly. There's two sides to that, theirs and mine. I'll post pictures once I redo my sculpt so you have a better idea of what I'm talking about.

I'm not too worried about resolution and the scalp/hair blend, simple gradients with a few thick lines should do the trick. I should test that though, thanks.

Eric Chadwick

Can you line up your vertices with the head vertices, then bend your vertex normals to match the vertex normals of the head? That should solve 99% of the seams, assuming the head normal map is fairly flat-colored there.

throttlekitty

Yes, my baldycap mesh has identical placement of vertices, and matching normals at each point. Their head normal map isn't very flat there, and leads up to a large bump for the cowl. This isn't final low poly or normal map, I want to get the seam solved first, since that area won't change from here on. (and make a real ear lol)


My baldycap sitting on top of the hero's head, verts are pushed out slightly to avoid z-fighting, but verts at the seam area are identically placed. Dota2 cosmetics are placed on top of a given heroes base model, and do not modify it.




Normal map closeup is of the spiky area around the temple. Both images use the hero's original normal map for the face and neck. The image on the left shows my unaltered normal map bake, the right shows basic masking to blend between mine and theirs, no getting crazy with painting in channels yet.

You can see where the the face normals start to bend up into that goofy bump, and I can't do much else about that, except ease the transition to something a little less jarring on my side of the border.


So, back to my question/script. In this layout, I'm stuck with the given UV layout for at least that border strip of polys, right? You can't arbitrarily change a normal map like that. The UVs not terrible, but they're laid out to make the face look good, and puts my model in a funny spot when I add geo to give the rows height and the ponytail. Given the 256x256 map, a more suitable layout is needed.

If I could apply my blended normal map as actual normals to an interim model, then bake that back to a normal map, I would have the freedom to then change UV layout as I wish. Dig?

Eric Chadwick

Is there any skin on the hero head, underneath your edge? Maybe an alpha blend could be used here? Otherwise I think you'll just have to tweak your normal map until the seam is minimized. Or, change your design to cover the seam with a headband or somesuch element.

Eric Chadwick

Oh also, be sure to export and view in the game, as opposed to in Maya alone.

throttlekitty

No, there is no scalp, and a bit of AO in the diffuse leading down from the cowl. This can be seen in an image I posted earlier in the thread. I'll be sure to check in-game shortly.

throttlekitty

// tkColorToNorm by throttlekitty
// Gets Vertex color and applies it as Vertex Normal.
//
// Usage: Apply an Object Space normal map to a high poly models' vertex colors via the Paint Vertex Color Tool.
// Select vertices and run tkColorToNorm.
// **This script temporarily turns of undo functionality to speed it up a little.
// **Recommended to run on smaller vertex selections.
//
// Making a game mod, I needed to blend between original geometry + textures and my own
// while being able to use a new UV layout.  Using this, I was able to blend normal maps
// using the original layout in Photoshop, and turn the pixel normals into vertex normals
// on a highly tesselated version of the low poly.
// From there, I was able to bake these new geometry normals back to my low poly with the new layout.



global proc tkColorToNorm () {
    int $undoState = `undoInfo -q -state`;
    undoInfo -state off;
    
    string $verts[] = `ls -fl -sl`;
   
    for ($vert in $verts){
        float $color[] = `polyColorPerVertex -q -rgb $vert`;
        polyNormalPerVertex -x ($color[0] * 2 - 1) -y ($color[1] * 2 - 1) -z ($color[2] * 2 - 1) $vert;
        }
    undoInfo -state $undoState;
}

Yay, got it working! In case anyone needs it, here it is. Probably more helpful to modders than production artists.

wailingmonkey

just wanted to pop in and say 'thank you!' for sharing your
script, throttlekitty...don't have a direct need for it now,
but may in the future and I appreciate you being a good egg
and posting it! If our paths cross at GDC or some such,
I've got a beer with your name onnit!