Is this a bad UV unwrap? Baking Problems

wclinker

Having some real difficulty here.

I have modeled a power armour helmet from Fallout with some complex elements to it (the two tubes on either side for example). I unwrapped it as accurately as possible but it's now giving me hell with baking a decent looking normal map and with using it in Substance Painter.
I have a high poly which needs some additional details on it but I don't want to touch it until I know what's going wrong and if I need to make my original model more low poly and then unwrap it again.



The unwrap at the moment. Tried my best to fit everything on there but I'm afraid some elements are too small to give a good looking result.


This is the normal map I'm getting from 3ds Max. Any ways to stop the cage from intersecting like that? And why are certain areas completely black?
Should I continue with the high poly then make a new low poly using retopology? Should I separate the UV's onto separate grids to give them a higher resolution?  

Any help would be greatly appreciated!  
(Ps. Smoothing groups are applied to the low poly and the unwrap has a seam on each hard edge).

Ojah

Not sure, but it might be that you're baking with mental ray? If so, try setting the renderer to scanline or set the visibility for the low poly in object properties to 0.0

wclinker

Played around with the render settings and mental ray actually gave me a better result than before. Not too sure what went wrong with the first few tests. 


Any advice for getting rid of the red areas? There's a lot of geometry that intersects quite easily on this model, could I separate it to avoid this problem then compile it into one normal map?

Revel

This is not the normal map though, it's just a preview of your mesh with whatever light(s) that your scene currently has. If you open the image that Max bake, normal map should mostly contain purple color for tangent normal space.

About the red color it's a ray miss, means your cage is not pushed high enough the encompass all your highpoly mesh. You can try to set the cage push value slightly higher and test render that again. Also, if you haven't aware there's a 'shaded' option on projection modifier that you can turn on to better preview the cage distance from your lowpoly to highpoly.

About the black area, well that's just preview so don't worry about that.

Personally I'll stay away from mental ray to bake texture. But that's just me, scanline give a better result with less setting as far as I know.

huffer

- If you want this to be a game ready asset, not intended for film, you can seriously optimize it - bake down all the small elements - keep a simple cylinder instead of all those small parts. It will be easier to do the unwrap, and it would tone down the scale of the piece, making it more fun to work on and finish. You could increase complexity on future assets.
- if cages are intersecting, you should explode the meshes (high and low together), so there are no pieces touching each other

wclinker

This is what I got after adjusting, don't know if it's totally right though...


Thanks for that advice. I didn't realise the render was just a preview. Learning as I go!
I've experimented with pushing the cage but the problem is if it goes too far then it interferes with the cage from another section. Meaning the ray miss is fixed in one area but reappears in another. What are your thoughts on the unwrap? Could it be less crowded perhaps?

JEmerson

wclinker said:

Thanks for that advice. I didn't realise the render was just a preview. Learning as I go!
I've experimented with pushing the cage but the problem is if it goes too far then it interferes with the cage from another section. Meaning the ray miss is fixed in one area but reappears in another. What are your thoughts on the unwrap? Could it be less crowded perhaps?

 Actually I think you could solve this rather easily. Set your timeline to 2 frames. Have frame 0 be the origin with all meshes/elements set to how you would like them to appear in-game (so high poly on one layer set at origin and low poly on another layer and set to origin). Then, activate auto key and key for frame 0 on all elements. Afterwards, move your time slider to the second frame, separate the meshes/elements so that they are still paired between high poly and low poly (same position), but not put together as a game ready mesh (you are exploding the mesh for an "exploded bake"). Set the second frame key for all elements. Turn off keying (and save). Now - go to frame 2 and try pushing your cage again. There should be little (if any) cage overlaps occurring. Keeping your time slider set to frame 2, go ahead and do a bake. See what kind of results that gives. No messing with vertices position, no messing with UV's. http://wiki.polycount.com/wiki/Texture_Baking#Solving_Intersections has some info on this for you.

instg8r

When I used to do my normal bakes in Max (now I pretty much use Handplane the majority of the time), I would "bake to material ID". Instead of having to explode your mesh, assign different material IDs to the different parts of your mesh. I'll then apply a multi/sub-object material with bright colors corresponding to the IDs on the mesh (to help me visualize the different parts I'll be baking). In the Projection Options check the box "Hit Only Matching Material ID" and Max will know to take those IDs into account when baking. It's a slick little workflow and many of the other programs you can bake in (Substance, Hanplane) incorporate a similar technique into their processes. If you have any questions about it, feel free to ask!

wclinker

Thanks for all the advice, appreciate it! I'll play around with material IDs and possibly an exploded bake, see what is easiest and works best. Will post results of anything works!
If I made those two complicated tubes on the sides cylinders, could a normal map give it the impression of having that segmented sort of shape? 

Revel

General idea for me in deciding normal map is that whenever the shape changes the silhouette of the mesh that much, normal map cant do anything about it and you need an actual geo for that. So in that 2 tubes at the side, normal map cant fake the extreme shape. But the smaller grill on the mouth piece, probably you can get away with just normal map. Try to keep you poly distribution even across the lowpoly mesh.

wclinker

Okay got it. I was thinking about doing that with the mouth grill. It would definitely cut down on the poly count. Going to revisit the model and see if I can optimise any other areas as well.

wclinker

Overcame my issues and finished this model! 
Thanks for all your help, check it out at:
http://polycount.com/discussion/182394/t-51b-helmet-fallout/p1?new=1