Issues Baking Normalmap
polycounter lvl 10
Hey everybody. I'm relatively new at this. Far enough along to start getting into the complicated issues, but not quite good enough yet to get them right.
I've recently started UVing and baking normals from high poly meshes. For the last week, I've been spending all my modeling time tweaking settings and rebaking the same normal over and over again. I've made some headway in that time, but still can't quite get it perfect. I figured it was time for me to head over here and start shooting questions and asking advice.
I'm working on a simple shutter. A big square. Nothing that'll make anyone gasp and ooh in wonder. I've got my 74 poly low res mesh, my overly complicated 2 million poly high res mesh I've been weathering in Zbrush, and my diffuse. For reference, here they are:
Low poly shutter with diffuse and UV
High poly mesh in Modo
After paying a bit of change for Luxologys alleyway tutorial and putting what I learned there into practice, this is the resulting normalmap.
As you can see, the thing is rife with unwanted bumps, smears, and seams. It's the closest I've come to what I want, but still nowhere near as nice as I'd like.
I could use a few pointers here. What exactly am I doing wrong, what can I improve? Any help will be greatly appreciated.
Edit: I got it! All I had to do was highlight the geometry I wanted to paste into the other UV instead of cutting the entire thing raw. Kind of a simple thing, and I feel like a total idiot for not doing it beforehand...but hey, live and learn.
Thanks for the help everyone.
I've recently started UVing and baking normals from high poly meshes. For the last week, I've been spending all my modeling time tweaking settings and rebaking the same normal over and over again. I've made some headway in that time, but still can't quite get it perfect. I figured it was time for me to head over here and start shooting questions and asking advice.
I'm working on a simple shutter. A big square. Nothing that'll make anyone gasp and ooh in wonder. I've got my 74 poly low res mesh, my overly complicated 2 million poly high res mesh I've been weathering in Zbrush, and my diffuse. For reference, here they are:
Low poly shutter with diffuse and UV
High poly mesh in Modo
After paying a bit of change for Luxologys alleyway tutorial and putting what I learned there into practice, this is the resulting normalmap.
As you can see, the thing is rife with unwanted bumps, smears, and seams. It's the closest I've come to what I want, but still nowhere near as nice as I'd like.
I could use a few pointers here. What exactly am I doing wrong, what can I improve? Any help will be greatly appreciated.
Edit: I got it! All I had to do was highlight the geometry I wanted to paste into the other UV instead of cutting the entire thing raw. Kind of a simple thing, and I feel like a total idiot for not doing it beforehand...but hey, live and learn.
Thanks for the help everyone.
Replies
1. don't use modo for baking out normalmaps if you can avoid it, max would be your best option and after that you have 3rd party tools like xnormal and I think crazy bump.
2. unless this is an important asset in your game, I would get rid of the small window thingies in the middle, and make the whole lowpoly 1 box with a hole in it.
3. if you're doing hardsurface works, you can either bevel the edges like you did, or split the uv's where smoothgroups would be, I usually do the latter which alongside max and smoothgroups works quite well. (I wrote an article about it)
I wish I had easy access to Max. Every tutorial I've seen or read shows it producing beautiful normals. Unfortunately, I'm stuck choosing between Modo or a 3rd party app.
I've tried Xnormal. Wasn't too terribly impressed. Granted I just ran the high and low through without much tweaking, and didn't bother with cages at all. I guess I'll be doing some reading up on it in the near future.
It'll be something people stare at up close, so I wanted to keep it fairly detailed. Plus, what you're looking at above is just half the model. I've already modeled and detailed the fins that fit inside the shutter body, but wanted to bake them separate to save on resources.
I'll check that out. Thanks.
One question, though. I have heard Modo can do something akin to Max's smoothing groups. If I were to stick with Modo instead of running things through Xnormal, could I achieve similar results to what I could get out of Max?
If you were to seperate the geometry manually - that would seperate the "baking cage" and only make things worse.
Sucks that Modo has no decent hard/soft edge or smoothing group support, I'd have thought that was pretty vital for anyone wanting to do game art work in it. Lightwave suffered from the same drawback, I don't understand it, since the LWO format actually supports vertex normals and smoothing groups, it's just LW and Modo have no way of actually setting this data inside the application...
From what I heard (never tried): to mimic smoothing groups you can just cut and paste the desired parts so they aren't a continuous mesh. Supposedly it is what smoothing groups does (except in smg you can assign different angles)... and yes it generates more vertex data since things aren't joined together (but I believe smgs do that too).
Correct me if I'm wrong
Actually it has the same vertex data, since smoothing groups split the normals along the hard edges.
The nice thing about 3ds max is that the cage is uneffected by the smoothing groups, so you can get really nice interpolation along the hard edges.
After the normalmap is baked, splitting the mesh will achieve the same result for the lowpoly, which is handy if you happen to work with an engine that doesn't support smoothing groups.
Well, I've followed some of the advice I learned in this thread and actually ended up getting some decentish normalmaps out of Xnormal. They're not quite as detailed as what Modo was putting out, but I don't have smoothing errors smeared all over the final results. Kind of a two good for one bad trade off. Truthfully, I didn't need that detailed of a normal to achieve the results I wanted, so it's all...well...it's alright.
I've also downloaded the Max 9 30 day trial. I'll try it out, see how it goes.
My problem I'm facing now is combining my UVs. I've cut and pasted UVs from one map to another without any problems before, so I have no idea why it's giving me problems now. This is what happens...
This is the final model. I've highlighed both UV maps to show how I want it to look.
What I intended on doing was going into the fins UV, cutting them, and pasting them into the shutter UV. Simple as that. Like I said, I've done it before. But when I try to do it, I get this, or occasionally this, instead of the intended results.
I dunno what it's doing there. It ain't doing what it's supposed to, that's for sure. The first deletes all the base shutter geometry from the UV and only paste bits and pieces of the fins, the other looks like it jams all the geometry into the fins.
Anyone have any idea what's going on here?
In Max the latter happens automatically, the two UVs are combined as well. The former is fairly simple too, add a UVW Unwrap modifier set to channel 1 and another set to channel 2, then select some faces, Copy, select the same faces in the other modifier, Paste.
but still give xnormal a better try, as mop said, you'll get better results
I believe you could set smoothing angle per material in lightwave, maybe it's the same for modo ?
I'm not quite following the terminology here. By channels, do you mean something akin to layers, or two separate maps? I'm in the latter territory.
Originally both the fins and the shutter were a single model on a single layer, though they were two separate objects. That worked fine up until I started baking the normalmaps. Both Modo and Xnormal would produce errors where the two intersected, so I separated them both into different models and baked each alone to try to get rid of the problem. It worked. It rendered faster, produced better results, all seemed well. But now, even after recombining them back into one model, I can't get the UVs to combine along with it. I'm now facing one model with two maps.
I've tried everything I can think of to fix this problem. And every single try nets the same two results you've seen above. Like I said before, I've cut and pasted UVs before and didn't have any issues. Only now has it started being difficult with me. The worst part is I don't know if it's a problem with the model, or if it's Modo flaking out on me.
I guess I could always redo the UV. But I don't want to do that until I have no other options.
an easier way to do it is to keep it one model but move the pieces apart
from each other (a reversible numerical distance is best), bake the single
normal map, then move them back. This prevents the intersections.
Gray is the final low-poly mesh. White wireframe is a copy of it, with the
parts moved away from each other. Colored pieces are the high-res meshes.
I moved the white wireframe mesh way from the high-poly meshes so it's easier
to see what's going on, but it should overlap the high-poly exactly for the bake.
When you do that you wind up missing a lot of the AO where those pieces would normally join, which isn't always ideal. What I will do in Max is build/duplicate a "shadow proxy" that I'll fit around each projection half - IE I'll make a shadow proxy copy of the purple half and sit it on top of the red half, and vice versa, but won't add those to the projection - the net result is your normals project fine, but you still get all the juicy AO bits you would get from a full mesh, and in one bake.
I managed to get everything recombined last night. All I had to do was select the polygons I wanted to cut and paste off the model, then use the Cut UV command. Beforehand, I was just using Cut UV on the whole map without highlighting anything.
Kind of a simple thing. I'm surprised I didn't try that sooner. But hey, live and learn, I guess.
Next thing I'm gonna try and do is figure out what the hell you're both talking about above, and see how much of Max I can figure out in 30 days. Thanks for the help everyone.
P.S: I'm sure I'll be back asking more stupid questions soon enough.
Actually, that can be ideal, if you really think about it.
When you bake those sort of low freq shape details out from a highres onto a low, you get shadows that dont actually match up to the shape of the lowpoly.
For example, you have a cylinder intersecting a box, and in your highpoly that cylinder is perfectly round, but in your low, that shape is only 8 sided. So what you get in the final result is this weird circle that doesnt actually match up with the shape of your lowpoly, and looks worse IMO.
Now how i get around this whole issue, is bake 2 seperate ao passes, one very quick one from the mesh put back together, you know ao from just the lowpoly, and use both of those maps in my texture.
I go through the entire process here, its actually simpiler than it seems.
http://wiki.polycount.net/Ambient_Occlusion_Map?highlight=(ambient)#head-c6f5bcc16779aa52a37c5a2739157459a39ca00b
Johny, I'll try to post something later. I'm at work right now and not feeling well so I just want to muscle through this thing I'm working on and book for the day.
If it helps, these are the notes I scribbled down in MSpaint one evening a long while ago after I had shut everything else down and then figured it out: