Normal Mapping in Max 2012
I've just come out of a Game Design course and feel hugely underqualified, and so I'm taking modelling into my own hands and seeing whether I can learn as I go. A few days with normal mapping would suggest "probably not". I've followed just about every normal mapping tutorial on the internet - it always works in those scenarios, but when I apply the same steps to my models it never even slightly works.
So I'm working on a basic U-boat to try and get a good working pipeline down, so I'm not too fussed about detailing or any of that - I'd be content with a textured, normal mapped, low-poly game model.
What I'm doing at the moment is experimenting with normal map baking - for the purposes of this experiment I've given the right hand side of the U-boat a planar, X-orientated projection.
Here's the high and low poly models occupying the same space:
And here it is with a shaded projection cage:
The render frame window (which apparently shows that the right side has baked quite nicely):
But then when I go to the output directory, I come across this cannibalisation of a normal map:
Just to be sure, I checked it when applied to my low-poly with a simple right side diffuse:
As you can see, it's just terrible! Whatever I do with the geometry or with the cage, it always finds a way to do something similar to the image shown above. Am I doing something severely wrong?
I was initially extremely excited about modelling and texturing, but ridiculous errors like this are really causing me to reconsider whether it's the industry for me - it seems that every model I make ends up getting binned as soon as I get to the normal mapping stage, since it inexplicably won't bake properly.
Excuse the negative tone, getting slapped in the face with exactly the same problem for every model I make is hugely demoralising. Thanks in advance for any advice or help.
Rob.
So I'm working on a basic U-boat to try and get a good working pipeline down, so I'm not too fussed about detailing or any of that - I'd be content with a textured, normal mapped, low-poly game model.
What I'm doing at the moment is experimenting with normal map baking - for the purposes of this experiment I've given the right hand side of the U-boat a planar, X-orientated projection.
Here's the high and low poly models occupying the same space:
And here it is with a shaded projection cage:
The render frame window (which apparently shows that the right side has baked quite nicely):
But then when I go to the output directory, I come across this cannibalisation of a normal map:
Just to be sure, I checked it when applied to my low-poly with a simple right side diffuse:
As you can see, it's just terrible! Whatever I do with the geometry or with the cage, it always finds a way to do something similar to the image shown above. Am I doing something severely wrong?
I was initially extremely excited about modelling and texturing, but ridiculous errors like this are really causing me to reconsider whether it's the industry for me - it seems that every model I make ends up getting binned as soon as I get to the normal mapping stage, since it inexplicably won't bake properly.
Excuse the negative tone, getting slapped in the face with exactly the same problem for every model I make is hugely demoralising. Thanks in advance for any advice or help.
Rob.
Replies
Right now it looks like the problem is coming from the planar map and that both sides of the sub are present in the same space and they overlap. There are three solutions:
1) Each side has its own unique space and nothing overlaps and everything fits inside the renderable area.
2) If both sides are the same you can unwrap half and them mirror it, and move the UV's for the mirrored half one tile to the right, outside of the renderable space. You do that by selecting the extra shells and in the lower right turn on the coordinate system and in U type 1 and hit enter, then turn off the coordinate system.
3) Select the back side of the sub, detach that UV shell, switch the editor to UW mode (top down view of your UV editor) and move that shell behind the the other, right now there are probably parts that poke through and it needs to be fully behind for the correct shell to fully render over the top of the back.
I followed both steps 1 and 3 (the sides aren't quite symmetrical), and this is the map I got:
I didn't detach the UV shell (don't know how), but assume that the blue border in the UV editor means it's detached.
Aside from the random array of 3 squares at the top (that, according to the UV Editor, don't exist), it's still just as bad.
Also have you got it as 'shell' ie is the submarine hollow like an easter egg?
if so that could be the problem. You would have to map the internal shell sperately also
If you're worried about it not working, you should start on just a small part of it and figure that out first.
I reset the Xform on both models and remapped/baked. Here's the result:
It's certainly cleared up the crazy artifacts from before, so it's more progress than I've made so far. But then the new map isn't much more practical than the old one in terms of it showing up correctly!
As a side note, what exactly does reset xform do?
Fixes everything! That's all we know
I think it's kind of the equivalent, or at least similar to 'delete history' in maya. So it resets the scaling and rotation etc, and deletes any other odd/leftover information that might be in there.
I know how to pelt unwrap in UVLayout, but I'm trying to develop my Max skills - I don't think I'll be too employable if I'm reliant on a wide array of third-party software for my work pipeline.
A further part of my problem is that I'm building the diffuse textures from orthographic images of the submarine - I wouldn't know where to start with pelt/cylinder unwraps, in terms of getting detail in the right place on the model. But then that's a whole different bag of cats.
A thing i learned is delete the mirrored half, then bake it all, then mirror over the geometry.
Also check your smoothing groups, if you cant be arsed, well atleast run Textools smoothing by Uv island script.
If you are playing around with xform then dont forget to check your normals with the normals modifier, they can be flipped from the xform.
It takes time and its really boring ( basicly what i have been doing all day now....) But after a while it makes sense. This technique gives you alot of control, and you get an idea where you should place the seams. I might be really stupid for doing it this way but all i care about is that it works.
[ame="
In the meantime, if anyone knows of any better methods of unwrapping in Max, I'd be very appreciative - I can see how the very precise level of control would be useful for laying out shells, but there always needs to be a balance between precision and time efficiency.
For your example, take all the noodly small bits to one side temporarily, and break apart the big shape into various pieces, relax them all flat, and then go through the small bits, split their seams in the relevant places.
Wait a mo, I will go and work out how to make a clip of the process...
Yes this is exactly what I do(and I think most people in some form or another) in modo, but there are a few automated tools for it, you just select your edges and run the unwrap tool, which has a few options for unwrap methods, and then I relax in addition to even out texal density etc.
It's not so much that I want to cut corners or anything, I understand that time spent usually increases the quality of the output - but taken to the extreme, in modelling terms, you could go as far as to position each individual vertex manually and then construct the polys from them, but that would take a ridiculous amount of time.
I think I probably phrased it slightly wrongly - I'm not asking for "the quickest" method or anything, more the method which would be considered "industry standard".
Would love a vid.
You seem to be saying that you wnat good advice and when it is given , you are asking for other ways to do it?
None of the methods that have been posted take 'that' much time or effort.
Eventually when you realise you are not getting the results you want you will have to go and do it properly anyway:)
You should never see uv mapping something that you quickly do at the end.
Personally I really enjoy laying out uv's but then I am a bit involoved
I like to be really sure of the steps of a process before I start it, to minimise how much has to be re-done. So rather than asking for other/quicker ways to do it right after being told one, I'm more looking for verification. For instance cw detailing his process, followed by EarthQuake saying that he and most others follow a very similar method, allows me to move forward with much more confidence than if I'd only heard the steps from cw with no further comment.
I think I may have given the wrong impression by planar mapping my model in the original post - I've always pelt unwrapped prior to this model, so this is very much stepping into unknown territory for me!
I do find the time-consuming parts of asset creation quite satisfying and rewarding, just less so when I have to do it to the same model several times over.
I also select groups of faces and hit quick peel which detaches, unwraps and relaxes that chunk. Which is another method I used in the past but manually relaxed the pieces.
Also, if I have plotted out my smoothing groups already then I know where I want some if not all of the UV seams, so I explode by smoothing group and then stitch pieces together or detach bits to arrange them better.
Loop + Stitch, Loop + Stitch, wash rinse repeat.
note to self, make more clips, they are fun-ish.
http://vimeo.com/30757635
Particularly pleased to see that there's a packing feature in Max.
You're right about the 3dsmax tutorials being outdated though! It's easy to forget how dusty that stuff is!
Further question: Half of the time when I open the UVW Editor, it doesn't let me use the freeform scaling handles - if I close and reopen the editor, it usually works. Similarly, sometimes when I apply the Unwrap UVW modifier, it doesn't allow me to select any of the faces/edges, and seems to refresh the modifier panels whenever I attempt to select something.
Are these signs that I've done something wrong, or are they just something inherent to Max 2012 that I just have to deal with?
One more thing - the green seam lines on the video seem really bold and clear - when I select and break UV edges, I can barely see the colour of the line since it sort of merges with the wireframe:
If you where in a sub-object mode like (polygon or edge) when you applied UVW unwrap, it will only operate on those faces you had selected.
THis happens with every modifier, whatever you have selected gets passed up the stack. If you want the modifier to be applied to the whole thing you need to exit sub-object mode, either by clicking the modifier label (name next to light bulb) or press 6 (with keyboard shortcut override turned on).
As for the UVW window, freeform was definitely selected, the cursor simply didnt change when moved over a handle - as I say, a close/reopen fixes it. I sometimes get a similar thing in the main Max window too, on random occasions finding myself unable to select/deselect things while modelling with Edit Poly. Again, save>close>open>load fixes that one.
What I have noticed is that Unwrap UVW REALLY doesn't like it when you go down the stack to Edit Poly to tweak smooth groups or geometry.
Switching the max display driver back to DX9, OpenGL or software mode clears up a lot misc issues for a lot of people. You do that by going to Customize > Preferances > Viewports Tab > Choose Driver and Configure driver.
Edit: Switching to DX9 fixed that weird UVW seams melting in with mesh wires, I have nice bold green lines now
Nice share!
I would personally trow a cylinder projection to those middle polygons tho.
I'm sure it will help many, cheers!
UV'ing can look complex, but it's very simple, it's just very time consuming. I also want to mention the power and useful ness of the stitch and the unfold mapping!
Xform doesn't fix everything. It's supposed to reset all positional, scaling, and rotational data back to zero. It doesn't always work.
What I usually do is create a box larger than my mesh and I avoid any scale, position, or rotational changes. I then attach my mesh to that box. Then in element mode, I delete the box. Done.
Normally if you just want to reset the transform data and zero it out without messing with the pivot point you can go into the hierarchy tab > reset transform > reset scale.
Edit: So I've unwrapped the whole model to a point where it's nicely laid flat (first time I've fully done it in Max, yay!). There's a bit of a problem though. The length of the main piece of it in relation to the size of the square UV restricts the scaling of the rest of the pieces, and so I end up wasting half of the UV space:
Should I split the main body of the sub into more pieces?
Edit 2: The fun isn't over yet! I figured I'd see if the normal map baked alright, to see if the problem from my original post had settled down. Result:
And I've taken an orthographic render from the side, with the normal map in the diffuse, to see how it compares to my most recent planar mapped attempt:
Funnily enough, it's almost exactly the same! Oh joy...
1) that bake is knackered - what's the high poly like? Something is really messed up there.
2) protip, textures can be rectangular, usually. If you really want square texture, just chop up that main body. As an excercise why not do that anyway!
also, when relaxing, do it by face angles if you can, it will usually make more symmetrical uv chunks, there seems to be some bits on there which are wonky. (the holes in the fuselage thing too, they are all scrunched up)
good luck - keep at it!
Here's what the high poly looks like in the viewport (not sure if I can get the wires visible on a render):
As for rectangular textures... mind=blown! I always thought they had to be square! What do I do to get the UV Editor to think in rectangles rather than squares?
I thought I was relaxing by face angles throughout, though I noticed the editor does like to default to edge angles - it's probably just caught me out once or twice, I'll go over them again.
edit: nvm just me beeing an idiot...have you tried baking in xnormal beacuse i cant really figure out what you have done here. have you rendered a version with raymiss check on? under the projection menu?
Dude tell me about it.
Im kinda in the same boat. I graduated University 3 years ago now with a 2:1, and felt like I hadnt even scratched the surface of games design...feeling well underqualified I've still not applied to a single developer...even though I studied it for 3 years.
To be honest my work back then was awful, and I wouldnt put any of it in my folio...so I'm starting from scratch and building my artwork/knowledge up.
Anyway, what I'm trying to say is keep going. Subscribe to a site like 3D motive maybe? Start out with the basics....follow tutorials until you can create your own pieces. Its better to get a lot of easy models finished than to go big straight away and never finish loads of complicated stuff. (Something I'm still trying to learn). :poly127:
It can also be a little tricky baking from high to low in max/maya. Try XNormal or NDo2 if you aint already? I know its good to get a workflow down but these are great programs and can be huge time savers. Although I'm determined to get Maya's transfer maps as my main normal mapping source!
Its the little victories that keep you going. Good luck.
Also, what school did you go to? I'm wondering if we went to the same school since they didn't teach you normal mapping. I went to Brown College by the way.
I went to Leeds Metropolitan University. Never mind normal mapping, they barely taught us anything - most of the stuff I handed in was self-taught. Because of that fact, I actually wrote a research assignment in my final year on normal mapping. I even wrote a step by step tutorial for baking in Max. That's partly why this issue is such a source of stress and frustration for me!
I had a feeling that might end up being the case... That would be awesome, because I'm really out of ideas on this one. Here's a link:
http://www.multiupload.com/DL25U0J730
Btw, you prolly already know this but you should watch this if you're learning baking in max: http://cg.tutsplus.com/tutorials/autodesk-3d-studio-max/how-to-bake-a-flawless-normal-map-in-3ds-max/
look in the menu where it says push and amount.
odd though, even without adjusting the cage at all, ie resetting it, I still don't have the problem that you had. the majority of my bake is blue?
And that's really odd that it works ok for you...
Edit: Newly inspired by the concept that it definitely has the potential to work, I tried a couple of random things.
Notably, switching to scanline renderer makes it come out ok:
Original problem solved! But this raises a question - was I really that wrong to use mental ray for map baking? I thought it was a far better renderer for all purposes!
But then I just tried rendering it 256x256 and it has the same issues! Perhaps I need a reinstall?
It messes up the padding, it incorrectly assigns the background as the environment color even when the correct color is set as the "background" in RTT. It also takes longer and unless you crank up the Final Gather settings which increases the render time it will spit out worse results that are blurry.
You also can run into a crash issue if you jump between mental ray and scanline too often in the same session...
About the only thing mental ray can do is bake AO... which is a bit of a gamble because it also fudges up the padding. Just about everyone I know uses scanline, its fast, efficient and works great. A lot of people use a custom light setup and bake AO in scanline also, I know I've had to resort to that when MentalRay won't create the AO correctly or doesn't give me the control I want.
If you're going to spend some CPU cycles on a higher end rendering engine, I would choose Vray, but even then what you get isn't really worth the time spent.
Statement of the thread! Good to know, I'll stick with scanline for map baking and mental ray for model rendering from now on.