hey guess what, normal map issues!

Count Vader

Hey guys, so here I was getting ready to start texturing my greatest masterpiece yet, and did a test run of normal mapping just to make sure this exact type of thing doesn't end up happening (ie an apparently unresolvable issue which persists throghout different combinations of baking/rendering software). Well, it ended up happening.

I am also not really technically inclined, and while I found a good deal of documentation online about dealing with smoothing errors, I didn't really get most of what they hell they were saying. Also I watched that video by the guy and that didn't help either!

Anyways submitted for your scrutinization is... my high poly:



Nothing crazy there, All one piece, very simple vehicle chassis.

Exhibit B... the low poly:



I put it all into one smoothing group. Everything seems to bake ok. Here is what the normal map looks like:



Yay its a normal map great


Ok, here is where the trouble starts. Example from max viewport is down below:



Please note the ugly a shit bands of shadow cascading down the wheel well, as well on the front where the turrent nest is. Puke!

Also the highlights/shadows coming from the corners of that indented bit on the side have a weird conical/radial quality to them, certainly not the flow I was going for in the high poly.

I know 3ds max has its issues when rendering normal maps in real time, so here's a marmoset screen cap for good measure. Even uglier




Can someone please help?

Am I just an effin dunce who doesn't 'get the technical limitations of what a normal map is supposed to do' or am I screwing something up hard core?

I'd really like to move on with this project, but I can't start painting the texture the the normal maps look so abbysmal!

Also please note I did attempt to use xnormal to generate the normal maps as well, result was more or less exactly the same.

For good measure, one last shot from max 2012 viewport with spec turned up:

odium

Smoothing groups dear boy! Make sure that each uv island is its own smoothing group. Your model looks faceted on the low poly, which is why it's looking so strange. See the other normal map thread a few threads down.

Count Vader

Thanks for the response, any way to assign smoothing groups based on pre existing UV chunks? Or do I have to manually select the groups in edit poly mode?

poopipe

If you have a face selection active in unwrapuvw it will be maintained if you add an edit poly modifies

Count Vader

ah, so it will! thank you! Will check back soon w/ result!

HAWK12HT

all you have to go through is this tutorial.

http://wiki.polycount.com/3DTutorials/Modeling%20High-Low%20Poly%20Models%20for%20Next%20Gen%20Games?action=show&redirect=3D_Tutorials%2FModeling_High-Low_Poly_Models_for_Next_Gen_Games

and please a request dont ruin this Beast APC from Aliens i may kill my self .

Count Vader

haha oh dude trust me, if i fucked up this badass design i'd hang myself in my closet...

HAWK12HT

I wont model a high poly for that faceted APC neways cause these days u can use bevel and chamgering on models without going overboard with polys. You wont have low poly but rather a mid poly.

http://speedhunters.com/archive/2011/11/17/behind-the-scenes-creating-cars-need-for-speed-run.aspx

As for that nasty black seam underside well.. im sure you can add the middle section that will hold wheels and a passage that leads to driver seats, as far as i know Bishop got in APC from main side door and not from somewhere else. This will get rid of your black seam right way.

And Smoothing groups work best on 45 degree angles not on 90 (APC) thats why you are seeing too many artifacts.

Hope it helps

HAWK12HT

oO oh dont do that, Allow me to do that for ya :P (i mean closet thing)

Count Vader

haha yeah, sadly I already put the time into the high poly! I needed hard surface practice. If you'd like to see, there it is. Hopefully it'll give you some faith that I didn't fuck it up yet :P

Count Vader

Also, in the first tutorial link you posted thats on the Polycount wiki, they reccomend putting everything into one smoothing group! Which is what I did too, but the guy in the tutorial sheet didn't end up with those same stupid shading artifacts

Oh can you please expand on the black seam issue further? I split up the UV islands into their own smoothing groups, which lessened the shading artifacts but is now generating those ugly jagged edges at the UV seams!

Quack!

Yes, always split your smoothing groups by uv islands.

So you split the smoothing groups by uv islands then rebaked a new normal map?

What was the padding on your bake settings? I don't know if you padding would come into play here.

HAWK12HT

I had no doubt on your abilities even befor, just cause you have courage to ask for a solution to a problem .

As for 1 SG (smooth group) for whole mesh its not quite true, and your APC is an example of it.
I would recommend you divide your UV layout like this "Top, Left, right, front, back, bottom" as separate UV islands, cause these are the area that are full 90 degree angles.
I hope u get what i mean .

HAWK12HT

then apply unique SG to each side as well. like 1,2,3,4,5,6

Trust me it will work, dont worry about all being one SG its not always possible unless you follow bevel, chamfer workflow on low poly (which will make ur APC as mid poly) only than you get nice results with one SG as if you have applied Turbosmooth.

EarthQuake

1SG *can* work if your entire workflow is synced up, ie: your baker and renderer using the same tangent space. Out of the box this isn't the case with Max, and it isn't the case with most game engines.

If you're working on this model just to display in Max viewport and not send off to a game engine, you can use 3Point Shader with the quality mode modifier which will sync things up and give you perfect results even on hard angled geometry like this.

If you're sending it off to a game engine like UDK, you need to make sure your lowpoly mesh normals look as good as possible before baking, this means using extra geometry/chambers, using smoothing groups etc etc to even out the smoothing. Just make sure you have a uv split everywhere you have a hard edge(smoothing group boundries) so that you don't get bake artifacts.

Even with a synced workflow its good to have hard edges along your uv borders, for a variety of technical reasons(better compression, easier to extract "detail maps" from crazybump, etc).

At the end of the day there is no "yes" or "no" answer to "can I use one smoothing group for my entire model?". Anyone who tells you "always" or "never" doesn't understand the situation.

Count Vader

Hey thank you all for the responses, you guys are awesome

I'm still putzing around trying to find the best compromise between low/high poly to eliminate smoothing issues. In the meanwhile, I'd like to address some of your individual comments.

@HAWK, you said to assign each 'side' of the model to a smoothing group because that's where the 90 degree angles happen. However with this model, there is that wide chamfer that runs along the top/bottom and interferes with any 90 degree angles that would take place. IE:



Sorry if this isn't even what you mean, I just wanted to double check

@Quack! Yup I split up the smoothing groups based on UV chunks, it sort of fixed the shading issue, however it resulted in ugly ass jagged lines where the seams come together. I didn't specify padding while baking, however I did move the UV islands apart from one another so they didn't intersect or touch in any way.

EDIT just realized theres 90 a degree angle within the circle I drew haha But I'm talking about the primary forms here, there are some smaller details that are 90 degrees. What I mean is the sides of the main chasis don't intersect @ 90 degrees.

@EarthQuake, this is a portfolio piece, it's going to just end up being a viewport/possible UDK render, it's not actualy going into a game. However I want to sell it as being passable for game usage.

At this stage I'm just considering texturing my current mesh without even bothering to make a low poly, just maybe cleaning it up a bit but leaving all the control edges and stuff. Currently the whole thing is ~200k triangles unsmoothed, is that too insanely high to pass of as a 'game ready' model?

EarthQuake

Count Vader wrote: »

is that too insanely high to pass of as a 'game ready' model?

Yes, and you're losing out on the very valuable learning experience of dealing with these problems and coming out with a working asset.

Try not to take the "90 degree" advice too literally, in reality you'll get smoothing errors with angles much less than 90 degrees even, the important thing is to understand that the more extreme the normal shift is, the more extreme the smoothing errors will be. To get them down to a tolerable level, you need to use hard edges or more geometry.

HAWK12HT

HAWK12HT

Yes EQ is right, dont take all info here as "literal" im sharing my experience and how i went about solving it, but im sure if this would be my model id do this same method and it will work . I dont wanna make you worry about another issue which comes with breaking UVs, and that is Normal seams when zoomed in way too close. Check out some games out there with sniper rifles "Crysis 2 map Sancturary" is an example (im not being picky just explaining fellow artist stuff so dont start a debate Crysis=Cryengine vs UDK). The walls look super detailed but if you zoom in with scope or actually jump on walls you can see a black line running in between 90 degree pieces. There is a whole lot info about this seam thing here and why it happens.

Back to topic i hope pic above gives you the idea of what is in my head,

Ace-Angel

Why is it 200K? To me, it looks like a chamfer at 2/3 divides would be enough for a model like this (if you're trying to make it look REALLY good), and it wouldn't pass the 10K at most.

Sorry if I missed something along the line in the thread, but I don't see anything indicating why you reached this figure. Mind showing the wireframe of your low-poly?

EDIT: OK, stupid me, you're talking about the high poly being cleaned up, which as EQ said, it counter-intuitive since it doesn't create a game ready asset.

Also, normals do all the work for details like the recess on the wheels, etc...you create the bulk and bake your normals on that, just as an example.

If you want to show you high-poly next to your low-poly, that is fine too.

Count Vader

GOOD NEWS EVERYONE, I implemented the advice of assigning each major plane of the vehicle to its own smoothing group/uv chunk, and got a pretty good result (shown here with a basic AO/skylight bake applied as well, scanline render)



Look good right? The normal seams i was complaining about before turned out WAY WAY less dramatic than last time. When I did this before, they were really noticeable and ugly, in this case as Hawk said they're only visible if you're zoomed in very close.

I may yet try to do the chamfered/1SG thing as well just to see how it looks, but realistically I just wanna start texturing this thing.

Here it is with wireframe if anyone is interested:



So thanks again for your help yall, I've learned a lot.. this started out as a HUGE headache since it was the first hard surface object I've attempted to normal map. If anyone has any closing crits/ advice i'd love to see them!

EarthQuake

Keep in mind that Max's scanline renderer is synced with Max's baker, so scanline renders will always look awesome - which is a big part of the problem.

Viewport shaders in max on the other hand....

Anyway, as Ace was sort of suggesting above, beveling/chamfering some of these shapes along the sharp edges will get you pretty far too, and not use up thatttt much geometry. Being the main element, with a pretty simple shape, you can easily afford to treat it well.

Count Vader

Yeah, I've heard max viewport renders its own bakes in messed up ways. I guess this begs the question, what can I consider a more 'objective' preview of my normals - the renderer because it uses the same tangent space as the baker, or the viewport shader because it's 'in real time' ?

Quack!

Count Vader wrote: »

Yeah, I've heard max viewport renders its own bakes in messed up ways. I guess this begs the question, what can I consider a more 'objective' preview of my normals - the renderer because it uses the same tangent space as the baker, or the viewport shader because it's 'in real time' ?

It all depends on what game engine you want the object to go into. So there is no real answer to that unless you specify which game engine you want to use. If you don't want to take it into an engine, do as EQ stated and use Quality Normals with 3point in the max viewport. However, I would highly recommend that you take your objects into one of the major game engines to prove that you understand those workflows.

HAWK12HT

YEAS! Told you it will work. Now do as others mentions go for full game engine import export thing and see where it takes you

Count Vader

Quack! wrote: »

However, I would highly recommend that you take your objects into one of the major game engines to prove that you understand those workflows.

Thisi s true, I haven't even delved into udk yet, but I figure I'll start presenting my models/assets in that, since obviously I need to develop a grasp of it anyway.

EDIT: I don't wanna spam this entire subforum with my normal map issues, so i'm just going to tack this on here and someone maybe will have an answer. I'm trying out this trick: http://www.polycount.com/forum/showthread.php?t=71995 Which requires rendering in mental ray, and I normally do normal maps in scanline. Anyone know why my normals come out of MR looking like this:

EarthQuake

Honestly I wouldn't waste my time with that stuff, I mean it works for very simple assets, but you're probably better off spending that time focusing on making a great asset the old-fashioned way.

Count Vader

Oh yeah, there's no way I would consider trying to use that trick as a substitute for regular HP modelling, it's just there's a particular piece on this APC that it would be the perfect candidate for, which is this turret in the front:

In all the reference pics I've seen the entire turret looks welded together as one piece, which is how i built the high poly, but it's proving a huge pain to unwrap/map the low poly version while still retaining that look of the turret barrels being integrated into the cylinder.

I just figured that mental ray trick is the perfect candidate to make the low poly of that turret easier, since I can just stick two cylinder into the main body of the turret and use that trick to render a normal map that makes them look like one piece.

Ace-Angel

I tried that bevel-technique not too long ago, on an asset that I had made which was kinda like your turret.

Don't know why, but both Scanline and MR didn't work, they either didn't bake anything (as in the process didn't start) or the map looked like the one you have.

The paradox is that when you render in MR with Fillet edges, it shows up correctly, just not as a bake, I guess they mixed a couple of wires in max 2011, etc...

Count Vader

right, well that post is quite old, so they've probably changed some stuff around since then.. shame though, would have been a pretty useful trick

artquest

Not that I'm looking to hijack your thread or anything :P but... since we're on the topic of normal maps.

Does anyone know what would cause me to have to flip my green channel to get proper lighting in a game engine? In maya the applied normal looks perfect however in game as the object rotates 360 degrees part of the time the indented areas appear bubbly(I also had some nasty seams). This problem was solved by inverting the green channel of my normal map.

I'm just wondering what is causing this? Should I be looking at my low poly normals, high poly normals, the tangents or is it something in the shader used by the game engine?

At least it's an easy fix if anyone else encounters this problem.

EarthQuake

There are two different "types" of normal maps, left handed and right handed, or -/+ green channel. You can adjust the "handedness" in maya and rebake(or set up a default) to avoid the whole issue. In the object properties or something. You can also set the "swizzle" in xnormal for the same effect, and somewhere in Max's bake options there is a place to change it too.