Hardsurface Normal map problems

JohnnyRaptor

Morning guys

im not usually doing a lot of hardsurface stuff, so iv run in to this problem i cant seem to sort out.

image of the problem,



normalmap



Its all modelled in 3dsmax. I've generated the normalmap with xNormal. Mesh is triangulated. iv tried what iv heard mentioned around the boards, no smoothing groups, 1 smoothing group, split UVs at hard edges, etc.

I know it has a lot of bad spots, but the main issue is that black blotch that i cant seem to get rid of!

i cant really add any more edges to the mesh, but it cant be that the mesh is just too low? im sure iv seen awesome normalmaps even on lowpoly crap like this.

any help or tips would be extremely appreciated! cheers!

Spitfire

If you have gone through all the standard procedures when baking hard edge stuff, you can copy your lowpoly and try to subD the copied lowpoly mesh one division without smoothing on, export that model into xnormal, bake it l and use the map on the lowpoly original without subD


Has worked for me a few times before.

Also be sure to try different programs to view your normalmap in, since 3dsmax is known to show minor errors on the surface

Dont forget, you can also paint on your normalmap in photoshop, go into photoshop and look at your normalmap and see if you see the error in there, if not, its probably the 3dsmax viewport

Mark Dygert

What are you using to display the low with the normal map? What does it look like if you click render. I guess there's a really good reason why you're trying to bake such a wild complex shape to something so flat?

Have you tried baking it in max?

cw

In max try the 3pointstudios shader, it should display flawlessly if you're assets are correct. If it doesn't you need to change your workflow a bit and rebake.

JohnnyRaptor

the screenshot is from max8 viewport, but it looks same or even worse when rendered.
The problem area isnt too wild i guess?. the circular stuff on the side, sure is pushing it tho, and i will remove that from final model.

havent tried in max, but maya 2009 is giving similar results. will try in max and see how it goes.


PS, forgot to include the problematic normalmap in original post, added it now.

Michael Knubben

I don't know why the rest thought they could solve this problem without more information, but you really need to tell us more. First off: Is the model softshaded/one smoothing group? I imagine so (because otherwise you would've had to breka up your uvs). Then, unless you're using 3dp's shaders with quality normals (and an engine that uses the same tangent basis), you might want to either add more geometry OR break it up into smoothing groups.

Go through the wiki (it's in my signature) and check out the normalmapping article before you continue. There's a lot of technical knowledge that would really help you there.

Spicypixel

AFAIK he is using max 8 and 3pointshader is 9+

JohnnyRaptor

aye, im in max8.

iv tried it with 1 smoothing group and several.
the original post is 1 smoothing group, and with 2 smoothing groups, i get this problem below.

i also tried breaking the UVs of the problem area, while having 1 smoothing group on the entire mesh, and it gave me a similar result to the image below.




iv tried so many different things im kind of confusing myself. its probably just some little thing iv never learnt or missing out on on this asset that is causing this. I will go through the wiki when i get home. its a bible of awesome knowledge! hopefully sorts me out for tomorrow.

edit,

i appreciate that i could combine/paint out the two different normalmaps in photoshop, but across a massive asset with tons of problems like this, it suddenly becomes a massive workload to do. so hopefully i can sort this out in the normalmap generation phase.

Shogun3d

Have you reset your xforms before exporting for xNormals, and what export format are you using? Also are you using ray distance or a cage?

JohnnyRaptor

yeah, xforms have been reset before exporting to OBJ using these settings,


in xNormal, i use ray distance. iv tried with both exported normals, avarage, and hard (setting the corresponding in max8, as exported, 1 smoothing group, no smoothing group)

Neavah

In xnormal is always a good idea to use 'average normals' (in depth explanation in the xnormal read-me - pictures - yay )

I don't know if this will really help your initial problem - but you can try using a cage instead. Couldn't hurt...

GoSsS

Is this possible to get the low and the Hi to give it a shot ?
I think this is a problem of Uvs/Smoothing groups + Cage

almighty_gir

i think the biggest problem you've got, is trying to bake your high poly down to such a badly reprisented low. i mean, there's a lot of shape to your high poly that will never EVER be captured in the low as it exists now.

and with the inexpense of polygons nowadays compared to shaders/material costs, you should really revise your low poly.

that's just me though, and while it probably won't help your current problem. it will help your overall presentation.

JohnnyRaptor

well, its not as easy as that unfortunately. its engine and hardware dependent i assume, tho i may be able to revise the lowpoly in the future depending on the performance and filesize of the final asset, even tho its already overbudget. Id like to get as good as possible normalmaps on this before redoing the lowpoly. im gonna read through the PC wiki tonite and take another hit at it tomorrow, meanwhile

iv uploaded the highpoly and lowpoly here

http://dl.dropbox.com/u/2516103/normalmapprob.rar

kudos to anyone who manages to crack this bastard...

almighty_gir

your object has about 7 different smoothing groups, after making it 1 group, it worked just fine for me.

GoSsS

Here is how i manage the seams (not really important 'cause i used only 1 smoothgroup after all) :



Only 1 smoothgroup

And here's the result (in marmoset ... run marmoset for that sort of asset ... yeah i know it's kinda useless )

JohnnyRaptor

ooooooh! dude thats badass! gotta give that a go! and yeah i use marmoset too, too freaking good eh!

JohnnyRaptor

im gonna blow my brains out soon...

1 smoothing group, avarage normals in xnormal, same seams




aaaaaaaaaah! what am i doing wrong?!


btw gir ur right, must have imported the obj right after one of my desperate attemps at finding the freaking problem....

JohnnyRaptor

eld

If you cant use the 3pointshader, then you have to split up the lowpoly in smoothinggroups and equally seperated uv-maps, since otherwise, if the lowpoly model has too much bad shading, it'll transfer over to the bake

(due to reasons discussed in the thread about bad bakes in max)

basicly, max bakes after render tangent basis, while it should be baking from the realtime one.

sampson

does it really have to be so lowpoly? theres no way your going to get a good bake anyway, the lowpoly and the highpoly are just too vastly different....

McGreed

If you have sharp edges, you should try use more then one smoothing group and seperate the UV islands into those groups. You will avoid most shading problems and you won't get that gradient in your normals.

vahl

had a quick try at it
first your uvs need to be cleaned up a lot, that's the first thing I did, then I added a smoothing group here since you can't use quality mode or object space which gets rid of the artefact, the seam, however is as good as it'll get without quality mode, you just have to add enough padding and separate the uvs where smoothing groups are.

JohnnyRaptor

vahl, i did like ur description, exported from max8, generated normalmaps in xnormal, no luck
looked totally messed up.

right before blowing my brains out while jumping out the window...i figured another try wouldnt hurt..

so i redid what u said..in maya 2009 and exported...same thing in xnormal...

BAM....problem solved... i cant believe it...

i documented it and here it is...what the fuck...



conclusion...max8 obj exporter is borked...

PS, thanks a ton to everyone for the help! gotta love this place...

cw

Now incorporated as the default in max, but still available for other versions - have you tried this?

http://www.guruware.at/main/index.html

Maybe it will help?

vahl

yeah max 8's obj was broken iirc, my normals were baked directly in max2009 tho, not maya2009 or xnormal, so you should get similar results without using 3 different softwares, plus, if you're viewing in maya, you should directly use maya's baker, which gives pretty awesome results, as long as you preview in maya.