Home Technical Talk

ugly normals

polycounter lvl 18
Offline / Send Message
dejawolf polycounter lvl 18
having problems with low-quality normals:
on the left, result after normalmapping. on the right,
the object to be mapped. preceding pics shows various
pictures of the low-poly mesh.
bah.jpg
cage.jpg
triangulation.jpg
normalmap.jpg

any normalmap-savvy people got some tips on how to improve the quality of these normals?
i'm using 3ds max 8 normalmapper.

Replies

  • Delaney King
    Offline / Send Message
    Delaney King polycounter lvl 18
    Make sure all your smoothing groups on the low poly model are set to 1. That way the ray cast is diagonal instead of horizontal, which means your 'skimming' the surface.

    That should fix a big portion of it for ya.
  • dejawolf
    Offline / Send Message
    dejawolf polycounter lvl 18
    well, heres 2 approaches, both with the normalmap not even looking remotely pretty.
    i fixed up the wires like per 128 showed, removing the red wires, and adding the green.
    one with no smoothing, where the normals seems fairly accurate, despite the hideous edges around each poly
    nosmoothgroups.jpg
    and one where i made everything into smoothing group1, which ive done before, but didn't do any good.
    edges looks decent, although they got a "jaggy" pattern.
    and the overledges are still messy.
    shoulders.jpg
    hmmh. i'll experiment some more. if you guys got any tutorial links on advanced normal mapping in max, i'd be interested. i'll try and read that section in the manual about the various features in the normalmapper, to see if any of them will make my normals magically more beautiful.
    very annoying having to fix up the edges, after a normalmap.
  • Joshua Stubbles
    Offline / Send Message
    Joshua Stubbles polycounter lvl 19
    It looks to me like in that first example, the low poly's UVs or faces are unwelded. Or you accidentally used an auto unwrap. Also, looking a the 2nd example, it seems your cage is still incorrect. You're not getting enough distance.

    Try increasing the cage distance from the base mesh so there's no clipping at all. Also, try NOT using the cage on a burn. Then last but not least, try using another app, such as Xnormal, to generate the normal map.

    You can turn the use of the cage off in RTT's projection menu:

    maxfg1.jpg
  • dejawolf
    Offline / Send Message
    dejawolf polycounter lvl 18
    i've done this with and without cage, the model is UV'ed, and the faces are welded properly, double checked that.
    you can see theres a big ugly blotch there on the lower part, in the middle of some UV's.
    i've even turned around the UV normals red and green space a few times, just to see if theres some errors there.
    i'll try the external UV program option, see if i get some better results.
  • MoP
    Offline / Send Message
    MoP polycounter lvl 18
    In your latest post, Dejawolf, the first image looks (as Vassago said) to have dodgy UVs (probably overlapping each other - if you're mirroring parts, move them out of the 0-1 range).
    The second image looks like a bad cage, the rays aren't catching the highpoly so it's baking black onto the lowpoly.

    I'd do what Per recommended and just make sure your UVs are layed out in one chunk, and if they're mirrored, move the mirrored side off the 0-1 UV range.
  • dejawolf
    Offline / Send Message
    dejawolf polycounter lvl 18
    hmm.. theres 1 overlapping UV for the left and right planar side. i'll try and modify them, and see how thing changes.
  • fritz
    Offline / Send Message
    fritz polycounter lvl 18
    in poops tutorial he mentions setting up the global supersampling as well. this can reduce some of the jaggies at least.
    bout halfway down:
    http://www.poopinmymouth.com/tutorial/normal_workflow_2.htm
  • Joshua Stubbles
    Offline / Send Message
    Joshua Stubbles polycounter lvl 19
    What are you doing to the mesh to corrupt it!? :|

    I've never had a mesh get "corrupt" in max.
    And from the looks of it per, the cage was still wrong, on yours :P
  • Joshua Stubbles
    Offline / Send Message
    Joshua Stubbles polycounter lvl 19
  • dejawolf
    Offline / Send Message
    dejawolf polycounter lvl 18
    wonder if theres a way to sue autodesk for shitty program design. or lost work.
    certainly if a crane you're working with on a construction site suddenly tips over, you'll have some serious insurance quarrels going on. hows things for working 2 months on a model, only to have it lost in a fatal crash by 3ds max,
    or worse, corrupted beyond salvage?
    they'll probably start ranting about backups, and such, but seriously, there should be laws against this kind of design malfunctionality,
    so that the end users can get a reliable tool, instead of one that breaks when you try to use it.
  • MoP
    Offline / Send Message
    MoP polycounter lvl 18
    Per: What hardcore tools are required to construct a 100-ish triangle mesh and bake a normal map from a subdivided EPoly?
    I do that all the time, and have never seen "mesh corruption" issues such as you speak of.

    dejawolf: What, do you turn off Autoback too? You don't save incrementally? Those are really, really bad workflow practices, regardless if you're working with the most stable program in the world or not. It's just common sense. What if your hard drive has a hardware failure? Do you have any external backups?
  • MoP
    Offline / Send Message
    MoP polycounter lvl 18
    I'm not insinuating these problems are imaginary. I use "quotation marks" all the time in responses, not as sarcasm or anything, just because that's what other people are saying, not me.

    Problems will be solved when you stop making smart-ass comments like:

    [ QUOTE ]
    your meshes will begin to corrupt as you start getting into the more hardcore tools.

    [/ QUOTE ]

    And just stay objective, please. I want to fix people's application problems just as much as you do, but I don't want to do it by implying that whoever isn't getting application issues aren't "pushing the limits" or being "hardcore". That's just unnecessary.
  • Ghostscape
    Offline / Send Message
    Ghostscape polycounter lvl 13
    What the hell do you mean by a corrupted mesh? I've had similar problems to deja's issue, and I'm totally confused as to what you mean by corrupted mesh.

    Is it an issue with not trangulating before rendering the nmap?
  • EarthQuake
    Its like, when a mesh no long has any sence of decency, and starts selling rock just to pay for it's gambling addictions. Eventually selling its children on the black market just because it can't afford to pay the rental payments on its patio furnature.
  • Joshua Stubbles
    Offline / Send Message
    Joshua Stubbles polycounter lvl 19
    stfu bitches. Stay on topic. laugh.gif

    Glad your issue was resolved, dejawolf. smile.gif
  • MoP
    Offline / Send Message
    MoP polycounter lvl 18
    Dejawolf: Any chance you could send me the original, fubar-ed mesh? I'd be interested to take a look at it smile.gif

    As far as mesh corruption issues go, do you use any custom scripts (written by people other than yourself) while modelling? If so, what are they called?
    Same question to you, Per - I wanna give them a try and see if they either improve my own workflow, or give me similar mesh problems, or both smile.gif
  • CrazyButcher
    Offline / Send Message
    CrazyButcher polycounter lvl 20
    software doesnt rate people, being hardcore or softcore, it doesnt matter, we are all the same dumb event triggers. Say you use some special maxscripts or not, at the end the backend is responsible to not corrupt your stuff. Maxscript doesnt give you any pointer access, you cannot fuck up memory within a mesh unless autodesk fucked up maxscript, which they are using increasingly, too.
    Max is a tool, it should always work regardless how we use it, it shouldnt corrupt data, regardless of input. It is one of its strength that it allows workflow enhancers via maxscript or plugin coding (the latter of course are perfect to fuck up everything, but maxscript shouldnt)

    A screwdriver should work for left- and right-handed people, but not start to melt the screw. Given the price tag, the long history of the software, one could expect and demand a more solid core. Likely we will (with luck) wait a few service packs, and things will smoothen a bit, but if we have bad luck, things will stay until version 10 comes, which we pay for again and again... that is what frustrates me, I dont need new mental-ray or other features, I need a solid core functionality.

    well we shouldnt accuse each other of breaking it because we dont use it "the right way", as there is none.

    Anyway let's storm autodesk's headquaters wink.gif our common beloved enemy. I wouldnt want to blame their coders either, the source is a giant mess, showing the long time it has been around. For autodesk there is no reason to use more coders and shift priorities, because most of us continue to pay, simply as we are used to it, and it has of course its strength. It just seems like a dilemma, which cannot be easily broken thru.
  • MoP
    Offline / Send Message
    MoP polycounter lvl 18
    [ QUOTE ]
    We shouldn't accuse each other of breaking it because we don't use it "the right way", as there is none.

    [/ QUOTE ]

    Absolutely smile.gif
  • Eric Chadwick
    Max has corrupted meshes on me quite a few times. I can attest that export/import to/from a non-max format will often clean out the gunk.
  • Delaney King
    Offline / Send Message
    Delaney King polycounter lvl 18
    I have this female character mesh that explodes on contact with the brush tools... its cool, kinda like a party trick.
  • dejawolf
    Offline / Send Message
    dejawolf polycounter lvl 18
    http://www.dejawolf.com/corrupto/footsoldier.zip
    http://www.dejawolf.com/corrupto/footsoldier_per.rar

    links to downloads of corrupted models.
    if you want, you can also try mirroring the whole arms.
    thats probably going to show some of the corruption.
    btw, the should elements isn't the only ones with normals that aint no good looking. i've had the problem all the way down the mesh.

    i don't use any specific modeling plugins on my side.
    just standard max modeling tools.
    although i use quite a few small self-made maxscripts for UVing.
  • MoP
    Offline / Send Message
    MoP polycounter lvl 18
    OK, I took a quick look at it this morning and had it working perfectly within 10 minutes.
    The only thing I'm not sure about was the highpoly segment of the shoulder bit, it seemed to be an XRef from a different file not included in footsoldier.zip , so I made a new one quickly by cloning the lowpoly section and adding some bevels.

    Here's a screenshot of the result, it just uses the original UVs, I didn't touch those:
    deja_normals.jpg

    Here's a run-down of what I noticed and what I did, maybe try these steps on your machine and see if it works.

    First of all, I noticed the low poly section was a mess when it came to XForms - it thought it's own scale was -107.24 or something on every axis, which is usually a very bad sign.
    So, I converted the object to Editable Poly (why was it EMesh anyway? smile.gif ), and Reset XForms on it get it back to positive scale.
    This action resulted in flipping all the normals (again, a sign that the object was fairly screwed in terms of scaling etc.), so I just selected all faces and flipped them back to face the original direction.

    Then I changed the renderer to Scanline, mental ray doesn't do a great job of rendering normals...
    I also un-checked "use advanced lighting" since this can sometimes screw with normal baking.

    I also applied one of your basic materials to all the objects, just in case (i used "ap-cuer", since it seemed fairly basic).

    Then I fired up Render To Texture, picked the highpoly source objects, hit "Reset" on the cage and used manual value to push it out until it was covering everything, quick and dirty.
    Turned on Global Supersampling in the Options -> Render Setup cos that always makes for a better normal render.
    Added a Normals Map output, just left everything else pretty much default.

    Baked that and got the above result - no dodgy pixellated lines, no black spots.
    The only visible normal problems are due to split UV seams, you could probably come up with a better UV layout to minimise the visibility of those.
    I'd probably put a split down the central symmetry, and uv-map each side as a whole chunk, and stack those UVs on top of each other (but move one set to the side by 1 UV unit to avoid errors).

    So yeah, if you wanna try those steps and see if it works, give it a shot and let me know what happens.
    I reckon the problems were mainly in the XForms, but the renderer and odd materials probably weren't helping.

    My solution was done without exporting/importing any meshes, or loading new scenes. The only unknown element in this whole solution is the XRef object that I couldn't load.
  • dejawolf
    Offline / Send Message
    dejawolf polycounter lvl 18
    hmm. ok, thanks MoP, didn't know Xforms were that crucial to modeling. i think i've fixed quite a few Xform problems by creating a box, and attaching the malfunctioning object to the box.
    but heres a question. how did the Xform become -107.4 in the first place?
    if i can't avoid doing whatever i do wrong in the future, and only fix the symptoms, i'm not very far.
    all i've done is modeled normally.
    i convert frequently from editable poly, to editable mesh.
    i find adding edges and turning edges is a lot simpler, and less buggy than in editable poly.
    another thing, i always work in perspective view, and when cutting with the cut tool, i find the cutting accuracy decrease, to a point where the cut radius is all-encompassing.
    cutting works fine in the orthos.
    so is there a chance, 3ds max modify the Xforms when i cut in perspective view?
    and why is such a useful utility hidden away?
    the manual reference doesnt really tell you that its capable of fixing normal mapping problems either.
    is there any other uses for Xforms?
  • dejawolf
    Offline / Send Message
    dejawolf polycounter lvl 18
    how come theres no blinking warning lights in the normalmapping utility when you use it? why isn't there a popup box that ask you to reset your xforms?
  • MoP
    Offline / Send Message
    MoP polycounter lvl 18
    Yeah, having to reset XForms is a pain - it's usually a rule of thumb though, if you're having any problems at all, try Reset XForm first smile.gif

    I think there are no warnings about it when rendering to texture, because sometimes people actually want their objects to have odd scale sizes (like when you instance mirror an object, it'll have negative scale on the axes you mirrored it across), so the program wouldn't be able to tell when the XForms were meant to be weird, or if they were just screwed up during the modelling process.

    If your scale transforms are at -107 on every axis, that means at some point along the way, you've scaled the object up, then mirrored it, or done some sort of transform operation on it. I'm pretty sure modifiers don't affect the XForm, only actual move/rotate/scale/mirror operations.

    Actually, if you look in the Heirarchy panel, you'll see a "Scale" button in the Reset area, if you hit this, it should reset the scale to 100% on all axes, quicker and easier than doing a Reset XForm and collapsing the stack, although not quite as comprehensive. It might fix certain issues, though, and is definitely worth doing if you don't mind losing your scale values (as they can sometimes be useful).

    One question - why do you keep switching between EMesh and EPoly? EPoly's Turn Edge tool is identical to EMesh's, and I don't understand how edge creation can be easier in EMesh than EPoly - what exactly do you mean by this?
  • dejawolf
    Offline / Send Message
    dejawolf polycounter lvl 18
    ok. but i was thinking a warning dialog, that tells you there might be something wrong if you try rendering this, before proceeding. besides, having the Xforms not reset, is a minority thing, applied to the masses, imho a pretty bad design decision. you always have to take Xforms into consideration, instead of only when you want to.

    for the Emesh, and Epoly:
    i couldn't find the turn edge tool in Epoly, so i just went with the turn edge tool in Emesh. theres also no way AFAIK, to select all edges, set auto edge to 5. which is quite handy when doing vehicle UVs, you want to select as few UV's as possible.
    Emesh is also good when you do a cut operation from a vertex to an edge, and it turns out Epoly created you an extra vertex in exactly the same position of the old one.
    turn to Emesh, select all, weld all, turn to Epoly.
    Emesh got a bit more freedom with welding vertices to vertices. and collapsing vertices is pretty good for LOD creation, when its better to have a lowpoly mesh, than a pretty mesh. its mostly a hangover from my all-Emesh days,
    when you couldn't shift-drag new polygons in Emesh.
    i sometimes like to select triangles too.
  • dejawolf
    Offline / Send Message
    dejawolf polycounter lvl 18
    btw, i had used mental ray before for ambient occlusion maps. i'm experimenting with dirt maps smile.gif
    anyways, i'll take all these things into consideration, and create a small checklist, for future reference for anyone who's unfortunate enough to have to deal with the confusing world of 3ds max normal mapping.
  • dejawolf
    Offline / Send Message
    dejawolf polycounter lvl 18
    oh yeah, thanks to everyone who's helped deepening my understanding of the mysterious normals.
    hope this thread is useful for anyone else who is having trouble with normals.
  • Rhinokey
    Offline / Send Message
    Rhinokey polycounter lvl 18
    the number one solution of all time to fixing ANY max errors is resetting x-forms, thats the first thing i do at any sign of a problem, its almost second nature now, sometimes if the problem persists after reseting make a box, and attach the object to the box (so it becomes part of the box,, not the other way around) then go to edit and delete the box polys.

    i've acheieved about 12 million corupted meshes since starting work in the industry, and i know a lotta coworkers here have had a lot of corrupted mesh issues,, a lot of times it comes from custom tools, other times it comes from being in max over 8 hours a day forever
  • JordanW
    Offline / Send Message
    JordanW polycounter lvl 19
    speaking of reset Xforms and it being the first line of defense against jacked up geoemtry...my reset Xforms button at work crashes my max now....every time, gues it's time for a reinstall laugh.gif
  • Pedro Amorim
    [ QUOTE ]
    Its like, when a mesh no long has any sence of decency, and starts selling rock just to pay for it's gambling addictions. Eventually selling its children on the black market just because it can't afford to pay the rental payments on its patio furnature.

    [/ QUOTE ]

    oh.. the story of my life!
  • dejawolf
    Offline / Send Message
    dejawolf polycounter lvl 18
    hmm, well i continued modeling, and i bounced into another error, AFTER resetting the Xform. on the leg.
    the only way to fix these errors, was to.. export the object, and then import it.
    for those who are willing, unhide the layer nicknack,
    reset Xform of the leg parts, and try rendering normals.
    i kept getting these faint wire patterns on the leg,
    after several Xform resettings, and various setting changes, i just exported and imported. worked fine then.
  • arshlevon
    Offline / Send Message
    arshlevon polycounter lvl 18
    sometimes reseting x forms dosent work for me.. the end all fix i have found is to create a cube at 0 0 0 with the keyboard option.. dosent matter the size.. then turn that into editable poly and attach my object.. then in element mode delete the cube.. this resets any crap that might be in the in the channels.. its not always transform information that fucks it up..

    oh and mop, the edit mesh has a much better turn edge, you can turn any edge..i think you can only turn invisible edges in editable poly, but i have scripts for turning any edge with edible poly, like edit mesh. polyboost also has a tool for this.
  • MoP
    Offline / Send Message
    MoP polycounter lvl 18
    Actually yeah, sorry, you're right about the edge turning thing, I just never really use that tool at all smile.gif

    I should probably look into getting polyboost, it looks insanely useful.
  • Joshua Stubbles
    Offline / Send Message
    Joshua Stubbles polycounter lvl 19
    [ QUOTE ]
    sometimes reseting x forms dosent work for me.. the end all fix i have found is to create a cube at 0 0 0 with the keyboard option.. dosent matter the size.. then turn that into editable poly and attach my object.. then in element mode delete the cube.. this resets any crap that might be in the in the channels.. its not always transform information that fucks it up..

    [/ QUOTE ]


    Yuppers. I've done this before, with objects that crash max. Works like a charm.
  • Michael Knubben
    I didn't really want to start a new thread for this, since i haven't tried it yet, but i came across it while googling for something else (which channel needs to be flipped for compliance with the D3 engine):
    http://www.codeproject.com/cs/media/NormalMapCompressor.asp

    It claims to compress Normalmaps without visual artifacts, let me know if it works well...

    Here's a screenshot, for the textually imparied:
    NormalMapCompressor.jpg
Sign In or Register to comment.