Home Technical Talk

Really need help with normal maps

polycounter lvl 3
Offline / Send Message
Thane- polycounter lvl 3

UPDATE ON POST #7, improvement, but still a few issues.



I just can't seem get this normal map looking okay using just 3DS Max. I feel like im close to solving it but i have literally been stuck on this for weeks. If anyone could walk me through this im having a couple of issue. Im going from 3DS max to Xnormal to EU3 (Chivalry).

First, what are the proper FBX settings for the LP and HP?

For the LP ,according to this from UE4 documentation (which i just now discovered ) the LP should have:
1. Smoothing groups
2. tangents and bi-normals
3. convert deforming dummies to bones
4. Preserve edge orientation (wth is this?)

What about the high poly, which setting should i have checked there??


Below are the settings in question and my result in 3DS MAX using render to texture.
FBXandLP_problem.jpg

Replies

  • AdvisableRobin
    Offline / Send Message
    AdvisableRobin polycounter lvl 10
    You're export settings seem fine. What issues are you having specifically? If its problems with the normal map post images of the normal map, high and low poly. This'll help us diagnose the issue.
  • huffer
    Offline / Send Message
    huffer interpolator
    Does the Chivalry engine use 3dsmax's normal map tangent space or mikk as xNormal? Can it import tangents and binormals data from fbx or does it recalculate them?
  • Thane-
    Offline / Send Message
    Thane- polycounter lvl 3
    Okay, here are some images and i linked to the .max file if that would help.

    .max file 6.8MB

    Current results in Chivalry editor (UE3)
    issues.jpg

    UVs

    sword_uvs.jpg


    High Poly

    HP.jpg


    Low Poly

    wires.jpg
  • Eric Chadwick
    What exactly is the problem?
  • Thane-
    Offline / Send Message
    Thane- polycounter lvl 3
    What exactly is the problem?

    -First the creasing on the zoomed in portion of the first picture in my second post.

    -The seam line along the side of the crossguard in the same pic. Thats two UV islands joining there. I exported the LP with 1 smoothing group using the settings according to UE4 documentation, like in the screenshot in my first post ( except split vertex unchecked )

    - The blade body halves look slightly rounded (concave). I made the HP flat which was the goal, but its hard to tell in the veiwport if the turbosmooth perhaps rounded it. Looks flat to me, but its hard to tell on my small 17" 1080p monitor.

    -Weird shadowing all over the place.

    Should the model that goes into UE3 have one smoothing group?


    XNORMAL SETTINGS


    HIGH POLY
    Exported from 3DS Max using the built in exporter using the UE4 rec settings

    Settings checkmarked:
    Average normals
    ignore per vertex color


    LOW POLY
    Exported LP and CAGE with same UE4 doc settings for FBX
    Both triangulated and identical
    LP has one smoothing group


    Settings checkmarked:
    Use cage
    Average normals
    Highpoly normals override tangent space[/quote]


    BAKING SETTINGS
    Closest ray hit: check
    ignore backface hits: check

    edge padding:6
    AA: 4x
  • Thane-
    Offline / Send Message
    Thane- polycounter lvl 3
    double post. Ill just use this for my next post.
    UPDATE: I solved issue 2 and the weird shadowing by using another model having all one smoothing group, which i sorta thought this did, but i guess maybe not... :/

    Okay, i realized that the documentation said to have "use exported normals" in Xnormal checked for the LP. I re-baked to much better results, but still have some issues below. I linked to a copy of the normal map i rendered this time.

    1. Still have one slight crease on the either side of the face of the cross guards. I guess you all would fix this in photoshop?

    2 One abrupt edge running all the way down in-between the two UV islands.

    3. Weird shadowing.

    I gave the material a constant vertex color node in the editor. I am new to UDK and new'ish to normal maps in general so if it looks like i am overlooking something incredibly simple, i probably am. These are UDK3 viewport renders and the normal map resolution is 2048x512.
    12.jpg


    shadows.jpg

    sword1_normals_exported%20normals.png
  • Thane-
    Offline / Send Message
    Thane- polycounter lvl 3
    Another weird thing. Because in the process of UV mapping the exploded model i made quite a few model/uv changes, i ended up having to use the exploded model to build the final model with.

    I had triangulated the best exploded result i had, using 1 smoothing group when exported, which only had the one minor issue left mentioned in the previous post which i can probably fix in photoshop. However, the finished blade model now exhibits a crease in another spot, despite being derived from the same model, with the same vertex positions and having the same triangulation and i used the same export/import setting afaik. Im just curious why this would be.

    The "good" one is on the right.
    issue%201256.jpg
  • Bedrock
    Offline / Send Message
    Bedrock polycounter lvl 10
    Hmm baking anything with one smoothing group was always disastrous for me. I would recommend having a separate smoothing group for every UV island, I use TexTools to do it for me. This also means that you will need to switch "Average Normals" to "Exported" for the low poly in Xnormal. I don't know if both the LP and the cage should have the same smoothing groups (I make the cages in xnormal because I hate myself) but it would make sense.

    As for the FBX export options, I don't recall having split per-vertex normals on, but I don't think that's the culprit.

    Also do refrain from fixing normal map issues in photoshop. I've done over 50 bakes in the past two months and it ALWAYS comes down to messing up some of the settings, inverted normals, split vertices, etc.
  • ZacD
    Offline / Send Message
    ZacD ngon master
    Udk doesn't have a synced normal map work flow so there will always be some issues. Adding smoothing groups/splits/hard edges do help some in situations like this. Make sure you are using the suggested xnormal work flow for UE3 that Epic has in their documentation. The UE4 doesn't exactly apply to UE3.
  • Thane-
    Offline / Send Message
    Thane- polycounter lvl 3
    @Bedrock: Interesting, i got the best bakes by choosing "exported" normals with 1 smoothing group too. Not sure what that means.

    I tried to use Textools to apply a smoothing group per uv island to the LP in Max, changing nothing else, but now the model looks horrible so something is obviously still wrong. I will go with what i have for now.

    @ZacD, checked out this July 2012 thread on UE, which i assume pertains to UE3. http://www.polycount.com/forum/showthread.php?t=102838

    It quotes the EU documentation saying to use 1 smoothing group as well, but in the following commnts there is quite a bit of disagreement.

    I frickin hate normal maps now. It seems like we should have a constantly updated normal map workflow guide with detailed settings for each possible combination of program. I wish i had more time or it would make a great way for me to pay back the community for all their tutorials and advice.
  • ZacD
    Offline / Send Message
    ZacD ngon master
    Handplane works pretty well for simplifying the different rules with different applications, it really sucks that you can't just use any normal map with any application.
Sign In or Register to comment.