Home Technical Talk

xNormal - MASTER THREAD

1394042444559

Replies

  • womball
    http://www.polycount.com/forum/showpost.php?p=1229131&postcount=259

    They seemed to have done it for the game.
    So perhaps there is a way of doing this in photoshop?
  • Eric Chadwick
    Read this post, you kind of need a tool that lets you paint the "flow" you want, in an abstracted way, similar to combing fur or somesuch. Not easy to do by hand in Photoshop!

    Edit... this might also be instructive... Painting a "flow" normalmap?
  • EarthQuake
    womball wrote: »
    http://www.polycount.com/forum/showpost.php?p=1229131&postcount=259

    They seemed to have done it for the game.
    So perhaps there is a way of doing this in photoshop?

    I know, I'm the one who made the asset. =)

    We had a tool that was... A simple 3d viewer that let you paint or "rake" in a certain direction, and the tool would output the aniso map. It was a pretty crap tool, it painted on both sides of the model and some other weird stuff.

    I blame Andres for this (MonkeyScience) I believe the code for the tool was actually lost in a SVN overhaul.

    So yeah, you would need either an entirely new tool, or a plugin for some existing tool that supports painting in 3d.
  • ianucci
    Offline / Send Message
    ianucci polycounter lvl 9
    Hey Santiago I've been playing with the curvature map and it appear xnormal does not differentiate between concave and convex surfaces.

    If you render to a map with two channels all the data is in one channel, the other being empty. I tried rendering to 3 channels but all this does is render the same information to the third channel but inverted.

    Still a useful map but it would be nice to have convexity and concavity separated.
  • jogshy
    Offline / Send Message
    jogshy polycounter lvl 17
    ianucci wrote: »
    Hey Santiago I've been playing with the curvature map....

    Still a useful map but it would be nice to have convexity and concavity separated.
    Pls, note xNormal can render two different kind of maps "convexity/concavity" and "curvature map". I think what you're looking for is the curvature one.
    You need to play a bit with the search distance to get good results.
    Take a look at the spiked ball example, make visible the HP and render a 2-channel curvature map. You should see two colors there ( red and green ).
    The curvature map is still a bit experimental though.
  • Vrav
    Offline / Send Message
    Vrav polycounter lvl 11
    Hey Santy, still loving your tool and using it for most everything. Just got access to CUDA and look forward to quicker AO bake times. :)

    I am working with an engine that uses "area-weighted smoothing of all triangle parameters (normal and tangents)." Is there some way to activate this for baking in xNormal, or an existing tangent basis calculator I can use to achieve this..?
  • jogshy
    Offline / Send Message
    jogshy polycounter lvl 17
    Vrav wrote: »
    I am working with an engine that uses "area-weighted smoothing of all triangle parameters (normal and tangents)." Is there some way to activate this for baking in xNormal, or an existing tangent basis calculator I can use to achieve this..?
    It's done by default. The default TBC takes care about the area of the triangles to weight the calculations.

    If you need more control I suggest to use the xNormal C++ SDK ( you'll need a programmer tho )
  • Vrav
    Offline / Send Message
    Vrav polycounter lvl 11
    Okay, thanks! I was apparently confused. Yeah, if it comes to that I might just.
  • Scruples
    Offline / Send Message
    Scruples polycounter lvl 10
    jogshy wrote: »
    Pls, note xNormal can render two different kind of maps "convexity/concavity" and "curvature map". I think what you're looking for is the curvature one.
    You need to play a bit with the search distance to get good results.
    Take a look at the spiked ball example, make visible the HP and render a 2-channel curvature map. You should see two colors there ( red and green ).
    The curvature map is still a bit experimental though.

    I am having the same problem, Green channel is ok, Red channel displays nothing.
    I tried playing with the range like you suggested but to no avail...I will keep trying different settings and figure this out.

    so far rendered about 10 different Curvature maps with different settings.
  • Scruples
    Offline / Send Message
    Scruples polycounter lvl 10
    sorry double post
  • ArYeS
    Offline / Send Message
    ArYeS polycounter lvl 8
    Hi,

    I'm having problems baking normal maps with xNormal. I have imported my own cage, which works good in Max, but I'm trying to get used to xNormal, because of triangulation problems I have in Max.

    xNormal bake however seems even more messed up. This is just the magazine, other details are even worse. Any clues?

    Thanks

    xnormalproblem.jpg
    right section is "Highpoly + imported cage"
  • oXYnary
    Offline / Send Message
    oXYnary polycounter lvl 18
    Hi jogshy. Do you have a bug list I dont want to repeat what already known.

    1. with non 2X2 maps (say 4X2) when using the normal map to cavity map conveter will render out the cavity 2X2.
    Unconfirmed.

    2. Red channel is (and again it just may be Maxs weird rendering ability to hide seams). Too pronounced. Even on straight edges on seams that otherwise would be hard to see if at all.

    3. Sometimes your sbm export either flips the uvs or the render does. (As in blue reverses). Export obj seems fine. (And yes xform used and all).

    Ill get some files up here.
  • oXYnary
    Offline / Send Message
    oXYnary polycounter lvl 18
    Ok here we go.

    Example

    1. Tools in program not understanding 2X1 etc. (I was wrong it flips the uv versus making 1 to 1).

    Original was 2048X1024
    normcavi.jpgcavires.jpg


    2. Red channel is over-pronounced causing seam (again may just be the way xnormal versus max handles these area) but damn. The results are significantly noticeable.

    RedX.gif

    beforeX.jpgafterX.jpg
    *Shown in max with 3 point shader. However results similar in Xnormal itself.

    3. Sometimes with sbm export if flips the polygons. I have narrowed this down in the sbm export to checking "export normals". If choose no normal or average/smoothing there is no issue.) )This is on the high rez, not low rez.

    reversedNorm.jpgnoNorm.jpg

    This was in max 2011. Xnormal 64bit
    Files: http://www.oxynary.com/downloads/x/export.7z

    Files. (Also not whole object, just part so some items will render funny-just focus on the "tube" area (interior string is present in full render).)
  • ArYeS
    Offline / Send Message
    ArYeS polycounter lvl 8
    I know this is not nice behavior for forums, but my post got posted as last on page 83, and I really don't have an idea how to fix my problem. Can anyone please look at that?
  • Ben Apuna
    @ArYeS:

    Well what exactly is the problem here?

    The normal map looks fairly OK.

    Is your high poly model completely surrounded by the cage? or are there parts that stick out of the cage? If yes then you need to push the cage out some more until nothing sticks out of it.

    Why is there only one side of a cage on that magazine?

    You should be baking to a "whole" low poly model, not deleting the half you plan to mirror. Just move the mirrored UVs over 1 unit in U or V. In that way your cage should also be "whole".

    What triangulation problems were you having in Max?

    From what I remember Max handled triangulation better than any other 3D app I have ever used (Maya, Max, Modo) because you could actually edit the triangulation by turning the hidden edges.
  • ArYeS
    Offline / Send Message
    ArYeS polycounter lvl 8
    @Ben:

    Thanks for replying, as for the max triangulation problems, Its probably problem in the model, and I am going to remodel it. I will do as you say for mirroring.

    But for normal map, if you look at the magazine side part, its really crooked and "indented lines" arent in right position aswell.
    Its should look like this (right one):
    http://upload.wikimedia.org/wikipedia/commons/c/ca/Stanag_mags.jpg

    I will post whole max and xNormal bakes when I get home.
  • Ben Apuna
    Yeah, I see what you mean. Usually distortion on a normal map from a bake is OK, it's just compensating for the distortion in your UVs compared to the actual polygons of the model.

    In this case looking at it a 2nd time it does look a bit extreme. You could try to minimize it by baking using the raycast method rather than the cage method.

    You can also get straighter lines my making sure all of your triangulation runs in the same direction like with tri stripping. Though not done for performance reasons, just so straight lines come out less wavy.

    strippingtesta.jpg

    Then again it might just be you've got really distorted UVs. It's hard to tell, try post a screen shot of your UVs overlayed on the normal map.
  • ArYeS
    Offline / Send Message
    ArYeS polycounter lvl 8
    As the whole model is more complex, I prefer using cage. Or should i try raycast method for whole weapon.

    So here are pictures of shading problems in max, should i separate things in smoothing groups better and split UVs there? Polycount for weapon + addons is already 12k, so I don't want to put a lot more polys there.

    And second picture is normal map with same cage in Max vs. xNormal, with red color I am pointing towards problems in max normal map, because of which I want to try baking same think in xNormal, as I heard it's better when dealing with such problems.

    Again, thanks for your time, I really don't know what to do here :)

    triangulationproblems.jpg
    normalcomparison.jpg
  • oXYnary
    Offline / Send Message
    oXYnary polycounter lvl 18
    Your going to hate me for saying it. But can you share your files? Might make it easier to see where your issue is. Im thinking personally it may be your export sbm settings. Try this. Export the high as an obj and the low as the sbm with the cage.

    Also add a xform /reset xform modifer to both and make them edit meshes versus polies.
  • ArYeS
    Offline / Send Message
    ArYeS polycounter lvl 8
    I exported everything as .ase from max to xNormal and looks good in xNormal 3D viewer. I reseted all xforms before porting anything.

    I'll try .smb exporting now.

    EDIT: So.. I tried exporting .smb + cage, bake is now different... not a lot better
    normalsmb.jpg

    If someone can take a look at mesh, and help me out I would be very excited. I can send files via email (send me your email via PM), if anyone uses GTalk and has time to chat about my problem -> aryes(dot)vuk is my ID.
  • oXYnary
    Offline / Send Message
    oXYnary polycounter lvl 18
    Ok what I found.

    Low:
    -Inverted faces
    -Overlapping geometry
    -No smoothing groups in some area.
    -Haven't separated all smoothing groups into own uv islands.
    -Anything over 90 really needs a new smoothing group.

    Detaching geometry is the same as smoothing groups in game. Using smoothing groups is easier to work with projection(doesnt :"blow out" from each other)


    High:
    "rats nest" in mesh Obj export. Meaning there is some geometry doing strange things somewhere on it. I think there is a tool that can help you identify but dont know name.
    Obj export gave best results. Sbm had the same issues Im encountering. (flipped faces)

    test_normals.PNG
  • ArYeS
    Offline / Send Message
    ArYeS polycounter lvl 8
    Thanks for all feedback guys. will get back to work and try to fix as much as I can :) I hope you will be able to see pretty cool results in gallery sometime soon :)
  • jogshy
    Offline / Send Message
    jogshy polycounter lvl 17
    oXYnary wrote: »
    1. with non 2X2 maps (say 4X2) when using the normal map to cavity map conveter will render out the cavity 2X2.
    Unconfirmed.
    I've processed the 2X1 image (512x256) you posted but works ok for me with the 3.17.4 I'm preparing, so the problem seems should be solved with that new version.
    2. Red channel is too pronounced.
    Well, you can't really compare TS-normal maps using different coordinate systems because the color channels will be different for sure:

    xNormal uses an OpenGL approach with bottom-left UV origin, right handed system with Y pointing up.

    3dsmax uses a D3D approach with top-left UV origin, right handed system with Z pointing up.

    About the ability of 3dsmax to hide the seams, I think there's is simply a problem with my TS computations. I'll revise my code.
    3. Sometimes your sbm export either flips the uvs or the render does. (As in blue reverses). Export obj seems fine. (And yes xform used and all).
    That's strange, yep. I'll revise the code of the exporter.
    Sometimes with sbm export if flips the polygons
    Well that's actually something strange.
    Some engines use CW faces as front faces while xNormal uses CCW faces... so the normals(and vertex indices) need to be reversed.... but I'll take a look because that's strange.

    Thanks for the bug report!

    @ArYeS, notice if you move the cage's triangle you'll modify the ray direction also ( which is useful for instance to deal with cylinders as is explained in the Poop's NM tutorials ). Also sure your UV's are well welded, perform a ResetXform before exporting AND avoid the old 3dsmax's max2obj exporter at any cost.

  • kary
    Offline / Send Message
    kary polycounter lvl 18
    jogshy, could you put a preset for a low poly (with cage/UVs/normals) and one that is blank (like now) for the sbm exporter? When exporting a lot of low polys (or the same one again and again more like) it's a bit annoying to tick those same boxes repeatedly. The obj exporter for max does something like that for different software, but I am not sure how difficult it is to implement.

    xNormal is a great piece of software. Thanks again :)
  • oXYnary
    Offline / Send Message
    oXYnary polycounter lvl 18
    jogshy wrote: »

    xNormal uses an OpenGL approach with bottom-left UV origin, right handed system with Y pointing up.


    Thanks Jogshy. FWIW though I did try it in the Xnormal viewer and got the same results.
  • jogshy
    Offline / Send Message
    jogshy polycounter lvl 17
    kary wrote: »
    jogshy, could you put a preset for a low poly (with cage/UVs/normals) and one that is blank (like now) for the sbm exporter?
    I'll add two buttons with presets for next release.
    Thanks!
  • Nexinumbra
    Offline / Send Message
    Nexinumbra polycounter lvl 12
    jogshy wrote: »
    If you are referring to the U/V tangent map like it's used in the UE3 engine... then I need to investigate it a bit :poly124:
    Oh man, I greatly hope that you extend your terrific program to have a UE3/3ds Max type of TBC. Having to export stuff from Zbrush at 3% of my highres polycount just so I can bake a normal map that I can use in UE3 sucks :(
  • prolow
    Offline / Send Message
    prolow polycounter lvl 11
    EarthQuake wrote: »
    The request here for Santiago to implement this is pretty misguided. You can't simply "bake" this, you need a full-fledged tool to "paint" the direction on top of the model. Its not something that you can just tick on a feature and bake. =P

    As you want to define a direction counter to the normal direction, the user must define this, its not "bake-able" data.

    sorry for missing this post and now quoting it a month later, but while quake is correct in that a "direction map" isn't something that can be just ticked on and baked. It IS bake-able data. as illustrated in the image below, which I previously posted in the xnormal forum on Eat3d
    anisodemos.gif

    As for it being "misguided" consider this:
    Assuming before there was a method to properly bake normal maps from geometry, normal maps weren't widely used because it was difficult to leverage their power with photoshop alone. And considering 90% of the surfaces in nex-gen games are shiny ass metal, I assert that anisotropic materials would be more widely used if there was a good method to properly bake the data. I know managing uv's on hi-res models can be unwieldy now, particularly for sculpted models, but better methods of creating the content would develop just as creating hi-res models for normal baking has.
    in conclusion, I'm taking your beef jerky and running Quake
  • EarthQuake
    This from a guy who didn't even show up for my Halloween party.
  • oXYnary
    Offline / Send Message
    oXYnary polycounter lvl 18
    Jogshy, do you know if xnormal works in wine? I'm thinking of building a dedicated render box out if old parts so my main box isn't so laggy and I can offload some of the work. Would be cheaper this way than having to buy another windows license just for something that will be rendering only.

    On top of this, last time I tried network rendering, it was only up for x86. Will the 64 side work soon? Will both the master and client have to be running the same bit (ie if master runs 32 bit so client must, or vice versa)?
  • Scruples
    Offline / Send Message
    Scruples polycounter lvl 10
    I feel like I am taking crazy pills, has anyone got the curvature map working without both red and green channels being identical? (identical with the exception of being inverted).
  • EarthQuake
    oXYnary wrote: »
    Jogshy, do you know if xnormal works in wine? I'm thinking of building a dedicated render box out if old parts so my main box isn't so laggy and I can offload some of the work. Would be cheaper this way than having to buy another windows license just for something that will be rendering only.

    On top of this, last time I tried network rendering, it was only up for x86. Will the 64 side work soon? Will both the master and client have to be running the same bit (ie if master runs 32 bit so client must, or vice versa)?

    Two quick things to try before you do any of that:

    Task manager->Process->Xnormal.exe->Set Priority-> Below Normal
    Task manager->Process->Xnormal.exe->Set Affinity-> Set to 3 or 1 CPU depending if you have dual or quad core.

    This should fix the problem with XN eating all your resources. Either/both of these should do the trick
  • oXYnary
    Offline / Send Message
    oXYnary polycounter lvl 18
    EarthQuake wrote: »
    Two quick things to try before you do any of that:

    Task manager->Process->Xnormal.exe->Set Priority-> Below Normal
    Task manager->Process->Xnormal.exe->Set Affinity-> Set to 3 or 1 CPU depending if you have dual or quad core.

    This should fix the problem with XN eating all your resources. Either/both of these should do the trick

    We are throwing away some old p4 based dual xeon machines here..
  • jogshy
    Offline / Send Message
    jogshy polycounter lvl 17
    I feel like I am taking crazy pills, has anyone got the curvature map working without both red and green channels being identical? (identical with the exception of being inverted).
    Well, the curvature map is still a bit experimental.
    I really lack models to test it depth so, pls, if you can send me some to debug I'll be very thankful.
    oXYnary wrote: »
    Jogshy, do you know if xnormal works in wine?
    AFAIK, nope ( because xN requires .NET 2.0 SP1 and to install that you'll need also Internet Explorer which is a big pain to setup under linux ).
    You can try with virtualisation but I think only Oracle's VirtualBox supports OpenGL 2.0 ( and, btw, not very well ).
    But don't worry, xn4 will be truly multiplatform.
    On top of this, last time I tried network rendering, it was only up for x86. Will the 64 side work soon? Will both the master and client have to be running the same bit (ie if master runs 32 bit so client must, or vice versa)?
    I'm afraid for the network renderer I'm using a library that does not support 64 bits. I would have to completely rewrite the renderer for xn3 which will be a bit painful currently.

    xn4's network renderer will be much better.
    This should fix the problem with XN eating all your resources. Either/both of these should do the trick
    Well, when xn3 was designed multicore machines didn't exist so I early adopted OpenMP as threading platform as it went available. The problem is that the current spec I use does not support thread priorities so the application just will eat all the resources.

    The trick you describe should work but you can also try to setup an environment variable called OMP_NUM_THREADS and set it to the # of cores you want to use. That have a problem though, ANY program using OpenMP will be affected, not only xNormal.

    For xn4 I'm letting you to set the # of threads and also the thread priority.

    When will be xn4 alpha 1 available? Well, the things are running very slow but very well. I hope Q1 2011.
  • Scruples
    Offline / Send Message
    Scruples polycounter lvl 10
    I sent you a model for debug, and a picture with the settings I was using which also explains in greater detail the problem.

    Thank You.
  • Scruples
    Offline / Send Message
    Scruples polycounter lvl 10
    Beta version of Xnormal is out, 3.17.4 and the curvature map seems to be working fine :D.
    (curvature map seems to render faster too).
  • jogshy
    Offline / Send Message
    jogshy polycounter lvl 17
    Yep, the 3.17.4 Beta 1 is available:

    - Added a post-render tool to control better the height map's clamping
    - Improved a bit the curvature map's algorithm
    - Corrected several bugs
  • Mouldtooth
    Just bought a license for toolbag. Can't get my license key to work...which sucks. Anybody else had this problem? I contacted costumer service, but I wanted to use it today. I'm I missing something? I should only enter my name and key right to get it to work, right?
  • Pedro Amorim
    Wrong thread man :P
  • Tea Monster
    Offline / Send Message
    Tea Monster polycounter lvl 15
    Hey, this may be more just a general baking question, if so, I'm sorry.

    I've got a cyborg type critter with pipes embedded in his skin. Some of the pipes are sort of involved. When I make my low poly, should I:

    1. create the pipes on the low poly as floating geometry, but trying to follow the shape of the pipes (I imagine this might get hairy on poly-count).
    2. Create the pipes as 'boxes' that just cover the shape of the pipes.
    3. Ignore the shape of the pipes entirely and just bake the whole thing down to the shape of the body.

    Here is an example of what I'm building:
    2yn2v15.jpg
  • SpeCter
    Offline / Send Message
    SpeCter polycounter lvl 14
    The parts which are near the body and not floating around could be approximated and the floating parts(the ones which are not directly at his body) should be modelled like that i guess.

    A mix between that should get desired results or i´m just talking bullshit again :D
    Maybe and just maybe you can get away with just following the shape a little bit if there are no wholes.

    It´s hard to give advice on this one, because the picture doesn´t help that much.
  • Tea Monster
    Offline / Send Message
    Tea Monster polycounter lvl 15
    Cheers for the advice. I'll see if I can 'mix and match' a bit. Sorry for the secrecy, but I'm doing him for a mod and the guys have asked me to keep him under wraps until the mod is out. I'm going to show him off as soon as it is out.
  • jogshy
    Offline / Send Message
    jogshy polycounter lvl 17
    Can't bake vertex colors with Xnormal. The image always comes out black, even though I exported the mesh between programs and the same .ase and .fbx files kept all vertex colors.

    So it must be Xnormal. What am I doing wrong? :[
    Currently, vertex colors can be only imported into xNormal using a ZBrush's Polypaint .OBJ or a SBM.
  • Ben Apuna
  • jogshy
    Offline / Send Message
    jogshy polycounter lvl 17
    So ASE.s or FBX. files don't work yet? Hmmmm. Need to figure that issue out somehow, since the models come from 3DSmax.
    If the model come for 3dsmax perhaps you could use the xNormal's SBM exporter/importer.
    COLLADA works for vertex colors, or at least DAEs made with Modo.
    Oh true, I forgot that I implemented the vertex colors for xsi, COLLADA and PLY too.
    I think for the final 3.17.4 I'm gonna implement it also for the ASE.
    For FBX is a huge mess.
  • Ben Apuna
    BTW, thanks for the COLLADA support Jogshy, I appreciate it very much.

    So true about FBX being a mess. For format that was supposed to be new "OBJ" it sure doesn't seem to be very standardized or supported very well by many apps yet. Even Autodesk's own 3D apps can't seem to get it right sometimes. I wish they would make FBX open source, that might help things along.
  • jogshy
    Offline / Send Message
    jogshy polycounter lvl 17
    @super happy cow, have you setup a cage in a way that covers completely the highpoly mesh? Render a "wireframe and ray fails map" and see if the rays are hitting correctly the meshes.
    Are you using the matchUV feature?
  • Nexinumbra
    Offline / Send Message
    Nexinumbra polycounter lvl 12
    Edit: Falling asleep the other night I realized that I hadn't deselected the ignore vertex color option (why does that even exist, I wonder)
    That option exists because people who export a Zbrush high poly mesh with polypaint colors UNcheck that box when they want to bake vertex colors. At least that's what I use it for - maybe it has other uses too, lol
  • Arcanox
    Any word on getting Material ID exclusion working in the newer versions?
  • Piflik
    Offline / Send Message
    Piflik polycounter lvl 12
    I have a quick question. Don't know if it has been asked before, but is there an option to use the closest hit always during baking instead of the furthest hit? (since I don't have any floating geometry in my current highpoly and such an option would be faster than adjusting a cage or playing around with ray distances...)
1394042444559
Sign In or Register to comment.