Home Technical Talk

xNormal - MASTER THREAD

1568101159

Replies

  • jogshy
    Offline / Send Message
    jogshy polycounter lvl 17
    [ QUOTE ]
    I was wondering if it would be possible to make an auto-assign random colors to groups/elements option? Also an option to go into 3d view, select highpoly elements and pick/change colors

    [/ QUOTE ]

    Yep yep good idea. Atm the to-do list has grown too much so need to "flush" some things but will be, indeed, a good feature to implement.
  • EarthQuake
    Yeah thats definately something i would LOVE to see as well. IF you could either do it per model, per-material group that would be awesome. Automatically loading .mtl files for the colors from objs would be even better.
  • jogshy
    Offline / Send Message
    jogshy polycounter lvl 17
    xNormal v3.9.1 has been released!

    It adds a Direct3D 9.0c graphics driver for the 3D viewer , hurray!

    hro_directx_left.jpg

    I was tired of crappy OpenGL drivers thats why included the Direct3D driver for the viewer.

    This driver can use Shader Model 1, 2 and 3 so people with low-end cards can now use the xNormal 3D viewer.

    A .X mesh importer has been added too.

    Added an ambient light controller inside the viewer.

    Improved the TIFF exporter. Now can save 8/16 or floating point pixels.

    Included also a small general overview tutorial in Flash as promised ( Wink ftw! ). More tutorials are coming, this was only a small test!

    Improved Windows Vista RC1 compatibility.

    Also solved tons of minor bugs and improved the speed of the OBJ importer a few.

    The bad news are that it has grown to 54Mb ( blame DirectX mad.gif )

    Download it at http://www.santyesprogramadorynografista.net/projects.aspx

    The change list is located at http://www.santyesprogramadorynografista.net/archives/xNormal_changes.txt

    The blog is at http://santyhammer.blogspot.com

    Now I will include some of the other features you requested.. also will try to include an option to subdivide the lowpoly model using Catmull-Clark so the displacement maps could be compatible with lowpoly+displacement highpoly reconstruction technique ( any idea about this? see http://www.silo3d.com/forum/showthread.php?t=7048 )

    Enjoy it!
  • ironbearxl
    Offline / Send Message
    ironbearxl polycounter lvl 18
    Thank you jogshy! Downloading now smile.gif
  • Renaud Galand
    Offline / Send Message
    Renaud Galand polycounter lvl 19
    wow, long time since my last visit in this thread, and I've to say that you're doing an amazing job for all of us jogshy. Downloading now as well smile.gif

    Thx again for the hard work !
  • Mongrelman
    Offline / Send Message
    Mongrelman polycounter lvl 18
    Awesome Jogshy! Looking forward to using this.

    I have a question, I was hoping to import a high poly model and apply a bump map to it in xNormal, and have the final result used to bake the normal map. Is this possible? I had a look in the documents and the fine detail map seemed to just be a map that is placed over the normal map and blended in, not actually applied to the high poly model.

    If the bump map feature isn't in, would it be possible to add it? Trying to export very large models can be impossible at times as the computer runs out of memory and it would be nice to just export a high poly mesh from zbrush and the bump map with the highest subd level details.

    Either way, great work and keep it up smile.gif
  • jogshy
    Offline / Send Message
    jogshy polycounter lvl 17
    [ QUOTE ]
    I was hoping to import a high poly model and apply a bump map to it in xNormal, and have the final result used to bake the normal map. Is this possible? I had a look in the documents and the fine detail map seemed to just be a map that is placed over the normal map and blended in, not actually applied to the high poly model

    [/ QUOTE ]

    Atm that's not possible. This is one of the "requested features" I want to implement now hehe laugh.gif
    If the normal map is in object space the detail is just "blended" but if it's in tangent space it requires more calculations and could be really slow. However, I'm coding that now and making some tests

    On the other hand, I wanna implement Catmull-Clark algorithm to subdivide the lowpoly model if required. See the link of Silo I posted. Some people want to reconstruct the highpoly sculpted model using the lowpoly model + a displacement map ( like the famous ZBrush monkey head ). Is a problem because xNormal atm just calculates the distance between the lowpoly model and the high one... To make a displacement map compatible with that technique I need to calculate the distance from the subdivided lowpoly model to the highpoly model.

    Soooooooooooooooo.... I'm going to implement:

    1) Micro Triangle Displacement. That will allow you to specify a displacement map for the highpoly model and use it through the bake-highpoly-base-texture-into-the-lowpoly-model existing feature... This will operate WITHOUT subdivision, so will be fast and memory efficient.

    2) Lowpolygon subdivision using Catmull-Clark ( for Silo/ZBrush compatibility), but also more schemes ( loop, butterly, etc )

    3) The linux and MacOSX ports.

    So is going to be a nightmare confused.gif !!!
  • Mongrelman
    Offline / Send Message
    Mongrelman polycounter lvl 18
    I think I understood what you said hehe.

    Good stuff, glad to see you are keeping busy laugh.gif
  • MoP
    Offline / Send Message
    MoP polycounter lvl 18
    You're insane, Jogshy... and we love you for it laugh.gif
  • jogshy
    Offline / Send Message
    jogshy polycounter lvl 17
    Just uploaded the version 3.9.2 to correct a bug in the 3DS mesh importer ( was showing an incorrect zero radio message ).

    Also changed a few the SDK headers to make it easier to use.
  • Caravaggio
    Offline / Send Message
    Caravaggio polycounter lvl 17
    Hi, I started messing with this program at 3.7 and found it to be really promising. I hadn't had anything to seriously do until now though so I thought I'd try the latest, and the normal render works wonderfully, but there's a problem when I go to the 3d viewer. It gives me this message:

    "unexpected error showing the viewer 3d."

    It happens with the example files to and the viewer worked with the 3.7 version, so i'm not sure. I did a forum search for "showing the viewer 3d" and it returned nothing, is there a log file I can pull to check for something specific? Sorry if it's pointed out elsewhere.

    Great program though, keep it up.

    edt: I just noticed on the last page you talk about an ATI bug. I'm using uvw maps but I wonder if that doesn't have something to do with it.
  • jogshy
    Offline / Send Message
    jogshy polycounter lvl 17
    that error happens with the 3.9.2.41613 (for x86)? HAd some hosting problems and there was a corrupted 3.9.2.xxx version for 10mins that you were able to download... so sure the build is 41613, that's the correct one.
  • Psionic
    Offline / Send Message
    Psionic polycounter lvl 18
    Very useful App and incredibly fast, Just saying "Thanks" wink.gif
  • Caravaggio
    Offline / Send Message
    Caravaggio polycounter lvl 17
    Sorry, I re-downloaded it, un-installed it, deleted the folder, rebooted and installed - twice, once from each server, and i'm still getting it.

    Is there a registry entry or something that I should delete to completely get rid of it? Did I have a major brain crash, windows xp is the 32 bit right? I'm not sure what's happening. (I'm running an Athlon 64 3800 on windows media center edition, ati x1300.)
  • Mongrelman
    Offline / Send Message
    Mongrelman polycounter lvl 18
    Any chance you might have been running the 32 bit version instead of the 64? I don't even know if that would cause crashes, I'm just throwing ideas in smile.gif
  • jogshy
    Offline / Send Message
    jogshy polycounter lvl 17
    Caravaggio,

    The x64 version won't install on Win32 though, so you can discard that ( if you try to execute the x64 on win32 an error dialog will appear at install phase )

    There are two things you can try to erase:

    1) Registry->Current User->Software->Santiago Orgaz->xNormal v3

    2) Erase all the XML files in the xNormal settings folder.

    Also you can go to the plugin manager and change the graphics driver to use inside the 3D viewer. You can choose between OpenGL and DirectX. Try both and see. Probably choosing the DirectX one will solve your problems.

    If you have ATI Catalyst 6.6 WARNING... there is a nasty OpenGL bug inside those drivers. Use 6.5 or 6.7+ then

    Finally, you can edit with the windows Notepad the xNormal_debugLog.txt file to see some log error/warning information which can help to discover what is happening.

    Hope it helps.
  • Caravaggio
    Offline / Send Message
    Caravaggio polycounter lvl 17
    Boom! Wonderful. I deleted the registry and just used the same download. I wonder if I didn't actually have ogl checked before, I should have checked if I did before I uninstalled it. Something to keep in mind for later I guess. (just for reference, I had 6.7 catalyst)

    Thanks a lot, I really like this program and the bump map inclusion. I've really gotten tired of the ati mapper and those huge damn nmf files.
  • Mongrelman
    Offline / Send Message
    Mongrelman polycounter lvl 18
    I've been using this a lot for previewing models, but come across a problem. I was just using it and it was working fine, then one of the times I reloaded the textures (after updating in photoshop), the light will now no longer move. Any idea how to fix this? Restarting the program didn't help.
  • Mongrelman
    Offline / Send Message
    Mongrelman polycounter lvl 18
    I discovered what happened. I'd accidentally turned on 'cast shadows', which locks the light in position. After turning this off things went back to normal.
  • Jarrod1937
    Offline / Send Message
    Jarrod1937 polycounter lvl 15
    excuse me for sounding stupid, but i just recently got into generating normal maps from high poly meshes.
    what is the advantage of using this program over say, 3ds max 7's normal map generation?
  • EarthQuake
    Its easier to use than max, generally just works. Will trace from both sides instead of just one if it misses on one side(or atleast if max can do this i dont know how). You can write plugins for it to load your own custom mesh formats, which is awesome if you want to get exact normals/tangents from your ingame models. Ambient occlusion is quite a bit faster..... A big thing for me is that its *not* max, no need to use up 2 gigs of memory just to run the program. You can also import VERY high res meshes as long as your computer has enough ram i've rendered out highres stuff that has like 5 million polys, which would be damn near imposible to import into max without it being a huge hassle

    Pros of max would be its more robust/has more options and a better cage system. Maps are also slightly better quality at times.
  • Jarrod1937
    Offline / Send Message
    Jarrod1937 polycounter lvl 15
    interesting, i think i'll try the program after i get off work,
    and yeah, 3ds max slows down a lot for me after 3 million+ polys.
  • janimatic
    hello jogshy !
    We talked on the silo forum...
    so how is catmult clark implementation going ?
    I've found plenty interesting source code and start to explore this myself!
    I hope you are having a lot of fun !

    [ QUOTE ]

    I'm going to implement:

    1) Micro Triangle Displacement. That will allow you to specify a displacement map for the highpoly model and use it through the bake-highpoly-base-texture-into-the-lowpoly-model existing feature... This will operate WITHOUT subdivision, so will be fast and memory efficient.

    2) Lowpolygon subdivision using Catmull-Clark ( for Silo/ZBrush compatibility), but also more schemes ( loop, butterly, etc )


    [/ QUOTE ]
  • jogshy
    Offline / Send Message
    jogshy polycounter lvl 17
    [ QUOTE ]
    hello jogshy !
    We talked on the silo forum...
    so how is catmult clark implementation going ?
    I've found plenty interesting source code and start to explore this myself!
    I hope you are having a lot of fun !


    [/ QUOTE ]

    Very slow frown.gif I have almost null spare time now blush.gif
  • Daz
    Offline / Send Message
    Daz polycounter lvl 18
    At the risk of looking stupid, I'm getting this error:

    "Error loading the models: The mesh Default has vertex UV's but no vertex indicies! Please sure if you fill the mesh->UV's you fill the mesh->UVIndices"

    I have no idea what this means. 'Fill' the mesh?? I can only assume the 'mesh default' is referring to my low poly model? It's an obj exported from maya, with good UV's. Can't find anything in this this thread. Would appreciate it if anyone had any insight, thanks.

  • EarthQuake
    It means you have polys that have no uvs, either delete all uvs or uv those polys. You cant have only some polys uv its xnormal gets mad. This probablly in your highpoly mesh, right?
  • EarthQuake
    Hey i got a question about the extra bump thing, will that let you just load a bump map that gest applied to your highres, or will it let you load one that will get combine with your normal map after everything is generated?

    I've been doing a lot of object space normal maps and its much harder to blend in painted bump detail with them as apposed to tangent, it works some times depending on which direction the poly is face but not other times because of how i do it in photoshop(set blue to 127, and overlay)
  • Daz
    Offline / Send Message
    Daz polycounter lvl 18
    [ QUOTE ]
    It means you have polys that have no uvs, either delete all uvs or uv those polys. You cant have only some polys uv its xnormal gets mad. This probablly in your highpoly mesh, right?

    [/ QUOTE ]

    Thanks mate, but that doesn't seem to specifically be the issue. I've checked and triple checked and my lo is completely UV'd. I've tried applying UV's to my hi and also deleting UV's from the hi, but I get the same error.
  • arshlevon
    Offline / Send Message
    arshlevon polycounter lvl 18
    hey daz i dont know if this will work, but are all the uv's on one map channel? and is it the first map channel? i dont know if this is a problem with xnormal, my experience with it has been pretty painless. the high poly shouldnt matter at all, so i am assuming its a problem with the low. if you want you can shoot me the low and i can see if i can reproduce the error, might be your machine. have you tried xnormal with any other meshes? i cant seem to use the viewer for some odd reason, but it seems to bake out normals just fine.
  • EarthQuake
    Yeah if you have multiple channels with obj it will just read the first. Whenever i get that error its usually unuved faces in the high so i dont know, if its something other than that... what are you exporting from? And arsh: high uv's do matter too btw, it needs them for diffuse baking. Even you you dont have that enabled it still checks for them.
  • Daz
    Offline / Send Message
    Daz polycounter lvl 18
    OK I'm a tard, I figured it out thanks guys. It wasn't a UV issue at all I don't think. At some point my maya>.obj export options had changed. I re-ticked materials, smoothing and normals to 'on', re-exported the meshes and it worked.

    Thanks for your help tho'.
  • jogshy
    Offline / Send Message
    jogshy polycounter lvl 17
    [ QUOTE ]
    will that let you just load a bump map that gest applied to your highres, or will it let you load one that will get combine with your normal map after everything is generated?

    [/ QUOTE ]

    The idea was to allow you to UV-texture-map the highpoly model with a heightmap, but I discarded it because that requires to calculate the tangent space in the highpoly model which can be really slow. On the other hand I could use the lowpoly tangent space, but I bet won't be accurate enough to produce good results confused.gif

    To add fine detail ( if you don't like the xNormal method ) try to use photoshop layer compositing + NVIDIA height-to-normalmap plugin. I think I've seen a tutorial by Mr.Poopping or Ben about that?

    ps: I'm uploading again the 3.9.3 because some people told me the mirrors were down/broken ...
  • EarthQuake
    Jogshy: that works fine for tangent space maps, but it dosent work for object space all of the time because of the color range it uses, do you know of a way to combine a hieght map with an object space map? Its easy to do with tangent yes.
  • EarthQuake
    heres an example, some bump detail i added to this normal in photoshop, on the top of the gun where the normals is all green it dosent seem to be overlaying in a way thats "correct" you can see some bump detail in game but i'm pretty sure its not being done properly. I circled those ares, you may have to zoom in to see.


    http://johnyontehspot.com/pix/c96normals1.jpg
    and without bump:
    http://johnyontehspot.com/pix/c96normals2.jpg
  • Joshua Stubbles
    Offline / Send Message
    Joshua Stubbles polycounter lvl 19
    Any chance of getting a generic cubemap lighting system of some kind?
    Something to mimic the overly-complex spherical harmonics would be great.
  • jogshy
    Offline / Send Message
    jogshy polycounter lvl 17
    [ QUOTE ]
    Any chance of getting a generic cubemap lighting system of some kind?
    Something to mimic the overly-complex spherical harmonics would be great.

    [/ QUOTE ]

    You can do cubemap diffuse lighting atm ( the diffuseGI thingy ) using the wonderful cubemaps that Ben donated!
    I wanna myself to practice a few with the spherical harmonics blush.gif
  • Joshua Stubbles
    Offline / Send Message
    Joshua Stubbles polycounter lvl 19
    Well, the way you have the "diffuseGI" setup isn't really lighting at all. It's a reflection map. If you use a high contrast image in that slot, the mesh comes out looking like a Christmas bulb. :P
  • Joao Sapiro
    Offline / Send Message
    Joao Sapiro sublime tool
    hey jogshy , check you PM wink.gif
  • Eric Chadwick
    [ QUOTE ]
    Well, the way you have the "diffuseGI" setup isn't really lighting at all. It's a reflection map. If you use a high contrast image in that slot, the mesh comes out looking like a Christmas bulb. :P

    [/ QUOTE ]
    But the point of diffuse cube lighting is to use a heavily-blurred image... see ATI's page for pics.
    http://ati.amd.com/developer/cubemapgen/index.html

    We use a different projection method for our diffuse cubes, it doesn't factor in the camera angle the way reflection does, but still if I use a non-blurred image it looks more like reflection than GI lighting.
  • Eric Chadwick
    Hey EQ, the bumps in those green areas, you could just invert the red/green channels of the normalmap being overlaid.

    But that doesn't really solve your problem. You need a way to get world space lighting into Photoshop, AND relative to the original surface directions.

    Might be a way in Body Paint though, you could project the details onto some crappy auto-UVs on the highpoly model, then bake from there using world-space lighting.

    Cool gun.
  • Joshua Stubbles
    Offline / Send Message
    Joshua Stubbles polycounter lvl 19
    oooooooh I see.

    There practically has to be no detail in it. Lol, much better now. Thanks for the clarification, Eric smile.gif
  • katzeimsack
    Offline / Send Message
    katzeimsack polycounter lvl 18
    http://www.crytek.com/images/gallery/cryengine2/facial_editor_2.jpg
    this crysis screen seems to use some kind ob cubemap lighting..
  • Joshua Stubbles
    Offline / Send Message
    Joshua Stubbles polycounter lvl 19
    I think Crysis uses the hardcore lighting of spherical harmonics. It's pushing everything else next gen, so I don't think they're going with something as simple as a cubemap.
  • Caravaggio
    Offline / Send Message
    Caravaggio polycounter lvl 17
    Again, sorry if it's asked before but culling this huge thread hasn't found me a similar problem. The program itself still works beautifully (though I do find myself playing with the fine details map to retain a decent contrast) so I'm thinking this one is a problem of my original meshes.

    Here's the normal map I'm making.
    rcspillarstoneheads01noia3.th.jpg
    ...The squares in the lower left are whats bothering me the most. Imagine a column with simple doric capitals, those make up the squares for it. The low poly mesh is squared, all the sides are perpendicular and parallel with hard edges. The high poly is basically the same thing with beveled edges as you can see by the normal map. What I can't figure out is why each side gets darker in the lower half. It's a flat surface in both models. My only guess is that it has something to do with the smoothing/phong settings in the obj. So I went back and actually disconnected all the sides of the square part just to be sure, and it still gives me this gradient color.

    Any ideas would be greatly appreciated.
  • EarthQuake
    thats to compensate for the normals(smoothing) of your mesh, unless you set the normals to each face(flat shaded) you will get gradients like that. If your lowres mesh is as simple as i think it may be you're probablly going to have to do that to get any decent results using tangent space without adding extra edges on your lowres to give you more accurate smoothing(chamfer to give a 45 degree angle on your hard edges isntead of 90).

    Or you could just use object space and it will work properly, just make sure your programers impliment it right. =)
  • Joshua Stubbles
    Offline / Send Message
    Joshua Stubbles polycounter lvl 19
    Yup.
    Object space = t3h s3x for props.
    Problem is, you can't really edit them in "post" that much. Not like you can with tangent space maps, anyways.
    That's where apps like Zbrush and Mudbox really help out.
  • jogshy
    Offline / Send Message
    jogshy polycounter lvl 17
    Caravaggio, can you post a screenshot of your lowpoly and highpoly model pls?

    I suspect is another mess with the Maya/3dsmax .OBJ exporter... I've seen tons of normal problems exporting with max2obj and other 3rd-party OBJ exporters. I put an example in the xN docs showing all this.

    Try to export the column in other format like MS3D/XSI/FBX/Collada/ASE ( and remember 3DS/LWO does not store vertex-per-face normals ).

    Sure too you are NOT using the "smooth vertex normals" inside xNormal ( which averages the vertex normals killing any hard edge ).

    Also, when you put your model into the 3D engine sure the programmers allow to use multiple vertex normals per "vertex", so the "hard edges" can be shown correctly.

    Btw, in order to optimize, xNormal always averages in the 3D viewer ( only in the 3D viewer, not in the normal map generation process ) the highpoly vertex normals. If you make a hard edged highpoly model the 3D viewer will show it always smoothed. This was to avoid the previous versions "de-indexing" slow step... OGL/D3D uses one-vertex-indexing only and to draw 2M triangles without vertex indexing was really slow. This does NOT apply for the lowpoly model and neither for the highpoly model outside the 3D viewer ( vertex normals are fully respected in these cases ).

    Hope it helps.
  • Caravaggio
    Offline / Send Message
    Caravaggio polycounter lvl 17
    Checking off the tangent space box seems to actually make the gradient an even higher contrast, and I don't really have a programmer to talk to as I'm trying to make static meshes for the upcoming unreal tournament game.

    Here's the model I made, in the editor they released for RoboBlitz.
    painpainpaintemp5bv1.th.png
    It gets more obvious if I use dynamic lights.
    painpainpaintempub9.th.png
    You can see where the gradients are causing problems in the top and bottom section sides. It's kind of depressing now though, even without the normal map problem I'm thinking of dropping this particular mesh.

    And I should have mentioned this before, but I'm using Cinema 4d, not maya or max. It doesn't exactly have the most robust obj exporter, so I wouldn't be surprised if that had something to do with it. It uses edge breaks instead of smoothing groups, so... ...yeah, now that I think about it, huh. Damn.

    I'm going to call a friend who has max and see the fbx or ase fixes it. I suppose I should have tried that first. Let me do a test or two and I'll report back so you have a ready answer for anyone else that might be using c4d too.

    It might also just be that problem outlined in 2.4.3, the missing space of normal project on hard edges. I need to rethink this one a little bit. Not exactly used to this whole normal map thing as you can see.

    edt: ase is giving me similar results. subdividing the the low poly mesh without smoothing just makes the colors go all wonky. Here's the meshes, the 90 degree low poly/smooth high poly isn't my biggest worry right now, unless you think that's part of it.
    painpainpaintemp10nt0.th.png
  • Joshua Stubbles
    Offline / Send Message
    Joshua Stubbles polycounter lvl 19
    I'm not sure I see the issue you're talking about on the sides of the object.

    If you're seeing some sort of color banding, I'm not seeing it. Are you using an LCD screen? Some can't display gradients very well, and give the appearence of banding.
  • jogshy
    Offline / Send Message
    jogshy polycounter lvl 17
    Caravaggio,

    I suspect that can be:

    1) OBJ exporter is messing your vertex normals. Try to export a simple cube... Edit the .OBJ with the Windows notepad and check the "vn" elements ( vertex normals ). If you see all the normals the same or not coherent that will be the cause.

    2) Perhaps C4D uses a different tangent basis coordinate system. Does C4D use UV origin at left-bottom? Perhaps you will need to use the "swizzle coordinates" to fit the OpenGL system. Also I see in the "perspective viewport" that C4D is using a Direct3D coordinate system ( with Z pointing from near to far ) and not an OpenGL one.
    Try to do this... When you generate the normalmap with xNormal specify swizzle as "X, Y, -Z" or "X, -Y, Z" and try again.

    3) Some programs ( like 3DSMAX ) need to do a "reset XForm" to update the normals and bounding box of a "moved" object before to export... Not sure what C4D does but if there is a button to update the object bbox and rotated normals, press it before to export from C4D. Also sure to "unify" your vertex normals ( sure all the vertex normals are pointing to outside ).

    Hope this helps.
1568101159
Sign In or Register to comment.