Home Technical Talk

xNormal - MASTER THREAD

1246759

Replies

  • CrazyButcher
    Offline / Send Message
    CrazyButcher polycounter lvl 20
    ... And lead us not into temptation ... wink.gif
  • jogshy
    Offline / Send Message
    jogshy polycounter lvl 17
    Like these ones, Pior?

    xN_shadows.jpg

    I'm using an anisotropic-variance-cascade-no need bias-hard on contact shadowmaps approach btw! Superslow of course frown.gif

    Mwahahaha! Ooooh, you haven't there the v3.6.0 yet tongue.gif Again, wannna thx to Pablo Hoyos for his model. I think I need new models, they're getting a bit old mwahaha!

    Need more tests before to release this new version but in a few... I wanna finish the 3.5.2 with some new things before.
  • MoP
    Offline / Send Message
    MoP polycounter lvl 18
    OHHHHHHHH! Lovely! laugh.gif
    I can maybe give you an example model if you want ... maybe the general?
  • EarthQuake
  • DDS
    Offline / Send Message
    DDS polycounter lvl 17
    Felicidades amigo smile.gif xNormal roxx
  • jogshy
    Offline / Send Message
    jogshy polycounter lvl 17
    That would be amazing MoP!
  • Joshua Stubbles
    Offline / Send Message
    Joshua Stubbles polycounter lvl 19
    That's HOT.
    You're adding fresnel also?? You're too damn good, jogshy smile.gif
  • EarthQuake
    Im getting a really bad crash here, after using the program for a while, i dont know how long it takes to do this, when i go into 3d viewer it will almost take down my entire system. Its such a hard crash it litterally makes my screen shake, really the video output starts to shake... After a little while it will give me mouse control back and i can kill it in task manager. Restarting the program dosent fix it, i have to reboot for it to work right again.

    On a p4 3.0, 3 gigs of ram with a 6800 GT
  • jogshy
    Offline / Send Message
    jogshy polycounter lvl 17
    Are you using WinXP 64bits or 32? 3Gb of ram using WinXP 32bits is DANGEROUS!!!!!!!!!!!!!!!!

    See this link:

    http://www.gehrytechnologies.com/catia/catia/catia_windows_large_memory.htm

    [ QUOTE ]

    Some driver may not load correctly when /3GB switch is added to boot.ini


    [/ QUOTE ]

    WinXP 32bits can handle usually 2Gb max... The 3Gb model is a trick and is HARD to configure it WELL!

    xNormal doesn't use the 3Gb model. I disabled it specifically some time ago ( check history in the docs, can't remember when ) because some people had problems with it. It is possible that your graphics driver doesn't like the 3Gb, specially if you run outta video ram ( see the indicator text inside the 3d viewer )... I would like to have 3Gb RAM to test, but unfortunally i'm pretty poor haha!
    Please sure the 3d viewer is not using more VRAM than your in-card VRAM and that your "AGP aperture size" is enough in case you run outta VRAM.

    I don't know whats the solution though ( prolly migrate to Win64? ). Atm can't compile xNormal for 64 bits because the 100 3rd-party libraries I am using ( xsi FTK, FBX, blah blah ) still aren't ported and can't mix 64 and 32 bits libraries... confused.gif

    So all I can do is

    A) See the routines again to check for any error.

    B) Recompile xNormal for Win64. Bad... Half the libraries can't be ported due to lazy 3rd party developers. Can't mix 32 and 64 libraries... Not sure if you can use >2Gb on Win64 for a 32-bits-compiled program.

    C) Re-enable 3Gb model on xNormal. Some people reported me errors when I had this enabled...

    D) Try to make xNormal to consume less memory. Sure! In fact i am working in a new sampling method which consumes 4x less memory than supersampling!

    So well... Does it occur every time you run the 3d viewer or after 20-30 application executions? Perhaps I'm not releasing well the textures or something and I am far far far from being a good Win64 programmer.
  • EarthQuake
    What happened was i rendered out normals a few times, viewed it fine. Then messed with the model in zbrush, re-exported the high res, rendered maps a few times until i got results i wanted and went to go back and test in the viewer again and it crashes and soon as i hit the button, so i cant tell you if its using too much vram or whatnot. Tomorrow ill try and take 1 gig out and see if that fixes the problem because really this program rocks and i could sacrafice a gig for it =)

    [edit] This happened twice to me, but the 2nd time i didnt reboot to fix it i just posted and went home.
  • jogshy
    Offline / Send Message
    jogshy polycounter lvl 17
    Try to avoid full screen antialiasing video modes btw... They take like 2x more VRAM memory.

    Are you using Win64 btw Earth? I think could cap a few(no dotXSI, no sound) the xNormal and provide a AMD64 version of the next version (3.5.2) to pass 2Gb RAM with no problems.
  • EarthQuake
    No its 32 bit, no fsaa.... Is there a way to run it in windowed mode btw? Having to use it in fullscreen is rather anoying, esp if i wanted to use it for texturing.
  • Joshua Stubbles
    Offline / Send Message
    Joshua Stubbles polycounter lvl 19
    Agreed. Windowed mode would be WAY better for the 3d viewer, I think. Just easier to deal with.
  • DDS
    Offline / Send Message
    DDS polycounter lvl 17
    as I usually make models with a lot of hard edges, I wondered if xNormal can create good normal maps for them, using just 1 Smoothing Group, you know...

    I've been surprised of seeing that in xNormal viewport it looks almost perfectly, while in Max and our engine it looks pretty crappy (basically the normal result...)

    What do you think about it?
    This is the Max screenshot (remember it looks like that in our engine too)
    This is how it looks in xNormal, almost perfect
    this is the normal map

    would be nice to know how you handle the normal maps to look so good!

    thx!
  • CrazyButcher
    Offline / Send Message
    CrazyButcher polycounter lvl 20
    very likely he computes tangents differently from 3dsmax and your engine. Jogshy, are you using nvmeshmender for tangent stuff ?
  • pior
    Offline / Send Message
    pior grand marshal polycounter
    Holy crap that's hot!

    Can you tweak the edge sharpness on these? But hey it's looking superb atm anyways laugh.gif

    Oh and btw, do you have multiple spotlights?

    I can also provide some other models if you need some. And I guess some people here could give a hand on the UI side of thing, to make your app even more kickass!

    Okay, ordering Dell NOW.
  • Snowfly
    Offline / Send Message
    Snowfly polycounter lvl 18
    my real job is software UI designer. smile.gif i hacked into the .exe but couldn't find the skin file anywhere!

    actually i've been debating whether or not to offer to help with the UI and decided against it. i kinda like the jogshy "programmer art" flavor.. besides the interface is efficient, and clear. although there are places where the workflow can be optimized (i.e. placing the high and low input meshes on the same tab). mop and eq seem to be using it a lot.. any good ideas for workflow improvements? smile.gif

    but since that's out in the open.. jogshy, mind if i send some ui mockups your way?
  • jogshy
    Offline / Send Message
    jogshy polycounter lvl 17
    First, thx all for the offers and the good feedback! Always need models and things to test and experiment! If you want I am always open to get nice art. Use jogshy@mixmail.com for small things or admin@santyesprogramadorynografista.net for large ones.

    Pior, take into consideration that in 6 months ( when Windows Vista comes to life ) you will need to buy other graphic card to run DX10 things tongue.gif

    Nope, I don't use NVMeshMender/NVTriStrip. I never liked it. Prefer my own de-indexing + stripping + TB calculation.
    Btw, the way I calculate tangents etc creates tons of vertices... the new version I am preparing optimizes a lot the meshes maintaining their aspect ( thx Earth for the advice on UV welding etc )

    I am planning UI skin(for the main UI) support for v3.5.2. Basically you will edit a very simple and easy XML file with the image names and some locations and sizes.
    For the 3D viewer, you can customize it using LUA ( is used to contruct World of Warcraft UIs or scripts in Far Cry )... I know it sux but originally wanted to experiment a few with scripting... Is a good way to customize menus, that's why tons of games uses it for their UIs.
    The problem is that the program has grown too much and now we have the entire screen covered with controls mwahaha! blush.gif To be honest I don't know why the hell I decided to use fullscreen controls and not stardard Windows things... Prolly because I tend to experiment too much with weird and ugly things hahaha! smile.gif

    I could pass the entire application to windowed mode like 3dsmax, maya or xsi if you want and use standard Windows controls ( well .NET ones tongue.gif ) just tell me what do you prefer. I love viewer in fullscreen because looks much better and you can use with multiple monitors but in that mode standard Windows controls can't be used and is hard to program...But for the Mac version I am preparing is also good because I can port very easy the OpenGL controls... If I design the viewer3D using Windows controls then I will have problems...

    To avoid headaches I will just add a checkbox to run the viewer in windowed mode laugh.gif

    See ya!
  • Eric Chadwick
    What are you using for your cubelight? Will it be customizable?

    I like where this is going, tons of features, it looks great.
  • jogshy
    Offline / Send Message
    jogshy polycounter lvl 17
    I will let you specify a cubemap for low-frecuency lighting. You can control the modulation color and intensity. You will need to create the cubemap with gauss-blur a lightprobe or something like that.. Have 0.0 experience creating this kind of cubes so.. haha! Any advice will be welcome grin.gif
  • Eric Chadwick
    Cubelights are great, we use them here a lot.

    ATI has a great tool for this, someone here turned me on to it...
    http://www.ati.com/developer/cubemapgen/index.html
  • Mongrelman
    Offline / Send Message
    Mongrelman polycounter lvl 18
    Wow, more awesomenessness from Jogshy (aka "The Machine"). Can't wait to get back to my own comp so I can play with this more.

    Would it be possible to generate displacement maps with xNormal? I know nothing about programming (except that it makes my head hurt) so have no idea if it is possible/too much work etc. But it would be great to have a free program can that can generate displacement maps from an arbitrary mesh. Apparently Melody can, but it just preferred to crash whenever I tried it.

    Also, as soon as you save a texture file in photoshop for example, will xNormal immediately update or do you have to reload the texture and go back into the 3D viewer. I'd check, but this isn't my computer frown.gif

    Three thumbs up! wink.gif
  • vahl
    Offline / Send Message
    vahl polycounter lvl 18
    man that some awesomeness, I have a big fat polied model to render, and xsi is driving me nuts these days so I'll definitely have a try at it !!! will give feedback asap, I'd be happy to provide models too.

    thanks again man, this prog seems to be already way better than 95% of the ones available to me right now.
    thanks a lot for keeping us updated !!
  • jogshy
    Offline / Send Message
    jogshy polycounter lvl 17
    Can the cubegen generate the diffuse lighting terms from a devebec light probe, Eric? Is not only to blur the ugly edges those appear in the faces when the graphics card filters it? I am trying to download more light probes and photoshop-gaussian-blur them as MoP suggested but my skills lack...

    To lupus:
    Yep yep, xNormal can generate displacement maps(signed,biased). Just select a file format with alpha channel and will be generated automatically with your normal map smile.gif

    If you are working inside the 3D viewer and you wanna change a texture just ALT+TAB, open photoshop and change it. The ALT+TAB back to xNormal 3D viewer and press the "Reload textures" button on the right wink.gif
  • KDR_11k
    Offline / Send Message
    KDR_11k polycounter lvl 18
    Can you make it possible to run the viewer in a window to make it easier to use that way?
  • Eric Chadwick
    [ QUOTE ]
    Can the cubegen generate the diffuse lighting terms from a devebec light probe, Eric? Is not only to blur the ugly edges those appear in the faces when the graphics card filters it? I am trying to download more light probes and photoshop-gaussian-blur them as MoP suggested but my skills lack...

    [/ QUOTE ] CubeMapGen includes all six bitmaps when it filters so you avoid edge discontinuities. Photoshop doesn't, so you get bad edges.

    CubeMapGen needs cubemap input, so those lightprobes need to be either six separate bitmaps, or if they're single bitmaps they need to be in Cross format or in DDS cube format.

    ATI wants a Cross format like shown here...
    http://gl.ict.usc.edu/HDRShop/tutorial/tutorial4.html
  • jogshy
    Offline / Send Message
    jogshy polycounter lvl 17
  • EarthQuake
    Personaly i would prefer to have standard 3d viewer controls, orbiting the mesh instead of the camera, but have the you controls can be nice too, it would be great if you could switch between the types of controls, because yours are more representive of how you would actually see a model in game, whereas orbiting model controls really help you look at every little part of the model quickly. Also an option to keep the light always emiting from the camera would be good, not just setting it to where the camera currently is.... Having an option for windowed mode would definately be a WIN in my book, you bring up good points as to why you use it so i think it would be best to have the option of doing both, currently in fullscreen it makes it hard to use it as a viewer while texturing because its fullscreen, it would be much easier to be able to quickly switch back and forth to photoshop if it was windowed. Also speaking of texturing could you make it auto update image maps? That would be nice.

    Ignore this stuff if some of it i've already said hehehe.

    [edit] Oh yes also i echo DDS's statements, if you could provide some more info on exactly what you do that we could pass on to our guys working on the engine so we can sync up how you display normals and how our engine displays normals that would be fantastic.
  • Jesse
    Offline / Send Message
    Jesse polycounter lvl 17
    Yes, this seems like an awesome tool but... we can't really make any good use of it unless we have a way to get your processed meshes in to other DCC tools or our engine. Do you have any plans for solving this? If this will happen via some exporter from your program, how will you handle things like skinned meshes?

    Really, in order for this tool to fit in to any pipeline it must either a) Not do anything proprietary to the mesh that can't be returned to the originating program or b) must sit as a processing step for final exporting to a format like .X or c) you must give programmers the info they need in order to process meshes exactly like you are doing it.

    Anyway, great job! I hope we can use your tool in production some day.

    Jesse Rapczak
    Boanerges Studios
  • EarthQuake
    Actually, wouldnt it be posible to simply mimic his process when we import meshes in game? If we're treating the meshes exactly the same when we import as he does when he imports shouldnt everything work properly?
  • Eric Chadwick
    AFAIK, for best normals the game importer has to recreate the tangent space per vertex. If xNormal is calculating these, it would help if it output them too.
  • CrazyButcher
    Offline / Send Message
    CrazyButcher polycounter lvl 20
    hence I was for nvmeshmender. it would have been a standardized way.

    eric, the problem is, noone knows what kind of vertices xNormal creates, considering the statements about the UV welding, and such it sounds like jogshy is creating quite many. At least more than max or most engines would do (judging by DDS's stuff)

    maybe some sort of interface to allow users to plugin their own tangent calculation ? that way every engine maker could use their own tangent solution, and work from their original vertexdata.
  • jogshy
    Offline / Send Message
    jogshy polycounter lvl 17
    I save a .ovb(openGL vertex buffer) XML simple file with the with the data you use in the viewer ( pos+normals+UVs+tangents+binormals+vcache optimized triangle faces+mesh name ). See the documentation and xN_cache directory for more info.

    You can tell your programer to parse it easy ( I include even a schema called ovb_schema.xsd )... If they use VS2005 is as easy as drag and drop the schema inside and it will autogenerate a .H file to read the DataSet.

    And yep I need to think a few how to pass back the xNormal data to your programers better. I see three options:

    A ) I could provide a small .LIB for C++ or a DLL to calculate the tangent basis as xNormal does. The programmer passes me triangles/vertices/attributes and I will return a nice mesh optimized and very complete structure. This library could be used in your exporter or engine. Sounds cool. I do my own NVMeshMender but working... This is going to be my first target because is the most flexible. Will do it in C++(.LIB and DLL) and .NET assembly too. Sorry , no Java I mad.gif that language badly.

    B) Plugin system like suggested... You make a plugin DLL that calculates the TB and then will be used inside xNormal to make the maps. Sounds good. This is my second target. Will implement a plugin system for xnormal once A) is completed. I will let you use your own raytracers and importers/exporters.

    C) Make a Maya, XSI and 3DSMAX importer/exporter. I don't like this option because when I import the xNormal output into Maya prolly will destroy the tangents/binormals .Also, the things I can export prolly won't fit well all the people. This is the way the ATI normalmapper worked and I think NMFs sucked. I am not sure about this option, specially when COLLADA is coming.

    Don't worry, a small SDK will be included with the v3.5.2 ( a small C library like NVMeshMender, ok? ) so you will be able to use it in your exporters and calculate all in the same way xNormal does.
  • CrazyButcher
    Offline / Send Message
    CrazyButcher polycounter lvl 20
    do you have some sort of "original vertex id" list ? that way one could map old vertex attributes (which are not in the .obj) to new generated vertices by you. (useful for the skinned stuff that Jesse mentioned)
  • EarthQuake
    Heres another feature request, would it be posible to get a window that shows you the normals or ambocc map while its rendering?
  • jogshy
    Offline / Send Message
    jogshy polycounter lvl 17
    Do you mean to show the pixels of the rendered-normalmap-faces in a window while is generating the normalmap or to put a textured rectangle in the 3D viewer with those textures? Think the time I spend rendering something in the screen is time that is added to the total process.
  • EarthQuake
    showing it whiles its generating, yes it would add time to the render most likely but the reason for this is so you can quickly see if something has generated wrong and stop the render.
  • EarthQuake
    im having lots of problems today with xnormal freezing while trying to render stuff, and telling me i have curropt memory when i set supersamling to 4x(2x works properly, but turning on ambocc with a mesh like this will just cause it to stall somewhere durring the render process) with a full-res zbrush model loaded(dont know how many polys, 180mb obj file) Im gonna take my work home and test it there most likely.
  • jogshy
    Offline / Send Message
    jogshy polycounter lvl 17
    Yeah I can add it a >>>SMALL<<< window to see the preview as is generating...

    Supersampling is a VERY memory cost operation as is implemented in the v3.5.1... I am changing it for adaptive multisampling for the 3.5.2 because consumes too much memory.

    To get maximum speed use cages.

    Also I am experimenting with the 3Gb model and have almost ready the x64 version (4Tb ram limit), just waiting Avid for some libraries.

    The v3.5.2 will be ready in very few days with a very preliminar SDK btw
  • Downsizer
    Offline / Send Message
    Downsizer polycounter lvl 18
    Is your hope to keep this a free application? I'll post screens in a week or so when i'm not so busy, but it gives great results. Not that I would mind paying for it.
  • jogshy
    Offline / Send Message
    jogshy polycounter lvl 17
    Okie a small change... will release the v3.6.0 and skip the 3.5.2... the list of changes is too much large for a minor version... The SDK I am preparing is changing ALL. Forget the "small" C library... I am making a collosal SDK and rewriting all as plugins to allow full customization. An example.. you can implement your own raytracer, mesh importers and exporters with custom vertex data ( allowing to import/export skinning info ), image loaders/savers and custom UIs... As well as new normalmap adaptive multisampling method ( which kills that crappy supersampling I-want-all-the-memory-in-your-PC-and-also-the-neighbor's-ram-too ) and tons tons tons of new things including a x64 version for persons with more than 2Gb of ram.

    I need some time to complete this >> never-ending << task list ooo.gif heheh You will be very pleased with the v3.6.0 I am sure. Just need a few more time for this upcoming mayor release, like a month tongue.gif

    And yes, xNormal is and will be always free. I will put a donate button in a few in my web once I consider xNormal is really ready for serious use because atm is far far far from what I wanna get grin.gif
  • thnom
    Offline / Send Message
    thnom polycounter lvl 18
    Jogshy you are a mammoth. You're like doing this for free but not only that you're listening to the people who its aimed for. You're a hollywood star to the community!
  • bearkub
    Offline / Send Message
    bearkub polycounter lvl 20
    Jogshy: If you need some mirrors to host your files, let me know.
  • jogshy
    Offline / Send Message
    jogshy polycounter lvl 17
    Thx bear, in fact I need mirrors because my web has very poor bandwith and these free upload sites just host the files 20 or 30 days before they erase the file. So yep yep! How can I upload to polycount or those mirrors the program please? grin.gif
  • KDR_11k
    Offline / Send Message
    KDR_11k polycounter lvl 18
    Hm, an option to ignore backsides, i.e. ignore surfaces of the highpoly model with normals that differ more than 90° from the lowpoly normal? That'd help in close spaces.
  • jogshy
    Offline / Send Message
    jogshy polycounter lvl 17
    OK!

    Just uploaded the version 3.6.0 BETA 1...
    I decided to put a small beta 1 before to release the final version because touched too much code and I'm pretty sure any ugly error will appear laugh.gif

    Changes include a complete C++ SDK with documentation, a new and improved antialiasing method that consumes 8x less memory and is much faster, superimproved glow effect, now xNormal is a full-plugin-oriented application, windowed mode, MoP's ubah example!, FBX mesh import is back, improved cage vertex picking, customizable camera sensibility, abort on model-reload, improved customizable tangent basis and UV welding, improved alpha-sorting, 3x shader optimization, mass center orbiting, 3Gb memory model, xsi5/collada 1.4.0 new importer, added a low-memory-consuption new raytracer for 90zillions of polys models, blah blah blah...

    With the new SDK and plugin system I think you can re-export the skinning bone info, vertex colors and other data. You can write mesh importers/exporters, image importers/exporters, tangent basis calculators, raycasters and graphics drivers. It comes with a quick guide in PDF and a DOxigen complete CHM doc. C++ .h files and VS2005 .libs and code examples are included.

    The bad news are that the program installer has grown to 23Mb.

    As usually:
    Change log is in http://www.santyesprogramadorynografista.net/archives/xNormal_changes.txt

    Download links are in
    http://www.santyesprogramadorynografista.net/projects.aspx

    Feel free to test it a few and any comments are welcome.

    ps: I know text is not well centered in viewer buttons and that the performace of the viewer 3D is not good atm :P
  • EarthQuake
    Holy crap thats one awesome list of features. You are easily the coolest programmer ever. Ill try to find some time to run some models through it.
  • MoP
    Offline / Send Message
    MoP polycounter lvl 18
    WOOOHOOOOOO! laugh.gif

    Jogshy returns bearing gifts! smile.gif

    I was sitting here just waiting for v3.6... yay! I'm downloading it now, gonna test it out with some things, and see if I run into any bugs wink.gif

    Thanks again!
  • ironbearxl
    Offline / Send Message
    ironbearxl polycounter lvl 18
    WOW jogshy, thank you for the hard work!

    When the program calculates normals with multiple meshes imported in different slots does it do them seperately in the order they were imported or all as one mesh?

    If it could do them seperately and then combine them all on a single normal/height + ambient map it would be a major time saver!
  • jogshy
    Offline / Send Message
    jogshy polycounter lvl 17
    [ QUOTE ]

    When the program calculates normals with multiple meshes imported in different slots does it do them seperately in the order they were imported or all as one mesh?

    If it could do them seperately and then combine them all on a single normal/height + ambient map it would be a major time saver!

    [/ QUOTE ]

    Yep, atm you can batch-render multiple objects and combine into the same normal map. xNormal will start rendering from top to bottom order ( in the lowpoly mesh list). This is specially good for tweaking floating geometry(touching the cages or the ray length independetly for each mesh) or to post-composite the normal map without using photoshop.

    Ouch I detected two bugs very strange:
    1) With certain cards, if you enable the full scene antialiasing the vertex picking doesn't work correctly ( can't lock-n-read a multisapled AA framebuffer to read z buffer?? )

    2) In some ATI cards the text is drawn blurred. If appears these cards can't disable the FSAA dynamically. Once you enable it you can't disable it...

    thx in advance for all the feeback!
1246759
Sign In or Register to comment.