Home Technical Talk

xNormal - MASTER THREAD

1141517192059

Replies

  • jogshy
    Options
    Offline / Send Message
    jogshy polycounter lvl 17
    [ QUOTE ]
    I think there's a memory leak there somewhere.

    [/ QUOTE ]
    Ok, thanks MoP, I think found it.

    [ QUOTE ]
    Apart from that it all seems good! smile.gif

    [/ QUOTE ]
    Discovered a problem toggling the video capture related to the F9 key

    Btw, do you know any good free codec apart from XviD? I'm gonna include it together with xNormal... is a good MPEG4 encoder... The default WinXP ones sux hehe ( indeo, Video1, etc )
  • dur23
    Options
    Offline / Send Message
    dur23 polycounter lvl 19
    Hey santiago,
    I'm getting that wicked hangup when rendering the AO. What was the issue that was discovered?

    (edit: for some strange reason i thought your name was josh....)
  • jogshy
    Options
    Offline / Send Message
    jogshy polycounter lvl 17
    [ QUOTE ]
    Hey josh,
    I'm getting that wicked hangup when rendering the AO. What was the issue that was discovered?

    [/ QUOTE ]
    In the new GPU AO tool or the typical method?

    In case the typical one:
    If you have a Core 2 need to install the microcode patch using Windows Update.

    Notice the AO can be VERY slow(hours) for complex meshes ( use 512x512, no AA, 16 samples per pixel ) and render the smiley example AO to test. If renders well but your model is slower... is not a hang really... just excesive rendering time tongue.gif

    In case the GPU tool:
    You need a Radeon 9500 or GeForceFX or above card.
    If goes terrible slow is because OpenGL enters a known software emulation mode which is a pain. Play with the "precision" mode until the speed is good ( and see the recommended settings table in the docs ).
  • dur23
    Options
    Offline / Send Message
    dur23 polycounter lvl 19
    So far it (with the smiley face of next-genishness) hangs on waiting completion. It has sat at this stage for 10 minutes now the progress bar hasn't even started yet.
  • Joao Sapiro
    Options
    Offline / Send Message
    Joao Sapiro sublime tool
    yo jogshy i still get the amboc to freeze and hitting the abort button wont abort it and ill have to ctrl alt delto close it. i tried rendering with a 512x512 amboc 16 samples and still nothing frown.gif
  • ScoobyDoofus
    Options
    Offline / Send Message
    ScoobyDoofus polycounter lvl 19
    Jogshy: For some reason when I export obj files from Silo into Xnormal they are invisible. I've tried several combinations of options in the export dialog box to no success. If I export from Silo into Max it looks fine. I just re-export the exact same thing into Xnormal, and Viola! It works. So...what gives?
  • jogshy
    Options
    Offline / Send Message
    jogshy polycounter lvl 17
    [ QUOTE ]
    So far it (with the smiley face of next-genishness) hangs on waiting completion.

    [/ QUOTE ]
    "Waiting completion"? That happens if you use the hxGrid distributed rendering without enabling the coordinator/agent. Are you using the hxGrid distributed renderer or the Default Scanline one? If you use the distributed renderer need to open one coordinator + some agents... or change to the default scanline renderer just to use your local machine.

    [ QUOTE ]

    Jogshy: For some reason when I export obj files from Silo into Xnormal they are invisible. I've tried several combinations of options in the export dialog box to no success. If I export from Silo into Max it looks fine. I just re-export the exact same thing into Xnormal, and Viola! It works. So...what gives?


    [/ QUOTE ]
    Probably need to set Silo OBJ export options to indicate the faces must be exported in CCW order as Maya does.
    Could be the same thing than exporting OBJs from Zbrush ( the flipX and flipY options ).
    Unfortunally I never used Silo so cannot tell you which ones, sorry.

    Have you tryed to export the model as other format like LWO?

    [ QUOTE ]

    i still get the amboc to freeze and hitting the abort button wont abort it and ill have to ctrl alt delto close it. i tried rendering with a 512x512 amboc 16 samples and still nothing


    [/ QUOTE ]
    Using the 3.12.0 beta 1 with the Default Scanline Renderer or the new GPU AO tool?
  • dur23
    Options
    Offline / Send Message
    dur23 polycounter lvl 19
    Fixed, thanks jogshy. What would cause the hang up fromt default settings for the ambient occulsion renderer?

    Oh boy, i must be dumb. My ambient occulsion is rendering a world space normal map instead of the ao map. why?
  • jogshy
    Options
    Offline / Send Message
    jogshy polycounter lvl 17
    [ QUOTE ]
    Fixed, thanks jogshy. What would cause the hang up fromt default settings for the ambient occulsion renderer?

    [/ QUOTE ]
    Nothing really, was a bug in the 3.11.0 and was fixed in the 3.11.1. The only thing you can do bad is to select the hxGrid distributed renderer without launching the coordinator+agents as happened to you. In theory if that happens some kind of socket/network timeout message should appear... not sure why none was displayed.

    Some people think is a hang but perhaps is just too slow rendering hehe... smirk.gif That's where the new GPU AO tool of the 3.12.0 Beta 1 can help a bit... and yep, perhaps I should add an "estimated time" to the progress bar, but i'm not sure the calculation could be done accurately ( which could as bad as the Windows Vista's file copy dialog heheheheh laugh.gif )

    [ QUOTE ]

    My ambient occulsion is rendering a world space normal map instead of the ao map. why?


    [/ QUOTE ]
    xNormal puts the bent normals in the RGB channel of the output map and the occlusion in the alpha channel. If you choose an output file format without alpha channel support(like BMP,JPG,etc) the occlusion won't be saved, just the bent normals.
  • dur23
    Options
    Offline / Send Message
    dur23 polycounter lvl 19
    I getcha. Shoulda known. Sorry about that. Thanks for this app. ps. I'm doing tests for my company, and if all goes well ill make sure the donations fly your way.
  • Joao Sapiro
    Options
    Offline / Send Message
    Joao Sapiro sublime tool
    yo jogshy, i tried the new beta when i wrote that, it still hangs up like the previous version , using scanline render. Also wouldnt it be better to make the ambient oclusion in the rgb channel ? since it is what we want ahahah , and maybe save the bent normals on another file ? it doesnt make sense that the image that we want more get lost in formats like bmp etc frown.gif

    i dont know what is happening for xnormal freeze rendering the amboc and the ram meter be almost full while doing it , in 3ds max it renders really fine, but i cant use normals from xnormal and amboc from max since stuff doesnt line up :S
  • jogshy
    Options
    Offline / Send Message
    jogshy polycounter lvl 17
    [ QUOTE ]
    yo jogshy, i tried the new beta when i wrote that, it still hangs up like the previous version , using scanline render.


    [/ QUOTE ]
    Are you sure it hangs really and is not just slow rendering? Do this... render AO for the smiley example. If takes more than 2 minutes in an AMD64 3500 then something is bad. On the other hand if renders ok then is just a slow speed problem.

    Can you post some screenshots and polycounts so I can see your settings pls?

    [ QUOTE ]

    Also wouldnt it be better to make the ambient oclusion in the rgb channel ? since it is what we want ahahah , and maybe save the bent normals on another file ? it doesnt make sense that the image that we want more get lost in formats like bmp etc frown.gif


    [/ QUOTE ]
    Ok, gonna save it into other file then.
  • Joao Sapiro
    Options
    Offline / Send Message
    Joao Sapiro sublime tool
    well it was going for like 5 hours and it was stuck on 60% for 4 hours , rendering a 512x512 with 50 samples ( 16 samples gets really crappy and unusual ) and it renders in like 16 minutes in 3ds max with full antialisaing ( but again, this ambient oclusion is worthless... )

    so there is something wrong i think, because i dont think the mesh is that complex :s ( the version you have is very outdated btw , i have a new one )

    thanks for the replys thou man laugh.gif
  • jogshy
    Options
    Offline / Send Message
    jogshy polycounter lvl 17
    [ QUOTE ]
    well it was going for like 5 hours and it was stuck on 60% for 4 hours


    [/ QUOTE ]
    The smiley example?
  • Joao Sapiro
    Options
    Offline / Send Message
    Joao Sapiro sublime tool
    nop , the smiley generated fine , but im finding just awkward something to take too much to render ahaha.
  • jogshy
    Options
    Offline / Send Message
    jogshy polycounter lvl 17
    [ QUOTE ]
    nop , the smiley generated fine , but i'm finding just awkward something to take too much to render ahaha.

    [/ QUOTE ]
    Ah ok, now I understand tongue.gif So is not a bug really, is just the rendering speed.

    I agree with you.. the current AO implementation is not very fast... I'm trying to optimize it a few for the final version, but cannot promise a good result blush.gif In fact I can tell you this is my personal nightmare! The damm AO speed confused.gif

    Hopefully you can use the new GPU AO tool in case you don't need very complicated features like cages/attenuation/normal overrides/etc.
  • Joao Sapiro
    Options
    Offline / Send Message
    Joao Sapiro sublime tool
    i will have my computer assembled ( the one i have 2 gigs of memory ) and ill see if it still hangs ahaha...


    ( i left this on during sleep , and it was 10 hours or so still hanged on the same place )

    but really check, render to texture from max and try to figure out how althou its weird, it can be pretty good memory wise with that smile.gif
  • jogshy
    Options
    Offline / Send Message
    jogshy polycounter lvl 17
    Omg, I found the dammmm AO bugggggggggggggggggggggggggg!
    So Johny was right laugh.gif

    If the lowpoly model has a vertex with a normal near (0,-1,0) will hang a bit in that face because a number goes to the infinite.. and cannot determine well the hit point and retries and retries until reachs a limit ( very slow ).
    Is corrected now and gonna include the patch for the final version.

    Thanks, Johny... found it with the file you sent me smile.gif
  • katzeimsack
    Options
    Offline / Send Message
    katzeimsack polycounter lvl 17
    hmmm ao doesn't work anymore frown.gif
    Rendering ao takes as long as it always took, but the whole map only is a mask, there is no ao!
    I tried every method (uniform, so on) but it doesn't work.. When I use the previous xn version everything is fine (same setting and model) frown.gif



    btw:
    A "shut PC down after map generation" option would be great smile.gif
  • Joao Sapiro
    Options
    Offline / Send Message
    Joao Sapiro sublime tool
    also santi please include images to what the diferences are for each type of ao baking please :S maybe do that for every effect, you render a normal map out of a random model part that allows to show something, then you in the normal map generation for example put an image of it in tangent and object space, in the ao dialog you put a certain number of samples and spread and render an ao bake with each type uniform, cosine etc, that way we could see what would be best etc smile.gif ( and save alot of time harr!)
  • jogshy
    Options
    Offline / Send Message
    jogshy polycounter lvl 17
    [ QUOTE ]

    A "shut PC down after map generation" option would be great


    [/ QUOTE ]
    Ok

    [ QUOTE ]

    include images to what the diferences are for each type of ao baking please


    [/ QUOTE ]
    Yep, some kind of realtime preview won't hurt. I should render real models instead of putting the static images in the map options's dialogs. Probably gonna add a button that could show a small preview with the current settings for the next version.

    Well, gonna try to finish all this and post a Beta 2, because i'm touching too much code.
  • EarthQuake
    Options
    Offline / Send Message
    Hey a thought just occured to me. Would it be posible to use a program like FAOGEN to bake ambocc into the very colors of a highres mesh, and then load that mesh up in xnormal and bake it to the low? I know it would be easy to do with a texutred baked from faogen, but that would take uving your highres which sucks. Also if this was possible would it actually save any memory as apposed to rendering ambocc with xnormal?
  • jogshy
    Options
    Offline / Send Message
    jogshy polycounter lvl 17
    [ QUOTE ]
    Would it be posible to use a program like FAOGEN to bake ambocc into the very colors of a highres mesh, and then load that mesh up in xnormal and bake it to the low?

    [/ QUOTE ]
    No need to use faogen really. You could use the new GPU AO tool in xNormal. Then use the "bake highpoly base texture in lowpoly" feature to project it into the lowpoly. Using this method could be very fast but the highpoly mesh need to be UV-mapped.

    Btw, with the ao-bug solved i'm rendering the AO using the "typical" projective way at very decent speed ( took 8h with the bug present, now takes 1 minute ) for the final 3.12.0.
  • Joshua Stubbles
    Options
    Offline / Send Message
    Joshua Stubbles polycounter lvl 19
    [ QUOTE ]
    (took 8h with the bug present, now takes 1 minute) for the final 3.12.0.

    [/ QUOTE ]

    Hot damn! Can I have your baby? smile.gif
    Thanks for the update, Jogshy. Looking forward to the new version.
  • Joao Sapiro
    Options
    Offline / Send Message
    Joao Sapiro sublime tool
    "ep, some kind of realtime preview won't hurt. I should render real models instead of putting the static images in the map options's dialogs. Probably gonna add a button that could show a small preview with the current settings for the next version.
    "

    wouldnt that waste a tad memory ? wouldbe cool , but i guess a good tangent where we can sort of see the shape and the resulting ao bakes would be enough smile.gif
  • jogshy
    Options
    Offline / Send Message
    jogshy polycounter lvl 17
    [ QUOTE ]
    wouldnt that waste a tad memory ?

    [/ QUOTE ]
    Nah, I'm gonna put the box-sphere scene ( which image appears in the AO dialog in blue tone ) which is very lightweight(a few Kbs) and very fast to compute... but for the next version, I wanna finish the 3.12.0 asap.
  • ironbearxl
    Options
    Offline / Send Message
    ironbearxl polycounter lvl 18
    Thanks Jogshy!

    Tried the Simple AO Gen, but I get this error:

    xn_error.jpg
  • jogshy
    Options
    Offline / Send Message
    jogshy polycounter lvl 17
    [ QUOTE ]

    Tried the Simple AO Gen, but I get this error:

    xn_error.jpg

    [/ QUOTE ]
    What graphics card do you use? Need a card with OpenGL 1.2 + shading language support + FBOs ( Radeon 9XXX/GeForce FX or above ) or to update your drivers if they are too old.
  • jogshy
    Options
    Offline / Send Message
    jogshy polycounter lvl 17
    Tic tac, tic tac...
    5 mins to something new...
  • Keg
    Options
    Offline / Send Message
    Keg polycounter lvl 18
    I think you mean tic tock :P
  • EarthQuake
  • jogshy
    Options
    Offline / Send Message
    jogshy polycounter lvl 17
    lol Earth...you are right... is tic tock in english then ? tongue.gif

    Well...whatever...

    xNormal 3.12.0 ( release candidate 1 ) has been released.

    Four major changes from the beta 1... the conemap tool is now implemented, solved AO problems, fixed the dilation filter and now the program is using multicore features better ( a 90% of the total code ).

    You have 5 days to discover bugs before I post the final version!

    ps: Requires latest November 2007 DirectX... sorry blush.gif Is included in the .exe installer though. Just need to download it in case you use the 7zipped package.

    thanks!
  • ironbearxl
    Options
    Offline / Send Message
    ironbearxl polycounter lvl 18
    [ QUOTE ]
    [ QUOTE ]

    Tried the Simple AO Gen, but I get this error:

    xn_error.jpg

    [/ QUOTE ]
    What graphics card do you use? Need a card with OpenGL 1.2 + shading language support + FBOs ( Radeon 9XXX/GeForce FX or above ) or to update your drivers if they are too old.

    [/ QUOTE ]

    Hey Jogshy, I'm using geforce 6800 and the newest drivers (163.75) the cpu mode gives the same error too.

    Edit: I will test it on the 3.12 beta!
  • EarthQuake
    Options
    Offline / Send Message
    Awesome stuff man, i'll give the new AO stuff a go tomorrow at work, what is the cone tool btw?
  • jogshy
    Options
    Offline / Send Message
    jogshy polycounter lvl 17
    [ QUOTE ]

    Hey Jogshy, I'm using geforce 6800 and the newest drivers (163.75) the cpu mode gives the same error too.


    [/ QUOTE ]
    Is not the Forceware 163.75 a Beta? Official one is 163.71 for WinXP and 163.69 for Vista.

    Works with the FP16 mode? Can you open the 3D viewer using the OpenGL graphics driver? Are you using a notebook or a desktop PC? The GF6800 is ok ( although with that card you will be limited to CPU or FP16, FP32 won't be posible ), problem must be in the drivers. Btw, the CPU requires OpenGL... but will use the CPU to perform some tasks intead of the GPU ( so should be more compatible but slower ).

    [ QUOTE ]
    what is the cone tool btw?

    [/ QUOTE ]
    Basically a next-next-gen improved parallax mapping effect with self-shadows.

    See this video
    http://www.youtube.com/watch?v=g93wOFan_gE
  • EarthQuake
    Options
    Offline / Send Message
    oh ok, thought it was some sort of content generation tool, but its just for viewing right?
  • jogshy
    Options
    Offline / Send Message
    jogshy polycounter lvl 17
    [ QUOTE ]
    oh ok, thought it was some sort of content generation tool, but its just for viewing right?

    [/ QUOTE ]
    Yep, for 3d engine bump mapping improvement... just need to pass a height map.. then the tool pre-computes the "cone map" so could be used by the engine when it load the texture.
    I plan to incorporate that technique for the upcoming DX10 graphics driver for xNormal 3.12.1 and xNormal 4.0.
    Is a very new thing.. I doubt any 3d engine is using it currently.
  • Slum
    Options
    Offline / Send Message
    Slum polycounter lvl 18
    interesting stuff.

    here's a pdf on cone step mapping. http://www.lonesock.net/files/ConeStepMapping.pdf
  • jogshy
    Options
    Offline / Send Message
    jogshy polycounter lvl 17
    [ QUOTE ]
    interesting stuff.

    here's a pdf on cone step mapping. http://www.lonesock.net/files/ConeStepMapping.pdf

    [/ QUOTE ]
    Yep and also http://developer.download.nvidia.com/books/gpu_gems_3/samples/gems3_ch18.pdf

    Btw, Fabio and Dummer collaborated in this tool laugh.gif I wanna thank them for all their help.

    Btw, the "cone", "quad" and, specially, the "relaxed" one method can be a bit slow... just need some patience.. computation is VEEEERY complicated ( and uses the GPU... don't panic if you see the screen to flick or to refresh slowly... the computation can require TeraflopS of power ). The "Lone" method is the fastest ( and uses the CPU ). I don't recommend to pass 256x256 bitmaps or will be really really slow.
  • Joao Sapiro
    Options
    Offline / Send Message
    Joao Sapiro sublime tool
    thanks jogshy ! will give it a go right nnow !
  • East
    Options
    Offline / Send Message
    East polycounter lvl 14
    I think it's a region specific thing. Here in Sweden it's "tick tack, tick tack" wink.gif
  • EarthQuake
    Options
    Offline / Send Message
    Trying to render a 2048 with 4x AA, gets passed the "rendering map" bit and is fine, memory drops from 1.4 gigs free to 600 megs free, then gets to normalizing height or somesuch and tells me i'm out of memory.... I still have 600 megs of ram free and its telling me i'm out of memory, after the map has been generated! =(((

    Also dilation seems even slower than it has been, its taking way longer to calculate dilation that to actually render the map now

    I dont know if theres a setting i'm missing somewhere but the ambocc seems to be as slow as ever, is there something special i need to do to use the GPU ambocc renderer?
  • EarthQuake
    Options
    Offline / Send Message
    Alright i found the "simple ambocc generator" but it only has inputs for one mesh, so all you can do is generate ambocc for your highres onto its own uvs? Not really much use for this, i thought you added in improved AO baking?
  • jogshy
    Options
    Offline / Send Message
    jogshy polycounter lvl 17
    [ QUOTE ]
    Trying to render a 2048 with 4x AA, gets passed the "rendering map" bit and is fine, memory drops from 1.4 gigs free to 600 megs free, then gets to normalizing height or somesuch and tells me i'm out of memory.... I still have 600 megs of ram free and its telling me i'm out of memory, after the map has been generated! =(((


    [/ QUOTE ]
    Probably runs outta memory for the preview window... well, almost the bitmap was generated and saved to disk?
    Well, if runs outta memory in the "normalizing heights" then something is really bad... gonna revise the code then ( I suspect there is a memory leak in some filter that is not deallocating RAM well ).
    Can you tell me if that happens to you with the old 3.11.1 pls?

    Btw, I hope you arent using Windows Vista, because is a memory eater ( specially the x64 ). I really really hate Vista.

    Two advices:

    1. Try this: instead of specifing a 2048x2048x4AA render a 8192x8192x1AA and manually scale down to 2kx2k using photoshop or other program.

    2. Caution with the output format you choose. For example, some formats cannot be saved in tiles or line by line so need to allocate a heavy RAM buffer before saving. The TGA and BMP exporter consume very low RAM. For example, OpenEXR or JPG consumes much more ram when saving.

    Well, I have a crazy idea in mind to save lots of RAM with the maps... gonna see if I can implement it for the final version.

    [ QUOTE ]

    Also dilation seems even slower than it has been, its taking way longer to calculate dilation that to actually render the map now


    [/ QUOTE ]
    Well, dilation is slow because need process a great number of pixels a lot of times. Fixed it to use multicore better... so should be faster, not slower smirk.gif. Also i'm planning to use the GPU in the future to make it faster, but i'm getting some problems because some graphics cards don't support textures greater than 2048.

    [ QUOTE ]

    i found the "simple ambocc generator" but it only has inputs for one mesh, so all you can do is generate ambocc for your highres onto its own uvs?

    Not really much use for this...


    [/ QUOTE ]
    You can use it to bake AO on any UV-mapped mesh ( lowpoly or highpoly ). To project the highpoly AO into the lowpoly one just use the "bake highpoly base texture into lowpoly" ( like the flag example does) feature after. That should be faster than the traditional way.

  • EarthQuake
    Options
    Offline / Send Message
    I was under the assumption that the GPU accelerated AO worked with the High->Low workflow =( =( =( =( =( Is there any way to make this happen?

    Now again if you could save the AO info into vertex colors or something like that then it would be cool, but having to uv highres models is just a huge waste of time to render quick AO imo... On complicated models it would be much more time to uv them instead of just waiting for a long ao bake, and while you're waiting for the AO bake you can actually do other stuff, whereas if you're uving, thats keeping you from doing something else you know.
  • jogshy
    Options
    Offline / Send Message
    jogshy polycounter lvl 17
    [ QUOTE ]
    I was under the assumption that the GPU accelerated AO worked with the High->Low workflow =( =( =( =( =( Is there any way to make this happen?


    [/ QUOTE ]
    I could adapt the simple AO GPU tool mechanism to the normal renderer. Just need a bit more time.

    [ QUOTE ]

    Now again if you could save the AO info into vertex colors or something like that then it would be cool


    [/ QUOTE ]
    I could output vertex AO info into a separate file if you want.

    [ QUOTE ]

    , but having to uv highres models is just a huge waste of time to render quick AO imo...


    [/ QUOTE ]
    And Zbrush AUV/GUV tiles?

    Well, gonna allow to use the simple AO gpu tool for models without UVs and output AO vertex values to a file(probably will convert the original mesh into a SBM file with per-vertex AO components added).

    Also gonna try to adapt the method i'm using in the simple AO GPU tool for the default renderer.

    And all this before this Monday... hmmm i'm gonna need steroids.
  • Joao Sapiro
    Options
    Offline / Send Message
    Joao Sapiro sublime tool
    you can do it man, this tool has came a loooong way, and its amazing your commitment to perfection !
  • EarthQuake
    Options
    Offline / Send Message
    hehehe awesome!
  • jogshy
    Options
    Offline / Send Message
    jogshy polycounter lvl 17
    Lol, discovered the cause of the out of memory after the height normalization... For some strange reason was allocating excesive RAM for the antialiasing mask... hahahahaha crazy.gif
  • jogshy
    Options
    Offline / Send Message
    jogshy polycounter lvl 17
    Gonna give me 2 weeks more to release the 3.12.0 final because I wanna reduce drastically the memory required to perform the render( no more damm outta memory mesages and you could render a 32k x 32k image with 32Mb of RAM or less). Also gonna try to introduce a realtime preview window and advanced antialiasing/sampling options.
    Basically i'm cloning Mental Ray's adaptive sampling and tiled(bucket) rendering.

    However, the GPU AO thing must wait for OpenGL 3.0, sorry... I'm not getting good results/experience with the current OpenGL ( due to software render fallback on unsupported blending and cannot swap to D3D due to 1M NVIDIA polygon limit ).
1141517192059
Sign In or Register to comment.