Home Technical Talk

Weird baking error, max RTT

polycounter lvl 12
Offline / Send Message
alexk polycounter lvl 12
I'm not sure why my screws in my picture is baking as if it's pointing inward. On my highpoly, the screw is beveled, but it's not bent like the way it's getting baked. I'm using the Max RTT to bake. I've noticed this on other things I've baked lately, in particular small bits. They are coming out with a bent direction that is not representive of the highpoly. How do I fix this?

boltsproblem.jpg

Replies

  • commander_keen
    Offline / Send Message
    commander_keen polycounter lvl 18
    Its because its projecting at an angle because of the smoothing groups in the low poly. You would either have to harden those smoothing groups and be really smart about how you adjust the cage, or add more detail to the low poly so that the surface directly above these details doesnt share an edge with another face with a very different normal.
  • 00Zero
  • alexk
    Offline / Send Message
    alexk polycounter lvl 12
    thanks for the tips. unfortunately, I still end up with the same results so I may not fully understand the suggestions given. However, I found a solution. All I did was UNcheck the "use cage" option in the RTT, and now my normals are baking properly.

    which brings me back to the tips, I'm not sure why I can't get them to work. I decided to harden all faces (seperate smoothing groups on all) to see if it made any difference and it didn't. In the projection cage, I don't do much to adjust it, not sure where I went wrong on that. All I do is reset it, and then adjust the "push" in the projection modifier until it covers any all the highpoly, but still no luck.
  • Mark Dygert
    Because your cage is bigger, the rays it shoots will be angled. If you draw a line from one vert on your cage corner to corner of the source that will give you some kind of idea what the rays look like. In the center they are straight but the closer to the edges they'll start to angle.

    If you use a flat plane the same size as your source, and just move it away from the surface it will give you most accurate projection.
  • alexk
    Offline / Send Message
    alexk polycounter lvl 12
    After a bit of experimenting, i'm understanding it a bit better. My question now is, how do you deal with capturing details that are sticking out on adjacent faces? Since you need to pull out one side of the cage, and then when you pull out the other side, it'll make the rays cast improperly? Heres my example:

    the 1st picture, the white lined is my low poly, it's a box. And you see my highpoly, there are some parts that stick out, so I need to pull out the cage to capture the vents on the right side, and also the top. But that causes my vents to bake in the same way my screws in my original post. I don't know if I'm beating a dead horse here, since I can just turn off "use cage". Is there something wrong in my workflow? I never see any of these errors in the P&P section

    cagesetupch5.jpg
  • Joshua Stubbles
    Offline / Send Message
    Joshua Stubbles polycounter lvl 19
    You won't get a good result by simply using a straight up BOX. You need to toss some chamfer on those edges, add more polys there. Your normals are too restricted otherwise. Either that or paint a normal map in photoshop...
  • EarthQuake
    Wait, since when has using a box been on the "list of things you're not allowed to normal map"?
    Because your cage is bigger, the rays it shoots will be angled.

    No, its not because the cage is bigger. Its because the normals on the low poly mesh are being averaged, making the direction the rays are being projected at skewed. The reason why you do not see a difference with using hard edges or not is because max averages the normals on your cage regardless. On the other hand not using cage, will use the exact normals of your low poly, which is good for projecting details without distortion, but bad for any hard edges(as you will miss detail at the edges).

    What i do is do a render using cage, and then a 2nd render with cage off. Then you can easily combine your results in photoshop, by loading one up, and using a layer mask to selectively choose the areas you want(usually taking the details from the no cage, and the edges from the cage). I do the same with AO, do 2 bakes, and just copy the layer mask from the normals.
  • 00Zero
    hmm. good suggestion EQ. i should do that next time.
  • commander_keen
    Offline / Send Message
    commander_keen polycounter lvl 18
    Yeah good suggestion EQ. I forgot that projection in max didnt care about smoothing groups.
  • Joshua Stubbles
    Offline / Send Message
    Joshua Stubbles polycounter lvl 19
    EarthQuake wrote: »
    Wait, since when has using a box been on the "list of things you're not allowed to normal map"?

    I didn't say you couldn't, jerky :) Just that you can't get great results from perfect box geometry (unless you do world/object space). You have to fudge stuff, add verts, or as you suggested, do multiple bakes.
    As a quick A > B workflow, adding more bakes and composites may not be the best idea. A simple chamfer and it would bake correctly the first time.

    But as with everything, there's always more than 1 way to do something.
  • Mark Dygert
    EQ apply Edit Normals to see which way the normals are facing, on a simple box, they are 90 degrees.
    3dsmaxCageNormals.jpg
    If you push a box cage, they are still 90. So how does it get off? I think we both understand the rays are being cast skewed, but not really why. Or maybe you do and I'm just not getting what you're saying?


    3dsmaxCage.jpg
    The problem he's having is that the cage, is in fact scaled bigger, so when the ray is cast from one point on the cage to the same point on the low poly the rays are angled.

    If he used a flat plane the same size as his high poly art and just hovered it (not scaling) above the piece it would render pretty accurately, but as you pointed out some details could be lost, which should have been addressed at the modeling stage, trench edges shouldn't be 90 degrees like a U, but angled like a V.


    Practical solution: Break off each face to its own element.
    So when its pushed, it explodes into planes that will project straight lines instead of scaling and angling. The quick way is just to use the detach tool in edit poly and set it to element.
    3dsmaxCageExplode.jpg


    No angling because the faces are mismatched sizes. They hover, and project down onto the high correctly. The important thing to remember is to model trenches with V shapes not U, or you'll miss details.
    3dsmaxCageExplodeFront.jpg
  • EarthQuake
    Again, even if you have hard edges on your lowpoly(as your first example clearly shows, a box with 1 smoothing group will obviously NOT have right angled normals), in MAX your cage will get averaged. This is the only way to keep a continuous mesh.

    This is what your cage looks like, even on faceted model. And the normals being averaged are what cause the distortion, not the size of the cage.

    cage.jpg
    What i was saying is that the cage being larger has nothing to do with it, it could be smaller, and you would still have the *same problem*.

    Honestly your "practical solution" is anything but, that would be a huge pain in the ass to do on anything but a simple box. Its better to have a workflow for all cases, then one extremely specific one-time use thing.

    Not to mention the fact that it doesn't actually work. =) All you're doing here is manually creating the same situation that turning off cage gives you.

    See, you're still missing detail on the edges with this method.

    edgerror.jpg
  • EarthQuake
    Vassago wrote: »
    I didn't say you couldn't, jerky :) Just that you can't get great results from perfect box geometry (unless you do world/object space). You have to fudge stuff, add verts, or as you suggested, do multiple bakes.
    As a quick A > B workflow, adding more bakes and composites may not be the best idea. A simple chamfer and it would bake correctly the first time.

    But as with everything, there's always more than 1 way to do something.

    Here are a couple examples you could do in that line of thinking, with the top, you would still get some distortion, and probabbly slightly worse results as far as smoothing errors etc.

    box_topo.jpg

    With the bottom, you would get a pretty accurate bake, and very good smoothing, but this is a wasteful method.

    When you start dealing with more complex meshes, adding this sort of stuff everywhere really becomes counter productive. Not to mention that rendering long, thin polygons like these is a bit of a bottleneck.

    So, while there may be more than 1 way to do something, there is generally a *best* way. If someone can actually provide a solid arguement to counter that i would love to hear it, because what i'm doing now with the 2 bakes is a bit of a pain, but the fact remains that it gives the best results in virtually every situation.
  • Mark Dygert
    Ahhh I get it now. Thanks =)
    I created a box and didn't set the smoothing groups to 1, so it wasn't able to average the normals. Which threw off my whole train of thought. I completely forgot that smoothing groups are nothing more then averaging. As soon as I saw your example I flashed back to the threads "how do I sub-d a box so it doesn't turn into a sphere" and "how do I fix my normal mapped box". I remembered the Edit Normals trick but I didn't remember what averaging was... hahaha sigh...

    What about using a different mesh for your cage? Xnormal and Max allow for a different mesh to be substituted don't they?
  • EarthQuake
    You can use a different mesh with xnormal, but it has to be the same # of verts, so it doesn't really help much. I think you can do that in max, but i'm not exactly sure how.

    I still think that just doing 2 bakes would be easier, really all you have to do is go to bake options, turn off use cage, change the file name and hit bake(maybe tweak the offset distance as well), its very simple and easy to do. It will take twice as long to render your maps, but rendering maps is pretty fast these days unless you have an ancient computer.

    One cool thing is that if you use object space maps, your "bake" mesh can have totally different geometry than your final lowpoly mesh, so you can go in and add a bunch of extra edges to get a clean bake when you're using OS.
  • Mark Dygert
    Yea its the same for max. No way to specify another cage unless it has the same vert count/numbering. So its useless for this case.

    BUT you could add the loops for baking only. Probably to an instanced copy of the low poly used just for baking. Or add Edit Poly on top of the stack and delete it after baking. But at that point, depending on the complexity of the object it could be speedier to do the double bake, instead of micro managing loops and averaging.
  • EarthQuake
    Is there a way to freeze normals or something? Because once you start adding geo to your bake mesh, you're gonna screw up the smoothing on the final, bake has to have the same normals as your final mesh for normals.

    But yeah in the end i think there are a lot of possible different things you can do to fudge the results, but imo the easiest is just to do 2 bakes.
Sign In or Register to comment.