3ds max render to texture

Ricky K.

I am getting triangles in normal map generated by render to texture.You may ignore the black artifacts as they are the ray miss check and I know how to fix it but I can't get rid of these triangles. I have tried various things such as targa, tiff, increased the push amount of the cage, and from the modifiers - smooth, mesh smooth, subdivide, tesselete. The current modifier on high poly mesh is Turbosmooth. If you need anymore info or need the 3dsmax file, please let me know.
The blue is normal map and the purple/pink image is from the rendered window.

Tits

Do you have smoothing group assigned to your low poly mesh?
Or worth doing a reset xform, add and collapse and edit poly on top see if it fixed the issue.
(I also suggest you using xnormal or knald for baking instead of max)

Mark Dygert

Using mental ray? Yea I wouldn't do that if I were you. If you must use max to bake, I suggest that you switch to scanline rendering and only use MR for AO. If you have access to another renderer like VRay, even better you can get everything you need without having to switch back and forth.

MR has all kinds of problems.
Aside from the ones you are running into, it also incorrectly colors the background black even if you set it to the proper color in RTT and it also butchers padding which gives you seams and makes your mipmaps look horrible.

Ricky K.

thanks to both of you, I am now looking for some xnormal and knald baking tutorials.As far as max is concerned, the default scanline renderer solved the black background problem in normal map but still gives triangles. Well applying smoothing group does in fact removes triangles but does not give the result one looks for.                           

And about vray, max gives 2 options for it, one is VRay RT and the other is Vray adv. 1) Vray rt disables render button in render to texture and 2)vray adv lasts for about 2 seconds when rendering but closes 3dsmax completely (like it crashes). Any idea of whats happening to vray in 3dsmax or what am I missing here ?   (I would like to share the file, plz give me the email id, etc)