Render to texture seams?

K47 · Mar 2013

I know. This question was shot many times around the internet but right now i`m having this problem and can`t seem to find/figure out a solution.

Even if i`m not an experienced modeler, I don`t think i had this issue before and since i reinstalled Max, I realized i get seams across the UV borders after bake.
I made sure that:
- I have a reasonable amount of edge padding (8 for 1024x1024)
- Used an averaged projection mesh (continuous cage) instead of explicit mesh normals
- Split the UV shells on hard edges (i actually used TexTools to create smoothing groups from UV shells)

And i still get seams. What am i doing wrong?

I`m attaching the max file in case someone is kind enough to take a look at my baking setup:
http://www.mediafire.com/?2aqad9ro6461605
Thanks!

Eric Chadwick · Mar 2013

Try this shader, along with the included modifier, see if it fixes the seam for you
http://www.3pointstudios.com/3pointshader_lite.shtml

K47 · Mar 2013

Funny thing is that`s the shader i`m using. I would try Xoliul but shader`s not the problem.

I`m also having seems in Fallout New Vegas engine.

Eric Chadwick · Mar 2013

Did you Reset Xform on the lowpoly before baking?
Did you change anything on the lowpoly mesh after baking?
Is this baked in 3ds Max with Render To Texture?
Did you edit the normal map in Photoshop or another image editor?

K47 · Mar 2013

1. Just tried to Reset Xform on lowpoly, still the same
2. No
3. Yes - using settings from Millenia`s tutorial
4. No

Ace-Angel · Mar 2013

Could you show what the low poly without the normal map looks like? Just the LP itself.

Did you use a cage in Max and did mess around with it outside of "Push" values?

Was the Reset Xform applied before or after the bake, and did you collapse it?

Does it show the same issue if you bake in another software like XN? What about saving in different format?

Are the shells Normalized?

Does flipping the channels change anything for the better?

Did you open the normal in Photoshop and save it? Photoshop might have used automatic Gamma Correction on the normal when you saved it, even if you edited nothing.

Did you double check the baking options? Max sometimes turns off your last options you selected for baking.

K47 · Mar 2013

Did you use a cage in Max and did mess around with it outside of "Push" values?
I used cage without messing with anything but the Push value.

Was the Reset Xform applied before or after the bake, and did you collapse it?
I applied Reset Xform before bake, i collapsed only the Xform modifier. The projection modifier was still in the stack.

Does it show the same issue if you bake in another software like XN? What about saving in different format?
I haven`t tried this yet.

Are the shells Normalized?
How do i normalize the shells?

Does flipping the channels change anything for the better?

It only gets worse.

Did you open the normal in Photoshop and save it? Photoshop might have used automatic Gamma Correction on the normal when you saved it, even if you edited nothing.
I used only the fresh bake without changing it in Photoshop

Did you double check the baking options? Max sometimes turns off your last options you selected for baking.
Yes, i always double chack

Can somebody confirm that my baking setup is good? I posted my max file here, i`ll share the link again: http://www.mediafire.com/?2aqad9ro6461605

Eric Chadwick · Mar 2013

Did you try turning off gamma, and rebaking?

Customize menu > Preferences > Preference Settings dialog > Gamma and LUT tab

Ace-Angel · Mar 2013

I just baked it now in XN at 1024 res, and it seems like there is still a seam, but it's not too obvious once you use Spec and Diffuse maps on it.

However, once I increased the size of the map, the seam becomes more evident.

I don't have Textools installed on my version of Max 2011, but I think it's honestly a shell size issue, your main body of the bottle has less ratio space given to it alongside the other parts I'm guessing, which is giving you the issue you. (?)

I would suggest normalizing the UV shells, making them all occupy 1:1 space values, and baking again from there.

Also, make sure to Smooth-Group ALL the seams, the main body of the bottle around the side and the neck have a seam, but are also 1SG, maybe cutting that will also help, don't know if that is my Max issue or something, but that' how the scene came to me.

Cheers.

K47 · Mar 2013

Ace-Angel, thanks for checking my .max file! UV shells were already normalized, they are all equal - the main body of the bottle doesn`t seem to have less ratio space - here`s an image of the bottle with uv test grid

Eric Chadwick · Mar 2013

Can you export OBJs of the low and the high, and post those? I'm running 2010 here.

K47 · Mar 2013

Eric Chadwick, Gamma/LUT Correction is turned off. Sure, i`ll post them now

K47 · Mar 2013

Here are the OBJ files

http://www.mediafire.com/?786kj4ijvw68rm8

Ace-Angel · Mar 2013

Oh well this is strange, the ones in my Max, in the top row looked like they took 1/2 of the space of the UV sheet.

The OBJ files look more clean, strange, I'll test them again now and see what's up.

Eric Chadwick · Mar 2013

Hmm, I got it to work correctly, and this is in Max 2010, before the "quality normal map" fix was added.

Import HP
Import LP
Press 0 to load Render To Texture
Padding 16
Pick... HP
---
Projection modifier... Reset the cage, Push a little (.05).
---
RTT... Add a NormalsMap
TGA, 32bit, Pre-Multiplied Alpha
1024x1024
Render To Files Only
Render
---
Hide the HP
Standard Material, Bump channel, Normal Bump
Bitmap, load the file
Bump channel = 100 (not the default 30)
Apply the material to the LP
Activate the Show Standard Map In Viewport button
Hold down on it for the flyout, choose the 3rd option (Show Hardware Map in Viewport)
---
No seam! Not perfect, because it needs the proper tangent basis. Max 2011+ should do a better job.

K47 · Mar 2013

Eric, you are correct, the seam is less visible but it`s still there.
If you increase the Multiplier/Strength value from 1.0 to 1.5 in 3Point Shader, the seam gets more obvious. I also imported the lowpoly into xNormal, applied the bake and the seam is still not gone. Also tried the bake in Fallout New Vegas getting the same result.

What`s the whole mystery on all this?

Obscura · Mar 2013

Man. Dont increase the normalmaps strength. It can cause problems like this. It is only a good option at tiled textures.

K47 · Mar 2013

Of course. But i`m just showing that the seam is still there

and that value is only for rendering in Max, it doesn`t actually change anything to the normal map file. And in xNormal and Fallout New Vegas the seam is still there no matter what. So there has to be another solution for this

Eric Chadwick · Mar 2013

Each program renders normal maps differently, because they use a different tangent basis. The baking tool and the game engine need to be "synched" to completely get rid of seams.

For example, the 3ds Max baker is only synched with the 3ds Max scanline renderer, and particular viewport shaders.

To completely kill seams, you need to find out what baking tool the developers used to make their normal maps.

Without that, you can only reduce seams, or hide them.

K47 · Mar 2013

I didn`t have this problem before, i baked more complex stuff like this:

http://fc04.deviantart.net/fs71/f/2013/051/c/8/3_by_kaya47-d5vo3xc.png
http://fc06.deviantart.net/fs71/f/2013/051/6/1/2_by_kaya47-d5vo3tw.png
http://fc06.deviantart.net/fs71/f/2013/051/c/d/1_by_kaya47-d5vo3kx.png

These images are rendered under Fallout engine.

And i remember the Max bakes showed OK in xNormal. Why is this happening? I can`t work any further because of this

RC-1290 · Mar 2013

K47 wrote: »

I didn`t have this problem before, i baked more complex stuff like this:

http://fc04.deviantart.net/fs71/f/2013/051/c/8/3_by_kaya47-d5vo3xc.png
http://fc06.deviantart.net/fs71/f/2013/051/6/1/2_by_kaya47-d5vo3tw.png
http://fc06.deviantart.net/fs71/f/2013/051/c/d/1_by_kaya47-d5vo3kx.png

These images are rendered under Fallout engine.

Those images actually show very similar artifacts, they're just less obvious. For example: near the Gatling gun mount, and on the transition from body to tail.

Do you think this might help you?

Obscura · Mar 2013

hmmm i met with this solution when i had problems with symmetried meshes with normalmaps

K47 · Mar 2013

Rc-1290, those are not necessarily normal map seams. I think they are diffuse map seams, that`s why are less obvious. I`d be happy if i could hide my seams that well again.

Obscura · Mar 2013

Those ARE normalmap seams. The specular is breaks because the normalmap have seams.

K47 · Mar 2013

Ok so there`s no way to get rid of that seam? Is this how it`s supposed to look?

Obscura · Mar 2013

nope, it need to be solved.one of the possible solutions what RC-1290 linked, but its not the best solution since it needs extra edges. i downloading your model now, and trying out a few things.

edit - Hmmm i baked the normalmap in max, exported/imported the mesh to udk, added the normalmap to the model with specular, and looked good. So i started guessing the problem is with the Fallout engine(i mean different tangent basis for example, what Eric said).Try out plz what RC-1290 linked, and lets see how it looks in engine after that.

RC-1290 · Mar 2013

What kind of normal maps does the Gamebryo engine use? Is it set up the same as the normal maps you're creating?

K47 · Mar 2013

Obscura, thanks for taking your time to download and bake that thing! Can you please import the lowpoly and normal map in xNormal and see if the seam is gone?

One thing i know, Fallout is using +X -Y +Z swizzle coordinates for normal maps. I followed this tutorial made by Millenia: https://www.youtube.com/watch?v=y_1ql8yH2Ow . He`s doing weapons for Fallout New Vegas and they look flawless.

Obscura · Mar 2013

Sure. This is is how it looks in xnormal with swizzle 'x+,y+,z+', if i bake the normal in max with 'right,up'( how he said in the video)(for udk,its 'right,down') orientations.
Looks ok not?

I think the problem was that you baked the normal with wrong orientation. Up was down i guess.

K47 · Mar 2013

If that was the case then the solution would`ve been simple: Flipping the green channel in Photoshop. Unfortunately doing that makes it worse. Might be some baking settings that i missed. Can you tell me exactly what you did for baking? Your normal map looks very good

Obscura · Mar 2013

First of all, i noticed, it doesnt matter what i do, the smoothing groups are not changing.So i added edit normals modifier, selected all normals, and pressed break.Collapsed this, then i ran the uvshells to smoothing groups script (I dont know the name of it, i found it here somewhere a long time ago).Then nothing special, just render to texture, added the projection, changed the orientation, saved to tga without any compression, and baked. After all of that, it looked good in Xnormal.

K47 · Mar 2013

The UV shells to smoothing groups script is called TexTools. I`m also using that

Anyway, i finally managed to get a clear render under xNormal after i exported the lowpoly again. It "seams" that my bake was not the problem, it was the lowpoly that i first exported. I`m gonna import the new poly to Fallout and see what happens

Render to texture seams?

Replies