Tangent space map errors / UV question

danshewan

So, if I'm ever going to start making headway on Nemo's office, I promised a deluge of questions, beginning with this one - actually, these two. I'd like to thank everyone in advance for their patience.

I'm working on the floor panel. Initially, I'd used a planar map to test bake, which looked fine on the surface, but predictably gave some pretty bad texture stretching on the vertical faces. So, I cut seams into the mesh where I thought they should be, and used a Pelt and Relax operation to get the shells laid out.





And the temporary UV layout:





So I used xNormal to generate the normal and AO maps at 512 x 512 resolution, and I'm getting these funky errors along the outer edges of the four panels when rendering out the normal maps in tangent space.





I've been struggling with this all day, and thought it was about time for someone more knowledgeable to give me a hint as to what I'm doing wrong. I've kept everything as clean as I can, made sure the low poly is close to the high poly, made sure the UV shells are neat and straight....so what did I forget to do, or what did I do wrong?

I tried generating a world-space map, but the errors are just as evident which leads me to suspect it's a problem with the geometry - but nothing immediately apparent leaps out at me as being at fault. Any suggestions? Apologies for starting yet another normal mapping thread, but it's driving me crazy.

Second (quicker) question - how would you go about unwrapping this area of the mesh?





The central cylindrical shape is too short for a cylindrical map to be of much use, and a planar map makes the texture stretching way too obvious. I'm thinking of splitting the seams as shown and laying the faces out in strips - I did try this earlier, but I got even more of those funky errors when I tried test-baking.

Thanks for reading, and for any suggestions.

PredatorGSR

That is a 90 degree angle and a hard edge, so if you want good smoothing on most engines you'll have to split the edges into separated strips in the uv map

In your uvs, the uvs on the vertical part don't seem to be visible on the uv map, unless its so small i can't see it.

danshewan

PredatorGSR wrote: »

That is a 90 degree angle and a hard edge, so if you want good smoothing on most engines you'll have to split the edges into separated strips in the uv map

In your uvs, the uvs on the vertical part don't seem to be visible on the uv map, unless its so small i can't see it.

Thanks, Konrad. I'll split the trims of the panels off into their own UV strips and see how that works out. Also, I didn't adjust the UV layout for the central cylindrical shape yet, which is why you can't see it.

rollin

so show us the normal mapped object with the problematic area on it

I rarely had problems with normalmap padding from xnormal
if it is all padding.. it's hard to see where your uv shells end

danshewan

Okay, here's the mesh with the (2048²) normal and AO maps applied using the 3Point shader lite. No changes have been made as per the suggestions above as of yet.

Autocon

The problem is your unwrap, nothing more. You didnt unwrap anything here correctly.

All you did was planer map the entire thing so your UV's are showing up as just the circle or the flat planes. Each of those planes and circle have height to them. The ring around them that gives them the height needs to be unwrapped. There are tons of different ways to unwrap them.

I suggest you look at some tutorials on unwrapping. I dont know any off the top of my head or I would link them but I'm sure there are some in the wiki.




As for padding. Even after you do a proper unwrap on this you will have a lot of "goofy" looking edges around the seams of your normal map. This is because this is exactly what padding is. Padding is taking the last pixel along the UV seam and duplicating it out however far you set your padding number to. So if you set it to 4 its adding 4 of the same pixels that are at the edge of the UV seam outwards away from the UV shell. This is also why you don't want to have UV shells touch each other.

The reason it uses the same pixel at the very end of the UV shell is so that when your texture is down mipped down it samples those pixels that are the same color so you dont get artifacting since when you down rez a texture it starts to blend pixels together.

PredatorGSR

Looks fairly good from this distance. Can you post your normal map with the uv's overlaid on top?

danshewan

Thanks for the input, Anthony.

I know the unwrap needed attention, but I thought the merely flattening the panel sections would be enough to minimize the errors, as opposed to separating the trims into their own UV strips. :poly136:

I'll post some images once I've sorted the layout.

Thanks again for everyone's feedback.

[HP]

Yeah, what Autocon said is completely right.

I also see too many 90º angles on your LP, avoid those at all cost. (Chanfer them)

You also need to work on your UV's as previously stated, hope this helps:
http://www.youtube.com/user/renderhjs#p/u
http://www.waylon-art.com/uvw_tutorial/uvwtut_01.html

danshewan

Thanks for the links, H

danshewan

Okay, so having read a multitude of normal map resources, and spending two days trying various approaches, I still don't feel any closer to figuring this shit out.

After a series of test unwraps and bakes, I tested this UV layout. The panels have seams cut into the corners, then pelted and relaxed to keep the problematic trims attached to the larger panel geometry. I specified a seam along the lowest edge of the central cylinder shape, and again used pelt and relax as I read that it's best to try and minimize UV shells wherever possible.





So with the entire mesh sharing a single smoothing group, I get this result:





So although the errors that were most evident on the panel trims aren't as bad, obviously the single smoothing group is giving me these gradients across the flat surfaces.

I had tried separating all the vertical trims into one smoothing group, with the flat surfaces on another, and unwrapped the mesh into separate UV shells where the changes in smoothing groups occurred.





However, this didn't do much in terms of result - despite the additional work in unwrapping, the results were comparable to those I'd initially posted.

So, the single smoothing group approach didn't work so well for the flat surfaces, and giving the vertical trims their own group didn't help with eliminating even some of the errors I'd initially encountered.

I also took padding into consideration, using a padding of 8 pixels when test baking out 512 maps, and going as high as 32 pixels when baking out the 2048 maps, according to what I read about mip-mapping.

Still no dice, which leads me to believe that it's definitely something wrong with the software. All the software I'm using simply must be broken. Just kidding, it's obviously me being a prize dumbass. I've given up on feeling stupid for not getting this stuff, I just want to figure it out.

Where am I going wrong?

Thanks for putting up with my idiocy. :poly136:

cw

Hey dude,

Sorry to hear you're having problems baking. I don't know how much it will help but I just did a quick test in max.




1 smoothing group, 3point studios shader and normals modifier, no fiddling in photoshop needed.

If you can bake your model using 3point's setup, and it looks great, then your assets and workflow are sound as a pound. The remaining obstacle is getting things to look as good as you can in your engine of choice, a whole other test of patience.

You can see the 'wobble' on the curved section, this can be tweaked some way with the cage, based on likely viewing angle of the area. You could try fiddling with splitting the UV shalls and smoothing groups there and it might work out better.

I hope it was useful! Sorry for my crappy example!

PredatorGSR

Not sure what to tell you, there's pretty much only two answers.

1. You can split the uvs everywhere you have a hard edge, and put hard edges everywhere you have an angle that is greater than 70-90 degrees, and you will get a clean bake in pretty much any engine, the only downside is it will be slightly harder to texture, and will use more uv space.

2. You can use a single smoothing group with joined uvs or split, doesn't really matter, and in some specific instances you will get good results, such as with the 3point shader and with Maya High Quality Viewport. However, most realtime engines that are available at this time will NOT handle the map cleanly, and you will get lots of smoothing errors.

Method #1 will result in the best result across the widest array of apps, and you haven't posted a uv layout that uses that method, just said you tried it and it didn't work, so no one can really tell you what you did wrong, but that is the proper method for most engines.

You also said you split it into 2 groups, this is not correct. You have to split the uvs and put hard edges EVERYWHERE there is a 90 degree angle, which means all the corners as well. So that means there should be hard edges on all the corners, and the uv strips should be chopped up into a bunch of smaller strips wherever you have a hard edge.

PredatorGSR

I don't know if you've looked at the polycount wiki, but there is a ton of info on normal mapping on there.

http://wiki.polycount.com/NormalMap?highlight=%28normal%29|%28mapping%29

danshewan

Thanks for the replies, guys. I really appreciate everyone's help and patience, and thanks for taking the time to create a test model, cw.

I'll try making some adjustments again later today, and will post the various maps / UV layouts.

danshewan

Okay, so some progress.

I put the offending trims into their own smoothing groups based on location of hard edges as per Konrad's advice, and separated their UV shells.





So, although the result was better having separated the hard edges into their own smoothing groups and isolating them into their own UV shells, I still noticed quite a bit of artifacting. With the UV editor window still open, I noticed that this artifacting could be solved by moving the UV shells of the trims closer to the larger shells they belong to. However, when I rebaked the maps with the shells closer, the artifacts were still present.

How does moving a UV shell while the maps are applied solve the errors, but a rebake doesn't? Obviously I don't want to be moving UV shells while the maps are applied, right?

So, this dirty hack resulted in this image, with placeholder diffuse and spec:





As I said, this result was only possible by moving the UV shells of the trims while the maps were applied - this is not how the bake looked initially. Don't get me wrong, it wasn't terrible, but there were still some funky artifacts along certain edges.

Here's the normal map as it is right now:





Surely moving UV shells (even just a little bit, I'm not talking major repacking or anything) after the bake is a really bad idea, even if it did solve the artifacting?

PredatorGSR

I usually bake at double size and size it down, this usually gives the best result. xNormal doesn't use super sampling (I think that's the term, could be wrong), which most other baking programs do which gives better results. Sometimes you just need a bigger map too.

That might fix the issue.

metalliandy

Yea moving anything after a bake is a bad idea...
Did you try exploding the mesh and then baking?

cw

That normal map is not picking up much of a bevel from the high poly. If you're baking with no cage from low poly with split normals the rays don't cast in a way where they will capture that info. Can you clarify your setup?

If you want to post your file (or just a bit of it) I will have a look at it?

SpeCter

You should apply some padding(if not already done)

danshewan

That's very kind of you to offer, cw. Here's the (2009) file.

http://dl.dropbox.com/u/4270892/art_deco_floor_panel.max

I'm going to keep tweaking / reading in the meantime. Thanks again.

cw

ok so i just baked it out of the box, like. Using max 2010 32 bit on my laptop. i turned off supersampliing to save a bit of time, so results maybe a bit rougher than usual.

Biggest issue you have IMO is the size of the thing and the resulting low texture resolution. Maybe worth breaking it up more? The detail in the middle doesnt have big enough bevels to hit more than one pixel, so the form is speckly and hard to read. Mayb exaggerate slightly if you stick with this texture size. If you subdivide the model more the issue may go away, though you might wat to add more defined bevels for visual interest's sake. The 'N' had no bele on it also, which means it will likely as not disappear when you bake and the bake process doesn't give a fig about height, just surface direction for the normal map. Little bevels on everything which is coplanar to the base surface but you want to actually show up is a good idea. Also useful to have a rough idea how big a bevel needs to be for the normal map resolution to pick it up nicely, and be aware anything smaller is a waste of time/effort to build.

Anyhoo, you probably know this stuff so feel free to ignore my wafflings.

here's a picoid:

teaandcigarettes

Just wondering, why don't you try baking it onto a flat plane? Those extrusions are so small that you shouldn't notice any difference once the scene is put together. However, what really bothers me is your high poly.

Here's a quick normal bake, displayed in Xnormal.





I have modified the mesh a little bit so that all major extrusions are sloped. I did not modify the details, as I simply meant to show you the issue; notice how shapes that point straight from the surface (N letter, smaller details, etc.) are hardly visible. Even if you use geometry to represent larger forms, your details will not be clearly visible unless you make your extrusions sloped. Here's another comparison that shows the issue.

Editing the mesh might take you a while, but if you do so, you will be able to bake it onto a flat plane and not worry about smoothing groups and such. Just make sure that you make the edges softer than I did



Hope that helps Dan

danshewan

Thanks so much for taking the time to look at the file, guys. I really appreciate it.

I'll try both solutions and see what looks the best.

Thanks again.