Extremely disappointing results from Substance Painter

PolyFarmer

I have been trying to teach myself Substance Painter for about 6 months now.  But the issue is, I can't seem to wrap my head around why this piece of software hates me so much:

When I finished modeling my current project, I had a high poly and a low poly mesh. I thought that would be enough to get normals baked from Painter. Alas! All I got was a blank. No change in any appearance of my lowpoly mesh.

I googled a bit and found out that I need to have all the UVs laid out, mesh exploded, normals averaged, blah blah. I did all that. Took me some time due to my full-time job but I did it in the end. Tried it again and this time, and whoa! I got some kind of weird dents all over the lowpoly mesh.

Then around New Year's Eve, I was busy dealing with some major life stuff so couldn't focus on the project for a solid 3 months.

I have recently picked it up again and read up a lot on Catmul Clark. Watched a few videos, understood how a cage can really help improve things. So I went back to the drawing board, spent a few weeks making a cage for my mesh and today I finally finished it. Lo and behold, I'm getting multiple types of issues everywhere on the mesh. This is not at all like what I had watched in the tutorials.

The first problem is the manifestation of wireframes during baking in the normal map:

The second problem is some kind of "leaky" artifacts on a few of the components:

The third problem is bake overlap, as in other components' bakes being manifested on parts of mesh:

This is a 4k res texture.

Yes, I use an exploded cage for the exploded high poly and low poly meshes. Yes, the UVs are laid out (Maya Auto UV; there's just too many components in this project for me to do it manually). Yes, I know the procedure to bake (watched a lot of tutorials before). Yes, this is an original subscription, I'm not using a counterfeit copy.

I'm really close to giving up on this software. It just does not like me. I've tried literally every solution on the web but I can't get the software to work as intended. Like I said, this is the third time I went back to the drawing board and wasted weeks just to get it to accept my mesh and give me a normal bake. But for some reason, it has always caused extremely weird issues that I've never seen anyone else encountering. But I've still read up on and followed all tutorials I could find, to a T.

Any help would be appreciated. Thank you.

poopipe

Seems like you've bitten off more than you can chew to be honest. 

I can assure you that painter works.  I look after a lot of  users (over 50) of varying experience and they all manage to get passable results out of it.

 baking is not straightforward in any software - tearing into something complicated before having a solid understanding of how to generate a clean model, clean UVs and a clean bake is asking for a massive bag of frustration. 

I say that because I can see from your results that you do not yet have a solid understanding. Tutorials mean nothing once you're not making the item in the tutorial so they don't count. 

I suggest you take a simple object and work on it until you get a good result.
If youve already done that please explain the whole project, show bake settings, uvs, how the texture sets are arranged etc.. 

Without context we're flailing around in the dark and won't be able to help

PolyFarmer

poopipe said:

Seems like you've bitten off more than you can chew to be honest. 

I can assure you that painter works.  I look after a lot of  users (over 50) of varying experience and they all manage to get passable results out of it.

 baking is not straightforward in any software - tearing into something complicated before having a solid understanding of how to generate a clean model, clean UVs and a clean bake is asking for a massive bag of frustration. 

I say that because I can see from your results that you do not yet have a solid understanding. Tutorials mean nothing once you're not making the item in the tutorial so they don't count. 

I suggest you take a simple object and work on it until you get a good result.
If youve already done that please explain the whole project, show bake settings, uvs, how the texture sets are arranged etc.. 

Without context we're flailing around in the dark and won't be able to help

Thanks for replying, poopipe!

Hmm... would it help if I sent the file over to you? Could you take a look and tell me where I can improve? I think even if I did, it'd be too much work for me to go back and redo everything for the fourth time. I just want to get over with this project now, it's getting unnecessarily dragged out.

PS: I did a simple test after the second try in Painter and it came out well. Didn't check too closely for the accuracy of the details but back then I was barely getting anything from the baking process, so even just seeing the details appear was a victory for me.

Flmn

Good UVs are essential for a good bake. Even if you use auto mapping, you should always use checker map to be sure, that every piece is unfolded.

On first picture wireframe is a bit strange - cant see the whole picture, but looks like it should be greatly simplified. Another thing I see - your edges on HP are too crisp - it can be almost impossible to capture this information into the map, especially on small object like a tread part. Object from 3rd picture seems simple - begin with it. Show us the UVs, because they are mess, I'm sure. And try not to use 4K maps until you really need to, especially if you're not experienced - that's the easy way to teach yourself some bad habits.

poopipe

I'd like to say yes to taking a look through the file but honestly I just don't have any time.

The chances are you'll  Be able to recover without having to redo everything if you break it up into manageable chunks. 

  It looks like you're making a tank to me.  I'd split that up into several parts over multiple texture sets or even projects as it's far too complex to try and cover in one hit. 

PolyFarmer

Flmn said:

Good UVs are essential for a good bake. Even if you use auto mapping, you should always use checker map to be sure, that every piece is unfolded.

On first picture wireframe is a bit strange - cant see the whole picture, but looks like it should be greatly simplified. Another thing I see - your edges on HP are too crisp - it can be almost impossible to capture this information into the map, especially on small object like a tread part. Object from 3rd picture seems simple - begin with it. Show us the UVs, because they are mess, I'm sure. And try not to use 4K maps until you really need to, especially if you're not experienced - that's the easy way to teach yourself some bad habits.

Thanks for replying! I understand UVs are an important part, but like I said there was just too much stuff in there. I had hoped Auto UVs would be enough. They've always worked fine. But now- if I try and fix it... well, my cage, my highpoly and my lowpoly would all need identical layouts. At this point, they're different meshes altogether so you can understand how tedious that would be. That ship has sailed.

I was using 4k maps so I can get a full idea of what I can get from the baking process. Because like I said, before I was barely getting anything. Imagine the 4k res as my magnifying tool. Just a quick test.

poopipe said:

I'd like to say yes to taking a look through the file but honestly I just don't have any time.

The chances are you'll  Be able to recover without having to redo everything if you break it up into manageable chunks. 

  It looks like you're making a tank to me.  I'd split that up into several parts over multiple texture sets or even projects as it's far too complex to try and cover in one hit. 

It's ok if you're busy.

Hmm... you're right regarding breaking it up, but I have already broken the pieces down as much as I could during the explosion:

That is correct. I'm trying to make a tank. Multiple texture sets? Can you elaborate?

defragger

You need to explode the parts before the bake to avoid overlap.

Also the edges on the High-poly meshes are waaay to sharp.

I can assure you that Substance Painter is not the problem here!

Try to understand the basic principles of normalmapping first. You can use basic test meshes like chamfered cubes.

You should read this:

https://polycount.com/discussion/81154/understanding-averaged-normals-and-ray-projection-who-put-waviness-in-my-normal-map/p1

... and later this as well:

https://polycount.com/discussion/147227/skew-you-buddy-making-sense-of-skewed-normal-map-details

poopipe

A texture set is analogous to a material
Each material you apply to the low res mesh before export will give you a texture set in Painter. 
Each texture set can have its own UVs and its own bakes. 

If you're baking that all in one go you'll have all sorts of problems.

Split your low poly up into logical chunks that fit onto a sensible sized map at a sensible texel density.  Assign a material to each chunk and export that to painter.
It's up to you whether you split the highpoly up the same way or not - you can have more than one highpoly in your list so might as well.. 

PolyFarmer

defragger said:

You need to explode the parts before the bake to avoid overlap.

Also the edges on the High-poly meshes are waaay to sharp.

I can assure you that Substance Painter is not the problem here!

Try to understand the basic principles of normalmapping first. You can use basic test meshes like chamfered cubes.

You should read this:

https://polycount.com/discussion/81154/understanding-averaged-normals-and-ray-projection-who-put-waviness-in-my-normal-map/p1

... and later this as well:

https://polycount.com/discussion/147227/skew-you-buddy-making-sense-of-skewed-normal-map-details

Thanks, I'll be sure to read them!

poopipe said:

A texture set is analogous to a material
Each material you apply to the low res mesh before export will give you a texture set in Painter. 
Each texture set can have its own UVs and its own bakes. 

If you're baking that all in one go you'll have all sorts of problems.

Split your low poly up into logical chunks that fit onto a sensible sized map at a sensible texel density.  Assign a material to each chunk and export that to painter.
It's up to you whether you split the highpoly up the same way or not - you can have more than one highpoly in your list so might as well.. 

Gotcha. But if I have been using a single texture set instead of multiple ones, a 4k res should have solved the problem, right? It should not have mattered if I do multiple texture sets with 1024x1024 res or a single one with 4k res, the details should have been preserved either way. At least that's what I thought should have happened.

I would like to avoid spiltting the highpoly and the cage if I can. I don't follow you when you say "you can have more than one highpoly in your list so might as well..". Can you elaborate please? As far as I understand, both the high definition mesh file as well as the low poly base mesh MUST have identical UV layouts and texture sets so that the software can extract normal information from the highpoly mesh and bake it onto the lowpoly successfully? No idea if the cage file needs identical UV layouts and texture sets too. My guess is- it doesn't? Because I guess it's just there to guide the min/max distance? Just taking wild guesses here hehe!

defragger

PolyFarmer said:

As far as I understand, both the high definition mesh file as well as the low poly base mesh MUST have identical UV layouts and texture sets so that the software can extract normal information from the highpoly mesh and bake it onto the lowpoly successfully?

That is not how it works. There is no need for UVs on your High-poly since raytracing is beeing used for baking. Usually I use just an auto generated cage (inflated low-poly).

It is explained on the polycount Wiki:

http://wiki.polycount.com/wiki/Texture_Baking

"Most normal map baking tools allow you to use a distance-based raycast. A ray is sent outwards along each vertex normal, then at the distance you set a ray is cast back inwards. Where ever that ray intersects the high poly mesh, it will sample the normals from it."

poopipe

what i meant about multiple high res meshes is that you can load several meshes and bake from them at the same time. 

wrt: the multiple texture sets thing.  
are you getting enough texel density on a single 4k map to represent the detail you need? 
how long do your bakes take?
how responsive is painter when working on a multilayer document at 4k on your hardware?  (a 16 core xeon with a 1080 isn't very responsive in my experience)
do you want to deal with a metric ton of crap in one go or would you rather deal with the same amount in smaller chunks?


re: the post above... 
The cage absolutely must have the same UV layout as the low res or your tangents won't match and the resultant normal map won't point in the right direction.  auto generated cages duplicate the UV information from the low res so they work fine.

cages are a crutch and I try to avoid using them - that said, you need to have a good grasp of all the other things that are going on in order to reliably bake anything complicated without one so probably best to carry on with the cage in the meantime

PolyFarmer

Thanks all. I'll keep these tips in mind. Going to publish my high poly model in the portfolio for now but I'll keep working on this.