Bake turned out wrong in Marmoset

wandering_snail

So, I'm very new to uving complex objects and baking in marmoset. I have been working on the uvs for a week, making sure they were straight and that the textures were in 2k but the bake still turned out wrong. It looks like it has artifacts in certain places and very pixelated up close. I do have some bars that are very large compared to the rest of the objects and I was making sure everything had the same texel density so could it be that the rest of the stuff turned out too low res? I honestly don't know what went wrong, I have been trying to fix any problems but it still turned out wrong. I'm so desperate at this point, I don't know if I can even solve this. 

Joopson

Can you post a shot of the issue? Either just the texture, the texture on the mesh, or both?

It's hard to troubleshoot without a visual aid

wandering_snail

Thank you! Here they are, I don´t know if I should send the uvs as well. I am afraid I might have done something terribly wrong with them but I was sure they´d be fine this time...

Joopson

The UVs could be useful, or the normal map itself, but barring that, I have a theory.

I think it comes down to:

- low resolution bake (I think mostly because of inefficient UV use. You could texture one (or if you neeed, 3 or 4) chain link(s), and save a lot of space)

- cage offset is too high (but really, you should "explode" the chains (I can see nearby chain links showing up in the normal map of any given chain link.), or use baking groups)

There could be other factors, but those are for sure a big part of the troubles.

EarthQuake

Yeah you could split the chains into two separate bake groups, alternate every other link in the chain, then you won't get those intersection errors. You could probably use bake groups in a few other spots too.

wandering_snail

- low resolution bake (I think mostly because of inefficient UV use. You could texture one (or if you neeed, 3 or 4) chain link(s), and save a lot of space)

Thank you for your answers!
Yes, probably but I already only used one chain link and the rest are instances and I actually moved their uvs out of the uv space so that they wouldn't interfere. The bars that are too huge though and take up too much space, should I make a different uv map for the bars and put the rest of the small objects in another one? I'm not very sure. 
@EarthQuake, I will look into the bake groups as well! 

I actually tried to bake one screw separately and it looked alright. It did seem pixelated when I zoomed in a lot though, is that normal or is it a problem as well?

Here is the map too, maybe I've done it wrong, I'm not sure, honestly. I stretched it like that in maya so that it would fit the space and also maya orders everything really randomly. Before I had it separated by parts, I'm not sure if that can cause a problem.

EarthQuake

something that can make sense in cases where you have something like long poles, and then small detailed areas, is to use tiling textures for the poles, and then a separate, uniquely UVed section for the small details. This way you can give both of them much higher texel density

wandering_snail

I want to add rust and such afterwards. Would tiling cause problems with that? I wouldn't want the textures to look like they are repeating. 
I looked up the bake groups as well and if I'm right I have those too. I made sure to name every object before importing to marmoset and it created these folders so the objects are all separate but regarding the chain links, there are so many that I'm not sure the numbers coincide in the low and high poly. Would it be a problem if I didn't import the instanced parts in marmoset? They will read the normal map just fine anyway since they share the same uvs, right? 

EarthQuake

If you want to add rust and unique variation, you could consider doing that via blending 2 materials together in your final engine. But yes, if you wanted to paint unique detail directly into the texture, you would want to have unique UVs instead of tiled.

For baking, if you're sharing UVs, you only need to bake one set of meshes for every shared instance. So you wouldn't necessarily need to import all of the links in your bake file.

wandering_snail

Thank you! Is it possible to separate my object into the poles and the rest of the stuff and have two different uv maps and then group them together to place them in unreal engine? Would it cause problems?

wandering_snail

Hi! I still have problems with my bake. I just can't understand why. I have separated all the other parts from the larger bars and I don't get pixels. I have redone the smoothing groups because I noticed they got messed up. The uvs have the same texel density. I have bake groups separated, I removed the instances... I have no idea what went wrong again. At some point I baked this and the screw turned out fine but I redid the smoothing groups because other parts were wrong and now everything is messed up. I'm at a complete loss. have se

EarthQuake

There seems to be two issues here:

1. For the bolt, it looks like some rays are missing. Double-check your cage to make sure it fits over the high poly mesh.
2. You've got some seam artifacts on the section below, this generally happens when you set up smoothing groups/hard edges, but don't break the UVs apart. Any area where you have a hard edge, you need to split the UVs to avoid this.

wandering_snail

Hello! Thank you for answering, I'm pretty sure I have done those things which is why I'm so baffled that I'm still getting these errors, like for example at some other point I didn't get them, it just seems so random.
Here is my cage, it kind of improved the screw but it still has those weird pixelated parts. There are other parts like this too. Also my uvs for that part before I packed it along with the other parts in maya. I couldn't get the ring part straight but I thought I had unwrapped them correctly.

wandering_snail

I tried exporting an obj and it might even look worse.

EarthQuake

It's difficult to say what the problem is with the bolt, what do your UVs look like there? Double check that the high poly bolt mesh is aligned correctly too - if the high poly bolt was rotated in relation to the low, I would expect to see errors like this.

For the hard edge problem, it looks like you have UV seams, but they do not make your hard edges. Compare the UV seams to the hard edges.

I'm not sure what the latest problem with OBJ is, but changing the file format likely won't solve these issues. I would guess you may have some sort of alignment issue now with the OBJ files, double-check that the high and low meshes are aligned as you would expect.

wandering_snail

Hello! Here are my uvs for the bolt! Maya packs them randomly like this so they are not all close together, could that cause problems?
I did check the hard edges and they seem fine but when I import back into max they are all undone, so could it be that maya messed them up? I did check the keep smooth groups box before exporting... And everything looks aligned too. Still have no idea what is wrong

EarthQuake

In the last image here, it looks like the high and low poly meshes don't line up very well on the top part - (one is hexagonal, the other is cylindrical), this will cause some bake errors as you're trying to project a shape onto another that doesn't quite match. This could explain the problem.

wandering_snail

Oh I see! It seems like I might have removed too many edge loops for the low poly! I will try again and see if it finally works! Thank you so much for your help!

Joopson

Btw, here are the obj settings I personally use for exporting from Maya for baking:



And the FBX settings I use:


It's definitely possible information is being lost or borked between programs, if your settings aren't including what they need to. Is there a reason you seem to be using both Max and Maya? It may be introducing more complication than it's worth.

wandering_snail

Thank you! I was following a tutorial that helped me with the texel density in maya but I will try to use textools in 3ds max because this is turning out to be a mess.