Artifacts, not sure where they're coming from.

J0NNYquid

Hey guys been working on a Halo: Reach DMR for a few weeks on and off http://www.polycount.com/forum/showthread.php?p=1910155#post1910155, and when I first baked everything out it looked fine, until I started adding in some color, then I noticed some artifacting. I know I probably checked or didn't check some specific box, or compressed something I wasn't supposed to compress. Anyways, I'm pulling my hair out trying to figure out where these artifacts are coming from.


Artifacts by camraab, on Flickr

What I've tried so far....

A friend said I should try rendering out my normal maps with a different super sampler, and at first it looked like it worked. I'm using Max,and I switched from Hammersley to Max 2.5 star. However, when I ran everything through dDo, I got a whole new type of artifacting, as you can see in the image above. The effect is severely more pronounced in dDo, this is a shot of it from the standalone Marmoset.

Everything looks fine on the files I put into dDo, something, somewhere along the line isn't right obviously, I just don't know where to look.

J0NNYquid

I have overlapping UV's, but they're offset 1 space to avoid anything like that. It's all over the place though, and it's pretty uniform. As far as geo goes what should I be looking for, the bake is super clean in all aspects, and when I look at in photoshop I don't see any of these artifacts. It's a 4096x4096 texture, it's pretty densely packed, but there should be more than enough room for everything. It's weird, it's only noticeable on the parts where little to nothing is actually being done by the normal maps, the predominately flat sections.

Ok, so in looking closer at the normal map bake, I can just barely make out some of the artifacts that I'm seeing in Marmoset. Not sure what I'm doing wrong in the bake to make that come about, apart from baking an 8K texture, which seems absurd, what am I missing?

Normals:


DMR_LOW_N by camraab, on Flickr

You can download the original from Flickr if you really want to look at it closely, this is straight out of max, no edits/fixes whatsoever.

AlecMoody

What artifacts are you referring to? The light noise in the specular highlights?

J0NNYquid

Is that what it is?, I apologize, technically I don't know my ass from a hole in the ground. If so what's the fix? The effect is exaggerated when I run it through dDo, I'll grab a screen quick. Light noise sounds right, it's not really noticeable unless a light is being directly cast on the surface.

EDIT:

dDo Examples


Artifacts3 by camraab, on Flickr


Artifacts2 by camraab, on Flickr

AlecMoody

You can try baking into 16bpp tif files, and then only converting into 8 bit precision once you are 100% done making changes to the normal map. Realistically though, in engine, this will get compressed and look much worse. I also don't know if DDO accepts 16 bpp normals.

what kind of changes to the normals are you making in DDO? I would be cautions about modifying your baked normals with a 2d tool.

AlecMoody

Yeah, looks like some kind of texture compression to me. Whats the final destination for this object? Is it just a portfolio object that will be shown in marmoset or is it going into a game engine?

Any subtle texture detail will cover this up. However, working with 16bpp normals should also help.

J0NNYquid

I'm not doing anything to the normals via dDo (that I'm aware of). I just plug it into the the "normals" map slot. I add additional detail via NDo2, but I keep the original layer unchanged.

I'll give tif files a shot, see what I get. Thanks for all your help so far, I really appreciate it.

EDIT: Sorry, should have waited for your response, this is just going to be a portfolio piece, so I can hide it if need, be, but I'd prefer not to, even if I'm the only one that will know, I'd rather have the correct process when I come out the other side of this project, as I've had this problem before. I render out my maps from Max in .tga format, but as I said, I'll give tif a shot.

EarthQuake

This is most likely a matter of bit depth. Max dithers so you get some noise, maya tends to introduce stair stepping effects, I like what max does better generally. When you actually texture your asset you're unlikely to see either type.

The last shot, you're 100% sure you dont have any detail layers affecting the height map in dDo? The height map is combined with the normals on export in dDo.

J0NNYquid

Alright, so the .tif appears to have worked. EQ, I don't believe so, to get those shot I just put in a TS Normal, CLR, AO, Cavity, and Wire into the automation hub, turned off the environmental preset, and used Standard Reflectance Max, then hit go.

Possible stupid questions incoming:

Alright, so I've rendered out my .tif, and I.....go where from here. As I've said, I struggle with the technical side of things, so bit depths, bit conversions. file types(and their subsequent strengths/weaknesses), all that stuff, confuses the hell of me. I've got my 16 bit .tif, but marmoset won't accept it. I've already got my NDO2 file that has all my added detail, is it as easy as placing my .tif normals on the base layer and just overlay everything? And if that's possible, what do I save it as to ensure I don't lose the cleanliness? of the .tif file?

I also notice I can't save the .tif directly as a .tga, or .dds, what's the normal plan of action for you at that point? I'll say it again, thanks for the help thus far guys, much appreciated.

EarthQuake

Toolbag will load a 16bit PSD I believe, just resave your tif as PSD.

Though the majority of game engines will not take a 16bit normal map, its just a really poor use of resources memory wise, so best not to stress too much about minor bit depth issues (again, that should go away when you actually texture the asset). Once an asset like this is in an actual game you'll have texture compression to worry about that will do more harm than the minor artifacts you're seeing here.

Also, sometimes saving an 8bit per channel file generated from a 16 bit file in photoshop will provide better results than a file baked to 8 bit.

J0NNYquid

Alright, well it appears I've discovered a workaround, but it makes no sense to me. Originally I wanted to run this through dDo, but dDo, for whatever reason, is doing something that I'm not aware of or familiar enough with it to catch. I'm using the standalone Marmoset Toolbag from here on out, and I'll do the textures the good ol' way. What ended up working was for me to just go map by map, re-baking AO, NORM, even CLR I found artifacting when I got really close. I re-rendered everything as a 16-bit .tif file, then saved them out as 16-bit .psds. I'm now loading those all into Marmoset and getting a perfectly clean result, no artifacts whatsoever.

Blessing in disguise maybe, I need to work on my texturing skills in general, so doing it by hand will be good practice, I just wish I knew why it was so messy in dDo, I'm not blaming the software, far from it, I know it's something on my end, just can't figure out what.

abmurray

EarthQuake wrote: »

Also, sometimes saving an 8bit per channel file generated from a 16 bit file in photoshop will provide better results than a file baked to 8 bit.

the truth of this cannot be overstated