Does anyone know a soft to de-noise height map from photogrammetry?
polycounter
That would keep sharp ridges , important detail . Not just blur everything. Maybe AI based? Does anything like this exist?
That would keep sharp ridges , important detail . Not just blur everything. Maybe AI based? Does anything like this exist?
Replies
Photoshop has various Smart Blur functions, where you can set thresholds for details to preserve.
Or maybe try converting it to mesh and using Topogun to filter the mesh?
if you're going down the mesh conversion route ...
zbrush has some pretty powerful masking/filtering options.
I'd suggest meshlab but I've got a feeling you'd hurl something expensive across the room if you had to use it ;)
I just happened to take lots of phtogrammetry series by my iphone taken very close enough to the subjects . I hoped it would compensate the lack of per photo resolution. Turned out whatever bright day it was and whatever close I shoot resulting mesh and height map from Reality Capture is very noisy and full of artifacts. Much worse than even with my old Sony mirror-less camera.
Perhaps it's all that modern AI picture improving in iphone that makes it only worse so I wonder maybe there is a special height trained AI to fix that. In Sampler maybe? Or elsewhere. Zbrush just blurs everything in my experience. Photoshop denoiser is a tad bit better but looks like not being trained for height maps at all.
Crap in, crap out, as they say. Chalk it up as a learning experience.
The weird thing is few years ago I did it with my old cheap iphone se which didn't have all that modern Ai stuff and with the shots taken under good light and close enough all worked pretty okish. Not as good as 40mpix camera of course but when close enough to compensate resolution still pretty acceptable.
as to the original question one technique you can use in photoshop is
Thanks Klunk. I m actually doing something like this in Substance Designer. The problem in Photoshop is you get zero visual feedback from the height map until you turn it into the normal map elsewhere at least . I should try it in Affinity Photo . I have an old version of it . Thankfully you can get live height to normal map preview from lighting live filter there.
I would look into Topaz's AI suite. They have AI denoising and upscaling tools. https://www.topazlabs.com/denoise-ai
Thanks EarthQuake. Have you tried it on a height map?
IMO Topaz AI denoiser is absolutely useless with height maps. Surprisingly it rather adds its own noise probably considering it as details reconstruction. But even with this reconstruction totally off it's still extra noise and sharpening artifacts.
Works a bit better on normal maps but nothing really "smart" , no better than Photoshop Ai "jpg artifacts remover"
No, this is meant more for general photography, so I'm sure it's not tuned for height maps or game art assets. What I would suggest is running it on your source images before processing them. I imagine this would give you cleaner geometry - unless the noise is being added by your photogrammetry app.
From the image you posted with the normal map, I don't see any noise that needs to be removed, so I'm not quite sure what you're trying to accomplish. If you can post your height map and highlight the problematic areas, someone may have better advice to give you.
It's a dry brook bed boulders I tried to scan . They supposed to be smoother especially low height ones that once was in water . Reality capture makes a lot of arbitrary surface irregularities . It's what I call noise. Sort of geometric noise . I wonder if there might be a remedy for that kind of "noise" . There is a smoother in Reality Capture itself but it rather blur everything indiscriminately while I need something that recognize sharp edges and shape of things like small twigs .
I am too lazy to wear a big camera all the time so it would be really great if photogrammetry would work with phone pictures somehow.
Can you post one of the input photographs?
If you're using a cell phone to take photos, most likely there is heavy noise reduction, sharpening, and potentially other post-effects applied to the image. Generally optical and sensor quality will be subpar, so a better camera would be a good idea. In any case, if your phone supports shooting raw, I would try that so you have control over the process. You can batch process in Lightroom when you find settings that you like, and optionally run them through Topaz (or similar) if you need more sophisticated noise reduction.
It's still not clear whether the problems you're trying to solve are from the input photos or the software, so there may be things for you to adjust in Reality Capture as well.
That said, noise reduction will tend to blur detail, there's really no way around that. So starting with an image that has less noise and more detail may be the only way to solve the problem. I don't think you'll have much luck attacking this from a "fix it in post" perspective - unless you're willing to do a lot of work masking what you do and do not want denoised.
Maybe one of the newer AI video-to-3d tools out there, https://youtu.be/jkiM9qPNC8w