How to cleanup photogrammetry mesh noise?

Oblivion2500

I created a mesh from Reality Capture (high quality) with 950 photos of a statue (using iPhone 12 Pro Max) that is smooth in real life. The mesh is about 2.8 million triangles total. Some small flaws with spots that are low poly/low detail. Overall, it looks great with the color texture on. However, without the color texture or up-close, you can see the triangles and "noise". How can I smooth this out on the whole mesh without removing any major detail or creases/sharp edges?

I'm using Blender 2.93 LTS. I'm willing to get ZBrush for a month via subscription (about $40) if that what it takes to get good quality clean surface and good quality auto-retopology with good edge flow. I'm also trying to retopologize the mesh manually with RetopoFlow 3.0 addon for Blender but it really can't handle more than 1 million poly meshes which kinda sucks (which is also blender fault with not being able to handle millions of polys like ZBrush). 

The goal with this statue meshes to showcase it as a prop in my portfolio with a good understanding of the process of making meshes with photogrammetry and also testing out the mesh in UE5 with nanite feature. 

poopipe

meshlab, houdini etc.. 

lluc21

Inside Blender you can try in sculpt with mesh filter tool, then in the tool select surface smooth. It might require fiddling a bit with the settings depending on the model

gnoop

I once tried  to use iphone  too but IMO  it's those extra magic AI tricks  Iphones are always   trying to do for pictures  or maybe their tiny matrix  or tiny lens   but I never  been able to get anything good  photogrammetry wise from iphone even saving as DNG .   Even doing gazillion super close pictures.   Same  thing  with small 12 mpix drones cameras  . 

 

There is always lots of mesh noise and  moreover weird artifacts .    Even pretty cheap 24mpx A6000 makes it way , way better.    As of killing  the noise I used  Zbrush "smooth"    and  height map  de-noising in Photoshop , some AI denoisers  in pictures.  But anyway  they degrade the actual details too.

My suggestion  don' t try to shoot at cloudy day at all. Iphone  would never has enough light and low enough  ISO  combination for crisp and noiseless pictures.  Better to deal with shadows.

My best results  are from Foveon matrix Sigma. It has best pixel per detail ratio  but it's so slow pacing , so much freezing after each few shots   it turns into a never ending  torture to take just 100 pictures series .  Still results with good enough lighting  are almost noiseless.

Oblivion2500

gnoop said:

I once tried  to use iphone  too but IMO  it's those extra magic AI tricks  Iphones are always   trying to do for pictures  or maybe their tiny matrix  or tiny lens   but I never  been able to get anything good  photogrammetry wise from iphone even saving as DNG .   Even doing gazillion super close pictures.   Same  thing  with small 12 mpix drones cameras  . 

 

There is always lots of mesh noise and  moreover weird artifacts .    Even pretty cheap 24mpx A6000 makes it way , way better.    As of killing  the noise I used  Zbrush "smooth"    and  height map  de-noising in Photoshop , some AI denoisers  in pictures.  But anyway  they degrade the actual details too.

My suggestion  don' t try to shoot at cloudy day at all. Iphone  would never has enough light and low enough  ISO  combination for crisp and noiseless pictures.  Better to deal with shadows.

My best results  are from Foveon matrix Sigma. It has best pixel per detail ratio  but it's so slow pacing , so much freezing after each few shots   it turns into a never ending  torture to take just 100 pictures series .  Still results with good enough lighting  are almost noiseless.

Yeah, I agree that the iPhone is very limited in quality for the camera. Any smartphone for that matter. I don't have any professional camera or any camera besides my iPhone Xs and my friend's phone which I used to take photos (iPhone 12 Pro Max).  The results still look great but I'm going to try Meshlab which is free and open-source for cleaning up models. Maybe it will do a decent enough job and then I will retopologized to for a low-poly model and a "high-quality high-poly model" for the purpose of it working in both UE4 or UE5 new nanite technology. 

I agree, taking a bunch of photos when it's slow is torture plus not only that, you're doing squats basically to take pictures on the low angle view. I find cloudy days to help a lot with the quality of my iPhone. iPhone 12 Pro has a really good light ISO. Huge difference between iPhone 12 Pro and my iPhone Xs. 

Oblivion2500

poopipe said:

meshlab, houdini etc.. 

I will check out meshlab which is free and looks really useful. 

Oblivion2500

lluc21 said:

Inside Blender you can try in sculpt with mesh filter tool, then in the tool select surface smooth. It might require fiddling a bit with the settings depending on the model

Holy crap! That actually worked really well. Looks a lot better! I will keep playing with the settings and see how it looks. Thanks for the tip!