Home Technical Talk

Vertex paint -Mesh or Poly

polycounter lvl 2
Offline / Send Message
Veronika polycounter lvl 2
Hi Guys, 
I have a short question: I prepare imported .fbx object for painting in Substance Painter,
is there any difference, on what I put "Vertex paint modifier" - on "Editable Poly" or on "Editable Mesh"?

Replies

  • Noors
    Offline / Send Message
    Noors greentooth
    To be complete, there's a difference if you paint by face :
    In edit mesh you have access to all triangles (faces) while in edit poly you only have access to "polygons".
    If you apply a vertexpaint modifier to an editable mesh, and go to face mode, the mesh is diplayed triangulated, which isnt the case with edit poly.
    I would avoid to paint only a triangle in edit mesh tho.

    If you paint on vertex level, there's no difference.
    Beware, I'm pretty sure fbx doesnt support vertex color by face anyway, unless you split the mesh.



  • Eric Chadwick
    Ah, I do know FBX supports per-face vertex color. It duplicates the color vertices, but not the mesh vertices. Kind of like splitting UVs. 

    There should be no export difference using Editable Mesh vs. Editable Poly as your base, except the triangulation. Which is important for normal mapping, if you've baked one from another mesh. http://wiki.polycount.com/wiki/Texture_Baking#Triangulation

    IMHO it's better to keep it as Editable Poly in 3ds Max (more tools, easier modeling), and use a Turn to Mesh modifier to triangulate for export. Or use a Turn to Poly modifier with Limit Polygon Size set to 3, same result though.
  • cptSwing
    Offline / Send Message
    cptSwing polycounter lvl 11
    Ah, I do know FBX supports per-face vertex color. It duplicates the color vertices, but not the mesh vertices. Kind of like splitting UVs.


    I too have heard of these ancient forlorn myths - when going from FBX to unity though, it never splits the vertex-colored faces, and I get a runny ol' mess.

  • Eric Chadwick
    That's cuz Unity sucks when it comes to vcolor.


  • cptSwing
    Offline / Send Message
    cptSwing polycounter lvl 11
    ^ Haha.. very fitting, smash those multiple sharp-and-nicely-split hues!

    I've had soul-crushing bouts of work where I had to split fairly complex meshes into hundreds of elements in order to transfer Vcolors correctly, even automated it's a pain. I guess one could prep an FBX export script with a routine that automatically split per-face vcolors before piping the mesh to the exporter, but I don't really have the leisure to root around maxscript for days ;)
  • cptSwing
    Offline / Send Message
    cptSwing polycounter lvl 11
    With some leisure time to burn, I did find this (by @PolyTools3D himself) ...

    (
    	   fn BreakVerticesByColorAndAlpha node =
    	   (
    		   mesh = copy node.mesh
    		 
    		 getmapface = meshop.getmapface
    		 getmapvert = meshop.getmapvert
       
    		   vertscolors = #()
    		 vertsalphas = #()
    		   colorsarray = #()
    		   vertsbyface = #()
    		   flagedverts = #{}
    		   
    		   for j = 1 to mesh.numfaces do
    		   (
    			   faceg = getface mesh j
    			   facec = getvcface mesh j
    			 facea = getmapface mesh -2 j
    			   
    			   for i = 1 to 3 do
    			   (
    				   vcolor = getvertcolor mesh facec[i]
    				 valpha = getmapvert mesh -2 facea[i]
    				   
    				   if vertscolors[faceg[i]] == undefined then
    				   (
    					   vertscolors[faceg[i]] = vcolor
    					 vertsalphas[faceg[i]] = valpha
    				   )else(
    					   if vertscolors[faceg[i]] != vcolor do flagedverts[faceg[i]] = true
    					 if vertscolors[faceg[i]] == vcolor and vertsalphas[faceg[i]] != valpha do flagedverts[faceg[i]] = true
    				   )
    			   )
    		   )
       
    		   setvertselection mesh flagedverts
    		   meshop.breakverts mesh flagedverts
       
    		   flagedverts = getvertselection mesh
    		   sharedfaces = meshop.getfacesusingvert mesh flagedverts
     
    		   for j in sharedfaces do
    		   (
    			   faceg = getface mesh j
    			   facec = getvcface mesh j
    			   facea = getmapface mesh -2 j
    			 
    			   for i = 1 to 3 where flagedverts[faceg[i]] do
    			   (
    				   vcolor = getvertcolor mesh facec[i]
    				 valpha = getmapvert mesh -2 facea[i]
    				 
    				 vrgba = vcolor + valpha
    				 found = finditem colorsarray vrgba
    				 
    				   if found == 0 then
    				   (
    					 append colorsarray vrgba
    					   append vertsbyface #([j,i])
    				   )else(
    					 append vertsbyface[found] [j,i]
    				   )
    			   )
    		   )
     
    		   for j in vertsbyface do
    		   (
    			   weldverts = #{}
    			   for i in j do
    			   (
    				   face = getface mesh i[1]
    				   weldverts[face[i[2]]] = true
    			   )
    			   meshop.weldvertsbythreshold mesh weldverts 0.0001
    		   )
    		   
    		   node.mesh = mesh
    		   
    	   )
    	   
    	   gc()
    	   st = timestamp(); sh = heapfree
    	   BreakVerticesByColorAndAlpha $
    	   format "time:% ram:% \n" (timestamp()-st) (sh-heapfree)
    )
    (via http://forums.cgsociety.org/showthread.php?t=1185723 )

    The script breaks at various junctions, but at least I'm not the first person to contemplate this. To work for all cases, it'd probably need to also copy pre-existing smoothing groups even after vertices are split, but that's a different story I guess.

  • Eric Chadwick
    Just tested this out again, and Unity is letting me use per-face vcolors. Not sure what's changed.
  • cptSwing
    Offline / Send Message
    cptSwing polycounter lvl 11
    Eh, I tried it earlier with results as I'd have expected (using Max 14 and the 2014 FBX). But prompted by your reply I just tried it with Max'17 and their newer FBX exporter and yup - it transfers colors-by-face alright. Awesome!
  • Noors
    Offline / Send Message
    Noors greentooth
    Yeah Clanker is me on cgtalk ^^
    Polytools is my hero.
    Cool that fbx supports it now.
    I do think poly and mesh trinagulate the same way, not 100% sure tho.
  • Veronika
    Offline / Send Message
    Veronika polycounter lvl 2
    Wow, thank you a lot for answers, I need to digest it. I did not suspect these two approaches has so much differences.
  • Veronika
    Offline / Send Message
    Veronika polycounter lvl 2
    Woho-ho , so many answers, I really need time to make a research to realize what they are about. 
    Thank you for your help! 

    For what purposes you could use Vpaint in Unity/Unreal? 
  • Eric Chadwick
    So many purposes! Anything you want to mask on a per-vertex or per-model basis.

    I'm using it in Unity to blend textures on terrains, to control wind bending on foliage, to bend water, etc.
  • monster
    Offline / Send Message
    monster polycounter
    Just tested this out again, and Unity is letting me use per-face vcolors. Not sure what's changed.
    Dude! Glad to know this. I've got a chunk of code I can edit out of our vertex color baking tools.
  • Raffles
    Offline / Send Message
    Raffles vertex
    So many purposes! Anything you want to mask on a per-vertex or per-model basis.

    I'm using it in Unity to blend textures on terrains, to control wind bending on foliage, to bend water, etc.
    That's actually a really good idea, I've never considered using vertex colours as a mask, as you would a pixel mask to blend textures etc.

    As per your other examples, obviously some 0-1 vertex colours would be used to scale the amount of wind applied right? But what about the water, what exact is bending water? :D
  • Eric Chadwick
    You can use vertex color as a cheap flow map. http://wiki.polycount.com/wiki/Flow_map

    One way to think about vertex color is it's just data per vertex, and makes automatic gradients from vert to vert, so it can be used for lots of things. 
  • cptSwing
    Offline / Send Message
    cptSwing polycounter lvl 11
    You can even free up memory by skipping on light bakes and baking shadows etc. to your vertices (sufficient mesh density given).. vertex colors and alpha rock. ;)
Sign In or Register to comment.