Texture to geometry tool??

melviso

https://knowledge.autodesk.com/support/maya/learn-explore/caas/CloudHelp/cloudhelp/2016/ENU/Maya/files/GUID-79763ED5-E7BD-408B-937B-92743EDDF57A-htm.html#WS17956D7ADBC6E7365144B89E117AE334916-7FE0

https://www.youtube.com/watch?v=qGJ7DcB-Rh0

Anyone know any opensource tool that can do this? I have looked up and down for a Blender addon. None. Also searched github for a standalone tool that can do this. Couldn't find any.
''When you want to use an image to define the edge boundary locations for faces on a polygonal mesh. Using this approach can save you time by not having to split each edge manually. You can use an image to define the edge locations instead.''

Very useful for complex geometry shapes and designs.

Using a color id map to generate edge boundaries for a mesh. Texture can also be tiled and mesh with edge boundaries can be generated.

lluc21

In blender you can convert any image to grease pencil object: https://www.youtube.com/watch?v=X4-3lzLK-R4

melviso

Thanks for the suggestion, lluc21.

However, this is somewhat close but isn't what I am looking for. The other creates a full balanced topology of the whole mesh directed by the color id map of the image. So if you have a complex or organic mesh, and you tile the texture, you can create those topology on the mesh easily.

I did try this but it is creating fill lines rather than one line on the black strokes when the image contains black lines on a white background. Someone wrote this as well in the comment section of the video. Also doesn't work well when using color id maps when the color threshold is tweaked.

poopipe

so - to be clear

You draw a spline, convert it to a bitmap, load the bitmap into maya, process it, and remove a bunch of faces so you end up with the same result you'd get if you just drew a spline in maya and extruded it? 

i feel like there's already an easier way... 


more seriously - the little python experiment i did a while back on alpha card generation should be able to achieve this (results are posted in the tech art WAYWO thread)  the underlying code is capable of handling non-convex shapes or polygons with holes iirc. If anyone wants to take it on I'd be happy to hand the code over (there are some awkward dependencies iirc).  

another caveman

you could save as .svg directly in Photoshop I think both Autodesk and Blender read them? and you'll have your spline

or here's how to export SVG from Photoshop without using Illustrator https://youtu.be/E1Yh3dLzKKk 
if it's sharp silhouette you could also subdivide a plane, displace it from that pic / texture you have, remove back part, keep your shape and decimate ?

melviso

poopipe said:

so - to be clear

You draw a spline, convert it to a bitmap, load the bitmap into maya, process it, and remove a bunch of faces so you end up with the same result you'd get if you just drew a spline in maya and extruded it? 

Hi poopipe,
What I am trying to do is create edge loops where color ids meet when tiled on a mesh:

Say this mesh for example, I just whipped this up, a random id map I just painted that is seamless that I tiled on a mesh in Blender.
This is the image from  Maya texture to geometry from the link in the first post:

I am guessing this tool in Maya would create a topology on the mesh I posted with the edge loops aligned where the id maps meet creating edge boundaries taking the topology of the mesh into account.
Also it would be cool to be able to select the faces or the edges using the image id colors as well.
I am looking for an opensource or free tool or blender addon that can do this but I am guessing that is not possible.

@another caveman  Thanks for the suggestion. This won't work because if you notice the shapes in mine would require a line where the colors meet which I can do manually but  to save time I am looking for an automated way. If I was to draw the lines in a white and black bitmap image, using inkscape, photoshop or grease pencil to convert to svg, lines would be created on both sides of the black stroke which is not what I am trying to do. I could create black patches but if I have a lot of many color ids that meet each other, picking which patch is black or white might become problematic for inkscape to create the lines.
Not to mention how to place the new svg converted mesh tiled on an uneven surface.

poopipe

ok so that's a bit more complex than the maya scenario. 

The maya tool converts an image to a 2d mesh - this is relatively straightforward, they seem to have done a good job with the triangulation based on the two examples here as well. As said above - this is relatively simple to implement given a decent triangulation library

To do the same on an arbitrary mesh is orders of magnitude more complicated and I'm not aware of any python libraries that can cope with it - with the possible exception of pymeshlab. 

with that in mind, It might be worth grabbing meshlab and seeing if you can make it do what you want - if it's possible through the UI it's most likely possible to write a tool that does it using their libraries.

also there's always houdini for these things

another caveman

Indeed unpractical . Hm you could look at having a boolean creating your seam from the edges of your input color map 
Houdini comes to mind for that kind of thing..I'd be very interested in seeing a Blender way
Or have a svg per color etc. It's kind of similar to processing for screen printing in real life, lol.

melviso

@poopipe Thanks a lot for the info. I did try MeshLab but the info was all over the place. I will give it another go. It would  be cool if we could still get the simple one for a single image for Blender. tt's still very very useful.

@another caveman  Probably the best way is to create the topology by hand. It would take more time but the end result would be worth it. I am interested in seeing this as well for Blender too. I am wondering if Geometry nodes in Blender can do this but its still in early development.

gnoop

melviso said:

Does anyone know how that thing works really? On what principles?       I am looking for something like that for pretty a while  and wonder does it really works in MAya and not one of useless features that  never working right ( 3d max has lots of  them for example)

Could I use it to make windows  on a building wall surface  for example?

Is there similar thing  in Houdini?     

poopipe

It detects shapes in the image - there's several ways to do that, it then generates a hull around them and  uses some derivative of Delaunay triangulation to build the surface. 

It's 1980s tech and has  been in Maya for over 10 years in one form or another 
It should be able to handle shapes that have holes so windows would be doable


I imagine Houdini has everything you'd need to make a tool that does it even if there isn't a prebuilt one. 

melviso

@poopipe In Meshlab, I am not sure you can tile a texture, you can assign one but no tiling,,I think. I couldn't find a filter that would can do this currently. Is there a way to create a new filter that can do this in Meshlab like in Houdini?

I found this:
https://www.youtube.com/watch?v=bCw7BN5J8Pk
It's for Blender but it does not work with the color ids like the one in Maya.

Also seems this is possible in ZBrush?
https://www.youtube.com/watch?v=9kSaJ1b4QZU

Seems Zbrush can do this? I am wondering if there is an open source remesher out there that works maybe with texture or vertex color.

gnoop

poopipe said:

It detects shapes in the image - there's several ways to do that, it then generates a hull around them and  uses some derivative of Delaunay triangulation to build the surface. 

It's 1980s tech and has  been in Maya for over 10 years in one form or another 
It should be able to handle shapes that have holes so windows would be doable


I imagine Houdini has everything you'd need to make a tool that does it even if there isn't a prebuilt one. 

They describe something  in  Maya help page . First  step is segmentation . My guess it's something like Substance Designer  flood  node .

Then If I am not wrong   the new  vertexes are usually generated as a regular matrix  first and need to be shifted then to  edges and merge together with a threshold .    Would it be all Substance Designer  solution I would use  bevel/distance  field  to give the shift a vector .   

Delaunay  triangulation  seams like working on a kind of blobs   and I am not sure  what should be center points .   Or maybe I just have no idea of what I am talking .         Just want to persuade  our soft guys to do something like that in our engine  directly .

   

Wonder  is there any published  docs about  all this. I couldn't find any  yet  or Google wrong  search words

poopipe

@gnoopThe flood fill method in designer is far more complicated to implement than the same thing in a  language with loops/memory - whoever put that together is a f*cking ninja (i think it was Nicholas)

But yes, a flood fill / magic wand type approach would be a perfectly sensible way to find the areas of color. 
Each area will then get a boundary defined by a polyline - that'll get optimised down a bit, there's a number of algorithms that fit cheaper polylines to more expensive ones.
The resulting polylines are used to define the boundaries of polygons  - this is what you feed to the triangulation algorithm - in the case of delaunay it distributes a load of points within the polygon and cleverly works out how to connect them up.

OpenCV contains everything you need for the first two parts
I've used this python library successfully for triangulation https://rufat.be/triangle/ .   
it's built off this https://www.cs.cmu.edu/~quake/triangle.html


@melviso
tesselate tex plane looks like it does almost everything you'd need - it might be worth poking the author as I can't imagine it would be a great deal of extra work to add the functionality you want.

BlenderBender

another caveman said:

you could save as .svg directly in Photoshop I think both Autodesk and Blender read them? and you'll have your spline

or here's how to export SVG from Photoshop without using Illustrator https://youtu.be/E1Yh3dLzKKk 
if it's sharp silhouette you could also subdivide a plane, displace it from that pic / texture you have, remove back part, keep your shape and decimate ?

I love this technique. I use it to model weapons. It works very well for swords, axes and stuff.

gnoop

Thanks a lot  poopipe        Looks like I found  something to play with  for  this summer weekends 

melviso

@poopipe I did. He is currently super busy and the triangles libraries has not been updated by its dev to the latest python version as well. so he might work on it in the future.
The addon gets most of the work done so its useful the way it is currently and I really appreciate the developer making it available for other artists to use. He did an awesome job and thank you for the help.

another caveman

you can either photoshop script or filter to get your colors turned to a trace
or substance designer (Edge Detect)

Then all that's left is svg import and sorting it (that could be script too I guess, or maybe you can try geometry nodes over it!)
You'll want export the invert as well for full geometry.

Complex illustrations with overlaps intersections etc might need one svg per color but some can be packed



Geometry isn't as clean as if it had been made from cylinders  as you see

melviso

@another caveman Yes, Breaking the colors down and converting to svg will be the best approach  like you suggested. Thanks for the suggestion and thanks for all the help.

Btw, I found two mesh callibration softwares on github:
Meshroom:
https://alicevision.org/
Trimesh:
https://github.com/mikedh/trimesh

I don't think Meshroom does this though based on color ids. Its for photogrammetry strictly. 

Trimesh...not sure how to install it. But I am guessing it has do do with python? is a library like triangle, the one used for Tessellate texture plane? I must say this past few days has me dabbling and researching about tools created by programmers, using git, cmd and phython as a total noob and I must say it is a fascinating world. Just makes me respect programmers/ software developers even more. It must be awesome to be able to create tools you need as an artist and a lot of hardwork too. Kudos to all programmers out there.