This is interesting, Using depth data to inform AI texture generation. The model is obviously camera-mapped, but I could see this improving by adding multiple angles. Developers could in theory train the AI using in-house content, to then auto-produce asset variants.
So far every AI texture generator I have seen or tried was so ridiculously pathetic I lost my hope. I am tired of textures and want to do something more challenging and thoughtful.
I hate Substance Designer, it's inconvenient as hell and a huge time sink vs just taking few photos for photogrammetry but at least it's an interesting puzzle and sport to waste your time on.
So I wonder why not just do some AI helpers for it . For example I am trying to write a function system that would allow to scale each involved noise to be working in scaling sync and be pixel grid snapping rather than using some arbitrary and blurry 0-1 floating values. I mean to do something with that pixel to pixel precision: https://polycount.com/discussion/231437/handpainting-normal-maps-pixel-by-pixel-100-photoshop-0-highpoly
Why not do an AI helper for such things instead of that pathetic texture generators. Hoped Adobe would do something AI and useful at least, nope . Still nothing at all.
It's a weird obsession of those who do those AI systems. Instead of doing tools that would help a person to be more efficient and free someone from annoying routine they try to replace a person. Still any area that requires actual intelligence not just relaxed "self-expression" and those AI are dumb as a rock.
I am looking forward when Google AI starts to find something really relevant to my inquiries instead of piles of useless advertisement junk . Although it is its primary purpose probably so my complain is futile.
It's a weird obsession of those who do those AI systems. Instead of doing tools that would help a person to be more efficient and free someone from annoying routine they try to replace a person. Still any area that requires actual intelligence not just relaxed "self-expression" and those AI are dumb as a rock.
Quite a good observation.
I have found Adobe's AI stuff for its other products to be incredibly useful, such as auto-transcription in Premiere, or automatic object selection in Photoshop. But their AI tools for substance are gimmicky and seem to barely be an improvement on B2M.
Photoshop object selection doesn't work for textures at all unfortunately, I mean to select something like crumbled layer of paint or mossy rock patches. Or rusty metal areas. I tried many times and it's rather annoying than helpful.
I am looking forward to see an AI de-lighter that wouldn't be just hi-pass.
Any other Ai I'v seen like sympligon or AI UV in SPainter was not that intelligent at all. Ai generated animation is pathetic too, well until it's a helper for a human being doing it.
CG has so much of boring routine that could be eliminated by AI , my guess, yet we see cool toys only. But my guess eliminating those routines requires actual intelligence.
Does anyone know what happened with Unity Art engine ? We have been subscribed for a while before Unity bought it, then lapsed because it wasn't that much helpful. Now I even can't find it.
I think in the near term the biggest problem with quality 3D ai generated models is the crazy data requirement.
It's one thing to correctly predict 1 pixel rgb color at a time for a 2 demensional color output. A 2d pixel output is 1 rgb color.
To correctly predict a pixel of a 3D model you'll need to predict the depth, color, roughness, metalness, emissive etc... (I am sure I am missing a few more PBR data). Clearly, it's coming. But I am not expecting something useful for production art anytime soon.
Like the previous posters have said. There seems to be a common issue for tools/ai developed for 3D artists that are usually not very good. I personally find that software development for multi-disciplinary fields like 3D art are often lacking in usefulness because so much gets lost in translation between the technical and the artistic.
In production we often get so many of the easy stuff wrong, it's hard for me to imagine that we'll somehow jump to getting the hard stuff right.
I am probably wrong on this as I didn't see the ai generated art images coming so soon. 🤷♂️
I played with ChatGPT during last weekend. Wondering if it would be able to find a few solutions for me in Blenders geometry nodes.
And amazing part it does very precise answers of what and how I should do things. It took me pretty a while to figure out that what it offers is not working at all . But it does look very clever indeed initially.
Then I thought ok . lets ask it something about a function graph I am trying to do In Substance Designer . And again it gave me a wall of text of what nodes and how I should connect and in what order to reach desired result.
An amazing thing there are no such nodes. GPT "inventing" them on the fly. I replied "there are no such nodes" It said sorry and broke showing me red color notice " something went wrong" Now I see that notice every time I ask something about Substance Designer.
The truth is that for a lot of 3D problems, whether it's UV or topology, good old math is most of the time still much better at giving you the best result (and faster) compared to AI. The AI wave we're seeing in 2D can only happen because it has access to millions of images to train on and the result just needs to be pretty. There is simply no 3D dataset out there allowing you to train an AI to not only build stuff that looks nice but is also technically sound and optimal.
That being said, we'll get there eventually, as more 3d content becomes available for training, and as ML models requires less and less data to "learn".
On the Substance side, the only 2 features that use AI today are the automatic framing/lighting inferred from a background image in Stager, and the Image to Material in Sampler. Both are 2 years old or so and we now have much better performing versions of those internally that will make their way into the apps soon-ish. In both cases, the key is to reach a critical mass in terms of quantity of training data, but also in terms of quality of that data.
Finally, to reply to your first message Gnoop, the reason we're seeing these very straightforward "artist replacing" use cases today, is because it's simply the easiest to implement, requires minimal investment, creates pretty enough results to appeal to a large audience and make $$$ quick.
The more serious, professional-oriented use cases actually take more time, effort and reflection to nail and will come a bit later, but they are the ones that will stick where I believe the "make art" types of use cases will get tired pretty quickly.
Yes, FOR NOW. But this stuff is advancing so rapidly that it feels like I have no hope. Just look at how fast 2D has progressed. Would you have imagined just 1 year ago that AI would be able to do all of this?
I mean the way it was created and for what purpose is something I anticipated.
They built something that is quick to monetise, maybe not monetise the way Midjournery and Lensa are doing it, but driving users so that you can get venture capital.
Also the people behind it are marketing people looking to monetise web3, nft and metaverse.
Its why they simply don't care about the ethical aspect of data set, or if there are any moral concerns whatsover. All they want is the fastest way to get to market.
The thing is that when you use this approach you risk oversaturation of your own product until your customers tire of it and move on to the next best thing.
In this sense, they aren't listening at all to what artists want, they just appropriated their work for machine learning and spat out something that they call disruptive.
The direction they think this is going is basically Ready Player one the problem is that most of the players here aren't ready and don't know what the end goal is.
If they had consulted artists, we could have gotten something that isn't designed to replace art but assist and augment it. You do see that from products like Substance Painter, Pixologic and Epic games and it is better managed (Though Allegorithmic and Pixologic are sell outs) and Epic seems reluctant to kill the AI hype given its pointless, erractic, free for all approach.
What I can see happening is that anything focused on getting that final render/3d model/concept art into market/metaverse will hype and then leave the public space where it will ultimately become worthless because of oversaturation.
The people making it might feel cool and sell some prints and move over to NFT's in the metaverse and hopefully remain there for the rest of their life.
A good parallel is the VR game, VRchat.
The program ships with stock VR avatars and they opened it to modding so its full of copyright and trademark breaches when it comes to custom characters.
The problem is that the developers now have no idea how to monetise it. The moment they try to sell it as a subscription they will have to disable infringing mods for fear of lawsuits but that will kill their community.
This is likely what will happen to midjourney, Dall-E etc once larger companies start protecting their copyrights and this depends on the scale of monetisation.
Its why midjournery casually says in its TOS, that you own the assets you create so long as you buy a subscription, they are removing their liability as a tech provider.
And I don't get how artists would be threatened, I mean artists that relied on making art for book covers, tshirts, prints might be but why would anyone hire a AI prompter for a role that has longevity?
All of this just fills me with anxiety. 3D Modeling changed my life. It's what gave me a measure of confidence. Initially, I was excited for AI. I saw those upscaled videos from 1900 and I was thinking this could be used to remaster my favorite games from the 90s. I could get rid of tedious 3D work such as UVs and making untileable textures tileable but to me it is CLEAR that the goal is to replace us, all in favor of some so-called "utopia". They could have done this properly from the start but the fact that they cause harm RIGHT NOW so they can make symbolic sacrifices to their idol PROVES to me that I can't trust them and that their so-called vision of the future may not be so utopic after all. The future is theoretical, the present is REAL and their harm is REAL. Just because horses were replaced by cars, copyists by the printing machine and so on DOES NOT MEAN IT HAS TO HAPPEN EVERY TIME. That is SUCH a bad argument. But don't worry, it's all in the name of PROGRESS, whatever that means. If they treat artists like this, why would they treat other professions any better?
We're all going to be a bunch of brainless consoomers eating up whatever this AI shits out. There will be nothing to aspire to, no skill to hone. At best we will be high tech janitors polishing up the turd. I can't think of a worst future.
well why not be "that guy" for once and side with this madness, couldn't someone just simply train an A.I. to use metahuman or makehuman, (steal poser models) and boom they basically do character work..something i was thinking about earlier last week and boom a thread for it to be posted to, nice(i mean the old meaning of this btw)...
"couldn't someone just simply train an A.I. to use metahuman or makehuman"
Well ... this already exists, in the form of the "mesh to metahuman" plugin. It analyses a front view capture of an arbitrary mesh (3d scan, sculpt, anything really) and then massages the base MetaHuman topology around it to fit the features. It basically mostly automates the process of otherwise manually tweaking the sliders in the MH editor. I can only assume that the feature recognition that happens (on the server btw, not locally) is AI-trained.
It already can, but the difference isn't as stark as with 2D. I've noticed a lot of 3D AI characters whether they are made by human or AI tend to look similar in terms of quality and design. The noticeable difference is that the AI can do it much faster.
Learning AI art generation tools will become mandatory like knowing photoshop. It will not replace Artists, but we will likely see a similar thing like on the Farm. Instead of 3 Tractors, one guy is driving one Large Tractor. The Job is not going anywhere but you'll not need as many people doing it any more, but you need to use more sophisticated tools to compete, and the level you are operating on is potentially much higher.
You'll see this already on artstation, where a medium good concept artist does a solid draft, then does generative stuff, then polishes that up, and becomes top 1% art. It's pretty clear that people not getting on that will be left behind.
I can imagine however that larger studios might have dedicated people to polish up content from others with AI help (probably for 2D), and / or dedicated people to polish up generated art, probably more the typical 3D artist, generating something baseline, then expanding on it, and mixing that with classical work. But we'll see, things are going to get crazy for sure, and standing out will matter more than ever.
My prediction is that at first, standing out will become important, then we'll get complete post processing style transfer, and that is basically nullified. Proportions / layouting / negative space might be handled by automatic level generation or might not and stay relevant for a long time. Character artists will play a lot of dress up. Being picky will stay relevant or even more so. I think a lot of environment artists will become more of a level designer hybrid probably, where the level designer does the art and the level gameplay simultaneously (which already many people kinda do).
Replies
yes it will, already does, as shown in some threads here regarding this topic
ie:
https://dreamfusion3d.github.io/
Why not?
The matter is just when, which should be too far from now.
https://onformative.com/work/ai-sculpting/
This is interesting, Using depth data to inform AI texture generation. The model is obviously camera-mapped, but I could see this improving by adding multiple angles. Developers could in theory train the AI using in-house content, to then auto-produce asset variants.
https://twitter.com/CarsonKatri/status/1600248730778681344
https://hackaday.com/2022/12/18/image-generating-ai-can-texture-an-entire-3d-scene-in-blender/
So far every AI texture generator I have seen or tried was so ridiculously pathetic I lost my hope. I am tired of textures and want to do something more challenging and thoughtful.
I hate Substance Designer, it's inconvenient as hell and a huge time sink vs just taking few photos for photogrammetry but at least it's an interesting puzzle and sport to waste your time on.
So I wonder why not just do some AI helpers for it . For example I am trying to write a function system that would allow to scale each involved noise to be working in scaling sync and be pixel grid snapping rather than using some arbitrary and blurry 0-1 floating values. I mean to do something with that pixel to pixel precision: https://polycount.com/discussion/231437/handpainting-normal-maps-pixel-by-pixel-100-photoshop-0-highpoly
Why not do an AI helper for such things instead of that pathetic texture generators. Hoped Adobe would do something AI and useful at least, nope . Still nothing at all.
It's a weird obsession of those who do those AI systems. Instead of doing tools that would help a person to be more efficient and free someone from annoying routine they try to replace a person. Still any area that requires actual intelligence not just relaxed "self-expression" and those AI are dumb as a rock.
I am looking forward when Google AI starts to find something really relevant to my inquiries instead of piles of useless advertisement junk . Although it is its primary purpose probably so my complain is futile.
It's a weird obsession of those who do those AI systems. Instead of doing tools that would help a person to be more efficient and free someone from annoying routine they try to replace a person. Still any area that requires actual intelligence not just relaxed "self-expression" and those AI are dumb as a rock.
Quite a good observation.
I have found Adobe's AI stuff for its other products to be incredibly useful, such as auto-transcription in Premiere, or automatic object selection in Photoshop. But their AI tools for substance are gimmicky and seem to barely be an improvement on B2M.
Photoshop object selection doesn't work for textures at all unfortunately, I mean to select something like crumbled layer of paint or mossy rock patches. Or rusty metal areas. I tried many times and it's rather annoying than helpful.
I am looking forward to see an AI de-lighter that wouldn't be just hi-pass.
Any other Ai I'v seen like sympligon or AI UV in SPainter was not that intelligent at all. Ai generated animation is pathetic too, well until it's a helper for a human being doing it.
CG has so much of boring routine that could be eliminated by AI , my guess, yet we see cool toys only. But my guess eliminating those routines requires actual intelligence.
Does anyone know what happened with Unity Art engine ? We have been subscribed for a while before Unity bought it, then lapsed because it wasn't that much helpful. Now I even can't find it.
I think in the near term the biggest problem with quality 3D ai generated models is the crazy data requirement.
It's one thing to correctly predict 1 pixel rgb color at a time for a 2 demensional color output. A 2d pixel output is 1 rgb color.
To correctly predict a pixel of a 3D model you'll need to predict the depth, color, roughness, metalness, emissive etc... (I am sure I am missing a few more PBR data). Clearly, it's coming. But I am not expecting something useful for production art anytime soon.
Like the previous posters have said. There seems to be a common issue for tools/ai developed for 3D artists that are usually not very good. I personally find that software development for multi-disciplinary fields like 3D art are often lacking in usefulness because so much gets lost in translation between the technical and the artistic.
In production we often get so many of the easy stuff wrong, it's hard for me to imagine that we'll somehow jump to getting the hard stuff right.
I am probably wrong on this as I didn't see the ai generated art images coming so soon. 🤷♂️
I played with ChatGPT during last weekend. Wondering if it would be able to find a few solutions for me in Blenders geometry nodes.
And amazing part it does very precise answers of what and how I should do things. It took me pretty a while to figure out that what it offers is not working at all . But it does look very clever indeed initially.
Then I thought ok . lets ask it something about a function graph I am trying to do In Substance Designer . And again it gave me a wall of text of what nodes and how I should connect and in what order to reach desired result.
An amazing thing there are no such nodes. GPT "inventing" them on the fly. I replied "there are no such nodes" It said sorry and broke showing me red color notice " something went wrong" Now I see that notice every time I ask something about Substance Designer.
Funny! I assumed as much, just like the images generated with AI, impressive at first glance and once you go closer it makes less and less sense
The truth is that for a lot of 3D problems, whether it's UV or topology, good old math is most of the time still much better at giving you the best result (and faster) compared to AI. The AI wave we're seeing in 2D can only happen because it has access to millions of images to train on and the result just needs to be pretty. There is simply no 3D dataset out there allowing you to train an AI to not only build stuff that looks nice but is also technically sound and optimal.
That being said, we'll get there eventually, as more 3d content becomes available for training, and as ML models requires less and less data to "learn".
On the Substance side, the only 2 features that use AI today are the automatic framing/lighting inferred from a background image in Stager, and the Image to Material in Sampler. Both are 2 years old or so and we now have much better performing versions of those internally that will make their way into the apps soon-ish. In both cases, the key is to reach a critical mass in terms of quantity of training data, but also in terms of quality of that data.
Finally, to reply to your first message Gnoop, the reason we're seeing these very straightforward "artist replacing" use cases today, is because it's simply the easiest to implement, requires minimal investment, creates pretty enough results to appeal to a large audience and make $$$ quick.
The more serious, professional-oriented use cases actually take more time, effort and reflection to nail and will come a bit later, but they are the ones that will stick where I believe the "make art" types of use cases will get tired pretty quickly.
al htis AI business doesn't bother me, carries on making stuff for fun :)
...feels like the end is coming any day now...
https://80.lv/articles/generating-text-to-3d-models-using-openai-s-point-e-gepetto-ai-unreal-engine/?fbclid=IwAR2a4VENGvNV2Jk7AuwLEqYbSQwOcZRnzqx_lzQmD2u5Zd1IDvFvgh3Eh7E
Not sure what is promising in that tech. Looks like garbage.
Yes, FOR NOW. But this stuff is advancing so rapidly that it feels like I have no hope. Just look at how fast 2D has progressed. Would you have imagined just 1 year ago that AI would be able to do all of this?
I mean the way it was created and for what purpose is something I anticipated.
They built something that is quick to monetise, maybe not monetise the way Midjournery and Lensa are doing it, but driving users so that you can get venture capital.
Also the people behind it are marketing people looking to monetise web3, nft and metaverse.
Its why they simply don't care about the ethical aspect of data set, or if there are any moral concerns whatsover. All they want is the fastest way to get to market.
The thing is that when you use this approach you risk oversaturation of your own product until your customers tire of it and move on to the next best thing.
In this sense, they aren't listening at all to what artists want, they just appropriated their work for machine learning and spat out something that they call disruptive.
The direction they think this is going is basically Ready Player one the problem is that most of the players here aren't ready and don't know what the end goal is.
If they had consulted artists, we could have gotten something that isn't designed to replace art but assist and augment it. You do see that from products like Substance Painter, Pixologic and Epic games and it is better managed (Though Allegorithmic and Pixologic are sell outs) and Epic seems reluctant to kill the AI hype given its pointless, erractic, free for all approach.
What I can see happening is that anything focused on getting that final render/3d model/concept art into market/metaverse will hype and then leave the public space where it will ultimately become worthless because of oversaturation.
The people making it might feel cool and sell some prints and move over to NFT's in the metaverse and hopefully remain there for the rest of their life.
A good parallel is the VR game, VRchat.
The program ships with stock VR avatars and they opened it to modding so its full of copyright and trademark breaches when it comes to custom characters.
The problem is that the developers now have no idea how to monetise it. The moment they try to sell it as a subscription they will have to disable infringing mods for fear of lawsuits but that will kill their community.
This is likely what will happen to midjourney, Dall-E etc once larger companies start protecting their copyrights and this depends on the scale of monetisation.
Its why midjournery casually says in its TOS, that you own the assets you create so long as you buy a subscription, they are removing their liability as a tech provider.
And I don't get how artists would be threatened, I mean artists that relied on making art for book covers, tshirts, prints might be but why would anyone hire a AI prompter for a role that has longevity?
All of this just fills me with anxiety. 3D Modeling changed my life. It's what gave me a measure of confidence. Initially, I was excited for AI. I saw those upscaled videos from 1900 and I was thinking this could be used to remaster my favorite games from the 90s. I could get rid of tedious 3D work such as UVs and making untileable textures tileable but to me it is CLEAR that the goal is to replace us, all in favor of some so-called "utopia". They could have done this properly from the start but the fact that they cause harm RIGHT NOW so they can make symbolic sacrifices to their idol PROVES to me that I can't trust them and that their so-called vision of the future may not be so utopic after all. The future is theoretical, the present is REAL and their harm is REAL. Just because horses were replaced by cars, copyists by the printing machine and so on DOES NOT MEAN IT HAS TO HAPPEN EVERY TIME. That is SUCH a bad argument. But don't worry, it's all in the name of PROGRESS, whatever that means. If they treat artists like this, why would they treat other professions any better?
We're all going to be a bunch of brainless consoomers eating up whatever this AI shits out. There will be nothing to aspire to, no skill to hone. At best we will be high tech janitors polishing up the turd. I can't think of a worst future.
well why not be "that guy" for once and side with this madness, couldn't someone just simply train an A.I. to use metahuman or makehuman, (steal poser models) and boom they basically do character work..something i was thinking about earlier last week and boom a thread for it to be posted to, nice(i mean the old meaning of this btw)...
"couldn't someone just simply train an A.I. to use metahuman or makehuman"
Well ... this already exists, in the form of the "mesh to metahuman" plugin. It analyses a front view capture of an arbitrary mesh (3d scan, sculpt, anything really) and then massages the base MetaHuman topology around it to fit the features. It basically mostly automates the process of otherwise manually tweaking the sliders in the MH editor. I can only assume that the feature recognition that happens (on the server btw, not locally) is AI-trained.
It will not replace Artists, but we will likely see a similar thing like on the Farm. Instead of 3 Tractors, one guy is driving one Large Tractor.
The Job is not going anywhere but you'll not need as many people doing it any more, but you need to use more sophisticated tools to compete, and the level you are operating on is potentially much higher.
You'll see this already on artstation, where a medium good concept artist does a solid draft, then does generative stuff, then polishes that up, and becomes top 1% art. It's pretty clear that people not getting on that will be left behind.
I can imagine however that larger studios might have dedicated people to polish up content from others with AI help (probably for 2D), and / or dedicated people to polish up generated art, probably more the typical 3D artist, generating something baseline, then expanding on it, and mixing that with classical work. But we'll see, things are going to get crazy for sure, and standing out will matter more than ever.
My prediction is that at first, standing out will become important, then we'll get complete post processing style transfer, and that is basically nullified. Proportions / layouting / negative space might be handled by automatic level generation or might not and stay relevant for a long time. Character artists will play a lot of dress up. Being picky will stay relevant or even more so. I think a lot of environment artists will become more of a level designer hybrid probably, where the level designer does the art and the level gameplay simultaneously (which already many people kinda do).
when it matures we will get better procedural generation tools than we have now
because that's all it is