Custom trained image model based on own weapon art
interpolator
Hey, we were playing around with doing custom models based on our own art as a test.
This has very mixed results based on the subject, but I want to share the approach for any artist wanting to try this with his own works.
If you want to try the same, you can use Leonardo.ai and feed it your own images but the most powerful UI is apparently Automatic1111 which Is kind of the go-to it seems right now. Our test only had 7 input images.
Some visual examples here below:
(3D Model by Moritz Mayer, done in Blender, game: Synthetik 2)
In the end this seems quite useful if you dont expect anything resembling a final artwork but more of "kitbashing components" but your image input must really fit and be consistent.
We tried it also using our icons, which in theory would be an amazing dataset and of very high resolution for an icon, but that did not work very well at all.
Input - This resulted in very bad things, (I don't have them for showing)
Then we tried "only grip icons" and gave less bad but still unredeemably bad results:
Giving a ton of images which for us seemed to be of the same style did not work. It seems to be more interested in subject than style, while (at least for us) we are more interested in a consistent style than subject.
Nevertheless this seems very powerful for concepting as honestly the generation was spitting out many interesting and wierd shapes which were not something we would usually come up with and you can pick out some of the great nuggets.
------
Bonus:
Midjourney Image to Image feature: Soviet PPSH + HK UMP
Since this is not using a custom dataset, this gives a very clear feeling of "generic gun" but spits out suprisingly high quality outputs and great parts, even if the proportions are quite terrible. Insanely easy to use tho, 6-8 months ago this would still give terrible results.
We are by far not very deep in this and there is definitely a lot more potential there. Id also look up "cozy UI" and Automatic1111 for the advanced user.
This has very mixed results based on the subject, but I want to share the approach for any artist wanting to try this with his own works.
If you want to try the same, you can use Leonardo.ai and feed it your own images but the most powerful UI is apparently Automatic1111 which Is kind of the go-to it seems right now. Our test only had 7 input images.
Some visual examples here below:
In the end this seems quite useful if you dont expect anything resembling a final artwork but more of "kitbashing components" but your image input must really fit and be consistent.
We tried it also using our icons, which in theory would be an amazing dataset and of very high resolution for an icon, but that did not work very well at all.
Input - This resulted in very bad things, (I don't have them for showing)
Then we tried "only grip icons" and gave less bad but still unredeemably bad results:
Giving a ton of images which for us seemed to be of the same style did not work. It seems to be more interested in subject than style, while (at least for us) we are more interested in a consistent style than subject.
Nevertheless this seems very powerful for concepting as honestly the generation was spitting out many interesting and wierd shapes which were not something we would usually come up with and you can pick out some of the great nuggets.
------
Bonus:
Midjourney Image to Image feature: Soviet PPSH + HK UMP
Since this is not using a custom dataset, this gives a very clear feeling of "generic gun" but spits out suprisingly high quality outputs and great parts, even if the proportions are quite terrible. Insanely easy to use tho, 6-8 months ago this would still give terrible results.
We are by far not very deep in this and there is definitely a lot more potential there. Id also look up "cozy UI" and Automatic1111 for the advanced user.
Replies
I'm a bit sad that there's no way our legal team would let me train something up on leonardo.ai, I'd love to see this sort of thing gaining traction
- AI image generators allowing for custom fine-tuning still rely on the initial training sourcing images without consent, so even if the images are used as concept only there is still an underlying ethical issue here.
- And because of that, it seems that the position of Valve is to deny the publishing of games containing such images or text, reportedly asking devs to remove it fully. Now whether this would apply here is up for debate, but maybe it isn't worth the risk ...
https://www.gamedeveloper.com/business/report-valve-won-t-publish-steam-titles-that-feature-ai-generated-content
If true (as sources aren't the strongest here ...) I feel like this position is not just motivated by the potential legal issues but also by their interest in maintaining a healthy platform and market.
Now beyond that and even if the ML model used here was fully custom-trained I feel like there is still the issue of the process potentially cheapening the product for not much of an added benefit. This feels similar to the case of book authors deciding to use AI art for their book cover instead of going for something perhaps more simple, but not involving artificial generation.
Overall that's definitely an interesting experiment. This reminds me of the initial enthusiastic excitement surrounding AI-generated images about 6 months ago, before the source of the training data became known.
Anyway, this is a neat experiment for sure. I like the idea of training a model from your own artwork. This could be very useful for kicking out new designs or variations for a world that has an established design language. I don't personally see any ethical issues with that either.
Using AI-derived designs as a base to paint over, almost like a thumbnail sketch generator, makes a lot of sense too. I think a lot of people look at AI and sort of assume it will take over the entire creation pipeline - but it can be effective as more of a brainstorming tool early in the process.
The scale of it is simply so massive that it makes it hard to fathom really ; so for the sake of illustrating it, here's a screenshot from HaveIBeenTrained. This screenshot happens to show a positive from one of my own self-searches, but that's hardly relevant - the thing that makes ones head spin is the sheer amount of art that has been used without consent. Without these images used in LAION the AI generators would output *nothing* - yet none of the authors or IP holders received any notice about their property being used. And that's not even mentioning the use of pictures from social media, medical records, porn (and of course revenge porn), and so on ...
BTW this is not an attack on @Shrike of course - as a matter of fact it would be hypocritical for me to do so as I too was playing around with these tools last year ... until I learned how they actually work, and how they constitute art theft on a massive scale.
If anything I hope this helps clarifies it. And for that alone I do believe that the experiment presented in this thread has value.
The tech is of course fascinating on a technical level, but the end doesn't justify the means. It's up to us as artists to decide whether we want to live in a world where we throw our principles of mutual respect out of the window in the name of art FOMO.
I'd argue that there is no such thing as "more ethical" - it is either ethical, or it isn't ... (but I understand what you're getting at of course).